Fine-tuning and customization

What this session covers

Fine-tuning teaches a base model a behavior by example — a house voice, a strict output format, a niche classification the base model keeps fumbling — by training it on hundreds or thousands of your own input/output pairs. It works, and it's still the last tool you should reach for. Prompting, few-shot examples, and RAG solve most problems faster and cheaper, with no training loop at all. Fine-tuning earns its place once you've exhausted those and still need consistency the prompt can't buy, or when you want the behavior without paying for a long prompt on every single call.

So this hour is as much about not fine-tuning as about doing it. When you do commit to it, the work is overwhelmingly the data: hundreds of consistent examples in the right format. The training job itself is a few lines and then a wait. We build and validate the dataset here, and the job stays guarded, because it costs money and takes time.

OpenAI

Fine-tune data is JSONL: one chat conversation per line, each carrying the system and user messages plus the assistant completion you want the model to imitate. The property that matters most is consistency — the same system prompt and the same output shape on every example, because whatever's inconsistent in your data is exactly what the model learns to be inconsistent about.

# Fine-tune data is JSONL — one chat conversation per line, each with the system
# and user messages and the assistant completion you want the model to learn.
# Consistency is everything: the same system prompt, the same output shape, on
# every example. Sloppy data teaches sloppy behavior.
EXAMPLES = [
    ("hello there", "Ahoy there, matey!"),
    ("where is the treasure?", "Arr, where be the treasure buried?"),
    ("I am hungry", "Me belly be growlin' for grub!"),
    ("good morning", "Mornin', ye scurvy dog!"),
]
SYSTEM = "You are a pirate translator. Rewrite the user's line in pirate speak."


def to_jsonl(examples: list[tuple[str, str]], system: str) -> str:
    lines = []
    for user, assistant in examples:
        lines.append(json.dumps({"messages": [
            {"role": "system", "content": system},
            {"role": "user", "content": user},
            {"role": "assistant", "content": assistant},
        ]}))
    return "\n".join(lines)

Launching takes a few lines — upload the file, create the job, poll until it finishes — and out comes a model id you call exactly like any other model. It sits behind a flag here so you don't spend money by accident.

# Launching is async and billable, so it's behind a flag. Upload the JSONL,
# create the job, then poll until it finishes — the job produces a model id you
# call exactly like any other model. (openai fine-tuning targets specific base
# models, e.g. gpt-4o-mini-2024-07-18.)
def launch(path: str) -> None:
    from openai import OpenAI
    client = OpenAI()
    uploaded = client.files.create(file=open(path, "rb"), purpose="fine-tune")
    job = client.fine_tuning.jobs.create(training_file=uploaded.id, model="gpt-4o-mini-2024-07-18")
    print(f"started job {job.id} — poll client.fine_tuning.jobs.retrieve('{job.id}') until status='succeeded'")

Gemini

Gemini's supervised tuning takes a simpler data shape — input/output pairs instead of full chat JSONL — but the discipline is identical, and so is the async, billable job.

# Gemini tuning takes input/output pairs directly. Same rule as everywhere:
# consistency across examples is what the model actually learns.
EXAMPLES = [
    ("hello there", "Ahoy there, matey!"),
    ("where is the treasure?", "Arr, where be the treasure buried?"),
    ("I am hungry", "Me belly be growlin' for grub!"),
    ("good morning", "Mornin', ye scurvy dog!"),
]


def build_dataset(examples):
    from google.genai import types
    return types.TuningDataset(
        examples=[types.TuningExample(text_input=u, output=a) for u, a in examples],
    )

The decision framework is worth carrying out of this chapter. Reach for prompting first (fast, free to change). Add few-shot examples when you need a specific style or format (still just a prompt). Add RAG when the gap is knowledge the model doesn't have — fine-tuning teaches behavior, and it does not teach facts, which trips people up constantly. Fine-tune only when you need consistent behavior at a scale or latency where stuffing examples into every prompt gets too slow or too expensive. And remember that a fine-tune is a commitment: new base models ship constantly, and yours pins you to the one you trained on until you redo the work.

Put it to work

Three docker-compose apps under code/showcase/<slug>/, arranged around the real workflow: prepare the data, try the cheaper alternative, then bootstrap more data. Same drill: bash bootstrap-secrets.sh, docker compose up --build, http://localhost:3000.

Showcase 1 — Dataset builder

Paste examples, get valid training JSONL — OpenAI chat format or Gemini pairs, side by side. No model call, because this is pure data plumbing, and data plumbing is where fine-tuning projects live or die. Put both formats next to each other and the shape difference jumps out.

Showcase 2 — Few-shot style

The alternative you should try first. A handful of example pairs in the prompt buys you a custom voice with no training and no cost, and you can iterate instantly. Run it and ask yourself honestly whether you'd still need a fine-tune.

Showcase 3 — Synthetic data

The chicken-and-egg fix: you need hundreds of examples and have five. Describe the task and a model drafts more, correctly formatted. You'd still review and expand them, since synthetic data is only where you start, but it beats staring at a blank file.

Together they trace the honest arc of a fine-tuning project: mostly data work, with a strong pull to solve the problem without training at all.

Run it

The README in this folder lists the Python version, the install line, the two environment variables, and the exact commands. Keys come from the untracked .env at the course root. The basic scripts build a dataset and only launch a job if you set RUN_FINETUNE=1 — that one costs money, so it's off by default.

Takeaways

Fine-tuning is real, and now and then it's the right answer, but treat it as the last resort. Prompting, few-shot, and RAG handle most of what people reach for fine-tuning to fix, and when the gap is knowledge the tool is RAG, because tuning changes how the model behaves and leaves what it knows alone. When you do fine-tune, the training call is a footnote and the data is the actual work: hundreds of consistent, correctly formatted examples, which is why the dataset builder and the synthetic generator matter more than the job that trains on their output. Weigh the commitment too — a fine-tune pins you to a base model in a world where better ones keep landing. Next week we put the whole course together and let the model drive: a single-loop agent with tools.