Fine-Tuning & Model Customization.

Fine-tuning takes a model that is already trained and keeps training it on a smaller, focused dataset. Pretraining from scratch starts from random weights and teaches the model language itself, from raw text at huge scale.

The mechanism is the same in both: gradient descent on a next-token prediction loss. The difference is the starting point and the scale.

That starting point drives the practical rules. The base model already holds the ability you want, so fine-tuning uses a small learning rate and few passes. A high learning rate or too many passes erode the general ability pretraining built.

Treat it as a ladder of cost and commitment. Start on the cheapest rung, and climb only when the one below clearly fails.

• Prompting changes nothing lasting, only each call's instructions. Always start here, and stay when the behavior can be put into words.
• RAG changes the facts in the context window, for private, fresh or changing knowledge.
• Fine-tuning changes the weights, so behavior sticks. Use it when prompts work but are long, slow or unreliable.
• Continued pretraining changes base knowledge and language, for a model that barely understands your domain's text or notation.

As a rough rule, RAG changes what the model answers with, and fine-tuning changes how it answers.

These stack rather than compete. Most production systems are prompting plus RAG, with a fine-tune added later for cost or consistency. Pretraining is rare, because it needs billions of tokens.

Supervised fine-tuning is the same next-token prediction as pretraining, run on your examples. The weights already exist, so each step only nudges them.

Each row is rendered with the chat template, then tokenized. The model does one forward pass and predicts a distribution over the next token at every position. Cross-entropy loss measures how surprised it was by your actual tokens. Backpropagation then works out how much each weight contributed to the loss. That gives a gradient, and the optimizer subtracts a small step.

for batch in loader:                  # one optimizer step per batch
    out = model(input_ids=batch["input_ids"], labels=batch["labels"])
    out.loss.backward()               # gradient for every trained weight
    optimizer.step()                  # weights move a little
    optimizer.zero_grad()

Nothing is inserted or looked up. The model becomes more likely to produce sequences that look like your answers. Fine-tuning learning rates are much smaller than pretraining ones, so each pass shifts behavior gently.

In practice it is a plain text file, usually JSONL, which holds one JSON object per line. Each line is one complete example: the input your model sees in production, plus the exact output you want back.

{"messages": [
  {"role": "system", "content": "Triage support tickets. Reply with JSON only."},
  {"role": "user", "content": "I was charged twice for order 4417."},
  {"role": "assistant", "content": "{\"category\": \"billing\", \"priority\": \"high\", \"order\": \"4417\"}"}
]}

In the real file that whole object sits on a single line.

Two things matter more than volume. Keep the structure identical on every line, with the same system prompt and the same output shape. The model copies whatever you repeat, including sloppiness. And examples should come from real traffic, so training inputs match production inputs.

You normally ship two files: a training set and a validation set.

Fine-tuning is good at teaching a model how to behave, not what to know. It shifts form, style and task habits, which you can show in examples but struggle to describe in a prompt.

The usual sweet spots:

• Consistent format, tone or house style. Examples beat a long, brittle style prompt.
• One narrow, repeated task, such as classifying, extracting or routing. The model only needs that job done well.
• Cost and latency. A small tuned model can match a large prompted one.
• Unusual output shapes or tool-call patterns. They are easier to demonstrate than to explain.

It also shortens prompts. Behavior baked into the weights does not need repeating on every call.

Fine-tuning is the wrong tool because it changes behavior, not storage. It shapes style, format and task shape. Facts land in the weights only as fuzzy statistics, so the model learns the shape of a confident answer and invents the details.

Three concrete problems:

• Recall is unreliable. A fact needs many repeated paraphrases before it sticks. A small SFT run rarely gets there.
• Facts change. Every correction means another training run and another eval pass. Retrieval needs one document edit.
• It can increase hallucination. Training on facts the base model never saw rewards confident guessing, and that habit spreads to other questions.

You also lose citations, per-user access control and instant removal.

As a rule of thumb, use RAG for what the model should know and fine-tuning for how it should answer.

Low-Rank Adaptation (LoRA) freezes the original model weights and trains a small set of extra weights beside them, called an adapter. The base model never changes, and the adapter carries your new behavior.

It became the default because it removes most of the cost without much loss in quality.

• You train a tiny fraction of the parameters, so one mid-range GPU is often enough.
• The result is an adapter file of megabytes, not a full model copy of gigabytes.
• One base model can serve many adapters, and you can swap or delete one instantly.
• On style, format and narrow task work, it usually matches full fine-tuning.

Hosted services run it too. Together AI trains LoRA by default, and Azure OpenAI in Microsoft Foundry fine-tunes with LoRA.