Skip to content

2. Preparing training data

Intermediate · 15 min read

Fine-tuning copies whatever is in your data, including the mistakes. A model trained on 500 excellent examples beats one trained on 5,000 mediocre ones, every time. Most of the work in fine-tuning is here, not in the training code.

The running example on pages 2-6: ShopKart support replies. Given a customer message, the model must reply in the company's format: a one-line category tag, then a short, friendly answer under 80 words, never promising refunds it can't authorise.

2.1 The format: chat JSONL

Almost every tool (OpenAI, Hugging Face TRL, Unsloth, Axolotl) accepts the same format: one JSON object per line, each holding a complete conversation:

import json

examples = [
    {"messages": [
        {"role": "system", "content": "You are ShopKart's support assistant."},
        {"role": "user", "content": "My order A100 arrived with a cracked screen."},
        {"role": "assistant", "content": "[damaged]\nSorry about the cracked screen! Please upload a photo in "
                                         "Orders > A100 > Report issue, and we'll arrange a free replacement "
                                         "or a refund review within 48 hours."},
    ]},
    {"messages": [
        {"role": "system", "content": "You are ShopKart's support assistant."},
        {"role": "user", "content": "where is my parcel?? ordered 6 days ago"},
        {"role": "assistant", "content": "[delivery]\nSorry for the wait! Could you share your order ID? "
                                         "I'll check the courier status right away."},
    ]},
]

with open("train.jsonl", "w", encoding="utf-8") as f:
    for ex in examples:
        f.write(json.dumps(ex, ensure_ascii=False) + "\n")

print(open("train.jsonl", encoding="utf-8").read()[:180] + "...")
Output
{"messages": [{"role": "system", "content": "You are ShopKart's support assistant."}, {"role": "user", "content": "My order A100 arrived with a cracked screen."}, {"role": "assista...

The model learns to produce the assistant turns given everything before them. Multi-turn conversations are fine: include the whole dialogue. Keep the system prompt identical in training and in production, because the model learns to behave this way in response to that prompt.

2.2 How many examples?

Goal Rough starting point
Fix a format or tone the base model nearly gets right 50-200
A new narrow task (classification into your categories, extraction) 300-1,000
Distil a large model's skill into a small model 1,000-10,000
Teach a new domain style across many tasks 5,000+

Start small: train on 100-200 great examples, evaluate, look at the failures, and add examples that cover them. Doubling data usually gives a steady but smaller improvement each time; doubling quality often helps more.

2.3 What good targets look like

The assistant messages are what the model will imitate. Each one should be exactly the answer you want in production:

  • Consistent: same format, same tone, same rules across every example. If two annotators write differently, the model learns a blur of both.
  • Correct: wrong answers are learned as eagerly as right ones.
  • Representative: the real mix of requests, including messy spelling, Hinglish, angry customers and off-topic messages, not just tidy ones.
  • Includes the hard cases: what to do when information is missing, when the request is out of scope, when to refuse or escalate. Models fine-tuned only on happy paths fail badly on everything else.

The best source is usually real interactions (past support tickets, human-written replies), cleaned and corrected by someone who knows the rules.

2.4 Validate every example

Training jobs fail (or silently learn garbage) on malformed rows. Check before you upload:

ALLOWED_ROLES = {"system", "user", "assistant"}
CATEGORIES = {"[damaged]", "[delivery]", "[refund]", "[other]"}

def validate(example: dict) -> list[str]:
    problems = []
    messages = example.get("messages")
    if not isinstance(messages, list) or len(messages) < 2:
        return ["no messages list"]
    for i, m in enumerate(messages):
        if m.get("role") not in ALLOWED_ROLES:
            problems.append(f"message {i}: bad role {m.get('role')!r}")
        if not str(m.get("content", "")).strip():
            problems.append(f"message {i}: empty content")
    if messages[-1].get("role") != "assistant":
        problems.append("last message is not from the assistant")
    reply = messages[-1].get("content", "")
    if reply.split("\n")[0] not in CATEGORIES:
        problems.append(f"reply doesn't start with a category tag: {reply[:20]!r}")
    if len(reply.split()) > 90:
        problems.append(f"reply too long ({len(reply.split())} words)")
    return problems

rows = examples + [
    {"messages": [{"role": "user", "content": "refund pls"}, {"role": "assistant", "content": ""}]},
    {"messages": [{"role": "user", "content": "Hi"}, {"role": "bot", "content": "Hello!"}]},
    {"messages": [{"role": "user", "content": "Cancel my order"},
                  {"role": "assistant", "content": "Sure, I've cancelled it."}]},
]
for n, row in enumerate(rows, 1):
    print(n, validate(row) or "ok")
Output
1 ok
2 ok
3 ['message 1: empty content', "reply doesn't start with a category tag: ''"]
4 ["message 1: bad role 'bot'", 'last message is not from the assistant', "reply doesn't start with a category tag: 'Hello!'"]
5 ['reply doesn\'t start with a category tag: "Sure, I\'ve cancelled"']

Row 5 is the dangerous kind: valid JSON, but the target breaks your format and promises something the assistant can't do. Validation rules should encode your business rules, not just the file format.

2.5 Deduplicate and remove PII

Duplicates make the model over-weight some examples and leak between your train and test sets. Normalise before comparing:

import re
import hashlib

def fingerprint(example: dict) -> str:
    user_text = " ".join(m["content"] for m in example["messages"] if m["role"] == "user")
    normalised = re.sub(r"[^a-z0-9 ]", "", user_text.lower())
    normalised = re.sub(r"\s+", " ", normalised).strip()
    return hashlib.sha1(normalised.encode()).hexdigest()

def dedupe(rows: list[dict]) -> list[dict]:
    seen, kept = set(), []
    for row in rows:
        fp = fingerprint(row)
        if fp not in seen:
            seen.add(fp)
            kept.append(row)
    return kept

def ticket(text, reply="[delivery]\nLet me check that for you."):
    return {"messages": [{"role": "user", "content": text}, {"role": "assistant", "content": reply}]}

raw = [ticket("Where is my parcel?"), ticket("where is my parcel"), ticket("WHERE IS MY PARCEL??"),
       ticket("My parcel hasn't arrived")]
print(len(raw), "->", len(dedupe(raw)))
Output
4 -> 2

Real customer data contains phone numbers, emails, addresses and card details. A fine-tuned model can memorise and repeat them, so scrub PII before training:

PII_PATTERNS = {
    "[EMAIL]": r"[\w.+-]+@[\w-]+\.[\w.]+",
    "[PHONE]": r"(?:\+91[\s-]?)?[6-9]\d{4}[\s-]?\d{5}\b",
    "[CARD]": r"\b(?:\d[ -]?){13,16}\b",
    "[PIN]": r"\b[1-9]\d{5}\b",
}

def scrub(text: str) -> str:
    for placeholder, pattern in PII_PATTERNS.items():
        text = re.sub(pattern, placeholder, text)
    return text

print(scrub("I'm Asha, asha.rao@gmail.com, call 98765 43210. Card 4111 1111 1111 1111, pin 751024."))
Output
I'm Asha, [EMAIL], call [PHONE]. Card [CARD], pin [PIN].

Regexes catch the obvious cases; for production datasets, use a dedicated PII detector (such as Microsoft Presidio) and spot-check by hand. Names are much harder to catch than numbers.

2.6 Split: train, validation, test

  • Train (~80%): what the model learns from.
  • Validation (~10%): used during training to spot overfitting.
  • Test (~10%): never used for training or tuning. Your final, honest score. Keep it fixed across experiments.

Split by label so rare categories appear in every set:

import random
from collections import defaultdict

def stratified_split(rows, label_of, val=0.1, test=0.1, seed=42):
    by_label = defaultdict(list)
    for row in rows:
        by_label[label_of(row)].append(row)
    rng = random.Random(seed)
    train, valid, testset = [], [], []
    for label, group in sorted(by_label.items()):
        rng.shuffle(group)
        n_test, n_val = max(1, round(len(group) * test)), max(1, round(len(group) * val))
        testset += group[:n_test]
        valid += group[n_test:n_test + n_val]
        train += group[n_test + n_val:]
    return train, valid, testset

labels = ["[delivery]"] * 50 + ["[damaged]"] * 25 + ["[refund]"] * 20 + ["[other]"] * 5
rows = [ticket(f"message {i}", f"{label}\nreply") for i, label in enumerate(labels)]
label_of = lambda r: r["messages"][-1]["content"].split("\n")[0]

train, valid, testset = stratified_split(rows, label_of)
for name, part in [("train", train), ("valid", valid), ("test", testset)]:
    counts = defaultdict(int)
    for r in part:
        counts[label_of(r)] += 1
    print(f"{name:5} {len(part):3}  {dict(sorted(counts.items()))}")
Output
train  80  {'[damaged]': 21, '[delivery]': 40, '[other]': 3, '[refund]': 16}
valid  10  {'[damaged]': 2, '[delivery]': 5, '[other]': 1, '[refund]': 2}
test   10  {'[damaged]': 2, '[delivery]': 5, '[other]': 1, '[refund]': 2}

If customers write in repeatedly, split by customer (or by conversation), not by message, so near-identical messages don't end up on both sides.

2.7 Check token lengths

Very long examples get truncated during training (the model never sees the end, which is the part it's supposed to learn) and cost more. Look at the distribution:

def approx_tokens(example: dict) -> int:
    return sum(len(m["content"]) for m in example["messages"]) // 4 + 4 * len(example["messages"])

lengths = sorted(approx_tokens(r) for r in examples * 50 + [ticket("x" * 12_000)])
p50, p95, longest = lengths[len(lengths) // 2], lengths[int(len(lengths) * 0.95)], lengths[-1]
print(f"median {p50} tokens, p95 {p95}, max {longest}")
print("too long for max_length=1024:", sum(n > 1024 for n in lengths))
Output
median 74 tokens, p95 74, max 3017
too long for max_length=1024: 1

Use a real tokenizer (tiktoken, or the model's own tokenizer from Hugging Face) for exact numbers. Set the training max_length above your p95, and drop or split the outliers.

2.8 Synthetic data from a stronger model

When you don't have enough real examples, a strong model can generate them: first varied user messages, then ideal replies following your rules. This is also how distillation works.

# no-run — needs ANTHROPIC_API_KEY
import anthropic
claude = anthropic.Anthropic()

RULES = open("support_rules.md").read()      # your format, tone and policies, written once

def generate_reply(customer_message: str) -> str:
    response = claude.messages.create(
        model="claude-opus-5-5", max_tokens=1024,
        system=f"Write the ideal ShopKart support reply.\n\n{RULES}",
        messages=[{"role": "user", "content": customer_message}],
    )
    return response.content[0].text

Rules for synthetic data:

  • Seed it with real examples so the variety is realistic, and ask explicitly for messy, angry, multilingual and edge-case messages.
  • Validate and review: run the validator from 2.4, and have a person read a random sample of at least 50.
  • Never generate your test set this way. Test on real data only, or you're measuring how well the small model copies the big one's quirks.
  • Check the model provider's terms on using outputs to train other models.

Interview questions

What does a fine-tuning dataset look like?

Usually chat-format JSONL: one JSON object per line with a messages list of system, user and assistant turns. The model learns to produce the assistant turns. For preference tuning, each row has a prompt plus a chosen and a rejected response.

How many examples do you need to fine-tune?

It depends on the task: 50-200 can fix a format the model nearly follows, a few hundred to a thousand for a new narrow task, thousands for distillation. Start with a small, high-quality set, evaluate, and add examples that target the observed failures. Quality and consistency matter more than volume.

How do you prevent data leakage between train and test sets?

Deduplicate after normalising text, split by customer or conversation rather than by message when similar messages repeat, keep the test set fixed and never use it for training or hyperparameter choices, and don't generate the test set synthetically.

What are the risks of training on customer data?

Memorisation and regurgitation of PII, learning agents' mistakes or bad habits, and legal or consent issues. Scrub PII (regex plus a detector like Presidio, then manual spot checks), correct or drop bad replies, and confirm you're allowed to use the data for training.

Practice

  • Add a rule to validate that rejects replies containing "refund processed" (only humans can approve refunds).
  • Write 20 real-looking customer messages, including five messy ones, and check them with the validator.
  • Extend scrub to catch Aadhaar-style 12-digit numbers written as 1234 5678 9012.

Next: 3. How fine-tuning works: LoRA and QLoRA Back to: Fine-Tuning overview · Notes overview