4. Fine-tuning with an API¶

Intermediate · 12 min read

The fastest way to fine-tune is to let a provider do it: upload your JSONL, start a job, and get back a model name you call like any other. No GPUs, no training code. The trade-offs: you can only fine-tune the models the provider offers, the model stays on their platform, and you pay their training and inference prices.

This page uses the OpenAI fine-tuning API (the most widely used) with the dataset from page 2. Other managed options are in 4.6.

Which models can be fine-tuned changes

Providers add and retire fine-tunable models regularly. Check the provider's fine-tuning docs for the current list and prices before you start; the model name below is an example.

4.1 Estimate the cost first¶

Training is billed per token processed: every token in your file, once per epoch.

def training_cost_usd(examples: int, avg_tokens: int, epochs: int, price_per_million: float) -> float:
    return examples * avg_tokens * epochs / 1e6 * price_per_million

# 800 examples x ~350 tokens, 3 epochs; example price $3 per 1M training tokens
print(f"${training_cost_usd(800, 350, 3, 3.00):.2f}")
# a much bigger distillation set
print(f"${training_cost_usd(20_000, 900, 2, 3.00):.2f}")

Output

$2.52
$108.00

Training a small task model is usually cheap. The bigger costs are building the data and the inference price: fine-tuned models often cost more per token than the same base model, which is worth checking against the prompt tokens you save.

4.2 Upload the files¶

# no-run — needs OPENAI_API_KEY; pip install openai
from openai import OpenAI

client = OpenAI()

train_file = client.files.create(file=open("train.jsonl", "rb"), purpose="fine-tune")
valid_file = client.files.create(file=open("valid.jsonl", "rb"), purpose="fine-tune")
print(train_file.id, valid_file.id)          # file-abc123..., file-def456...

Run your own validator (page 2) first. The API checks the format, but not your business rules.

4.3 Start a supervised fine-tuning job¶

# no-run
job = client.fine_tuning.jobs.create(
    model="gpt-4o-mini-2024-07-18",            # an example fine-tunable base model
    training_file=train_file.id,
    validation_file=valid_file.id,
    suffix="shopkart-support",                 # appears in the new model's name
    seed=42,
    method={
        "type": "supervised",
        "supervised": {"hyperparameters": {"n_epochs": 3}},   # or "auto"
    },
)
print(job.id, job.status)                      # ftjob-..., validating_files

Leave the other hyperparameters (batch_size, learning_rate_multiplier) on "auto" for the first run. Change one thing at a time afterwards, guided by the validation loss and your evals.

4.4 Monitor it¶

# no-run
import time

while True:
    job = client.fine_tuning.jobs.retrieve(job.id)
    print(job.status, job.trained_tokens)
    if job.status in ("succeeded", "failed", "cancelled"):
        break
    time.sleep(60)

for event in client.fine_tuning.jobs.list_events(job.id, limit=10):
    print(event.created_at, event.message)     # step N: training loss=..., validation loss=...

print(job.fine_tuned_model)                    # ft:gpt-4o-mini-2024-07-18:your-org:shopkart-support:AbC123

Jobs take minutes to hours depending on size and queue. The dashboard shows the training and validation loss curves; watch for the overfitting pattern from page 3. Intermediate checkpoints (client.fine_tuning.jobs.checkpoints.list(job.id)) are usable models too: if the last epoch overfit, an earlier checkpoint may evaluate better.

4.5 Use the fine-tuned model¶

It's just a model name. Use the same system prompt as in training:

# no-run
reply = client.chat.completions.create(
    model=job.fine_tuned_model,
    messages=[
        {"role": "system", "content": "You are ShopKart's support assistant."},
        {"role": "user", "content": "parcel says delivered but nothing came!!"},
    ],
    temperature=0,
)
print(reply.choices[0].message.content)

Then evaluate it against the base model with a good prompt, on your test set (page 6), before switching any traffic.

Preference tuning with DPO¶

When you can say which of two answers is better more easily than you can write the perfect answer, use Direct Preference Optimization. Each row has a prompt, a preferred reply and a worse one:

{"input": {"messages": [{"role": "user", "content": "I want a refund NOW"}]},
 "preferred_output": [{"role": "assistant", "content": "[refund]\nI understand, and I'm sorry. Could you share the order ID? I'll start a refund review right away."}],
 "non_preferred_output": [{"role": "assistant", "content": "[refund]\nRefunds take 5-7 days. Please be patient."}]}

# no-run
job = client.fine_tuning.jobs.create(
    model=sft_model_name,                      # commonly: DPO on top of your SFT model
    training_file=dpo_file.id,
    method={"type": "dpo", "dpo": {"hyperparameters": {"beta": 0.1}}},
)

beta controls how far the model may move from the starting model: higher keeps it closer. Preference pairs are also easy to collect from production: thumbs up/down, or an agent's edits to a drafted reply.

4.6 Other managed options¶

Option	Models	Notes
OpenAI	Selected GPT models	SFT, DPO and reinforcement fine-tuning; model stays on OpenAI
Google Vertex AI	Gemini and open models	Supervised tuning, integrated with Google Cloud
Amazon Bedrock	Selected Amazon, Meta and partner models	Custom models inside your AWS account
Azure AI Foundry	Selected OpenAI and open models	Enterprise controls on Azure
Together AI, Fireworks, Hugging Face	Open models (Llama, Qwen, Gemma, Mistral…)	LoRA fine-tuning as a service; often you can download the adapter and serve it anywhere

If you need to own the weights, run on-premise, or fine-tune a model no provider offers, train it yourself: next page.

Interview questions¶

Walk me through fine-tuning a model with the OpenAI API.

Prepare and validate chat-format JSONL with train and validation splits; upload them with files.create(..., purpose="fine-tune"); create a job with fine_tuning.jobs.create specifying the base model, files and method (supervised, with epochs); monitor status, events and loss; then call the returned ft:... model name with the same system prompt used in training, and evaluate it against a well-prompted base model on a held-out test set.

How is fine-tuning priced?

Training is billed per token processed (dataset tokens × epochs). Inference on the fine-tuned model is billed per token, often at a higher rate than the base model. Total cost of ownership also includes data preparation, evaluation runs and retraining when the base model is retired.

What's DPO and when would you use it?

Direct Preference Optimization trains on pairs of a preferred and a non-preferred response to the same prompt, pushing the model toward the preferred style without a separate reward model. Use it after SFT when quality differences are easier to judge than to write, such as tone, helpfulness or conciseness.

Managed fine-tuning or training it yourself?

Managed: fastest, no infrastructure, but limited to offered models, data goes to the provider, and you're tied to their inference. Self-hosted: full control of open models and weights, on-premise options, any technique, but you need GPUs, ML engineering and serving infrastructure.

Practice¶

Estimate the training cost of your own dataset with the calculator in 4.1.
Write five DPO preference pairs for a task you know well.
Find the current list of fine-tunable models and prices for one provider and note what changed recently.

Next: 5. LoRA fine-tuning an open model Back to: Fine-Tuning overview · Notes overview