Get structured JSON from an LLM

Beginner → Intermediate · 12 min · tested with openai 3.x and pydantic 2

Chatbots return prose. Real applications need data: a ticket category to route on, fields to save in a database, a list of extracted entities. The goal is to go from a messy customer email to a typed Python object you can trust, every time.

---

1. Why "reply in JSON" isn't enough

Asking nicely mostly works, until it doesn't:

Sure! Here's the JSON you asked for:
```json
{"intent": "refund", "urgency": 4}
```

The model added a friendly sentence and a code fence, so json.loads() crashes. Other days it forgets a field, invents a category you don't handle, or writes "urgency": "high" instead of a number. You can patch around it:

import json
import re

def extract_json(text: str) -> dict:
    match = re.search(r"\{.*\}", text, re.S)        # grab from the first { to the last }
    return json.loads(match.group(0))

print(extract_json('Sure! Here is the JSON:\n```json\n{"intent": "refund", "urgency": 4}\n```'))
# → {'intent': 'refund', 'urgency': 4}

But that only fixes the wrapping, not wrong fields or values. The real fix is to give the model a schema and let the API enforce it.

---

2. Describe the output as a Pydantic model

from typing import Literal
from pydantic import BaseModel, Field

class SupportTicket(BaseModel):
    name: str
    email: str
    intent: Literal["refund", "delivery", "complaint", "other"]
    order_id: str | None = Field(description="Order ID like A100, or null if not mentioned")
    urgency: int = Field(ge=1, le=5, description="1 = can wait, 5 = urgent")

Each piece helps the model:

• Literal gives a closed list of choices, so you never get a category your code doesn't handle.
• str | None with a description tells it what to do when the information is missing (instead of inventing an order ID).
• Field(description=...) is read by the model. Write it like an instruction.

---

3. Structured Outputs with the OpenAI SDK

Pass the model class as response_format to .parse(). The API constrains generation to your schema, and the SDK hands back an instance of your class:

from openai import OpenAI

client = OpenAI()

email = """Hi, this is Asha Rao (asha@example.com). My order A100 arrived broken
and I need my money back before the weekend, please!"""

completion = client.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Extract a support ticket from the customer's email."},
        {"role": "user", "content": email},
    ],
    response_format=SupportTicket,
)

message = completion.choices[0].message
if message.refusal:                          # the model can decline (e.g. harmful content)
    print("Refused:", message.refusal)
else:
    ticket = message.parsed                  # a SupportTicket, already validated
    print(ticket)
    print(ticket.intent, ticket.urgency)
# → name='Asha Rao' email='asha@example.com' intent='refund' order_id='A100' urgency=4
# → refund 4

ticket is a real object: ticket.urgency is an int, your editor autocompletes the fields, and ticket.model_dump() gives a dict for your database. Behind the scenes the SDK converted the class to a JSON schema and sent it as response_format={"type": "json_schema", ...}.

Anthropic (Claude) versionOlder models / other providers: JSON mode

import anthropic

claude = anthropic.Anthropic()

response = claude.messages.parse(
    model="claude-opus-5-5",
    max_tokens=1024,
    system="Extract a support ticket from the customer's email.",
    messages=[{"role": "user", "content": email}],
    output_format=SupportTicket,
)
ticket = response.parsed_output          # a SupportTicket
print(ticket.intent, ticket.urgency)

If a provider or model doesn't support schemas, JSON mode at least guarantees valid JSON (not the right fields). Mention JSON and the fields in the prompt, then validate yourself:

reply = client.chat.completions.create(
    model="gpt-4o-mini",
    response_format={"type": "json_object"},
    messages=[
        {"role": "system", "content": "Return JSON with keys: name, email, intent, order_id, urgency (1-5)."},
        {"role": "user", "content": email},
    ],
)
ticket = SupportTicket.model_validate_json(reply.choices[0].message.content)

---

4. Validate and retry

Schemas guarantee the shape. They can't guarantee the values make sense, and JSON mode guarantees even less. Pydantic catches what's wrong, with messages precise enough to send back to the model:

from pydantic import ValidationError

bad = '{"name": "Ravi", "email": "r@x.com", "intent": "cancel", "order_id": null, "urgency": 9}'
try:
    SupportTicket.model_validate_json(bad)
except ValidationError as err:
    for e in err.errors():
        print(e["loc"], e["msg"])
# → ('intent',) Input should be 'refund', 'delivery', 'complaint' or 'other'
# → ('urgency',) Input should be less than or equal to 5

A retry loop that feeds the errors back:

def extract_ticket(email: str, max_attempts: int = 3) -> SupportTicket:
    messages = [
        {"role": "system", "content": "Return JSON with keys: name, email, intent, order_id, urgency (1-5)."},
        {"role": "user", "content": email},
    ]
    for attempt in range(max_attempts):
        reply = client.chat.completions.create(
            model="gpt-4o-mini", response_format={"type": "json_object"}, messages=messages,
        )
        text = reply.choices[0].message.content
        try:
            return SupportTicket.model_validate_json(text)
        except ValidationError as err:
            messages += [
                {"role": "assistant", "content": text},
                {"role": "user", "content": f"That JSON is invalid:\n{err}\nReturn corrected JSON only."},
            ]
    raise ValueError(f"No valid ticket after {max_attempts} attempts")

Add your own business rules as Pydantic validators (for example, order_id must match ^[A-Z]\d{3}$), and they get the same retry treatment.

---

5. Extract many items at once

Wrap a list in a model to pull every item out of a document in one call:

class Product(BaseModel):
    name: str
    price_inr: int | None
    in_stock: bool

class ProductList(BaseModel):
    products: list[Product]

completion = client.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Extract products: Blue kurta Rs 899 (in stock), "
                                          "steel bottle Rs 349 (sold out), gift card (price on request)"}],
    response_format=ProductList,
)
for p in completion.choices[0].message.parsed.products:
    print(p)

Most APIs need the top level to be an object, which is why the list is wrapped in ProductList.

---

Tips for schemas the model fills well

Do	Why
Use Literal / enums for categories	No surprise values
Make "might be missing" fields X | None and say when to use null	Stops invented data
Put a reasoning: str field first when the task needs judgment	The model "thinks" before answering the later fields
Keep nesting shallow and field names obvious	Fewer mistakes, cheaper tokens
Use temperature=0 for extraction	Consistent results
Don't stream when you must validate before showing	You can't validate half a JSON object

---

Checklist

• Output shape is a Pydantic model, passed as response_format (or output_format for Claude).
• Refusals are handled.
• Categories are Literals; optional fields allow None with a description.
• Business rules are validators; failures are retried with the error message, a limited number of times.
• Extraction runs at temperature 0, and a few real examples are covered by tests.

See Structured output & tool calling and Pydantic for GenAI for the deeper version.

Next: Count tokens and estimate cost →