Skip to content

5. Structured output prompting

Intermediate · 12 min read

When code consumes the model's answer, prose isn't enough: you need fields you can trust. Native structured output (JSON schema mode) guarantees valid JSON on providers that support it. But valid JSON can still hold wrong values: a guessed date, an invented field, a label outside your list. This page is about the prompt and the checks around it.

5.1 Describe every field, not just its name

A field name is a weak hint. Say what it means, its format, and what to do when it's missing:

Extract these fields from the invoice text. Reply with JSON only.

- invoice_number: string exactly as printed, e.g. "INV-2026-0042"
- invoice_date: ISO date YYYY-MM-DD; convert "3rd Oct 2026" to "2026-10-03"
- total_inr: number, the final amount payable including GST, without the ₹ sign or commas
- gst_number: the 15-character GSTIN of the SELLER, or null if not printed
- category: one of "software", "hardware", "services", "other"

If a field isn't in the text, use null. Never guess or calculate a value that isn't printed.

The last line matters most. Without it, models fill gaps with plausible inventions.

5.2 Validate in code

Even with schema mode, check values: types, enums, formats and cross-field rules.

import json
import re
from datetime import date

CATEGORIES = {"software", "hardware", "services", "other"}

def validate(raw: str) -> tuple[dict | None, list[str]]:
    try:
        data = json.loads(raw)
    except json.JSONDecodeError as e:
        return None, [f"not valid JSON: {e.msg}"]
    errors = []
    if not re.fullmatch(r"INV-\d{4}-\d{4}", str(data.get("invoice_number"))):
        errors.append("invoice_number: expected INV-YYYY-NNNN")
    try:
        date.fromisoformat(str(data.get("invoice_date")))
    except ValueError:
        errors.append("invoice_date: expected YYYY-MM-DD")
    if not isinstance(data.get("total_inr"), (int, float)) or data["total_inr"] <= 0:
        errors.append("total_inr: expected a positive number")
    gst = data.get("gst_number")
    if gst is not None and not re.fullmatch(r"[0-9A-Z]{15}", str(gst)):
        errors.append("gst_number: expected 15 characters or null")
    if data.get("category") not in CATEGORIES:
        errors.append(f"category: must be one of {sorted(CATEGORIES)}")
    return data, errors

good = '{"invoice_number": "INV-2026-0042", "invoice_date": "2026-10-03", "total_inr": 11800, "gst_number": null, "category": "software"}'
bad = '{"invoice_number": "42", "invoice_date": "3rd Oct", "total_inr": "11,800", "category": "cloud"}'
print(validate(good)[1])
for err in validate(bad)[1]:
    print("-", err)
Output
[]
- invoice_number: expected INV-YYYY-NNNN
- invoice_date: expected YYYY-MM-DD
- total_inr: expected a positive number
- category: must be one of ['hardware', 'other', 'services', 'software']

5.3 Repair: send the errors back

When validation fails, retry with the specific errors. The model usually fixes them on the first retry. Cap the retries and fall back to a human queue.

def fake_llm(prompt: str) -> str:
    """First call returns a sloppy answer; a repair prompt gets a corrected one."""
    if "Fix these errors" in prompt:
        return '{"invoice_number": "INV-2026-0042", "invoice_date": "2026-10-03", "total_inr": 11800, "gst_number": null, "category": "software"}'
    return bad

def extract(text: str, max_attempts: int = 3) -> dict:
    prompt = f"Extract the invoice fields as JSON.\n<invoice>{text}</invoice>"
    for attempt in range(1, max_attempts + 1):
        raw = fake_llm(prompt)
        data, errors = validate(raw)
        print(f"attempt {attempt}: {len(errors)} error(s)")
        if not errors:
            return data
        prompt = (f"Your previous reply:\n{raw}\n\nFix these errors and reply with corrected JSON only:\n"
                  + "\n".join(f"- {e}" for e in errors))
    raise ValueError("extraction failed; send to the review queue")

print(extract("Invoice INV-2026-0042 dated 3rd Oct 2026 ... Total ₹11,800"))
Output
attempt 1: 4 error(s)
attempt 2: 0 error(s)
{'invoice_number': 'INV-2026-0042', 'invoice_date': '2026-10-03', 'total_inr': 11800, 'gst_number': None, 'category': 'software'}

5.4 Let the model say "I'm not sure"

For each uncertain field, a confidence or evidence field makes review cheap: reviewers check only the low-confidence fields, and the evidence shows where the value came from.

{
  "total_inr": {"value": 11800, "evidence": "Grand Total: ₹11,800.00", "confidence": "high"},
  "gst_number": {"value": null, "evidence": null, "confidence": "high"}
}

Verify the evidence in code: the quoted text must actually appear in the source. That catches invented values.

5.5 Classification prompts

Classification is structured output with one field. Three rules make it reliable:

  1. Define each label in one line, with the boundary between similar labels ("billing = payments and invoices; refund requests are refund, not billing").
  2. Always include an escape label (other or unclear), so the model isn't forced into a wrong class.
  3. Ask for the label after a one-line reason when labels are subtle, or the label alone when speed matters.

5.6 Prompt-only JSON vs native structured output

Prompt-only ("reply with JSON") Native schema mode / tool calling
Valid JSON Usually, not guaranteed Guaranteed on supported models
Correct values Needs validation Still needs validation
Works with any model Yes Only providers that support it
Best for Quick prototypes, small local models Production extraction

Use native structured output when you can, and keep the field descriptions, nulls rule and value validation either way.

Interview questions

How do you stop an LLM from inventing values during extraction?

Tell it explicitly to use null for anything not in the text and never to guess or calculate; describe each field precisely; ask for the evidence quote per field and check in code that it appears in the source; and validate formats and ranges before using the data.

Native JSON mode guarantees valid JSON. Why validate anyway?

Valid syntax isn't correct content. The model can still pick a value outside your business rules, misread a date, choose a wrong label or invent a field value. Validate types, enums, formats and cross-field rules, and repair or escalate failures.

What do you do when the output fails validation?

Retry with the specific validation errors and the previous reply, which usually fixes it on the first attempt. Cap retries, log failures with the prompt version, and send persistent failures to a human review queue.

Practice

  • Add a rule to validate: invoice_date can't be in the future.
  • Change fake_llm so the repair also fails, and confirm extract raises after 3 attempts.

Next: 6. Prompt injection & defensive prompting Back to: Prompt Engineering overview · Notes overview