3. Reasoning techniques¶
Intermediate · 13 min read
A model generates one token at a time, and each token can only build on what came before. If the answer comes first, there's no room to work it out. Reasoning techniques give the model space to think before it commits, or check its work after. They cost extra tokens and time, so use them where accuracy on multi-step problems matters.
3.1 Chain of thought, with tags¶
Ask for the reasoning first and the final answer in a separate, parseable place:
Work through the problem step by step inside <thinking> tags.
Then give only the final answer inside <answer> tags.
import re
def parse(reply: str) -> dict:
thinking = re.search(r"<thinking>(.*?)</thinking>", reply, re.S)
answer = re.search(r"<answer>(.*?)</answer>", reply, re.S)
return {"thinking": thinking.group(1).strip() if thinking else "",
"answer": answer.group(1).strip() if answer else None}
reply = """<thinking>3 seats at ₹1,200 = ₹3,600. Group discount 10% for 3+ seats: ₹360 off -> ₹3,240.
Convenience fee ₹40 per seat = ₹120. Total ₹3,360.</thinking>
<answer>₹3,360</answer>"""
print(parse(reply)["answer"])
print(parse("The total is ₹3,360.")["answer"]) # no tags: treat as a failure and retry
Show users only the answer. Log the reasoning: it's the fastest way to see why a wrong answer happened.
3.2 Self-consistency: ask several times, take the majority¶
For questions with one correct answer (a number, a label), sample the model several times at a higher temperature and vote. Independent reasoning paths that agree are more likely to be right.
from collections import Counter
def fake_llm(question: str, seed: int) -> str:
"""Stands in for a model sampled at temperature 0.8: usually right, sometimes not."""
answers = ["₹3,360", "₹3,360", "₹3,240", "₹3,360", "₹3,480"]
return f"<thinking>...</thinking><answer>{answers[seed % len(answers)]}</answer>"
def self_consistency(question: str, n: int = 5) -> tuple[str, float]:
votes = Counter(parse(fake_llm(question, seed=i))["answer"] for i in range(n))
answer, count = votes.most_common(1)[0]
return answer, count / n
answer, agreement = self_consistency("Total for 3 seats?")
print(f"{answer} (agreement {agreement:.0%})")
The agreement rate is a free confidence signal: below a threshold (say 60%), route to a human or a stronger model. The cost is n times one call, so keep it for high-value decisions.
3.3 Decomposition: least-to-most¶
For a hard problem, first ask the model to list the sub-questions, then answer them in order, feeding each answer into the next. Each step is easier than the whole.
Step 1 prompt: "List the sub-questions needed to answer: <question>. Output one per line, simplest first."
Step 2 prompt: "Answer sub-question 1. Facts so far: <none>"
Step 3 prompt: "Answer sub-question 2. Facts so far: <answer 1>"
...
Final prompt: "Using these facts, answer the original question: <question>"
This is prompt chaining applied to reasoning, and it's the idea behind agent planning.
3.4 Step-back prompting¶
Before answering a specific question, ask a more general one first and use its answer as context:
| Specific question | Step-back question |
|---|---|
| "Can my 2019 contract be ended with 15 days' notice?" | "What are the general rules for ending a contract under this agreement?" |
| "Why does this Python loop leak memory?" | "What are common causes of memory growth in long-running Python loops?" |
The general answer brings the relevant principles into the context, and the specific answer then reasons from them. In RAG it also improves retrieval: search with both questions.
3.5 Self-critique and revision¶
Generate, then ask the model (or a second prompt) to check the draft against explicit criteria and fix it:
CRITIQUE = """Review the draft reply below against these checks:
1. Does every number match the order details?
2. Does it promise anything the refund policy doesn't allow?
3. Is it under 80 words?
List each failed check, then write a corrected reply inside <final> tags.
If all checks pass, copy the draft into <final> unchanged.
<order>{order}</order>
<policy>{policy}</policy>
<draft>{draft}</draft>"""
print(CRITIQUE.format(order="Order 1182, ₹1,499, delivered 2 Oct",
policy="Refunds within 7 days of delivery",
draft="Sure! We'll refund ₹1,999 right away."))
Review the draft reply below against these checks:
1. Does every number match the order details?
2. Does it promise anything the refund policy doesn't allow?
3. Is it under 80 words?
List each failed check, then write a corrected reply inside <final> tags.
If all checks pass, copy the draft into <final> unchanged.
<order>Order 1182, ₹1,499, delivered 2 Oct</order>
<policy>Refunds within 7 days of delivery</policy>
<draft>Sure! We'll refund ₹1,999 right away.</draft>
Specific checks work; "check your answer" alone mostly makes the model approve its own draft. Critique works best when the checker sees something the writer didn't: the source data, a rubric, or test results for code.
3.6 Prompting reasoning models¶
Reasoning models already think internally before answering. Prompt them differently:
| Standard model | Reasoning model |
|---|---|
| Ask for step-by-step thinking | Don't. It already reasons; scripted steps can make it worse |
| Many worked examples | Clear goal, constraints and success criteria; few or no examples |
| Short context is fine | Give all the relevant data; it can use it well |
| Cheap and fast | Set an effort level or thinking budget; use for hard tasks only |
3.7 Which technique when¶
| Problem | Try |
|---|---|
| Multi-step maths, logic, policy rules | Chain of thought with tags, or a reasoning model |
| One correct answer, high stakes | Self-consistency with an agreement threshold |
| Long, many-part task | Decomposition into a chain |
| Needs background principles | Step-back question first |
| Output must meet a checklist | Generate, then critique against the checklist |
| Simple extraction or classification | None of these; they add cost without gain |
Interview questions¶
Why does chain-of-thought prompting improve accuracy?
The model generates tokens left to right, and each token can only build on earlier ones. Writing intermediate steps first gives later tokens, including the final answer, more relevant computation to build on. Separating the reasoning and answer with tags keeps the output parseable.
What is self-consistency, and what does it cost?
Sample several reasoning paths at a non-zero temperature, extract each final answer and take the majority. It improves accuracy on problems with one correct answer and gives an agreement-based confidence score, at n times the cost and latency of one call.
How should you prompt a reasoning model differently?
Give it the goal, constraints, success criteria and all relevant data, but don't script its steps or ask it to think step by step. Control cost with an effort level or thinking budget, and use it only where the task benefits.
Practice¶
- Make
self_consistencyreturnNonewhen agreement is below 60%, and test it with a different answer list. - Write a step-back question for: "Should this RAG chatbot use chunks of 300 or 800 tokens?"
Next: 4. Prompt chaining & routing Back to: Prompt Engineering overview · Notes overview