Skip to content

3. Planning

Intermediate · 12 min read

A ReAct agent decides one step at a time. That's flexible, but on longer tasks it wanders: it repeats searches, forgets sub-goals, and spends many model calls reasoning about trivial next steps. Planning separates deciding what to do from doing it. A planner writes the steps once, an executor runs them, and the planner is only called again when something goes wrong.

3.1 Step-by-step vs plan-first

ReAct (step by step) Plan-and-execute
Model calls One big-model call per step One planning call, then cheap execution (small model or plain code)
Adapts to surprises Immediately At re-plan points
Keeps the overall goal in view Can drift on long tasks The plan holds the goal
Parallel steps Hard Natural: independent steps run together
Best for Short, exploratory tasks Longer tasks with a clear structure: research, reports, migrations 
flowchart LR
    G[Goal] --> P[Planner LLM<br/>writes the steps]
    P --> E[Executor<br/>runs steps, tools]
    E -->|step failed| R[Re-planner] --> E
    E -->|all done| S[Synthesiser<br/>writes the answer]

3.2 A plan with dependencies

Ask the planner for structured output: a list of steps, each with an ID, a tool, its input and the steps it depends on. Steps with no unmet dependencies can run in parallel. The standard library's graphlib works out the order:

import json
from graphlib import TopologicalSorter

# What a planner model returned for: "Compare our Q3 revenue in India and the US and suggest one action."
plan = json.loads("""[
  {"id": "s1", "tool": "sql", "input": "Q3 revenue, India", "after": []},
  {"id": "s2", "tool": "sql", "input": "Q3 revenue, US", "after": []},
  {"id": "s3", "tool": "search_docs", "input": "Q3 marketing campaigns", "after": []},
  {"id": "s4", "tool": "compare", "input": "s1 vs s2", "after": ["s1", "s2"]},
  {"id": "s5", "tool": "write", "input": "summary + one action", "after": ["s3", "s4"]}
]""")

graph = TopologicalSorter({step["id"]: step["after"] for step in plan})
graph.prepare()
batch = 1
while graph.is_active():
    ready = list(graph.get_ready())
    print(f"batch {batch}: run in parallel -> {ready}")
    graph.done(*ready)
    batch += 1
Output
batch 1: run in parallel -> ['s1', 's2', 's3']
batch 2: run in parallel -> ['s4']
batch 3: run in parallel -> ['s5']

Five steps finish in three rounds instead of five. Validate the plan before running it: unknown tools, missing dependencies and cycles (graphlib raises CycleError) should go back to the planner as an error, just like a tool error in ReAct.

3.3 Executing, failing and re-planning

Real steps fail: an API is down, a table doesn't exist, a search finds nothing. Don't restart the whole task. Send the failure and the results so far to the planner, and let it revise only the remaining steps:

def run_tool(step: dict) -> str:
    if step["input"] == "Q3 revenue, US":
        raise RuntimeError("table revenue_us does not exist")           # a realistic failure
    return f"<result of {step['tool']}({step['input']})>"

def replan(failed: dict, error: str, done: dict) -> list[dict]:
    """Stand-in for a planner call that sees the error and the results so far."""
    print(f"  re-plan: {failed['id']} failed ({error}); planner suggests the regional table instead")
    return [{"id": failed["id"] + "b", "tool": "sql", "input": "Q3 revenue from revenue_by_region WHERE region='US'",
             "after": failed["after"]}]

def execute(plan: list[dict], max_replans: int = 2) -> dict:
    results, queue, replans = {}, list(plan), 0
    while queue:
        step = queue.pop(0)
        deps = [d for d in step["after"] if d not in results]
        if deps:                                      # wait for dependencies (sequential here for clarity)
            queue.append(step)
            continue
        try:
            results[step["id"]] = run_tool(step)
            print(f"  ok   {step['id']}: {step['tool']}({step['input']})")
        except Exception as err:
            if replans == max_replans:
                raise
            replans += 1
            new_steps = replan(step, str(err), results)
            replacement = new_steps[-1]["id"]
            for s in queue:                           # later steps now wait for the replacement step
                s["after"] = [replacement if d == step["id"] else d for d in s["after"]]
            queue = new_steps + queue
    return results

results = execute([dict(s) for s in plan])
print(f"\n{len(results)} steps completed:", list(results))
Output
  ok   s1: sql(Q3 revenue, India)
  re-plan: s2 failed (table revenue_us does not exist); planner suggests the regional table instead
  ok   s2b: sql(Q3 revenue from revenue_by_region WHERE region='US')
  ok   s3: search_docs(Q3 marketing campaigns)
  ok   s4: compare(s1 vs s2)
  ok   s5: write(summary + one action)

5 steps completed: ['s1', 's2b', 's3', 's4', 's5']

Two rules keep re-planning safe: cap the number of re-plans (an agent can otherwise re-plan forever), and give the planner the error message and the results so far, not just "it failed".

3.4 Reflection: critique and revise

Reflection adds a check before the result is accepted. One call drafts, another critiques against explicit criteria, and the draft is revised until the critic is satisfied or a round limit is reached:

CRITERIA = ["mentions both countries", "gives exactly one action", "under 60 words"]

def critique(draft: str) -> list[str]:
    """Stand-in for a critic LLM; returns the criteria the draft fails."""
    problems = []
    if not ("India" in draft and "US" in draft):
        problems.append(CRITERIA[0])
    if draft.count("Action:") != 1:
        problems.append(CRITERIA[1])
    if len(draft.split()) > 60:
        problems.append(CRITERIA[2])
    return problems

drafts = [   # what the writer model produced in each round
    "India revenue grew 18% while the US fell 4%.",
    "India revenue grew 18% while the US fell 4%. Action: move 10% of US ad spend to India.",
]
for round_no, draft in enumerate(drafts, start=1):
    problems = critique(draft)
    print(f"round {round_no}: {'accepted' if not problems else 'revise: ' + ', '.join(problems)}")
    if not problems:
        break
Output
round 1: revise: gives exactly one action
round 2: accepted

Reflection works best when the critique checks specific, verifiable criteria (format, required facts, tests passing). "Is this good?" critiques tend to approve their own work. For code, the strongest critic is not an LLM at all: run the tests.

3.5 ReWOO: plan once with placeholders

ReWOO (Reasoning WithOut Observation) goes further: the planner writes every step up front, using placeholders (#E1, #E2) for results it hasn't seen yet. Workers fill them in with no extra LLM reasoning, and a final call writes the answer. It uses far fewer tokens than ReAct, but can't adapt mid-way.

rewoo_plan = [("#E1", "geocode_city", "Bengaluru"),
              ("#E2", "get_weather", "#E1"),
              ("#E3", "llm_answer", "Will it rain? Use #E2")]
evidence = {}
fake_tools = {"geocode_city": lambda x: "12.97,77.59",
              "get_weather": lambda x: f"80% rain at 18:00 for {x}",
              "llm_answer": lambda x: f"answer from: {x}"}
for var, tool, arg in rewoo_plan:
    for ref, value in evidence.items():                 # substitute earlier results into this step's input
        arg = arg.replace(ref, value)
    evidence[var] = fake_tools[tool](arg)
    print(f"{var} = {tool}({arg!r}) -> {evidence[var]!r}")
Output
#E1 = geocode_city('Bengaluru') -> '12.97,77.59'
#E2 = get_weather('12.97,77.59') -> '80% rain at 18:00 for 12.97,77.59'
#E3 = llm_answer('Will it rain? Use 80% rain at 18:00 for 12.97,77.59') -> 'answer from: Will it rain? Use 80% rain at 18:00 for 12.97,77.59'

3.6 Choosing a planning style

Task Style
2–5 steps, exploratory ReAct
Clear structure, many steps, independent parts Plan-and-execute with parallel batches
Quality bar is checkable (format, tests, facts) Add reflection
Fixed shape, cost-sensitive ReWOO, or just a workflow
Fully predictable Don't plan with an LLM: write the workflow in code

Interview questions

What is plan-and-execute, and why use it over ReAct?

A planner LLM writes the steps up front (often as structured JSON with dependencies), an executor runs them, and the planner is called again only to re-plan after failures. It keeps long tasks on track, uses fewer expensive reasoning calls, and lets independent steps run in parallel. ReAct adapts faster to surprises but can drift on long tasks.

How should an agent handle a failed step in a plan?

Give the planner the error and the results so far, and ask it to revise only the remaining steps; update the dependencies of later steps; cap the number of re-plans; and escalate to a human or fail gracefully when the cap is hit.

What is reflection in agents, and when does it help?

A critique step evaluates a draft against explicit criteria and the agent revises until it passes or a round limit is reached. It helps when quality is checkable (format, required facts, tests). Vague self-critique tends to approve its own output; external checks such as tests or validators are better critics.

Practice

  • Add a cycle to the plan in 3.2 (s1 after s5) and catch graphlib.CycleError.
  • Make the replacement step in 3.3 fail too and watch max_replans stop the run.
  • Add a fourth criterion to critique and a third draft that passes it.

Next: 4. Memory Back to: AI Agents overview · Notes overview