Skip to content

7. Multi-agent patterns

Advanced · 13 min read

A single agent with good tools solves most problems. Multiple agents help when one agent's prompt and tool list grow too large to manage: each specialist gets a short prompt and a few tools, and a coordinator decides who works next. The price is more model calls, more latency and more ways to fail. This page covers the main patterns, built from scratch, and when each one is worth it.

7.1 The patterns

Pattern How control flows Good for
Supervisor A coordinator agent picks the next specialist, reviews results, decides when done Clear specialist roles: research, SQL, writing
Handoffs (swarm) The active agent transfers the conversation to another agent Customer conversations: triage → billing → tech support
Hierarchical Supervisors of supervisors; each team is a sub-graph Large systems split by department or domain
Parallel fan-out Several agents work on parts at once; a step merges results Independent sub-tasks: multi-source research, several reviewers
Debate / critique Agents argue or review each other; a judge decides High-stakes answers where quality beats cost

7.2 Supervisor, from scratch

The supervisor sees the goal and a shared blackboard of results so far, and returns the next worker or "FINISH". Workers are ordinary agents (or plain functions) with their own tools:

WORKERS = {
    "sql_analyst": lambda task, board: "India Q3 revenue ₹4.2 Cr (+18%); US ₹6.1 Cr (−4%).",
    "researcher":  lambda task, board: "Q3: India ran a festival campaign; US paused paid search in August.",
    "writer":      lambda task, board: ("India grew 18% on the festival campaign while the US fell 4% after paid "
                                        "search was paused. Action: restart US paid search in Q4."),
}

def supervisor(goal: str, board: dict) -> str:
    """Stand-in for a supervisor LLM with structured output: {"next": "<worker>" | "FINISH"}."""
    if "sql_analyst" not in board:
        return "sql_analyst"
    if "researcher" not in board:
        return "researcher"
    if "writer" not in board:
        return "writer"
    return "FINISH"

def run_team(goal: str, max_rounds: int = 6) -> str:
    board = {}
    for round_no in range(1, max_rounds + 1):
        nxt = supervisor(goal, board)
        print(f"round {round_no}: supervisor -> {nxt}")
        if nxt == "FINISH":
            return board["writer"]
        if nxt not in WORKERS:
            board["_error"] = f"unknown worker {nxt!r}"          # fed back to the supervisor next round
            continue
        board[nxt] = WORKERS[nxt](goal, board)
        print(f"          {nxt}: {board[nxt][:70]}")
    return "Stopped: round limit reached."

print("\nRESULT:", run_team("Explain Q3 revenue in India vs the US and suggest one action."))
Output
round 1: supervisor -> sql_analyst
          sql_analyst: India Q3 revenue ₹4.2 Cr (+18%); US ₹6.1 Cr (−4%).
round 2: supervisor -> researcher
          researcher: Q3: India ran a festival campaign; US paused paid search in August.
round 3: supervisor -> writer
          writer: India grew 18% on the festival campaign while the US fell 4% after pai
round 4: supervisor -> FINISH

RESULT: India grew 18% on the festival campaign while the US fell 4% after paid search was paused. Action: restart US paid search in Q4.

The supervisor never does the work itself; it only routes. In a real system it's a small, fast model call with structured output, and workers get only the slice of the blackboard they need, which keeps their prompts short.

7.3 Handoffs between agents

In conversational systems the active agent changes as the conversation moves. Each agent can hand off with a special tool (transfer_to_billing), and the next agent continues with the full history:

AGENTS = {
    "triage": lambda msg: ("handoff", "billing") if "refund" in msg or "charged" in msg
              else ("handoff", "tech") if "error" in msg or "crash" in msg else ("reply", "How can I help?"),
    "billing": lambda msg: ("reply", "I've started a refund for the duplicate charge: 3–5 working days."),
    "tech": lambda msg: ("reply", "Please update the app to 4.2; that fixes the crash on login."),
}

def converse(message: str, active: str = "triage", max_handoffs: int = 3) -> str:
    for _ in range(max_handoffs + 1):
        kind, value = AGENTS[active](message.lower())
        if kind == "reply":
            return f"[{active}] {value}"
        print(f"  {active} hands off to {value}")
        active = value
    return "[system] Too many handoffs: connecting you to a person."

for message in ["I was charged twice for my order", "The app crashes with an error on login"]:
    print(message)
    print(" ", converse(message))
Output
I was charged twice for my order
  triage hands off to billing
  [billing] I've started a refund for the duplicate charge: 3–5 working days.
The app crashes with an error on login
  triage hands off to tech
  [tech] Please update the app to 4.2; that fixes the crash on login.

The max_handoffs cap matters: two agents that each think the other should answer will bounce a customer forever.

7.4 Parallel fan-out and merge

When sub-tasks are independent, run them at the same time and merge the results. Wall-clock time becomes the slowest worker, not the sum:

import time
from concurrent.futures import ThreadPoolExecutor

def reviewer(focus: str) -> str:
    time.sleep(0.3)                                         # stand-in for one LLM call (~0.3 s here)
    return {"security": "No secrets in logs; add rate limiting to /login.",
            "performance": "N+1 query in order history; batch it.",
            "readability": "Rename helper `do_it` to `apply_discount`."}[focus]

start = time.perf_counter()
with ThreadPoolExecutor(max_workers=3) as pool:
    reviews = dict(zip(["security", "performance", "readability"],
                       pool.map(reviewer, ["security", "performance", "readability"])))
elapsed = time.perf_counter() - start
for focus, text in reviews.items():
    print(f"{focus:12} {text}")
print(f"3 reviews in {elapsed:.1f}s (sequential would be ~0.9s); a merge step now combines them into one report")
Output
security     No secrets in logs; add rate limiting to /login.
performance  N+1 query in order history; batch it.
readability  Rename helper `do_it` to `apply_discount`.
3 reviews in 0.3s (sequential would be ~0.9s); a merge step now combines them into one report

With async LLM clients, use asyncio.gather instead of threads. In LangGraph, the Send API fans out to parallel nodes, and a reducer merges their outputs.

7.5 The cost of more agents

Every extra agent adds model calls, and each call re-sends context. A rough comparison for the same task:

def run_cost(calls: int, avg_input_tokens: int, avg_output_tokens: int,
             price_in: float = 0.50, price_out: float = 2.00) -> float:
    """Dollars per task; prices per 1M tokens are placeholders."""
    return calls * (avg_input_tokens * price_in + avg_output_tokens * price_out) / 1e6

designs = {
    "single agent, 4 tool steps":          run_cost(5, 3000, 250),
    "supervisor + 3 workers":              run_cost(4 + 3 * 3, 2500, 300),   # 4 routing calls + 3 workers x 3 steps
    "supervisor + 3 workers + 2 critics":  run_cost(4 + 3 * 3 + 2, 2800, 300),
}
base = designs["single agent, 4 tool steps"]
for name, cost in designs.items():
    print(f"{name:36} ${cost * 1000:6.2f} per 1,000 tasks  ({cost / base:.1f}x)")
Output
single agent, 4 tool steps           $ 10.00 per 1,000 tasks  (1.0x)
supervisor + 3 workers               $ 24.05 per 1,000 tasks  (2.4x)
supervisor + 3 workers + 2 critics   $ 30.00 per 1,000 tasks  (3.0x)

That multiplier also applies to latency and to compounding failure. Add an agent only when your evals show a single agent failing because of prompt or tool overload, and when the gain is worth the cost.

7.6 Making multi-agent systems work

  • Clear contracts: each agent has one job, a short prompt, its own tools, and a defined output format.
  • Shared state, not chat: pass structured results through a blackboard or graph state, rather than having agents "talk" in free text, which drifts and costs tokens.
  • One owner of the final answer: the supervisor or a writer merges and checks; nobody else replies to the user.
  • Limits everywhere: rounds, handoffs, steps per worker, total cost per task.
  • Trace across agents: one trace ID for the whole task, so you can see which agent made which call.

Frameworks implement these patterns for you: LangGraph (supervisor and swarm libraries, subgraphs, Send), CrewAI (role-based crews), AutoGen / AG2 (conversational agents) and the OpenAI Agents SDK (handoffs).

Interview questions

Supervisor vs handoff (swarm) architectures: what's the difference?

In a supervisor design, a coordinator chooses which specialist works next and decides when the task is finished: control is centralised and easy to reason about. With handoffs, the active agent transfers control directly to another agent: decentralised, natural for conversations that move between departments. Both need caps (rounds or handoffs) to avoid loops.

When is a multi-agent system better than a single agent?

When a single agent's prompt or tool list has grown too large and evals show it choosing badly, when sub-tasks are independent and can run in parallel, or when distinct expertise or permissions must be separated. Otherwise a single agent with good tools is cheaper, faster and more reliable.

How do agents share information in a multi-agent system?

Preferably through structured shared state (a blackboard or graph state) that each agent reads from and writes to, with each agent seeing only what it needs. Free-text conversations between agents are flexible but drift, cost more tokens and are harder to debug.

Practice

  • Make the supervisor in 7.2 return an unknown worker name once and check the error path.
  • Give the triage agent in 7.3 a message about both a refund and a crash. What should happen?
  • Add a fourth reviewer to 7.4 and confirm the total time barely changes.

Next: 8. Evaluating agents Back to: AI Agents overview · Notes overview