Skip to content

4. When plain SDK code is better

Intermediate · 10 min read

The RAG pipeline took about 60 lines of plain Python. A tool-calling agent, below, takes about 40. Model providers' own SDKs now handle streaming, structured output, tool calling and retries. So "Should we use LangChain?" is a real engineering decision, not a default. Interviewers ask it to see whether you understand what a framework actually does for you.

4.1 What a framework costs, and what it buys

You get You pay
Integrations: loaders, vector stores, many providers behind one interface Extra dependencies, and version upgrades that break code
Ready-made patterns: agents, retrievers, advanced RAG An abstraction between you and the actual prompt and API request
Persistence, interrupts and streaming (LangGraph) Framework concepts the whole team must learn
Tracing integrations (LangSmith and others) Harder debugging: deep stack traces, hidden defaults
Faster prototypes Provider features arrive in the framework later than in the provider's SDK

4.2 A tool-calling agent in plain Python

Here's the agent loop that create_agent gives you, written directly. A fake LLM stands in for the API call, so it runs offline. The message format is simplified, but the control flow is exactly what runs in production.

import json

def get_order_status(order_id: str) -> str:
    return {"A100": "Shipped, arriving Friday"}.get(order_id, "Order not found")

def search_policies(query: str) -> str:
    return "Refunds are processed within 5 working days of approval."

TOOLS = {"get_order_status": get_order_status, "search_policies": search_policies}

def fake_llm(messages: list[dict]) -> dict:
    """Stands in for client.chat.completions.create(..., tools=SCHEMAS): asks for tools first, then answers."""
    results = [m["content"] for m in messages if m["role"] == "tool"]
    if not results:
        return {"role": "assistant", "content": None, "tool_calls": [
            {"id": "call_1", "name": "get_order_status", "arguments": json.dumps({"order_id": "A100"})},
            {"id": "call_2", "name": "search_policies", "arguments": json.dumps({"query": "refund time"})},
        ]}
    return {"role": "assistant", "content": "Order A100: " + results[0] + ". " + results[1], "tool_calls": []}

def run_agent(question: str, llm=fake_llm, max_turns: int = 5) -> str:
    messages = [{"role": "system", "content": "You are a support assistant. Use tools; never guess."},
                {"role": "user", "content": question}]
    for turn in range(max_turns):
        reply = llm(messages)
        messages.append(reply)
        if not reply["tool_calls"]:
            return reply["content"]                                   # no tool calls: final answer
        for call in reply["tool_calls"]:
            args = json.loads(call["arguments"])
            try:
                result = TOOLS[call["name"]](**args) if call["name"] in TOOLS else f"Unknown tool {call['name']}"
            except Exception as err:                                  # tool errors go back to the model, not the user
                result = f"Tool error: {err}"
            print(f"turn {turn}: {call['name']}({args}) -> {result}")
            messages.append({"role": "tool", "tool_call_id": call["id"], "content": result})
    return "Sorry, I couldn't finish that. Let me connect you to a person."

print(run_agent("Where is order A100, and how long do refunds take?"))
Output
turn 0: get_order_status({'order_id': 'A100'}) -> Shipped, arriving Friday
turn 0: search_policies({'query': 'refund time'}) -> Refunds are processed within 5 working days of approval.
Order A100: Shipped, arriving Friday. Refunds are processed within 5 working days of approval.

Everything is visible: the exact messages, the turn limit, how tool errors are handled, and what happens when the agent gives up. When it misbehaves, you set a breakpoint in your loop, not in a framework's internals. See the tool loop for the real message protocol.

4.3 Switching providers without a framework

"We might change models" is the most common reason given for adopting a framework. A thin adapter of your own covers it. Here the same conversation goes to two providers whose APIs differ: one takes the system prompt as a message, the other as a separate parameter.

class OpenAIStyle:
    def payload(self, model: str, messages: list[dict]) -> dict:
        return {"model": model, "messages": messages}

class AnthropicStyle:
    def payload(self, model: str, messages: list[dict]) -> dict:
        system = "\n".join(m["content"] for m in messages if m["role"] == "system")
        return {"model": model, "system": system, "max_tokens": 1024,
                "messages": [m for m in messages if m["role"] != "system"]}

PROVIDERS = {"openai": OpenAIStyle(), "anthropic": AnthropicStyle()}

conversation = [{"role": "system", "content": "Be concise."}, {"role": "user", "content": "What is RAG?"}]
for name, adapter in PROVIDERS.items():
    print(f"{name:9}", json.dumps(adapter.payload("model-name", conversation)))
Output
openai    {"model": "model-name", "messages": [{"role": "system", "content": "Be concise."}, {"role": "user", "content": "What is RAG?"}]}
anthropic {"model": "model-name", "system": "Be concise.", "max_tokens": 1024, "messages": [{"role": "user", "content": "What is RAG?"}]}

A production version adds a complete() method per adapter that calls the SDK and maps the response back to one shape, plus streaming. That's roughly 100–200 lines you own, test and understand. Alternatively, a dedicated gateway such as LiteLLM, or an OpenAI-compatible endpoint, gives you multi-provider access without adopting a full framework.

4.4 Decision table

Situation Lean towards
One provider, a clear flow (RAG, extraction, classification) Plain SDK
Latency- or cost-critical hot path, where every token and call must be explicit Plain SDK
A small team that needs to debug production fast Plain SDK, or a minimal framework surface
Many document sources, connectors or vector stores to integrate quickly LangChain / LlamaIndex integrations
RAG-heavy app needing advanced retrieval (auto-merging, sentence windows, rerankers) LlamaIndex
Agents with loops, branches, long-running state, human approval, resumable runs LangGraph
A quick prototype or hackathon to test an idea Framework, then reassess
A team already standardised on a framework, with tracing and evals built around it Stay consistent

The question isn't really "framework or not". It's which parts should be yours. Control flow, prompts and business rules decide product quality, and you need to see them. Connectors and persistence plumbing are commodities, so let a library carry them.

4.5 The hybrid most teams settle on

flowchart LR
    subgraph Yours [Your code — explicit, tested, traced]
        P[Prompts & templates] --- F[Control flow / agent loop] --- B[Business rules, permissions, citations]
    end
    subgraph Library [Libraries — commodity plumbing]
        L[Loaders & parsers] --- V[Vector store clients] --- S[Provider SDKs / gateway]
        G[LangGraph: only if you need<br/>checkpoints, interrupts, resumable runs]
    end
    Yours --> Library
  • Own the core: prompts, the agent or RAG control flow, permissions and citations live in plain code you can read.
  • Borrow the edges: use library loaders, splitters and vector-store clients where they save real time.
  • Adopt LangGraph for state, not for style: when you actually need persistence, interrupts or complex branching, it's cheaper than building those yourself.
  • Hide it behind your interfaces: wrap any framework object in your own function or class, so swapping it later is a local change.
  • Pin versions and trace everything, framework or not. You must be able to see the exact prompt sent to the model.

4.6 Moving between the two

  • Framework → plain: prototype with a framework, then replace the hot path with SDK code once the flow is stable and you know which parts matter. Keep the framework's integrations if they're still useful.
  • Plain → framework: start plain. When you find yourself writing checkpointing, pause/resume or a graph executor, that's the signal to adopt LangGraph. Your node functions move over almost unchanged (see 3.3).

4.7 How to answer this in an interview

A good answer is balanced and specific, not "frameworks bad" or "always LangChain":

"It depends on what the framework would own. For a single-provider RAG service, I'd write the core in plain SDK code, because it's about a hundred lines and I want full visibility into prompts, latency and failures. I'd still use library loaders and vector-store clients. If the product needed a long-running agent with human approval and resumable state, I'd use LangGraph, because building checkpoints and interrupts myself isn't a good use of time. Either way I'd hide it behind our own interfaces, pin versions and trace every call."

Interview questions

When would you not use LangChain?

When the flow is simple and stable (single-provider RAG, extraction, classification), when latency and cost need every call to be explicit, or when the team needs to debug production quickly without framework internals in the way. Modern provider SDKs already handle tool calling, structured output and streaming. I'd still consider library integrations, such as loaders and vector-store clients, where they save time.

What does a framework actually give you that SDK code doesn't?

Integrations behind common interfaces. Ready-made patterns (agents, retrievers, advanced RAG). In LangGraph's case, persistence, interrupts, streaming and resumable execution. Also tracing hooks. The cost is abstraction, dependency churn, hidden prompts and defaults, and harder debugging.

How do you avoid lock-in if you do adopt a framework?

Keep prompts, business rules and control-flow decisions in your own code. Wrap framework objects behind your own interfaces (a retriever function, an LLM client class). Pin versions, and keep eval sets that are framework-agnostic, so you can verify a migration. Tracing should capture the actual prompts, independent of the framework.

Practice

  • Add a max_tool_calls budget to run_agent that stops after 3 tool calls in total.
  • Add a third adapter to PROVIDERS for a provider that puts messages under a different key, and print its payload.
  • For a project you've built, list which parts you'd keep in plain code and which you'd take from a library.

Next: AI Agents: ReAct from scratch, planning, memory, multi-agent, evaluation and safety. Back to: Frameworks overview · Notes overview