Skip to content

5. Parent–child retrieval & GraphRAG

Advanced · 14 min read

Flat RAG treats a corpus as a bag of independent chunks. That breaks in two ways. A chunk small enough to match precisely is often too small to understand. And some questions can't be answered from any single chunk, because the answer is spread across documents connected by relationships. Parent–child retrieval fixes the first problem. Knowledge graphs and GraphRAG fix the second.

5.1 Parent–child (small-to-big) retrieval

Chunking forces a trade-off: small chunks match precisely, large chunks give context. Parent–child retrieval refuses to choose. It indexes small child chunks for matching, and each child points to its parent (a section or a large chunk). You search over children and send their parents to the LLM.

flowchart LR
    subgraph Parent [Parent: section 'Express shipping' — sent to the LLM]
        C1[child: Express shipping takes 1 day.]
        C2[child: It costs 9 dollars.]
        C3[child: Order before 2 pm for same-day dispatch.]
    end
    Q[Question] -->|matches| C2
    C2 -->|parent_id| Parent
import math
import re
from collections import Counter

STOP = {"a", "an", "the", "is", "are", "to", "of", "and", "in", "it", "for", "on", "by", "do", "does", "how",
        "what", "much", "who", "which", "i", "be", "before", "can"}

def embed(text: str) -> Counter:
    words = re.findall(r"[a-z0-9]+", text.lower())
    return Counter(w.rstrip("s") if len(w) > 3 else w for w in words if w not in STOP)

def cosine(a: Counter, b: Counter) -> float:
    dot = sum(a[t] * b[t] for t in a)
    norm = math.sqrt(sum(v * v for v in a.values())) * math.sqrt(sum(v * v for v in b.values()))
    return dot / norm if norm else 0.0

PARENTS = {   # parent_id -> full section text (stored in a doc store or as metadata, NOT embedded)
    "shipping#express": "Express shipping takes 1 day. It costs 9 dollars. Order before 2 pm for same-day dispatch.",
    "shipping#standard": "Standard shipping takes 3 to 5 days. It is free on orders over 50 dollars.",
    "returns#how": "Start a return from your orders page within 30 days. Print the label and drop the parcel off.",
}
CHILDREN = [   # small chunks that ARE embedded, each pointing to its parent
    {"id": f"{pid}/{i}", "parent_id": pid, "text": s, "vector": embed(s)}
    for pid, text in PARENTS.items()
    for i, s in enumerate(re.split(r"(?<=\.)\s+", text))
]

def parent_child_search(question: str, k_children: int = 3, k_parents: int = 2) -> list[str]:
    q = embed(question)
    hits = sorted(CHILDREN, key=lambda c: -cosine(q, c["vector"]))[:k_children]
    parent_ids = []
    for hit in hits:                                   # dedupe: several children can share a parent
        print(f"  child {hit['id']:22} {cosine(q, hit['vector']):.2f}  {hit['text']}")
        if hit["parent_id"] not in parent_ids:
            parent_ids.append(hit["parent_id"])
    return parent_ids[:k_parents]

question = "How much does express shipping cost, and when must I order?"
for pid in parent_child_search(question):
    print(f"→ send parent {pid}: {PARENTS[pid]}")
Output
  child shipping#express/0     0.37  Express shipping takes 1 day.
  child shipping#express/1     0.24  It costs 9 dollars.
  child shipping#standard/1    0.18  It is free on orders over 50 dollars.
→ send parent shipping#express: Express shipping takes 1 day. It costs 9 dollars. Order before 2 pm for same-day dispatch.
→ send parent shipping#standard: Standard shipping takes 3 to 5 days. It is free on orders over 50 dollars.

The child "It costs 9 dollars." alone would leave the LLM guessing what costs 9 dollars. Its parent supplies the context, plus the 2 pm cut-off the user also asked about. That sentence wasn't in the top children at all, but it arrives with its parent.

Practical notes:

  • Typical sizes are children of 100–300 tokens and parents of 1,000–2,000 tokens, or whole sections.
  • Store parents in a document store or key–value store keyed by parent_id. Only children go in the vector index.
  • Sentence-window retrieval is the lightweight variant. Index single sentences and, at query time, return the matched sentence plus n sentences either side, with no separate parent store.
  • Parents use more prompt tokens. Send fewer of them (2–4), and make sure two hits from the same parent count once.

5.2 Where vector search can't help

Two kinds of question defeat any chunk-based retriever, however good the reranker:

Type Example Why chunks fail
Multi-hop "Who manages the team that owns the payments service?" The answer needs two facts in different documents, and the second doesn't mention "payments" at all
Global / aggregate "What are the main themes across all customer complaints this quarter?" The answer is a summary of the whole corpus. Top-5 chunks see a sliver of it 
DOCS = {
    "services.md#1": "The payments service is owned by the Checkout team.",
    "teams.md#4": "The Checkout team is managed by Asha Rao.",
    "teams.md#7": "The Search team is managed by Vikram Shah.",
    "services.md#2": "The payments service processes card and UPI transactions.",
}
q = embed("Who manages the team that owns the payments service?")
for doc_id, text in sorted(DOCS.items(), key=lambda d: -cosine(q, embed(d[1]))):
    print(f"{cosine(q, embed(text)):.2f}  {doc_id:14} {text}")
Output
0.55  services.md#1  The payments service is owned by the Checkout team.
0.33  services.md#2  The payments service processes card and UPI transactions.
0.18  teams.md#4     The Checkout team is managed by Asha Rao.
0.18  teams.md#7     The Search team is managed by Vikram Shah.

The first hop ranks first, but the chunk with the actual answer (teams.md#4) ties with the Search team's manager. Nothing in the question says "Checkout", so search can't know which manager is relevant. Only the first chunk's content links them.

5.3 Knowledge graphs: follow the relationships

A knowledge graph stores facts as (subject, relation, object) triples. In GraphRAG pipelines, an LLM extracts them from every chunk at indexing time, with structured output. A multi-hop question then becomes a walk along the edges:

TRIPLES = [   # extracted by an LLM from the chunks above, each with the chunk it came from
    ("payments service", "owned_by", "Checkout team", "services.md#1"),
    ("Checkout team", "managed_by", "Asha Rao", "teams.md#4"),
    ("Search team", "managed_by", "Vikram Shah", "teams.md#7"),
    ("payments service", "processes", "UPI transactions", "services.md#2"),
]

graph: dict[str, list[tuple[str, str, str]]] = {}
for subj, rel, obj, source in TRIPLES:
    graph.setdefault(subj, []).append((rel, obj, source))

def follow(start: str, relations: list[str]) -> list[tuple[str, str, str, str]]:
    """Walk a fixed path of relations from a start entity; return each hop with its source."""
    path, node = [], start
    for rel in relations:
        step = next(((r, o, s) for r, o, s in graph.get(node, []) if r == rel), None)
        if step is None:
            break
        path.append((node, step[0], step[1], step[2]))
        node = step[1]
    return path

for subj, rel, obj, source in follow("payments service", ["owned_by", "managed_by"]):
    print(f"{subj} --{rel}--> {obj}   [{source}]")
Output
payments service --owned_by--> Checkout team   [services.md#1]
Checkout team --managed_by--> Asha Rao   [teams.md#4]

Each hop keeps its source chunk, so the final answer can still cite both documents. In a real system an LLM turns the question into the start entity and relation path (or into a Cypher query for a graph database like Neo4j), and the walk's facts and source chunks go into the prompt.

5.4 GraphRAG: communities for global questions

Microsoft's GraphRAG (2024) uses the graph to answer the global questions that plain RAG can't. At indexing time:

  1. Extract entities and relationships from every chunk with an LLM.
  2. Cluster the entity graph into communities, groups of densely connected entities, using the Leiden algorithm, at several levels of granularity.
  3. Summarise each community with an LLM ("the Checkout team, the payments service and their incidents...").

At query time there are two modes:

Mode For How it works
Local search Questions about specific entities ("What does the Checkout team own?") Find the matching entities, then gather their neighbours, relationships, community summaries and source chunks
Global search Corpus-wide questions ("What are the main themes?") Map: ask the question of every community summary. Reduce: combine the partial answers into one

Here is the clustering idea in miniature. Connected groups of entities form communities. Real GraphRAG uses Leiden, which also splits large connected groups into tighter ones.

EDGES = [("payments service", "Checkout team"), ("Checkout team", "Asha Rao"), ("payments service", "UPI"),
         ("Search team", "Vikram Shah"), ("Search team", "search API"), ("search API", "Elasticsearch")]

parent = {}
def find(x):
    parent.setdefault(x, x)
    while parent[x] != x:
        parent[x] = parent[parent[x]]
        x = parent[x]
    return x

for a, b in EDGES:
    parent[find(a)] = find(b)                 # union: a and b are in the same community

communities: dict[str, list[str]] = {}
for entity in sorted(parent):
    communities.setdefault(find(entity), []).append(entity)
for i, members in enumerate(communities.values(), start=1):
    print(f"community {i}: {members}")        # each one gets an LLM-written summary at indexing time
Output
community 1: ['Asha Rao', 'Checkout team', 'UPI', 'payments service']
community 2: ['Elasticsearch', 'Search team', 'Vikram Shah', 'search API']

5.5 When is a graph worth it?

Plain / hybrid RAG Parent–child Knowledge graph / GraphRAG
Fixes Most fact lookups Chunks too small to understand Multi-hop and global questions
Indexing cost Embeddings only Embeddings only An LLM call per chunk, plus community summaries (often 10–100× more)
Query latency Low Low Local: medium. Global: high (map over many summaries)
Updating content Easy: re-embed changed chunks Easy Harder: re-extract, re-cluster, re-summarise
Use when Default Answers need surrounding context Relationship-heavy data (org charts, systems, contracts, research) or "summarise everything" questions

Start with hybrid search, reranking and parent–child. Add a graph when your eval set shows multi-hop or global questions failing, not because GraphRAG is fashionable. A lighter middle ground is to extract entities as metadata and filter or boost by them, with no full graph.

Interview questions

What is parent–child retrieval, and why use it?

Index small child chunks for precise matching, each linked to a larger parent (a section). Search over children, map the hits to their parents, deduplicate, and send the parents to the LLM. It separates the chunk size that's best for retrieval from the size that's best for generation. The cost is more prompt tokens per result. Sentence-window retrieval is a lighter variant.

What questions does plain vector RAG fail on, and how does GraphRAG help?

Multi-hop questions, where facts in different documents must be connected, and global questions that summarise the whole corpus. GraphRAG extracts entities and relationships with an LLM, clusters them into communities with Leiden, and writes community summaries. Local search walks entities and their neighbourhoods. Global search maps the question over community summaries and reduces the partial answers.

What are the downsides of GraphRAG?

Indexing is expensive: an LLM call per chunk plus community summaries. Extraction errors become wrong edges. Updates require re-extraction and re-clustering. Global search is slow and costly. It's worth it for relationship-heavy data or corpus-level questions, and should be justified by evals.

Practice

  • Change k_parents to 1 in parent_child_search and check what context is lost.
  • Add the triple ("Asha Rao", "reports_to", "CTO", "org.md#1") and walk three hops.
  • Add an edge joining the two communities in 5.4 and see them merge. Why does Leiden exist?

Next: 6. Citations Back to: RAG overview · Notes overview