In the enterprise, AI isn't just answering simple FAQs; it's digesting massive, complex documents. A standard 100-page PDF—like a commercial lending manual or a regulatory compliance guide—can easily translate to 40,000+ tokens.

If you try to shove a 100-page document into a single LLM prompt, you will hit three massive walls:

  1. Context Limits: You exceed the model's maximum context window.

  2. The "Lost in the Middle" Phenomenon: LLMs notoriously forget or ignore information buried in the middle of long prompts.

  3. Cost & Latency: Processing 40k tokens per query is prohibitively expensive and slow.

To solve this, we cannot rely on simple "Retrieve-and-Read" RAG. We need a Multi-Agent Map-Reduce Architecture powered by LangGraph, combined with a Parent-Child Indexing strategy.

The Real-World Use Case: TechBank’s Commercial Lending Policy

Imagine you are building an AI underwriting assistant for TechBank. The bank has a 100-page "Commercial Real Estate (CRE) Lending Policy Manual".

The User Query:

"What are the specific Debt-Service Coverage Ratio (DSCR) requirements, and what environmental risk assessments are mandated for a $5M warehouse loan located in a designated flood zone?"

To answer this, the AI must find the DSCR rules (Page 14), the loan size thresholds (Page 45), and the environmental/flood zone policies (Page 88), and then synthesize them into a single, coherent underwriting checklist.

Technology Stack

ComponentTechnologyRole in Architecture
OrchestrationLangGraphManages the multi-agent state, memory, and Map-Reduce flow.
LLM ProviderAzure OpenAI (GPT-4o)Handles the complex reasoning and synthesis.
Document ParsingLlamaParse / UnstructuredExtracts text, tables, and headers from the 100-page PDF.
Vector Databasepgvector (PostgreSQL)Stores embeddings and handles the Parent-Child retrieval.
CachingRedisCaches frequent policy queries to save LLM costs.
ObservabilityLangSmithTraces the Map-Reduce steps for compliance auditing.
452

The Strategy: Parent-Child Indexing + Map-Reduce Agents

1. Ingestion: Parent-Child Chunking

We don't just chunk the PDF into 500-token pieces.

2. Execution: The Map-Reduce Multi-Agent Flow

Instead of passing 20 retrieved chunks to one LLM call (which causes context overflow), we use LangGraph to orchestrate three agents:

  1. The Retriever: Finds the top 15 relevant Child chunks and fetches their Parent context.

  2. The Mapper Agent: Loops through the retrieved chunks one by one (or in tiny batches). It extracts only the specific policy rules relevant to the query. This keeps the context window per call tiny (< 2k tokens).

  3. The Reducer Agent: Takes the extracted rules from the Mapper and synthesizes them into the final underwriting answer.

End-to-End Implementation

Let's build this resilient, context-aware system using LangGraph.

Step 1: Define the Enterprise State

Our state needs to track the original query, the raw retrieved chunks, the mapped extractions, and the conversation memory.

from typing import TypedDict, List, Annotated, Optionalimport operator
from langgraph.graph import StateGraph, END

class BankPolicyState(TypedDict):
    # Memory: Conversation history
    messages: Annotated[List[str], operator.add]
  
    # The user's complex underwriting query
    query: str
  
    # Raw data retrieved from the 100-page PDF (Parent-Child chunks)
    retrieved_contexts: List[str] 
  
    # The Map-Reduce state:
    # 1. Mapper extracts facts from each chunk individually
    mapped_extractions: Annotated[List[str], operator.add]
  
    # 2. Reducer synthesizes the final answer
    final_underwriting_decision: str
  
    # Tracking for loop prevention and observability
    current_phase: str
    chunks_processed: int

Step 2: Build the Multi-Agent Nodes

Node 1: The Retriever (Parent-Child Search)

This node simulates querying the vector database. In a real app, it searches the small "Child" embeddings but returns the large "Parent" text.

def retriever_agent(state: BankPolicyState) -> BankPolicyState:
    print("🔍 [Retriever] Searching 100-page CRE Policy Manual...")
  
    # Simulating Parent-Child retrieval. 
    # We found relevant sections on DSCR, Loan Limits, and Flood Zones.
    mock_parent_chunks = [
        "PAGE 14 CONTEXT: DSCR Requirements. For commercial loans > $1M, the global DSCR must be >= 1.25x. For speculative real estate, it must be >= 1.35x.",
        "PAGE 45 CONTEXT: Loan Sizing. Loans between $2M and $10M require approval from the Regional Credit Committee and must include a full environmental Phase I ESA.",
        "PAGE 88 CONTEXT: Environmental Risk. Properties in FEMA designated flood zones require specialized flood insurance coverage equal to 100% of the replacement cost, and the loan-to-value (LTV) cannot exceed 65%."
    ]
  
    return {
        "retrieved_contexts": mock_parent_chunks,
        "current_phase": "mapping",
        "messages": ["Retriever: Found 3 highly relevant policy sections."]
    }

Node 2: The Mapper Agent (Context Limit Bypass)

This is the secret sauce. Instead of passing all 3 chunks to the LLM at once, the Mapper processes them to extract only the facts needed for the specific query. (In production, this node would use LangGraph's Send API to process chunks in parallel).

def mapper_agent(state: BankPolicyState) -> BankPolicyState:
    print("🗺️ [Mapper] Extracting specific rules from retrieved chunks...")
  
    extractions = []
    # Simulating the LLM extracting facts from each chunk individually
    for chunk in state["retrieved_contexts"]:
        # Prompt to LLM: "Extract only the rules relevant to a $5M warehouse loan in a flood zone from this text: {chunk}"
        if "DSCR" in chunk:
            extractions.append("Rule 1: Global DSCR must be >= 1.25x (1.35x if speculative).")
        elif "Loan Sizing" in chunk:
            extractions.append("Rule 2: $5M loan requires Regional Credit Committee approval and Phase I ESA.")
        elif "Environmental" in chunk:
            extractions.append("Rule 3: Flood zone requires 100% replacement cost flood insurance and max 65% LTV.")
          
    return {
        "mapped_extractions": extractions,
        "chunks_processed": len(state["retrieved_contexts"]),
        "current_phase": "reducing",
        "messages": [f"Mapper: Extracted {len(extractions)} specific policy rules."]
    }

Node 3: The Reducer Agent (Synthesis)

Now that we have a clean, concise list of extracted rules (instead of 10,000 tokens of raw PDF text), the Reducer agent can easily synthesize the final answer without exceeding context limits.

def reducer_agent(state: BankPolicyState) -> BankPolicyState:
    print("📝 [Reducer] Synthesizing final underwriting checklist...")
  
    # The context here is tiny! Just the extracted rules.
    rules_text = "\n".join(state["mapped_extractions"])
  
    # Simulating LLM synthesis
    final_decision = f"""
    UNDERWRITING CHECKLIST FOR $5M WAREHOUSE LOAN (FLOOD ZONE):
    ---------------------------------------------------------
    Based on the CRE Policy Manual, the following conditions apply:
    1. {state['mapped_extractions'][0]}
    2. {state['mapped_extractions'][1]}
    3. {state['mapped_extractions'][2]}
  
    ACTION REQUIRED: Request borrower's DSCR calculations, order Phase I ESA, and verify flood insurance certificates before presenting to Regional Credit Committee.
    """
  
    return {
        "final_underwriting_decision": final_decision,
        "current_phase": "complete",
        "messages": ["Reducer: Final underwriting checklist generated."]
    }

Step 3: Compile the LangGraph

We wire the agents together in a sequential flow.

def build_bank_policy_graph():
    workflow = StateGraph(BankPolicyState)

    workflow.add_node("retriever", retriever_agent)
    workflow.add_node("mapper", mapper_agent)
    workflow.add_node("reducer", reducer_agent)

    workflow.set_entry_point("retriever")
  
    # Sequential flow: Retrieve -> Map -> Reduce
    workflow.add_edge("retriever", "mapper")
    workflow.add_edge("mapper", "reducer")
    workflow.add_edge("reducer", END)

    return workflow.compile()

app = build_bank_policy_graph()

Running the System

Let's process the complex query that would normally break a standard RAG system.

initial_state = {
    "messages": [],
    "query": "What are the DSCR requirements and environmental assessments for a $5M warehouse loan in a flood zone?",
    "retrieved_contexts": [],
    "mapped_extractions": [],
    "final_underwriting_decision": "",
    "current_phase": "start",
    "chunks_processed": 0
}

result = app.invoke(initial_state)

print("\n--- Agent Memory Trace ---")
for msg in result["messages"]:
    print(f"• {msg}")

print("\n--- Final Underwriting Decision ---")
print(result["final_underwriting_decision"])

Output Trace:

🔍 [Retriever] Searching 100-page CRE Policy Manual...
🗺️ [Mapper] Extracting specific rules from retrieved chunks...
📝 [Reducer] Synthesizing final underwriting checklist...

--- Agent Memory Trace ---
• Retriever: Found 3 highly relevant policy sections.
• Mapper: Extracted 3 specific policy rules.
• Reducer: Final underwriting checklist generated.

--- Final Underwriting Decision ---
UNDERWRITING CHECKLIST FOR $5M WAREHOUSE LOAN (FLOOD ZONE):
---------------------------------------------------------
Based on the CRE Policy Manual, the following conditions apply:
1. Rule 1: Global DSCR must be >= 1.25x (1.35x if speculative).
2. Rule 2: $5M loan requires Regional Credit Committee approval and Phase I ESA.
3. Rule 3: Flood zone requires 100% replacement cost flood insurance and max 65% LTV.
...

Enterprise Best Practices for Large Document RAG

  1. Parallelize the Mapper: In the code above, the Mapper processes chunks sequentially for simplicity. In production, use LangGraph’s Send API to map over the retrieved chunks in parallel. This reduces latency from seconds to milliseconds.

  2. Implement a "Router" Agent: Not every query requires a Map-Reduce. Add a Supervisor node at the beginning. If the user asks, "What is the bank's routing number?", route it to a simple, single-shot RAG node. Only trigger the heavy Map-Reduce graph for complex, multi-page synthesis.

  3. Use Semantic Caching: A 100-page policy document doesn't change daily. Use Redis to cache the mapped_extractions for specific document chunks. If another loan officer asks a similar question, bypass the Mapper and go straight to the Reducer.

  4. Citation Tracking: In the Mapper node, ensure the LLM returns the page_number or chunk_id alongside the extracted rule. Pass this metadata to the Reducer so the final output includes clickable citations (e.g., [Source: Page 88]), which is mandatory for banking compliance.

Conclusion

Handling a 100-page PDF in an enterprise environment isn't about finding an LLM with a massive context window; it's about architecting the workflow to respect context limits. By combining Parent-Child Indexing for precise retrieval with a LangGraph Multi-Agent Map-Reduce pattern for processing, we bypass the "Lost in the Middle" problem. We keep the context window small, the token costs low, and the reasoning sharp. For TechBank, this means their underwriters get accurate, synthesized, and fully cited policy guidance in seconds, turning a 100-page manual into a highly actionable AI assistant.