Citation Grounding for Compliance-Heavy Industries using LangGraph

In standard Retrieval-Augmented Generation (RAG), the Large Language Model (LLM) acts as a synthesizer. But in compliance-heavy industries—like finance, healthcare, and legal synthesis is not enough. Traceability is mandatory.

If an AI tells a compliance officer that a transaction violates an Anti-Money Laundering (AML) policy, the officer cannot simply "trust the AI." They need to see the exact page, paragraph, and sentence from the regulatory document that proves it. This is where Citation Grounding comes in. Standard linear RAG chains (like basic LangChain) struggle here because they cannot easily "check their own work." To build a truly grounded system, we need a framework that supports loops, state management, and self-correction. In this end-to-end guide, we will build a Citation-Grounded RAG system using LangGraph, applied to a real-world use case: Automated KYC/AML (Know Your Customer / Anti-Money Laundering) Review in Retail Banking.

The Real-World Use Case: KYC/AML Document Review

The Scenario: A global bank is onboarding a corporate client. The client submits a complex 200-page packet detailing their Ultimate Beneficial Owners (UBOs), corporate structure, and source of funds.

The Task: The compliance team must verify if the UBO structure complies with the bank’s internal AML policy and external sanctions guidelines.

The Problem: Reading 200 pages of dense legal text against a 500-page internal policy manual takes hours.

The AI Solution: A RAG system that answers: "Is the UBO structure compliant?"

The Compliance Requirement: The AI must output the answer alongside exact, verbatim quotes from the policy manual. If it cannot find an exact quote, it must state "Insufficient information" rather than hallucinate a compliance rule.

Why LangGraph for Citation Grounding?

To guarantee citations, we cannot rely on a single LLM prompt. We must implement a Generate-Verify-Correct loop:

1. Retrieve: Fetch relevant policy chunks.

2. Generate: The LLM drafts an answer and extracts exact quotes as citations.

3. Verify (The Grounding Node): A deterministic Python script checks if the LLM's "exact quotes" actually exist in the retrieved text.

4. Route: If the citations are fake or missing, LangGraph routes the state back to the Generate node with an error prompt, forcing the LLM to self-correct.

Step 1: Data Ingestion & Metadata Strategy

Citation grounding fails if your chunks lack context. When chunking your compliance documents, you must preserve strict metadata.

# Example of a perfectly structured document chunk for grounding
document_chunk = {
    "page_content": "Section 4.2: Any corporate entity with a UBO holding more than 25% equity must submit a notarized proof of funds.",
    "metadata": {
        "source": "Global_AML_Policy_2026.pdf",
        "page_number": 42,
        "section_id": "SEC_4.2",
        "chunk_id": "doc_123_chunk_4"
    }
}

Step 2: Defining the LangGraph State

We need a state that tracks the documents, the generated answer, the citations, and a verification flag to control the loop.

from typing import TypedDict, List, Optional
from langchain_core.documents import Document
from pydantic import BaseModel, Field

# Pydantic model to force the LLM to output structured citations
class Citation(BaseModel):
    exact_quote: str = Field(description="The exact verbatim quote from the text")
    source_doc: str = Field(description="The document name")
    page_number: int = Field(description="The page number")

class GroundedAnswer(BaseModel):
    answer: str = Field(description="The final compliance answer")
    citations: List[Citation] = Field(description="List of exact quotes supporting the answer")
    is_compliant: Optional[bool] = Field(description="True if compliant, False if not, None if unknown")

class GraphState(TypedDict):
    question: str
    documents: List[Document]
    generated_response: Optional[GroundedAnswer]
    verification_status: str # "pending", "verified", "failed"
    retry_count: int
    error_feedback: Optional[str]

Step 3: Building the LangGraph Nodes

Now we build the nodes. The magic happens in the generate_node (which uses structured output) and the verify_citations_node (which acts as the deterministic ground-truth checker).

Node 1: Retrieve

def retrieve_node(state: GraphState):
    # In production, this queries your Vector DB (e.g., Pinecone, Milvus)
    # We mock it here for the example
    question = state["question"]
    
    # Mock retrieval
    docs = [
        Document(page_content="Section 4.2: Any corporate entity with a UBO holding more than 25% equity must submit a notarized proof of funds.", 
                 metadata={"source": "Global_AML_Policy_2026.pdf", "page_number": 42}),
        Document(page_content="Section 5.1: If proof of funds is not notarized, the onboarding must be escalated to the Senior Compliance Officer.", 
                 metadata={"source": "Global_AML_Policy_2026.pdf", "page_number": 55})
    ]
    return {"documents": docs, "verification_status": "pending"}

Node 2: Generate (with Structured Output)

We use LangChain's with_structured_output to force the LLM to return our Pydantic model. This prevents the LLM from writing flowing text where citations are hard to parse.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o", temperature=0)
structured_llm = llm.with_structured_output(GroundedAnswer)

def generate_node(state: GraphState):
    question = state["question"]
    docs = state["documents"]
    error_feedback = state.get("error_feedback", "")
    
    context = "\n\n".join([f"[Source: {d.metadata['source']}, Page: {d.metadata['page_number']}]\n{d.page_content}" for d in docs])
    
    prompt = f"""You are a strict banking compliance AI. 
    Answer the question using ONLY the provided context.
    
    CONTEXT:
    {context}
    
    QUESTION: {question}
    
    {f"PREVIOUS ATTEMPT FAILED: {error_feedback}. Please ensure your exact_quote matches the context VERBATIM." if error_feedback else ""}
    
    Rules:
    1. If the answer is not in the context, set answer to "Insufficient information in provided policy."
    2. Citations MUST be exact, verbatim substrings from the context. Do not paraphrase the exact_quote.
    """
    
    response = structured_llm.invoke(prompt)
    return {"generated_response": response, "verification_status": "pending"}

Node 3: Verify Citations (The Grounding Engine)

This is the most critical node. We do not ask the LLM to verify itself. We use deterministic Python string matching to ensure the exact_quote actually exists in the retrieved documents.

def verify_citations_node(state: GraphState):
    docs = state["documents"]
    response = state["generated_response"]
    retry_count = state.get("retry_count", 0)
    
    # Combine all document text for searching
    full_context = " ".join([doc.page_content for doc in docs])
    
    failed_quotes = []
    
    # Deterministic check: Does the exact quote exist in the context?
    for citation in response.citations:
        # We use a slight normalization (lowercase, strip) for robustness, 
        # but in strict compliance, you might require exact character matching.
        if citation.exact_quote.strip().lower() not in full_context.lower():
            failed_quotes.append(citation.exact_quote)
            
    if not failed_quotes:
        return {"verification_status": "verified", "retry_count": retry_count}
    else:
        feedback = f"The following quotes were not found verbatim in the context: {failed_quotes}. Regenerate with exact quotes."
        return {
            "verification_status": "failed", 
            "error_feedback": feedback,
            "retry_count": retry_count + 1
        }

Step 4: Assembling the LangGraph Workflow

Now we wire the nodes together. We add a conditional edge: if verification fails, we loop back to generate_node. If it passes (or we hit max retries), we end.

from langgraph.graph import StateGraph, END

def route_after_verification(state: GraphState):
    if state["verification_status"] == "verified":
        return "end"
    elif state["retry_count"] >= 2: # Prevent infinite loops
        return "end" 
    else:
        return "regenerate"

# Build the graph
workflow = StateGraph(GraphState)

# Add nodes
workflow.add_node("retrieve", retrieve_node)
workflow.add_node("generate", generate_node)
workflow.add_node("verify", verify_citations_node)

# Add edges
workflow.set_entry_point("retrieve")
workflow.add_edge("retrieve", "generate")
workflow.add_edge("generate", "verify")

# Add conditional edge for the self-correction loop
workflow.add_conditional_edges(
    "verify",
    route_after_verification,
    {
        "regenerate": "generate",
        "end": END
    }
)

# Compile the graph
grounded_rag_app = workflow.compile()

Step 5: Execution and Real-Time Output

Let's run the graph with a real compliance question.

# The Compliance Question
query = "The corporate applicant has a UBO with 30% equity, but their proof of funds is not notarized. What is the compliance action?"

# Invoke the graph
final_state = grounded_rag_app.invoke({
    "question": query, 
    "documents": [], 
    "generated_response": None, 
    "verification_status": "pending", 
    "retry_count": 0, 
    "error_feedback": None
})

# Print the Grounded Result
result = final_state["generated_response"]
print(f"Answer: {result.answer}")
print(f"Is Compliant: {result.is_compliant}")
print("\n--- VERIFIABLE CITATIONS ---")
for i, cite in enumerate(result.citations, 1):
    print(f"[{i}] \"{cite.exact_quote}\"")
    print(f"    -> Source: {cite.source_doc}, Page: {cite.page_number}\n")

Expected Output:

Answer: The onboarding must be escalated to the Senior Compliance Officer because the UBO holds more than 25% equity and the proof of funds is not notarized.
Is Compliant: False

--- VERIFIABLE CITATIONS ---
[1] "Any corporate entity with a UBO holding more than 25% equity must submit a notarized proof of funds."
    -> Source: Global_AML_Policy_2026.pdf, Page: 42

[2] "If proof of funds is not notarized, the onboarding must be escalated to the Senior Compliance Officer."
    -> Source: Global_AML_Policy_2026.pdf, Page: 55

Output

Conclusion

In compliance-heavy industries, an AI without citations is a liability. By combining Structured Output (Pydantic) with Deterministic Verification inside a LangGraph loop, you transition RAG from a "creative assistant" to a "verifiable compliance engine". This architecture ensures that every claim made by the AI is anchored to the ground truth, giving compliance officers the confidence to actually trust the system.