Why "Looks Good" Is Not a Success Criterion
In digital banking loan management and customer support, multi-step reasoning chains answer questions like "Can I refinance my auto loan at a lower rate given my recent credit score improvement?" or "Why was my hardship deferment application denied last week?" These require sequential retrieval (loan terms → payment history → credit profile → policy rules), intermediate computations (DTI recalculation, eligibility scoring), and regulatory validation.
Most teams define success implicitly: the agent stops when it generates a response. This is catastrophic in lending. A response that sounds confident but cites an outdated interest rate, miscalculates DTI by omitting a co-borrower’s debt, or misses a mandatory adverse action disclosure isn’t a success it’s a fair lending violation. Convergence must be explicitly defined as a structured, multi-dimensional predicate evaluated against live state. The graph doesn’t terminate when generation completes; it terminates when all success criteria are satisfied or when bounded exhaustion triggers escalation. This article demonstrates implementing formal convergence definitions for loan management RAG, with code-level enforcement in a stateful LangGraph multi-agent system.
Real-Time Use Case: NovaBank Loan Management & Support Assistant
The Workflow
NovaBank’s AI assistant handles 150K+ monthly interactions across:
Loan Management: Refinancing eligibility, payment schedule changes, hardship programs, payoff quotes, escrow analysis, rate lock inquiries, cosigner release requests.
Customer Support: Application status, document requests, denial explanations, fee disputes, regulatory disclosures, complaint acknowledgment.
Each query triggers a multi-step chain: retrieve loan contract → fetch live account state → pull credit bureau data → apply policy rules → validate regulatory compliance → generate response. Chains range from 3 steps (simple balance inquiry) to 8+ steps (refinance eligibility with DTI recalculation).
Why Implicit Termination Fails in Lending
| Failure Mode | Implicit Termination Behavior | Explicit Convergence Fix |
|---|---|---|
| Stale rate cited | Agent generates response with cached rate; chain ends | Freshness gate blocks termination until live rate confirmed |
| DTI miscalculation | Agent omits co-borrower debt; response generated | Computational verification node validates DTI before allowing completion |
| Missing adverse action notice | Agent explains denial without FCRA-mandated language | Regulatory checklist gate requires explicit disclosure confirmation |
| Circular policy lookup | Agent re-fetches same policy doc 6 times | State-change delta check detects zero-progress loops |
| Partial eligibility answer | Agent says "you may qualify" without specific conditions | Completeness predicate requires enumerated conditions + next steps |
| Unverified computation | Agent calculates monthly savings without showing formula | Audit trail gate requires computation trace in state |

The Convergence Framework: Five Success Dimensions
Convergence is not a boolean. It is a structured vector of five dimensions, each with typed pass/fail criteria evaluated against graph state. All five must pass for CONVERGED; any failure triggers targeted repair or bounded escalation.
| Dimension | Definition | State Fields Checked | Failure Action |
|---|---|---|---|
| Factual Grounding | Every numeric/contractual claim traces to verified state | verified_facts, source_citations | Re-retrieve + regenerate |
| Computational Integrity | All derived values have auditable computation traces | computation_trace, intermediate_values | Recompute with validation |
| Regulatory Completeness | All mandatory disclosures present and version-correct | disclosure_checklist, policy_version | Inject missing disclosures |
| State Freshness | All live data within TTL for decision-sensitive fields | data_timestamps, freshness_thresholds | Re-fetch stale sources |
| Actionable Completeness | Response includes specific next steps, not vague guidance | action_items, eligibility_conditions | Regenerate with specificity |
Step 1: State Schema with Convergence-First Design
Every convergence dimension maps to typed state fields. The schema makes success criteria machine-evaluable.
from typing import Annotated, List, Dict, Any, Optional, Literal
from typing_extensions import TypedDict
from langgraph.graph.message import add_messages
from datetime import datetime
import operator
class VerifiedFact(TypedDict):
"""A factual claim traced to a specific source in state."""
claim: str
value: Any
source_field: str # e.g., "loan_contract.interest_rate", "credit_report.score"
source_timestamp: datetime
verified: bool
class ComputationStep(TypedDict):
"""One step in a derived-value calculation with full audit trail."""
operation: str # e.g., "DTI = total_debt / gross_income"
inputs: Dict[str, Any]
output: Any
formula_reference: str # Policy section or formula ID
timestamp: datetime
class DisclosureCheck(TypedDict):
"""Status of a mandatory regulatory disclosure."""
disclosure_id: str # e.g., "FCRA_adverse_action", "TILA_rate_change"
required: bool
present_in_response: bool
version_match: bool
policy_section: str
class ConvergenceResult(TypedDict):
"""Structured evaluation of all five convergence dimensions."""
factual_grounding: bool
computational_integrity: bool
regulatory_completeness: bool
state_freshness: bool
actionable_completeness: bool
overall_converged: bool
failed_dimensions: List[str]
repair_actions: List[Dict[str, Any]]
eval_timestamp: datetime
class LoanAssistantState(TypedDict):
# === CONVERSATION STATE ===
messages: Annotated[list, add_messages]
customer_id: str
session_id: str
current_module: Literal["loan_management", "customer_support"]
# === RETRIEVED CONTEXT ===
loan_contract: Optional[Dict[str, Any]]
account_state: Optional[Dict[str, Any]]
credit_profile: Optional[Dict[str, Any]]
policy_rules: Optional[Dict[str, Any]]
# === CONVERGENCE STATE (typed, machine-evaluable) ===
verified_facts: Annotated[List[VerifiedFact], operator.add]
computation_trace: Annotated[List[ComputationStep], operator.add]
disclosure_checklist: List[DisclosureCheck]
data_timestamps: Dict[str, datetime]
freshness_thresholds: Dict[str, int] # seconds
# === GENERATION OUTPUT ===
generated_response: Optional[str]
action_items: List[str]
eligibility_conditions: List[str]
# === CONVERGENCE EVALUATION ===
convergence_result: Optional[ConvergenceResult]
convergence_cycle: int
max_convergence_cycles: int
# === REPAIR STATE ===
repair_history: Annotated[List[Dict[str, Any]], operator.add]
pending_repairs: List[Dict[str, Any]]
# === FINAL OUTPUT ===
final_response: Optional[str]
resolution_status: Literal["converged", "repaired", "escalated", "exhausted"]
# === AUDIT ===
audit_trail: Annotated[List[Dict[str, Any]], operator.add]
processing_start_time: datetime🔑 Key Insight:
verified_facts,computation_trace, anddisclosure_checklistare not documentation—they are convergence predicates. The convergence evaluator reads these fields and returns structured pass/fail per dimension. If they’re empty or incomplete, convergence fails by definition.
Step 2: Fact Verification Node — Grounding Dimension
async def fact_verification_node(state: LoanAssistantState) -> dict:
"""
Extracts every factual claim from generated response and verifies
against retrieved state. Populates verified_facts for convergence eval.
"""
response = state.get("generated_response", "")
loan = state.get("loan_contract", {})
account = state.get("account_state", {})
credit = state.get("credit_profile", {})
verified_facts = []
# Extract and verify interest rate claims
import re
rate_matches = re.findall(r'(\d+\.\d+)%\s*(?:APR|interest rate|rate)', response, re.IGNORECASE)
for rate_str in rate_matches:
claimed_rate = float(rate_str)
actual_rate = loan.get("current_apr")
verified_facts.append(VerifiedFact(
claim=f"Interest rate {rate_str}%",
value=claimed_rate,
source_field="loan_contract.current_apr",
source_timestamp=state.get("data_timestamps", {}).get("loan_contract", datetime.min),
verified=abs(claimed_rate - actual_rate) < 0.001 if actual_rate else False
))
# Verify balance/payment claims
balance_matches = re.findall(r'\$[\d,]+\.?\d*', response)
actual_balance = account.get("outstanding_principal")
for amount_str in balance_matches:
claimed = float(amount_str.replace("$", "").replace(",", ""))
if actual_balance and abs(claimed - actual_balance) < 0.01:
verified_facts.append(VerifiedFact(
claim=f"Balance {amount_str}",
value=claimed,
source_field="account_state.outstanding_principal",
source_timestamp=state.get("data_timestamps", {}).get("account_state", datetime.min),
verified=True
))
# Verify credit score claims
score_matches = re.findall(r'(?:credit\s+score|FICO)\s*(?:of\s*)?(\d{3})', response, re.IGNORECASE)
actual_score = credit.get("fico_score")
for score_str in score_matches:
claimed_score = int(score_str)
verified_facts.append(VerifiedFact(
claim=f"Credit score {score_str}",
value=claimed_score,
source_field="credit_profile.fico_score",
source_timestamp=state.get("data_timestamps", {}).get("credit_profile", datetime.min),
verified=claimed_score == actual_score if actual_score else False
))
return {
"verified_facts": verified_facts,
"audit_trail": [{
"node": "fact_verification",
"facts_checked": len(verified_facts),
"unverified_count": sum(1 for f in verified_facts if not f["verified"]),
"timestamp": datetime.utcnow().isoformat()
}]
}Step 3: Computation Validation Node — Integrity Dimension
async def computation_validation_node(state: LoanAssistantState) -> dict:
"""
Validates all derived computations (DTI, savings, eligibility scores)
have complete audit trails. Recomputes if trace is missing or inconsistent.
"""
response = state.get("generated_response", "")
existing_trace = state.get("computation_trace", [])
account = state.get("account_state", {})
credit = state.get("credit_profile", {})
new_steps = []
# DTI computation validation
if "dti" in response.lower() or "debt-to-income" in response.lower():
# Extract claimed DTI
dti_match = re.search(r'(?:DTI|debt.to.income)\s*(?:of\s*)?(\d+\.?\d*)%?', response, re.IGNORECASE)
if dti_match:
claimed_dti = float(dti_match.group(1))
# Recompute from source data
total_monthly_debt = account.get("total_monthly_obligations", 0)
gross_monthly_income = credit.get("verified_monthly_income", 0)
if gross_monthly_income > 0:
computed_dti = (total_monthly_debt / gross_monthly_income) * 100
new_steps.append(ComputationStep(
operation="DTI = total_monthly_debt / gross_monthly_income * 100",
inputs={
"total_monthly_debt": total_monthly_debt,
"gross_monthly_income": gross_monthly_income
},
output=round(computed_dti, 2),
formula_reference="POLICY-LOAN-003 §4.2",
timestamp=datetime.utcnow()
))
# Flag mismatch for convergence evaluator
if abs(claimed_dti - computed_dti) > 1.0:
new_steps[-1]["output"] = f"MISMATCH: claimed={claimed_dti}, computed={computed_dti}"
# Monthly savings computation for refinance scenarios
if "sav" in response.lower() and ("refinance" in response.lower() or "lower rate" in response.lower()):
current_payment = account.get("current_monthly_payment", 0)
new_rate = state.get("loan_contract", {}).get("refinance_offer_rate")
remaining_term = account.get("remaining_term_months", 0)
if new_rate and remaining_term > 0:
# Simplified payment calc for validation
new_payment = compute_monthly_payment(
account.get("outstanding_principal", 0),
new_rate / 100 / 12,
remaining_term
)
savings = current_payment - new_payment
new_steps.append(ComputationStep(
operation="Monthly savings = current_payment - new_payment",
inputs={
"current_payment": current_payment,
"new_rate": new_rate,
"remaining_term": remaining_term,
"computed_new_payment": round(new_payment, 2)
},
output=round(savings, 2),
formula_reference="POLICY-REFI-007 §2.1",
timestamp=datetime.utcnow()
))
return {
"computation_trace": new_steps,
"audit_trail": [{
"node": "computation_validation",
"steps_validated": len(new_steps),
"timestamp": datetime.utcnow().isoformat()
}]
}
Join the conversation! Your thoughts help the community grow.