Tracing Reasoning Failures Across Multiple Agent Steps in Retail

When we build Multi-Agent AI systems, we often assume that if each individual agent is smart, the whole system will be smart. But in reality, multi-agent systems often fail not because one agent made a massive error, but because of a cascade of small reasoning failures across multiple steps.

As AI engineers, standard logging (just printing inputs and outputs) is no longer enough. We need to trace the reasoning—the "why" behind every decision.

Let’s explore how to trace these cascading failures using a real-time Retail domain use case.

The Analogy: The Corporate Game of Telephone

Imagine a game of "telephone" in a corporate office.

The CEO whispers, "We need to boost morale."

The VP tells HR, "The CEO wants a party."

HR tells the event team, "The CEO wants an expensive party."

The event team books a $100,000 yacht cruise.

If you only look at the final output (the yacht booking), you won't understand how the disaster happened. You have to trace the message back through every person's interpretation to find where the reasoning drifted. Multi-agent AI works exactly the same way.

The Real-Time Use Case: Smart Retail Supply Chain

Imagine a large retail chain using a Multi-Agent System to automate inventory replenishment. The system has three agents working in a sequence:

Trend Analyst Agent: Monitors social media and sales data to predict product demand.
Inventory Manager Agent: Checks current warehouse stock levels and storage capacity.
Procurement Agent: Generates Purchase Orders (POs) to suppliers based on the previous agents' recommendations.

The Scenario: A Mid-Summer Disaster

It is June (peak summer). A popular sci-fi movie set in a snowy apocalypse is released, causing a viral TikTok trend around "Heated Winter Jackets."

Here is how the cascading reasoning failure happens:

Step 1 (Trend Agent): Sees a 5000% spike in social media mentions for "Heated Winter Jackets."
- Agent's internal reasoning: "High social buzz = high customer demand. I must recommend stocking up." (Failure: It ignores the current season and assumes the trend is for immediate wear, not cosplay/collection).
Step 2 (Inventory Manager): Receives the recommendation. Checks the database and sees they only have 50 jackets left.
- Agent's internal reasoning: "Trend Agent says demand is high, and stock is low. I will approve a massive restock." (Failure: It blindly trusts the Trend Agent without applying common-sense seasonal logic).
Step 3 (Procurement Agent): Receives the approval.
- Agent's internal reasoning: "I have approval to buy 10,000 jackets. I will generate a $500,000 Purchase Order." (Failure: It lacks a final financial/seasonal guardrail).

The Result: The retail chain spends half a million dollars buying heavy winter jackets in the middle of June.

Why Standard Logging Failed

If we only use standard logging, we would just see:

[INFO] Trend Agent Output: {"action": "RESTOCK", "quantity": 10000}
[INFO] Inventory Agent Output: {"status": "APPROVED"}
[INFO] Procurement Agent Output: {"PO_ID": "12345", "amount": 500000}

Looking at this, everything looks perfectly normal! The JSON is valid, the API calls succeeded. But the business logic is completely flawed. To catch this, we must trace the Chain of Thought (CoT) and the State across the agents.

How to Trace Reasoning Failures (The Solution)

To prevent these cascading failures, we need to implement Reasoning Tracing. Here are the three pillars of tracing:

1. Shared State with "Thought Process" Tracking

Instead of agents just passing final JSON decisions, they must pass a shared state object (using Pydantic) that records their internal reasoning.

2. The "Critic" or "Reviewer" Agent

Before the final action is executed, a Reviewer agent (or a deterministic code guardrail) analyzes the entire reasoning_trace of the previous agents to look for logical gaps.

3. Semantic Observability Tools

In production, we use tools like LangSmith, Arize Phoenix, or OpenTelemetry to visualize the agent graph. These tools allow us to click on any node in the graph and read the exact LLM prompt and reasoning trace that led to a decision.

Code Implementation: Tracing the Failure in Python

Let’s look at how we can build this using Python and Pydantic. We will create a shared state that forces agents to declare their reasoning, and a Reviewer that catches the flaw.

from pydantic import BaseModel, Field
from typing import List, Literal
from datetime import datetime

# 1. Define the Shared State with Reasoning Traces
class AgentStep(BaseModel):
    agent_name: str
    action: str
    reasoning: str = Field(description="The internal thought process of the agent")
    timestamp: datetime

class SupplyChainState(BaseModel):
    product_name: str
    current_month: int = Field(description="Current month (1-12)")
    execution_trace: List[AgentStep] = []

# 2. Simulating the Agents
def trend_analyst_agent(state: SupplyChainState) -> SupplyChainState:
    # The agent makes a flawed reasoning step
    step = AgentStep(
        agent_name="TrendAnalyst",
        action="RESTOCK",
        reasoning="Social media mentions for Heated Winter Jackets spiked 5000%. High buzz means high immediate demand.",
        timestamp=datetime.now()
    )
    state.execution_trace.append(step)
    return state

def inventory_manager_agent(state: SupplyChainState) -> SupplyChainState:
    step = AgentStep(
        agent_name="InventoryManager",
        action="APPROVE_RESTOCK",
        reasoning="TrendAnalyst recommended restocking. Current stock is only 50 units. I approve restocking 10,000 units.",
        timestamp=datetime.now()
    )
    state.execution_trace.append(step)
    return state

# 3. The Reviewer Agent (The Tracing & Guardrail Mechanism)
def reviewer_agent(state: SupplyChainState) -> str:
    print("\n--- REVIEWER AGENT: Analyzing Reasoning Trace ---")
    
    # The reviewer looks at the ENTIRE trace, not just the last step
    for step in state.execution_trace:
        print(f"[{step.agent_name}] Reasoning: {step.reasoning}")
        
    # Deterministic Guardrail + LLM Logic
    # Let's simulate the Reviewer noticing the temporal reasoning failure
    current_month = state.current_month
    is_winter_product = "Winter" in state.product_name
    
    if is_winter_product and current_month in [4, 5, 6, 7, 8]:
        return "REJECTED: Critical reasoning failure detected. The TrendAnalyst failed to account for the current season (June). Do not issue PO."
    
    return "APPROVED: Proceed to Procurement."

# --- Running the Pipeline ---
if __name__ == "__main__":
    # Initialize state for June (Month 6)
    initial_state = SupplyChainState(
        product_name="Heated Winter Jacket",
        current_month=6 
    )
    
    # Execute the agents
    state_after_trend = trend_analyst_agent(initial_state)
    state_after_inventory = inventory_manager_agent(state_after_trend)
    
    # Execute the Reviewer to trace and catch the failure
    final_decision = reviewer_agent(state_after_inventory)
    
    print(f"\n--- FINAL DECISION ---")
    print(final_decision)

Output of the Code

--- REVIEWER AGENT: Analyzing Reasoning Trace ---
[TrendAnalyst] Reasoning: Social media mentions for Heated Winter Jackets spiked 5000%. High buzz means high immediate demand.
[InventoryManager] Reasoning: TrendAnalyst recommended restocking. Current stock is only 50 units. I approve restocking 10,000 units.

--- FINAL DECISION ---
REJECTED: Critical reasoning failure detected. The TrendAnalyst failed to account for the current season (June). Do not issue PO.

Key Takeaways for AI Engineers

Never Pass "Black Box" Outputs: Always force your agents to output their reasoning alongside their action. If you can't read the agent's thought process, you can't debug it.
Trace the Chain, Not Just the Node: A failure in Agent 3 is often caused by Agent 1. Observability tools must allow you to trace the entire graph execution.
Implement "Critic" Patterns: Always put a Reviewer Agent or a deterministic code guardrail at the end of a multi-agent chain to evaluate the combined reasoning of all previous steps before taking real-world action.
Use Pydantic for State Management: As shown in the code, using Pydantic models for your agent state ensures that the data passed between agents is strictly typed, validated, and easy to inspect.

By shifting our focus from output validation to reasoning tracing, we can build multi-agent systems that are not just autonomous, but actually reliable enough for enterprise retail environments.