AI Agents  

Building an Enterprise Multi-Agent ReAct Loop with Raw Ollama and LangGraph

In the modern enterprise AI landscape of 2026, relying on high-level LLM wrappers often introduces unacceptable latency, opaque prompt formatting, and unnecessary dependencies. For highly regulated or performance-critical environments, engineering teams are increasingly bypassing abstractions to interact directly with local inference engines like Ollama. However, abandoning wrappers doesn't mean abandoning orchestration. By combining Raw Ollama (via direct HTTP calls) with LangGraph, we can build highly controlled, stateful, multi-agent systems with persistent memory and Retrieval-Augmented Generation (RAG). In this end-to-end guide, we will build an Enterprise IT & Compliance Helpdesk Multi-Agent System.

1. The Real-Time Enterprise Use Case

The Scenario: An employee at "GlobalCorp" submits a complex, multi-intent query:

"I'm locked out of my VPN, and I also need to know the new Q3 2026 remote work compliance policy regarding hardware usage."

The Architecture:

  1. Memory & State: The system recalls the user's department from past interactions to tailor the compliance policy.

  2. Supervisor Agent: A routing node analyzes the intent and delegates to the specialized worker.

  3. ReAct Worker Agent (Raw Ollama): Executes a Thought-Action-Observation loop. It uses raw Ollama to reason through the steps without LangChain LLM wrappers.

  4. Tools:

    • reset_vpn_credentials (Action)

    • search_enterprise_knowledge_base (RAG)

370

2. Prerequisites & Setup

Ensure you have Ollama running locally with a capable model (e.g., llama3.1:8b or qwen2.5:14b).

# Start Ollama
ollama serve
ollama pull llama3.1:8b

# Install Python dependencies
pip install langgraph httpx faiss-cpu numpy sentence-transformers

3. Code Implementation

Step 1: The Raw Ollama Client

Instead of using langchain-ollama, we write a lightweight, raw HTTP client using httpx. This gives us exact control over the payload, eliminating wrapper overhead and ensuring enterprise security compliance.

import httpx
import json
import re
from typing import List, Dict, Any

class RawOllamaClient:
    def __init__(self, base_url: str = "http://localhost:11434", model: str = "llama3.1:8b"):
        self.base_url = base_url
        self.model = model
        self.client = httpx.Client(timeout=60.0)

    def chat(self, messages: List[Dict[str, str]], temperature: float = 0.1) -> str:
        """Direct raw HTTP call to Ollama's /api/chat endpoint."""
        payload = {
            "model": self.model,
            "messages": messages,
            "stream": False,
            "options": {"temperature": temperature}
        }
        
        response = self.client.post(f"{self.base_url}/api/chat", json=payload)
        response.raise_for_status()
        return response.json()["message"]["content"]

Step 2: Enterprise RAG & Tools Setup

We set up a lightweight FAISS vector store for the RAG tool and define our IT action tools.

import numpy as np
import faiss
from sentence_transformers import SentenceTransformer

# 1. Initialize Embedding Model
embedder = SentenceTransformer('all-MiniLM-L6-v2')

# 2. Mock Enterprise Knowledge Base
documents = [
    "Q3 2026 Remote Work Policy: Employees must use company-issued M3 MacBooks or approved Dell XPS laptops. Personal hardware is strictly prohibited for accessing production environments.",
    "VPN Reset Protocol: Password resets are handled via the Okta self-service portal. IT cannot manually reset VPN tokens.",
    "Compliance Rule 4B: All remote workers must complete the annual cybersecurity phishing simulation by August 15, 2026."
]

# Build FAISS Index
doc_embeddings = embedder.encode(documents)
dimension = doc_embeddings.shape[1]
index = faiss.IndexFlatL2(dimension)
index.add(np.array(doc_embeddings))

def search_knowledge_base(query: str) -> str:
    """RAG Tool: Searches internal enterprise policies."""
    query_embedding = embedder.encode([query])
    distances, indices = index.search(np.array(query_embedding), k=1)
    return documents[indices[0][0]]

def reset_vpn_credentials(employee_id: str) -> str:
    """Action Tool: Triggers VPN reset workflow."""
    # In reality, this calls an internal REST API or ServiceNow
    return f"VPN reset link successfully sent to the registered email for employee {employee_id}."

# Tool Registry
TOOLS = {
    "search_knowledge_base": search_knowledge_base,
    "reset_vpn_credentials": reset_vpn_credentials
}

Step 3: Defining LangGraph State and Memory

We use LangGraph’s StateGraph to manage the agent's memory and conversational state. We will use SqliteSaver for persistent memory across sessions.

from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, END
from langgraph.graph.message import add_messages
from langgraph.checkpoint.sqlite import SqliteSaver
import sqlite3

# Define the State
class AgentState(TypedDict):
    # The add_messages annotation allows LangGraph to automatically append messages
    messages: Annotated[list, add_messages]
    agent_scratchpad: str
    user_context: dict # Memory: stores user department, ID, etc.

# Initialize persistent memory (Enterprise would use PostgresSaver here)
sqlite_conn = sqlite3.connect("enterprise_memory.sqlite", check_same_thread=False)
memory = SqliteSaver(sqlite_conn)

Step 4: Building the ReAct Agent Node (Raw Ollama)

This is the core of the implementation. We format a strict ReAct prompt, pass it to our RawOllamaClient, and parse the output to decide the next step.

def format_react_prompt(state: AgentState) -> str:
    """Formats the strict ReAct prompt for the local model."""
    history = "\n".join([f"{m['role']}: {m['content']}" for m in state['messages']])
    scratchpad = state.get('agent_scratchpad', '')
    
    # Inject user memory into the system prompt
    user_ctx = state.get('user_context', {})
    ctx_str = f"User Department: {user_ctx.get('department', 'General')}. Employee ID: {user_ctx.get('emp_id', 'Unknown')}."

    prompt = f"""You are an Enterprise IT & Compliance Assistant. 
Context: {ctx_str}

You have access to the following tools:
1. search_knowledge_base(query: str) - Searches company policies and documentation.
2. reset_vpn_credentials(employee_id: str) - Resets VPN access for a user.

Conversation History:
{history}

Agent Scratchpad (Your previous thoughts and actions):
{scratchpad}

Respond using EXACTLY this format:
Thought: [Your reasoning about what to do next]
Action: [The tool to use, MUST be one of: search_knowledge_base, reset_vpn_credentials]
Action Input: [The input for the tool in JSON format]
OR, if you have the final answer:
Thought: I now have all the information needed.
Final Answer: [Your comprehensive response to the user]
"""
    return prompt

def parse_react_output(text: str):
    """Parses the raw Ollama output for ReAct components."""
    action_match = re.search(r"Action: (.*?)\nAction Input: (.*?)$", text, re.DOTALL)
    final_match = re.search(r"Final Answer: (.*)$", text, re.DOTALL)
    
    if final_match:
        return {"type": "final", "content": final_match.group(1).strip()}
    elif action_match:
        tool_name = action_match.group(1).strip()
        try:
            tool_input = json.loads(action_match.group(2).strip())
        except json.JSONDecodeError:
            tool_input = {"input": action_match.group(2).strip()}
        return {"type": "action", "tool": tool_name, "input": tool_input}
    
    # Fallback if model breaks format
    return {"type": "final", "content": text}

def react_agent_node(state: AgentState):
    """The core ReAct loop node using Raw Ollama."""
    prompt = format_react_prompt(state)
    
    # Call Raw Ollama
    ollama = RawOllamaClient()
    raw_response = ollama.chat([{"role": "user", "content": prompt}])
    
    parsed = parse_react_output(raw_response)
    
    if parsed["type"] == "final":
        # Append final answer to messages and clear scratchpad
        return {
            "messages": [{"role": "assistant", "content": parsed["content"]}],
            "agent_scratchpad": ""
        }
    else:
        # Append thought/action to scratchpad to maintain loop context
        new_scratchpad = state.get('agent_scratchpad', '') + f"\n{raw_response}\n"
        return {
            "messages": [], # Don't pollute main chat history with internal thoughts
            "agent_scratchpad": new_scratchpad,
            "next_tool": parsed["tool"],
            "tool_input": parsed["input"]
        }

Step 5: Tool Execution and Routing Nodes

We need nodes to execute the tools and a conditional edge to route the flow.

def tool_executor_node(state: AgentState):
    """Executes the tool selected by the ReAct agent."""
    tool_name = state.get("next_tool")
    tool_input = state.get("tool_input", {})
    
    if tool_name not in TOOLS:
        observation = f"Error: Tool {tool_name} not found."
    else:
        # Execute the tool
        observation = TOOLS[tool_name](**tool_input)
    
    # Add observation to scratchpad so the agent can reason about it next
    new_scratchpad = state['agent_scratchpad'] + f"Observation: {observation}\n"
    
    return {
        "agent_scratchpad": new_scratchpad,
        "next_tool": None,
        "tool_input": None
    }

def should_continue(state: AgentState):
    """Conditional edge: checks if the agent is done or needs to execute a tool."""
    if state.get("next_tool"):
        return "execute_tool"
    return "end"

Step 6: Assembling the Multi-Agent Graph

We compile the nodes and edges into a LangGraph StateGraph.

# Build the Graph
workflow = StateGraph(AgentState)

# Add Nodes
workflow.add_node("react_agent", react_agent_node)
workflow.add_node("execute_tool", tool_executor_node)

# Define Edges
workflow.set_entry_point("react_agent")
workflow.add_conditional_edges(
    "react_agent",
    should_continue,
    {
        "execute_tool": "execute_tool",
        "end": END
    }
)
# After tool execution, always go back to the agent to reason about the observation
workflow.add_edge("execute_tool", "react_agent")

# Compile with Memory
app = workflow.compile(checkpointer=memory)

4. Execution and Real-Time Demo

Let's run the agent. We will pass a thread_id to LangGraph to demonstrate persistent memory.

def run_enterprise_agent(user_query: str, thread_id: str, user_context: dict):
    config = {"configurable": {"thread_id": thread_id}}
    
    # Initial state
    initial_state = {
        "messages": [{"role": "user", "content": user_query}],
        "agent_scratchpad": "",
        "user_context": user_context
    }
    
    print(f"\n--- Running Thread: {thread_id} ---")
    print(f"User: {user_query}")
    
    # Stream the execution
    for output in app.stream(initial_state, config):
        for node_name, node_output in output.items():
            if node_name == "react_agent" and node_output.get("messages"):
                print(f"\n[Final Assistant Response]:\n{node_output['messages'][0]['content']}")
            elif node_name == "execute_tool":
                print(f"\n[Tool Executed] -> Scratchpad updated with Observation.")

# --- SCENARIO 1: First interaction ---
user_ctx_1 = {"emp_id": "EMP-992", "department": "Finance"}
query_1 = "I'm locked out of my VPN, and I also need to know the new Q3 2026 remote work compliance policy regarding hardware usage."

run_enterprise_agent(query_1, thread_id="thread_1", user_context=user_ctx_1)

# --- SCENARIO 2: Follow-up interaction (Testing Memory) ---
# Notice we don't pass the context again; LangGraph memory handles the thread state.
query_2 = "Thanks. Also, when is the phishing simulation deadline for my department?"

run_enterprise_agent(query_2, thread_id="thread_1", user_context={})

Expected Output Flow:

  1. Thought 1: The user needs a VPN reset and hardware policy info. I will start with the VPN reset using their Employee ID.

  2. Action 1: reset_vpn_credentials -> {"employee_id": "EMP-992"}

  3. Observation 1: VPN reset link successfully sent...

  4. Thought 2: VPN is handled. Now I need to search the knowledge base for Q3 2026 remote work hardware policy.

  5. Action 2: search_knowledge_base -> {"query": "Q3 2026 remote work compliance policy hardware"}

  6. Observation 2: Q3 2026 Remote Work Policy: Employees must use company-issued M3 MacBooks...

  7. Thought 3: I have all the information needed.

  8. Final Answer: Provides a synthesized response confirming the VPN reset and detailing the M3/Dell hardware policy.

  9. Follow-up (Query 2): The agent remembers the user is in "Finance" from the thread memory and accurately answers the phishing simulation deadline (August 15, 2026) without needing the context passed again.

5. Enterprise Takeaways

By decoupling the LLM inference from the orchestration layer, we achieve several critical enterprise requirements:

  1. Zero Wrapper Latency: Bypassing LangChain's ChatOllama wrapper reduces token-processing overhead and allows for custom streaming implementations if needed.

  2. Strict Prompt Control: Local models (like Llama 3.1) can be highly sensitive to formatting. Writing the raw HTTP payload ensures our ReAct prompt structure is never altered by an abstraction layer.

  3. Stateful Multi-Agent Routing: LangGraph’s StateGraph allows us to seamlessly inject memory (user_context) and manage the ReAct scratchpad without losing the conversational thread.

  4. Security & Compliance: Raw HTTP calls can be easily routed through enterprise API gateways, audited, and restricted to internal VPCs, ensuring no telemetry leaks to third-party wrapper libraries.

This architecture provides a robust, production-ready foundation for 2026's enterprise AI deployments, balancing the raw power of local inference with the sophisticated state management required for complex, multi-step workflows.


Membership not found