AI Agents  

Enabling Microsoft Agent Framework multi-turn sessions with GraphRAG router checkpoints

TL;DR:

  • Resume Microsoft 365 Agents Playground sessions without losing routed context.

  • Capture router spans with session identifiers, checkpoint usage, and fallback metadata.

  • Preserve Azure AI Foundry model router policies while meeting Part 5 evaluation gates.

  • Align session storage improvements with Microsoft Agent Framework checkpoint guidance.

Introduction

Keeping conversational memory inside a single session is what makes GraphRAG answers actionable, because every turn depends on the classifier understanding the history it already routed. The earlier installments supplied those building blocks—routing patterns that coordinate sequential, concurrent, and handoff workflows; evaluation gates that locked in TaskAdherence 80% and IntentResolution 100%; and the Azure AI Foundry model router that selects Balanced, Cost, or Quality modes per request Use model router with Foundry agents. This part (7th) finishes the arc by layering checkpoint-aware session storage on top, letting Microsoft 365 Agents Playground resume the same conversation while the router contract stays intact.

Development

RouterWorkflow in src/workflows/router.py still anchors the routing contract with _MIN_ROUTING_CONFIDENCE_SCORE = 80, _CLASSIFIER_MAX_ATTEMPTS = 3, and deterministic fallback to sequential when the classifier exhausts retries. Unknown labels and explicit out_of_context results continue to degrade into sequential, preventing retrieval fan-out when context is missing. The same module now emits richer telemetry: every call records the classified label, the routed workflow, the fallback reason, and the session context that triggered the decision. On the storage side, src/agents/session_store.py keeps TTL, LRU eviction, and compaction rules while exposing ActiveWorkflowRun metadata so a resumed session can map directly to the checkpoint ID returned by Microsoft Agent Framework. Playground traffic exercises the same /api/messages surface provided by run_router_chatbot.py, meaning multi-turn validation happened on the production entry point instead of a bespoke harness. Quality gates remain unchanged: run_batch_evaluation.py and run_redteam.py still publish the Part 5 metrics and safety evidence.

Azure Monitor setup now lives in src/core/observability.py, and both the router chatbot (run_router_chatbot.py) and evaluation helper (src/evaluation/monitoring/otel_setup.py) call configure_azure_monitor_exporters() so spans, metrics, and logs share the same resource metadata without copy/pasted instrumentation code.

Router spans keep policy observable

RouterWorkflow still enforces the 80-point confidence threshold and retries transient classifier errors before falling back to sequential. Telemetry now mirrors that contract by logging session context inside router.workflow.select spans:

with _TRACER.start_as_current_span("router.workflow.select") as span:
    self._span_set_if_present(span, "router.session_id", session_telemetry.get("session_id"))
    self._span_set_if_present(span, "router.turn_index", session_telemetry.get("turn_index"))
    self._span_set_if_present(span, "router.confidence_score", router_outcome.classification.confidence_score)
    self._span_set_if_present(span, "router.fallback_reason", router_outcome.fallback_reason)

This transparency supports the per-request optimization model described in Microsoft’s model router guidance, where a single endpoint chooses between Balanced, Cost, and Quality modes without sacrificing policy enforcement.

Session continuity mirrors Microsoft guidance on checkpoints

Microsoft Agent Framework recommends durable checkpoint storage for long-running workflows Microsoft Agent Framework Workflows – Checkpoints. Part 7 follows that guidance by surfacing ActiveWorkflowRun metadata through the session store:

@dataclass(slots=True)
class ActiveWorkflowRun:
    workflow_run_id: str
    checkpoint_id: str
    workflow_type: str
    status: str = "interrupted"
    last_step: str | None = None

When the Playground reconnects, get_or_create retrieves this structure, RouterWorkflow resumes from the stored checkpoint ID, and the span metadata records which checkpoint was used. The result is evidence-backed continuity without introducing a new proxy service.

Evaluation pipeline and workflow patterns stay intact

The multi-turn validation preserves the quality guardrails introduced earlier. The latest Foundry batch evaluation (10 rows) still reports TaskAdherence 0.8, IntentResolution 1.0, Relevance 1.0, Coherence 1.0, and ResponseCompleteness 1.0, while red-team runs continue to log non-zero evaluated attacks. Operators continue to confirm those numbers via uv run python -m evaluation.scripts.run_batch_evaluation --foundry and uv run python -m evaluation.scripts.run_redteam --flow cloud-model, giving the same evidence pipeline Part 5 established. Those results align with the specification that keeps sequential, concurrent, and handoff workflows as routed targets. In practice, that means the router can still delegate research-style questions to the sequential pipeline, parallel fact gathering to the concurrent path, and specialist escalations to the handoff workflow—now with the assurance that the session store can replay the correct checkpoint if a conversation resumes after a pause.

Operational observations from the Playground session

The Microsoft 365 Agents Playground run doubled as an operational test. Because the endpoint was unchanged, the scenario covered inbound Bot Framework messages, connector delivery status, typing keep-alives, and router telemetry in one trace. Operators still start the stack with uv run python run_mcp_server.py to expose GraphRAG tools and uv run python run_router_chatbot.py to host the /api/messages endpoint before driving the Playground session. The log excerpt below shows that both turns share the same session identifier (8a9cf8519191c73decd1ec056adae276), increment turn_index, and report memory_hits on the second turn—demonstrating that the session store replayed prior history before routing. That audit trail lets operators answer “who resumed what, and why did the router choose that path?” without digging through raw checkpoints.

Telemetry and evidence

2026-08-27T17:58:29.118Z router_chatbot.message_processed INFO

{
  "conversation_id": "31be9110-30f5-467b-a029-9c0614bb8c5b",
  "session_id": "8a9cf8519191c73decd1ec056adae276",
  "turn_index": 1,
  "routed_workflow": "handoff",
  "classifier_status": "success",
  "lock_hold_ms": 30391.008,
  "memory_hits": 0
}


2026-08-27T17:59:34.812Z router_chatbot.message_processed INFO

{
  "conversation_id": "31be9110-30f5-467b-a029-9c0614bb8c5b",
  "session_id": "8a9cf8519191c73decd1ec056adae276",
  "turn_index": 2,
  "routed_workflow": "handoff",
  "classifier_status": "success",
  "lock_hold_ms": 22770.348,
  "memory_hits": 1
}

Playground session log excerpts from local (the same is exported to Azure Monitor) showing a shared session ID, increasing turn_index, and memory reuse.

ms-playground-1

ms-playground-2

ms-playground-3

Conversation capture from Microsoft 365 Agents Playground (first turn of the 2026-08-27 session) showing the router escalating to the specialist workflow.

The attached second-turn capture from the same session shows the follow-up question and the handoff response completing, confirming that the resumed conversation keeps the handoff context intact without relying on a separate transcript export.

2026-08-27T17:59:12.331Z agents.router_classifier INFO "Router classifier invoking chat completions on deployment 'model-router'"
2026-08-27T17:59:14.683Z workflows.router INFO "Router selected 'handoff' workflow"
2026-08-27T17:59:34.485Z workflows.router_chatbot_server INFO
{
  "event": "router_chatbot.progress_status_sent",
  "status_text": "Composing specialist handoff answer..."
}
2026-08-27T17:59:34.812Z workflows.router_chatbot_server INFO
{
  "event": "router_chatbot.message_processed",
  "session_id": "8a9cf8519191c73decd1ec056adae276",
  "turn_index": 2,
  "routed_workflow": "handoff",
  "memory_hits": 1,
  "lock_hold_ms": 22770.348
}

Local telemetry confirming Azure Monitor exporters initialized and the resumed session emitted router metadata.

traces
| where cloud_RoleName == "router-chatbot"
| where customDimensions.session_id == "8a9cf8519191c73decd1ec056adae276"
| project timestamp, name, customDimensions.router_routed_workflow
| order by timestamp asc
timestampnamerouter_routed_workflow
2026-08-27T17:59:14.683Zrouter.workflow.selecthandoff
2026-08-27T17:59:24.508Zrouter.workflow.stephandoff
2026-08-27T17:59:34.812Zrouter.workflow.completehandoff

Application Insights traces filtered by session identifier show the router spans, and routed workflow now stored in Azure Monitor.

traces-1

workflow-select-turn-1

Application Insights capture illustrating the query and resulting spans for the resumed session.

traces-2

workflow-select-turn-2

A second capture (attached) highlights the follow-up turn on the same operation ID, confirming that both traces remain under the shared session identifier.

Attached close-ups of the router.workflow.select span for turns one and two enumerate the classified workflow, routed workflow, confidence score, attempts, session identifier, and turn index exactly as emitted by the telemetry pipeline. They provide a direct visual audit of the metadata contract without depending on a transcript artifact or console logs.

{
  "dataset": "eval_router_data.jsonl",
  "metrics": {
    "task_adherence": 0.8,
    "intent_resolution": 1.0,
    "relevance": 1.0,
    "coherence": 1.0,
    "response_completeness": 1.0
  },
  "rows_evaluated": 10,
  "published_to_foundry": true
}

Batch evaluation summary generated by run_batch_evaluation.py, matching the Part 5 quality floor.

Key takeaways

  • Multi-turn readiness strengthened the existing router rather than spawning a new service, aligning with Microsoft’s checkpoint guidance for long-running workflows.

  • Keeping a single endpoint maintains operational clarity while Azure AI Foundry model router continues per-request optimization for cost and quality.

  • Part 5 quality and safety thresholds remain the baseline; session memory arrives without relaxing governance.

  • Playground telemetry makes checkpoint usage auditable through shared session IDs, turn indexes, and memory hit counters.

References

Membership not found