Enterprise Knowledge and Memory: Foundry IQ, Fabric IQ, Work IQ, and Memory Stores

For the last few years, almost every company built Retrieval-Augmented Generation (RAG) the same way: take a user's question, turn it into a vector embedding, grab the top 5 matching paragraphs from a search index, and paste them into the prompt.

That works fine for simple questions like "What is our vacation policy?"

It fails miserably when a manager asks a multi-part question like: "Compare our Q3 cloud spending in Europe against our Q2 forecast, and check if any delayed engineering projects in SharePoint explain the difference." A single search query cannot answer that. The system needs to break the question into pieces, search financial data in one system, search project docs in another system, look at what is missing, run a second follow-up search, and then combine the findings with exact citations.

In Microsoft Foundry, that active, multi-step research loop is built into the platform through Foundry IQ, Fabric IQ, Work IQ, and Memory Stores.

The Four Pillars of Knowledge and Memory in Foundry

Think of these four features as the different ways an experienced employee finds information:

+-----------------------------------------------------------------------+
|                    How Your Agent Knows Things                        |
+-----------------------------------+-----------------------------------+
| 1. Foundry IQ (Azure AI Search)   | 2. Fabric IQ (Microsoft Fabric)   |
| Searches unstructured PDFs,       | Queries structured lakehouses,    |
| manuals, and vector indexes       | warehouses, and OneLake tables    |
| with multi-hop query planning.    | without copying data out.         |
+-----------------------------------+-----------------------------------+
| 3. Work IQ (M365 Graph Context)   | 4. Foundry Memory Stores          |
| Connects to emails, Teams chats,  | Remembers who the user is across  |
| SharePoint sites, and meetings    | separate conversations (facts,    |
| respecting user permissions.      | preferences, past decisions).     |
+-----------------------------------+-----------------------------------+

Foundry IQ (Unstructured Knowledge Bases): Built on top of Azure AI Search, Foundry IQ turns passive vector search into an agentic research engine. Instead of one-shot retrieval, it plans sub-queries, runs hybrid vector + semantic search (vector_semantic_hybrid), checks if the retrieved chunks actually answer the question, and rewrites the query if needed.
Fabric IQ (Structured Analytics Data): Connects your agent directly to Microsoft Fabric Lakehouses and OneLake tables so the agent can answer analytical and financial questions without you writing custom SQL-generation pipelines.
Work IQ (Workplace Context): Connects the agent to organizational knowledge inside Microsoft 365 (SharePoint, OneDrive, Teams channels) while strictly honoring the signed-in user's access permissions.
Memory Stores (Long-Term Personalization): While a Conversation only remembers the messages inside the current chat thread, a Memory Store extracts lasting facts about a user (such as "User manages the East US AKS cluster and prefers Python examples") and automatically injects those facts into future conversations weeks later.

Grounding an Agent with Azure AI Search and Inline Citations

Let's connect an Azure AI Search index to a Foundry Prompt Agent and stream the response while extracting the exact source URL citations.

Prerequisites for Keyless Security

When connecting Azure AI Search to your Foundry project, never use admin API keys in production, especially if you use a private Virtual Network (key authentication is blocked on private VNets). Instead, enable the System-Assigned Managed Identity on your Foundry account and assign it two RBAC roles on your Azure AI Search resource:

Search Index Data Contributor
Search Service Contributor

Python Example: Streaming Grounded Answers with Inline Citations

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    AISearchIndexResource,
    AzureAISearchQueryType,
    AzureAISearchTool,
    AzureAISearchToolResource,
    PromptAgentDefinition,
)

PROJECT_ENDPOINT = "https://your-resource.services.ai.azure.com/api/projects/your-project"
SEARCH_CONNECTION_NAME = "corporate-policy-search"
SEARCH_INDEX_NAME = "hr-and-security-policies"

project = AIProjectClient(
    endpoint=PROJECT_ENDPOINT,
    credential=DefaultAzureCredential(),
)
openai_client = project.get_openai_client()

# 1. Resolve the full resource connection ID from the friendly connection name
search_conn = project.connections.get(SEARCH_CONNECTION_NAME)

# 2. Create an Agent wired to the search index using hybrid vector + semantic ranking
agent = project.agents.create_version(
    agent_name="policy-research-agent",
    definition=PromptAgentDefinition(
        model="gpt-5-mini",
        instructions=(
            "You are a corporate policy assistant. Always ground your answers in the search tool "
            "and include inline citations."
        ),
        tools=[
            AzureAISearchTool(
                azure_ai_search=AzureAISearchToolResource(
                    indexes=[
                        AISearchIndexResource(
                            project_connection_id=search_conn.id,
                            index_name=SEARCH_INDEX_NAME,
                            query_type=AzureAISearchQueryType.VECTOR_SEMANTIC_HYBRID,
                            top_k=5,
                        )
                    ]
                )
            )
        ],
    ),
)

# 3. Stream the grounded response and extract URL citations
stream = openai_client.responses.create(
    stream=True,
    tool_choice="required",
    input="What are the encryption requirements for storing customer backups?",
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="")
    elif event.type == "response.output_item.done":
        if event.item.type == "message" and event.item.content[-1].type == "output_text":
            for annotation in event.item.content[-1].annotations:
                if annotation.type == "url_citation":
                    print(f"\n[Source Citation: {annotation.url}]")

Adding Long-Term User Memory Across Sessions

Conversations are temporary; users start new chat threads all the time. To make your agent remember facts across separate sessions, you attach a Cosmos DB Memory Store (agent-framework-azure-cosmos-memory or project.beta.memory_stores).

Before every run, the memory provider searches past memories for the current user_id and injects only the facts that pass two strict filters:

top_k (Must be an integer >= 1): The maximum number of relevant memory facts to inject into the prompt (typically 3 to 5 so you do not clutter the context window).
min_confidence (Must be between 0.0 and 1.0): The minimum relevance score required before a memory is trusted. Setting min_confidence=0.75 prevents outdated or loosely related memories from derailing the current conversation.

Storage Layer	What It Stores	Lifespan	Backing Service in Standard Setup
Conversations API	Exact turn-by-turn messages, tool calls, and outputs	Single chat session / thread	Customer-owned Azure Cosmos DB
Vector Store (File Search)	Chunked documents uploaded directly to an agent	Tied to project or conversation	Customer-owned Azure Storage + AI Search
Foundry IQ / Fabric IQ	Enterprise knowledge bases and OneLake tables	Permanent corporate data	Azure AI Search & Microsoft Fabric
Memory Store	Distilled user facts, preferences, and summaries	Cross-session (weeks/months)	Azure Cosmos DB with vector indexing