Understanding Microsoft Foundry: From Simple Model APIs to Production AI Agents

Think about what happens when you call a standard language model API like GPT-5 or Claude on its own. You send a text prompt over the internet, and you get text back. It is like talking to a very smart brain sitting in a glass jar. It can answer questions, write code, or summarize an email, but it cannot log into your company database, click buttons in a browser, remember who you were yesterday, or follow company security policies on its own.

If you want to turn that brain in a jar into an actual digital coworker, you need to build an office building around it. You need an ID badge system so security knows which databases the worker is allowed to touch. You need a shared filing cabinet where conversation history is stored safely. You need a toolbox full of verified company software, and a manager watching the logs to make sure nothing breaks.

That office building is Microsoft Foundry.

Formerly known as Azure AI Studio and Azure AI Foundry, Microsoft Foundry is the single Azure platform where teams pick models, connect company data, bundle tools, and run AI agents in production. Instead of stitching together ten different cloud services by hand, you get one unified resource and one project endpoint that handles the entire stack.

What Changed in 2026 (And Why Old Tutorials Break)

If you read older blog posts from 2024 or 2025, almost all of that code is now retired or replaced. In August 2026, Microsoft officially retired the legacy Azure OpenAI Assistants API and the azure-ai-inference package.

Everything in Microsoft Foundry now revolves around four big upgrades:

One Unified Project Endpoint: You no longer create messy Hubs and separate OpenAI resources. You create a single Microsoft Foundry resource (Microsoft.CognitiveServices/accounts of kind AIServices) and work inside a Project endpoint that looks like https://your-resource.services.ai.azure.com/api/projects/your-project.
The Responses API (openai/v1/responses): With the old Assistants API, you had to create a thread, add a message, start a run, and write a while loop to poll the server every second until it finished. The new Responses API is synchronous by default. You send one request, the server runs the model and any attached tools, and you get the answer back in a single call.
One Unified SDK (Azure.AI.Projects and azure-ai-projects 2.3+): Separate agent packages are gone. Agents, conversations, evaluations, memory stores, and toolboxes all live inside a single client library.
Managed Toolboxes and Hosted Agents: You can define lightweight server-side Prompt Agents with a few lines of configuration, or deploy full custom C# and Python applications as Hosted Agents directly into Foundry's managed runtime.

Legacy Setup (Classic Portal)	Current Microsoft Foundry Setup
Hub resource + Child project + Separate OpenAI resource	Single Foundry Resource (`AIServices`) + Child Project
Assistants API (`Threads`, `Messages`, and polling `Runs`)	Responses API (`Responses` and stateful `Conversations`)
Multiple endpoints per service	Single Project Endpoint (`.services.ai.azure.com/api/projects/`)
Separate `azure-ai-agents` and `azure-ai-inference` packages	Unified `Azure.AI.Projects` (.NET) and `azure-ai-projects` (Python)
Classic RBAC (`Cognitive Services OpenAI User`)	Unified RBAC (`Foundry User`, `Foundry Project Manager`, `Foundry Owner`)

How a Request Flows Through Microsoft Foundry

When a user sends a message from a web app or Microsoft Teams into your Foundry project, it passes through five clean layers:

[ Client App / Teams / Copilot ]
               |
               v
[ 1. Foundry Control Plane ]  --> Checks Entra Agent ID & runs Prompt Shields
               |
               v
[ 2. Foundry Agent Service ]  --> Loads state from Conversations & Memory Store
               |
       +-------+-------+
       |               |
       v               v
[ 3. Toolboxes ]  [ 4. Foundry IQ ] --> Runs MCP Tools & searches company data
       \               /
        \             /
         v           v
[ 5. Foundry Models / Router ] --> Generates grounded response & logs OTel trace

Writing Your First Agent with the Unified SDK

Let's look at how clean the code is today. First, install the current packages and sign in with the Azure CLI (az login). We never hardcode API keys in code; DefaultAzureCredential automatically uses your local Azure login during development and switches to Managed Identity in production.

C# (.NET) Implementation

Install the NuGet packages:

dotnet add package Azure.AI.Projects
dotnet add package Azure.AI.Projects.Agents
dotnet add package Azure.AI.Extensions.OpenAI
dotnet add package Azure.Identity

Create a versioned Prompt Agent and chat with it over a stateful conversation:

using System;
using Azure.Identity;
using Azure.AI.Projects;
using Azure.AI.Projects.Agents;
using Azure.AI.Extensions.OpenAI;
using OpenAI.Responses;

#pragma warning disable OPENAI001

var projectEndpoint = "https://your-resource.services.ai.azure.com/api/projects/your-project";
var agentName = "incident-helper";

// 1. Connect to your Microsoft Foundry project using keyless Entra ID auth
AIProjectClient projectClient = new(
    endpoint: new Uri(projectEndpoint),
    tokenProvider: new DefaultAzureCredential());

// 2. Define and version a declarative Prompt Agent
ProjectsAgentDefinition definition = new DeclarativeAgentDefinition("gpt-5-mini")
{
    Instructions = "You are a cloud operations assistant. Give clear, numbered troubleshooting steps."
};

ProjectsAgentVersion agent = projectClient.AgentAdministrationClient.CreateAgentVersion(
    agentName,
    options: new(definition));

Console.WriteLine($"Created Agent: {agent.Name}, Version: {agent.Version}");

// 3. Create a server-side Conversation to hold multi-turn history
ProjectConversation conversation = projectClient.ProjectOpenAIClient
    .GetProjectConversationsClient()
    .CreateProjectConversation();

// 4. Bind the Responses client to our agent and conversation
ProjectResponsesClient responsesClient = projectClient.ProjectOpenAIClient
    .GetProjectResponsesClientForAgent(
        defaultAgent: agentName,
        defaultConversationId: conversation.Id);

// Turn 1
ResponseResult firstTurn = responsesClient.CreateResponse(
    "Our web app is returning HTTP 502 Bad Gateway after a deployment.");
Console.WriteLine(firstTurn.GetOutputText());

// Turn 2 (Automatically remembers Turn 1 through the Conversation ID)
ResponseResult secondTurn = responsesClient.CreateResponse(
    "How do I check that in Azure App Service specifically?");
Console.WriteLine(secondTurn.GetOutputText());

Python Implementation

Install the Python packages:

pip install "azure-ai-projects>=2.3.0" azure-identity

Run the exact same workflow in Python:

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition

PROJECT_ENDPOINT = "https://your-resource.services.ai.azure.com/api/projects/your-project"
AGENT_NAME = "incident-helper"

# 1. Initialize the unified Foundry Project Client
project = AIProjectClient(
    endpoint=PROJECT_ENDPOINT,
    credential=DefaultAzureCredential(),
)

# 2. Create a versioned Prompt Agent
agent = project.agents.create_version(
    agent_name=AGENT_NAME,
    definition=PromptAgentDefinition(
        model="gpt-5-mini",
        instructions="You are a cloud operations assistant. Give clear, numbered troubleshooting steps.",
    ),
)
print(f"Created Agent: {agent.name}, Version: {agent.version}")

# 3. Get an OpenAI client pre-bound to this agent
openai_client = project.get_openai_client(agent_name=AGENT_NAME)

# 4. Start a multi-turn Conversation
conversation = openai_client.conversations.create()

response1 = openai_client.responses.create(
    conversation=conversation.id,
    input="Our web app is returning HTTP 502 Bad Gateway after a deployment.",
)
print(response1.output_text)

response2 = openai_client.responses.create(
    conversation=conversation.id,
    input="How do I check that in Azure App Service specifically?",
)
print(response2.output_text)

Designing Your Infrastructure: Basic vs. Standard Setup

When you provision Microsoft Foundry for a real company, you have to choose between two deployment modes: Basic Setup and Standard Setup.

Basic Setup (Fast Prototyping): Microsoft manages the underlying storage and database for your conversations behind the scenes in a Microsoft-owned tenant. You can spin it up in two minutes to test ideas, but you cannot put it behind a private corporate firewall or inspect the raw database tables.
Standard Setup (Enterprise Production): You bring your own Azure Cosmos DB account (for conversation state), Azure Storage Account (for uploaded files), and Azure AI Search instance. All customer data stays inside your own Azure subscription, encrypted with your own keys, and isolated inside your own Virtual Network (VNet) using Private Endpoints.

Production Rule of Thumb: Use Basic Setup in a sandbox subscription to test models and prompts. Always use Standard Setup with DefaultAzureCredential and VNet private endpoints for staging and production workloads.