Running an AI model inside Kubernetes solves one problem, but it creates another.
Your application now needs a secure way to call the model.
A common approach is to create an API key, store it in a Kubernetes Secret, mount or inject that credential into the application, and make sure it is rotated when necessary.
That works, but there is another option when both the application and the AI model are running inside the same Kubernetes environment.
Kubernetes service account token authentication.
Foundry Local on Azure Local now supports Kubernetes service account token authentication for in-cluster inference. The feature was introduced with the September 2026 2609 extension release.
With this approach, a workload can use its Kubernetes service account identity when calling a Foundry Local ModelDeployment. Kubernetes RBAC determines whether that workload is allowed to make the request.
This removes the need to create and distribute a separate long-lived credential for every internal application.
The important part is that this authentication method is intended for data-plane inference. It is not a replacement for authentication used to manage Foundry Local resources through the control plane.
What Is Foundry Local on Azure Local?
Foundry Local on Azure Local brings AI model inference to Azure Local infrastructure running an Arc-enabled Kubernetes cluster.
Instead of sending inference requests to a remote AI service, organizations can deploy models into their own environment.
A simplified architecture looks like this:
Application
|
v
AKS Arc / Kubernetes
|
+----------------------+
| |
v v
Application Pod Foundry Local
|
v
ModelDeployment
|
v
AI Model
The platform provides Kubernetes-native model deployment and management while exposing OpenAI-compatible inference APIs for generative workloads.
Foundry Local currently supports CPU and GPU inference, multiple model deployments, multi-node deployments, disconnected environments, and both catalog and customer-provided models.
Why Authentication Matters for In-Cluster Inference
Imagine a Kubernetes cluster running three applications:
Namespace: production
orders-api
reporting-api
customer-support-api
The cluster also has an AI model:
phi-4-mini
All three applications can potentially reach the model endpoint.
But should all three be allowed to use it?
Probably not.
For example:
orders-api
-> Allowed
reporting-api
-> Allowed
unknown-debug-pod
-> Denied
This is where identity becomes important.
An application should not gain access simply because it can reach a Kubernetes Service.
The inference endpoint needs to know:
Who is calling?
Is that identity trusted?
Is that identity authorized?
Kubernetes service accounts provide an identity that can be attached to workloads.
What Is a Kubernetes Service Account?
A Kubernetes ServiceAccount gives a workload an identity inside a Kubernetes cluster.
For example:
apiVersion: v1
kind: ServiceAccount
metadata:
name: ai-client
namespace: app
A Pod can then run using that service account:
apiVersion: v1
kind: Pod
metadata:
name: inference-client
namespace: app
spec:
serviceAccountName: ai-client
containers:
- name: client
image: my-inference-client:latest
The important relationship is:
Pod
|
+-- ServiceAccount
|
+-- Kubernetes identity
Foundry Local can use that identity when the workload makes an inference request.
What Changed in September 2026?
The September 2026 2609 release added Kubernetes service account token authentication for Foundry Local on Azure Local.
The feature supports data-plane inference requests from workloads running inside Kubernetes.
The service account identity is authorized through Kubernetes RBAC.
Microsoft also describes the tokens as short-lived and audience-bound.
The feature is enabled by default with:
satAuth.enabled: true
It supports pod-to-pod inference across namespaces and is particularly useful for environments with limited or no cloud connectivity.
This is a meaningful change for applications that already use Kubernetes identity as part of their security model.
API Keys vs Service Account Tokens
Before looking at the configuration, it helps to compare the two approaches.
Area | API Key | Kubernetes Service Account Token |
|---|
Credential type | Secret/API key | Kubernetes identity token |
Credential storage | Kubernetes Secret | Managed through Kubernetes identity |
Rotation | Application/platform responsibility | Short-lived token model |
Identity | Credential-based | Workload identity |
Good for | General clients | In-cluster workloads |
Cloud dependency | Can work without cloud authentication | Designed for Kubernetes-native access |
RBAC integration | Separate credential management | Kubernetes RBAC |
Long-lived secret | Usually yes | No |
Pod-to-pod use | Possible | Natural fit |
This does not mean API keys are obsolete.
API keys remain useful for clients that do not have a Kubernetes service account identity.
The service account approach is mainly attractive when the caller is already running inside the cluster.
The Basic Request Flow
With service account authentication, the flow looks like this:
Application Pod
|
v
Kubernetes ServiceAccount
|
v
Service Account Token
|
v
Foundry Local Inference Endpoint
|
v
Authentication
|
v
Authorization / RBAC
|
v
ModelDeployment
|
v
AI Model
The application does not need to manually create a separate API key for itself.
Instead, its Kubernetes identity becomes part of the authentication process.
A Simple Service Account
Start with a dedicated service account:
apiVersion: v1
kind: ServiceAccount
metadata:
name: ai-inference-client
namespace: application
Apply it:
kubectl apply -f service-account.yaml
Verify it:
kubectl get serviceaccount \
ai-inference-client \
-n application
You should see the service account listed.
Use a Dedicated Service Account Per Application
Avoid using one generic service account for every application.
For example, this is easier to reason about:
orders-api
-> orders-ai-client
reporting-api
-> reporting-ai-client
support-api
-> support-ai-client
than:
all applications
-> default
The dedicated approach gives you a much clearer authorization boundary.
If orders-api is compromised, you can revoke or change its permissions without automatically affecting every other application.
Why You Should Avoid the default Service Account
Every Kubernetes namespace normally has a default service account.
It is convenient, but it is not a good identity boundary for applications that need different permissions.
Instead of:
serviceAccountName: default
use:
serviceAccountName: ai-inference-client
This makes the application's security relationship explicit.
Create a Role for Inference Access
Kubernetes RBAC controls which identities can perform specific actions.
A simplified example looks like this:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: foundry-inference-client
namespace: application
rules:
- apiGroups: [""]
resources: ["services"]
verbs: ["get"]
However, the exact RBAC rules required for your Foundry Local deployment should come from the current Foundry Local authentication configuration rather than being copied blindly from a generic Kubernetes example.
The important principle is:
ServiceAccount
|
v
RBAC
|
v
Allowed operations
Give the workload only the permissions it actually needs.
Bind the Role to the Service Account
The RoleBinding connects the identity to the permissions.
For example:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: foundry-inference-client
namespace: application
subjects:
- kind: ServiceAccount
name: ai-inference-client
namespace: application
roleRef:
kind: Role
name: foundry-inference-client
apiGroup: rbac.authorization.k8s.io
The relationship becomes:
ai-inference-client
|
v
RoleBinding
|
v
Role
|
v
Allowed permissions
This is the normal Kubernetes RBAC model.
Tokens Are Short-Lived
One of the advantages of service account token authentication is that the token model is designed around short-lived credentials.
That is different from placing a long-lived API key in a Secret and keeping it there indefinitely.
Conceptually:
API key
Create
|
v
Store
|
v
Use
|
v
Rotate
|
v
Replace
Whereas service account authentication is closer to:
Pod identity
|
v
Short-lived token
|
v
Request
|
v
Token expires
|
v
New token
This reduces the need for applications to implement their own credential rotation workflow.
Microsoft specifically describes Foundry Local's Kubernetes service account tokens as short-lived and audience-bound.
Audience Binding Matters
An audience-bound token is intended for a particular service or audience.
That is important because a token issued for one purpose should not automatically be accepted everywhere.
The general idea is:
Token
|
+-- Identity
|
+-- Expiration
|
+-- Audience
The inference service validates that the token is intended for it.
This provides an additional boundary beyond simply checking whether the token belongs to a known Kubernetes service account.
Cross-Namespace Inference
Foundry Local's service account authentication supports pod-to-pod inference across namespaces.
For example:
Namespace: application
----------------------
client-pod
serviceAccount:
ai-inference-client
|
|
v
Namespace: foundry
------------------
ModelDeployment
This is useful because production Kubernetes clusters often separate workloads into namespaces.
You may want:
application namespace
for business applications and:
foundry namespace
for AI infrastructure.
The identity can cross that boundary while authorization remains controlled.
Why Cross-Namespace Access Needs Care
Cross-namespace access is useful, but it should not become:
Every namespace
|
v
Every model
Instead, define clear application-to-model relationships.
For example:
orders namespace
|
+-- orders-ai-client
|
+-- approved inference endpoint
and:
analytics namespace
|
+-- analytics-ai-client
|
+-- approved inference endpoint
This keeps access easier to audit.
Calling the OpenAI-Compatible Endpoint
Foundry Local provides an OpenAI-compatible chat completions endpoint:
/v1/chat/completions
The platform's current API reference lists the Chat Server on port 5000 for OpenAI-compatible chat completions and audio transcription.
A typical request body looks familiar to developers who already use OpenAI-compatible APIs:
{
"model": "phi-4-mini",
"messages": [
{
"role": "user",
"content": "Explain Kubernetes service accounts."
}
]
}
The authentication mechanism changes.
The application uses its Kubernetes workload identity rather than distributing a separate static credential.
Example Application Pod
A simple application deployment can reference the service account:
apiVersion: apps/v1
kind: Deployment
metadata:
name: ai-client
namespace: application
spec:
replicas: 2
selector:
matchLabels:
app: ai-client
template:
metadata:
labels:
app: ai-client
spec:
serviceAccountName: ai-inference-client
containers:
- name: client
image: my-ai-client:latest
ports:
- containerPort: 8080
The important line is:
serviceAccountName: ai-inference-client
This tells Kubernetes which service account identity the Pod should use.
The Application Should Not Hard-Code Tokens
Avoid code like:
TOKEN = "eyJhbGciOi..."
or:
API_KEY = "my-production-key"
inside the application image.
Credentials should not be baked into:
Dockerfile
Source code
Git repository
Container image
Application configuration
The application should obtain its credential through the Kubernetes-supported identity mechanism.
This keeps the credential lifecycle separate from application code.
API Key Authentication Still Has Its Place
Foundry Local currently supports API key authentication as another option.
For example, the documented API-key flow retrieves the deployment's primary key from a Kubernetes Secret and sends it using:
Authorization: Bearer <API_KEY>
This remains useful when:
The caller is outside Kubernetes.
A legacy application expects a static credential.
You need a simple integration.
The client cannot participate in Kubernetes service account identity.
Microsoft's current inference documentation lists API key and Microsoft Entra ID JWT authentication for general inference scenarios.
The service-account approach adds another option specifically suited to workloads already running inside Kubernetes.
Service Account Authentication Is for the Data Plane
This distinction is easy to miss.
The new SAT support applies to data-plane inference.
It does not mean that the service account can automatically manage Foundry Local deployments.
For example:
Data plane
-----------
Send prompt
Generate response
Run inference
versus:
Control plane
-------------
Create deployment
Update deployment
Delete deployment
Manage models
The current Foundry Local API separates these responsibilities.
The control plane provides management operations, while the Chat Server exposes OpenAI-compatible inference endpoints.
Do not give an inference-only application unnecessary control-plane privileges.
A Good Security Boundary
A production setup can look like this:
Kubernetes Cluster
|
+------------+------------+
| |
v v
Application Namespace Foundry Namespace
| |
| |
Application Pod ModelDeployment
| |
| |
ServiceAccount Model
|
v
SAT Token
|
+------------------------->
The application authenticates as itself.
The model infrastructure remains separately managed.
Why This Is Useful in Disconnected Environments
One of the strongest use cases is an environment with limited or no internet connectivity.
Foundry Local on Azure Local supports disconnected operations, where model artifacts and required platform dependencies can be supplied locally rather than requiring outbound internet access during normal operation.
In such environments, relying heavily on external identity or credential-management services can make architecture more complicated.
Kubernetes service account identity already exists inside the cluster.
That makes it a natural fit:
Application
|
v
Kubernetes identity
|
v
Foundry Local
|
v
Local model
No separate cloud authentication round trip is required for the in-cluster identity itself.
Service Account Tokens vs Microsoft Entra ID
Foundry Local supports different authentication patterns for different clients.
Kubernetes SAT
Best suited for:
Pod
|
v
Pod
inside the Kubernetes environment.
Microsoft Entra ID
Useful when:
User
Application
Service Principal
Managed Identity
|
v
Foundry Local
needs Azure identity and RBAC integration.
Microsoft's Entra authentication guide uses Azure RBAC roles such as Cognitive Services OpenAI User for inference access.
A simple decision table:
Scenario | Recommended approach |
|---|
Pod calling model in same cluster | Kubernetes SAT |
Internal Kubernetes service | Kubernetes SAT |
External enterprise application | Entra ID or API key |
Developer testing | API key can be simpler |
Disconnected in-cluster application | Kubernetes SAT |
Human user access | Entra ID |
The exact choice depends on how the application is deployed and managed.
Checking the Service Account
Use:
kubectl get serviceaccount \
ai-inference-client \
-n application
Then inspect it:
kubectl describe serviceaccount \
ai-inference-client \
-n application
For modern Kubernetes versions, do not assume that a long-lived Secret containing a service account token will automatically exist.
Kubernetes uses projected service account tokens for workloads, which is a better fit for short-lived credentials.
Testing the Identity
Before troubleshooting Foundry Local, verify the Pod's Kubernetes identity.
For example:
kubectl get pod \
-n application \
-l app=ai-client \
-o jsonpath='{.items[0].spec.serviceAccountName}'
You should see:
ai-inference-client
If the output is:
default
your Deployment is not using the intended service account.
That simple check can save a lot of debugging time.
Test Network Connectivity Separately
Authentication and networking are two different problems.
A Pod can have the correct identity and still be unable to reach the model endpoint.
Check:
DNS
Service
Port
NetworkPolicy
Gateway
TLS
Authentication
Authorization
A useful troubleshooting order is:
Pod identity
↓
DNS resolution
↓
TCP connectivity
↓
TLS
↓
Authentication
↓
Authorization
↓
Inference
Do not immediately assume a 401 or 403 is a network issue.
Understanding Common Errors
401 Unauthorized
Usually indicates an authentication problem.
Possible causes include:
Missing token
Invalid token
Expired token
Wrong audience
Authentication configuration mismatch
403 Forbidden
Usually points toward authorization.
The identity may be valid but not permitted to perform the requested operation.
Check Kubernetes RBAC and the Foundry Local authorization configuration.
404 Not Found
Check:
Endpoint
Namespace
ModelDeployment
Gateway path
Model name
Connection Timeout
Start with:
Service
DNS
NetworkPolicy
Gateway
Firewall
TLS
Do not start with RBAC.
Use Least Privilege
The application does not need access to every model simply because it can authenticate.
Think about permissions as:
Identity
|
+-- Model A: allowed
|
+-- Model B: denied
|
+-- Control plane: denied
This is much safer than:
Identity
|
+-- Everything: allowed
Least privilege is particularly important when AI endpoints can process sensitive business data.
Protect the Model Endpoint
Authentication is only one part of the security design.
Foundry Local on Azure Local also supports endpoint exposure through Kubernetes Gateway API. Current deployments can use internal exposure by default, external exposure through a LoadBalancer gateway, or no gateway exposure depending on configuration.
For an application that runs inside the same cluster, an internal endpoint is usually preferable to unnecessarily exposing the model outside the cluster.
The general principle is:
Internal caller
|
v
Internal endpoint
rather than:
Internal caller
|
v
External load balancer
|
v
Back into cluster
unless external access is actually required.
Keep TLS in the Architecture
Authentication tells you who is making the request.
TLS protects the request while it is traveling through the network.
You generally want both:
Authentication
+
Authorization
+
Encryption
Foundry Local's current architecture supports TLS and controlled endpoint exposure through its Gateway API-based networking stack.
Do not treat authentication as a replacement for encryption.
Service Accounts Are Not User Accounts
A Kubernetes service account represents a workload.
It is not intended to represent an individual human user.
For example:
Human
|
v
Application
|
v
ServiceAccount
|
v
Foundry Local
If you need to know which human initiated a request, your application should maintain that application-level identity separately.
The service account identifies the workload making the request.
Multiple Applications Can Use One Model
A single model deployment can serve multiple applications.
For example:
orders-api --------\
\
reports-api ---------> Phi model
/
support-api --------/
Each application can have its own service account:
orders-ai-client
reports-ai-client
support-ai-client
This gives you independent identity boundaries even though they share the same model deployment.
That is a much cleaner approach than giving every application the same API key.
Service Account Identity and Multi-Node Inference
Foundry Local now supports multi-node Kubernetes deployments for larger inference workloads.
The platform can distribute inference workloads across multiple nodes, including high-parameter generative models, while maintaining Kubernetes-native management.
The application does not need to know which node is running the model.
It simply calls the inference endpoint.
The identity follows the application workload rather than being tied to a particular node.
This is one of the practical benefits of using workload identity instead of node-specific credentials.
A Practical Production Architecture
A more complete design could look like this:
Kubernetes Cluster
|
+-------------------+-------------------+
| |
v v
Application Namespace Foundry Namespace
| |
+-----+------+ +------+
| | |
v v v
orders-api reports-api ModelDeployment
| | |
v v v
ServiceAccount ServiceAccount AI Model
| |
+------token-+
|
v
Internal Inference
Endpoint
|
v
Foundry Local
The important boundaries are:
Application identity
+
Kubernetes RBAC
+
Internal network
+
TLS
+
Inference authorization
That gives you a much stronger design than simply putting an API key into every application.
Common Mistakes
Using the Default Service Account
Create a dedicated service account for the application.
Sharing One Service Account Everywhere
Separate identities make auditing and revocation easier.
Treating SAT as a Control-Plane Credential
SAT support is for data-plane inference.
Hard-Coding Tokens
Never put tokens into source code or container images.
Assuming Every Token Is Long-Lived
Service account tokens are designed to be short-lived.
Ignoring the Audience
An audience-bound token is intended for a particular service.
Granting Excessive RBAC Permissions
An inference client should not automatically receive permissions to manage deployments.
Exposing the Model Publicly
Keep inference traffic internal when external access is not required.
Debugging Authentication Before Testing Connectivity
First establish that the Pod can actually reach the endpoint.
Treating Authentication and Authorization as the Same Thing
Authentication answers:
Who are you?
Authorization answers:
What are you allowed to do?
Both matter.
When Kubernetes SAT Is a Good Choice
Use Kubernetes service account token authentication when:
Your application already runs inside Kubernetes.
The model is deployed inside the same Kubernetes environment.
You want workload-based identity.
You want to avoid distributing static API keys.
You want short-lived credentials.
You need pod-to-pod inference.
You operate in a disconnected or restricted environment.
You already use Kubernetes RBAC as part of your security model.
It is less useful when the client is an external application with no Kubernetes identity.
In that case, API key or Microsoft Entra ID authentication may be more appropriate.
Summary
Foundry Local on Azure Local now supports Kubernetes service account token authentication for in-cluster inference. This lets a Kubernetes workload call a deployed AI model using its service account identity instead of requiring a separate API key for every application.
The service account token is short-lived and audience-bound, while Kubernetes RBAC provides the authorization layer. The approach also supports inference across namespaces, making it practical to keep applications and AI model infrastructure in separate namespaces.
API keys and Microsoft Entra ID remain useful for other scenarios. Kubernetes SAT is mainly a strong fit when both the caller and the model are part of the Kubernetes environment.
For production deployments, use dedicated service accounts, grant only the permissions the application needs, keep inference traffic private when possible, and remember that data-plane inference access is different from permissions to manage Foundry Local resources.
Join the conversation! Your thoughts help the community grow.