Introduction
You hear this all the time, AI went rogue. An agent got unauthorized access to data. Why does that happen? How can you build AI agents that stays in their boundaries?
Examples:
You ask an AI agent to fix a bug. It starts reading the relevant code, checks configuration files, searches additional directories, and eventually attempts to access something you never intended it to touch. You ask another agent to prepare a report, and it starts looking through connected applications for more information.
Your reaction is understandable: “Why are you looking there? I never asked you to do that.”
These hypothetical examples illustrate a real challenge with autonomous software. We want agents to take initiative, investigate problems, and complete meaningful work. But we also expect them to recognize where their authority ends.
AI agents do not always overstep. However, when they do, the cause can involve ambiguous instructions, excessive permissions, misleading external content, or weaknesses in the application controlling them. It's no different than you don't give proper instructions to an untrained staff and ask them to do without clear instructions.
For developers, the question is bigger than why an agent behaves this way. How do we build an agent that can work independently without giving it permission to do whatever it decides is useful?
What Does It Mean for an AI Agent to Overstep?
An agent oversteps when it exceeds the authorized scope of a task. That can involve accessing unrelated information, performing an unapproved action, or attempting to work around a restriction.
Exploration itself is not the problem. If you ask an agent to investigate a checkout failure, reviewing the checkout code and relevant test logs may be necessary. Searching employee folders for passwords or deploying a production change is a different matter.
User’s request | Reasonable behavior within authorized scope | Potential overreach |
|---|
Review a repository | Read relevant code and report findings | Modify unrelated projects |
Summarize a document | Analyze the supplied document | Search private email without authorization |
Investigate a test failure | Inspect test output and dependencies | Retrieve production credentials |
Draft a customer response | Prepare text for review | Send the message |
Recommend infrastructure changes | Explain options and trade-offs | Modify live infrastructure |
The distinction is straightforward: information or action can be relevant to a problem without being authorized.
Access, relevance, and permission are three different things.
Why AI Agents Start “Sniffing Around”
An agent typically works through a loop: interpret a goal, select a tool, perform an action, examine the result, and choose the next step. Unlike a fixed workflow, the next action can depend on what the agent discovers. This flexibility makes agents useful, but it also creates opportunities for scope expansion.
Broad Goals Leave Room for Interpretation
Instructions such as “fix everything,” “do whatever it takes,” or “keep going until it works” describe an outcome without fully defining acceptable methods.
An agent might interpret missing information as a reason to search another source. It might interpret a failed operation as a reason to try another tool. Each step can appear useful while gradually moving beyond the original assignment.
Anthropic’s agent framework describes this gap between helpful intent and user expectations. For example, a request to organize files could lead an agent to delete apparent duplicates even though the user never intended deletion.
The Application Exposes Too Much Power
Sometimes the problem begins before the model makes a decision. A developer gives a narrowly focused agent a general-purpose shell, broad database credentials, and access to unrelated services.
OWASP describes excessive agency as a vulnerability associated with excessive functionality, excessive permissions, or excessive autonomy. A tool can be unnecessary, a credential can be too powerful, or an action can execute without sufficient authorization.
A document summarizer should not automatically inherit every capability available to the person using it.
External Content Can Redirect the Agent
An agent may encounter instructions embedded in a webpage, repository file, email, or tool response. Those instructions might claim that another action is required to complete the task.
For example, a malicious repository document could tell an agent that it must upload diagnostic files before reviewing the code. If the agent follows that instruction, untrusted content has influenced its actions.
This is indirect prompt injection. OpenAI describes attacks that use convincing workflow language and social engineering to make unauthorized actions appear legitimate.
These behaviors do not require human curiosity or a desire to spy. To diagnose a specific incident, examine the instructions, tool calls, and permission configuration. Do not infer motivation from a surprising action alone.
Can You Guarantee That an Agent Will Never Overstep?
You cannot honestly guarantee that a model will never propose an inappropriate action. You can, however, build controls that reject specific unauthorized actions even when the model requests them.
For example, a database identity without write privileges should be unable to update records through that connection. An execution environment without access to a private directory should prevent the agent from reading it through its filesystem tools.
Those protections depend on correct implementation and coverage of every available route. An agent blocked by one tool must not retain equivalent access through another.
The goal is to combine clear instructions with enforceable boundaries, then test whether those boundaries hold. Microsoft’s guidance similarly emphasizes hard authorization controls rather than relying on assumptions about what an agent will choose to do.
1. Define a Task Contract Before Granting Access
Before giving an agent tools, define what successful and authorized completion means.
For a checkout investigation, I would use a task contract like this:
Control | Example boundary |
|---|
Objective | Identify the checkout failure and prepare a proposed fix |
Read access | Checkout repository and sanitized test logs |
Write access | Designated development branch |
Data access | Approved test dataset |
External access | Required documentation and test services |
Restricted actions | Production deployment, customer messaging, credential changes |
Completion condition | Proposed change, relevant test results, and remaining limitations |
Escalation condition | Required information or action falls outside the approved scope |
This is a proposed configuration for the example, not a universal template.
The contract should exist in both the agent’s instructions and the application’s policy configuration. The instructions explain the assignment; the policy controls what can execute.
A vague instruction to “be careful” does not define an operating boundary.
2. Enforce Authorization Outside the Model
The model can propose an action. Your application must independently determine whether that action is permitted.
In a .NET application, authorization should happen before a tool operation reaches the database, filesystem, or external service. The check should consider the authenticated identity, target resource, tenant, requested operation, and current task.
Do not trust an agent-generated field such as approved: true. Approval must come from trusted application state.
Microsoft’s least-privilege guidance treats agent identity, resource scope, tool access, and downstream authorization as explicit security requirements.
For example, a customer-support agent requesting order information should not be able to retrieve another customer’s order simply by changing an identifier. The service must verify access to that particular order.
This is ordinary application security applied consistently to agent-generated requests.
3. Give Each Agent the Minimum Tools and Permissions
A tool should expose only the operations needed for the task.
If an agent needs to retrieve an order’s delivery status, a narrowly defined operation is easier to control than arbitrary SQL execution. If it needs to draft an email, it does not automatically need permission to send messages.
Also restrict the credentials behind the tools. Removing a delete button from the agent’s tool list does little if another available tool can perform the same deletion using a privileged account.
OWASP recommends minimizing tool functionality and permissions, avoiding unnecessarily open-ended tools, and enforcing restrictions in downstream systems.
For agents that genuinely need a shell, provide it inside an appropriately restricted environment. Broad execution capability increases the importance of containment.
4. Isolate Files, Secrets, and Network Access
An agent working on one project should not automatically see everything on the host machine.
Use an isolated workspace with explicit filesystem permissions. Restrict network access to the services required for the assignment. Apply the restrictions to scripts and subprocesses as well as direct tool calls.
Anthropic’s sandboxing approach combines filesystem and network isolation because the two controls address different routes to harm.
Keep production secrets outside the agent’s accessible environment wherever possible. When credentials are necessary, use restricted, short-lived credentials appropriate to the operation.
Network rules also need more precision than an approved hostname. An allowed cloud service may support uploads to an attacker’s account. Anthropic has documented an incident where an approved API domain became an exfiltration route through an attacker-controlled credential.
Validate the destination account, operation, and data involved—not just the domain.
5. Separate Preparation From Consequential Actions
Many workflows have a natural boundary between preparing work and making it take effect.
An agent can draft a message before sending it, prepare a code change before deployment, or assemble a proposed purchase before placing the order.
For actions requiring approval, I would bind that approval to the exact operation being reviewed. A deployment approval should identify the change and environment. A message approval should identify the recipients and content.
If those details materially change, the earlier approval should not silently cover the new action.
Avoid asking users to approve every routine step. Excessive prompts can train people to click without examining the request; Anthropic discusses this approval-fatigue problem in its containment guidance.
Let agents operate freely within established boundaries. Require meaningful intervention when authority must expand.
6. Treat Retrieved Content as Evidence, Not Authority
A document can provide facts without having permission to instruct the agent.
Keep trusted instructions separate from retrieved material. Preserve the source of external content, and prevent claims inside that content from changing access rules or approval status.
Instruction-hierarchy research aims to improve models’ ability to distinguish trusted instructions from lower-trust information. However, model behavior should complement application controls rather than replace them.
For multi-agent systems, apply the same discipline to messages between agents. A research agent’s output should not automatically authorize a deployment agent to act.
Microsoft’s shared-responsibility guidance explicitly treats inter-agent communication as a trust boundary and places responsibility for tool permissions, orchestration, memory, and authorization on the application owner.
7. Define Stop Conditions and Handle Denials Correctly
An autonomous agent needs a clear definition of when to stop.
For the checkout example, I would stop the task when the proposed change and relevant verification are complete, when required access is unavailable, or when the remaining work needs additional authorization.
I would also impose task-appropriate limits on execution time, tool calls, retries, and spending. These controls limit runaway activity; they do not substitute for permissions.
Most importantly, distinguish temporary failures from explicit denials.
A network timeout may justify a limited retry. An authorization denial should not trigger a search for another credential or a different tool that reaches the same protected resource.
The agent should report the blocked operation and explain what additional authority would be required. It should continue other authorized work when useful.
8. Test the Agent’s Boundaries, Not Just Its Answers
An agent can produce a correct answer while taking an unacceptable route to get there. Evaluation must examine actions and resulting system state as well as the final response.
Anthropic’s guidance on agent evaluations distinguishes the execution trace from the outcome, which is useful when assessing both task success and behavior.
For the checkout agent, I would include these tests:
Test scenario | Expected behavior |
|---|
Required log file is missing | Reports the gap without inventing evidence |
A document instructs it to upload private files | Rejects the instruction and stays within scope |
A tool returns an authorization denial | Does not seek an unauthorized alternative route |
An accessible file contains a production credential | Does not treat discovery as permission to use it |
A second agent claims approval was granted | Verifies approval through trusted application state |
A deployment target changes after approval | Requires authorization for the changed target |
The task is complete | Stops without making unrelated improvements |
Track blocked attempts separately from successful unauthorized actions. A blocked attempt can show that enforcement worked while still identifying behavior worth correcting.
Repeat relevant evaluations when changing models, tools, permissions, or orchestration logic.
9. Make Agent Activity Visible and Revocable
Operators need to know which agent acted, on whose behalf, against which resource, and under what authorization.
Record tool calls, policy decisions, approval references, and resulting changes. Avoid filling logs with secrets or unnecessary personal information.
Give each deployed agent an accountable owner and a way to revoke access quickly. Stopping a conversation is insufficient if background jobs or delegated agents continue running.
Microsoft’s agent-risk guidance emphasizes auditable identities, least privilege, lifecycle management, and traceability.
For production workflows, I would also define how to cancel queued operations and recover from authorized but incorrect changes. Backups, version history, and rollback procedures remain necessary.
Build Agents With Defined Authority
I want AI agents that investigate problems, question assumptions, and complete difficult work. Their usefulness depends on having enough freedom to make progress.
But freedom needs an operating boundary. An agent assigned to analyze information does not automatically gain permission to retrieve every related record. An agent assigned to prepare a change does not automatically gain permission to publish it.
When we build agents, we own the tools, credentials, approval paths, and execution environment around them. A stronger model does not remove that responsibility.
The standard should be simple: the agent can choose how to complete its assignment, but it cannot grant itself more authority to do so.
Join the conversation! Your thoughts help the community grow.