Code review has traditionally focused on reading a pull request and identifying potential problems before the code is merged. Automated CI pipelines then take over to compile the project, execute tests, run static analysis, and perform other validation.
GitHub Copilot Code Review with shell tools changes the boundary between these activities.
Instead of only examining source code, an agent can use supported shell tools to perform development tasks such as running build commands and tests, inspecting their output, and using those results to improve its review.
This creates a workflow closer to an AI-assisted development environment:
Pull Request
|
v
Copilot Code Review
|
v
Inspect Repository
|
v
Run Build / Tests
|
v
Analyze Results
|
v
Review Findings
|
v
Human Validation
The important question is not whether AI can execute a command.
The more useful question is whether allowing an AI reviewer to execute repository commands produces better review feedback without introducing security or trust problems.
What Are Shell Tools in Copilot Code Review?
A shell tool allows an agent to interact with the repository through command-line operations.
For a code review workflow, this can enable actions such as:
Build the project
Run unit tests
Inspect generated files
Check project structure
Execute repository scripts
Analyze command output
For example, a .NET project might use:
dotnet build
followed by:
dotnet test
A JavaScript project could instead use:
npm test
or:
npm run build
The exact commands depend on the repository.
The important difference is that the agent can obtain evidence from executing the project's own tooling rather than relying entirely on static inspection.
Why Static Review Is Not Always Enough
Consider this pull request:
public async Task<User?> GetUserAsync(int id)
{
return await repository.FindAsync(id);
}
The code may appear correct.
But perhaps the repository interface has changed:
Task<User?> FindAsync(
int id,
CancellationToken cancellationToken);
A static review might identify the mismatch.
A build provides definitive evidence:
error CS7036:
There is no argument given that corresponds
to the required parameter 'cancellationToken'.
The agent can then use the build output as additional context.
This creates a useful feedback loop:
Code
|
v
Build
|
+---- Success ----> Continue Review
|
+---- Failure ----> Investigate Error
|
v
Review Finding
Running Tests During Review
Compilation is only one part of validation.
Suppose a change modifies authorization logic:
if (user.IsAdmin)
{
return true;
}
The code may compile successfully.
But existing tests might reveal a behavioral regression:
Test: NonAdminCannotDeleteArticle
Result: Failed
Expected: Forbidden
Actual: Success
The test result gives the agent evidence that static analysis alone cannot provide.
A stronger review workflow therefore looks like:
Source Analysis
+
Build
+
Tests
|
v
Combined Review
Example: .NET Pull Request
Suppose a repository contains:
src/
tests/
Directory.Build.props
MyApp.sln
A code review agent can inspect the project and determine that the appropriate validation commands are:
dotnet build MyApp.sln
and:
dotnet test MyApp.sln
The results can then be incorporated into the review.
A successful build:
Build succeeded.
0 Error(s)
does not prove that the pull request is correct.
It only establishes that the solution compiles.
The agent still needs to review the behavior.
Build Success Is Not Review Success
This distinction is essential.
Consider:
Build
|
+--> Passed
|
v
Tests
|
+--> Passed
|
v
Security
|
+--> Potential Issue
|
v
Human Review
All tests passing does not mean that the code is secure or architecturally appropriate.
Similarly, a successful build does not verify:
Business requirements
Correct authorization behavior
Data migration safety
API compatibility
Performance characteristics
Operational behavior
Shell tools provide additional evidence. They do not replace engineering judgment.
Agent-Executed Commands and Trust
Allowing an AI agent to execute shell commands introduces an important security consideration.
A shell command can potentially:
Read files
Modify files
Access environment variables
Execute project scripts
Consume CPU and memory
Access network resources depending on the environment
Trigger side effects
Therefore, shell access should not be treated as equivalent to a harmless read-only code analysis feature.
The security model of the environment matters.
Why Repository Scripts Need Special Attention
Many repositories contain scripts that perform more than developers expect.
For example:
{
"scripts": {
"test": "node scripts/test.js",
"build": "node scripts/build.js"
}
}
Running:
npm test
does not necessarily execute a simple test runner.
It executes whatever the repository defines.
Similarly, a shell script such as:
./build.sh
may invoke several additional commands.
Before allowing an agent to execute repository commands automatically, teams should understand what those commands actually do.
A Safer Validation Model
A useful enterprise workflow is to separate commands into trust levels.
Low-Risk Commands
Examples:
dotnet build
dotnet test
npm test
go test ./...
These are generally intended for local validation, although their actual behavior should still be inspected.
Medium-Risk Commands
Examples:
npm run integration-test
./scripts/generate-code.sh
make test
These may interact with additional tools or services.
High-Risk Commands
Examples include scripts that:
Modify infrastructure
Deploy resources
Delete data
Change cloud configuration
Access production systems
Rotate credentials
These should not be automatically executed simply because an AI reviewer encounters them in a repository.
Preventing Dangerous Tool Usage
The principle should be:
AI can validate code
|
v
AI should not automatically control production
CI environments should use least privilege.
If a review agent only needs to compile and test a project, it should not have:
Production credentials
Deployment permissions
Database administrator access
Cloud account owner privileges
Unrestricted secret access
This follows the same security principles used for any automated CI worker.
Shell Tools and Secrets
Environment variables deserve special attention.
A process might have access to:
DATABASE_URL
API_KEY
CLOUD_TOKEN
PACKAGE_REGISTRY_TOKEN
Even if the agent does not intentionally request these values, repository commands might access them.
Therefore, shell-enabled review environments should use appropriately scoped credentials and secret-management controls.
Never assume that a shell tool is safe simply because the AI is instructed not to print secrets.
Security should be enforced by the execution environment.
Test Isolation
Tests themselves can have side effects.
For example:
Integration Tests
|
v
Database
|
v
External Service
If an AI reviewer executes those tests against a shared environment, the tests could modify real data or consume external resources.
Prefer:
AI Review
|
v
Isolated Test Environment
|
+--> Temporary Database
+--> Mock Services
+--> Test Credentials
This is safer and more reproducible.
How Shell Results Improve Review Quality
Suppose Copilot identifies a potentially incorrect API change:
public IActionResult Get(int id)
{
return Ok(service.Get(id));
}
The agent can inspect the project and discover that:
service.Get()
was changed to:
Task<Item?> GetAsync(int id);
Running the build produces concrete evidence.
Instead of a speculative comment such as:
"This may cause a compilation problem."
the review can be based on:
"The project fails to compile because the service now
returns Task<Item?> while this action expects Item."
This makes the review more actionable.
Shell Tools and Test-Driven Review
Test failures can provide even stronger evidence.
Imagine a change to a discount calculation:
var total = price * quantity;
The agent runs:
dotnet test
and receives:
Failed:
OrderTotal_WithDiscount_ReturnsExpectedValue
Expected: 90
Actual: 100
Join the conversation! Your thoughts help the community grow.