AI coding tools can generate an entire feature in minutes. They can create API endpoints, database queries, authentication logic, configuration files, and application tests.
However, generated code can also introduce security vulnerabilities. An AI tool might use an unsafe SQL query, select an outdated dependency, expose a credential, or implement authorization incorrectly.
Reviewing every generated line manually becomes difficult when developers use AI tools throughout the development process.
Automated security scanning helps identify these problems before code reaches production. Tools such as GitHub CodeQL, Semgrep, Gitleaks, and dependency scanners can examine source code, dependencies, and repository history.
GitHub has also introduced AI-powered security scanning to identify vulnerabilities that traditional static analysis might miss.
In this article, we will examine how to scan AI-generated code, configure automated checks in GitHub Actions, and review security findings before merging a pull request.
Why AI-Generated Code Needs Automated Scanning
Consider a developer who asks an AI coding assistant to create a customer search API.
The assistant generates the controller, database query, and response model. The application builds successfully, and the basic tests pass.
However, the generated query might look like this:
def search_customer(cursor, customer_name):
query = (
"SELECT * FROM customers "
"WHERE name = '" + customer_name + "'"
)
cursor.execute(query)
return cursor.fetchall()
The problem is that user input is directly added to the SQL statement. An attacker could provide specially crafted input and change how the database executes the query.
A safer implementation uses parameterized SQL:
def search_customer(cursor, customer_name):
query = """
SELECT *
FROM customers
WHERE name = %s
"""
cursor.execute(query, (customer_name,))
return cursor.fetchall()
This example shows why successful compilation and unit tests are not enough.
Security scanning can help identify unsafe coding patterns before the application is deployed.
Understand the Different Security Scans
No single security scanner identifies every vulnerability. A practical setup combines several types of checks.
Security scan | What it checks | Example tools |
|---|
Static code analysis | Unsafe code patterns and data flows | CodeQL, Semgrep |
Dependency scanning | Known vulnerabilities in packages | Dependabot, pip-audit |
Secret scanning | Exposed API keys and credentials | GitHub Secret Scanning, Gitleaks |
Infrastructure scanning | Insecure cloud and deployment configurations | Checkov, Trivy |
AI-powered security scanning | Additional vulnerability patterns using AI | GitHub AI Scan |
Each tool examines a different part of the application.
For example, CodeQL can identify SQL injection risks in supported languages. A dependency scanner can identify a vulnerable library version. A secret scanner can detect an API key accidentally added by an AI assistant.
Using these checks together provides better coverage than relying on one tool.
Scan AI-Generated Code With GitHub CodeQL
CodeQL is GitHub's semantic code analysis engine. It examines source code and represents its behavior as a database that security queries can analyze.
It can identify vulnerabilities such as:
CodeQL supports multiple programming languages, including C#, Java, JavaScript, Python, Go, and C/C++.
For example, suppose an AI assistant generates the following Python function:
import sqlite3
def get_account(account_id):
connection = sqlite3.connect("app.db")
query = (
"SELECT * FROM accounts "
f"WHERE id = {account_id}"
)
result = connection.execute(query)
return result.fetchall()
The function directly inserts the account ID into the SQL query.
A static analysis tool can examine how the input reaches the database operation and identify a possible SQL injection vulnerability.
Configure CodeQL in GitHub Actions
Create a workflow named .github/workflows/codeql.yml.
name: CodeQL Security Analysis
on:
push:
branches:
- main
pull_request:
branches:
- main
schedule:
- cron: '30 2 * * 1'
permissions:
contents: read
jobs:
analyze:
name: Analyze Code
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language:
- python
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
- name: Perform CodeQL analysis
uses: github/codeql-action/analyze@v4
This example configures CodeQL to analyze Python code.
The workflow runs when code is pushed to the main branch, when a pull request targets main, and on a weekly schedule.
CodeQL produces security findings that appear in the repository's code scanning results.
For production repositories, pin third-party and GitHub Actions to reviewed commit SHAs and follow GitHub's recommended workflow configuration.
If your project uses C#, Java, or JavaScript, update the language configuration accordingly. Some compiled languages also require a build configuration.
Use GitHub AI Scan for Additional Vulnerability Detection
Traditional static analysis is useful, but it has limitations.
CodeQL uses security queries and data-flow analysis to identify known vulnerability patterns. However, not every language or framework receives the same level of coverage.
GitHub AI Scan adds AI-powered security detection to code scanning.
There are different AI Scan capabilities:
AI Scan for pull requests examines eligible pull request changes in selected languages and frameworks not covered by CodeQL.
AI Scan for repositories examines the default branch across supported languages and can identify additional vulnerabilities.
AI application scanning focuses on security risks in AI applications and agents.
For example, an organization might have a repository containing Python, PHP, Terraform, and Dockerfiles.
CodeQL provides analysis for supported languages, while AI Scan can extend coverage to additional languages and frameworks.
AI Scan findings appear with an AI label so developers can distinguish them from CodeQL alerts.
Understand its limitations
GitHub AI Scan is not a replacement for CodeQL or other security tools.
Its availability, billing, language coverage, and behavior depend on the particular AI Scan feature.
For example, repository-wide AI Scan is currently in public preview and runs on demand against the default branch. It does not automatically scan every commit.
AI Scan findings on pull requests are advisory and do not currently block merges through repository rulesets.
AI-generated findings can also contain false positives. Developers must review the reported issue before deciding whether to fix it.
Treat AI Scan as an additional source of security findings rather than the only security gate.
Check Dependencies Introduced by AI Tools
AI coding assistants sometimes recommend packages that are outdated, vulnerable, or unnecessary.
There is another problem: an AI model might suggest a package name that does not exist or is similar to a popular package.
This can create a dependency confusion or typosquatting risk if developers install packages without checking their source.
For Python projects, use pip-audit to check known vulnerabilities.
Install it:
pip install pip-audit
Scan the project dependencies:
pip-audit -r requirements.txt
For JavaScript projects, use:
npm audit
For GitHub repositories, Dependabot can also identify vulnerable dependencies and suggest updates.
Before accepting a new dependency generated by an AI tool, check:
Whether the package exists in the expected registry.
Whether the package name is correct.
Whether it is actively maintained.
Whether it has known security advisories.
Whether the application actually needs it.
Whether the selected version supports your runtime.
Do not automatically install every package suggested by an AI assistant.
Detect Secrets in AI-Generated Files
An AI assistant might accidentally place credentials inside a source file or configuration file.
For example:
DATABASE_URL = (
"postgresql://admin:[email protected]/app"
)
PAYMENT_API_KEY = "sk_example_secret"
These values should not be committed to a repository.
GitHub Secret Scanning can identify supported credential patterns in repositories. Push protection can also prevent supported secrets from being pushed.
You can use Gitleaks for an additional repository scan.
Install Gitleaks according to its official installation instructions and run:
gitleaks git .
This scans the Git repository for potential secrets.
If a real credential is detected, revoke or rotate it. Deleting the value from the latest version of the file does not remove it from Git history.
For applications, use environment variables or a dedicated secrets manager rather than storing credentials in source code.
Add Dependency and Secret Checks to the Pull Request Workflow
Code scanning should run automatically instead of depending on developers remembering to execute security tools.
For example, you can create a GitHub Actions workflow for Python dependency and secret checks.
name: Dependency and Secret Security
on:
pull_request:
branches:
- main
permissions:
contents: read
jobs:
dependency-check:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.14'
- name: Install pip-audit
run: |
python -m pip install --upgrade pip
python -m pip install pip-audit
- name: Scan dependencies
run: |
pip-audit -r requirements.txt
secret-check:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Scan for secrets
uses: gitleaks/gitleaks-action@v2
This example runs two independent jobs.
The dependency job checks the Python packages for known vulnerabilities.
The secret-check job scans the Git repository for potential exposed credentials.
Use versions and configuration approved by your organization. Pin external actions to reviewed commit SHAs for production workflows, and configure the appropriate permissions and licensing requirements.
You can combine these jobs with CodeQL and your existing test workflow.
For a stronger merge policy, configure required status checks and branch protection. Do not assume every security finding automatically blocks a pull request.
Review Security Findings Before Fixing Them
Automated scanners can identify possible vulnerabilities, but not every alert represents an exploitable issue.
For example, a scanner might report an unsafe function because user input reaches it. However, the application might validate and sanitize that input earlier in the execution path.
Review each finding carefully.
A useful review process includes:
Identify the affected file and line.
Understand the vulnerability category.
Follow the input data through the application.
Check whether authentication and authorization are enforced.
Verify whether the issue can be reproduced.
Apply a suitable fix.
Run the security scan again.
Add a regression test where appropriate.
Pay particular attention to authentication, authorization, database queries, file uploads, cryptographic operations, and external API integrations.
AI-generated fixes also need review. A proposed fix can introduce a different vulnerability or change application behavior.
Common Mistakes to Avoid
Depending on only one scanner
CodeQL, dependency scanners, and secret scanners identify different problems. Use multiple checks.
Ignoring new dependencies
An AI-generated package recommendation should be verified before installation.
Treating every alert as a confirmed vulnerability
Review the code path and determine whether the reported issue is actually exploitable.
Assuming AI Scan replaces traditional analysis
AI-powered scanning complements existing tools. It does not replace CodeQL, dependency auditing, or secret detection.
Allowing security findings to remain unresolved
Assign ownership to security findings, prioritize them based on risk, and track their resolution.
Treating a clean scan as proof of security
Automated tools cannot identify every business logic issue, authorization mistake, or application-specific vulnerability.
Summary
AI-generated code should go through the same security checks as code written by developers. CodeQL can identify vulnerable code patterns, GitHub AI Scan can provide additional AI-powered detection, dependency scanners can identify known package vulnerabilities, and secret scanning can catch exposed credentials. Run these checks automatically on pull requests and review findings before merging. No single scanner can guarantee secure code, so combine automated analysis with application testing and human review. This makes security a regular part of AI-assisted development rather than something addressed only after deployment.
Join the conversation! Your thoughts help the community grow.