As organizations adopt AI coding assistants across multiple development teams, AI usage can become difficult to measure. Developers may use different Copilot models for different tasks, while repositories, prompts, context size, and agentic workflows can all affect resource consumption.
Simply counting how many developers use Copilot does not provide enough information to understand usage patterns.
A better approach is to track model usage at an appropriate level and analyze metrics such as model selection, token consumption, task type, team, repository, and time period.
This article explains how to design a practical token-usage tracking system for AI-assisted development and how teams can use the resulting data for capacity planning, governance, and cost analysis.
What Is Per-Model Token Usage?
Token usage represents the amount of text processed by an AI model.
At a simplified level:
Developer Request
↓
Input Tokens
↓
AI Model
↓
Output Tokens
↓
Generated Response
The total usage can be represented as:
Total Tokens
=
Input Tokens
+
Output Tokens
In agentic workflows, additional model calls may occur:
User Task
↓
Model Call 1
↓
Tool Call
↓
Model Call 2
↓
Tool Call
↓
Model Call 3
↓
Final Response
Therefore, a single developer request can involve multiple model interactions.
Actual usage and billing behavior depends on the Copilot plan, model, feature, and current GitHub policies. Organizations should use the usage information provided by their GitHub environment rather than attempting to infer exact billing from token counts alone.
Why Teams Should Track Model Usage
Consider an organization with several engineering teams:
Team A → Model X
Team B → Model Y
Team C → Model X
Team D → Multiple Models
Without usage data, it is difficult to answer questions such as:
Which models are most frequently used?
Which teams use AI assistance most heavily?
Which repositories generate the most activity?
Which workflows consume the most model context?
Are developers using expensive or resource-intensive models for simple tasks?
Has usage changed after introducing agentic workflows?
Usage analytics can provide these answers.
Usage Data Model
Start by defining a normalized internal model.
public sealed record ModelUsage(
string UserId,
string Team,
string Repository,
string Model,
long InputTokens,
long OutputTokens,
DateTimeOffset Timestamp)
{
public long TotalTokens =>
InputTokens + OutputTokens;
}
This model deliberately separates input and output tokens.
That distinction becomes useful when analyzing why usage is increasing.
For example, large repository context can increase input usage even when generated responses remain relatively small.
Aggregating Usage by Model
Once usage records are collected, LINQ can be used to aggregate them.
var usageByModel = usageRecords
.GroupBy(x => x.Model)
.Select(group => new
{
Model = group.Key,
Requests = group.Count(),
InputTokens = group.Sum(x => x.InputTokens),
OutputTokens = group.Sum(x => x.OutputTokens),
TotalTokens = group.Sum(x => x.TotalTokens)
})
.OrderByDescending(x => x.TotalTokens)
.ToList();
This provides a model-level summary.
For example:
Model Requests Total Tokens
-------------------------------------
Model A 4,200 18.4M
Model B 2,700 11.1M
Model C 1,900 6.8M
These numbers are illustrative only.
A production system should calculate them from actual usage data.
Aggregating Usage by Team
Model-level usage does not explain organizational behavior.
Add team-level analysis:
var usageByTeam = usageRecords
.GroupBy(x => x.Team)
.Select(group => new
{
Team = group.Key,
Developers = group
.Select(x => x.UserId)
.Distinct()
.Count(),
Requests = group.Count(),
TotalTokens = group.Sum(x => x.TotalTokens)
})
.OrderByDescending(x => x.TotalTokens)
.ToList();
This can reveal differences between teams.
For example:
Team Developers Requests
---------------------------------------
Platform 18 9,400
Payments 12 7,800
Internal Tools 7 2,100
However, raw totals should not be used to rank teams without considering team size and workload.
Normalize Usage Per Developer
A team with 30 developers will naturally produce more usage than a team with five developers.
Calculate a normalized metric:
var averageUsagePerDeveloper =
usageRecords
.GroupBy(x => x.Team)
.Select(group =>
{
var developers = group
.Select(x => x.UserId)
.Distinct()
.Count();
return new
{
Team = group.Key,
TotalTokens = group.Sum(x => x.TotalTokens),
TokensPerDeveloper =
developers == 0
? 0
: group.Sum(x => x.TotalTokens)
/ developers
};
})
.ToList();
This makes comparisons more meaningful.
Still, tokens per developer should be treated as an analytical metric rather than a productivity score.
High usage does not automatically mean high productivity.
Tracking Usage by Repository
Repository-level information can identify where AI assistance is concentrated.
var usageByRepository = usageRecords
.GroupBy(x => x.Repository)
.Select(group => new
{
Repository = group.Key,
Requests = group.Count(),
TotalTokens = group.Sum(x => x.TotalTokens)
})
.OrderByDescending(x => x.TotalTokens)
.ToList();
This can help identify repositories that may benefit from:
A high usage value should trigger investigation, not automatic criticism.
Tracking Usage Over Time
Time-based analysis helps identify trends.
var dailyUsage = usageRecords
.GroupBy(x => x.Timestamp.Date)
.Select(group => new
{
Date = group.Key,
Requests = group.Count(),
TotalTokens = group.Sum(x => x.TotalTokens)
})
.OrderBy(x => x.Date)
.ToList();
The resulting trend might show:
Date Total Tokens
--------------------------
Monday 420K
Tuesday 510K
Wednesday 480K
Thursday 690K
Friday 730K
A sudden increase should be investigated.
Possible causes include:
A new team adopting Copilot
Increased agentic usage
Larger repository context
A new development project
Changes in model usage
Repeated automated workflows
Comparing Models Fairly
Raw token usage should not be used to determine that one model is more efficient.
Consider:
Model A
Requests: 100
Tokens: 1M
Completed Tasks: 90
Model B
Requests: 100
Tokens: 600K
Completed Tasks: 60
Model B uses fewer tokens but completes fewer tasks.
A better analysis combines usage with task outcomes.
For example:
public sealed record ModelPerformance(
string Model,
long TotalTokens,
int CompletedTasks);
Then calculate:
public static double TokensPerCompletedTask(
ModelPerformance performance)
{
if (performance.CompletedTasks == 0)
{
return double.PositiveInfinity;
}
return (double)performance.TotalTokens
/ performance.CompletedTasks;
}
This still should not be interpreted as a universal efficiency benchmark.
The definition of "completed task" must be consistent.
Measuring Cost
Token counts and actual costs are related but not necessarily identical.
Depending on the product and plan, organizations may have:
Therefore, do not create a fake price calculation such as:
var cost = totalTokens * 0.00001;
unless that rate is explicitly defined for the exact service and usage category being analyzed.
Instead, separate usage from pricing:
public sealed record UsageSummary(
string Model,
long InputTokens,
long OutputTokens,
long TotalTokens,
decimal? ReportedCost);
If reliable cost information is available, store it separately.
Detecting Unusual Usage
Usage analytics can also help identify anomalies.
For example:
var threshold = 1_000_000L;
var highUsageUsers = usageRecords
.GroupBy(x => x.UserId)
.Select(group => new
{
UserId = group.Key,
TotalTokens = group.Sum(x => x.TotalTokens)
})
.Where(x => x.TotalTokens > threshold)
.ToList();
A threshold should not automatically indicate misuse.
A developer working on a large migration may legitimately produce substantially more AI activity than another developer.
Use anomaly detection as a signal for investigation rather than an automatic enforcement mechanism.
Privacy Considerations
Usage analytics can involve sensitive information.
Depending on the telemetry collected, records may contain:
User identifiers
Repository names
Model names
Prompts
Generated code
Tool information
Timestamps
Do not collect more information than necessary.
If the goal is simply model usage reporting, you may not need to store complete prompts or generated responses.
A minimal usage record might contain:
User/Team
Repository
Model
Input Tokens
Output Tokens
Timestamp
This is considerably safer than storing complete AI conversations.
Data Retention
Define a retention policy before collecting usage data.
For example:
Raw Usage Data
↓
Short Retention
↓
Aggregated Metrics
↓
Longer Retention
Aggregated information can often provide long-term trend analysis without retaining every detailed interaction.
The appropriate retention period depends on organizational security, privacy, compliance, and operational requirements.
Building a Usage Dashboard
A useful dashboard should provide multiple views.
Model Usage
Model A 52%
Model B 31%
Model C 17%
Team Usage
Platform 40%
Payments 30%
Applications 20%
Other 10%
Trend
Week 1 1.2M tokens
Week 2 1.5M tokens
Week 3 1.8M tokens
Week 4 2.1M tokens
Efficiency
Model
Requests
Tokens
Completed Tasks
A dashboard should provide enough context to explain changes rather than simply presenting a large number.
Common Mistakes
Treating Token Usage as Productivity
More AI usage does not automatically mean better developer productivity.
Comparing Teams by Raw Usage
Larger teams naturally produce more activity.
Normalize where appropriate.
Assuming Token Count Equals Cost
Actual cost depends on the applicable product and pricing model.
Storing Complete Conversations Unnecessarily
Usage analytics usually does not require storing every prompt and response.
Using Fixed Anomaly Thresholds Without Context
A large usage spike can have legitimate reasons.
Ignoring Model Mix
Total organizational usage can increase simply because developers switch to models or workflows with different usage characteristics.
Best Practices
Define the metrics before collecting data.
Track input and output usage separately.
Aggregate usage by model, team, repository, and time.
Normalize team comparisons where appropriate.
Keep usage and billing as separate concepts.
Minimize stored prompt and response data.
Apply appropriate retention policies.
Restrict access to individual-level usage information.
Investigate anomalies rather than automatically penalizing users.
Combine usage data with task outcomes.
Document how metrics are calculated.
Revisit dashboards as Copilot capabilities change.
Advantages and Disadvantages
Advantages
Provides visibility into AI adoption
Helps identify model usage patterns
Supports capacity planning
Helps organizations understand resource consumption
Enables trend analysis
Can highlight unusual usage
Supports governance discussions
Disadvantages
Usage data can be sensitive
Token counts do not directly represent productivity
Cost models can be complex
Large datasets require storage and processing
Individual usage metrics can be misinterpreted
Model and product changes can affect historical comparisons
Troubleshooting Usage Reports
If usage numbers appear incorrect:
Verify that all relevant usage sources are being collected.
Check timestamp and timezone handling.
Confirm that input and output tokens are not being counted twice.
Check whether retries generate additional records.
Verify model names are normalized.
Remove duplicate events.
Compare aggregated data with a known sample.
Check whether usage records represent requests or individual model calls.
Validate the reporting period.
Document any limitations in the data source.
For example, these two measurements are not necessarily equivalent:
100 User Requests
and:
100 Model Calls
An agentic workflow could produce several model calls for a single user request.
A Practical Data Pipeline
A production usage analytics system can follow:
Copilot Usage Data
↓
Collection
↓
Normalization
↓
Validation
↓
Aggregation
↓
Analytics
↓
Dashboard
The normalization layer is especially important.
Different data sources may represent models, users, repositories, and timestamps differently. Converting them into a common internal schema makes downstream reporting much easier.
Example Summary Model
A final dashboard can consume an aggregated model:
public sealed record ModelUsageSummary(
string Model,
int Requests,
long InputTokens,
long OutputTokens,
long TotalTokens,
int CompletedTasks);
This provides a clean separation between raw telemetry and reporting.
Conclusion
Tracking GitHub Copilot usage across teams is more useful when it goes beyond simply counting requests. Model selection, input and output token consumption, repositories, teams, task outcomes, and time-based trends provide a much clearer picture of how AI-assisted development is being used.
However, usage metrics should be interpreted carefully. Token consumption is a resource metric, not a direct measure of developer productivity or code quality.
A practical enterprise solution should collect only the information required, normalize it into a consistent schema, separate usage from actual billing, protect individual-level data, and combine consumption metrics with meaningful development outcomes.
With that foundation, organizations can understand how AI coding models are being adopted, identify changing usage patterns, and make better decisions about model selection and AI development governance without turning raw usage numbers into misleading productivity scores.