What Is Application Insights?
Azure Application Insights is a monitoring and diagnostics service that is part of the Azure Monitor platform. Think of it as a flight recorder for your Copilot Studio agent — it silently captures everything that happens during user conversations so you can examine it later.
Without it, you only see high-level stats inside Copilot Studio. With it connected, you see every message, every topic triggered, every API call, and every error — in detail.
Key Capabilities
Real-time telemetry via Live Metrics
Error and exception tracking with full detail
Performance monitoring — latency and API durations
Custom event logging from within topic flows
KQL querying — powerful ad-hoc data exploration
Smart Detection — automated anomaly alerts
Copilot Studio Dashboard — pre-built visual workbook
Why Connect It to Copilot Studio?
The built-in Analytics tab shows high-level engagement metrics but cannot tell you which conversation failed, which API returned an error, or what triggered a content filter. Application Insights fills every one of those gaps.
| Built-in Analytics | Application Insights |
|---|---|
| Session counts only | Full conversation transcripts |
| Topic engagement % | Per-message telemetry with timestamps |
| Escalation rate | Exception details and stack traces |
| No querying capability | Full KQL query engine |
| No custom events | Custom event logging from topic actions |
| Limited data retention | Configurable retention up to 730 days |
Prerequisites
Before getting started, ensure you have:
A Copilot Studio environment with an active custom agent
An active Azure subscription with permission to create resources
Owner or Contributor role on an Azure Resource Group
Access to the Azure Portal:
portal.azure.comYour agent must be published at least once before telemetry flows
Step-by-Step Setup Guide
Step 1: Create an Application Insights Resource
In the Azure Portal, search Application Insights and click Create.
Fill in:
Subscription
Resource Group
Name
Region
Set Resource Mode to Workspace-based.
Click Review + Create.
Step 2: Copy the Connection String
Open the new resource and navigate to the Overview page.
Find the Connection String field and click the copy icon.
Do not use the old Instrumentation Key — it is deprecated.
Step 3: Open Agent Settings
In Copilot Studio:
Open your agent.
Click Settings (top-right corner).
Select Advanced from the left navigation panel.
Step 4: Paste the Connection String
Under the Application Insights section:
Paste the Connection String.
Configure the logging options (see Section 5).
Click Save.
Step 5: Publish the Agent
Application Insights changes only take effect after publishing.
Click Publish.
Confirm the publish operation.
Both test-pane and live-channel conversations will now be captured.
Step 6: Verify the Connection
Navigate to:
Azure Portal → Application Insights Resource → Monitoring → Metrics
Select:
Metric: Users
Aggregation: Unique
After a few test conversations, you should see at least one unique user appear.
Logging Configuration Options
After pasting the Connection String, configure the following settings in the Advanced settings page.
| Toggle | What It Does & Recommendation |
|---|---|
| Log Conversation Transcripts | Captures full message text. Turn ON for debugging; turn OFF for privacy-sensitive deployments. |
| Log Sensitive Activity Properties | Includes personal data from events. Leave OFF by default and enable only during specific debugging sessions. |
| Log Generative AI Responses | Captures full AI-generated answers. Useful when troubleshooting content filter issues or answer quality problems. |
| Enable Enhanced Transcripts | Adds rich metadata such as topic names and action results. Highly recommended for production monitoring. |
Privacy Note: Always review your organization's data handling policies before enabling transcript logging. The Log Sensitive Activity Properties setting is OFF by default and should remain off unless required for a specific debugging scenario.
Understanding Telemetry Tables
Application Insights stores data across several tables. Each table serves a different purpose.
| Table | Contains & When to Use |
|---|---|
| customEvents | Primary table for conversation start/end events, topic triggers, message events, and AI responses. Start investigations here. |
| customDimensions | JSON metadata inside customEvents containing ConversationId, TopicName, DesignMode, ContentFiltered status, and more. |
| exceptions | Unhandled runtime errors and exceptions. Use when conversations fail unexpectedly. |
| traces | Free-text log messages and custom diagnostics generated from topic actions. |
| requests | HTTP requests made by the agent, including Power Automate flows and HTTP actions. |
| dependencies | External service calls used to analyze latency and integration performance. |
Finding Error Logs
The Failures Tab
Navigate to:
Application Insights → Investigate → Failures
This view provides:
Failed requests and exceptions over time
Most common error types
Individual error drill-downs
Full exception details
Use this as your first stop when users report failures.
The Logs (KQL) Editor
Navigate to:
Application Insights → Monitoring → Logs
This opens the KQL editor where you can run custom queries against all telemetry data.
Live Metrics
Navigate to:
Application Insights → Investigate → Live Metrics
This view provides:
Real-time requests
Response times
Failures
Live telemetry with sub-second latency
Keep this open while testing your agent for immediate feedback.
Essential KQL Queries for Error Investigation
KQL (Kusto Query Language) reads naturally from left to right, with each operation passing data to the next.
View All Production Events (Exclude Test Chats)
customEvents
| extend isDesignMode = customDimensions['DesignMode']
| where isDesignMode == 'False'
| project timestamp, name, customDimensions, session_Id, user_Id
| order by timestamp desc
Find All Errors and Exceptions
exceptions
| where timestamp > ago(7d)
| project timestamp, type, outerMessage, details, session_Id
| order by timestamp desc
Investigate a Specific Conversation by ID
customEvents
| where customDimensions contains 'YOUR-CONVERSATION-ID-HERE'
| project timestamp, name, customDimensions
| order by timestamp asc
Find Responsible AI Content Filter Errors
customEvents
| where customDimensions contains 'ContentFiltered'
| project timestamp, name, customDimensions, session_Id, user_Id
| order by timestamp desc
Track Daily Active Users Over 14 Days
let queryStartDate = ago(14d);
let queryEndDate = now();
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| summarize uc = dcount(user_Id) by bin(timestamp, 1d)
| render timechart
Most Frequently Triggered Topics
customEvents
| extend topicName = tostring(customDimensions['TopicName'])
| where isnotempty(topicName)
| summarize count() by topicName
| order by count_ desc
| take 20
The Copilot Studio Dashboard Workbook
Application Insights includes a pre-built Copilot Studio Dashboard workbook.
Accessing the Dashboard
Navigate to Application Insights → Monitoring → Workbooks
Open Copilot Studio Dashboard
Review session volumes, topic success rates, and error metrics
Customizing the Dashboard
Click Edit
Add, remove, resize, or reposition tiles
Add custom KQL visualizations
Save your changes
Sharing the Dashboard
Use the Share button to provide access to teammates.
Users require at least the Reader role on the Application Insights resource.
Pro Tips
Pin important tiles to an Azure Dashboard
Reader access is sufficient; Copilot Studio access is not required
Export workbook data to Power BI for organization-wide reporting
Setting Up Alerts
Proactive alerting helps identify issues before users report them.
Navigate to:
Application Insights → Monitoring → Alerts → New Alert Rule
| Alert Scenario | What to Configure |
|---|---|
| Error rate spike | Exception count exceeds a threshold within a 5-minute window |
| High latency | Average response time exceeds 3 seconds |
| Content filter rate | ContentFiltered events exceed a defined threshold |
| Zero traffic | Session count drops to zero during expected operating hours |
After configuring conditions and thresholds:
Create an Action Group.
Configure notifications:
Email
Microsoft Teams
Webhook
Assign the Action Group to the alert.
Tips for Effective Monitoring
Filter DesignMode in Every Query
Events generated from the Copilot Studio test pane have:
DesignMode = True
Always exclude them from production analysis.
Use:
| where customDimensions['DesignMode'] == 'False'
Use Connection String, Not Instrumentation Key
Microsoft has deprecated Instrumentation Keys.
Always configure Application Insights using the Connection String.
Always Publish After Configuration Changes
Telemetry settings do not become active until the agent is republished.
Use Conversation ID as Your Primary Debug Key
When troubleshooting:
Identify the conversation timestamp.
Find the Conversation ID.
Run Query 8.3 to reconstruct the entire conversation flow.
Configure Data Retention Appropriately
Default retention:
90 days
Maximum retention:
730 days
Review organizational compliance requirements before increasing retention, especially when transcript logging is enabled.
Troubleshooting Common Issues
| Problem | What to Check |
|---|---|
| No data appearing in Application Insights | Verify the agent was published after adding the Connection String. Wait 5–10 minutes after the first interaction and confirm the Connection String was copied correctly. |
| DesignMode shows NULL in queries | Use tolower(tostring(customDimensions['DesignMode'])) == 'false' to handle different event formats. |
| ContentFiltered errors appearing unexpectedly | Review the transcript and identify the specific message or prompt that triggered the filter. |
| Exceptions table is empty but agent errors exist | Search customEvents and filter for names containing "Error" or "Exception". |
| Live Metrics showing nothing | Keep the page open while actively sending messages. Live Metrics only displays recent activity. |
Quick Reference Cheat Sheet
| Task | Where to Go / What to Use |
|---|---|
| Connect Application Insights | Copilot Studio → Settings → Advanced → Application Insights |
| View real-time activity | Application Insights → Investigate → Live Metrics |
| See all errors | Application Insights → Investigate → Failures |
| Write custom queries | Application Insights → Monitoring → Logs |
| View the visual dashboard | Application Insights → Monitoring → Workbooks → Copilot Studio Dashboard |
| Set up alerts | Application Insights → Monitoring → Alerts → New Alert Rule |
| Configure data retention | Application Insights → Usage and Estimated Costs → Data Retention |
| Filter test conversations | ` |
| Find errors for a conversation | customEvents | where customDimensions contains 'CONV-ID' |
| Find content filter events | customEvents | where customDimensions contains 'ContentFiltered' |
Summary
Application Insights transforms a Copilot Studio agent from a black box into a fully observable system. The setup process is straightforward: create an Application Insights resource, copy the Connection String, configure it in Copilot Studio, and publish the agent. Once connected, you gain access to detailed telemetry, conversation diagnostics, exception tracking, KQL-based investigations, dashboards, and proactive alerts. For effective monitoring, always filter out DesignMode test traffic, use Conversation IDs for troubleshooting, and configure alerts so issues are detected before users report them.
Join the conversation! Your thoughts help the community grow.