What are some best practices for using Azure Application Insights in production environments?
What are some best practices for using Azure Application Insights in production environments?
1. Control cost & data volume (the #1 production concern)
• Enable adaptive sampling — the SDK automatically drops a % of telemetry under load while keeping metrics statistically accurate. On by default in ASP.NET Core; tune MaxTelemetryItemsPerSecond .
• Use fixed-rate sampling when you need consistent, predictable ingestion across multiple services (so correlated traces aren't half-sampled).
• Set a daily cap on the resource to prevent bill shock — but understand it drops data once hit, so alert before you reach it.
• Filter noise with ITelemetryProcessor — drop health-check pings, successful 200s on /ping , noisy dependencies you don't care about.
2. Connection strings, not instrumentation keys
• Always use the connection string, not the legacy InstrumentationKey (deprecated; keys don't carry the regional endpoint and break sovereign clouds).
• Never hardcode it — pull from Key Vault or App Configuration; inject via APPLICATIONINSIGHTS_CONNECTION_STRING . Better still, use managed identity where supported.
3. Correlation & distributed tracing
• Adopt the W3C Trace Context standard so traces flow across service boundaries, queues, and HTTP calls — critical in microservices.
• Set cloud_RoleName per service so the Application Map shows distinct components instead of one blob.
• Prefer the OpenTelemetry-based Azure Monitor distro for new apps — it's the strategic direction and vendor-neutral.
4. Structured, correlated logging
• Log through ILogger (it flows into App Insights as traces with operation correlation) rather than dumping raw strings.
• Use structured properties, not string interpolation: logger.LogInformation("Order {OrderId} shipped", id) — so you can query/filter by OrderId in KQL.
• Set the right log level in production (Information/Warning) — Debug/Trace floods ingestion and cost.
5. Don't log sensitive data (PII / secrets)
• Scrub PII, tokens, connection strings, and full request bodies with a telemetry processor/initializer before it leaves the process.
• Be careful with dependency telemetry capturing query strings or SQL with embedded params.
6. Custom telemetry that matters
• Track business KPIs with custom events/metrics ( TrackEvent , GetMetric ) — "OrdersPlaced", "PaymentFailed" — not just infra.
• Use GetMetric() (pre-aggregated) instead of TrackMetric() for high-frequency counters — far cheaper and won't be sampled away.
• Add custom dimensions via a TelemetryInitializer (tenant ID, region, app version) so every item is queryable and filterable.
7. Proactive monitoring, not just dashboards
• Availability tests (URL ping / standard tests) to catch outages before users report them.
• Smart Detection & metric alerts on failure rate, P95 latency, dependency failures, and exception spikes — route to Action Groups (Teams/PagerDuty/email).
• Live Metrics for real-time firefighting during deployments/incidents.
8. Retention, workspace & querying
• Use workspace-based App Insights (the classic standalone model is retired) — unifies logs/metrics in Log Analytics and enables cross-resource KQL.
• Tune retention (default 90 days) and export to storage or a Basic/Archive tier for cheap long-term/compliance data.
• Master KQL — requests , dependencies , exceptions , traces , joined on operation_Id — this is where real diagnosis happens.
9. Release health & deployment correlation
• Stamp telemetry with application version ( ai.application.ver ) so you can compare error rates before vs. after a deploy and spot a bad release fast.
• Annotate releases on charts so latency/error changes line up with deployments.
10. Performance & reliability of the SDK itself
• App Insights buffers and sends telemetry asynchronously — but on shutdown, Flush() and allow drain time (esp. Functions/short-lived processes) or you lose the last batch.
• Watch overhead — heavy custom logging in hot paths adds latency; sample and aggregate.
• Reuse a single TelemetryClient (it's thread-safe); don't new one up per request.