India's Unified Payments Interface (UPI) is a marvel of modern fintech. Processing over 14 billion transactions a month, it operates on a mandate of zero downtime and sub-second latency.
At the fintech company where I work, we built an in-house, real-time machine learning system designed to detect fraudulent UPI transactions. This uses an ensemble of XGBoost and Graph Neural Networks (GNN) to analyze transaction context, device fingerprints, and user behavior, returning an approve/decline decision in under 50 milliseconds.
But in the world of high-scale ML, models don't fail in isolation; they fail in the messy intersection of data, infrastructure, and human behavior.
Here is the story of how a massive e-commerce flash sale broke our AI, how we diagnosed it in the middle of the night, and the lessons we learned.
The Incident: The 11:30 PM PagerDuty Alert
It was a Friday night in October. The "Mega Festive Sale" had just gone live across major Indian e-commerce platforms. UPI volumes were spiking 4x above our daily average.
At 11:32 PM, my phone buzzed. Then it buzzed again. Then it didn't stop.
Alert 1: CRITICAL: AI False Positive Rate (FPR) exceeded 5% threshold.
Alert 2: CRITICAL: Customer support ticket volume for "UPI Payment Failed" up 600%.
A False Positive Rate (FPR) of 5% in a payment system is catastrophic. It means that out of every 100 legitimate transactions, 5 are being wrongly flagged as fraud and declined. Customers were trying to buy TVs and smartphones, entering their UPI PIN, and getting a "Transaction Declined" message.
![48]()
Triage: Isolating the Blast Radius
I jumped on a war room call with the SRE (Site Reliability Engineering) and Data Engineering teams. Our first instinct was always to check the infrastructure.
Inference Servers: CPU and memory utilization were at 65%. Healthy.
Model Latency: P99 latency was 42ms. Well within our 50ms SLA.
Network: No packet drops between the API gateway and the ML inference cluster.
Conclusion: The AI system was running perfectly fast. It just wasn't making the right decisions. This wasn't an infrastructure failure; it was a logic or data failure. The model was highly confident in its wrong predictions.
The Diagnosis: Digging into the "Black Box"
To figure out why the model was declining legitimate transactions, we needed to look inside the black box. We pulled a sample of 500 recently declined transactions and ran them through our local explainability pipeline using SHAP (SHapley Additive exPlanations) values.
SHAP tells us exactly which features pushed the model's prediction toward "Fraud."
When we aggregated the SHAP values for the declined transactions, a massive red flag appeared. The top feature driving the "Fraud" prediction was user_30d_avg_txn_value (the user's average transaction value over the last 30 days).
The model was flagging these users because their user_30d_avg_txn_value was drastically lower than the current transaction amount.
I asked the data engineer, "What is the actual value of user_30d_avg_txn_value for these users in the database?"
He ran a quick query and went pale. "It's zero. For all 500 users, the feature value is exactly 0.0."
The "Aha!" Moment
Why would a user's 30-day average transaction value suddenly be zero?
We traced the data flow. This AI system doesn't calculate this on the fly; it fetches it in real-time from our Feature Store (a high-speed Redis cluster).
Because of the massive spike in UPI traffic during the flash sale, the Redis cluster was experiencing micro-latencies. To prevent the 50ms SLA from being breached, our engineering team had previously written a "fallback" mechanism in the feature-fetching microservice.
If the Feature Store took longer than 10ms to respond, the code would abort the fetch and return a default value to the model to keep latency low.
The fatal flaw: The default fallback value for a missing numerical feature was hardcoded as 0.0.
The Chain of Failure
Flash sale traffic spiked.
Feature Store (Redis) experienced slight latency.
The microservice timed out and returned 0.0 for user_30d_avg_txn_value.
A legitimate user tried to pay ₹45,000 for a phone.
The AI model saw: Current Txn = ₹45,000, but 30-day Average = ₹0.0.
The model logically concluded: "This user has never spent money before, and is suddenly trying to drain their account. This is an account takeover! BLOCK IT."
The Fix: Stopping the Bleeding
It was 12:15 AM. We had been bleeding false declines for 45 minutes. We needed to stop the bleeding immediately before implementing a permanent fix.
Immediate Mitigation (12:20 AM)
We couldn't fix the Feature Store latency in 5 minutes. Instead, we used our ML deployment platform to instantly route 100% of AI system traffic to our Fallback Rules Engine. This was a simple, hardcoded heuristic system (e.g., "Block if transaction > ₹1,00,000 and device is new").
Result: The False Positive Rate dropped from 5% back to 0.2% within 3 minutes. Customers could pay again.
Short-Term Fix (Next Morning)
We updated the feature-fetching microservice. We changed the fallback value from 0.0 to NULL. We then updated the XGBoost model's preprocessing pipeline to handle NULL values by imputing the global population average, rather than zero. We also horizontally scaled the Redis Feature Store cluster to handle the festive load.
Post-Mortem and Lessons Learned
The next day, we held a blameless post-mortem. The incident was a harsh but invaluable lesson in MLOps. Here are the core takeaways we implemented:
1. Never Default to Zero in Financial ML
In many domains, defaulting a missing numerical feature to 0 is fine. In financial fraud detection, 0 is a highly anomalous value. We established a strict engineering standard: Missing data must be imputed using statistical baselines (mean/median) or handled via native NULL support in the model, never defaulted to zero.
2. Feature Store SLAs are Model SLAs
We used to treat the Feature Store as just another database. This incident proved that the Feature Store is a core component of the ML model. If the features degrade, the model degrades. We implemented strict SLAs for the Feature Store and added circuit breakers that gracefully degrade the model's confidence score if feature latency spikes, rather than feeding it garbage data.
3. Explainability is for Debugging, Not Just Compliance
We originally implemented SHAP values to satisfy RBI (Reserve Bank of India) compliance requirements for explainable AI. During this incident, SHAP transitioned from a compliance tool to our primary debugging instrument. Without it, we would have spent days guessing why the model was failing.
4. Shadow Mode is a Lifesaver
For our next major model update, we implemented a "Shadow Mode" deployment. The new model runs in production, scoring real transactions, but its decisions are logged and ignored (the old model makes the actual decision). This allows us to monitor the False Positive Rate of the new model in real-time against live traffic without risking customer friction.
Conclusion
Building AI for UPI is thrilling. You are building systems that process the lifeblood of the Indian economy at a scale unmatched anywhere else in the world.
But as we learned at 11:30 PM on a Friday, an AI model is only as good as the data feeding it in real-time. Diagnosing the incident taught us that in production ML, the hardest bugs aren't usually in the math or the algorithms; they are in the plumbing. And in high-stakes systems like UPI, ensuring the plumbing is just as robust as the AI is the only way to keep the digital tollbooth open.