RCA Library
Pool exhaustion on payments-api
SEV2Service: payments-api · 2025-11-23
- Root cause: Unbounded ORM pool + traffic burst
- Detection: Alert + agent correlation with Cloud SQL metrics
- TTD: 4m · TTR: 22m
- Fix: Cap pool + HPA tune; canary stable
Edge gateway 5xx spike
SEV2Service: edge-gateway · 2025-11-14
- Root cause: Bad upstream subset after deploy
- Detection: Synthetic + log anomaly
- TTD: 2m · TTR: 15m
- Fix: Shifted traffic + rollback
BQ cost overrun
SEV3Service: analytics · 2025-11-01
- Root cause: Unbounded export job
- Detection: Budget alert + query audit
- TTD: 12m · TTR: 40m
- Fix: Throttle extractor + limit bytes scanned