← Log
TILDraft2025-06-02
TIL: alert fatigue is a design problem
A short note on why deleting alerts often improves reliability more than tuning thresholds.
- Topics
- SRE
- Alerting
- SLOs
- Tags
- til
- alerting
- slos
- reliability
The symptom
A team had 120 active alert rules and a 40% false-positive rate. They asked for help tuning thresholds. The real fix was removing alerts that did not map to a user-impacting symptom.
The rule
Every alert must answer: what symptom is this protecting, who is impacted, and what is the expected first action? If any answer is missing, the alert is a candidate for deletion.