Field Notes
Quarantine, fix, or delete: deciding the fate of a flaky suite
Flaky suites earn political cover because deleting tests feels reckless. Leaving them red-or-ignored is often riskier: they train teams to dismiss signals.
Prefer quarantine with an expiry
Move the suite out of the blocking job with a named owner and a review date no later than two sprints out. Quarantine without expiry becomes a graveyard.
Fix when the signal is real
If the suite catches the defect class you care about—but only sometimes—invest in isolation, test data, and waits that match the product’s real timing. Do not “fix” by loosening assertions until anything passes.
Delete when the criterion moved
Many flakes guard flows the product no longer ships, or duplicate a faster API check. Deletion is a strategy decision, not a moral failing. Record what criterion the suite used to serve so nobody resurrects it from nostalgia.