Regression suite rescue
Cut the nightly regression run from 6 hours to 11 minutes and flake to under 1%.
- Role
- QA lead
- Stack
- PlaywrightTypeScriptGitHub ActionsDocker
- 6h → 11m
- Suite runtime
- 40% → 0.8%
- Flake rate
- -64%
- Escaped defects / quarter
The state of things
The suite ran overnight, failed most mornings, and every failure was triaged as "probably flaky." It had stopped being a signal and become a ritual.
What actually fixed it
Not better waits. The fixes that mattered were structural:
- Kill shared state. Every test seeded its own data through the API and tore it down. Test independence bought parallelism for free.
- Quarantine, don't delete. Flaky tests moved to a non-blocking lane with an owner and a deadline, so the main gate went green and stayed meaningful.
- Fail loudly and specifically. Custom matchers so a failure names the business rule it broke, not the selector it could not find.
The lesson
The runtime win came from parallelism, but parallelism was impossible until state was isolated. Almost every "slow suite" problem is a state problem.