Key takeaways
- A pass rate measures the tests you have, not the requirements you committed to.
- Traceability — each test linked to the requirement it verifies — turns “how many passed” into “what is covered”.
- Coverage of committed stories, open defects by severity and untested changes are better release signals than one percentage.
- A quality gate is only useful if its decision is recorded: who made it, when, and on what evidence.
- Importing automation results keeps the evidence current without a spreadsheet rebuilt before every release.
What a pass rate hides
A suite can pass 98% of its tests while three of the stories committed to this release have no tests at all — because nobody wrote them, or because the tests that exist still check last quarter’s behaviour. The number looks healthy because it only counts what exists.
Trace tests to requirements
Linking every test to the requirement it verifies changes the questions you can answer before a release:
- Which committed stories have no approved test?
- Which requirements changed after their tests were approved?
- Which failing tests block which stories — and are any of those stories in this release?
Signals worth putting in front of the decision
| Signal | What it tells you |
|---|---|
| Coverage of committed stories | Whether everything promised for this release has at least one approved test |
| Latest results, per story | Whether what is covered actually works on this build |
| Open defects by severity, linked to stories | What is known to be broken, and what it affects |
| Changes since the last test run | What went into the build without being tested again |
| Gate decision and approver | Who said go, when, and on what evidence |
Make the gate a recorded decision
A quality gate held as a meeting leaves no trace. A gate recorded as a decision — the approver, the time, the evidence they saw and any exceptions they accepted — is something an auditor, a regulator or the next release manager can read.
It also makes exceptions visible. Shipping with a known defect is sometimes the right call, but it should be a decision someone owns, not something nobody noticed.
Keep it current without a spreadsheet
- Import results from CI rather than copying them, so the evidence is never older than the last build.
- Protect approved tests, so a change creates a new version instead of quietly rewriting what was signed off.
- Draft tests as stories arrive, from their acceptance criteria, so coverage keeps pace with scope.