Skip to content

QA-CI-007 — Retry masks test failures ​

← Back to the rule catalog

Generated from the rule registry. Get the same detail from the CLI with mjolnir explain QA-CI-007.

FieldValue
Severitywarning
Confidencehigh
Tierextended
Measured FP rate0% (n=11)
Evidence levelE2
QA impactFlaky-test risk (FLAKY-RISK)
False-positive risk (author estimate)low
Autofix availableno
Languagesyaml
Frameworksgithub-actions, azure-pipelines
Detection strategyLEXICAL
Introduced inv0.1.0

Why this fails in production ​

Retrying tests until they pass hides flaky and intermittent failures — the green check no longer means the suite passed.

What gets flagged (real detector output) ​

Job `test` wraps a test command in an automatic retry action.

Example from this rule's own must-fire fixture: tests/fixtures/QA-CI-007/must-fire/retry-wrapper.yml

The fix ​

Remove the retry wrapper; investigate the underlying flakiness instead.

Confirmed NOT to fire on the corresponding clean pattern ​

Verified against tests/fixtures/QA-CI-007/must-not-fire/clean.yml — a legitimate, similar-looking pattern this rule correctly leaves alone.

Corpus-measured false-positive risk ​

Real occurrence counts from npm run corpus:audit against actively-maintained OSS repos — reproduce yourself, don't just trust this table (see the false-positive audit):

RepoOccurrences
apache-airflow1
keycloak-keycloak1
positive-fixtures10

Released under the MIT License.