Skip to content

QA-CI-010 — Tests skipped where they must block ​

← Back to the rule catalog

Generated from the rule registry. Get the same detail from the CLI with mjolnir explain QA-CI-010.

FieldValue
Severityerror
Confidencemedium
Tierquarantine
Measured FP rate10% (n=10)
Evidence levelE2
QA impactFalse-green risk (FALSE-GREEN)
False-positive risk (author estimate)medium
Autofix availableno
Languagesyaml
Frameworksgithub-actions
Detection strategyFRAMEWORK (regex heuristic on parsed workflow AST)
Introduced inv0.4.0

Why this fails in production ​

A job or step gated behind a condition that can evaluate to "skip" on the exact runs that matter (e.g. if: github.event_name == 'push' on a workflow that's supposed to gate pull requests, or a path filter that excludes the files that actually changed) means the test suite that's supposed to block a bad merge never actually runs on that merge. GitHub reports a skipped required check as "neutral," which several branch-protection configurations treat as passing — the PR merges having never been tested at all.

What gets flagged (real detector output) ​

Job `test` runs tests but its `if:` condition skips it on pull requests.

Example from this rule's own must-fire fixture: tests/fixtures/QA-CI-010/must-fire/skip-on-pr.yml

The fix ​

Remove the skip-on-PR condition, or split into a dedicated PR test job and require THAT one in branch protection.

Confirmed NOT to fire on the corresponding clean pattern ​

Verified against tests/fixtures/QA-CI-010/must-not-fire/clean.yml — a legitimate, similar-looking pattern this rule correctly leaves alone.

Corpus-measured false-positive risk ​

Real occurrence counts from npm run corpus:audit against actively-maintained OSS repos — reproduce yourself, don't just trust this table (see the false-positive audit):

RepoOccurrences
microsoft-playwright-mcp1
positive-fixtures9
reflex-dev-reflex1
streamlit-streamlit1

Released under the MIT License.