My Pull Request Was Green. My Tests Never Ran.

One of my stacked pull requests had a green checkmark next to it. GitHub said the branch was clean and ready. But when we checked which jobs had run, my test suite wasn’t on the list.

Here’s what gh pr checks returned for that PR:

SUCCESS Socket Security: Pull Request Alerts
SUCCESS Socket Security: Project Report

Two checks. Both green. Both from a GitHub App that has nothing to do with my test suite. My CI workflow runs three jobs, build + vet + test, golangci-lint, and govulncheck. None of them appear in that list, because none of them ran. The PR looked ready to merge without any CI test results for the change.

The trigger filter matches the wrong branch

The changes were stacked. PR #76 was built on top of PR #74’s branch rather than on main, because the second change depended on the first. That let work continue while the first PR was still open, but my CI configuration didn’t account for it.

My workflow file started like this:

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

That branches: filter under pull_request reads like it means “run on pull requests from these branches.” It doesn’t. It filters on the base branch, the one you’re merging into. My stacked PR targeted fix/path-posix-promote, not main, so the filter excluded it and GitHub skipped the whole workflow. Silently. No skipped-job placeholder, no warning, no row in the checks table saying a workflow decided to sit this one out. The checks table just doesn’t mention it.

The push trigger didn’t save me either, since that one was pinned to main.

The fix was deleting a single line:

   pull_request:
-    branches: [main]

That’s the entire change. The push trigger stays pinned to main so feature-branch pushes don’t produce a duplicate run, and pull requests now get checked no matter what they target.

“Clean” doesn’t mean what you want it to mean

The other misleading signal was mergeStateStatus, which came back as CLEAN.

CLEAN means nothing is blocking the merge. That’s it. That’s the whole definition. And “nothing is blocking” is trivially true when no checks exist at all. An empty rollup and a fully passing rollup produce the same word.

Both signals looked fine, and I still had no CI test results for the change. The green checkmark covered the two Socket Security checks. It told me nothing about the jobs I expected to run.

The merge rule needs to be specific: every expected CI job must succeed for the revision being merged. In this repo, that means build + vet + test, golangci-lint, and govulncheck. A passing lint job doesn’t help if the tests never started. Neither does a list of five green checks if they’re the wrong five.

Getting CI running

After the workflow fix merged, the feature change landed as a new PR based on main. That PR is #79, and gh pr checks on it returns:

SUCCESS govulncheck
SUCCESS golangci-lint
SUCCESS build + vet + test
SUCCESS Socket Security: Pull Request Alerts
SUCCESS Socket Security: Project Report

Five checks instead of two, including all three CI jobs. By then, both the workflow and the base branch had changed, so this wasn’t a test of the filter change in isolation. It did confirm that CI ran on the replacement PR.

There was a branch cleanup problem along the way. Deleting the lower branch after merging it was supposed to retarget the PR stacked on top. GitHub closed it instead. And a closed PR whose base branch no longer exists can’t be retargeted or reopened, so #76 is permanently closed with its commits stranded. The recovery was to rebase onto main with git rebase --onto and open #79 as a fresh PR.

So if you’re working in a stack, retarget the dependent PR at main before you merge and delete anything underneath it. Order matters, and the failure mode is not reversible.

The things worth keeping

Four specifics, in case you want to go check your own setup this afternoon:

  1. pull_request: branches: filters the base branch, not the head. If you stack PRs and that filter names only main, your stacked work runs no CI.
  2. A skipped workflow leaves no trace in the checks list. It isn’t a gray “skipped” row. It’s absent, which looks identical to a repo that simply has fewer checks.
  3. Check every expected CI job on the revision you’re merging. A green merge status or the right number of rows isn’t enough. The expected jobs need to be present and successful.
  4. Deleting a base branch closes the PRs stacked on it, and that close is not recoverable through the UI. Retarget first.

That filter had been in place for a while. Every PR targeting main ran CI, so it looked fine until we started stacking changes. The workflow file hadn’t changed. How we used branches had.

Run gh pr checks <PR-number> on a recent PR and look for the jobs you expect. Check their names and results. I want to know that my tests passed before a merge, and that starts with making sure they ran.

Sources

I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].

DevOps Github Ci