In this post, I’ll talk about how I used the Git bisect tool to find the specific commit that caused my test to fail.
Let’s say I create a simple project with an entry point that says, “Welcome to this app!” I then write a test asserting that the entry page says, “Welcome to this app!” My app is running and my one test is passing. Great!
I want to add a few things, so I commit a bunch of little modifications to my main page.
After a while, I run the test suite, and I see that my one test is no longer passing. My test was passing initially but, somewhere along the way, it stopped passing. I want to find out exactly when my test started failing.
Examining My Commit History
I know the test was passing in my initial commit but is not passing now.
My commit history looks like this:
commit 0fc90a0876ca253f01624304b274b7243a
add banana pie text
❌ test not passing
commit fb62b20dd48c0e668fb12d02b04bf09290
add banana bread text
❓ test maybe passing
commit 3c0aeb0cc03910c9c51c46638e2c72fc50
remove welcome text and add other text
❓ test maybe passing
commit e05af554acb68ad52e4bd9a7711f92edf0
modify text after header text
❓ test maybe passing
commit a5b79ec15c653676053292420d4ed29b9c
add text under the welcome text 2
❓ test maybe passing
commit 1322afcc38c06945d0501ec81c9ec791ed
add text under the welcome text
❓ test maybe passing
commit 4b10e5bef83965e977e02ff2a88c3b58ab
building out main page
✅ test passing
I know my test was passing at the building out main page commit and but it’s failing now. I want to find the commit where the test went from passing to failing. Assuming that checking CI is not an option, I could start with the building out main page commit and individually check out each commit to identify where the stopped passing. That would work okay for my little example project with a few commits but would quickly become untenable if this was a large project with many contributors.
Using Git Bisect to Find the Commit That Caused the Failure
Instead, I’ll reach for a handy tool: git bisect. I reach for git bisect whenever I want to identify the exact commit that introduced something. I often use it to identify which commit caused degradation in performance or introduced a bug. However, this tool could also easily be used to identify a commit responsible for speeding up the performance or fixing a styling issue.
In this example, I want to identify which commit caused my test asserting that I would see “Welcome to this app!” on the home page to start failing. I know my test was passing on the building out main page (4b10e5bef83965e977e02ff2a88c3b58ab) commit but is now failing at the most recent commit add banana pie text(0fc90a0876ca253f01624304b274b7243a). I need to identify where between those two commits the test went from passing to failing. Luckily, the “good” commit (where the test was passing) and the “bad” commit (where the test was failing) are the only things I need to start using git bisect.
I’ll start a git bisect with:
git bisect start Next, I’ll provide the “good” commit and the “bad” commit:
git bisect bad # current commit is bad
git bisect good 4b10e5bef83965e977e02ff2a88c3b58ab # "building out main page" commit is good
Then, git bisect will find the commit between the “good” commit and the “bad” commit and check that commit out. Then, it’s up to me to decide if this newly checked-out commit is a “good” commit or a “bad” commit. In this example, the commit is “good” if my one test passes and “bad” if the test fails. I run the main page test and it passes, so I’ll mark the currently checked out commit as “good”:
git bisect good
I’ve allowed git bisect to narrow the possible commits down, but I’m not quite finished yet. There is still a small range of commits between the “good” commit and the “bad” commit. Again, git bisect will pick a commit in the middle and I’ll have to determine the status of the commit by running my test. This time, the test fails, so I’ll mark the currently checked out commit as “bad”:
git bisect bad
The bisect has narrowed down the commits enough that it is able to print out the offending commit:
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[3c0aeb0cc03910c9c51c46638e2c72fc50a958c0] remove welcome text and add other text
I now can see that my “remove welcome text and add other text” commit broke my test. That makes a lot of sense!
An Even Faster Way to Use Git Bisect
What if I had way more commits and I didn’t want to have to individually go through and specify if a commit is “good” or “bad”?
Instead, I could provide the git bisect with a program or script that determines the status of a given commit. If the script exits successfully for the commit, the commit will be marked as “good.” Otherwise, on failure, the commit will be marked as “bad.” The script enables the git bisect to determine where the status went from “good” to “bad.”
In my case, the status is determined by the success of my test, so the script I provide to git bisect would be running the test:
git bisect yarn test
The result?
commit 3c0aeb0cc03910c9c51c46638e2c72fc50a958c0
remove welcome text and add other text
The script definitely made the process a little faster. What an easy way to find where I broke the test!
Now that I know where the problem is, it’s time to fix it.