Git bisect: the bug hunter!

Git bisect is a command, available since Git 2.6.0. It makes use of a binary search algorithm to find which commit in history introduced a specific change (bug or improvement).
This command helps with quick browsing of the commits history, and progressively reduces its search perimeter. This approach allows us to identify the changeset causing a problem or a side-effect, even when we have no idea where to look.

Which use cases?

Git bisect is a tool, not a silver bullet. To make the most of its potential without losing precious time, one needs to know its typical use cases. Thus, the command is more or less relevant for certain contexts.

It is particularly efficient when:

• The bug or change to look for is clearly identified, systematically reproducible via a fixed scenario.
• You are not sure where and when the impacting change took place (if it's a side-effect of a transverse component, or an unexpected performance improvement, for example).
• There are a lot of commits to browse and you have absolutely no idea what to look for (needle in a haystack: the case of an unexplicable bug running on your nerves).
• You have the possibility to test all commits (or most of them) atomically. Which means, each commit is stable enough to allow a replaying of the test scenario.
• The test scenario is quite fast. Things will be way smoother and nicer if we are testing things using a quick process which only takes a small amount of time, instead of, say 20 minutes of build at every git bisect step, which adds a huge risk of getting discouraged.

How does it work?

To better understand how the tool operates, here are the main steps when using git bisect for a bug hunt:

1. Start by giving git bisect a "bad" commit, known because the bug is present in this state and you can reproduce it.
2. Then, give it a "good" commit far enough in the past, known because the bug is absent in this state and you don't reproduce it.
3. Git bisect will automatically check you out to the commit which is right in the middle of the interval delimited by your two "good" and "bad" commits.
4. You should now test this commit by replaying your test scenario (build, launch, feature test... it's up to you) and tell git bisect whether this commit is "good" (with git bisect good) or "bad" (with git bisect bad).
5. Git bisect checks you out to a new commit, based on the info you gave it so far. vous déplace alors vers un nouveau commit, basé sur les informations fournies jusqu'ici. More specifically, it goes to the middle of the interval between your oldest bad commit and your newest good commit.
6. Test this commit like the previous ones, qualify it with "good" or "bad", etc.
7. This iterative search process ends on its own once git bisect is done browsing your history, and tells you which commit introduced the bug.

As you will have understood, git bisect has no inherent knowledge of your problem and needs you to be rigorous while testing and tagging commits, so it can provide accurate results.

If at some point of the binary search, you cannot assert whether a commit is "good" or "bad" (for example, if it was a work in progress, not yet stabilized or with build failing) you can use git bisect skip to go to the next step. git bisect skip will ignore this commit, but at the end of the process, you will be provided with a list of more than one potentially faulty commit, including the ones you ignored during the search.

Example

Let's illustrate with a real life case.

You want to find which commit broke your amazing dependency injection system, which exists since release 6.0.1 of your frontend project.

The problem is: you have no idea where to start because a lot of changes have been made all over the codebase since this release, and you don't know where the problem originated. At all.

You will start git bisect as follows, after checking out to the current commit where you encounter the issue:

git bisect start
git bisect bad  # Current commit is bad
git bisect good v6.0.1  # The tag of v6.0.1 is functional

Once this first step is out of the way, git bisect will select the first commit you will have to test, based on your input, and will bring you (= check you out) to it. It will then display something like:

Bisecting: 675 revisions left to test after this (roughly 10 steps)

It's time to re-compile (and maybe restart) this version of the code to test the scenario again.

If this version works as expected, type:

git bisect good

If this version is broken (bug is present), type:

git bisect bad

Which brings us to a new display, such as:

Bisecting: 337 revisions left to test after this (roughly 9 steps)

Keep repeating this process: compile your code, test it, and launch the appropriate command to tag each commit that git bisect will present you.

After a few iterations, there will be no more commits to test and inspect, and the command will display both reference and description of the first bad commit it has identied. Victory!

In your Git history, the reference refs/bisect/bad will be pointing on that commit, and visible in the Git tree as long as you don't execute git bisect reset.

Alternate terms

Sometimes, we are not after a bug but an unexpected improvement. In such cases, the "good" and "bad" terms can be quite confusing (we are looking for a positive change, so the semantics are reversed).

For the sake of clarity, we can use the old and new terms instead of good and bad.

Example : You start git bisect with a "new" commit containing the coveted improvement, and an "old" commit which doesn't. This can be a performance improvement, for example, if you don't know what triggered it.

For each commit containing this improvement, tag them as "new". Tag the others as "old".

At the end of the search, git bisect will indicate the first "new" commit containing the improvement.

You cannot mix good/bad and old/new in one git bisect search. To change them, you will have to use git bisect reset or start a new search from scratch.

You can also use your own terms for a git bisect search, like so:

git bisect start --term-old <term-old> --term-new <term-new>

For example, for our performance improvement:

git bisect start --term-new fast --term-old slow

These changes, though being pure syntactic sugar, can ease up the search process.

Commands for advanced use

git bisect (visualize|view)
Displays a list of the remaining suspects, using git log.

git bisect log
Displays a list of all commits tagged as good or bad so far.

git bisect replay
Replays a saved git bisect search from an external log file.

If you made mistake while tagging a commit, you can save your search log in an external file, edit it, then use these commands to get back on track:

git bisect reset
git bisect replay my-file

git bisect run {cmd}
If you happen to have a script which can assert whether the current source code is good or bad, you can start an automated binary search using this command:

git bisect run my_script arguments

Your script must return a 0 exit code if the source code is good, and any code between 1 and 127 (included, except for 125) if the source code is bad.

The 125 special code must only be used when the source code cannot be tested, and will trigger a skip.

If any other code is returned, the search will be aborted.

Automation potential

The git bisect run command can be used in automation for continuous integration, in order to automatically find faulty commit(s) when a build fails.

For instance, if we have a simple script capable of building our app and returning an appropriate exit code depending on the result, we can automate git bisect on an interval of commits:

(master) $git bisect start HEAD <sha1>
Bisecting: xxx revisions left to test after this (roughly x steps)
[A commit SHA1] A commit message

((bisect/bad~512)|BISECTING) $ git bisect run test.sh

running ../test.sh
Bisecting: xxx revisions left to test after this (roughly x steps)
[Another commit SHA1] Another commit message

...

running ../test.sh

And finally, the result:

Bisecting: 1 revision left to test after this (roughly 1 step)
[465194af92951519c7da6542eaca0c56ee09fcd9] I have no idea what I'm doing

465194af92951519c7da6542eaca0c56ee09fcd9 is the first bad commit
commit 465194af92951519c7da6542eaca0c56ee09fcd9
Author: Jean-Michel Apeuprès <jean-mi.approximately@jean-mi-is-the-best.fr>
Date:   Sat Feb 8 16:39:47 2014 +0100
   I have no idea what I'm doing

{A detailed list of changes in the changeset}

bisect run success

So, this is the end of this article ! Git bisect is a very interesting command with a high potential for automation. When used in a relevant context, it can be much more efficient and rigorous than a manual search.

For more info, feel free to read the Git official documentation page.