Some useful Git commands for CI

Posted 18 September 2017
Tagged with builds-and-ci, git

I spend a lot of time writing build scripts for interacting with Git repos. But Git’s documentation is notoriously opaque, and a lot of my Git knowledge comes from word-of-mouth rather than reading the docs. In that spirit of sharing, these are few of the Git commands I find useful when writing build scripts.

$ git rev-parse --show-toplevel
/Users/alexwlchan/repos/alexwlchan.net

This command gives you an absolute path to the top-level of your repository.

Often scripts expect to be run from the root of a repo, so assume that $(pwd) gives you that path – but that’s not always true. Using this command always gives you the top of your repo.

Some examples of how I use this:

To share an entire repo into a Docker container (I’m on the way to running everything build-related inside Docker)
To refer to specific files in a repo within build scripts
To open a repo in my text editor or Git client, but from anywhere in the tree

$ git rev-parse --show-prefix
_posts/2017/

While reading the git-rev-parse docs for this post, I stumbled upon this similar command. This tells you where you are, relative to the top-level of the repo.

I don’t have a use for this yet, but I feel like it’s bound to come in handy.

$ git diff --exit-code
$ echo $?
0

$ echo "Hello world" >> README.md
$ git diff --exit-code
diff --git a/README.md b/README.md
index a562e4a..a20d083 100644
--- a/README.md
+++ b/README.md
@@ -27,3 +27,4 @@ $ make serve

 A local copy of the site will be served on <http://localhost:5757>.
 If you make changes to the source files, this version will automatically update.
+Hello world

$ echo $?
1

Running git diff will show you any unstaged changes. Adding --exit-code will cause it to have a non-zero exit code when you have a non-empty diff.

I use this to check that automated tasks have run correctly:

Some of my repos use code formatters like scalafmt and pyformat. To check that the formatters have run correctly on a patch, I have CI tasks that run the autoformat, then use this Git command to check there aren’t any changes – that is, that the autoformat was a no-op.
On repos with automatic code generation where the autogen is expensive, I sometimes check in the autogen artefacts. Ensuring they’re committed correctly is a similar process: run the autogen in CI, then check it didn’t change anything – that all the autogenerated files are already checked in.

This is a pattern I originally read in the Hypothesis Makefile.

(If you just want an exit code and no printed diff, add --quiet.)

$ git diff --name-only
Gemfile
Gemfile.lock
Makefile
README.md

$ git diff --name-only e7791a0 82eb01c
_plugins/static_file_generator.rb
install_jekyll.sh

Adding the --name-only flag to git diff gives you a list of files with changes.

In larger projects, it can be useful to skip some of the tests/builds if there aren’t any changes to relevant files. Output from this command can be input in a decision whether to run certain parts of the build.

A few examples:

In pull requests on the Wellcome Platform repo, we skip running tests on pull requests if there aren’t relevant changes compared to current master. This reduces the build time on PRs, and allows us to merge faster. In a similar vein, builds on master always run the full set of tests, but we only deploy if the patch contains interesting changes.
At my last job, people wanted to get notifications for changes to certain files – for example, if you were responsible for the licensing code, get an email every time there was a change to the licensing files. I don’t remember whether we finished it, but I remember starting to build it based on output from this command.
In Hypothesis, we do a deployment and release for every PR with changes to the core package – and it’s this command which tells us whether we need a release.

On its own, this command isn’t very useful – it’s when you wrap your own logic around it that it becomes really powerful.

$ git merge-base --is-ancestor master my-branch; echo $?
0

$ git merge-base --is-ancestor other-branch my-branch; echo $?
1

This final command can tell you if one commit is the ancestor (directly precedes) another commit. It exits with code 0 for yes, code 1 for no.

The example is based on the following branch structure:

A chain of five commits on my-branch (red), with a new branch (other-branch, green) forking at the third commit (master, blue). — There are three commits up to *master* (marked in blue). There are then additional two commits atop master to *my-branch* (red), and three commits in a different branch from master, named *other-branch* (green).

Most of the time, I’m comparing against master – more specifically, the shared master that’s been pushed to GitHub. Checking that shared master is an ancestor of my current branch is a good way to ensure I don’t inadvertently blat changes in master.

Let’s look at an example.

The same chain of five commits, but now master (blue) is ahead of the point of divergence. — Now *master* has advanced another two commits (blue), while *other-branch* (green) hasn't moved.

Consider the two commits in master but not in other-branch. Suppose they contain important changes, and we’ve deployed them to a staging server. If we do a deployment from other-branch, we’ll deploy our changes and revert the already-deployed changes from master. This opens the door for mistakes and confusion – so we can use --is-ancestor to check we’re only building on top of current master, not some earlier version of it.

In this case, a deployment from other-branch would be rejected until we’d rebased against master.

I use all of these commands on a regular basis, but I bet there are others I’m missing. What Git commands do you find indispensable? What do you wish more people knew about? I’d love to know what you find useful – write your own blog post, or tell me on Twitter.