Compiling all test binaries without running them

Update: compiling multiple go test binaries at once may be supported by the go toolchain itself, since https://go.dev/cl/466397 🤞 – this post still has some additional optimisations which may be useful for very large projects though…

See this gist. from this comment on GitHub which inspired this post.

Being able to compile all tests without running them is a really useful tool as a pre-commit or pre-push hook, especially when running those tests is slow. Pushing tests that fail is bad enough, tests that don’t even compile is something we should definitely avoid! Unfortunately, the obvious-sounding invocation go test -c -o /dev/null ./... doesn’t work, since the -o flag isn’t supported when targetting multiple packages (./...).

You can skip running the test using the -exec flag, and setting that to a no-op like /bin/true. For medium sized projects that will be enough to get your compilation feedback in a second or two. However if you want even faster “does this compile?” feedback, on very large projects, you can take this a step further by also skipping linking and vetting…

The Short Version

Put this script in your path somewhere, call it go-build-no-link

#!/usr/bin/env bash
set -euo pipefail

# Functions named skip/X are used in place of binaries called X.
skip/link() { touch "$1"; }
skip/vet() { true; }

TOOL="${1:-}"
# If the first arg to the tool is to get the full version, just let that happen.
[[ "${2:-}" == "-V=full" ]] && { "$@"; exit; }

# Grab the names of all functions prefixed skip/ in this file.
# shellcheck disable=SC2207
SKIP=($(declare -F | grep -E 'skip\/' | cut -d' ' -f3))
# Throw away the first arg (binary name) as we've now consumed it.
shift
TOOLNAME="$(basename "$TOOL")"

# Loop through the skip/... functions looking for a match.
for T in "${SKIP[@]}"; do
	F="skip/$TOOLNAME" # The corresponding function name.
	[[ "$F" = "$T" ]] || continue
	# We got a match, so call our shim function and exit.
	"$F" "$@"
	exit
done
# If we got this far, just run what was asked for.
"$TOOL" "$@"

Then run the following command to quickly check that all your tests are compiling without having to run them, link them, or vet them, and you’re done! Happy tests that compile 😀

go test -exec=true -toolexec=go-build-no-link ./...

I also added a convenience script to my path to run this, and a quick and dirty benchmarking script to compare the results with both a warm and a cold cache.

The Long Version

If you’re curious what is going on here, read on…

Skip running tests using the go test -exec flag

You can do this trivially using the go test -exec flag, by passing in a no-op binary that doesn’t run anything, e.g. /bin/true like this:

go test -exec /bin/true

The -exec flag runs the specified program rather than the test binary itself.

But you can go even further and also avoid linking the test binaries to save even more time. In fact this would go for general compilation checks as well…

Skip linking tests using the go test -toolexec flag

The -toolexec is used to run other programs used in the build, including the linker. To speed things up further, you can insert a shim executable here that just performs a no-op instead of calling the actual linker.

The first code block in this post (above) skips linking and vetting the code. That’s my rewrite of @howardjohn’s gist, but also skips vetting without using the -vet=off flag.

#!/bin/bash

# Usage: go test -exec=true -toolexec=go-compile-without-link -vet=off ./...
# Preferably as an alias like `alias go-test-compile='go test -exec=true -toolexec=go-compile-without-link -vet=off'`
# This will compile all tests, but not link them (which is the least cacheable part)

if [[ "${2}" == "-V=full" ]]; then
  "$@"
  exit 0
fi
case "$(basename ${1})" in
  link)
    # Output a dummy file
    touch "${3}"
    ;;
  # We could skip vet as well, but it can be done with -vet=off if desired
  *)
    "$@"
esac

Note: compiling with linking turned off is not a guarantee that code will compile with linking on. However, it’s still a fast and useful heuristic that will eliminate a large class of programmer errors.

I hope you enjoyed this post. Let me know where I’ve gone wrong (or right!) @fieldnotes_tech on twitter.