Integrating Go and Bazel tests - go

Question

In my CI system, I have various go scripts that I run to analyze my Go code. For example, I have a script that validates if various main files can start a long running app successfully. For this I run the go script via go run startupvalidator -pkgs=pkg1,pkg2,pk3. I am interested in using Bazel to be able to utilize the cache for this since if pkg1 has not changed startupvalidator would be able to hit the cache for pkg1 and then run a fresh run for pkg2 and pkg3.
I thought about a couple different ways to do this but none of them feel correct. Is there a "best" way to accomplish this? Is this a reasonable use case for bazel?
I thought about creating a bash script where I run something like:
go run startupvalidator $1
With a BUILD.bazel file containing
sh_binary(
name = "startupvalidator-sh",
sources = [":startupvalidator.sh"],
deps = [
"//go/path/to/startupvalidator",
],
)
I also thought about placing a similar sh_test in the BUILD.bazel file for each pkg1, pkg2, and pkg3 so that I could run bazel run //go/pkg1:startupvalidator.
However, this doesn't actually work. Does anyone have feedback on how I should go about this? Any directions or pointers are appreciated.

Answer 1

To take advantage of the caching for test results, you need a *_test which you run with bazel test. Maybe the only part you're missing is that bazel run simply runs a binary (even if it's a test binary), while bazel test is looking for an up-to-date test result which means it use the cache?
You also need to split up the binary so changing the code in pkg2 doesn't affect the test action in pkg1. The action's key in the cache includes the contents of all its input files, the command being run, etc. I'm not sure if your startupvalidator has the various main functions compiled into it, or if it looks for the binaries at runtime. If it compiles them in, you'll need to build separate ones. If it's loading the files at runtime, put the files it looks for in data for your test rule so they're part of the inputs to the test action.
I'd do something like this in pkg1 (assuming it's loading files at runtime; if they're compiled in then you can just make separate go_test targets):
sh_test(
name = 'startupvalidator_test',
srcs = ['startupvalidator_test.sh'],
deps = ['#bazel_tools//tools/bash/runfiles'],
data = ['//go/path/to/startupvalidator', ':package_main'],
)
with a startupvalidator_test.sh which looks like:
# --- begin runfiles.bash initialization v2 ---
# Copy-pasted from the Bazel Bash runfiles library v2.
set -uo pipefail; f=bazel_tools/tools/bash/runfiles/runfiles.bash
source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \
source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \
source "$0.runfiles/$f" 2>/dev/null || \
source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \
source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \
{ echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e
exec $(rlocation workspace/go/path/to/startupvalidator) \
-main=$(rlocation workspace/pkg1/package_main)
# --- end runfiles.bash initialization v2 ---
I'm assuming that package_main is the thing loaded by startupvalidator. Bazel is set up to pass full paths to dependencies like that to other binaries, so I'm pretending that there's a new flag that takes the full path instead of just the package name. The shell script uses runfiles.bash to locate the various files.
If you want to deduplicate this between the packages, I would write a macro that uses a genrule to generate the shell script.