Module dependency caching issue during build - go

Question

Recently swapped to using Go 1.11 release and am trying to convert our projects over to using the new module system. However, I'm running into a frustrating issue with the caching system (I've thus far run with GOCACHE=off because of unrelated issues in the past, but that's not an option with modules).
The below command log is based on using a fresh upgrade of my system Go to 1.11 using Homebrew (I typically use gvm to install and manage Go versions, but swapped to system build for this to see if gvm was the issue). I set my GOPATH to a temporary directory and moved the source code out of the default GOPATH (it is currently at ~/gotemp/statsbot).
For this command log, I'm nuking the cache first, to get it in a fresh state. I'm then doing a build (to pull in any missing dependencies, which should also recache deps) followed by a test on one of the subdirectories.
[08:34:48] statsbot (feature/bot-rebuild-gomod) $ rm -rf on
[08:34:49] statsbot (feature/bot-rebuild-gomod) $ go clean --cache
[08:34:50] statsbot (feature/bot-rebuild-gomod) $ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="on"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/var/folders/ck/bf8_n8hn34j7bh7hqqppxrtm0000gn/T/tmp.3ga5Iq09"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/kaedys/gotemp/statsbot/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ck/bf8_n8hn34j7bh7hqqppxrtm0000gn/T/go-build065167780=/tmp/go-build -gno-record-gcc-switches -fno-common"
[08:34:53] statsbot (feature/bot-rebuild-gomod) $ go build .
[08:35:01] statsbot (feature/bot-rebuild-gomod) $ ginkgo stats
Failed to compile stats:
# golang.org/x/text/runes
/var/folders/ck/bf8_n8hn34j7bh7hqqppxrtm0000gn/T/tmp.3ga5Iq09/pkg/mod/golang.org/x/text#v0.3.0/runes/cond.go:10:2: can't open import: "golang.org/x/text/transform": open on/51/51b9b2cf5a27a7ff06942814ad6ef031de64dc6d773c835916067578ea1ae3a1-d: no such file or directory
# golang.org/x/text/encoding
/var/folders/ck/bf8_n8hn34j7bh7hqqppxrtm0000gn/T/tmp.3ga5Iq09/pkg/mod/golang.org/x/text#v0.3.0/encoding/encoding.go:19:2: can't open import: "golang.org/x/text/encoding/internal/identifier": open on/4c/4c2fad64312f6ec34b38755ccecb96bb31c6995211f39e0eb4112abc2fac560c-d: no such file or directory
# gopkg.in/fsnotify.v1
/var/folders/ck/bf8_n8hn34j7bh7hqqppxrtm0000gn/T/tmp.3ga5Iq09/pkg/mod/gopkg.in/fsnotify.v1#v1.4.7/kqueue.go:18:2: can't open import: "golang.org/x/sys/unix": open on/77/77790588f9aed90a52efdb1dac069953e1ea2bf82db8627a47a9492862b39c6e-d: no such file or directory
Ginkgo ran 1 suite in 4.123499564s
Test Suite Failed
I see this same error message in a number of build situations. go build . will usually succeed, but sometimes will fail with similar messages. Running go clean --cache invariable fixes these issue, but I don't really want to run that before every build, and it doesn't work when trying to use Ginkgo to test since that seems to involve several successive compilations.
The error itself is what's baffling, because these files do exist:
[08:40:49] statsbot (feature/bot-rebuild-gomod) $ ll on/4c
total 112
-rw-r--r-- 1 kaedys staff 18346 Sep 14 08:40:19 2018 4c1d5db302dff0381c402160d7fd2bd89a6dafd3edc61b3dbd5e8f1a85108fb7-d
-rw-r--r-- 1 kaedys staff 27980 Sep 14 08:40:45 2018 4c2fad64312f6ec34b38755ccecb96bb31c6995211f39e0eb4112abc2fac560c-d
-rw-r--r-- 1 kaedys staff 175 Sep 14 08:40:20 2018 4cbbc40c46eba0ef41f3ebd23470c3327215fa6e05e269c276c6635da79f2e8e-a
-rw-r--r-- 1 kaedys staff 13 Sep 14 08:40:20 2018 4cffcbd4a7f1255ca3696f5e24e0b1045d14c25f1f90cbbb077b5d6f2d11e68c-d
The second file in that ll is the file the second error message complained was missing, so I can only assume that either it was trying to access the file while the file was still being generated, or it was looking in a different directory (which is difficult to tell, since the error message only reports a relative path for the file).
Posting here to see if anyone knows how to solve this or if it's something funky in my environment setup before creating an official issue on the Go github.

Answer 1

Your GOCACHE environment variable is set to "on". Value of GOCACHE must be a folder, and the special "off" value means to not cache build and test results and to not use already cached data of those. There is no special value "on".
You don't have to set GOCACHE, as it has a default value. Quoting from Command Go: Build and test caching:
The go command caches build outputs for reuse in future builds. The default location for cache data is a subdirectory named go-build in the standard user cache directory for the current operating system. Setting the GOCACHE environment variable overrides this default, and running 'go env GOCACHE' prints the current cache directory. You can set the variable to 'off' to disable the cache.