After upgrading from Go 1.2.1 to 1.3 (Windows 7 64 bit) "go build" execution time has increased from around 4 to over 45 seconds. There were no other changes except the go version update. Switching off the virus scanner seems to have no effect. Any clues?
You probably have dependencies that are being recompiled each time. Try go install -a mypackage to rebuild all dependencies.
Removing $GOPATH/pkg also helps to ensure you don't have old object files around.
Building with the -x flag will show you if the toolchain is finding incompatible versions.
I have the exact same problem, running this command solves it:
go get -u -v github.com/mattn/go-sqlite3
Another tip: http://kokizzu.blogspot.co.id/2016/06/solution-for-golang-slow-compile.html
Using go1.6,
Simply run go build -i.
It will compile all the dependencies and store them at $GOPATH/pkg/*/* as .a files.
Later when you run go run main.go, everything is much faster.
What s really great is that if you use vendored dependencies (IE: a vendor folder in your project), deps are built appropriately within $GOPATH/pkg/**/yourproject/vendor/**
So you don t have to go get install/get/whatever and have a mix of vendor / global dependencies.
I suspect you got to re-build .a files after deps update (glide update or smthg like this), but i did not test that yet.
After Go 1.10, you'd just need to type go build. You'd not need to type: go build -i.
From the draft Go 1.10 document, here.
Build & Install
The go build command now detects out-of-date packages purely based on the content of source files, specified build flags, and metadata stored in the compiled packages. Modification times are no longer consulted or relevant. The old advice to add -a to force a rebuild in cases where the modification times were misleading for one reason or another (for example, changes in build flags) is no longer necessary: builds now always detect when packages must be rebuilt. (If you observe otherwise, please file a bug.)
...
The go build command now maintains a cache of recently built packages, separate from the installed packages in $GOROOT/pkg or $GOPATH/pkg. The effect of the cache should be to speed builds that do not explicitly install packages or when switching between different copies of source code (for example, when changing back and forth between different branches in a version control system). The old advice to add the -i flag for speed, as in go build -i or go test -i, is no longer necessary: builds run just as fast without -i. For more details, see go help cache.
I just experienced the same problem - updating from 1.4 to 1.5. It seems that the olds versions are somehow incompatible or are being rebuild every time as go build -x shows. Executing go get -v invalidates all packages or refetches them, I am not quite sure and go build -x shows quite less output.
You can build sqlite3 like this:
cd ./vendor/github.com/mattn/go-sqlite3/
go install
After that your project will b built much faster.
If you try as all other said but still not work, I suggest removing the directory of $GOPATH such as :
sudo rm -rf $GOPATH
cd yourproject
go get -d
go get -u -v github.com/mattn/go-sqlite3
Related
i'm trying to install a library inside a package. However I don't understand where it compiles to.
Structure is like so:
package/cmd/library
I can install other executable targets fine with go install. My paths are set correctly. However now I want to build my shared library target and deploy it somewhere (this deployment step can be done manually). I'm running into two different issues.
Issue one, I can't seem to install it at all:
go install -buildmode=c-shared bpackage/cmd/library#latest
Returns with:
go install: no install location for directory /home/tpm/go/pkg/mod/package/cmd/library outside GOPATH
For more details see: 'go help gopath'
which tells me that it installs somewhere other than in my gopath, I'm just not sure where that might be.
Issue 2, using the -o flag doesn't work with go install, so I can't seem to alter the output location to place it inside the GOPATH (i did try setting the GOBIN to within my gopath, but since other commands work fine I don't think this should be causing any issue)
Quoting Ian from https://github.com/golang/go/issues/24253
Note that it doesn't make a great deal of sense to use go install -buildmode=c-shared. The expectation is that people will use go build -buildmode=c-shared -o foo.so. The only point of using -buildmode=c-shared is to use the shared library somewhere, and using go install without -o will put the shared library in a relatively unpredictable place.
I always build with -i flag to install packages and .a files are installed in GOPATH/pkg directory.
GOCACHE directory shown by go env GOCACHE seems to store cache files as well.
What's difference between them?
And what I wanna know is both of them should be saved if I want to make build time faster?
TLDR; The cache folder is internal to the go tool and its working should be opaque to the user, and its purpose is to speed up builds and tests. For example if you use a version control system (such as git), switching between branches or versions, the GOPATH/pkg may only contain package files of one version. The go cache folder may contain (partially) compiled packages of multiple branches and versions, speeding up future builds when you switch between branches and versions.
The cache folder was introduced in Go 1.10:
The go build command now maintains a cache of recently built packages, separate from the installed packages in $GOROOT/pkg or $GOPATH/pkg. The effect of the cache should be to speed builds that do not explicitly install packages or when switching between different copies of source code (for example, when changing back and forth between different branches in a version control system). The old advice to add the -i flag for speed, as in go build -i or go test -i, is no longer necessary: builds run just as fast without -i. For more details, see go help cache.
So you don't need to use -i anymore for fast builds.
Some quotes from the output of go help cache:
The go command caches build outputs for reuse in future builds.
The default location for cache data is a subdirectory named go-build
in the standard user cache directory for the current operating system.
Setting the GOCACHE environment variable overrides this default,
and running 'go env GOCACHE' prints the current cache directory.
The go command periodically deletes cached data that has not been
used recently. Running 'go clean -cache' deletes all cached data.
The build cache correctly accounts for changes to Go source files,
compilers, compiler options, and so on: cleaning the cache explicitly
should not be necessary in typical use. However, the build cache
does not detect changes to C libraries imported with cgo.
If you have made changes to the C libraries on your system, you
will need to clean the cache explicitly or else use the -a build flag
(see 'go help build') to force rebuilding of packages that
depend on the updated C libraries.
The go command also caches successful package test results.
See 'go help test' for details. Running 'go clean -testcache' removes
all cached test results (but not cached build results).
The cache folder is also used to store test results, so in some circumstances, the cached results may be presented without running the tests again.
Your questions is self-answering:
$ ls $(go env GOCACHE)
$ cat $(go env GOCACHE)/README
and
$ ls $(go env GOPATH)/pkg
As you can see - there is nothing similar between them:
GOPATH/pkg - compiled packages that remains static between builds. Those files are not "cashe" files actually
GOCACHE - collections of build artefacts that constantly changes between builds
More elaborate answer could be done by examining sources of go build
I'm running a project using Go Modules with 1.11.4 on Ubuntu, running in WSL.
My problem is that I'm having getting incremental builds to work as I expect. Perhaps this is due to me misunderstanding how it's supposed to, but I'd be glad if someone could clarify if that is the case.
Just as an example, if I do go build ./... then everything is built, as expected.
If I now do go build ./... again without any changes, my expectation was that due to the incremental builds, this time nothing would be built. But it builds everything again. I tried doing go build -i ./... (even though my understanding is that -i isn't needed anymore from 1.10), but the result is the same. This has been puzzling me for some time, as after reading the documentation I indeed expected the go build command to produce incremental builds.
The other day I realized that if instead I do go install ./... first, and then go install ./... again a second time, the second time around nothing is built, as I would expect. If I change a single module and run go install ./... again, then only that module is rebuilt, again what I would expect. So this gives me incremental builds.
So my questions are
1) Did I misunderstand go build ./... and how it handles incremental builds? Do I need to use go install instead?
2) Typically, we build the modules one by one, using the -o flag to specify an output path. Using go install instead, there is no -o option to specify an output path. Is there anything I can do to achieve something similar to -o using go install?
Thanks!
Till Go 1.4.2 when i run go install after running go build, i could find binary file in my current folder. Hence following Linux command was working
$ go build && go install && ./executable
But after installing go 1.5, when i run same command i get,
-bash: ./executable: No such file or directory
and when i checked, there is no executable to find. Did go install behavior changed in Go 1.5?
Yes, the behaviour has changed in Go 1.5:
If 'go install' (with no arguments, meaning the current directory) succeeds, remove the executable written by 'go build', if present. This avoids leaving a stale binary behind during a sequence like:
go build
<test, mostly works, make small change>
go install
Before this CL, the current directory still has the stale binary from 'go build'. If $PATH contains dot, running the name of the program will find this stale binary instead of the new, installed one.
I can't find anything mentioning that in the release notes though. Might be a documentation issue.
It seems like the solution is to use the binary that go install has produced.
EDIT: Here is the issue on the Go issue tracker if you want to follow on updates. Should be fixed by 1.5.1.
Every time I try to compile my program after (this morning) upgrading go from 1.1.1 to 1.1.2 (on Windows 7 64 bits), I get error message like:
C:\Users\VonC\prog\go\src\github.com\spf13\hugo>go build -o hugo.exe main.go
# github.com/spf13/hugo/hugolib
hugolib\page.go:23: import C:\Users\VonC\prog\go\pkg\windows_amd64/github.com/emicklei/hopwatch.a:
object is [windows amd64 go1.1.1 X:none]
expected [windows amd64 go1.1.2 X:none]
I tried a go clean -r, but the error message persists?
What is the right clean command to use?
Actual solution:
I reproduced the issue with:
calling the 1.1.2 go.exe directly (I didn't have %GOROOT%\bin in my path)
with GOROOT pointing to the previous 1.1.1 installation folder (I kept go 1.1.1 and 1.1.2 installed in separated folders).
If you are sticking with the default go setup (ie: one C:\go installation directory, and %GOROOT%\bin in your PATH), you won't see this error.
But if you do see this error:
make sure %GOROOT% is consistent with the go.exe you are calling
go install -a as explained below. The go clean mentioned below won't be necessary.
As jnml comments:
the Go build system is supposed to figure out any obsolete stuff in $GOPATH/pkg and (transitively) rebuild it on demand.
Original solution:
In the "Remove object files" section of "Command go" page, I missed the go clean -i option:
-i
The -i flag causes clean to remove the corresponding installed archive or binary (what 'go install' would create).
And those .a file (like hopwatch.a) are precisely what go install generates for libraries (in Windows).
So the full clean command, to make sure go rebuild everything, would be:
cd C:\Users\VonC\prog\go\src\github.com\spf13\hugo
go clean -r -i
go install -a
That will rebuild and install everything, including all dependent packages.
The -a is actually a build option, which forces rebuilding of packages that are already up-to-date.
As usual, go clean -r -n would show you what would be cleaned (-n: preview option).
It doesn't hurt to be sure of what will be deleted... before actually deleting anything.