I'm running a project using Go Modules with 1.11.4 on Ubuntu, running in WSL.
My problem is that I'm having getting incremental builds to work as I expect. Perhaps this is due to me misunderstanding how it's supposed to, but I'd be glad if someone could clarify if that is the case.
Just as an example, if I do go build ./... then everything is built, as expected.
If I now do go build ./... again without any changes, my expectation was that due to the incremental builds, this time nothing would be built. But it builds everything again. I tried doing go build -i ./... (even though my understanding is that -i isn't needed anymore from 1.10), but the result is the same. This has been puzzling me for some time, as after reading the documentation I indeed expected the go build command to produce incremental builds.
The other day I realized that if instead I do go install ./... first, and then go install ./... again a second time, the second time around nothing is built, as I would expect. If I change a single module and run go install ./... again, then only that module is rebuilt, again what I would expect. So this gives me incremental builds.
So my questions are
1) Did I misunderstand go build ./... and how it handles incremental builds? Do I need to use go install instead?
2) Typically, we build the modules one by one, using the -o flag to specify an output path. Using go install instead, there is no -o option to specify an output path. Is there anything I can do to achieve something similar to -o using go install?
Thanks!
Related
Here's the scenario:
in package myPackage I make a breaking change. No worries, it's private, nobody cares
in project myProject I would like to use the new myPackage version.
running go get -u ./... in myProject will not update the package, since go will find errors (due to the breaking changes).
I can see how this is a good idea, but in this very case I would like go to force the update, break everything, so I can follow the compiler errors until my code is finally ready.
Is there any way to achieve this ?
The -d flag causes go get to download the requested packages but not build them. (The -d flag is planned to be the default behavior of go get starting with Go 1.18.)
go get -d -u ./... should perform the update you intend.
i'm trying to install a library inside a package. However I don't understand where it compiles to.
Structure is like so:
package/cmd/library
I can install other executable targets fine with go install. My paths are set correctly. However now I want to build my shared library target and deploy it somewhere (this deployment step can be done manually). I'm running into two different issues.
Issue one, I can't seem to install it at all:
go install -buildmode=c-shared bpackage/cmd/library#latest
Returns with:
go install: no install location for directory /home/tpm/go/pkg/mod/package/cmd/library outside GOPATH
For more details see: 'go help gopath'
which tells me that it installs somewhere other than in my gopath, I'm just not sure where that might be.
Issue 2, using the -o flag doesn't work with go install, so I can't seem to alter the output location to place it inside the GOPATH (i did try setting the GOBIN to within my gopath, but since other commands work fine I don't think this should be causing any issue)
Quoting Ian from https://github.com/golang/go/issues/24253
Note that it doesn't make a great deal of sense to use go install -buildmode=c-shared. The expectation is that people will use go build -buildmode=c-shared -o foo.so. The only point of using -buildmode=c-shared is to use the shared library somewhere, and using go install without -o will put the shared library in a relatively unpredictable place.
I am trying to install the dependencies for a Go project that uses Go modules. I have GO111MODULES=on in my environment.
No matter what command I run (go build, go get, go clean --modcache, ...) it fails with output something like:
sdgluck$ go build .
go: finding cloud.google.com/go v0.26.0
go: cloud.google.com/go#v0.26.0: unknown revision refs/tags/v0.26.0
go: error loading module requirements
The last line seems to appear in the output for every command, alongside some mention of cloud.google.com/go.
How can I fix this and just get Go to install the modules for this project?
As stated in a comment below, the proper command to clear the Go modules cache is go clean -modcache (just one dash).
If that still doesn't work then you can try manually clearing your Go mod folder.
For example, if your GOPATH was /Users/spongebob/go:
rm -rf /Users/spongebob/go/pkg/mod
go build and go run are very slow on a tiny program I have (cgo invocations in particular). I'd like go to cache the binary so that it only rebuilds when the source is newer. I would use a simple Makefile with a % rule, but the language designers claim that go's build support doesn't need Makefiles.
Is there another alternative I've overlooked? Does the go community prefer another build system, maybe hash-based instead, for caching and reusing build products?
go build and go install will soon (Go 1.10, Q1 2018) be much faster: see this thread and this draft document.
The go command now maintains a cache of built packages and other small metadata (CL 68116 and CL 75473). The cache defaults to the operating system-defined user cache directory but can be moved by setting $GOCACHE.
Run "go env GOCACHE" to see the current effective setting. Right now the go command never deletes anything from the cache. If the cache gets too big, run "go clean -cache" instead of deleting the directory. That command will preserve the cache's log.txt file. In a few weeks I'll ask people to post their log.txt files to a Github issue so that we can evaluate cache size management approaches.
The main effect of the build cache is that commands like "go test" and "go build" run fast and do incremental builds always, reusing past build steps as aggressively as possible.
You do not have to use "go test -i" or "go build -i" or "go install" just to get fast incremental builds. We will not have to teach new users those workarounds anymore. Everything will just be fast.
Note that go install won't installs dependencies of the named packages: see "What does go build build?".
I wrote a tool that happens to solve this as a side effect. go build alone will not check if the executable it's producing is already up to date. go install does, and if you tweak it to install to a location of your choice, then you'll get the desired result, similar to go build.
You can see the behaviour you describe by doing something like this:
$ go get -d github.com/anacrolix/missinggo/cmd/nop
$ time go run "$GOPATH"/src/github.com/anacrolix/missinggo/cmd/nop/*.go
real 0m0.176s
user 0m0.142s
sys 0m0.048s
That's on a warm run. go run will link on every invocation, just as go build would. Note that github.com/anacrolix/missinggo/cmd/nop is an program that does absolutely nothing.
Here's invoking the same package, using my tool, godo:
$ time godo github.com/anacrolix/missinggo/cmd/nop
real 0m0.073s
user 0m0.029s
sys 0m0.033s
For larger programs, the difference should be more pronounced.
So in summary, your standard tooling option is to use go install, or an alternative like godo.
After upgrading from Go 1.2.1 to 1.3 (Windows 7 64 bit) "go build" execution time has increased from around 4 to over 45 seconds. There were no other changes except the go version update. Switching off the virus scanner seems to have no effect. Any clues?
You probably have dependencies that are being recompiled each time. Try go install -a mypackage to rebuild all dependencies.
Removing $GOPATH/pkg also helps to ensure you don't have old object files around.
Building with the -x flag will show you if the toolchain is finding incompatible versions.
I have the exact same problem, running this command solves it:
go get -u -v github.com/mattn/go-sqlite3
Another tip: http://kokizzu.blogspot.co.id/2016/06/solution-for-golang-slow-compile.html
Using go1.6,
Simply run go build -i.
It will compile all the dependencies and store them at $GOPATH/pkg/*/* as .a files.
Later when you run go run main.go, everything is much faster.
What s really great is that if you use vendored dependencies (IE: a vendor folder in your project), deps are built appropriately within $GOPATH/pkg/**/yourproject/vendor/**
So you don t have to go get install/get/whatever and have a mix of vendor / global dependencies.
I suspect you got to re-build .a files after deps update (glide update or smthg like this), but i did not test that yet.
After Go 1.10, you'd just need to type go build. You'd not need to type: go build -i.
From the draft Go 1.10 document, here.
Build & Install
The go build command now detects out-of-date packages purely based on the content of source files, specified build flags, and metadata stored in the compiled packages. Modification times are no longer consulted or relevant. The old advice to add -a to force a rebuild in cases where the modification times were misleading for one reason or another (for example, changes in build flags) is no longer necessary: builds now always detect when packages must be rebuilt. (If you observe otherwise, please file a bug.)
...
The go build command now maintains a cache of recently built packages, separate from the installed packages in $GOROOT/pkg or $GOPATH/pkg. The effect of the cache should be to speed builds that do not explicitly install packages or when switching between different copies of source code (for example, when changing back and forth between different branches in a version control system). The old advice to add the -i flag for speed, as in go build -i or go test -i, is no longer necessary: builds run just as fast without -i. For more details, see go help cache.
I just experienced the same problem - updating from 1.4 to 1.5. It seems that the olds versions are somehow incompatible or are being rebuild every time as go build -x shows. Executing go get -v invalidates all packages or refetches them, I am not quite sure and go build -x shows quite less output.
You can build sqlite3 like this:
cd ./vendor/github.com/mattn/go-sqlite3/
go install
After that your project will b built much faster.
If you try as all other said but still not work, I suggest removing the directory of $GOPATH such as :
sudo rm -rf $GOPATH
cd yourproject
go get -d
go get -u -v github.com/mattn/go-sqlite3