For example, when I run this
GOOS="windows" GOARCH="386" go install ./something
The executable would be created at
$GOROOT/bin/$GOOS_$GOARCH/something.exe
I want the executable to be created at
path/to/my/module/bin/$GOOS_$GOARCH/something.exe
EDIT
To clarify, I ask the question in the context of cross compiled builds inside a module.
I get the desired result using this command inside my module. I'd like to avoid specifying the path.
GOOS="windows" GOARCH="386" go build -o ./bin/$GOOS_$GOARCH/something.exe ./something
So, is it possible to get the same with go install?
Setting GOBIN results in an error
GOBIN=$(pwd)/bin GOOS="windows" GOARCH="386" go install ./something
# go install: cannot install cross-compiled binaries when GOBIN is set
Don't use go install for cross compiling. Use go build with -o flag.
Here is some history regarding GOBIN, robpike posted an issue stating "I have $GOBIN set but the go tool is ignoring it when installing..."
This article mentions
"When cross compiling, you should use go build, not go install. This is the one of the few cases where go build is preferable to go install...
The reason for this is go install always caches compiled packages, .a files, into the pkg/ directory that matches the root of the source code...
This logic also holds true for the standard library, which lives in /usr/local/go/src, so will be compiled to /usr/local/go/pkg/$GOOS_$GOARCH. This is a problem, because when cross compiling the go tool needs to rebuild the standard library for your target, but the binary distribution expects that /usr/local/go is not writeable.
Using go build rather that go install is the solution here, because go build builds, then throws away most of the result (rather than caching it for later), leaving you with the final binary in the current directory, which is most likely writeable by you"
Related
i'm trying to install a library inside a package. However I don't understand where it compiles to.
Structure is like so:
package/cmd/library
I can install other executable targets fine with go install. My paths are set correctly. However now I want to build my shared library target and deploy it somewhere (this deployment step can be done manually). I'm running into two different issues.
Issue one, I can't seem to install it at all:
go install -buildmode=c-shared bpackage/cmd/library#latest
Returns with:
go install: no install location for directory /home/tpm/go/pkg/mod/package/cmd/library outside GOPATH
For more details see: 'go help gopath'
which tells me that it installs somewhere other than in my gopath, I'm just not sure where that might be.
Issue 2, using the -o flag doesn't work with go install, so I can't seem to alter the output location to place it inside the GOPATH (i did try setting the GOBIN to within my gopath, but since other commands work fine I don't think this should be causing any issue)
Quoting Ian from https://github.com/golang/go/issues/24253
Note that it doesn't make a great deal of sense to use go install -buildmode=c-shared. The expectation is that people will use go build -buildmode=c-shared -o foo.so. The only point of using -buildmode=c-shared is to use the shared library somewhere, and using go install without -o will put the shared library in a relatively unpredictable place.
I'm using Go modules in my project and in my build system (e.g. Travis CI) I'm downloading a command-line utility (written in Go) with go get to assist with my build process, e.g.:
go get github.com/mitchellh/gox
However, this go get causes the file to be added to my go.mod file. This is contaminating the build environment, causing it to be "dirty" (since there are changes to some files tracked in git, in this case go.mod and go.sum), and I use git describe --always --dirty --tag to describe my build, which shows up as "dirty".
Is there a way to "go get" a binary just to download it, without adding it to the go.mod/go.sum?
I've tried setting GOPATH to somewhere else, even then, go get updates the go.mod/go.sum to add this as an // indirect dependency.
dir="$(mktemp -d)"; \
env GOPATH="$dir" go get github.com/mitchellh/gox && \
mv "$dir/bin/gox" "$(go env GOPATH)"/bin/gox
Go 1.16 onwards
Go 1.16 (released February 2021) includes a change that makes it possible to install a binary without affecting go.mod.
Issue 40276 tracks the proposal:
cmd/go: 'go install' should install executables in module mode outside a module
This was implemented in CL 254365. As part of this change, you can run e.g.:
go install golang.org/x/tools/cmd/goimports#latest
to install a binary without affecting go.mod.
To install a specific version, replace #latest with e.g. #v0.1.5.
Hopefully in Go 1.14 there will be a new flag for go get that does exactly what you are asking. This is tracked in issue #30515 "cmd/go: offer a consistent global install command".
Prior to that, you have a few different options.
Go 1.12 and 1.13: change directory
If you are using Go 1.12 or later, the simplest solution is probably to move outside your current module to a directory without a go.mod prior to doing the go get, such as:
$ cd /tmp
$ go get github.com/foo/bar#v1.2.3
$ cd - # return to prior directory
Go 1.11, 1.12, 1.13+: gobin
gobin is a module-aware command to install or run binaries that provides additional flexibility, including the ability to install without altering your current module's go.mod. See the gobin README and FAQ for more details.
Go 1.11: temporary module
If you are using Go 1.11 with modules, the first step is probably to upgrade to Go 1.12 or 1.13 given there are many improvements in modules. If you are required to use Go 1.11 and want to use the #version syntax without updating your current module's go.mod, then one approach is to create a temporary module:
cd $(mktemp -d) && go mod init tempmod && go get github.com/foo/bar#v1.2.3
This is because in Go 1.11, you can't use the #version syntax unless you are in a module, which was relaxed in Go 1.12. This approach has been automated by a simple shell script by #rogpeppe.
Additional Details
In general, the go command in module-module always determines what module it is "in", which is based on the current working directory when you invoke the go command. (You could make an analogy to how make without any args will look for a makefile in the current working directory, or how historically go build without any args will build the current working directory, etc.).
With modules, go get looks for a go.mod file in the current working directory or any of its parents, and go get will use the constraints listed in any go.mod as part of solving for versions, as well as update the go.mod if needed based on doing the go get. That is why your go.mod file is updated if you run go get from within an existing module.
On the other hand, starting with Go 1.12, if you are in a directory that is not part of any module (that is, the directory does not have a go.mod, nor do any of its parents), then there is no go.mod to update, but the go command is still able to operate in module mode and use the #version syntax.
From the Go 1.12 release notes:
When GO111MODULE is set to on, the go command now supports module-aware operations outside of a module directory, provided that those operations do not need to resolve import paths relative to the current directory or explicitly edit the go.mod file. Commands such as go get, go list, and go mod download behave as if in a module with initially-empty requirements. In this mode, go env GOMOD reports the system's null device (/dev/null or NUL).
Per go help build:
The -mod build flag provides additional control over updating and use of go.mod.
If invoked with -mod=readonly, the go command is disallowed from the implicit automatic updating of go.mod
I have a repository with a group of nested go packages organized as follows:
$GOPATH/src/
- mypackage/common/utils.go
- mypackage/app1/main.go
- mypackage/app2/main.go
...
It compiles to a handful of binaries. For releasing, I'm cross-compiling for a multitude of platforms/archs (I deploy repeatedly with a different GOOS and GOARCH). I'm trying to write the compilation results to a directory of my choice, but I'm fighting with the toolchain.
I can:
Combine GOBIN and go install when compiling for my own architecture
(i.e. not cross compiling):
# build + output all binaries in /somedir/bin
# this works great when compiling for my local architecture.
GOBIN=/somedir/bin go install mypackage/app1 mypackage/app2 mypackage/app3
But unfortunately, GOBIN conflicts with cross-compilation e.g:
# Throws: "cannot install cross-compiled binaries when GOBIN is set"
GOBIN=/somedir/bin GOARCH=amdm64 GOOS=darwin go install mypackage/app1 mypackage/app2
Use go build with GOOS=X and GOARCH=Y for each subpackage
# cross compile one of the binaries
cd /output/darwin-386-bin/ && \
GOOS=darwin GOARCH=386 go build mypackage/app1
# but if given multiple binaries, there is no output. (as documented)
GOOS=darwin GOARCH=386 go build mypackage/app1 mypackage/app2
When go build is given multiple packages, it no longer emits binaries -- it just checks that the code compiles. To emit all the binaries, it seems I have to run go build once for each subpackage, so it takes longer, esp when building with -a.
Another possible issue with using go build like this, is that it's potentially mixing binaries and intermediary results across multiple architectures in the same workspace. But maybe that's just a matter of taste. Hopefully the toolchain keeps cached results separate for different architectures.
How is this handled in practice? Should I treat each subpackage as an individual package, despite their shared common code (would it then be safe to build them all in parallel)?
Related articles:
go build vs go install: https://groups.google.com/forum/#!topic/golang-nuts/s8Csz3-7EXA (Little info on cross-compilation, or multipackage setups)
cross compile with go 1.5.: https://dave.cheney.net/2015/08/22/cross-compilation-with-go-1-5 (does not specifically address multipackage / common code compilation. Also, the statement about having to recompile the standard library into /usr/local/go/pkg seems to no longer hold in 1.9.1, as far as I can tell)
Go build multiple nested packages: Go build multiple/nested packages? . ( The accepted answer, go build ./... behaves the same as passing multiple packages in a single build command, and therefore outputs no binaries. The thread also does not cover cross compilation)
What I'm about to suggest feels like a hack at best, but it might work for you if you're cross-compiling stuff in a container or an isolated build environment.
You can drop the GOBIN from the install command:
# I'm not on this platform, so this is a cross compile
GOOS=darwin GOARCH=amd64 go install mypackage/app1 mypackage/app2
Assuming your package is in $GOPATH/src/mypackage the above command will install the two binaries to:
$GOPATH/bin/darwin_amd64/{app1, app2}
and the .a compile dependency files in:
$GOPATH/pkg/darwin_amd64/mypackage/{app1,app2,common}
If you're running this in a for loop for all the platforms you plan on supporting, one nuisance with this process is that when you pass GOOS=x GOARCH=y matching your local architecture (i.e. not cross compiling), then the executables will be placed directly in $GOPATH/bin/ and not $GOPATH/bin/x_y/.
(packages on the other hand always end up in $GOPATH/pkg/x_y/, cross compilation or not).
To determine your local architecture, you can follow steps in this answer: https://stackoverflow.com/a/35669816/5556676 .
Simulating GOBIN=/foo/bin GOOS=x GOARCH=y go install mypackage/app{1,2,3}
Changing $GOPATH has little effect on where go install writes output. With some go commands, like go get, you tweak where packages are installed by adding a new component at the front of $GOPATH. But in go 1.9, go install will always copy binaries in the bin/ folder that's a sibling of the src/ which contains the packages to install.
Instead of using GOBIN=/foo/bin, you can pretend your source files are in /foo/src/mypackage (you may use a symlink), and then do GOPATH=/foo GOOS=x GOARCH=y go install mypackage/app{1,2,3}. This will put binaries in /foo/bin, because of the behavior I describe in the previous paragraph.
It's probably simpler to just grab the binaries from where you expect them to be though, than mucking with copying source trees around.
go build allow to precise binary name with the -o command line option.
go install doesn't, but it deploy the binary and cache package compilation.
How to either change the binary name with go install? or cache compilation with go build?
A simple answer would be "use go install with correct package name", but please, where is the option I am missing?
You can use the -i flag to go build:
The -i flag installs the packages that are dependencies of the target.
go build -i -o binary packagename
When you type go help install you get
usage: go install [build flags] [packages]
Install compiles and installs the packages named by the import paths,
along with their dependencies.
For more about the build flags, see 'go help build'.
For more about specifying packages, see 'go help packages'.
See also: go build, go get, go clean.
The only build flags allowed are
-a
force rebuilding of packages that are already up-to-date.
-n
print the commands but do not run them.
-p n
the number of programs, such as build commands or
test binaries, that can be run in parallel.
The default is the number of CPUs available, except
on darwin/arm which defaults to 1.
-race
enable data race detection.
Supported only on linux/amd64, freebsd/amd64, darwin/amd64 and windows/amd64.
-msan
enable interoperation with memory sanitizer.
Supported only on linux/amd64,
and only with Clang/LLVM as the host C compiler.
-v
print the names of packages as they are compiled.
-work
print the name of the temporary work directory and
do not delete it when exiting.
-x
print the commands.
-asmflags 'flag list'
arguments to pass on each go tool asm invocation.
-buildmode mode
build mode to use. See 'go help buildmode' for more.
-compiler name
name of compiler to use, as in runtime.Compiler (gccgo or gc).
-gccgoflags 'arg list'
arguments to pass on each gccgo compiler/linker invocation.
-gcflags 'arg list'
arguments to pass on each go tool compile invocation.
-installsuffix suffix
a suffix to use in the name of the package installation directory,
in order to keep output separate from default builds.
If using the -race flag, the install suffix is automatically set to race
or, if set explicitly, has _race appended to it. Likewise for the -msan
flag. Using a -buildmode option that requires non-default compile flags
has a similar effect.
-ldflags 'flag list'
arguments to pass on each go tool link invocation.
-linkshared
link against shared libraries previously created with
-buildmode=shared.
-pkgdir dir
install and load all packages from dir instead of the usual locations.
For example, when building with a non-standard configuration,
use -pkgdir to keep generated packages in a separate location.
-tags 'tag list'
a list of build tags to consider satisfied during the build.
For more information about build tags, see the description of
build constraints in the documentation for the go/build package.
-toolexec 'cmd args'
a program to use to invoke toolchain programs like vet and asm.
For example, instead of running asm, the go command will run
'cmd args /path/to/asm <arguments for asm>'.
Where -i and -o flags are no possible to install command and are available to build command only.
Now, The only option that I see to change the name of the Go install binary name is manually mv oldnamebinary newnamebinary. At least, this behaviour is in GO 1.6, maybe on coming releases can change.
we can run below commands to generate binary executable with the desired name and path:
optional if you want dep to be downloaded and installed
go install
to build the binary at path and name desired:
go build -o path/binary_name
example: go build -o ~/go/bin/mybinaryname
In GO 1.6+ you can rename your main file to whatever you want.
Example :
mv main.go myApp.go
Then just run go install myApp.go
Your myApp is ready to exec, and the location is in $GOROOT/bin
After upgrading from Go 1.2.1 to 1.3 (Windows 7 64 bit) "go build" execution time has increased from around 4 to over 45 seconds. There were no other changes except the go version update. Switching off the virus scanner seems to have no effect. Any clues?
You probably have dependencies that are being recompiled each time. Try go install -a mypackage to rebuild all dependencies.
Removing $GOPATH/pkg also helps to ensure you don't have old object files around.
Building with the -x flag will show you if the toolchain is finding incompatible versions.
I have the exact same problem, running this command solves it:
go get -u -v github.com/mattn/go-sqlite3
Another tip: http://kokizzu.blogspot.co.id/2016/06/solution-for-golang-slow-compile.html
Using go1.6,
Simply run go build -i.
It will compile all the dependencies and store them at $GOPATH/pkg/*/* as .a files.
Later when you run go run main.go, everything is much faster.
What s really great is that if you use vendored dependencies (IE: a vendor folder in your project), deps are built appropriately within $GOPATH/pkg/**/yourproject/vendor/**
So you don t have to go get install/get/whatever and have a mix of vendor / global dependencies.
I suspect you got to re-build .a files after deps update (glide update or smthg like this), but i did not test that yet.
After Go 1.10, you'd just need to type go build. You'd not need to type: go build -i.
From the draft Go 1.10 document, here.
Build & Install
The go build command now detects out-of-date packages purely based on the content of source files, specified build flags, and metadata stored in the compiled packages. Modification times are no longer consulted or relevant. The old advice to add -a to force a rebuild in cases where the modification times were misleading for one reason or another (for example, changes in build flags) is no longer necessary: builds now always detect when packages must be rebuilt. (If you observe otherwise, please file a bug.)
...
The go build command now maintains a cache of recently built packages, separate from the installed packages in $GOROOT/pkg or $GOPATH/pkg. The effect of the cache should be to speed builds that do not explicitly install packages or when switching between different copies of source code (for example, when changing back and forth between different branches in a version control system). The old advice to add the -i flag for speed, as in go build -i or go test -i, is no longer necessary: builds run just as fast without -i. For more details, see go help cache.
I just experienced the same problem - updating from 1.4 to 1.5. It seems that the olds versions are somehow incompatible or are being rebuild every time as go build -x shows. Executing go get -v invalidates all packages or refetches them, I am not quite sure and go build -x shows quite less output.
You can build sqlite3 like this:
cd ./vendor/github.com/mattn/go-sqlite3/
go install
After that your project will b built much faster.
If you try as all other said but still not work, I suggest removing the directory of $GOPATH such as :
sudo rm -rf $GOPATH
cd yourproject
go get -d
go get -u -v github.com/mattn/go-sqlite3