TL;DR: Where do .so files end up when you install plugins with go install -buildmode=plugin?
I have a project that uses plugins. The layout is something like this:
myproject/
├── main.go
└── modules
├── bar
│ └── main.go
└── foo
└── main.go
When I run go install the binary gets installed OK.
But I would also like to run go install for each of my modules and have them available to the main binary everywhere on the system.
If I run go install -buildmode=plugin from inside a module folder (say, modules/foo) the command runs to completion but I can't find the resulting file anywhere.
Installing a normal package ends up in:
GOPATH/pkg/<goos>_<goarch>_dynlink/path/to/parent/folder/packagename.a
Installing main packages end up in:
GOPATH/bin/foldername
(where foldername is the parent folder of the main package you install, it'll get an .exe extension on windows).
When you "go install" a plugin (using -buildmode=plugin), that ends up in
GOPATH/pkg/<goos>_<goarch>_dynlink/path/to/parent/folder/foldername.a
Related
I have the following project structure, outside of GOPATH.
. // Project root
├── Dockerfile
├── .env
├── README.md
└── src
├── main.go
├── go.mod
├── go.sum
├── internal
│ ├── somepackage
│ │ ├── main.go
│ │ └── types.go
│ ├── someother
│ │ ├── main.go
│ │ ├── oauth.go
│ │ └── types.go
│ └── models
│ └── main.go
└── pkg
├── somepackage
│ └── main.go
└── anotherpackage
└── main.go
I want to run my Go module code located in the src directory.
When I cd into the src directory and go run . or go build . my code, it works perfectly.
When I stand at the root of my project, I am unable to run go run ./src or go build ./src. I get the following error.
src/service.go:8:2: cannot find package "web-service/internal/auth" in any of:
/usr/lib/go/src/web-service/internal/auth (from $GOROOT)
/home/miloertas/Packages/go/src/web-service/internal/auth (from $GOPATH)
src/endpoints.go:3:8: cannot find package "web-service/internal/handlers" in any of:
/usr/lib/go/src/web-service/internal/handlers (from $GOROOT)
/home/miloertas/Packages/go/src/web-service/internal/handlers (from $GOPATH)
It's important that my source code remains in this src directory.
It is equally important that I am able to run and build my code from the root of my project (For example the .env file is located at the root of the repository).
I am therefore looking for a way to run or build my code in the src directory from the root of my project.
I tried moving the go.mod at the root of the project and running and ran go run ./src but this causes issues of its own:
The go command is now unable to locate all the sub-packages in internal and pkg
VSCode is now lost and executing tests is impossible for some reasons (Mainly because all sub-packages are not found).
Since Go 1.18, it's now possible to achieve this with Go workspaces.
Using the following directory structure
parent-dir/
└─ go.work
hello-world/
├─ go.mod
└─ main.go
You can run the hello-world module from the parent-dir using go run hello-world.
go.work
go 1.18
use ./hello-world
go.mod
module hello-world
go 1.18
Note: it is possible, not recommended as pointed out by #Volker
It's important that my source code remains in this src directory. It is equally important that I am able to run and build my code from the root of my project (For example the .env file is located at the root of the repository).
These two requirements are contradictory. You have to let go of one.
Especially the second one is unfounded: Do not use go run, use go build. Make the path to look for the .env file a command line option to your program (Go is not PHP or JavaScript, there simply is no project or source root for the executing binary). Or build the executable somewhere but execute it in you project root.
Note that having a src folder is -- to put it mildly -- uncommon.
I tried moving the go.mod at the root of the project and running and ran go run ./src but this causes issues of its own:
Well, start by not using go run at all, use go build. And then try building the actual main package. All the go tooling works best on packages, not on file system folders. If your module is named playing.hardball/for-unspecific-reasons and package main is in src try go build playing.hardball/for-unspecific-reasons/src.
Takeaways even if this doesn't work out the way you want:
Do not use go run. The reasons are manyfold, it is useful to run single file scripts and a loaded footgun for basically every other use case.
The go tool works on import paths. In simple cases the import path can be inferred from the filesystem.
A compiled executable has no notion of a "project directory", "source", "classpath" or whatever, it is a standalone executable runnable everywhere and completely detached from its sources.
Make all filesystem lookup path a configuration option (cmdline flag or environment variable); provide practical defaults (e.g. ./); use that when running your executable to announce where to find static stuff like .env files, templates, icons, css files, etc.
Here is the directory structure of my project (~/go/src/bitbucket.org/a/b):
├── cmd
│ ├── c
│ │ └── main.go
│ └── d
│ └── main.go
├── config
│ ├── config.go
│ ├── default.json
│ └── development.json
├── go.mod
├── go.sum
├── log
│ └── log.go
├── main.go
I need to compile 2 binaries (one for each module in cmd/).
I have tried running GO111MODULE=on go build ./cmd/c from project root (~/go/src/bitbucket.org/a/b). It silently finishes without doing anything.
I also tried running GO111MODULE=on go build -o test ./cmd/c. It created 29kb test file. When i add execution rights to it and run, it finishes with error:
./test: 2: ./test: Syntax error: newline unexpected
I have tried using go 1.12.5 and go 1.11.10.
Also when i put main.go file from any of the cmd directories to project root directory and build, the compiler builds it just fine (binary file size is ~33mb).
Is it possible to use 2 compiler entry points in a single project?
You can use go install ./... and it will create the executables in $GOPATH/bin directory if you are looking to get the executables
And regarding the multiple compiler entry points, you can. You can build using go build ./cmd/c ./cmd/d . but you cannot get the executables as per the GO Documentation -o can be only used in the case of single package. Instead you can write a makefile to get all the executables with a single make target.
And regarding the error that you are seeing, I would need more information. When I tried to build a sample application, everything works fine. I am not using GO111MODULE flag though.
go build gives me stackoverflow executable
go build -o test ./cmd/c gives me test executable
go build ./cmd/c gives me c executable
For your convenience I uploaded the project to github repo
I'm in the process of relearning Go. I installed the latest Go version (1.7.1) using gvm and I am looking to build a simple rest api app using gin. I installed it using glide get https://github.com/gin-gonic/gin (glide) and that created a "vendor" folder on my project root. Running my app though, go run main.go, I encounter this error
main.go:3:8: cannot find package "github.com/gin-gonic/gin" in any of:
/home/yowmamasita/.gvm/gos/go1.6.3/src/github.com/gin-gonic/gin (from $GOROOT)
/home/yowmamasita/.gvm/pkgsets/go1.6.3/global/src/github.com/gin-gonic/gin (from $GOPATH)
It is not resolving the "vendor" directory glide just created
.
├── glide.lock
├── glide.yaml
├── main.go
├── README.md
└── vendor
└── github.com
└── gin-gonic
└── gin
Not sure what's happening here, I thought after 1.5, it should be able to resolve imports from "vendor" directories without doing anything. I even added my projects folder on my $GOPATH
/home/yowmamasita/.gvm/pkgsets/go1.7.1/global:/home/yowmamasita/goprojects
What am I doing wrong here? I tried 1.6.3 too and I get the same error.
Please make sure:
Add the workspace (/home/yowmamasita/goprojects) to $GOPATH variable.
Typically under workspace there will be three directories which are bin, pkg and src. More details
You can omit pkg and bin, but the project which is using vendor packages or your custom package must be placed under $GOPATH/src, otherwise go compiler will not recognized it.
More discussions can be found here and here
The structure should look like:
$GOPATH
└── src
└── YOURPROJECT1
├── source codes #1
└── vendor/
└── YOURPROJECT2
├── source codes #2
└── vendor/
I have this folder structure for my fib package:
$ tree
.
└── src
└── fib
├── fib
│ └── main.go
├── fib.go
└── fib_test.go
(main.go is in package main, fib(_test).go is in package fib)
GOPATH is set to $PWD/src, GOBIN is set to $PWD/bin. When I run go install fib/fib, I get a file called fib in the directory bin (this is what I expect):
$ tree bin/
bin/
└── fib
But when I set GOOS or GOARCH, the directory in the form GOOS_GOARCH is created:
$ GOARCH=386 GOOS=windows go install fib/fib
$ tree bin/
bin/
└── windows_386
└── fib.exe
This is not what I want. I'd like to have the file fib.exe in the bin directory, not in the sub directory bin/windows_386.
(How) is this possible?
That doesn't seem possible, as illustrated in issue 6201.
GOARCH sets the kind of binary to build.
You might be cross-compiling: GOARCH might be arm.
You definitely don't want to run the arm tool on an x86 system.
The host system type is GOHOSTARCH.
To install the api tool (or any tools) you need to use
GOARCH=$(go env GOHOSTARCH) go install .../api
and then plain 'go tool' will find them.
In any case (GOARCH or GOHOSTARCH), the go command will install in a fixed location that you cannot change.
The phrase "I (don't) want" is incompatible with the go tool; the go tool works how it works. You can a) copy the file to where you want it to be after installing it with the go tool or b) compile it yourself, e.g. by invoking 6g manually (here you can specify the output). If you are unhappy with how the go tool works, just switch to a build tool of your liking, e.g. plain old Makefiles. Note that the go tool helps you there too, e.g. by invoking the compiler via go tool 6g
When I build a Go binary, I usually do something like this:
go build -ldflags "-X main.basedir somevar" -o mybuilddir/bin/myfile mypackage/main"
this builds the binary and places it in a custom directory. But this doesn't keep the "intermediate" package files beneath pkg/, which would speed up the next compilation runs.
The solution would be go install, but I cannot specify an output directory. It seems to be possible to set the binary directory with GOBIN, but I am unable to specify the name of the executable (always main).
What is a possible solution to this problem?
Custom destionation directory
Custom name (not main)
Keep intermediate generated package files (.a)
This is the src directory of GOPATH:
GOPATH/src$ tree
.
└── mypackage
├── packagea
│ └── packagea.go
├── packageb
│ └── packageb.go
└── main
└── mypackage.go
With go install, the package files (.a) are created in $GOPATH/pkg, with go build, I can't find the .a files anywhere.
Update Nov. 2017: Go 1.10 (Q1 2018) will add caching for go build and go install: see "go build rebuilds unnecessarily".
Original answer (2014)
With go install, the package files (.a) are created in $GOPATH/pkg, with go build, I can't find the .a files anywhere.
As mentioned in "How does the go build command work ?"
The final step is to pack the object file into an archive file, .a, which the linker and the compiler consume.
Because we invoked go build on a package, the result is discarded as $WORK is deleted after the build completes.
If we invoke go install -x two additional lines appear in the output
mkdir -p /home/dfc/go/pkg/linux_arm/crypto/
cp $WORK/crypto/hmac.a /home/dfc/go/pkg/linux_arm/crypto/hmac.a
This demonstrates the difference between go build and install;
build builds,
install builds then installs the result to be used by other builds.