Being very new to Go, I'm trying to import a third party library into a vendor folder. I follow the instructions given by the Go docs, but didn't find anything about third party libraries.
Update (2019)
The Go environment is slowly starting to move away from tools like dep and towards the native Go tooling around modules. While explaining models is outside the scope of this answer, you can look into modules from the following places:
https://blog.golang.org/modules2019
https://github.com/golang/go/wiki/Modules
tldr
Install dep: go get -u github.com/golang/dep/cmd/dep
In your project run: dep init
answer
The easiest way to solve this problem in my opinion is using the dep dependency management tool. This tool is very widely in use and is very easy to use. Here is a typical workflow:
First you should install the dep program.
go get -u github.com/golang/dep/cmd/dep
Now you have access to the dep command. Full documentation can be found here: https://golang.github.io/dep/
This is how you get 3rd party libraries into your vendor directory. In the example below we will use the url router github.com/gorilla/mux.
First, in you code import the libraries like normal.
package main
import "github.com/gorilla/mux"
func main {
r := mux.NewRouter()
r.HandleFunc("/", HomeHandler)
r.HandleFunc("/products", ProductsHandler)
r.HandleFunc("/articles", ArticlesHandler)
http.Handle("/", r)
}
Now all we have to do to get it sed up is run the dep init command. This will look for all of your imports and create a vendor directory for you with all of your needed dependencies. Note that dep automatically analyzes your imports.
Dep in depth
Once you have initialized dep you can start work on your project as normal. When you add a new library you can run the dep ensure command to get the newly added 3rd party libraries in the vendor directory.
In addition, dep gives you the capability to lock down on particular versions of 3rd party libraries. dep init initializes your project with two files: Gopkg.toml and Gopkg.lock. The Gopkg.toml file contains assertions about which dependencies will be at what version. For example, if you wantted the gorilla mux library to stay at version v1.4.0, you could add the following line to your Gopkg.toml:
[[constraint]]
name = "github.com/gorilla/mux"
version = "=v1.4.0"
Dep also has functionality to upgrade dependencies, remove unused dependencies from vendor, and much more. Look at the documentation for more details. https://golang.github.io/dep/
Related
I am new to go and have trouble using a library that does not have a go.mod file. The library is https://github.com/yourbasic/graph and I tried installing it according to the instructions given in the go docs. Since I cannot make a request for the version on the repo, I used the #latest flag, so in order to install, I executed
go install github.com/yourbasic/graph#latest
This however fails and says
package github.com/yourbasic/graph is not a main package
Is there a proper way to install libraries like that? I assume my way of proceeding to just copy the files into a directory within my project is not very clean.
As #JimB said: instead of attempting to install it,
Import the package in your code where applicable.
Run go get github.com/yourbasic/graph#latest — which will download the module (to the local cache) and update your module's go.mod file.
Build.
The steps 1 and 2 can be swapped, but then the generated go.mod entry will have the // indirect comment which will disappear next time you run go mod tidy.
I am working on a golang project which have a few packages (folders). My purpose is to pull out a certain package into a separate module, but before I do this I want to see the package dependencies. Is there any tool out there which I can run to view the package dependencies. I am not talking about 3rd party modules but just simple folders in the same repo. I checked around for a few tools but could not find any.
go list -f {{.Deps}} path/to/your/package gives you all direct and indirect dependencies.
For Go modules you can do go mod graph or go list -m all.
I was wondering if there is a way to automatically download all the imports.
So let's assume I need to use github.com/gorilla/mux and several other packages in my code base. Should I manually go to my ~/go/src and run go get repo or is there a smarter way of doing dependency management. I am using Goland IDE for my development.
if there is a way to automatically download all the imports
You can download all imported pkgs and their dependencies by running go get from the command line.
I am using Goland IDE for my development
I'm using Goland too. When imports can't be found (ie the import path is highlighted in red), you can place your typing caret over it and press alt + enter and select go get ... from the popup window to automatically import.
There are several approaces:
Simply go get github.com/gorilla/mux which will download sources in your $GOPATH and will be resolved automatically when compiling
Use dependency management (godep, glide[deprecated])
Use modules (experimental feature in Go 1.11 - Module. Check more here)
If you want a good and stable solution, use dep (.First you have to install it, then run:
cd $GOPATH/src/path/to/project
dep init
dep ensure -add github.com/gorilla/mux
You will see a new folder vendor in your project and 2 new dependency configuration files Gopkg.lock and Gopkg.toml.
Read more about godep here.
Then run your main file as usual.
You can use dep package manager which will go through your code and automatically import all the packages you use in you code. If you are working with >go1.11 I would suggest to use newly added go mod.
I come from a ruby background, and I just started learning go. Is there any standard way to install 3rd-party libraries that's comparable to RubyGems?
Since go1.11 released, we have an official go package management tools, the Go Modules.
The difference between go modules and other package management tools is go modules does not rely on $GOPATH. The project must be placed outside of $GOPATH. If your project is already inside a $GOPATH but you wanted to use package management tools, then I suggest to see the old answer below.
Usage:
mkdir testproject
cd testproject
# init project as go module with root package name is testproject
go mod init testproject
# install 3rd party library, it will be stored inside testproject/vendor
go get github.com/labstack/echo
go get github.com/novalagung/gubrak
the go mod init command generates Go.mod file (similar like Gemfile for ruby). You can either install the 3rd party libraries through the usual go get command, or by adding the library metadata into Go.mod file then perform go mod tidy.
More informations about Go Modules: https://blog.golang.org/using-go-modules
Old answer
Go does have package management tool as well, it's called dep.
Usage example:
cd $GOPATH/src
mkdir testproject
cd testproject
# init project
dep init
# install 3rd party library
dep ensure -add github.com/labstack/echo
dep ensure -add github.com/novalagung/gubrak
dep generates Gopkg.toml file (similar like Gemfile for ruby). You can either install the 3rd party libraries through dep ensure -add command, or by adding the library metadata into Gopkg.toml then perform dep ensure.
Btw, there is also few other alternatives other than dep. For more information please take a look at https://github.com/golang/go/wiki/PackageManagementTools.
I've recently started with Go 1.11 and love the modules. Apart from runtime dependencies I need to work with go modules during the build, e.g. during go generate.
How can I install a specific build dependency (e.g. github.com/aprice/embed/cmd/embed) and run that specific tool from which folder? Is go get the right tool for doing so?
If you get an error
I was not seeing the dependency that I wanted added to the go.mod and I was getting this error:
internal/tools/tools.go:6:5: import "github.com/UnnoTed/fileb0x" is a program, not an importable package
(fileb0x is the thing I'm trying to add)
I'm not 100% clear on the sequence of events that fixed it, but I did all of these things:
Using a "tools" package
I made a tools directory:
mkdir -p internal/tools
I put the tools package inside of it (as mentioned above):
internal/tools/tools.go:
// +build tools
package tools
import (
_ "github.com/UnnoTed/fileb0x"
)
Note that the tag is mostly not important. You could use foo:
// +build foo
However, you cannot use ignore. That's a special predefined tag.
// +build ignore
// NO NO NO NO NO
// `ignore` is a special keyword which (surprise) will cause
// the file to be ignore, even for dependencies
Updating go.mod
The best way is probably to run go mod tidy:
go mod tidy
However, before I did that I ran a number of commands trying to figure out which one would cause it to go into go.mod:
go install github.com/UnnoTed/fileb0x # didn't seem to do the trick
go get
go generate ./...
go build ./...
go install ./...
go mod vendor
Later I did a git reset and rm -rf ~/go/pkg/mod; mkdir ~/go/pkg/mod and found that go mod tidy did well enough on its own.
vendoring
In order to actually take advantage of the modules cache in a project you need to copy-in the source code
go mod vendor
That will grab all dependencies from go.mod
You also need to change nearly all of your go commands to use -mod=vendor in any Makefiles, Dockerfiles or other scripts.
go fmt -mod=vendor ./... # has a bug slated to be fixed in go1.15
go generate -mod=vendor ./...
go build -mod=vendor ./...
That includes go build, go get, go install, and any go run called by go generate (and even the go generate itself)
//go:generate go run -mod=vendor github.com/UnnoTed/fileb0x b0x.toml
package main
// ...
https://github.com/golang/go/issues/25922 proved helpful for me, especially
when using build-only dependencies with modules the main point is version selection (not installing these!)
To avoid installing you can modify your //go:generate directive to something like:
//go:generate go run golang.org/x/tools/cmd/stringer ARGS
There is also the best practices repo: https://github.com/go-modules-by-example/index/blob/master/010_tools/README.md
The convention is to add a file named "tools.go" that is guarded by a build constraint and imports all required tools:
// +build tools
package tools
import (
_ "github.com/aprice/embed/cmd/embed"
)
https://github.com/golang/go/issues/25922#issuecomment-412992431
The tools are then installed as usual in one of
$GOBIN
$GOPATH/bin
$HOME/go/bin
You may also want to follow https://github.com/golang/go/issues/27653, which discusses future explicit support for tools.
tools.go is a great solution if you're building an app or service. But if you're building a library, tools.go still leaks dependencies to things consuming your library (your tools are still there as indirect dependencies, and go mod tidy will pull them in since it considers every possible target). That's not the end of the world since those modules never end up in the actual built binaries of the consumer, but it's still messy.
https://github.com/myitcv/gobin/issues/44 is probably the most promising approach to fixing this long term, but short term I've used a combination of the "internal module" approach explained there along with https://github.com/izumin5210/gex.
First, I install gex globally:
GO111MODULE=off go get github.com/izumin5210/gex/cmd/gex
Then before actually using gex I create a structure like this:
myproject/
\
- go.mod: module github.com/ysamlan/myproject
\
internal/
\
tools/
- go.mod: module github.com/ysamlan/myproject/tools
To install a build-only tool I just cd internal/tools and run gex --add (sometool), which puts that tool in internal/tools/bin. CI scripts and other folks that want to build my stuff locally just need to run cd internal/tools && gex --build to reliably and reproducibly populate the tool binaries, but the top-level go.mod is unchanged.
The key piece there is creating that internal/tools/go.mod file with a different module path than the one the root project uses, and then only running gex from that directory.