I have an external package that apart from the usual go get, needs to run make install in its $GOPATH/src directory in order to use it (performs some makefile and git magic).
Trying to use this package with modules means a copy of it is downloaded to the vendor library using go mod vendor. However, this copy is not a git repository so running make install inside the package's vendor folder fails.
Does this mean that the package cannot be used in a module and I have to revert to using GOPATH?
Does this mean that the package cannot be used in a module
Yes.
Contact the author and make him to check in what the makefile does.
Related
I am new to go and have trouble using a library that does not have a go.mod file. The library is https://github.com/yourbasic/graph and I tried installing it according to the instructions given in the go docs. Since I cannot make a request for the version on the repo, I used the #latest flag, so in order to install, I executed
go install github.com/yourbasic/graph#latest
This however fails and says
package github.com/yourbasic/graph is not a main package
Is there a proper way to install libraries like that? I assume my way of proceeding to just copy the files into a directory within my project is not very clean.
As #JimB said: instead of attempting to install it,
Import the package in your code where applicable.
Run go get github.com/yourbasic/graph#latest — which will download the module (to the local cache) and update your module's go.mod file.
Build.
The steps 1 and 2 can be swapped, but then the generated go.mod entry will have the // indirect comment which will disappear next time you run go mod tidy.
I am trying to download, make some tweaks and build a golang project from GitHub. The project's instructions are:
go get github.com/<vendor>/<projectName>
cd src/github.com/<vendor>/<projectName>
go build .
That used to work in the past — before enabling Go Modules.
Now I have GO111MODULE=on (go version go1.15.4 linux/amd64). When running the first command, go downloads the project as a module and all its dependencies.
But then there is no src/github.com/<vendor>/<projectName> folder anymore. Moreover, the is no folder named <projectName> anywhere in the system.
Instead, there is folder pkg/mod/github.com/<vendor> which contains the project folder with weird symbols in its name (exclamation marks etc.) and version identifiers.
How do I get the project folder available for making tweaks and builds?
As pointed by #Volker, good old git clone should be used.
It turns out that it should be used instead of go get github.com/<vendor>/<projectName> (no idea why the project vendor recommends that):
git clone git://github.com/<vendor>/<projectName>
cd <projectName>
go get ./...
# do tweaks here
go build .
If your goal is tweaks, the easiest way it use to use go mod vendor.
https://golang.org/ref/mod#go-mod-vendor
The go mod vendor command constructs a directory named vendor in the main module's root directory that contains copies of all packages needed to support builds and tests of packages in the main module
I've installed a package using go modules (go get in Go 1.13) and now I want to remove it. In the documentation there is nothing about this and in go get docu neither.
Removing the package from go.mod manually doesn't solve the issue so it remains in go.sum.
How should I remove a package in a clean way?
Found it https://go.dev/blog/using-go-modules#removing-unused-dependencies
go mod tidy
So basically, once the package is not being imported in any package you can perform a go mod tidy and it will safely remove the unused dependencies.
And if you are vendoring the dependencies, then run the command below to make the module changes be applied in the vendor folder:
go mod vendor
#jesugmz answer doesn't say, what if you wanna remove a currently using package in go modules.
So, if you're using go modules (you have a go.mod file in your project) and you want to remove a currently using package, check $GOPATH/pkg/mod/ directory and simply remove the package named package#version.
For example, if you have github.com/some/project package installed, you should run the following command:
rm -rf $(go env GOPATH)/pkg/mod/github.com/some/project#v1.0.0
You can find the using package version in go.mod file.
I come from a ruby background, and I just started learning go. Is there any standard way to install 3rd-party libraries that's comparable to RubyGems?
Since go1.11 released, we have an official go package management tools, the Go Modules.
The difference between go modules and other package management tools is go modules does not rely on $GOPATH. The project must be placed outside of $GOPATH. If your project is already inside a $GOPATH but you wanted to use package management tools, then I suggest to see the old answer below.
Usage:
mkdir testproject
cd testproject
# init project as go module with root package name is testproject
go mod init testproject
# install 3rd party library, it will be stored inside testproject/vendor
go get github.com/labstack/echo
go get github.com/novalagung/gubrak
the go mod init command generates Go.mod file (similar like Gemfile for ruby). You can either install the 3rd party libraries through the usual go get command, or by adding the library metadata into Go.mod file then perform go mod tidy.
More informations about Go Modules: https://blog.golang.org/using-go-modules
Old answer
Go does have package management tool as well, it's called dep.
Usage example:
cd $GOPATH/src
mkdir testproject
cd testproject
# init project
dep init
# install 3rd party library
dep ensure -add github.com/labstack/echo
dep ensure -add github.com/novalagung/gubrak
dep generates Gopkg.toml file (similar like Gemfile for ruby). You can either install the 3rd party libraries through dep ensure -add command, or by adding the library metadata into Gopkg.toml then perform dep ensure.
Btw, there is also few other alternatives other than dep. For more information please take a look at https://github.com/golang/go/wiki/PackageManagementTools.
I'm using go 1.11 with module support. I understand that the go tool now installs dependencies automatically on build/install. I also understand the reasoning.
I'm using docker to build my binaries. In many other ecosystems its common to copy over your dependency manifest (package.json, requirements.txt, etc) and install dependencies as a separate stage from build. This takes advantage of docker's layer caching, and makes rebuilds much faster since generally code changes vastly outnumber dependency changes.
I was wondering if vgo has any way to do this?
It was an issue #26610, which is fixed now.
So now you can just use:
go mod download
For this to work you need just the go.mod / go.sum files.
For example, here's how to have a cached multistage Docker build: (source)
FROM golang:1.17-alpine as builder
RUN apk --no-cache add ca-certificates git
WORKDIR /build
# Fetch dependencies
COPY go.mod go.sum ./
RUN go mod download
# Build
COPY . ./
RUN CGO_ENABLED=0 go build
# Create final image
FROM alpine
WORKDIR /
COPY --from=builder /build/myapp .
EXPOSE 8080
CMD ["./myapp"]
Also see the article Containerize Your Go Developer Environment – Part 2, which describes how to leverage the Go compiler cache to speed up builds even further.
You may use the go mod vendor command which will create a vendor folder in the main module's root folder, and copy all dependencies into it. After this you may pass the -mod=vendor param to the go tool, and then dependencies from the vendor folder will be used to build / compile / test your app.
So what you may do to speed up your builds:
Run the go mod vendor command to have an actual version of your dependencies.
Save / cache this vendor folder.
During builds, restore this vendor folder, and build / install your app by passing the -mod=vendor argument to the go tool, so no dependencies will be downloaded, but the content of the vendor folder will be used.
Quoting from go help mod:
Modules and vendoring
When using modules, the go command completely ignores vendor directories.
By default, the go command satisfies dependencies by downloading modules
from their sources and using those downloaded copies (after verification,
as described in the previous section). To allow interoperation with older
versions of Go, or to ensure that all files used for a build are stored
together in a single file tree, 'go mod vendor' creates a directory named
vendor in the root directory of the main module and stores there all the
packages from dependency modules that are needed to support builds and
tests of packages in the main module.
To build using the main module's top-level vendor directory to satisfy
dependencies (disabling use of the usual network sources and local
caches), use 'go build -mod=vendor'. Note that only the main module's
top-level vendor directory is used; vendor directories in other locations
are still ignored.
I wanted to re-download all the dependencies using go mod, this is what I did:
Go to your GOROOT
sudo rm -rf pkg/mod/
Go to the directory where the go.mod file exists
go mod download
You can use a package manager, There are many of them like dep, glide, and govendor. dep is newer and is going to be integrated into go toolchain as official dependency management tool.
We also make docker images for go applications and we use dind to make those images and we prepared a CI/CD image with all dependencies preinstalled to make the builds faster. Though, it took a little bit of scripting to glue everything together.
Moreover, layering up the dependencies could result in big size of docker images. I suggest try multi-stage builds which could help making images super lite.