I am trying to build a Go program in a offline VM. The program only has one dependency which is https://github.com/go-log/log.
When trying to build the program it says.
go: download github.com/go-log/log v0.2.0
However since I do not have internet it won't work. I have tried downloading the code from Github and placing it in the directories that I thought would be correct.
c:\Program Files\Go\pkg\mod\github.com\go-log\log#0.2.0
and
c:\users\user\go\pkg\mod\github.com\go-log\log#0.2.0
When doing this it still attempts to download from the internet. I am not sure what I need to do. Do I need to compile the log program?
You can use Vendoring in golang.
go mod vendor also creates the file vendor/modules.txt that contains a list of vendored packages and the module versions they were copied from. When vendoring is enabled, this manifest is used as a source of module version information, as reported by go list -m and go version -m. When the go command reads vendor/modules.txt, it checks that the module versions are consistent with go.mod. If go.mod has changed since vendor/modules.txt was generated, the go command will report an error. go mod vendor should be run again to update the vendor directory.
In an online machine use go mod download and go mod vendor in the folder containing your go.mod.
This will download and place your requirements in a folder named vendor in the same directory.
Move this folder to your offline machine behind your go.mod file.
use go build -mod=vendor to build your project. this way go compiler will use dependencies stored in the vendor folder.
You probably want go mod vendor. Emphasis mine:
The go mod vendor command constructs a directory named vendor in the main module’s root directory that contains copies of all packages needed to support builds and tests of packages in the main module. Packages that are only imported by tests of packages outside the main module are not included. As with go mod tidy and other module commands, build constraints except for ignore are not considered when constructing the vendor directory.
When vendoring is enabled, the go command will load packages from the vendor directory instead of downloading modules from their sources into the module cache and using packages those downloaded copies. See Vendoring for more information.
See also: what is the purpose of `go mod vendor` command?
Given a simple main.go that imports github.com/go-log/log/print:
package main
import (
"github.com/go-log/log/print"
)
type MyLogger struct{}
func (l MyLogger) Print(v ...interface{}) {}
func (l MyLogger) Printf(format string, v ...interface{}) {}
func main() {
printer := print.New(MyLogger{})
printer.Log("Blah")
}
... running go mod vendor will vendor those files and you will end up with the following in your vendor folder:
|-github.com
| |-go-log
| | |-log
| | | |-LICENSE
| | | |-print
| | | | |-print.go
|-modules.txt
The next time you go build, the version of print from the vendor folder will be used instead.
Related
What is the golang command that is equivalent to npm install
npm install downloads all the dependencies listed in the package.json file.
Having said that, what is the command that downloads all the dependencies in the go.mod file?
If you have only a go.mod and you have Go 1.16 or later:
If you just want to run your code, use go build or go run . - your dependencies will be downloaded and built automatically
If you want to save a copy of your dependencies locally, use go mod vendor
Both the above will create a go.sum file (this is maintained by the Go Tools - you can ignore it)
The vendor command will create a vendor folder with a copy of all the source code from your dependencies. Note: If you do use the vendor approach, you will need to run go mod vendor if there is a change in your dependencies, so that a copy is downloaded to the vendor folder. The advantage is your code will build without an internet connection. The disadvantage is you need to keep it up to date.
That should get you started for every day use.
If you'd like to know all about modules, this is a good source.
I am trying to fetch a Go project and copy the dependencies under the vendor/ directory so I have the complete source code of the project and its dependencies in my project. However, even after doing that, deleting the packages under $GOPATH/pkg/mod and rebuilding cause the Go compiler to re-fetch all dependencies, which takes a considerable of time.
This is what I did:
# Fetch the project, e.g. influx/telegraf
go get -d github.com/influxdata/telegraf
# CD into the project
cd $GOPATH/src/influxdata/telegraf
# Fetch the modules under vendor/ directory
go mod vendor
After calling the last command, Go will fetch all the dependencies under pkg/mod. Not sure why it is doing that, but I assume it is because it needs to build the project normally, and then move the fetched dependencies under the vendor/ folder. After that, I can build successfully. However, to make sure I no longer require those dependencies, I deleted the pkg/mod directory completely and rebuilt the project. Go compiler, for some reason, fetched the packages again.
Is there anything I am doing wrong?
Thanks!
The vendor folder is not used automatically in all cases.
To make sure dependencies are loaded from the main module's vendor folder, pass -mod=vendor to the go tool.
The vendor folder if present is only used automatically (if not specified otherwise with -mod=mod) if the go version specified by go.mod file is 1.14 or higher.
These are detailed in Command go: Maintaining module requirements:
If invoked with -mod=vendor, the go command loads packages from the main module's vendor directory instead of downloading modules to and loading packages from the module cache. The go command assumes the vendor directory holds correct copies of dependencies, and it does not compute the set of required module versions from go.mod files. However, the go command does check that vendor/modules.txt (generated by 'go mod vendor') contains metadata consistent with go.mod.
If invoked with -mod=mod, the go command loads modules from the module cache even if there is a vendor directory present.
If the go command is not invoked with a -mod flag and the vendor directory is present and the "go" version in go.mod is 1.14 or higher, the go command will act as if it were invoked with -mod=vendor.
Setup:
Our project is using golang 1.12.14
We are using go build -mod=vendor
Issue: When new dependencies are added to go.mod the vendor folder isn't updated and people are committing code and forgetting to run go mod vendor to update the folder. My understanding is that since -mod=vendor specifies to use packages from the vendor folder, the go.mod file will have discrepancies from what we are actually using when building the project.
Question: Should go mod vendor be added to a pre-commit hook?
As of Go 1.14, the go command automatically checks for consistency between the vendor directory and the go.mod file whenever the vendor directory is used. In addition, it uses the vendor directory by default if the module specifies go 1.14 or higher (see https://tip.golang.org/doc/go1.14#go-command).
As of today, the oldest supported version of the Go toolchain is Go 1.15.13.
So if you upgrade to a supported version of the Go toolchain, it should not be necessary to run go mod vendor as a pre-commit hook. The go command itself will flag inconsistencies whenever the vendor directory is used.
I've installed a package using go modules (go get in Go 1.13) and now I want to remove it. In the documentation there is nothing about this and in go get docu neither.
Removing the package from go.mod manually doesn't solve the issue so it remains in go.sum.
How should I remove a package in a clean way?
Found it https://go.dev/blog/using-go-modules#removing-unused-dependencies
go mod tidy
So basically, once the package is not being imported in any package you can perform a go mod tidy and it will safely remove the unused dependencies.
And if you are vendoring the dependencies, then run the command below to make the module changes be applied in the vendor folder:
go mod vendor
#jesugmz answer doesn't say, what if you wanna remove a currently using package in go modules.
So, if you're using go modules (you have a go.mod file in your project) and you want to remove a currently using package, check $GOPATH/pkg/mod/ directory and simply remove the package named package#version.
For example, if you have github.com/some/project package installed, you should run the following command:
rm -rf $(go env GOPATH)/pkg/mod/github.com/some/project#v1.0.0
You can find the using package version in go.mod file.
If we have bin directory already where executables go then what is the need of pkg directory? Please explain.
The pkg directory contains Go package objects compiled from src directory Go source code packages, which are then used, at link time, to create the complete Go executable binary in the bin directory.
We can compile a package once, but link that object into many executables. For example, the fmt package appears in almost every Go program. It's compiled once but linked many times, a big saving.
~/go/pkg
From How to Write Go Code, we know ~/go/pkg can store third party library. For example:
Create two file main.go and go.mod:
//main.go
package main
import (
"fmt"
"github.com/google/go-cmp/cmp"
)
func main() {
fmt.Println(cmp.Diff("Hello World", "Hello Go"))
}
//go.mod
module example.com/user/hello
go 1.13
Install a third party library
$ ls ~/go/pkg/mod/github.com/google/
...
$ go install example.com/user/hello
go: finding github.com/google/go-cmp v0.5.2
go: downloading github.com/google/go-cmp v0.5.2
go: extracting github.com/google/go-cmp v0.5.2
$ ls ~/go/pkg/mod/github.com/google/
... go-cmp#v0.5.2
$ ls ~/go/pkg/mod/github.com/google/go-cmp#v0.5.2
And then, you will see a lot of go files, not compile files.
$ ls ~/go/pkg/mod/github.com/google/go-cmp#v0.5.2/cmp/
... example_test.go options.go ...
pkg in user project
This pkg is a directory/package of user project. You can see it as Library and it's OK to use by external applications. For more things, you can check here.
You put your source code in src directory while pkg is the directory that holds compilation output of your actual source code. If you use multiple libraries/packages you will have different output with extension .a for each one, A linker should be responsible for linking and combining all of them together to produce one final executable in bin directory.
As pkg and bin are more specific to the machine or operating system into which you build your actual source code so it is not recommended to share both of them, your repo should have only your actual code.
A side note, if you plan to use docker containers, pkg dir should be ignored as we may build the source code in windows for example while you import/mount your code into linux container; at this time pkg will have compiled files that are only valid for windows