E.g., github security alery signaled:
Upgrade github.com/opencontainers/runc to version 1.0.0-rc95 or later. For example:
How can I do it in golang with minimal code changes? This package is not being declared in go.mod (seems like it's indirect dependency).
When I run go get -u, I can see a whole bunch of undirect dependencies added to my go.mod which is a bit overkill:
require (
cloud.google.com/go v0.N.0 // indirect <--------
Is there a way to update directy that dependency?
Pretty sure go get -u github.com/opencontainers/runc should do it.
Related
I'm trying to compare the behavior of go mod tidy (and the resulting content of go.sum) to the output of go list -m all.
Reading the docs, I understand go.sum contains the whole list of dependent modules declared in go.mod and in dependencies' go.mod files, go list -m all shows the modules really loaded during the execution.
As an example, an application including logrus and prometheus like this:
go.mod
module mytest
go 1.14
require (
github.com/prometheus/common v0.4.0
github.com/sirupsen/logrus v1.8.1
)
main.go
package main
import "github.com/sirupsen/logrus"
import "github.com/prometheus/common/version"
func main() {
logrus.Info("Hello World")
logrus.Infof("Prometheus info: %v", version.Info())
}
After go mod tidy, go.sum shows both logrus v1.8.1, requested by the go.mod, and 1.2.0, dependency of prometheus v0.4.0; go list -m all shows only v1.8.1.
go.sum
[...]
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.8.1 h1:dJKuHgqk1NNQlqoA6BTlM1Wf9DOH3NBjQyu0h9+AZZE=
github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
[...]
output of go list
[...]
github.com/sirupsen/logrus v1.8.1
[...]
Is it correct to say the modules really used by the application are listed by go list -m all?
The underlying problem is that a static code analysis detects insecure module versions listed in go.sum, but actually these versions don't show up in go list -m all, hence they shouldn't be really used by the application, but only downloaded during build phase to select the proper minimal version.
Some reference:
https://go.dev/ref/mod#go-mod-tidy
go mod tidy acts as if all build tags are enabled, so it will consider
platform-specific source files and files that require custom build
tags, even if those source files wouldn’t normally be built.
https://go.dev/ref/mod#go-sum-files
The go.sum file may contain hashes for multiple versions of a module.
The go command may need to load go.mod files from multiple versions of
a dependency in order to perform minimal version selection. go.sum may
also contain hashes for module versions that aren’t needed anymore
(for example, after an upgrade).
https://github.com/golang/go/wiki/Modules#is-gosum-a-lock-file-why-does-gosum-include-information-for-module-versions-i-am-no-longer-using
[...]In addition, your module's go.sum records checksums for all
direct and indirect dependencies used in a build (and hence your
go.sum will frequently have more modules listed than your go.mod).
https://github.com/golang/go/wiki/Modules#version-selection
The minimal version selection algorithm is used to select the versions
of all modules used in a build. For each module in a build, the
version selected by minimal version selection is always the
semantically highest of the versions explicitly listed by a require
directive in the main module or one of its dependencies.
As an example, if your module depends on module A which has a
require D v1.0.0, and your module also depends on module B which
has a require D v1.1.1, then minimal version selection would choose
v1.1.1 of D to include in the build (given it is the highest listed
require version). [...] To see a list of the selected module versions
(including indirect dependencies), use go list -m all.
Yes, it correct to say the modules really "used" by the application are listed by go list -m all (as per documentation you provided the link of). By "used", it means the package selected at build time for the compilation of the go code of your application.
We had a similar issue with a static analysis tool and we had to change the configuration to use the output of go list -m all (dumped in a file) instead of go.sum.
I have a question about dependencies in golang.
My application defines a go.mod like this:
module my.host.com/myapp
require (
ext1.com/module1 v0.0.1
)
go 1.14
The dependency relationship is：
ext1.com/module1 v0.0.1 depends on ext3.com/module3 v0.0.3
A security scan detects ext3.com/module3 v0.0.3 is insecure and must be updated to v0.0.4.
Is there a way to "force" myapp to get only module3 v0.0.4, overriding the directives defined in module1 v0.0.1 go.mod?
Let's say ext1.com/module1 v0.0.1 is already at the latest version, so upgrading it doesn't work.
Would "replace" work?
module my.host.com/myapp
require (
ext1.com/module1 v0.0.1
)
replace ext3.com/module3 v0.0.3 => ext3.com/module3 v0.0.4
go 1.14
Thanks in advance!
Run go get -u ext3.com/module3#v0.0.4.
This upgrades the module to at least the v0.0.4
Given the dependency main -> B -> C, when main requires a higher version of C than that required by B, the higher version is selected, with // indirect.
See this https://go.dev/ref/mod#go-mod-file-require
If the go directive specifies go 1.16 or lower, the go command adds an indirect requirement when the selected version of a module is higher than what is already implied (transitively) by the main module’s other dependencies. That may occur because of an explicit upgrade (go get -u ./...)
I quote this part because your go.mod has go 1.14
I wrote a program and want to encapsulate some logic.
So I did new module and pull it in my git. Link for git looks like
gitlab.xxx.ru/group/subgroup/proj
but when I tried to get it with go get, I got error
fatal: «https://xxx.ru:#gitlab.xxx.ru/group/subgroup.git/» unreachable: URL using bad/illegal format or missing URL
Go tried to load subgroup instead project.
I made folder gitlab.xxx.ru/group/subgroup/ in $GOPATH/src/ and clone my project.
And now it wrote
could not import gitlab.xxx.ru/group/subgroup/proj (no required module provides package "gitlab.xxx.ru/group/subgroup/proj")
So, if I understand correctly, in Golang 1.16 I can't just put my project in the correct directory and I can't use local packages.
How to fix loading from my GitLab and load it with ssh?
Thank you.
UDP go.mod in my proj.
module gitlab.xxx.ru/group/subgroup/proj
go 1.16
require (
golang.org/x/sys v0.0.0-20210608053332-aa57babbf139
golang.org/x/text v0.3.6
)
You may be hitting the intended behaviour from Gitlab which will prevent go from fetching the list of subgroups while trying to compute project dependencies. Since go get requests are unauthenticated, existing projects not under the root group are invisible and cannot be found.
To overcome this limitation, which Gitlab has yet to provide a solution, you can use one of the following two approaches:
Have the project located at the root and not in a subgroup (not always possible)
Leverage the .git extension as well as the replace directive in the project which imports your module (see below)
Go is able to fetch the project living under a subgroup if it knows the version control qualifier (.git). To indicate this, make sure you import the project using the following format gitlab.xxx.ru/group/subgroup/proj.git
While this alone works, it would force you to have all those .git in your imports. To overcome this, you also need a replace directive in your go.mod so you can use the original import path.
In the end, the project which imports your module should have a go.mod that look like this:
require(
gitlab.xxx.ru/group/subgroup/proj v1.7.0
)
replace(
gitlab.xxx.ru/group/subgroup/proj => gitlab.xxx.ru/group/subgroup/proj.git v1.7.0
)
I am trying to convert https://github.com/appscode/voyager from glide to go mod.
I am getting an error like below:
go: github.com/Sirupsen/logrus#v1.4.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
go: error loading module requirements
How do I find out the source of this old Sirupsen module?
How do I find out the source of this old Sirupsen module?
Use the Go 1.13 beta (go get golang.org/dl/go1.13beta1 && go1.13beta1 download) or even better, try the latest Go on tip / master (go get golang.org/dl/gotip && gotip download).
Go 1.13 has improved error messages in general. It should help in your case, including most likely showing the import chain leading up to the error.
For example:
$ gotip build .
go: example.com/temp/mod imports
github.com/docker/libcompose/docker imports
github.com/Sirupsen/logrus: github.com/Sirupsen/logrus#v1.4.2: parsing go.mod:
module declares its path as: github.com/Sirupsen/logrus
but was required as: github.com/sirupsen/logrus
In that example, you can see that docker/libcompose/docker is importing the old and now incorrect uppercase version of Sirupsen/logrus.
The most common reason people see a Sirupsen/logrus vs. sirupsen/logrus case mismatch is when importing github.com/docker/docker or one of the other docker repos. Importing docker repos is a bit confusing with modules, including because:
The docker/docker repo does not follow semver.
There is a very old v1.13.1 semver tag on the docker/docker repo.
Even though it is a couple years old, it is still the "latest" semver tag on that repo, and hence that old version is picked by default by the go command if you don't ask for a more specific version.
That old docker/docker version imports the old and not incorrect uppercase Sirupsen/logrus, which can then trigger the error reported in the question above.
The docker client package has had breaking changes after v1.13.1.
There is generally confusion about docker/docker vs. docker/engine repositories, and about what import paths to use.
The docker repos do not have go.mod files.
For the docker/docker repo, the import path remains github.com/docker/docker, but it needs to come from github.com/docker/engine, so the recommended approach is often for a docker importer to do import "github.com/docker/docker" and edit their go.mod to something like this:
require (
github.com/docker/docker v1.13.1
)
replace github.com/docker/docker => github.com/docker/engine <tag-or-commit-hash>
Docker issue #39302 tracks trying to document how to import docker repos when using modules.
Using go modules, I would like to replace labix.org/v2/mgo with github.com/globalsign/mgo. The http://labix.org/mgo repository is unmaintained and has been forked to https://github.com/globalsign/mgo
my code is stored outside $GOPATH in the directory ~/git/foo
I'm using go1.11
other go modules are working (for example go list -m all lists other modules, the files go.mod and go.sum are updating automatically. See the full file below)
I've tried the following in the go.mod file:
replace labix.org/v2/mgo => github.com/globalsign/mgo v0.0.0-20181015145952-eeefdecb41b842af6dc652aaea4026e8403e62df
Running go build gives the following error:
build github.com/foo/bar: cannot find module for path labix.org/v2/mgo
The documentation in go help modules discusses Pseudo-Versions eg v0.0.0-yyy.., which I'm trying to use because the tags on https://github.com/globalsign/mgo are of the form r2018.06.15 and not v1.2.3 (semantic versioning).
In addition go help modules says:
Pseudo-versions never need to be typed by hand: the go command will accept
the plain commit hash and translate it into a pseudo-version (or a tagged
version if available) automatically. This conversion is an example of a
module query.
However I can't work out the command for generating a pseudo-version when I'm in the cloned github.com/globalsign/mgo (located at $GOPATH/src/github.com/globalsign/mgo). Hence the pseudo-version I've manually generated may be wrong.
The full go.mod file looks like:
module github.com/foo/bar
replace labix.org/v2/mgo => github.com/globalsign/mgo v0.0.0-20181015145952-eeefdecb41b842af6dc652aaea4026e8403e62df
require (
github.com/DATA-DOG/godog v0.7.8
github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38 // indirect
github.com/alecthomas/colour v0.0.0-20160524082231-60882d9e2721 // indirect
github.com/alecthomas/repr v0.0.0-20181024024818-d37bc2a10ba1 // indirect
...
)
Try the go edit -replace command, without modifying the line of the original package.
This way you don't need to know the exact pseudo version beforehand, just the commit hash id.
So in your case, don't manually modify go.mod require section of "labix.org/v2/mgo", and run this command to append your commit "eeefdec" of your fork, on top of labix version:
$ GO111MODULE="on" go mod edit -replace=labix.org/v2/mgo=github.com/globalsign/mgo#eeefdec
After the first time you build or test your module, GO will try to pull the new commit, and then will generate the "replace" line with the correct pseudo version for you. So then you'll see on the bottom of go.mod:
replace labix.org/v2/mgo => github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
When using the replace directive, leave the pseudo-version out.
It's also stated here, which points to an open issue.
Probably off-topic, but I've mostly used replace when I wanted to use a local version of some dependency. Why not import the forked lib which you'd like to use (instead of the original non-maintained one) and have mod resolve it properly?
The source being replaced (in this case labix.org/v2/mgo) also needs to be added to the require list with a version of v0.0.0 (even though it won't be downloaded). In the replace the source doesn't need a version but the target does.
However I haven't worked out how to auto-generate the pseudo version for the target (github.com/globalsign/mgo in this case), so I still needed to manually generate it.
Here's a better go.mod file:
1 module foo.bar/qux
2
3 replace labix.org/v2/mgo => github.com/globalsign/mgo v0.0.0-20181015145952-eeefdecb41b842af6dc652aaea4026e8403e62df
4
5 require (
6 github.com/DATA-DOG/godog v0.7.8
7 github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38 // indirect
<snip>
21 golang.org/x/image v0.0.0-20181116024801-cd38e8056d9b // indirect
22 labix.org/v2/mgo v0.0.0
23 )
Notice the require of labix.org on line 22; go mod tidy accepts this.
However I now come up against a different issue, the use of internal packages (referred to in issues like this: https://github.com/golang/go/issues/23970). I still haven't solved this new problem.
% go build main.go
/home/sonia/go/pkg/mod/github.com/globalsign/mgo#v0.0.0-20181015145952-eeefdecb41b842af6dc652aaea4026e8403e62df/auth.go:38:2:
use of internal package github.com/globalsign/mgo/internal/scram not allowed