The background:
Two Applications - A and B
Application 'A' contains two go.mod files
A/pkg/test/go.mod: (I keep here application specific structures which are gonna be reused by another services willing to integrate)
module A/pkg/test
go 1.14
require (
some_dependencies vx.x.x
)
A/go.mod: (root module importing pkg/test as local module)
module A
go 1.14
require (
A/pkg/test v0.0.0
)
replace A/pkg/test => ./pkg/test/
Now Application 'B' would like to re-use Application's A pkg/test package by simply importing it
The Adventage:
This solution lets any integrating services pull only dependencies of pkg/test module instead of importing whole application A tree
Current Solution:
Application 'B' imports pkg/test module of Application 'A' using following go.mod:
module B
go 1.14
require(
some_dependencies vx.x.x
A/pkg/test v0.0.0
)
replace A/pkg/test => gitlab.com/A/pkg/test v0.0.0-02345798575346-72cs44671e34
Now I really do not like using commit-timestamp_commit-sha approach here.
The problem:
I would like to use TAGS in order to import A/pkg/test module.
Repository A has a tag, say v2.0.0, created. When I replace v0.0.0-02345798575346-72cs44671e34 with v2.0.0, after running go mod tidy/download I am getting following error output:
reading gitlab.com/A/pkg/test/pkg/test/go.mod at revision pkg/test/v2.0.0: unknown revision pkg/test/v2.0.0
The question:
I assume bad package naming/module naming/tagging etc. might lead to such error. The question is what am I missing here in order to make it clear and work?
To take a nested go-modules package within a git repo, just add a tag to the directory that contains the go.mod file - and then append your version.
So in your case, the directory is A/pkg/test and lets say you wanted to use semver v0.0.1 you tag like so:
git tag -a "A/pkg/test/v0.0.1" -m "initial test checkin"
git push origin --tags # push all local tags
# git push origin "A/pkg/test/v0.0.1" # push a single tag
Please Note: from the Go wiki Should I have multiple modules in a single repository? - Russ Cox (the creator of vgo i.e. today's go-modules) warns:
For all but power users, you probably want to adopt the usual
convention that one repo = one module. It's important for long-term
evolution of code storage options that a repo can contain multiple
modules, but it's almost certainly not something you want to do by
default.
Related
I've used openapi-generator to generate multiple SDK separately and put it in the same parent dir:
sdks
--- bar-api
------ go.mod
--- foo-api
------ go.mod
I'd like to be able to import bar-api in my client code. Initially there's bar-api's go.mod was generated as:
module github.com/coolrepo/bar-api
go 1.13
but I couldn't fix the imports in my client code:
bar "github.com/cool-repo/bar-api"
so what I did was:
sdks
--- bar-api
------ go.mod
---go.mod
i.e., created sdks/go.mod:
module github.com/coolrepo
go 1.13
and manually edited bar-api's go.mod to:
module github.com/coolrepo/bar-api
require (
...
)
replace github.com/coolrepo => ./..
go 1.15
Is there a better way to fix it? This one seems to work but looks kinda hacky with this replace.
The “better way to fix it” is to put all of the generated Go packages inside a single module, instead of splitting up the repo into multiple modules.
rm bar-api/go.mod
go mod tidy
You only really need to split a repo into separate modules if you need to be able to tag a release for one set of packages independently from another set of packages in the same repo. For generated service APIs, that probably isn't worth the development overhead of juggling versions for multiple modules.
I wrote a program and want to encapsulate some logic.
So I did new module and pull it in my git. Link for git looks like
gitlab.xxx.ru/group/subgroup/proj
but when I tried to get it with go get, I got error
fatal: «https://xxx.ru:#gitlab.xxx.ru/group/subgroup.git/» unreachable: URL using bad/illegal format or missing URL
Go tried to load subgroup instead project.
I made folder gitlab.xxx.ru/group/subgroup/ in $GOPATH/src/ and clone my project.
And now it wrote
could not import gitlab.xxx.ru/group/subgroup/proj (no required module provides package "gitlab.xxx.ru/group/subgroup/proj")
So, if I understand correctly, in Golang 1.16 I can't just put my project in the correct directory and I can't use local packages.
How to fix loading from my GitLab and load it with ssh?
Thank you.
UDP go.mod in my proj.
module gitlab.xxx.ru/group/subgroup/proj
go 1.16
require (
golang.org/x/sys v0.0.0-20210608053332-aa57babbf139
golang.org/x/text v0.3.6
)
You may be hitting the intended behaviour from Gitlab which will prevent go from fetching the list of subgroups while trying to compute project dependencies. Since go get requests are unauthenticated, existing projects not under the root group are invisible and cannot be found.
To overcome this limitation, which Gitlab has yet to provide a solution, you can use one of the following two approaches:
Have the project located at the root and not in a subgroup (not always possible)
Leverage the .git extension as well as the replace directive in the project which imports your module (see below)
Go is able to fetch the project living under a subgroup if it knows the version control qualifier (.git). To indicate this, make sure you import the project using the following format gitlab.xxx.ru/group/subgroup/proj.git
While this alone works, it would force you to have all those .git in your imports. To overcome this, you also need a replace directive in your go.mod so you can use the original import path.
In the end, the project which imports your module should have a go.mod that look like this:
require(
gitlab.xxx.ru/group/subgroup/proj v1.7.0
)
replace(
gitlab.xxx.ru/group/subgroup/proj => gitlab.xxx.ru/group/subgroup/proj.git v1.7.0
)
I have my projects that have many packages which import each other and import outside packages. When I make a change to one of my low lever packages, and then push it to git it is fine and works in that section. When I go get it for use in another project that was working perfectly I now get this go get this error:
module declares its path as: github.com/xdg-go/scram
but was required as: github.com/xdg/scram
None of my code uses either of those directly. It looks like it automatically updated some lower external packages and broke things the used to then old import.
How do I either find out the package that is importing the wrong name or stop all auto-updates?
The go.mod file at github.com/xdg/scram declares itself as github.com/xdg-go/scram:
module github.com/xdg-go/scram
go 1.11
require (
github.com/xdg-go/stringprep v1.0.2
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2
)
The go.mod file should be updated to reflect the correct import path.
Unfortunately if this module is for you an indirect dependency, the best fix possible is to update whatever project you import that is directly importing it.
When that is not an option, a solution to this error is to clone the problematic repository locally and use the replace directive in your go.mod file:
module mymodule
replace github.com/xdg/stringprep => ../strprep
go 1.16.2
require (
github.com/divjotarora/mgo v0.0.0-20190308170442-1d451d2a3149
)
where ../strprep is where the code of the required module exists in your local machine, relative to the go.mod file of your project.
The downside of this of course is that you have to replicate this palliative fix wherever you plan to go get your modules.
Note also:
divjotarora/mgo is just a random example of a project that imports one of those packages using their old import path.
I'm using xdg/stringprep as an example because I can't find modules that import xdg/scram instead, but apparently it suffers from the same issue
Beside, you can use:
go mod why <package> to find out why a certain package is listed as a dependency of your project
go mod graph to show the full dependency graph. The output is in <package> <requirement> format
I am developing multiple microservices, which require different modules (which shall be available like modules on github, but private)
My first tests with Go were all located within one package, which gets quite messy after some time
I'm coming from the Java side of programming - with loads of packages - which keep stuff clear and clean.
(This also works with public modules like on github.com/xyz/module1 github.com/xyz/module2 github.com/xyz/module3)
I just need this for private modules - how can I do so?
This is what I tried yet:
Directory of my go sources:
C:\my\path\to\go\src\
In this dir I have multiple subdirs containing modules (actually more than listed here)
my-module1
my-module2
my-module3
For each folder I called go mod init but I get the message that
package my-module1/util is not in GOROOT (c:\go\src\my-module1\util)
which is obviously right, as my private libraries reside in C:\my\path\to\go\src\
Importing packages from github with go get ... is working without troubles (those packages will be loaded but copied to c:\go\src)
Working with all files in one folder works but is not the desired solution (I need to create multiple microservices therefore I want to be able to create different projects with custom executeables and or tests)
What am I doing wrong?
If more information is needed I will provide it - just let me know what
NOTE: packages without go file in package main cannot be installed via go install. This system looks pretty confusing to me - as modules cannot be found...
finally i created a multimodule
means i have a root folder (lets call it root) which contains a go.mod file
this defines the root module
it requires all my submodules
and it replaces the absolute path into a relative.
root
* aa
* ab
* bc
main gets defined as
module root
go 1.15
require (
root/aa v0.0.0
root/ab v0.0.0
root/bc v0.0.0
)
replace (
root/aa => ./aa
root/ab => ./ab
root/bc => ./bc
)
all the submodules have a very simple definition (this is equal for all, as long as you do not have any dependencies)
module root/aa
Using go modules, I would like to replace labix.org/v2/mgo with github.com/globalsign/mgo. The http://labix.org/mgo repository is unmaintained and has been forked to https://github.com/globalsign/mgo
my code is stored outside $GOPATH in the directory ~/git/foo
I'm using go1.11
other go modules are working (for example go list -m all lists other modules, the files go.mod and go.sum are updating automatically. See the full file below)
I've tried the following in the go.mod file:
replace labix.org/v2/mgo => github.com/globalsign/mgo v0.0.0-20181015145952-eeefdecb41b842af6dc652aaea4026e8403e62df
Running go build gives the following error:
build github.com/foo/bar: cannot find module for path labix.org/v2/mgo
The documentation in go help modules discusses Pseudo-Versions eg v0.0.0-yyy.., which I'm trying to use because the tags on https://github.com/globalsign/mgo are of the form r2018.06.15 and not v1.2.3 (semantic versioning).
In addition go help modules says:
Pseudo-versions never need to be typed by hand: the go command will accept
the plain commit hash and translate it into a pseudo-version (or a tagged
version if available) automatically. This conversion is an example of a
module query.
However I can't work out the command for generating a pseudo-version when I'm in the cloned github.com/globalsign/mgo (located at $GOPATH/src/github.com/globalsign/mgo). Hence the pseudo-version I've manually generated may be wrong.
The full go.mod file looks like:
module github.com/foo/bar
replace labix.org/v2/mgo => github.com/globalsign/mgo v0.0.0-20181015145952-eeefdecb41b842af6dc652aaea4026e8403e62df
require (
github.com/DATA-DOG/godog v0.7.8
github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38 // indirect
github.com/alecthomas/colour v0.0.0-20160524082231-60882d9e2721 // indirect
github.com/alecthomas/repr v0.0.0-20181024024818-d37bc2a10ba1 // indirect
...
)
Try the go edit -replace command, without modifying the line of the original package.
This way you don't need to know the exact pseudo version beforehand, just the commit hash id.
So in your case, don't manually modify go.mod require section of "labix.org/v2/mgo", and run this command to append your commit "eeefdec" of your fork, on top of labix version:
$ GO111MODULE="on" go mod edit -replace=labix.org/v2/mgo=github.com/globalsign/mgo#eeefdec
After the first time you build or test your module, GO will try to pull the new commit, and then will generate the "replace" line with the correct pseudo version for you. So then you'll see on the bottom of go.mod:
replace labix.org/v2/mgo => github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
When using the replace directive, leave the pseudo-version out.
It's also stated here, which points to an open issue.
Probably off-topic, but I've mostly used replace when I wanted to use a local version of some dependency. Why not import the forked lib which you'd like to use (instead of the original non-maintained one) and have mod resolve it properly?
The source being replaced (in this case labix.org/v2/mgo) also needs to be added to the require list with a version of v0.0.0 (even though it won't be downloaded). In the replace the source doesn't need a version but the target does.
However I haven't worked out how to auto-generate the pseudo version for the target (github.com/globalsign/mgo in this case), so I still needed to manually generate it.
Here's a better go.mod file:
1 module foo.bar/qux
2
3 replace labix.org/v2/mgo => github.com/globalsign/mgo v0.0.0-20181015145952-eeefdecb41b842af6dc652aaea4026e8403e62df
4
5 require (
6 github.com/DATA-DOG/godog v0.7.8
7 github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38 // indirect
<snip>
21 golang.org/x/image v0.0.0-20181116024801-cd38e8056d9b // indirect
22 labix.org/v2/mgo v0.0.0
23 )
Notice the require of labix.org on line 22; go mod tidy accepts this.
However I now come up against a different issue, the use of internal packages (referred to in issues like this: https://github.com/golang/go/issues/23970). I still haven't solved this new problem.
% go build main.go
/home/sonia/go/pkg/mod/github.com/globalsign/mgo#v0.0.0-20181015145952-eeefdecb41b842af6dc652aaea4026e8403e62df/auth.go:38:2:
use of internal package github.com/globalsign/mgo/internal/scram not allowed