This question already has answers here:
How to use a module that is outside of "GOPATH" in another module?
(2 answers)
Closed 4 years ago.
I'm building an application using a 'Micro Service' architecture.
This means that I have different applications.
The truth is that some logic is in a 'shared' library.
See the following directory structure:
ROOT/
├── Service 1/
│ ├── src
│ ├──── app.go
├── Service 2/
│ ├── src
│ ├──── app.go
└── Lib/
├── Lib 1
│ ├── src
│ ├──── app.go
Service 1, Service 2, and Lib 1, are all initialized with the go mod command.
For Service 1, this resulted in a go.mod file with the following contents.
module github.com/kevin-de-coninck/datalytics/services/serviceOne
For Service 2, this resulted in a go.mod file with the following contents.
module github.com/kevin-de-coninck/datalytics/services/serviceTwo
For Lib 1, this resulted in a go.mod file with the following contents.
module github.com/kevin-de-coninck/datalytics/lib/libOne
The import statements of Service 1 contains a reference to the Lib 1
import (
"github.com/kevin-de-coninck/datalytics/lib/libOne"
)
However, when I try to build the application, the following output is showed:
go: finding github.com/kevin-de-coninck/datalytics/lib/libOne latest
go: finding github.com/kevin-de-coninck/datalytics/lib latest
go: finding github.com/kevin-de-coninck/datalytics latest
build github.com/kevin-de-coninck/datalytics/services/serviceOne/src:
cannot find module for path github.com/kevin-de-coninck/datalytics/lib/libOne
How can I resolve this issue so that i can use my LibOne package without making it public or whithout copying it across all the services?
Kind regards
After some fiddling around, I have found my answer.
It seems that the 'go.mod' and 'go.sum' file(s) should be placed in the 'src' directory.
After doing that, the application can be build and executed.
Related
I have a golang application structure like this:
.
├── calc
│ ├── go.mod
│ ├── main.go
│ └── Makefile
├── go.mod
├── LICENSE
├── num
│ ├── go.mod
│ └── num.go
└── README.md
Where calc is an "application" where I'm importing the num package to add 2 numbers.
calc/go.mod
go 1.15
require github.com/github_username/goapp/num v0.2.1
num/go.mod
module github.com/github_username/goapp/num/v0.2.1
go 1.15
go.mod
module github.com/github_username/goapp/v0.2.1
go 1.15
When in /calc, and I run go run main.go, I get the following:
go: github.com/github_username/goapp/num#v0.2.1: reading github.com/github_username/goapp/num/num/go.mod at revision num/v0.2.1: unknown revision num/v0.2.1
What am I doing wrong? The github repo has the annotated tags.
For further context, I'm mimicking a production setup where we have six different mini golang services in folders such as calc, calc2, etc. where each "calc" service has a go.mod file.
module github.com/github_username/goapp/num/v0.2.1
Is nonsense. The semver version tag "v0.2.1" does not belong into the module name.
(Note that for major versions > 1, e.g. 4.3.1, the major version becomes part of the name like in module github.com/user/proj/folder/v4).
And one more: There are no source code belonging to the root go.mod so this module makes no sense whatsoever.
You really should not make that many modules.
Are you working with private repositories?
if yes, then you need to configure OAuth authentication:
export GITHUB_TOKEN=MY_GITHUB_TOKEN
git config --global url."https://${GITHUB_TOKEN}:x-oauth-basic#github.com/".insteadOf "https://github.com/"
Now, if you don't have the necessity to use private repositories!
Turn your repositories to the public that will solve the problem too.
I am having issues trying to deploy a Google cloud function in Go 1.11 using Go modules. I have have the following code structure in my GOPATH:
└── example
├── models
│ ├── go.mod
│ └── models.go
└── load
├── fn.go
├── go.mod
├── go.sum
└── vendor
└── ....
the load/go.mod looks like the following:
module github.com/example/load
require (
github.com/example/models v0.0.0
)
replace github.com/example/models => ../models
When I try to deploy the function using the command
gcloud functions deploy load-data --entry-point GCSNewFileTrigger --runtime go111 --trigger-resource new_data --trigger-event google.storage.object.finalize
I get the following error:
Deploying function (may take a while - up to 2 minutes)...failed.
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: go: parsing /models/go.mod: open /models/go.mod: no such file or directory
go: error loading module requirements
The commands go mod vendor and go mod verify run successfully locally and I can see my local package models in the vendor folder of load
Modules are preferred by the builder over vendor. If there is a go.mod, modules will be used. When you upload your function, it only includes the directory with your function at the root, not any directories one level up. So, when there is a go.mod and you have a replace directive pointing one level up, it will not work.
The solution is to vendor and not upload the go.mod/go.sum files. When using gcloud, you can create a .gcloudignore file to do this for you. See https://cloud.google.com/functions/docs/concepts/go-runtime#specifying_dependencies for more detail.
Disclaimer: I work at Google and on this product.
I'm writing a library package which depends on certain imports but I'm not sure how to handle it correctly.
Let me start with the directory structure:
go/src/github.com/
├── developer A/
│ ├── project 1
│ └── project 2
│
└── developer B/
└── project 3
└── vendor
└── project 4
Project 1 is a library. It is used in project 2 and gets pulled into 2s vendor folder. Therefore, project 1 should contain all its dependencies such that clients (e.g. project 2) don't need to pull them as well. However, one dependency of project 1 is project 4 which is contained in project 3s vendor folder. It is essential that this dependency is always exactly the version vendored by project 3. Go doesn't allow imports to point to packages inside vendor folders, so I can't import it directly from there. How do I solve this with govendor?
Go won't let you reach into another project's vendor directory. It sounds like your intention is to ensure versions. This is what go modules are tasked to do. Take a look at the wiki for more information.
I have a repository structure as follows :-
xyz/src
1. abc
- p
- q
- r
2. def
- t
- u
- v
3. etc
- o
- m
- n
I have created a .mod file in src and run go build ./...
Except for local packages everything is fine. So if abc/p is being used in def then it throws the following exception :- cannot find module providing package abc/p. The idea behind keeping the .mod file in src package was to make sure the path is being found from where the mod file is located. Can anyone suggest where should the mod file ideally should be? also i tried placing it one directory above in xyz but still same issue as well as i created one for each sub directory. I am bit confused on this. Will I have to create separate repository for abc and etc. But considering gopath which earlier used to work for the same I think module should also be able to do the same. Any suggestions?
The most common and easiest approach is a single go.mod file in your repository, where that single go.mod file is placed in the root of your repository.
Russ Cox commented in #26664:
For all but power users, you probably want to adopt the usual convention that one repo = one module. It's important for long-term evolution of code storage options that a repo can contain multiple modules, but it's almost certainly not something you want to do by default.
The Modules wiki says:
For example, if you are creating a module for a repository
github.com/my/repo that will contain two packages with import paths
github.com/my/repo/foo and github.com/my/repo/bar, then the first
line in your go.mod file typically would declare your module path as
module github.com/my/repo, and the corresponding on-disk structure
could be:
repo/
├── go.mod <<<<< Note go.mod is located in repo root
├── bar
│ └── bar.go
└── foo
└── foo.go
In Go source code, packages are imported using the full path including
the module path. For example, if a module declared its identity in its
go.mod as module github.com/my/repo, a consumer could do:
import "example.com/my/repo/bar"
That imports package bar from the module github.com/my/repo.
I have a single go.mod in the root of my go application. I am using the following structure inspired by Kat Zien - How Do You Structure Your Go Apps
At the minute one of my applications looks like this
.
├── bin
├── cmd
│ ├── cli
│ └── server
│ └── main.go
├── pkg
│ ├── http
│ │ └── rest
| │ # app-specific directories excluded
│ └── storage
│ └── sqlite
All packages are imported via their full path, i.e. import "github.com/myusername/myapp/pkg/http/rest" otherwise it causes problems all over the place and this was the one change I had to make going from $GOPATH to go mod.
go mod then handles all the dependencies it discovers properly as far as I've discovered so far.
I try understand how to organize go code using "internal" packages. Let me show what the structure I have:
project/
internal/
foo/
foo.go # package foo
bar/
bar.go # package bar
main.go
# here is the code from main.go
package main
import (
"project/internal/foo"
"project/internal/bar"
)
project/ is outside from GOPATH tree. Whatever path I try to import from main.go nothing works, the only case working fine is import "./internal/foo|bar". I think I do something wrong or get "internal" package idea wrong in general. Could anybody make things clearer, please?
UPDATE
The example above is correct the only what I need was to place project/ folder under $GOPATH/src. So the thing is import path like the project/internal/foo|bar is workable if we only import it from project/ subtree and not from the outside.
With modules introduction in Go v1.11 and above you don't have to specify your project path in $GOPATH/src
You need to tell Go about where each module located by creating go.mod file. Please refer to go help mod documentation.
Here is an example of how to do it:
project
| go.mod
| main.go
|
\---internal
+---bar
| bar.go
| go.mod
|
\---foo
foo.go
go.mod
project/internal/bar/go.mod
module bar
go 1.14
project/internal/bar/bar.go
package bar
import "fmt"
//Bar prints "Hello from Bar"
func Bar() {
fmt.Println("Hello from Bar")
}
project/internal/foo/go.mod
module foo
go 1.14
project/internal/foo/foo.go
package foo
import "fmt"
//Foo prints "Hello from Foo"
func Foo() {
fmt.Println("Hello from Foo")
}
project/main.go
package main
import (
"internal/bar"
"internal/foo"
)
func main() {
bar.Bar()
foo.Foo()
}
Now the most important module
project/go.mod
module project
go 1.14
require internal/bar v1.0.0
replace internal/bar => ./internal/bar
require internal/foo v1.0.0
replace internal/foo => ./internal/foo
Couple things here:
You can have any name in require. You can have project/internal/bar if you wish. What Go think it is URL address for the package, so it will try to pull it from web and give you error
go: internal/bar#v1.0.0: malformed module path "internal/bar": missing dot in first path element
That is the reason why you need to have replace where you tell Go where to find it, and that is the key!
replace internal/bar => ./internal/bar
The version doesn't matter in this case. You can have v0.0.0 and it will work.
Now, when you execute your code you will have
Hello from Bar
Hello from Foo
Here is GitHub link for this code example
The packages have to be located in your $GOPATH in order to be imported. The example you gave with import "./internal/foo|bar" works because it does a local-import. internal only makes it so code that doesn't share a common root directory to your internal directory can't import the packages within internal.
If you put all this in your gopath then tried to import from a different location like OuterFolder/project2/main.go where OuterFolder contains both project and project2 then import "../../project/internal/foo" would fail. It would also fail as import "foo" or any other way your tried due to not satisfying this condition;
An import of a path containing the element “internal” is disallowed if
the importing code is outside the tree rooted at the parent of the
“internal” directory.
Now if you had the path $GOPATH/src/project then you could do import "foo" and import "bar" from within $GOPATH/src/project/main.go and the import would succeed. Things that are not contained underneath project however would not be able to import foo or bar.
below way is more scalable, especially when you plan to build multiple binaries
github.com/servi-io/api
├── cmd/
│ ├── servi/
│ │ ├── cmdupdate/
│ │ ├── cmdquery/
│ │ └── main.go
│ └── servid/
│ ├── routes/
│ │ └── handlers/
│ ├── tests/
│ └── main.go
├── internal/
│ ├── attachments/
│ ├── locations/
│ ├── orders/
│ │ ├── customers/
│ │ ├── items/
│ │ ├── tags/
│ │ └── orders.go
│ ├── registrations/
│ └── platform/
│ ├── crypto/
│ ├── mongo/
│ └── json/
The folders inside cmd/ represents the number of binaries you want to build.
for more
Also to check: When you are using your externally imported object types: make sure you are prefixing them with the namespace they're in. As a golang newbie, I didn't know I had to do that, and was wondering why is VS Code just removing my import (for not being used) when I saved. It's because I had to prefix the imported object with the namespace name:
Example:
import (
"myInternalPackageName" // works fine as long as you follow all instructions in this thread
)
//Usage in code:
myInternalPackageName.TheStructName // prefix it, or it won't work.
If you don't put the namespace prefix before the object/struct name, VS code just removes your import for being unused, and then you still have the error: "Can't find TheStructName"... That was very confusing, and I had to do a build without VS code through the command line to understand that.
The point is: I had to specify the package prefix when actually using the struct from that internal package.
If you don't want to use the qualifier prefix when using imported objects, use:
import . "thePath" // Can use contents without prefixing.
Reference: What does the '.' (dot or period) in a Go import statement do?