I am trying to learn Go modules dependency manager but getting an error while importing auto-generated proto definitions. Have been struggling with this for 2 days. Any help will be greatly appreciated.
Directory structure:
➜ ~/go-service-skeleton> tree .
.
├── protobuf
│ └── test_service
│ ├── test_service_config.pb.go
│ └── test_service_config.proto
├── src
│ └── test_service
│ ├── go.mod
│ ├── main.go
│ └── server
│ └── server.go
Error (last error is particularly concerning):
➜ ~/go-service-skeleton/src/test_service go build
bootstrap.go:11:2: no required module provides package github.com/Sirupsen/logrus; to add it:
go get github.com/Sirupsen/logrus
bootstrap.go:12:2: no required module provides package github.com/sirupsen/logrus; to add it:
go get github.com/sirupsen/logrus
bootstrap.go:13:2: no required module provides package gopkg.in/yaml.v2; to add it:
go get gopkg.in/yaml.v2
server/server.go:8:2: package protobuf/test_service is not in GOROOT
~/go-service-skeleton/src/test_service echo $GOPATH
/Users/****/go-service-skeleton:/Users/****/go-service-skeleton/protobuf
~/go-service-skeleton/src/test_service echo $GOROOT
Within your test_service module, the only dependencies visible to the go command are the ones declared in its go.mod file.
The simplest fix is to put all of the source code you need inside the main module. Note that per https://blog.golang.org/generate, “if the containing package is intended for import by go get, once the file is generated (and tested!) it must be checked into the source code repository to be available to clients.”
Related
Envs
GO111MODULE=on
golang:1.15.2-alpine
What I want to do
I implemented some features in a local project and I want to confirm the behavior before pushing
changes to the remote repo.
But my project can't access the new package.
Error text
When docker-compose run 'go build' I got his error.
Both 'github.com/Asuha-a/URLShortener/api/pb/url' and 'github.com/Asuha-a/URLShortener/api/utility' don't exist in the remote repo because it's not pushed yet.
main.go:10:2: module github.com/Asuha-a/URLShortener/api#latest found (v0.0.0-20201114002535-3dac2ebd322a), but does not contain package github.com/Asuha-a/URLShortener/api/pb/url
main.go:12:2: module github.com/Asuha-a/URLShortener/api#latest found (v0.0.0-20201114002535-3dac2ebd322a), but does not contain package github.com/Asuha-a/URLShortener/api/utility
Project architecture
I deleted some of the files look like not related to this issue.
$ tree
.
├── api
│ ├── Dockerfile
│ ├── go.mod
│ ├── go.sum
│ ├── main.go
│ ├── pb
│ │ ├── url // new package
│ │ ├── url_grpc.pb.go
│ │ ├── url.pb.go
│ │ └── url.proto
│ ├── services
│ │ ├── url
│ │ │ ├── db
│ │ │ │ ├── settings.go
│ │ │ │ └── url.go
│ │ │ ├── Dockerfile
│ │ │ ├── go.mod
│ │ │ ├── go.sum
│ │ │ └── main.go // failed building
│ └── utility // new package
│ ├── jwt.go
│ └── url.go
├── docker-compose.yml
What I tried
I found this issue.
Importing local changes of a package without pushing code in Go
That says I can access local packages by adding 'replace MODULE_URL => /PATH/TO/MODULE/DIR'.
I edited the 'api/services/url/go.mod' like this.
module github.com/Asuha-a/URLShortener/api/services/url
go 1.15
replace github.com/Asuha-a/URLShortener/api => ../../.
require (
github.com/jackc/pgx/v4 v4.9.2 // indirect
github.com/satori/go.uuid v1.2.0
golang.org/x/crypto v0.0.0-20201112155050-0c6587e931a9 // indirect
golang.org/x/text v0.3.4 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/grpc v1.33.2
gorm.io/datatypes v1.0.0
gorm.io/driver/postgres v1.0.5
gorm.io/gorm v1.20.6
)
But I got this error:
main.go:10:2: module github.com/Asuha-a/URLShortener/api#latest found (v0.0.0-20201114002535-3dac2ebd322a, replaced by ../../.), but does not contain package github.com/Asuha-a/URLShortener/api/pb/url
main.go:12:2: module github.com/Asuha-a/URLShortener/api#latest found (v0.0.0-20201114002535-3dac2ebd322a, replaced by ../../.), but does not contain package github.com/Asuha-a/URLShortener/api/utility
Then I tried the full path '/home/asuha/go/src/github.com/Asuha-a/URLShortener/api'.
But I got this error:
main.go:10:2: github.com/Asuha-a/URLShortener/api#v0.0.0-00010101000000-000000000000: replacement directory /home/asuha/go/src/github.com/Asuha-a/URLShortener/api does not exist
main.go:12:2: github.com/Asuha-a/URLShortener/api#v0.0.0-00010101000000-000000000000: replacement directory /home/asuha/go/src/github.com/Asuha-a/URLShortener/api does not exist
I want to know
How to fix this error?
Some basics first:
A module is a set of packages versioned together.
A module has a name declared in the go.mod, e.g. github.com/anyone/someproject
A package is imported by its import path (the import path basically is package identity).
A package belonging to a module must have an import path beginning with the module name. In the example above any package belonging to the module github.com/anyone/someproject must have an import path like e.g. github.com/anyone/someproject/whatever/hierarchy/pkgname
You never need to replace packages from the same module. replace is for replacing other modules.
You declared the module
module github.com/Asuha-a/URLShortener/api/services/url
That makes the name of your module "github.com/Asuha-a/URLShortener/api/services/url" and only packages with import path like module "github.com/Asuha-a/URLShortener/api/services/url/I/really/like/deeply/nested/stuff" would belong to that package.
Clearly you utility package (btw. don't do such packages, this is fundamental code smell!) doesn't belong to your module: You probably try to import it as "github.com/Asuha-a/URLShortener/api/utiliyt" but this is not part of the module "github.com/Asuha-a/URLShortener/api/services/url".
The fact that the folder utility is "below" the go.mod file in the filesystem doesn't overrule the fact that your module declares on its first line: "All my packages have import paths starting with github.com/Asuha-a/URLShortener/api/services/url! Anything else does not belong to me!"
You probably should name your module at least module github.com/Asuha-a/URLShortener/api (if not module github.com/Asuha-a/URLShortener).
I urge you to (re-)read "How to Write Go Code" and stick to it.
Takeaway:
Modules are really just a set of packages with the same import path prefix (the module name) versioned together.
I have to create a desktop app on top of existing Go backend code, since this requirement and my knowledge in web development I'm using Wails.
With Wails CLI I generated the desktop app project in a subfolder of the back end main project.
This is how the folder structure looks like:
backend
├── [...]
├── logger
│ └── logger.go
├── gui
│ └── desktopApp
│ ├── build
│ │ └── desktopApp
│ ├── frontend
│ │ └── [...]
│ ├── go.mod
│ ├── go.sum
│ ├── main.go
│ └── project.json
├── main.go
├── go.mod
└── go.sum
The logger has been imported in the desktopApp and everything works during the development and testing via browser. But when I build the desktop app with the command wails build from the desktopApp folder I have got the following error:
go: <domain>/<owner>/backend/logger: unrecognized import path "<domain>/<owner>/backend"
I imported the backend in the desktop app like this:
/backend/gui/desktopApp/go.mod
module desktopApp
go 1.15
require (
<domain>/<owner>/backend v0.0.0
)
./gui/desktopApp/main.go
package main
import (
"github.com/leaanthony/mewn"
"github.com/wailsapp/wails"
"<domain>/<owner>/backend/logger"
)
// Using the logger package normally
How can I fix my issue?
I see that you use go-module and store code in a repo with sub-module
The problem is that you forgot add "desktopApp"-prefix
import (
"github.com/leaanthony/mewn"
"github.com/wailsapp/wails"
"desktopApp/backend/backendPackage"
)
Right way use modules👇🏼
Fix go.mod of desktopApp module
module <domain>/<owner>/desktopApp
go 1.15
fix backends import , because it belongs to desktopApp
import (
"github.com/leaanthony/mewn"
"github.com/wailsapp/wails"
"<domain>/<owner>/desktopApp/backend/backendPackage"
)
I solved my issue changing go.mod file of the nested desktop app like this:
module desktopApp
go 1.15
require (
<domain>/<owner>/backend/logger
)
// Added this line
replace <domain>/<owner>/backend/logger => ../../
So I created a simple test project to demonstrate the issue I'm having in a much larger code base here https://github.com/cleblanc189/test.go
.
├── go.mod
├── main.exe
├── main.go
├── server
│ ├── v1beta1
│ │ ├── foo.go
│ │ └── go.mod
│ └── v1ga
│ ├── foo.go
│ └── go.mod
└── service
├── v1beta1
│ ├── foo.go
│ └── go.mod
└── v1ga
├── foo.go
└── go.mod
I'd like to be able to reference service/v1beta1 from server/v1beta1 but I'm getting this error
go: test.go/src/server/v1beta1#v0.0.0 requires
test.go/src/service/v1beta1#v0.0.0: unrecognized import path "test.go/src/service/v1beta1" (https fetch: Get https://test.go/src/ser
vice/v1beta1?go-get=1: dial tcp: lookup test.go: no such host)
In foo.go from server/v1beta1 I import like this
svc "test.go/src/service/v1beta1"
and have a go.mod file like below;
require test.go/src/service/v1beta1 v0.0.0
replace test.go/src/service/v1beta1 => ../../service/v1beta1
When I try to cd src && go mod tidy I get that error above. Also if anyone knows should I even have a go.mod above the src directory?
Appreciate any help anyone can offer. This is maddening.
Your go.mod has to declare the module path/name, typically like this:
module test.go
Then all the imports in your code will be taken relative to that. When you import something relative to your declared module name, the go tool will automatically pick up the code from the right place locally.
Not sure where your src comes into play though... is everything in folder src? See this post and accompanying code for a complete example you could start from.
I have the following project structure
.
├── README.md
├── protos
│ ├── my-proto-output.pb.go
│ └── my-proto.proto
└── grpc-backend
├── client
│ └── client.go
├── go.mod
├── go.sum
├── main.go
└── a-submodule
Inside grpc-backend/main.go I have:
package main
import (
pb "github.com/my-user/my-repo/protos"
"google.golang.org/api/option"
"google.golang.org/grpc"
)
Inside go.mod I have
go 1.13
require (
google.golang.org/api v0.14.0
google.golang.org/grpc v1.21.1
)
module github.com/my-user/my-repo/grpc-backend
However, when I run cd grpc-backend && go build I get
go: finding github.com/my-user/my-repo latest
go: finding github.com/my-user/my-repo/protos latest
go: downloading github.com/my-user/my-repo v0.0.0-20200103231607-5a754c449f99
verifying github.com/my-user/my-repo#v0.0.0-20200103231607-5a754c449f99: github.com/my-user/my-repo#v0.0.0-20200103231607-5a754c449f99: reading https://sum.golang.org/lookup/github.com/my-user/my-repo#v0.0.0-20200103231607-5a754c449f99: 410 Gone
I don't have any tags like that, I'm in a private repo only commiting to master at the moment. Furthermore, proto's are in this same git repo under the package myrepo.
Questions:
Did I organize this incorrectly?
Where did that version come from? It's not in go.sum
Your module does not include the protos package, and thus, the generated grpc source. Move your go.mod one level up, and change the module name to github.com/my-user/my-repo so the module includes all packages.
I have some problems with how Go importing and modules works.
I have a project with this structure:
My-Project
|- Program 1
|- main.go
|- go.mod
|- Program 2
|- main.go
|- go.mod
|- Support
|- go_file.go
The two programs have different module "definition". I need to include the "Support" folder, which is shared between the two programs. I tried
import "My-Project/Support"
inside Program1's main.go, but this is not working, because Support can’t be resolved. How can I do it?
As given, your Support/go_file.go is not in any module so it's not importable using Go modules.
Either use a single module for all three packages or use three separate modules (i.e. put Support in it's own module). You could also use sub-modules.
Single module:
For a single module you'd do something like:
└── My-Project
├── go.mod
├── Program\ 1
│ └── main.go
├── Program\ 2
│ └── main.go
└── Support
└── go_file.go
And the project level go.mod would be something like:
module github.com/My-Name/My-Project
(substitute with appropriate module name) and then both main packages would import the support package with:
import "github.com/My-Name/My-Project/Support"
Note Go packages normally do not start with capitals like this.
More idiomatic for such a setup with a single repository which mixes packages and commands is a layout that puts commands into a cmd subdirectory (e.g. see the golang.org/x/tools layout). This would look something like this:
└── My-Project
├── cmd
│ ├── program1
│ │ └── main.go
│ └── program2
│ └── main.go
├── go.mod
├── internal
│ └── support
│ └── support.go
└── other-package
└── some-other-package.go
Here the support package is in an internal sub-directory which makes it non-importable from outside My-Project where-as the other-package is importable by anyone.
Again, go.mod would define the module name which sets the prefix to be used for all import statements.
Separate modules:
To use separate modules you'd do something like:
└── My-Project
├── Program\ 1
│ ├── go.mod
│ └── main.go
├── Program\ 2
│ ├── go.mod
│ └── main.go
└── Support
├── go_file.go
└── go.mod
Here My-Project/Program 1/go.mod would be something like:
module "github.com/My-Name/My-Project/Program 1"
require (
github.com/My-Name/My-Project/Support
)
The imports would be the same as for the single module example.
Similarly for Program 2. For Support/go.mod it'd look something like:
module github.com/My-Name/My-Project/Support
For local development you'll probably also want/need a replace directive in each program's go.mod to use your local working copy of the support module (go mod edit -replace github.com/My-Name/My-Project/Support=../Support should add this).