I have the following project structure
.
├── README.md
├── protos
│ ├── my-proto-output.pb.go
│ └── my-proto.proto
└── grpc-backend
├── client
│ └── client.go
├── go.mod
├── go.sum
├── main.go
└── a-submodule
Inside grpc-backend/main.go I have:
package main
import (
pb "github.com/my-user/my-repo/protos"
"google.golang.org/api/option"
"google.golang.org/grpc"
)
Inside go.mod I have
go 1.13
require (
google.golang.org/api v0.14.0
google.golang.org/grpc v1.21.1
)
module github.com/my-user/my-repo/grpc-backend
However, when I run cd grpc-backend && go build I get
go: finding github.com/my-user/my-repo latest
go: finding github.com/my-user/my-repo/protos latest
go: downloading github.com/my-user/my-repo v0.0.0-20200103231607-5a754c449f99
verifying github.com/my-user/my-repo#v0.0.0-20200103231607-5a754c449f99: github.com/my-user/my-repo#v0.0.0-20200103231607-5a754c449f99: reading https://sum.golang.org/lookup/github.com/my-user/my-repo#v0.0.0-20200103231607-5a754c449f99: 410 Gone
I don't have any tags like that, I'm in a private repo only commiting to master at the moment. Furthermore, proto's are in this same git repo under the package myrepo.
Questions:
Did I organize this incorrectly?
Where did that version come from? It's not in go.sum
Your module does not include the protos package, and thus, the generated grpc source. Move your go.mod one level up, and change the module name to github.com/my-user/my-repo so the module includes all packages.
Related
I am trying to learn Go modules dependency manager but getting an error while importing auto-generated proto definitions. Have been struggling with this for 2 days. Any help will be greatly appreciated.
Directory structure:
➜ ~/go-service-skeleton> tree .
.
├── protobuf
│ └── test_service
│ ├── test_service_config.pb.go
│ └── test_service_config.proto
├── src
│ └── test_service
│ ├── go.mod
│ ├── main.go
│ └── server
│ └── server.go
Error (last error is particularly concerning):
➜ ~/go-service-skeleton/src/test_service go build
bootstrap.go:11:2: no required module provides package github.com/Sirupsen/logrus; to add it:
go get github.com/Sirupsen/logrus
bootstrap.go:12:2: no required module provides package github.com/sirupsen/logrus; to add it:
go get github.com/sirupsen/logrus
bootstrap.go:13:2: no required module provides package gopkg.in/yaml.v2; to add it:
go get gopkg.in/yaml.v2
server/server.go:8:2: package protobuf/test_service is not in GOROOT
~/go-service-skeleton/src/test_service echo $GOPATH
/Users/****/go-service-skeleton:/Users/****/go-service-skeleton/protobuf
~/go-service-skeleton/src/test_service echo $GOROOT
Within your test_service module, the only dependencies visible to the go command are the ones declared in its go.mod file.
The simplest fix is to put all of the source code you need inside the main module. Note that per https://blog.golang.org/generate, “if the containing package is intended for import by go get, once the file is generated (and tested!) it must be checked into the source code repository to be available to clients.”
Suppose I have a monorepo and there are several individual golang services:
root
└── services
├── svc1
│ ├── go.mod
│ ├── go.sum
│ └── main.go
└── svc2
├── go.mod
├── go.sum
└── main.go
Whereas svc2 will depend on svc1 in the future. Also, it is a requirement that svc1 and svc2 can be released individually.
Is there a way I can publish the module indivdually? The go mod documentation only assume that there is only repo so it doesn't provide such flexibility.
To tag a release version for a module within a subdirectory of a repository, add the subdirectory as a prefix of the tag, like svc1/v0.1.0.
Per https://golang.org/ref/mod#vcs-version:
If a module is defined in a subdirectory within the repository …, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
Envs
GO111MODULE=on
golang:1.15.2-alpine
What I want to do
I implemented some features in a local project and I want to confirm the behavior before pushing
changes to the remote repo.
But my project can't access the new package.
Error text
When docker-compose run 'go build' I got his error.
Both 'github.com/Asuha-a/URLShortener/api/pb/url' and 'github.com/Asuha-a/URLShortener/api/utility' don't exist in the remote repo because it's not pushed yet.
main.go:10:2: module github.com/Asuha-a/URLShortener/api#latest found (v0.0.0-20201114002535-3dac2ebd322a), but does not contain package github.com/Asuha-a/URLShortener/api/pb/url
main.go:12:2: module github.com/Asuha-a/URLShortener/api#latest found (v0.0.0-20201114002535-3dac2ebd322a), but does not contain package github.com/Asuha-a/URLShortener/api/utility
Project architecture
I deleted some of the files look like not related to this issue.
$ tree
.
├── api
│ ├── Dockerfile
│ ├── go.mod
│ ├── go.sum
│ ├── main.go
│ ├── pb
│ │ ├── url // new package
│ │ ├── url_grpc.pb.go
│ │ ├── url.pb.go
│ │ └── url.proto
│ ├── services
│ │ ├── url
│ │ │ ├── db
│ │ │ │ ├── settings.go
│ │ │ │ └── url.go
│ │ │ ├── Dockerfile
│ │ │ ├── go.mod
│ │ │ ├── go.sum
│ │ │ └── main.go // failed building
│ └── utility // new package
│ ├── jwt.go
│ └── url.go
├── docker-compose.yml
What I tried
I found this issue.
Importing local changes of a package without pushing code in Go
That says I can access local packages by adding 'replace MODULE_URL => /PATH/TO/MODULE/DIR'.
I edited the 'api/services/url/go.mod' like this.
module github.com/Asuha-a/URLShortener/api/services/url
go 1.15
replace github.com/Asuha-a/URLShortener/api => ../../.
require (
github.com/jackc/pgx/v4 v4.9.2 // indirect
github.com/satori/go.uuid v1.2.0
golang.org/x/crypto v0.0.0-20201112155050-0c6587e931a9 // indirect
golang.org/x/text v0.3.4 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/grpc v1.33.2
gorm.io/datatypes v1.0.0
gorm.io/driver/postgres v1.0.5
gorm.io/gorm v1.20.6
)
But I got this error:
main.go:10:2: module github.com/Asuha-a/URLShortener/api#latest found (v0.0.0-20201114002535-3dac2ebd322a, replaced by ../../.), but does not contain package github.com/Asuha-a/URLShortener/api/pb/url
main.go:12:2: module github.com/Asuha-a/URLShortener/api#latest found (v0.0.0-20201114002535-3dac2ebd322a, replaced by ../../.), but does not contain package github.com/Asuha-a/URLShortener/api/utility
Then I tried the full path '/home/asuha/go/src/github.com/Asuha-a/URLShortener/api'.
But I got this error:
main.go:10:2: github.com/Asuha-a/URLShortener/api#v0.0.0-00010101000000-000000000000: replacement directory /home/asuha/go/src/github.com/Asuha-a/URLShortener/api does not exist
main.go:12:2: github.com/Asuha-a/URLShortener/api#v0.0.0-00010101000000-000000000000: replacement directory /home/asuha/go/src/github.com/Asuha-a/URLShortener/api does not exist
I want to know
How to fix this error?
Some basics first:
A module is a set of packages versioned together.
A module has a name declared in the go.mod, e.g. github.com/anyone/someproject
A package is imported by its import path (the import path basically is package identity).
A package belonging to a module must have an import path beginning with the module name. In the example above any package belonging to the module github.com/anyone/someproject must have an import path like e.g. github.com/anyone/someproject/whatever/hierarchy/pkgname
You never need to replace packages from the same module. replace is for replacing other modules.
You declared the module
module github.com/Asuha-a/URLShortener/api/services/url
That makes the name of your module "github.com/Asuha-a/URLShortener/api/services/url" and only packages with import path like module "github.com/Asuha-a/URLShortener/api/services/url/I/really/like/deeply/nested/stuff" would belong to that package.
Clearly you utility package (btw. don't do such packages, this is fundamental code smell!) doesn't belong to your module: You probably try to import it as "github.com/Asuha-a/URLShortener/api/utiliyt" but this is not part of the module "github.com/Asuha-a/URLShortener/api/services/url".
The fact that the folder utility is "below" the go.mod file in the filesystem doesn't overrule the fact that your module declares on its first line: "All my packages have import paths starting with github.com/Asuha-a/URLShortener/api/services/url! Anything else does not belong to me!"
You probably should name your module at least module github.com/Asuha-a/URLShortener/api (if not module github.com/Asuha-a/URLShortener).
I urge you to (re-)read "How to Write Go Code" and stick to it.
Takeaway:
Modules are really just a set of packages with the same import path prefix (the module name) versioned together.
I have to create a desktop app on top of existing Go backend code, since this requirement and my knowledge in web development I'm using Wails.
With Wails CLI I generated the desktop app project in a subfolder of the back end main project.
This is how the folder structure looks like:
backend
├── [...]
├── logger
│ └── logger.go
├── gui
│ └── desktopApp
│ ├── build
│ │ └── desktopApp
│ ├── frontend
│ │ └── [...]
│ ├── go.mod
│ ├── go.sum
│ ├── main.go
│ └── project.json
├── main.go
├── go.mod
└── go.sum
The logger has been imported in the desktopApp and everything works during the development and testing via browser. But when I build the desktop app with the command wails build from the desktopApp folder I have got the following error:
go: <domain>/<owner>/backend/logger: unrecognized import path "<domain>/<owner>/backend"
I imported the backend in the desktop app like this:
/backend/gui/desktopApp/go.mod
module desktopApp
go 1.15
require (
<domain>/<owner>/backend v0.0.0
)
./gui/desktopApp/main.go
package main
import (
"github.com/leaanthony/mewn"
"github.com/wailsapp/wails"
"<domain>/<owner>/backend/logger"
)
// Using the logger package normally
How can I fix my issue?
I see that you use go-module and store code in a repo with sub-module
The problem is that you forgot add "desktopApp"-prefix
import (
"github.com/leaanthony/mewn"
"github.com/wailsapp/wails"
"desktopApp/backend/backendPackage"
)
Right way use modules👇🏼
Fix go.mod of desktopApp module
module <domain>/<owner>/desktopApp
go 1.15
fix backends import , because it belongs to desktopApp
import (
"github.com/leaanthony/mewn"
"github.com/wailsapp/wails"
"<domain>/<owner>/desktopApp/backend/backendPackage"
)
I solved my issue changing go.mod file of the nested desktop app like this:
module desktopApp
go 1.15
require (
<domain>/<owner>/backend/logger
)
// Added this line
replace <domain>/<owner>/backend/logger => ../../
So I created a simple test project to demonstrate the issue I'm having in a much larger code base here https://github.com/cleblanc189/test.go
.
├── go.mod
├── main.exe
├── main.go
├── server
│ ├── v1beta1
│ │ ├── foo.go
│ │ └── go.mod
│ └── v1ga
│ ├── foo.go
│ └── go.mod
└── service
├── v1beta1
│ ├── foo.go
│ └── go.mod
└── v1ga
├── foo.go
└── go.mod
I'd like to be able to reference service/v1beta1 from server/v1beta1 but I'm getting this error
go: test.go/src/server/v1beta1#v0.0.0 requires
test.go/src/service/v1beta1#v0.0.0: unrecognized import path "test.go/src/service/v1beta1" (https fetch: Get https://test.go/src/ser
vice/v1beta1?go-get=1: dial tcp: lookup test.go: no such host)
In foo.go from server/v1beta1 I import like this
svc "test.go/src/service/v1beta1"
and have a go.mod file like below;
require test.go/src/service/v1beta1 v0.0.0
replace test.go/src/service/v1beta1 => ../../service/v1beta1
When I try to cd src && go mod tidy I get that error above. Also if anyone knows should I even have a go.mod above the src directory?
Appreciate any help anyone can offer. This is maddening.
Your go.mod has to declare the module path/name, typically like this:
module test.go
Then all the imports in your code will be taken relative to that. When you import something relative to your declared module name, the go tool will automatically pick up the code from the right place locally.
Not sure where your src comes into play though... is everything in folder src? See this post and accompanying code for a complete example you could start from.