Im getting the following error once i run the protocol command to generate pb.go file
But I can generate the pb.go file. how do I mitigate the following error
Missing 'go_package' option in "job.proto",
please specify it with the full Go package path as
a future release of protoc-gen-go will require this be specified.
"Import path" is the path another package would use to import the generated code, e.g. github.com/me/myproject/model or You can simply define the import path based on your preference.
You can simply define your optional import path as follows
option go_package = ".;<Your_Import_path>";
For example, we can assume the package path as "/pub", So the statement as follows.
option go_package = ".;pub";
Then you can simply execute the protoc command to generate the pb.go file
protoc -I=<ABS_PATH_OUTPUT_DIR> --go_out=<ABS_PATH_PROTO_FILE>
Related
From here:
https://developers.google.com/protocol-buffers/docs/reference/go-generated#package
It is written:
There is no correlation between the Go import path and the package
specifier in the .proto file. The latter is only relevant to the
protobuf namespace, while the former is only relevant to the Go
namespace. Also, there is no correlation between the Go import path
and the .proto import path.
So, if I understand correctly - the "package" in your .proto file is just for importing other .proto's whereas the "option go_package" is what determines the package name generated into your .go files.
My question is why was this done? If you have the following in your .proto file:
syntax = "proto3";
package cool.app.v1;
option go_package = "github.com/some-user/cool-app/pkg/app/proto";
message Test{}
Then in other proto files you can reference Test using cool.app.v1.Test. But, then in the generated golang file - you end up with the following package (that last string after the last "/"):
package proto
Why wouldn't protoc just use the package specified in the .proto file? Are there other languages generated by protoc that require the same "option" override for the package name?
I'm trying to make my Go project using hexagonal architecture as described here.
In my project I'm using a gRPC communication generated with protoc from .proto file.
The directories structure:
|- grpc.proto
|-internal
|-core
|-domain
|-services
|- grpcprotocol
And my grpc.proto file has go_package option that points to a specific directory in my Go project
syntax = "proto3";
option go_package = "github.com/myuser/myrepo/internal/core/services/grpcprotocol";
...
Using protoc --go_out=internal/core/domain --go_opt=paths=source_relative --go-grpc_out=internal/core/services/grpcprotocol --go-grpc_opt=paths=source_relative ports.proto I'm able to generate both grpc.pb.go file in internal/core/domain directory and grpc_grpc.pb.go file inside internal/core/services/grpcprotocol directory.
However, grpc.pb.go has a go package named grpcprotocol while it should have a package named domain (I also use other types defined there in separate Go files).
Also grpc_grpc.pb.go file has code that uses types defined in grpc.pb.go without imports (it treats it as if it was defined in the same package).
Is it possible to split those two files into separate Go packages and enforce code that's in grpc_grpc.pb.go to import types from domain directory instead of treating them as defined in the same package?
Your best solution here is too separate the code that you want in grpcprotocol and the one you want into domain, into separate files. Such as:
domain.proto
syntax = "proto3";
package domain;
option go_package = "github.com/myuser/myrepo/internal/core/domain";
//...
grpc.proto
syntax = "proto3";
package grpcprotocol;
option go_package = "github.com/myuser/myrepo/internal/core/services/grpcprotocol";
//...
Then you could import your domain.proto in your grpc.proto, by simply writing import "domain.proto";, provide a --proto_path if domain.proto and grpc.proto are not in the same directory. And finally to reference an object from domain.proto in grpc.proto you can write:
domain.AnObject
After that you can take advantage of the --go_opt=module and --go-grpc_opt=module, to strip the module name in go_package and generate the code at the right place. Like:
protoc --go_out=. --go_opt=module=github.com/myuser/myrepo --go-grpc_out=. --go-grpc_opt=module=github.com/myuser/myrepo *.proto
What this will do is that, it will remove github.com/myuser/myrepo from each go_package and basically start from the root of your module. That's why you can do a --go_out=. and --go-grpc_out=..
Hope that helps, let me know how I can further improve my answer.
Notes
Protobuf package and go_package are not the same. The former is only used for protobuf in order to give context and it extends the qualified name. The go_package is used during go code generation.
The package in the proto file is optional, it makes things more clear and nobody can misuse you proto file without specifying the fully qualified name (little bit safer, if named properly).
There is gRPC api https://github.com/LTD-Beget/antivirus/tree/master/proto/v1
I want to generate files for GO from this API, but there is no way to add go_package option.
In old version of protoc was cmd option as import_path, and it can can be generated for example by next command
root#d887087e9fa2:/shared/api# protoc -I/usr/local/include -I/shared/api --go_out=import_path=antivirus_proto_v1,M/shared/api/antivirus/proto/v1/antivirus.proto=antivirus_proto_v1,M/shared/api/antivirus/proto/v1/structures.proto=antivirus_proto_v1,plugins=grpc:internal/grpc /shared/api/antivirus/proto/v1/antivirus.proto /shared/api/antivirus/proto/v1/structures.proto
But with new version of protoc
protoc-gen-go: no such flag -import_path
--go_out: protoc-gen-go: Plugin failed with status code 1.
follow the guidelines
In order to generate Go code, the Go package's import path must be provided for every .proto file (including those transitively depended upon by the .proto files being generated). There are two ways to specify the Go import path:
by declaring it within the .proto file, or
by declaring it on the command line when invoking protoc.
but when i declaring it on cmd
root#d887087e9fa2:/shared/api# protoc -I/usr/local/include -I/shared/api --go_out=generated --go-grpc_out=generated --go_opt=M/shared/api/antivirus/proto/v1/antivirus.proto=antivirus/antivirus --go_opt=M/shared/api/antivirus/proto/v1/structures.proto=antivirus/structures /shared/api/antivirus/proto/v1/antivirus.proto /shared/api/antivirus/proto/v1/structures.proto
protoc-gen-go: unable to determine Go import path for "antivirus/proto/v1/structures.proto"
Please specify either:
• a "go_package" option in the .proto source file, or
• a "M" argument on the command line.
See https://developers.google.com/protocol-buffers/docs/reference/go-generated#package for more information.
what am i missing?
I am unable to compile google's fhir proto for go
1. I was able to generate annotations.pb.go with warnings which can be resolved
protoc --proto_path=proto --go_out=. proto/annotations.proto
2020/05/27 12:42:17 WARNING: Missing 'go_package' option in "annotations.proto",
please specify it with the full Go package path as
a future release of protoc-gen-go will require this be specified.
See https://developers.google.com/protocol-buffers/docs/reference/go-generated#package for more information.
2. Unfortunately, I am unable to resolve fixing issues surrounding file not found.
Example the profile_config.proto which contained the following imports
import "proto/annotations.proto";
import "proto/r4/core/codes.proto";
import "proto/r4/core/datatypes.proto";
Attempting to executed resulted in "not found"
protoc --proto_path=proto --go_out=. proto/profile_config.proto
proto/annotations.proto: File not found.
proto/r4/core/codes.proto: File not found.
proto/r4/core/datatypes.proto: File not found.
Perhaps these proto files can only be used only with java and any other language would require modification on the files.
Regarding 1., you need to edit every .proto you want to compile and add the go_package option. For instance:
option go_package = "github.com/my-org/my-proj/go/gen/fhir/proto"
Regarding 2., you are setting --proto_path=proto, which leads protoc to search for proto/annotations.proto into the following path:
./proto/proto/annotations.proto
If you don't set this option or set it to --proto_path=. you will be able to compile.
I also suggest taking a look at this pull-request.
I am fairly certain this is operator error and I am at the point I am not thinking clearly.
Here is the setup:
$GOPATH/src/github.com/<company>/<service a>/proto/a.proto
$GOPATH/src/github.com/<company>/<service b>/proto/b.proto
etc.
Now in the proto file I am using imports similar to go (perhaps the issue) such that a.proto has:
import "github.com/<company>/<service b>/b.proto"
I have possibly two separate issues.
I cannot get the import to compile properly using go:generate protoc
I cannot get the output a.pb.go file to be placed in the $GOPATH/src/github.com/<company>/<service a>/proto/ path.
I have attempted multiple configurations probably not in the correct combination.
Using option go_package = "github.com/<company>/<service b>/proto" in each .proto file
Multiple variations of go generate;
go:generate protoc --proto_path=.:$GOPATH/src --go_out=$GOPATH/src a.proto
go:generate protoc --proto_path=.:$GOPATH/src --go_out=. a.proto
go:generate protoc --go_out=import_prefix=github.com/<company>/:. api.proto
I clearly have a poor understanding on how protoc looks at import paths and file outputs. Anyone point me in the direction of what I am doing wrong?
Thanks!
Update #1
In a.proto;
option go_package = "github.com/<company>/<service a>/proto";
import "github.com/<company>/<service b>/proto/b.proto";
and the go generate;
//go:generate protoc --proto_path=$GOPATH/src --go_out=$GOPATH/src/github.com/<company>/<service a>/proto a.proto
Which is called from a go file in the proto directory with the a.proto.
I received the error;
a.proto: File does not reside within any path specified using --proto_path (or -I). You must specify a --proto_path ch encompasses this file. Note that the proto_path must be an exact prefix of the .proto file names -- protoc is too dumb to figure out when two paths (e.g. absolute and relative) are equivalent (it's harder than you think).
I have confirmed $GOPATH is to the expected location.
Solution
Thanks to Shivam Jindal for pointing me in the correct direction. The import is exactly as described in his solution. The output was a problem of my misusing both --go_out and option go_package. I used the go_package to specify the output location and --go_out to specify the root similar to --proto_path. Now everything works.
option go_package = "github.com/<company>/<service a>/proto";
and
//go:generate protoc --proto_path=$GOPATH/src/ --go_out=$GOPATH/src/ $GOPATH/src/github.com/<company>/<service a>/proto/a.proto
Thanks!
Firstly, option go_package is not meant for other dependency import at all, it's the Go package name where the new proto bindings for Go (a.pb.go file) will be placed.
Now coming to the output file location, I can see you are using go-generate. Firstly it depends from which directory you are invoking that if the path used in --go_out= is relative path. I would say use absolute paths. If you want to put the output file in that location you mentioned, use --go_out=$GOPATH/src/github.com/<company>/<service a>/proto/ in go-generate.
To correctly import the other file b.proto in your a.proto use the fully qualified import path as you have done. Just that use --proto_path $GOPATH/src in go-generate. Also please update the question with the errors you are seeing in case it does not work.
Please see this for more information on import paths.