I'm new to Protocol Buffers and gRPC stuff. Now I'm trying to build a client/server architecture with grpc + grpc-gateway in Go.
I tried to follow some examples but I always end up with the same problem.
After generating the code with protoc i run go build and I get this error:
proto/helloworld/hello_world.pb.gw.go:64:2: cannot use msg (type *HelloReply) as type protoreflect.ProtoMessage in return argument:
*HelloReply does not implement protoreflect.ProtoMessage (missing ProtoReflect method)
proto/helloworld/hello_world.pb.gw.go:98:2: cannot use msg (type *HelloReply) as type protoreflect.ProtoMessage in return argument:
*HelloReply does not implement protoreflect.ProtoMessage (missing ProtoReflect method)
This is go.mod:
module github.com/riccardopedrielli/grpc-gateway-test
go 1.15
require (
github.com/golang/protobuf v1.4.3
github.com/grpc-ecosystem/grpc-gateway/v2 v2.2.0
google.golang.org/genproto v0.0.0-20210207032614-bba0dbe2a9ea
google.golang.org/grpc v1.35.0
google.golang.org/protobuf v1.25.0
)
This is hello_world.proto:
syntax = "proto3";
package helloworld;
import "google/api/annotations.proto";
option go_package = "github.com/riccardopedrielli/grpc-gateway-test/proto/helloworld";
// Here is the overall greeting service definition where we define all our endpoints
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {
option (google.api.http) = {
get: "/v1/example/echo/{name}"
};
}
}
// The request message containing the user's name
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
This is the link to the repository: https://github.com/riccardopedrielli/grpc-gateway-test
A difference I see between the generated go files is that they are importing different protobuf libraries.
The one generated by protoc-gen-go imports github.com/golang/protobuf/proto.
The one generated by protoc-gen-grpc-gateway imports google.golang.org/protobuf/proto.
Could this be the cause of the problem?
Still it's not clear to me which one should be used and how to force the same in both the generators.
I'm new to grpc and quite lost at this point, so I could have omitted some important informations. Any suggestion will be welcomed.
Thank you
Ok I solved the issue.
I had installed protoc via snap and the stable channel had version 3.11.4
Now I upgraded to 3.14.0 and everything is working well.
For generating the stubs, we can use either protoc or buf. protoc is the more classic generation experience used widely in the industry. Still, it has a pretty steep learning curve. buf is a newer tool built with user experience and speed in mind. It also offers linting and breaking change detection, and something protoc doesn’t provide.
You should check out the tutorial series on gRPC-Gateway, i.e., https://grpc-ecosystem.github.io/grpc-gateway/docs/tutorials/. Also, you can refer to my simple hello world program, which uses gRPC-Gateway, i.e., https://github.com/iamrajiv/helloworld-grpc-gateway.
Related
I am using the jackc/pgx driver alongside the GORM library to interface with a PostgreSQL database.
I have an instance where I have to check the PostgreSQL error code and handle a certain error type differently. When using the pgx driver, the GORM methods return a *pgconn.PgError type as the error, which contains a field with the specific error code.
In order to access that field, I must cast the error to a *pgconn.PgError, but for some reason this is failing:
res := tx.Take(&f, "id = ?", id)
if res.Error != nil {
if pqErr, ok := res.Error.(*pgconn.PgError); ok {
// does not reach here
} else {
fmt.Printf("Error type: %T\n", res.Error)
// Output: "Error type: *pgconn.PgError"
}
}
Notes:
The pgx and pgconn packages are inside the same project, so it's not the case that they are returning different versions of a type with the same name. In other words, I only have one import in my go.mod.
The returned value is not nil.
A debugger reveals that the type is a *pgconn.PgError.
You've solved your own issue, but here's some perhaps helpful background, and how I found the source.
Packages of the same name can exist in the same program, so long as they have different import paths. For example, the standard library has both math/rand and crypto/rand, each called rand. This is the first hint of how *pgconn.PgError and *pgconn.PgError are not the same: they come from different import paths.
When modules in Go make major revisions, they are supposed to change their import path. This is to preserve backwards compatibility with respect to import paths. Note that this is usually done by updating the module declaration in the go.mod file, rather than actually moving the code into a sub-directory. For example, see this commit where pgx was bumped from v4 to v5. This is the second hint: code from the pgx project is available under multiple import paths (due to the multiple major versions).
With this background in mind, I used the git tags to view the repository at the latest v4.x.x release. I noticed that oddly, the pgconn package did not exist in v4. This seemed to rule out the idea of a github.com/jackc/pgx/v4/pgconn vs github.com/jackc/pgx/v5/pgconn conflict. I then Google searched for "pgconn" and found the github.com/jackc/pgconn repository, where I saw in the README:
This version is used with pgx v4. In pgx v5 it is part of the https://github.com/jackc/pgx repository.
From the other information you've given, your mistake may have been using the import path "github.com/jackc/pgx/pgconn". As shown in the example code for pgx, the current import path you should be using for the base module is "github.com/jackc/pgx/v5", and packages within it would be specified similarly, e.g., "github.com/jackc/pgx/v5/pgconn".
As #HymnsForDisco correctly pointed out in the comments, both github.com/jackc/pgconn and github.com/jackc/pgx/pgconn exist. It turns out that the returned error was from the former, whereas I was importing the latter in my code.
To confirm, I added the following line:
fmt.Println("Error path: ", reflect.TypeOf(res.Error).Elem().PkgPath())
// Output: "Error path: github.com/jackc/pgconn"
Changing my import to "github.com/jackc/pgconn" resolved the issue.
Handling errors in a gRPC service commonly requires both a status message and error codes. Both have two definitions:
Google APIs definition (googleapis/go-genproto) - the generated Go packages for common protocol buffer types, and the generated gRPC code for Google's gRPC APIs
gRPC definition (grpc/grpc-go) - the Go implementation of gRPC
The Go packages for both definitions of Status and Codes are:
Google APIs
Status: google.golang.org/genproto/googleapis/rpc/status
Codes: google.golang.org/genproto/googleapis/rpc/code
gRPC
Status: google.golang.org/grpc/status
Codes: google.golang.org/grpc/codes
Since I'm a client of my own gRPC service and not a client of an existing Google gRPC API, I want to use the gRPC definitions of Status and Code.
However, the gRPC proto file for Status is actually copied from Google APIs definition. See https://github.com/grpc/grpc/tree/master/src/proto/grpc/status. The go_package of status.proto is also unchanged, so both the Google API and gRPC definitions use the following Go package
option go_package = "google.golang.org/genproto/googleapis/rpc/status;status";
The upshot is the only way to use Status when defining an API is by importing it with
import "google/rpc/status.proto";
...and importing the language bindings in Go with
import (
"google.golang.org/genproto/googleapis/rpc/status"
)
// Go server code...
But as stated earlier, this is wrong since I'm not a client of a Google API, but rather my own gRPC service. Therefore the language bindings should be imported with
import (
"google.golang.org/grpc/status"
)
// Go server code...
As expected if I switch to importing the gRPC language bindings and try and return a Status message to the API client, I get a compile error
cannot use &(status.Status literal)
(value of type *"google.golang.org/grpc/internal/status".Status) as
*"google.golang.org/genproto/googleapis/rpc/status".Status value
This is caused by my .proto file using the Google API definition of Status while the server implementation (in Go) uses the gRPC definition.
The problem impacts error codes since Google APIs uses signed 32 bit integers (int32) whereas gRPC uses unsigned 32 bit integers (uint32).
Questions
Is my assertion that I should be using the gRPC definition of Status and Codes correct?
If my assertion is correct, how can I use the gRPC definition of Status when it's packaged for Google APIs?
We need to distinguish a few cases. Some of them are obvious, some are not.
Just returning Status from a gRPC handler
If your proto schema (.proto files) doesn't define messages that use Status or Code directly, then the gRPC handlers can satisfy the return type error simply with "google.golang.org/grpc/status".Error(), or Newf().Err(). And that's about it.
Example:
// implements SomeServiceServer unary RPC GetFoo
func (s *SomeService) GetFoo(ctx context.Context, req *grpc.FooRequest) (*grpc.FooResponse, error) {
// status is "google.golang.org/grpc/status"
return nil, status.Error(codes.Unimplemented, "coming soon")
Using Status in your .proto files
In this case, you are forced to use the googleapis implementation. As you already have seen, the status.proto Go package is defined as:
option go_package = "google.golang.org/genproto/googleapis/rpc/status;status";
So let's say you have the following .proto file, where the imported status.proto is just a copy-paste of the gRPC status.proto as per this question:
syntax = "proto3";
package test;
import "status.proto";
option go_package = ".;main";
message Foo {
string a = 1;
google.rpc.Status status = 2;
}
with directory structure as:
/protos
|_ status.proto
|_ test.proto
and you compile the above with:
cd protos && protoc -I=. --go_out=. test.proto
breathe ...then the generated Go code will have the following import
import (
status "google.golang.org/genproto/googleapis/rpc/status"
)
and you must satisfy that by go get google.golang.org/genproto.
So about your first question, you can only use Status from googleapis in proto files, because that's how status.proto declares its Go package.
Using generated googleapis Status in Go code
Since the imported Go package is from googleapis that is what you must use in your Go code in order to initialize such messages:
package main
import (
"fmt"
googleapis "google.golang.org/genproto/googleapis/rpc/status"
)
func main() {
foo := &Foo{
A: "foo",
Status: &googleapis.Status{
Code: int32(code.Code_OK),
Message: "all good",
},
}
// fmt.Println(foo)
}
Yes but I must use grpc-go Status in my Go code
You can't. protoc generates code with the packages described above. If you absolutely NEED to construct these Status fields using grpc-go, you can use Status.Proto:
package main
import (
"fmt"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func main() {
foo := &Foo{
A: "foo",
Status: status.New(codes.OK, "all good").Proto(),
}
fmt.Println(foo)
}
Just for the record, the opposite is also possible with status.FromProto:
package main
import (
"fmt"
googleapis_codes "google.golang.org/genproto/googleapis/rpc/code"
googleapis "google.golang.org/genproto/googleapis/rpc/status"
"google.golang.org/grpc/status"
)
func main() {
gapisStatus := &googleapis.Status{
Code: int32(googleapis_codes.Code_OK),
Message: "all good",
}
grpcStatus := status.FromProto(gapisStatus)
fmt.Println(grpcStatus)
}
As a less well-behaved alternative, you can simply copy-paste the status.proto sources into your project and manually change the go_package:
option go_package = "google.golang.org/grpc/status;status";
This way protoc will generate the Go code with this import, and your own sources will be able to follow suit. Of course this means you now have your own fork of status.proto.
Hi I am trying to generate the simple protobuf file in Go language
syntax = "proto3";
package gen;
message EvtKeepAlive
{
string SvcName = 2;
}
In the header I see that the package uses two different proto go implementations, one from github.com and one from google.golang.org. As far as I understand the latter supersedes the former, so is this file generation valid?
// versions:
// protoc-gen-go v1.25.0-devel
// protoc v3.13.0
// source: common.proto
package gen
import (
proto "github.com/golang/protobuf/proto"
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
)
The file is valid; see the comments from dsnet in response to this issue:
The only reason the newly generated .pb.go files depend on the
deprecated proto package is to enforce a weak dependency on a
sufficiently new version of the legacy package. This is necessary
because not everyone is using Go modules such that the Go toolchain
would enforce this dependency constraint. I wasn't fond of adding it,
but I think it's probably necessary to keep at least for a few months.
Is error type in Go "Error" or "error"? It bugged me that in the Tour it is with small first letter so I looked around and found here with small e yet here in source code it is with big capital letter.
Also how can it be without big capital letter yet still visible outside of package?
Just started learning Go so I might have missed something basic, thanks.
error is the type, lowercase. Just like with int and string it doesn't need to be visible as it is built-in to Go:
A good blog post on error handling
The runtime package you're referring to has an Error interface. The type there is an interface not error:
Package runtime
type Error interface {
error
// RuntimeError is a no-op function but
// serves to distinguish types that are run time
// errors from ordinary errors: a type is a
// run time error if it has a RuntimeError method.
RuntimeError()
}
The Error interface identifies a run time error.
I am using wgo for dependency management in Golang (although I think wgo has little to do with this), wgo has a folder structure like this
project/
.gocfg/
gopaths
vendor.json
vendor/
src/
github.com_or_whatever/
I have a library I coded myself which uses an nsq-go type in one of the exported methods:
func AddNsqSubscription(
topic, channel string,
handler nsq.Handler,
config *nsq.Config) error { }
The library is called messi and I import the nsq-go like so "messi/vendor/src/github.com/bitly/go-nsq"
The problem comes when I try to use this library in another project. For instance, in a project called scribe I have the following code (notice the imports):
import (
"scribe/vendor/src/github.com/bitly/go-nsq"
"scribe/vendor/src/messi"
)
//...
nsqHandler := nsq.HandlerFunc(func(message *nsq.Message) error {
msgHandler(MessiMessage{message})
return nil
})
return messi.AddNsqSubscription(destination, subdestination, nsqHandler, nsq.NewConfig())
When I go build the following error is returned:
cannot use nsqHandler (type "scribe/vendor/src/github.com/bitly/go-nsq".HandlerFunc) as type "messi/vendor/src/github.com/bitly/go-nsq".Handler in argument to messi.AddNsqSubscription:
"scribe/vendor/src/github.com/bitly/go-nsq".HandlerFunc does not implement "messi/vendor/src/github.com/bitly/go-nsq".Handler (wrong type for HandleMessage method)
have HandleMessage("scribe/vendor/src/github.com/bitly/go-nsq".Message) error
want HandleMessage("messi/vendor/src/github.com/bitly/go-nsq".Message) error
Why? I do not really know what is going on. The code go-nsq imported is exactly the same, yet golang wants that this code comes from the same folder?
What am I doing wrong?
Packages in Go are identified by full import path, not by name.
For example in the standard library there are two different packages with the same name template but different import paths: text/template and html/template.
You should make sure that go-nsq package is imported using the same path.