I'm trying to generate Go file from proto file but it doesn't have json definition in the method's input definition. Should I add the json definition by myself or there were something wrong with my script. Thank you, I sincerely appreciate your help.
Proto file
message RateRequest {
string Base = 1;
string Destination = 2;
}
Generated file
type RateRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
//No json definition here
Base string `protobuf:"bytes,1,opt,name=Base,proto3" json:"Base,omitempty"`
Destination string `protobuf:"bytes,2,opt,name=Destination,proto3" json:"Destination,omitempty"`
}
Protoc script
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
protos/currency.proto
grpcurl
grpcurl --plaintext -d '{Base: "GBP", Destination: "USD"}' localhost:9092 Currency.GetRate
// Error invoking method "Currency.GetRate": error getting request data: message type RateRequest has no known field named base
Since the error is
error getting request data: message type RateRequest has no known field named base
And the json is
json:"Base,omitempty"`
Then it seems it's looking for the wrong field, it should be
json:"base,omitempty"`
Related
when I use kitex to start an examples, I Got an error like below
for my step:
mkdir -p Protobuf-test
new file whose name is "echo.proto" and content is like this
syntax = "proto3";
option go_package = "echo";
package echo;
message Request {
string msg = 1;
}
message Response {
string msg = 1;
}
service EchoService {
rpc ClientSideStreaming(stream Request) returns (Response){} // 客户端streaming
rpc ServerSideStreaming(Request) returns (stream Response){} // 服务端streaming
rpc BidiSideStreaming(stream Request) returns (stream Response){} //双向流
}
open a terminal executed a command
kitex -type protobuf -module echoTest -service echoTest echo.proto
please give me some advise, thanks very mush
I take a comparison to using "thrift", it is ok, in the directory "xx/kitex_gen/echo", there has a file named echo.go, but not when using protobuf as model
I am trying to learn GRPC from the official doc, Here is the tutorial I have followed grpc-go
Generating the proto using this command
protoc --go_out=$PWD helloworld/helloworld.proto
This above command will generate the file helloworld.pb.go without any problem but the problem is the code for the client stub is missing from the generated file
syntax = "proto3";
package helloworld;
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
The actual error i am getting from the client connection which is
undefined: helloworld.NewGreeterClient
And this has occurred from the line c := pb.NewGreeterClient(conn) in the greeter_client/main.go file
The reason behind because the client stub not generated in the generated file
Issue resolved i have some problems with the command
Here is the actual command
protoc --go_out=plugins=grpc:$PWD helloworld.proto
Add --I to your command. e.g.
protoc -I helloworld --go_out=${PWD} helloworld/*.proto
I have a Go struct for which I want to generate an OpenAPI schema automatically. Once I have an OpenAPI definition of that struct I wanna generate JSONSchema of it, so that I can validate the input data that comes and is gonna be parsed into those structs.
The struct looks like the following:
// mySpec: io.myapp.MinimalPod
type MinimalPod struct {
Name string `json:"name"`
// k8s: io.k8s.kubernetes.pkg.api.v1.PodSpec
v1.PodSpec
}
Above struct is clearly an augmentation of what Kubernetes PodSpec is.
Now the approach that I have used is to generate definition part for my struct MinimalPod, the definition for PodSpec will come from upstream OpenAPI spec of Kubernetes. PodSpec has a key io.k8s.kubernetes.pkg.api.v1.PodSpec in the upstream OpenAPI spec, this definition is injected from there in my Properties. Now in my code that parses above struct I have templates of what to do if struct field is string.
If the field has a comment that starts with k8s: ... the next part is Kubernetes object's OpenAPI definition key. In our case the OpenAPI definition key is io.k8s.kubernetes.pkg.api.v1.PodSpec. So I retrieve that field's definition from the upstream OpenAPI definition and embed it into the definition of my struct.
Once I have generated an OpenAPI definition for this struct which is injected in Kubernetes OpenAPI schema's definition with key being io.myapp.MinimalPod. Now I can use the tool openapi2jsonschema to generate JSONSchema out of this one. Which generates a JSONSchema file named MinimalPod.json.
Now jsonschema tool and the file MinimalPod.json can be used for validating input given to my tool parser to see if all fields were given right.
Is this the right approach of doing things, or is there a tool/library and if I feed Go structs to it, it gives me OpenAPI schema? It would be fine if it does not identify where to inject Kubernetes OpenAPI schema from even automatic parsing of Go structs and giving OpenAPI definition would be much appreciated.
Update 1
After following #mehdy 's instructions, this is what I have tried:
I have used this import path github.com/kedgeproject/kedge/vendor/k8s.io/client-go/pkg/api/v1 to import the PodSpec definition instead of k8s.io/api/core/v1 and code looks like this:
package foomodel
import "github.com/kedgeproject/kedge/vendor/k8s.io/client-go/pkg/api/v1"
// MinimalPod is a minimal pod.
// +k8s:openapi-gen=true
type MinimalPod struct {
Name string `json:"name"`
v1.PodSpec
}
Now when I generate the same with flag -i changed from k8s.io/api/core/v1 to github.com/kedgeproject/kedge/vendor/k8s.io/client-go/pkg/api/v1
$ go run example/openapi-gen/main.go -i k8s.io/kube-openapi/example/model,github.com/kedgeproject/kedge/vendor/k8s.io/client-go/pkg/api/v1 -h example/foomodel/header.txt -p k8s.io/kube-openapi/example/foomodel
This is what is generated:
$ cat openapi_generated.go
// +build !ignore_autogenerated
/*
======
Some random text
======
*/
// This file was autogenerated by openapi-gen. Do not edit it manually!
package foomodel
import (
spec "github.com/go-openapi/spec"
common "k8s.io/kube-openapi/pkg/common"
)
func GetOpenAPIDefinitions(ref common.ReferenceCallback) map[string]common.OpenAPIDefinition {
return map[string]common.OpenAPIDefinition{
"k8s.io/kube-openapi/example/model.Container": {
Schema: spec.Schema{
SchemaProps: spec.SchemaProps{
Description: "Container defines a single application container that you want to run within a pod.",
Properties: map[string]spec.Schema{
"health": {
SchemaProps: spec.SchemaProps{
Description: "One common definitions for 'livenessProbe' and 'readinessProbe' this allows to have only one place to define both probes (if they are the same) Periodic probe of container liveness and readiness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes",
Ref: ref("k8s.io/client-go/pkg/api/v1.Probe"),
},
},
"Container": {
SchemaProps: spec.SchemaProps{
Ref: ref("k8s.io/client-go/pkg/api/v1.Container"),
},
},
},
Required: []string{"Container"},
},
},
Dependencies: []string{
"k8s.io/client-go/pkg/api/v1.Container", "k8s.io/client-go/pkg/api/v1.Probe"},
},
}
}
I get only this much of the configuration generated. While when I switch back to "k8s.io/api/core/v1" I get config code auto generated which is more than 8k lines. What am I missing here?
Here definition of k8s.io/client-go/pkg/api/v1.Container and k8s.io/client-go/pkg/api/v1.Probe is missing while when I use k8s.io/api/core/v1 as import everything is generated.
Note: To generate above steps, please git clone https://github.com/kedgeproject/kedge in GOPATH.
You can use kube-openapi package for this. I am going to add a sample to the repo but I've tested this simple model:
// Car is a simple car model.
// +k8s:openapi-gen=true
type Car struct {
Color string
Capacity int
// +k8s:openapi-gen=false
HiddenFeature string
}
If you assume you created this file in
go run example/openapi-gen/main.go -h example/model/header.txt -i k8s.io/kube-openapi/example/model -p k8s.io/kube-openapi/example/model
(you also need to add a header.txt file). You should see a new file created in example/model folder called openapi_generated.go. This is an intermediate generated file that has your OpenAPI model in it:
func GetOpenAPIDefinitions(ref common.ReferenceCallback) map[string]common.OpenAPIDefinition {
return map[string]common.OpenAPIDefinition{
"k8s.io/kube-openapi/example/model.Car": {
Schema: spec.Schema{
SchemaProps: spec.SchemaProps{
Description: "Car is a simple car model.",
Properties: map[string]spec.Schema{
"Color": {
SchemaProps: spec.SchemaProps{
Type: []string{"string"},
Format: "",
},
},
"Capacity": {
SchemaProps: spec.SchemaProps{
Type: []string{"integer"},
Format: "int32",
},
},
},
Required: []string{"Color", "Capacity"},
},
},
Dependencies: []string{},
},
}
}
From there you should be able to call the generated method, get the model for your Type and get its Schema.
With some go get magic and changing the command line a little, I was able to generate the model for your model. Here is what you should change in your code:
package model
import "k8s.io/api/core/v1"
// MinimalPod is a minimal pod.
// +k8s:openapi-gen=true
type MinimalPod struct {
Name string `json:"name"`
v1.PodSpec
}
and then change the run command a little to include PodSpec in the generation:
go run example/openapi-gen/main.go -h example/model/header.txt -i k8s.io/kube-openapi/example/model,k8s.io/api/core/v1 -p k8s.io/kube-openapi/example/model
Here is what I got: https://gist.github.com/mbohlool/e399ac2458d12e48cc13081289efc55a
I am having problem with compiling the .proto file. Looking to generate REST endpoints from the .proto files. Below is the code and error :
syntax = "proto3";
package pb;
import "google/protobuf/empty.proto";
import "google/api/annotations.proto";
service UrlShortener {
rpc Hello(HelloRequest) returns (HelloResponse);
rpc Encrypt(EncryptRequest) returns (EncryptResponse);
rpc Decrypt(DecryptRequest) returns (DecryptResponse) {
option (google.api.http) = {
get: "/{hash}"
};
}
}
message HelloRequest {
string Name = 1;
}
message HelloResponse {
string Message = 1;
}
message EncryptRequest {
string OriginalUrl = 1;
}
message EncryptResponse {
UrlMap ResponseMap = 1;
}
message DecryptRequest {
string hash = 1;
}
Error :
github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis: warning: directory does not exist.
google/api/annotations.proto: File not found.
urlshortener.proto: Import "google/api/annotations.proto" was not found or had errors.
Please help with fixing this.
I tried : go get -u github.com/grpc-ecosystem/grpc-gateway
But it failed saying : no buildable go source files in path.
I think you have more than one errors in your definition
You are missing the syntax version at the very beginning of your definition:
syntax = "proto3";
There are some undefined types
service.proto:32:3: "UrlMap" is not defined.
service.proto:12:40: "DecryptResponse" is not defined.
You are importing and unused empty.proto
You can use the googleapies from
{GOPATH}/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis
Then run using:
protoc -I${GOPATH}/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis -I/usr/local/include -I. service.proto --go_out=plugins=grpc:.
I made the previous changes and it compiles, so I have the service.pb.go file
Edited:
Take a look of this grpc-gateway, maybe can help you
https://github.com/grpc-ecosystem/grpc-gateway
Found out the solution : The problem is that google/api/annotations has moved from the earlier path grpc-ecosystem/grpc-gateway/third_party/googleapis to https://github.com/grpc-ecosystem/grpc-gateway/tree/master/third_party/googleapis/google/api.
Running the following resolved the error : go get -u github.com/grpc-ecosystem/grpc-gateway/...
Everything works when I have my proto file defining a service with a streaming response. However when I return a single value, then the generated code is invalid!
Proto File:
syntax = "proto3";
package analyticsServicesGlobal;
option go_package = "generated/analyticsServices";
service HealthCheck {
rpc Health (HealthCheckParam) returns (HealthCheckPayload) {}
}
message HealthCheckPayload {
string message = 1;
}
message HealthCheckParam {
string response = 1;
}
The code generation completes without issue : protoc --go_out=plugins=grpc:$GOPATH/src/gauss minimal.proto
The ide and command line compilation return this error in the generated Go code though:
./minimal.pb.go:130: cannot use handler (type func("gauss/vendor/golang.org/x/net/context".Context, interface {}) (interface {}, error)) as type grpc.UnaryHandler in argument to interceptor
./minimal.pb.go:139: cannot use _HealthCheck_Health_Handler (type func(interface {}, "gauss/vendor/golang.org/x/net/context".Context, func(interface {}) error, grpc.UnaryServerInterceptor) (interface {}, error)) as type grpc.methodHandler in field value
This issue shows up for any service where a non-stream response is returned. I have create this minimal proto file to demonstrate, but every similar service definition has the same issue.
I am on OSX using Go version 1.8.1, protoc version 3.3.0, and the latest protoc plugin for Go go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
I'm sure I'm doing something wrong given the lack of reports on this issue - but I don't see what. Any suggestions are appreciated!