protoc documentation is very minimal. I often see something like this and I could not find any documentation.
protoc -I routeguide/ routeguide/route_guide.proto --go_out=plugins=grpc:routeguide
What does plugins=grpc: do? Does it need to be used together with
go_out? If so, this is pretty confusing in my opinion since
go_out is supposedly just specifying the output directory.
https://github.com/golang/protobuf has this example.
protoc --go_out=plugins=grpc,import_path=mypackage:. *.proto
This is more confusing since it has additional import_path.
Based on the documentation, plugins=grpc,import_path=mypackage: is considered additional parameter (since it is terminated by :) and . denotes the output directory.
Is it correct? I still do no know know what plugins=grpc and import_path=mypacakge do.
How to write proto
syntax = "proto3";
message Request {
int64 phone_number =1;
}
message Response {
int64 phone_number = 1;
string message = 2;
int32 status_code =3;
}
service CreateService{
rpc Service (Request) returns (Response);
}
To Generate code:
#generate gRPC-Code
protoc oauth2/proto/oauth.proto --go_out=plugins=grpc:.
For more know about the proto and gRPC those repos contains the basic to advance with good examples
https://github.com/SXerox007/gRPC-Remote-Procedure-Call-
https://github.com/SXerox007/protos-
Related
Here's the minimal example:
import "gopkg.in/alecthomas/kingpin.v2"
...
fooCodes = kingpin.Flag("foo_codes", "List of codes").Default().Strings()
and I tried passing --foo_codes=AA,BB,CC that didn't work unfortunately so I had to use
--foo_codes=AA
--foo_codes=BB
--foo_codes=CC
instead as a workaround (I'm passing those args in a yaml file. Is there a better option that I can configure via kingpin?
I guess you can use HintOptions or HintAction, for more details see these examples: https://github.com/alecthomas/kingpin/blob/master/_examples/completion/main.go
I call an HTTP request which returns a protobuf response. I don't have the .pb files, so I try to convert that to a readable format such as JSON with indexed fields.
Using protoc and the command protoc --decode-raw < test.bin
The output looks like this:
2: 1623325235623240
4: "\037\213\010\000\000\000\000\000\000\000\265[\tTS\327\272\346\204\220J\324g\232Zz\264\n\251\366\366\272\332K\r\220 \364\3425\377Y\335\272\223\253\200#\333\256\304=VKu\036\254%G\3501k\371^\273ZQ\225fg\030e\342\322"\271\207=\222|\366d/\307\25"
5: 1
7: 1
9 {
2: "MethodUpdate"
3 {
1: "ASfeyGDdgsSDgddSDheyWw"
2: 0
}
4: 11
}
I know that everything is meta data on the file and the actual content is in the field 4. From my knowledge, this is also a protobuf message that should have a similar output. If I copy paste the content of the field 4 and run the protoc command again, I get the error Failed to parse input.
Is there any way to parse the protobuf response to a readable format such as JSON?
I found similar questions, but non of the suggested answers worked in my case.
You ought be able to reverse-engineer the protobuf.
Then, you could use this to decode the message using your preferred language and convert that into JSON.
See Encoding for an overview of how the wire-format is constructed.
I'm using protobuf and I'm generating JAVA classes from the following proto file.
syntax = "proto3";
enum Greeting {
NONE = 0;
MR = 1;
MRS = 2;
MISS = 3;
}
message Hello {
Greeting greeting = 1;
string name = 2;
}
message Bye {
string name = 1;
}
option java_multiple_files = true;
Now I need to add some code to the generated files and I found that is possible using a custom plugin (https://developers.google.com/protocol-buffers/docs/reference/java-generated#plugins). I'm trying to generate that plugin in Java, something like this.
public class Test {
PluginProtos.CodeGeneratorResponse.getDefaultInstance();
/* Code to get generated files from java_out and use the insertion points */
codeGeneratorResponse.writeTo(System.out);
}
And then I run
protoc --java_out=./classes --plugin=protoc-gen-demo=my-plugin --demo_out=. example.proto
The problem is that on my Test.java main method I don't know how to get access to the files created by the option --java_out so that I can use their insertion points. Currently the CodeGeneratorResponse for the default instance is empty (no files).
Does anybody know how can I get the CodeGeneratorResponse from the --java_out so that I can add more code to the generated classes?
Thanks in advance.
I recently struggled with this as well and wasn't able to find a good answer. I finally figured it out after staring at the comments within the CodeGeneratorResponse message for a while.
What threw me off at first was that I was thinking of plugins as a pipeline, where the output from one feeds into the next. However, each plugin gets the exact same input (the parsed .proto files expressed via CodeGeneratorRequest messages), and all the generated code from the plugins (including the built-in ones) gets combined into the output file. However, plugins may modify the output from the previous plugins, which is what insertion points are designed for.
Specifically to your question, you would add a file to the response with the name field getting set to the name of the generated Java file, the insertion_point field getting set to the name of the insertion point at which you want to add code, and the content field getting set to the code you want inserted at that point.
I found this article helpful in creating a simple plugin (in this case in python). As a simple test, I modified the generate_code function from that article to look like this:
def generate_code(request, response):
for proto_file in request.proto_file:
f = response.file.add()
f.name = "Test.java"
f.insertion_point = "outer_class_scope"
f.content = "// Inserting a comment as a test"
Then I ran protoc with the plugin:
$ cat test.proto
syntax = "proto3";
message MyMsg {
int32 num = 1;
}
$ protoc --plugin=protoc-gen-sample=sample_proto_gen.py --java_out=. --sample_out=. test.proto
$ tail -n3 Test.java
// Inserting a comment as a test
// ##protoc_insertion_point(outer_class_scope)
}
Your plugin just needs to be some executable which reads a CodeGeneratorRequest message from stdin and writes a CodeGeneratorResponse message to stdout, so could certainly be written in Java instead. I just chose python as I'm generally more comfortable with it and found this simple example.
As a reference, here's a plugin I wrote for generating code based on custom protobuf options.
I have made a custom python plugin.
To run my plugin i use the command below:
protoc --plugin=protoc-gen-custom=my_plugin_executable_file --custom_out=./build test.proto
So i think that, you have to generate an executable file from your .java file and use it in your command.
I have a golang struct which contains references to some other structs. Is there an automated way to generate the .proto file from the structs ?
For example:
type A struct {
a int
b B
}
type B struct {
c []C
}
type C struct {
x int
}
should generate:
message A, B, C etc. proto3 is preferred.
https://github.com/kubernetes/kubernetes/tree/master/cmd/libs/go2idl seems to have something related but is undocumented. Any options ?
I'm find the package,Generate .proto files from Go source code:
proteus (https://github.com/src-d/proteus)
Proteus /proʊtiəs/ is a tool to generate protocol buffers version 3 compatible .proto files from your Go structs, types and functions.
The motivation behind this library is to use Go as a source of truth for your models instead of the other way around and then generating Go code from a .proto file, which does not generate idiomatic code.
Generate protobuf messages
//proteus:generate
type User struct {
Model
Username string
}
type Model struct {
ID int
CreatedAt time.Time
}
This example will generate the following protobuf message.
message User {
int32 id = 1;
google.protobuf.Timestamp created_at = 2;
string username = 3;
}
Install
go get -v gopkg.in/src-d/proteus.v1/...
Requirements
There are two requirements for the full process.
protoc binary installed on your path
go get -u github.com/gogo/protobuf/...
Usage
You can generate the proto files, the marshal/unmarshal and the rest of protobuf stuff for your Go types, the RPC client and server interface and the RPC server implementation for your packages. That is, the whole process.
proteus -f /path/to/protos/folder \
-p my/go/package \
-p my/other/go/package
You can generate proto files only using the command line tool provided with proteus.
proteus proto -f /path/to/output/folder \
-p my/go/package \
-p my/other/go/package
--verbose
You can also only generate gRPC server implementations for your packages.
proteus rpc -p my/go/package \
-p my/other/go/package
NOTE: Of course, if the defaults don't suit your needs, until proteus is extensible via plugins, you can hack together your own generator command using the provided components. Check out the godoc documentation of the package.
If anyone just need to generate pure protobuf messages without any gogo, mogo, blogo syntax you can use https://github.com/anjmao/go2proto which I wrote recently. It's super simple and just generates proto messages from given go source package containing structs. Also it supports go modules.
If someone find here in GO Get Error. You just need GO111MODULE=off
GO111MODULE=off go get -v gopkg.in/src-d/proteus.v1/...
GO111MODULE=off go get -v -u github.com/gogo/protobuf/...
Hi I am using a flag when testing in go:
file_test.go
var ip = flag.String("ip", "noip", "test")
I am only using this in one test file. And it works fine when only testing that one test file, but when I run:
go test ./... -ip 127.0.0.1 alle of the other test file saying: flag provided but not defined.
Have you seen this?
Regards
flag.Parse() is being called before your flag is defined.
You have to make sure that all flag definitions happen before calling flag.Parse(), usually by defining all flags inside init() functions.
If you've migrated to golang 13, it changed the order of the test initializer,
so it could lead to something like
flag provided but not defined: -test.timeout
as a possible workaround, you can use
var _ = func() bool {
testing.Init()
return true
}()
that would call test initialization before the application one. More info can be found on the original thread:
https://github.com/golang/go/issues/31859#issuecomment-489889428
do not call flag.Parse() in any init()
I'm very late to the party; but is this broken (again) on Go 1.19.5?
I hit the same errors reported on this thread and the same solution reported above (https://github.com/golang/go/issues/31859#issuecomment-489889428) fixes it.
I see a call to flags.Parse() was added back in go_test.go in v1.18
https://go.googlesource.com/go/+/f7248f05946c1804b5519d0b3eb0db054dc9c5d6%5E%21/src/cmd/go/go_test.go
I am only just learning Go so it'd be nice to have some verification from people more skilled before I report this elsewhere.
If you get this, when running command via docker-compose then you do incorrect quoting. Eg.
services:
app:
...
image: kumina/openvpn-exporter:latest
command: [
"--openvpn.status_paths", "/etc/openvpn_exporter/openvpn-status.log",
"--openvpn.status_paths /etc/openvpn_exporter/openvpn-status.log",
]
First is correct, second is wrong, because whole line counted as one parameter. You need to split them by passing two separate strings, like in first line.