How can I add my own code to JAVA generated classes from proto file? - protocol-buffers

I'm using protobuf and I'm generating JAVA classes from the following proto file.
syntax = "proto3";
enum Greeting {
NONE = 0;
MR = 1;
MRS = 2;
MISS = 3;
}
message Hello {
Greeting greeting = 1;
string name = 2;
}
message Bye {
string name = 1;
}
option java_multiple_files = true;
Now I need to add some code to the generated files and I found that is possible using a custom plugin (https://developers.google.com/protocol-buffers/docs/reference/java-generated#plugins). I'm trying to generate that plugin in Java, something like this.
public class Test {
PluginProtos.CodeGeneratorResponse.getDefaultInstance();
/* Code to get generated files from java_out and use the insertion points */
codeGeneratorResponse.writeTo(System.out);
}
And then I run
protoc --java_out=./classes --plugin=protoc-gen-demo=my-plugin --demo_out=. example.proto
The problem is that on my Test.java main method I don't know how to get access to the files created by the option --java_out so that I can use their insertion points. Currently the CodeGeneratorResponse for the default instance is empty (no files).
Does anybody know how can I get the CodeGeneratorResponse from the --java_out so that I can add more code to the generated classes?
Thanks in advance.

I recently struggled with this as well and wasn't able to find a good answer. I finally figured it out after staring at the comments within the CodeGeneratorResponse message for a while.
What threw me off at first was that I was thinking of plugins as a pipeline, where the output from one feeds into the next. However, each plugin gets the exact same input (the parsed .proto files expressed via CodeGeneratorRequest messages), and all the generated code from the plugins (including the built-in ones) gets combined into the output file. However, plugins may modify the output from the previous plugins, which is what insertion points are designed for.
Specifically to your question, you would add a file to the response with the name field getting set to the name of the generated Java file, the insertion_point field getting set to the name of the insertion point at which you want to add code, and the content field getting set to the code you want inserted at that point.
I found this article helpful in creating a simple plugin (in this case in python). As a simple test, I modified the generate_code function from that article to look like this:
def generate_code(request, response):
for proto_file in request.proto_file:
f = response.file.add()
f.name = "Test.java"
f.insertion_point = "outer_class_scope"
f.content = "// Inserting a comment as a test"
Then I ran protoc with the plugin:
$ cat test.proto
syntax = "proto3";
message MyMsg {
int32 num = 1;
}
$ protoc --plugin=protoc-gen-sample=sample_proto_gen.py --java_out=. --sample_out=. test.proto
$ tail -n3 Test.java
// Inserting a comment as a test
// ##protoc_insertion_point(outer_class_scope)
}
Your plugin just needs to be some executable which reads a CodeGeneratorRequest message from stdin and writes a CodeGeneratorResponse message to stdout, so could certainly be written in Java instead. I just chose python as I'm generally more comfortable with it and found this simple example.
As a reference, here's a plugin I wrote for generating code based on custom protobuf options.

I have made a custom python plugin.
To run my plugin i use the command below:
protoc --plugin=protoc-gen-custom=my_plugin_executable_file --custom_out=./build test.proto
So i think that, you have to generate an executable file from your .java file and use it in your command.

Related

Why are optional inputs on my Gradle custom task not working?

I have a build.gradle with the following contents:
task myTask {
inputs.file("input.txt").optional()
doLast { println "input.txt exists = " + file("input.txt").exists() }
}
If input.txt doesn't exist, it fails with:
File '/Users/skissane/testgradle/input.txt' specified for property '$1' does not exist.
What I am trying to do, is run a custom script–which is written in Groovy, and runs inside the Gradle build under doLast, not as an external process–which takes the input.txt file as input, and the script's behaviour and output will change based on what is in that input file. But it is an optional input file – the script will still generate output (albeit different output) even if the input file doesn't exist.
Things I have tried so far:
Remove .optional(), change it to .optional(true): no difference in results
Instead of .optional(), wrap it in if (file("input.txt").exists()) {: this works, but seems ugly. Why doesn't .optional() work?
Have I misunderstood what .optional() is meant to do? Because another answer suggests it is the right way to solve my problem, but it isn't working.
(I am using Gradle 6.8.3. I tried upgrading to the latest Gradle 7.2, the same problem occurs, although 7.2 has more detailed error messages.)
optional() can't be used to mark the file itself as optional. optional() just means that the input property is optional, and the task is still valid if no files at all are specified; but if a file is specified, it must exist.
As such, optional() isn't really useful in this kind of custom task declared directly in build.gradle. It is really intended for defining new task types in plugins, when one defines a new task input property other than inputs, and wants to make it optional to declare files for that property. It is the property itself which is made optional, not the files in it. On a custom task, declaring inputs as optional is pointless because it is already optional to begin with.
Right now (as of version 7.2), Gradle doesn't have any way to mark a file as an optional input, other than through if (file("input.txt").exists()) {. Hopefully they might add that feature in some future Gradle version.
(Thanks to James Justinic who answered my post about this on Gradle forums.)

How can I use SaveData to write an ASCII file in ParaView 3.98.1?

I am writing an automation script for an old project and I need some help with pvpython from Paraview 3.98.1. The function SaveData() in this version does not exist. I found its implementation here and moved it to my code. How can I save a file as ASCII? Calling it like SaveData(filename, proxy=px, FileType='Ascii') saves my files as binaries (awkward behavior).
I need this version because some of my codes in the scripting pipeline handle very specific vtk files. Using the SaveData() function of the latest versions ended up creating different metadata in my final files, and when I process them it ends up destroying my results. It is easier at the moment to use an older version of Paraview than to modify all my codes.
Edit:
The website is not working now, but it was yesterday. Maybe it is an internal problem? Anyway, the code is attached below.
# -----------------------------------------------------------------------------
def SetProperties(proxy=None, **params):
"""Sets one or more properties of the given pipeline object. If an argument
is not provided, the active source is used. Pass a list of property_name=value
pairs to this function to set property values. For example::
SetProperties(Center=[1, 2, 3], Radius=3.5)
"""
if not proxy:
proxy = active_objects.source
properties = proxy.ListProperties()
for param in params.keys():
pyproxy = servermanager._getPyProxy(proxy)
pyproxy.__setattr__(param, params[param])
# -----------------------------------------------------------------------------
def CreateWriter(filename, proxy=None, **extraArgs):
"""Creates a writer that can write the data produced by the source proxy in
the given file format (identified by the extension). If no source is
provided, then the active source is used. This doesn't actually write the
data, it simply creates the writer and returns it."""
if not filename:
raise RuntimeError ("filename must be specified")
session = servermanager.ActiveConnection.Session
writer_factory = servermanager.vtkSMProxyManager.GetProxyManager().GetWriterFactory()
if writer_factory.GetNumberOfRegisteredPrototypes() == 0:
writer_factory.UpdateAvailableWriters()
if not proxy:
proxy = GetActiveSource()
if not proxy:
raise RuntimeError ("Could not locate source to write")
writer_proxy = writer_factory.CreateWriter(filename, proxy.SMProxy, proxy.Port)
writer_proxy.UnRegister(None)
pyproxy = servermanager._getPyProxy(writer_proxy)
if pyproxy and extraArgs:
SetProperties(pyproxy, **extraArgs)
return pyproxy
# -----------------------------------------------------------------------------
def SaveData(filename, proxy=None, **extraArgs):
"""Save data produced by 'proxy' in a file. If no proxy is specified the
active source is used. Properties to configure the writer can be passed in
as keyword arguments. Example usage::
SaveData("sample.pvtp", source0)
SaveData("sample.csv", FieldAssociation="Points")
"""
writer = CreateWriter(filename, proxy, **extraArgs)
if not writer:
raise RuntimeError ("Could not create writer for specified file or data type")
writer.UpdateVTKObjects()
writer.UpdatePipeline()
del writer
# -----------------------------------------------------------------------------
The question is answered here (also my post). I used SaveData() to save a binary file with the proxy I need and then used DataSetWriter() to change my FileType to ASCII. It is not a beautiful solution since SaveData() is supposed to do that, but it does the job.

protoc documentation in golang is pretty confusing

protoc documentation is very minimal. I often see something like this and I could not find any documentation.
protoc -I routeguide/ routeguide/route_guide.proto --go_out=plugins=grpc:routeguide
What does plugins=grpc: do? Does it need to be used together with
go_out? If so, this is pretty confusing in my opinion since
go_out is supposedly just specifying the output directory.
https://github.com/golang/protobuf has this example.
protoc --go_out=plugins=grpc,import_path=mypackage:. *.proto
This is more confusing since it has additional import_path.
Based on the documentation, plugins=grpc,import_path=mypackage: is considered additional parameter (since it is terminated by :) and . denotes the output directory.
Is it correct? I still do no know know what plugins=grpc and import_path=mypacakge do.
How to write proto
syntax = "proto3";
message Request {
int64 phone_number =1;
}
message Response {
int64 phone_number = 1;
string message = 2;
int32 status_code =3;
}
service CreateService{
rpc Service (Request) returns (Response);
}
To Generate code:
#generate gRPC-Code
protoc oauth2/proto/oauth.proto --go_out=plugins=grpc:.
For more know about the proto and gRPC those repos contains the basic to advance with good examples
https://github.com/SXerox007/gRPC-Remote-Procedure-Call-
https://github.com/SXerox007/protos-

How to pass "args" while using Intellij IDEA [example list 3.11 in Odersky's Scala book (2nd Ed)]

I basically try to run the example 3.11 in Odersky's book (Programming in Scala). I am using Intellij IDE. While runing the code, the "else" branch got executed.
The screen capture is here:
The source is here in case you need it to try:
package ch3
import scala.io.Source
object l3p11 extends App{
def widthOfLength(s: String) = s.length.toString.length
if (args.length > 0){
val lines = Source.fromFile(args(0)).getLines().toList
val longestLine = lines.reduceLeft(
(a, b) => if (a.length > b.length) a else b
)
val maxWidth = widthOfLength(longestLine)
for (line <- lines){
val numSpaces = maxWidth - widthOfLength(line)
val padding = " " * numSpaces
println(padding + line.length + "|" + line)
}
}
else
Console.err.println("Please enter filename")
}
The reason, I think, is becuase I did not pass args correctly (say here I want to pass the source file l3p11.scala as the args). I tried several option, but have not find a way to pass the args correctly for the code to be executed in the "if" branch. There are two directions in my mind to resolve this problem:
1. Find the right way to pass args in Intellij IDE
Run Scala in commond line, a similar command such as
$ scala l3p11.scala l3p11.scala
should be able to pass the args correctly. But my current setting gives "bash: scala: command not found". I currently use scala REPL to run scala code following the set up given in Odersky's Coursera course on Scala. I think I need to change the set up in orde run scala directly, instead of using "sbt->console" to invoke the scala interpreter like what I am doing now.
Any suggestion on either direction (or other directions that I have not thought of) to resolve the problem is welcome.
Update 1:
Direction 2 works after I reinstall scala. (My to be corrected understanding is that the installation of sbt does not provide an executable binary of scala to be included in the environment list for Windows. Therefore, scala command cannot be found before). After installation of scala directly:
$ scala l3p11.scala l3p11.scala
gives the expected results. But I still have not figured out how to get this result with Intellij IDEA.
Update 2:
I revisited the "Program arguments" option after Joe's confirmation. The reason I was not be able to get it work before was that I only add "l3p11.scala". Adding the complete path from working directory "src/main/scala/ch3/l3p11.scala" solved the problem. The result is as following:
To pass command-line arguments when running a program in IntelliJ IDEA, use the "Edit Configurations …" menu item under "Run". Choose the entry for your main program. There's a "Program arguments" text field where you specify the arguments to pass to the program.
I'm not super familiar on how it will run on windows but if you are able to run it directly from the command line then I think you'll need to compile first, that's the scalac command. So:
$ scalac l3p11.scala
then you can run just with the class name, not sure if you would need quotes on the arg:
$ scala l3p11 l3p11.scala

How to install a directory recursively with waf

I currently use following valadoc build task to generate a api documentation for my vala application:
doc = bld.new_task_gen (
features = 'valadoc',
output_dir = '../doc/html',
package_name = bld.env['PACKAGE_NAME'],
package_version = bld.env['VERSION'],
packages = 'gtk+-3.0 gee-1.0 libxml-2.0 x11 gdk-x11-3.0 libpeas-gtk-1.0 libpeas-1.0 config xtst gdk-3.0',
vapi_dirs = '../vapi',
force = True)
path = bld.path.find_dir ('../src')
doc.files = path.ant_glob (incl='**/*.vala')
This tasks creates a directory html in the output directory including several subdirectories with html and picture files.
What I am know trying to do is to install such files to /usr/share/doc/projectname/html/. To do so I added the following to the wscript_build file (following the documentation I have found here):
output_dir = doc.bld.path.find_or_declare('../doc/html')
doc.outputs = output_dir.ant_glob (incl='**/*')
doc.bld.install_files('${PREFIX}/share/doc/projectname/html', doc.outputs)
However this leads to an error "Missing node signature". Does anyone know how to get around this error? Or is there a simple way to install a directory recursively with waf?
You can find a full-fledge sample here.
I had a similar issue with generated files and had to update the signature for the corresponding Node objects. Try creating a task:
def signature_task(task):
for x in task.generator.bld.path.find_dir('../doc/html').ant_glob('**/*', remove=False):
x.sig = Utils.h_file(x.abspath())
To the top of you build rule, try adding:
#Support running task groups serially
bld.post_mode = Build.POST_LAZY
Then at the end of your build, add:
#Previous tasks belong to a group
bld.add_group()
#This task runs last
bld(rule=signature_task, always=True, name="signature_task")
There is an easier way using relative_trick.
bld.install_files(destination,
bld.path.ant_glob('../doc/html/**'),
cwd=bld.path.find_dir('../doc/html'),
relative_trick=True)
This gets a list of files from the glob, chops off the prefix, and puts it into the destination.

Resources