I am building a microservice application adopting gRPC as the communication protocol among the microservices. I realised that I have to copy and paste the proto files to all microservices. Futhermore, if there is a change in definition, I will have to C&P again to all relevant microservices.
How should the proto files be shared? Preferably there is a common that we can call to upload and download the updated proto files.
Related
Spring cloud-config-server has a built in mechanism to communicate with git repository and read files which are stored there. Then cloud-config-server also has built in mechanism to expose endpoints to clients (normally spring boot apps) which can read those files and use them as configurations.
This is well documented in spring documentation as can be seen from bellow.
According to doc
Spring Cloud Config Server
Spring Cloud Config Server provides an HTTP
resource-based API for external configuration (name-value pairs or
equivalent YAML content).
Also as documented about the serving format
Serving Alternative Formats
The default JSON format from the
environment endpoints is perfect for consumption by Spring
applications, because it maps directly onto the Environment
abstraction. If you prefer, you can consume the same data as YAML or
Java properties by adding a suffix (".yml", ".yaml" or ".properties")
to the resource path. This can be useful for consumption by
applications that do not care about the structure of the JSON
endpoints or the extra metadata they provide (for example, an
application that is not using Spring might benefit from the simplicity
of this approach).
It can also support txt format
Serving Plain Text
Instead of using the Environment abstraction (or
one of the alternative representations of it in YAML or properties
format), your applications might need generic plain-text configuration
files that are tailored to their environment.
But considering that spring cloud config server has the built in mechanism to communicate with a git repository and also exposes endpoints to the clients to consume the delivered files, it would make sense for other type of files to be able to be served from those endpoints as well.
It could be for example .pdf , .xslx , or even .zip
For example let's assume that the configured git repository contains the file myFile.zip in featureA branch. Then the call under the exposed path of type /{application}/{profile}[/{label}] for example as
serverUrl:serverPort/myApp/default/featureA/myFile.zip is able to deliver the file but is always delivered as raw .txt file which then corrupts the content of the original file existing in git.
I have already found the solution, but invested many hours on it and it was strange that it was not documented in spring documentation. So it is probably good to exist here as well to spare some time from others having the same issue.
As discussed under this issue, spring-cloud-config-server runs under the hood with the help of a normal spring-boot app. Considering that spring-boot has built in content negotiation mechanism it is able to consume and produce different content as well.
As for spring-cloud-config-server it is possible to fetch binary files from git as well as other files (ex zip, pdf, word, xlsx ...) if the call is made with the header Accept: application/octet-stream . This way the call to serverUrl:serverPort/myApp/default/featureA/myFile.zip is able to deliver a copy of the original file myFile.zip without any corruption.
I'm using grpc in the asp.net project with clean architecture, where should I put proto files and grpc services which layer, and which folder
When taking Clean Architecture (from Uncle Bob) very strict gRPC should be considered as a "framework" (technical detail) and so all code depending on it (including proto files) should be outside the domain/application layer and in the outer most layer.
I am building a Spring Boot Microservices using protobuf and gRPC for communication. However, I realised that I will need to define the entities in all microservices that requires it.
Instead of the straight forward method of copy and paste the raw file (not recommended), I can think of 2 methods:
sharing raw proto file
sharing proto generated files
Which is the proper way of doing? If I am sharing raw proto files, how can I share the proto files properly across microservices?
When sharing across languages, copying raw protos between repositories is typical. Some build systems like Bazel don't need to copy the protos, but most do. When copying protos it is important there there is a single well-known canonical copy of the protos and all other copies are bit-identical to a version of the canonical copy.
But when sharing in a Java-centric collection of projects, creating a canonical Java package for the generated code is superior as it is easier to use and it helps make sure only one copy of the generated code is in the classpath.
A typical Java protobuf Jar will include both the raw protos and the generated code; the raw protos are automatically included by both the Maven and Gradle Protobuf plugins. You then depend on that Jar like normal and it provides the dependency for Java code and Protobuf definitions. The Maven and Gradle Protobuf plugins automatically find .proto files in dependencies and add them to the include path (-I) of protoc when generating code.
I am new in grpc i don't know how to use it with spring boot but using the below link
https://github.com/saturnism/grpc-java-by-example/tree/master/simple-grpc-server
https://github.com/saturnism/grpc-java-by-example/tree/master/simple-grpc-client
note* : - first is for server project and second is for client project.
i have created a project on grpc with spring boot but i can'nt getting understand one thing in this that in grpc client project how can i use classes which are generated by protobuf in the project of grpc server. because it is not creating any proto file in grpc client project then how can i use the classes of grpc server project in grpc client project or can we create one project for grpc server and client instead of creating a diffrent project for both.
I have two queries to ask related to this question one:-
1. How to use classes of grpc generated by protobuf compiler in another project like if client and server are two different project and only server have proto generated files and client wants to use same classes.
How can i create all these thing in a single project means client and server in one project and then how can i run this project with step by step demo.
There are two ways you can do this:
Copy the .proto files between the two projects, and have each one generate their own copies of the generated code. This is probably the easiest, and allows you to avoid checking in the generated code into source control. The downside to this approach is that the .proto files can get out of date if you modify one and not the other.
Keep the .proto in the same repository of both the client and server, and make both depend on the generated code. This allows the proto to be modified for the client and server at the same time, but requires the code to live in the same repository (this is sometimes called the "Monorepo" approach). The downside to this is that the client and server repos may get too big, and need to be split up.
Google (the author of Protobuf) typically uses option #2, but many users of Protobuf prefer option 1. I would highly recommend regenerating the classes each time, and not check in the generated code. The ABI of Protobuf classes can change occasionally, and you would lose the backwards compatibility of Protobuf.
I have created a sample spring boot grpc application and posted in here
https://javabelazy.blogspot.com/
use the dependency net.devh.grpc-server-spring-boot-starter in your pom
create a protofile (sample service code)
service PingPongService {
rpc ping(PingRequest) returns (PongResponse) {
option (google.api.http) = { get: "/v1/grpc/{ping}" };
}
generate stubs for proto file using io.grpc:protoc-gen-grpc-java:1.30.0:exe
use nettyserver
set the port to 9090 (default) grpc.server.port=9090 in application properties
I have used https://github.com/yidongnan/grpc-spring-boot-starter recently. You will get most of the spring features along with grpc using this library.
There is yidongnan/grpc-spring-boot-starter (DOC) which implement springboot autoconfiguration starter for both client and server.
It implements #GrpcServer and #GrpcClient.
#GrpcService, which will add service to grpc server and start server automatically.
Annotation that marks gRPC services that should be registered with a gRPC server.
If spring-boot's auto configuration is used, then the server will be created
automatically. This annotation should only be added to implementations of
BindableService (GrpcService-ImplBase).
#GrpcClient, which will create channel and stub for client automatically
Example: #GrpcClient("myClient") <-> grpc.client.myClient.address=static://localhost:9090
nils server sample
nils client sample
Based on these samples, I also implement my simple server and client sample:
ppdouble/springboot-grpc-server-sample
ppdouble/springboot-grpc-client-sample
You can based on those samples implement your project or implement a new springboot autoconfiguration starter.
What is the best way to serve public static files uploaded at run-time by users using Spring (Boot) and (embedded) tomcat?
Moreover, is there an elegant way to upload and serve static file trees directly?
We are building a CMS that enables internal users like designers or front-end developers to directly upload their work on our server. It often consists in several files linked together by a relative path, so it could be really easier to have something to handle file trees directly. Files are always static as dynamic features only run on client-side.It could be helpful to purpose implementation ideas if there is no standard solution available (like using a zip then recreating its internal organisation into tomcat public folder).