I'm experimenting with Protobuf and gRPC and was going through the Go basics tutorial: https://www.grpc.io/docs/languages/go/basics/
From what I could see in the documentation about the ClientConn it seems that it is safe to use the same ClientConn concurrently. But I can't anything about the stub/client that one creates...
Say I was receiving data via HTTP POST requests and I wanted to forward that data via gRPC as protobuf and then respond to the HTTP request. Should I create a client/stub every time I process a HTTP request, or can I create the client/stub when setting up the HTTP server and pass in the client/stub when setting up HTTP Handlers?
Example: I assume this is okay
var opts []grpc.DialOption
conn, err := grpc.Dial("127.0.0.1:1234", opts...)
if err != nil {
log.Panic(err)
}
defer conn.Close()
grpcService := service.Service{GRPC: conn}
http.HandleFunc("/", util.ValidateRequest(grpcService.HandleRoot))
// listen to port
http.ListenAndServe(fmt.Sprintf("%s:%d", viper.GetString("server.address"), viper.GetInt32("server.port")), nil)
but what about this
var opts []grpc.DialOption
conn, err := grpc.Dial("127.0.0.1:1234", opts...)
if err != nil {
log.Panic(err)
}
defer conn.Close()
client := pb.NewEventShipperClient(conn)
grpcService := service.Service{GRPC: conn, Client: client}
http.HandleFunc("/", util.ValidateRequest(grpcService.HandleRoot))
// listen to port
http.ListenAndServe(fmt.Sprintf("%s:%d", viper.GetString("server.address"), viper.GetInt32("server.port")), nil)
Thread safety ("is it safe to run this concurrently") is covered in the Generated-code reference:
Thread-safety: note that client-side RPC invocations and server-side RPC handlers are thread-safe and are meant to be run on concurrent goroutines. But also note that for individual streams, incoming and outgoing data is bi-directional but serial; so e.g. individual streams do not support concurrent reads or concurrent writes (but reads are safely concurrent with writes).
So, yes, methods on the struct returned by pb.NewEventShipperClient(conn) can be called concurrently.
You can also check this yourself by looking at the implementation of pb.NewEventShipperClient. Currently (I guess this may change) the only member variable will be an interface (cc grpc.ClientConnInterface - implemented by *ClientConn) with each method using the connection to Invoke a gRPC call.
Please note, however, that you need to be careful in your implementation of service.Service to ensure it is also threadsafe and note the caveat regarding streams.
Related
What is the proper way to manage resource ownership in golang? Suppose I have the following:
db, err := sql.Open("mysql", "role#/test_db")
am := NewResourceManager(db)
am.DoWork()
db.Close()
Is it typical to always have the calling function maintain ownership and responsibility for closing resources? This feels a little weird to me because after closing, am still retains a reference and could try to use db if I or someone else is not careful later on (I guess this is a case for defer; however, if I want to pass the ResourceManager am back from this block, how would I even defer the closing of the file properly? I actually want it to stay open when this block finishes execution). I find that in other languages I often want to allow the instance to manage the resource and then clean it up when it's destructor is called, like this toy python example:
class Writer():
def __init__(self, filename):
self.f = open(filename, 'w+')
def __del__(self):
self.f.close()
def write(value):
self.f.write(value)
Unfortunately, there are no destructors in golang. I'm not sure how I would do this in go other than something like this:
type ResourceManager interface {
DoWork()
// Close() ?
}
type resourceManager struct {
db *sql.DB
}
func NewResourceManager(db *sql.DB) ResourceManager {
return &resourceManager{db}
}
db, err := sql.Open("mysql", "role#/test_db")
am := NewResourceManager(db)
am.DoWork()
am.Close() // using method shortening
But this seems less transparent, and I'm not sure how to communicate that the ResourceManager also needs to be Close()'d now. I'm finding this a frequent stumbling block, i.e. I also want to have a resource manager that holds a gRPC client connection, and if these types of resources aren't managed by resource managing objects, it seems like my main function is going to be cluttered with a lot of resource management, i.e. opening and closing. For instance, I could imagine a case where I wouldn't want main to know anything about the object and it's resources:
...
func NewResourceManager() ResourceManager {
db, err := sql.Open("mysql", "role#/test_db")
return &resourceManager{db}
}
...
// main elsewhere
am := NewResourceManager()
am.DoWork()
You chose a bad example, since you generally would reuse a database connection, instead of opening and closing one for each use. Hence, you would pass the db connection to functions using it and do the resource management in the caller without any need for an resource manager:
// Imports etc omitted for the sake of readability
func PingHandler(db *sql.DB) http.Handler (
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if err := db.ping(); err != nil {
http.Error(w,e.Error(),500)
}
})
)
func main(){
db,_ := sql.Open("superdb",os.Getenv("APP_DBURL"))
// Note the db connection will only be closed if main exits.
defer db.Close()
// Setup the server
http.Handle("/ping", PingHandler(db))
server := &http.Server{Addr: ":8080"}
// Create a channel for listening on SIGINT, -TERM and -QUIT
stop := make(chan os.Signal, 1)
// Register channel to be notified on said signals
signal.Notify(stop, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)
go func(){
// When we get the signal...
<- stop
ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
// ... we gracefully shut down the server.
// That ensures that no new connections, which potentially
// would use our db connection, are accepted.
if err := server.Shutdown(ctx); err != nil {
// handle err
}
}
// This blocks until the server is shut down.
// AFTER it is shut down, main exits, the deferred calls are executed.
// In this case, the database connection is closed.
// And it is closed only after the last handler call which uses the connection is finished.
// Mission accomplished.
server.ListenAndServe()
}
So in this example, there is no need for a resource manager and I honestly can not think of an example actually needing one. In the rare cases I needed something akin to one, I used sync.Pool.
However, for gRPC client connections, there is no need to maintain a pool, either:
[...]However, the ClientConn should manage the connections itself, so if a connection is broken, it will reconnect automatically. And if you have multiple backends, it's possible to connect to multiple of them and load balance between them. [...]
So the same principle applies: Create a connection (pool), pass it around as required, ensure that it is closed after all work is done.
Go proverb:
Clear is better than clever.
– Robert Pike
Instead of hiding resource management somewhere else, manage the resources as closely to the code where they are used and as explicitly as you possibly can.
I was following some of the tutorial for creating bidirectional grpc client and server. Client will pass some value and when last maximum value changed on server it'll response client with current max. Finally I'd like to write down some of the test cases but I have no experience with testing scenarios that's why I'm not sure if I'm doing the correct thing or not.
func TestClientConnection(t *testing.T) {
creds, _ := credentials.NewClientTLSFromFile("../server-cert.pem", "")
conn, err := grpc.Dial(address, grpc.WithTransportCredentials(creds))
if err != nil {
t.Error("Had problem with connection, NOT PASSED")
}
defer conn.Close()
c := proto.NewHerdiusServerClient(conn)
stream, err := c.CheckMax(context.Background())
if err != nil {
t.Error("Had problem with stream, NOT PASSED")
return
}
err = stream.Send(&proto.MaxRequest{Val: int32(10)})
err = stream.Send(&proto.MaxRequest{Val: int32(12)})
err = stream.Send(&proto.MaxRequest{Val: int32(13)})
err = stream.Send(&proto.MaxRequest{Val: int32(9)})
if err != nil {
t.Error("Had problem with stream, NOT PASSED")
return
}
return
}
Right now when I test this scenario wiht go test it passes but I also want to test if something received from server side.
My second question was If I want to tear this test to different scenarios for example to check is server connected or is stream connected or it received response from server side, how can I do that? Should I create another class to retrieve connection and streaming and use on test functions?
Create a contest with timeout context.WithTimeout, and after sending your data call Recv on the stream. Check if you receive anything within the timeout.
The specifics depend on the protocol here - you may need a goroutine to Recv if you have to send server data at the same time.
As for your second question, the Go philosophy is to have clear, explicit, readable tests for each scenario. It's OK if some code is duplicated. It's much more important that each test in isolation is readable and understandable. In cases where the tests are very repetitive one should use table driven tests, but in the cases you describe that sounds like separate tests to me.
It's useful to have tests that "build up" functionality. One test to test connection, the other connection and sending, yet another connection and sending and receiving. This way when tests fail, by rerunning them individually you can very quickly isolate the problem even before you look at the tests' code.
I am taking a look at the tutorials of grpc
https://grpc.io/docs/tutorials/basic/go.html
The grpc unary call looks something like this
conn, err := grpc.Dial(*serverAddr)
if err != nil {
...
}
defer conn.Close()
client := pb.NewRouteGuideClient(conn)
feature, err := client.GetFeature(context.Background(), &pb.Point{409146138, -746188906})
if err != nil {
...
}
I wanted to know if I call
client.GetFeature
from multiple threads, is it thread safe?
Looking into this issue you can learn that:
#rubenv asks:
Can I use a client from different threads in parallel?
#iamqizhao replies:
On client, if you want to perform multiple rpc in parallel, you should
spawn multiple goroutines to do that since the rpc is
synchronous/blocking
the answer is yes, however, a stream can't be shared (source).
#trevorgray, these kinds of concurrency topics are apparently still
not documented, per #682.
I am trying to understand GoLang "Go" together with gRPC and to make a simple service scalable.
Lets say I have a client1 that calls a service1(adds numbers) that calls service2(determines if the result is prime), and service2 returns the result to service1 that returns the result to client1 all via gRPC.
When I use protocol buffers "proto3" and generate the Go code via protoc.
I get generated methods that call the service in one particular way.
I see no distinction to call the methods asynchronously "Go".
And the underlying call seems to be "Invoke" which I believe is synchronous,the call returns once a result is received.
How do I make service1 "performant", I know I can run this in a cluster and have copies, but that would mean I can only serve clients as per the amount of instances within the cluster.
I want a "single" service to be able to serve multiple clients(e.g. 1000) .
Here is a simple server and I am not sure if this is performant or not:
I do know that the getprime function does dial every time,
and this could probably be moved to make this dial persist and be re-used; But more importantly I want to make a simple performant scaleable service and get a good understanding.
(A)
Perhaps the whole design is incorrect and the service1 should just return
as soon as the instruction is received "ack", do the addition and send the next request to sercice2 which determines if the answer is prime or not; again service2 just responds with an acknowledgement of the request being received. Once prime is determined by the service2 a call is made to the client with an answer.
If (A) above is the better approach, then still please explain the bottlenecks below; what happens when multiple clients are processed?
The call to "Listen" does what, "blocks, or does not block", etc.
package main
import (
pb "demo/internal/pkg/proto_gen/calc"
"fmt"
"golang.org/x/net/context"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
"log"
"net"
)
const (
port = ":8080"
)
type service struct {
}
func (s *service) Calculate(ctx context.Context, req *pb.Instruction) (*pb.Response, error) {
var answer float64
answer = req.Number1 + req.Number2
// call service prime
p := getprime(int(answer))
pa := pb.PrimeAnswer{Prime: p}
return &pb.Response{Answer: answer, Prime: &pa}, nil
}
const (
primeAddress = "127.0.0.1:8089"
)
func getprime(number int) bool {
conn, err := grpc.Dial(primeAddress, grpc.WithInsecure())
if err != nil {
log.Fatalf("Did not connect to prime service: %v", err)
}
defer conn.Close()
client := pb.NewPrimeServiceClient(conn)
p := pb.PrimeMessage{"", float64(number)}
r, err := client.Prime(context.Background(), &p)
if err != nil {
log.Fatalf("Call to prime service failed: %v", err)
}
return r.Prime
}
func main() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterCalculatorServer(s, &service{})
reflection.Register(s)
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
Thanks for your question. It is true that gRPC-Go is sync only; that is your Unary RPC(the one in your example) will return only when the RPC has finished (got a response from the server).
About performance:
The Dial operation establishes an underlying connection which may be expensive. So it not wise to do it every time getprime is called. A better way is to create a client, keep it around and make calls to the prime server on it. This way only first RPC incurs the cost of connection.
For each RPC request a server gets we launch a goroutine to process that request. So in general, this should scale fairly well.
About (A): It is not uncommon for a service handler to make an RPC call to yet another server and wait for its response before returning back.
Note that there's no way for a server to make call to the client.
To phrase what JimB said as an answer: "Synchronous" means that the function that makes the remote call waits for a reply before continuing, not that the whole server or client does. The server is normally multithreaded, even when processing synchronous calls; it can accept and work on a second call while it's responding to the first.
And similarly, if a client has multiple concurrent tasks that each have a gRPC call running, that won't block the process. Clients like that could include net/http servers serving end users, or gRPC servers handling multiple RPCs.
Where you might add explicit go statements is if you want to do something else from the specific function making the RPC call. For example, if you want to issue several RPC calls at once then wait for all their results to come in, you could write code following the examples of fan-out calls.
I have a function that makes a call to an external API using a Go http.Client, parses the result, and uses the result in the template executed afterwards. Occasionally, the external API will respond slowly (~20s), and the template execution will fail citing "i/o timeout", or more specifically,
template: :1:0: executing "page.html" at <"\n\t\t\t\t\t\t\t\t\...>: write tcp 127.0.0.1:35107: i/o timeout
This always coincides with a slow API response, but there is always a valid response in the JSON object, so the http.Client is receiving a proper response. I am just wondering if anyone could point me towards what could be causing the i/o timeout in the ExecuteTemplate call.
I have tried ResponseHeaderTimeout and DisableKeepAlives in the client transport (both with and without those options) to no avail. I've also tried setting the request's auto-close value to true to no avail. A stripped-down version of the template generation code is below:
func viewPage(w http.ResponseWriter, r *http.Request) {
tmpl := pageTemplate{}
duration, _ := time.ParseDuration("120s")
tr := &http.Transport{
ResponseHeaderTimeout: duration,
DisableKeepAlives: true,
}
client := &http.Client{Transport: tr}
req, _ := http.NewRequest("GET", "http://example.com/some_function", nil)
req.Close = true
resp, _ := client.Do(req)
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
var res api_response // some struct that matches the JSON response
err = json.Unmarshal(body, &res)
t, _ := template.New("page.html")
err = t.ExecuteTemplate(w, "page.html", tmpl)
}
The timeout on this line:
err = t.ExecuteTemplate(w, "page.html", tmpl)
means that the outgoing response is timing out when being written into, so nothing you change in the locally created client should affect it. It also does make sense that a slow response from that client increases the chance of the timeout on w, since the deadline is set when the response is created, before your handler is called, so a slow activity from your handler will increase the chances of a timeout.
There's no write timeout on the http.Server instance used by http.ListenAndServe, so you must be setting the Server.WriteTimeout field explicitly on the created server.
As a side note, there are errors being ignored in that handler, which is a strongly discouraged practice.