I am taking a look at the tutorials of grpc
https://grpc.io/docs/tutorials/basic/go.html
The grpc unary call looks something like this
conn, err := grpc.Dial(*serverAddr)
if err != nil {
...
}
defer conn.Close()
client := pb.NewRouteGuideClient(conn)
feature, err := client.GetFeature(context.Background(), &pb.Point{409146138, -746188906})
if err != nil {
...
}
I wanted to know if I call
client.GetFeature
from multiple threads, is it thread safe?
Looking into this issue you can learn that:
#rubenv asks:
Can I use a client from different threads in parallel?
#iamqizhao replies:
On client, if you want to perform multiple rpc in parallel, you should
spawn multiple goroutines to do that since the rpc is
synchronous/blocking
the answer is yes, however, a stream can't be shared (source).
#trevorgray, these kinds of concurrency topics are apparently still
not documented, per #682.
Related
I'm experimenting with Protobuf and gRPC and was going through the Go basics tutorial: https://www.grpc.io/docs/languages/go/basics/
From what I could see in the documentation about the ClientConn it seems that it is safe to use the same ClientConn concurrently. But I can't anything about the stub/client that one creates...
Say I was receiving data via HTTP POST requests and I wanted to forward that data via gRPC as protobuf and then respond to the HTTP request. Should I create a client/stub every time I process a HTTP request, or can I create the client/stub when setting up the HTTP server and pass in the client/stub when setting up HTTP Handlers?
Example: I assume this is okay
var opts []grpc.DialOption
conn, err := grpc.Dial("127.0.0.1:1234", opts...)
if err != nil {
log.Panic(err)
}
defer conn.Close()
grpcService := service.Service{GRPC: conn}
http.HandleFunc("/", util.ValidateRequest(grpcService.HandleRoot))
// listen to port
http.ListenAndServe(fmt.Sprintf("%s:%d", viper.GetString("server.address"), viper.GetInt32("server.port")), nil)
but what about this
var opts []grpc.DialOption
conn, err := grpc.Dial("127.0.0.1:1234", opts...)
if err != nil {
log.Panic(err)
}
defer conn.Close()
client := pb.NewEventShipperClient(conn)
grpcService := service.Service{GRPC: conn, Client: client}
http.HandleFunc("/", util.ValidateRequest(grpcService.HandleRoot))
// listen to port
http.ListenAndServe(fmt.Sprintf("%s:%d", viper.GetString("server.address"), viper.GetInt32("server.port")), nil)
Thread safety ("is it safe to run this concurrently") is covered in the Generated-code reference:
Thread-safety: note that client-side RPC invocations and server-side RPC handlers are thread-safe and are meant to be run on concurrent goroutines. But also note that for individual streams, incoming and outgoing data is bi-directional but serial; so e.g. individual streams do not support concurrent reads or concurrent writes (but reads are safely concurrent with writes).
So, yes, methods on the struct returned by pb.NewEventShipperClient(conn) can be called concurrently.
You can also check this yourself by looking at the implementation of pb.NewEventShipperClient. Currently (I guess this may change) the only member variable will be an interface (cc grpc.ClientConnInterface - implemented by *ClientConn) with each method using the connection to Invoke a gRPC call.
Please note, however, that you need to be careful in your implementation of service.Service to ensure it is also threadsafe and note the caveat regarding streams.
Bit of a newb to both Go and GRPC, so bear with me.
Using go version go1.14.4 windows/amd64, proto3, and latest grpc (1.31 i think). I'm trying to set up a bidi streaming connection that will likely be open for longer periods of time. Everything works locally, except if I terminate the client (or one of them) it kills the server as well with the following error:
Unable to trade data rpc error: code = Canceled desc = context canceled
This error comes out of this code server side
func (s *exchangeserver) Trade(stream proto.ExchageService_TradeServer) error {
endchan := make(chan int)
defer close(endchan)
go func() {
for {
req, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
log.Fatal("Unable to trade data ", err)
break
}
fmt.Println("Got ", req.GetNumber())
}
endchan <- 1
}()
go func() {
for {
resp := &proto.WordResponse{Word: "Hello again "}
err := stream.Send(resp)
if err != nil {
log.Fatal("Unable to send from server ", err)
break
}
time.Sleep(time.Duration(500 * time.Millisecond))
}
endchan <- 1
}()
<-endchan
return nil
}
And the Trade() RPC is so simple it isn't worth posting the .proto.
The error is clearly coming out of the Recv() call, but that call blocks until it sees a message, like the client disconnect, at which point I would expect it to kill the stream, not the whole process. I've tried adding a service handler with HandleConn(context, stats.ConnStats) and it does catch the disconnect before the server dies, but I can't do anything with it. I've even tried creating a global channel that the serve handler pushes a value into when HandleRPC(context, stats.RPCStats) is called and only allowing Recv() to be called when there's a value in the channel, but that can't be right, that's like blocking a blocking function for safety and it didn't work anyway.
This has to be one of those real stupid mistakes that beginner's make. Of what use would GPRC be if it couldn't handle a client disconnect without dying? Yet I have read probably a trillion (ish) posts from every corner of the internet and noone else is having this issue. On the contrary, the more popular version of this question is "My client stream stays open after disconnect". I'd expect that issue. Not this one.
Im not 100% sure how this is supposed to behave but I note that you are starting separate receive and send goroutines up at the same time. This might be valid but is not the typical approach. Instead you would usually receive what you want to process and then start a nested loop to handle the reply .
See an example of typical bidirectional streaming implementation from here: https://grpc.io/docs/languages/go/basics/
func (s *routeGuideServer) RouteChat(stream pb.RouteGuide_RouteChatServer) error {
for {
in, err := stream.Recv()
if err == io.EOF {
return nil
}
if err != nil {
return err
}
key := serialize(in.Location)
... // look for notes to be sent to client
for _, note := range s.routeNotes[key] {
if err := stream.Send(note); err != nil {
return err
}
}
}
}
sending and receiving at the same time might be valid for your use case but if that is what you are trying to do then I believe your handling of the channels is incorrect. Either way, please read on to understand the issue as it is a common one in go.
You have a single channel which only blocks until it receives a single message, once it unblocks the function ends and the channel is closed (by defer).
You are trying to send to this channel from both your send and receive
loop.
When the last one to finish tries to send to the channel it will have been closed (by the first to finish) and the server will panic. Annoyingly, you wont actually see any sign of this as the server will exit before the goroutine can dump its panic (no clues - probably why you landed here)
see an example of the issue here (grpc code stripped out):
https://play.golang.org/p/GjfgDDAWNYr
Note: comment out the last pause in the main func to stop showing the panic reliably (as in your case)
So one simple fix would probably be to simply create two separate channels (one for send, one for receive) and block on both - this however would leave the send loop open necessarily if you don't get a chance to respond so probably better to structure like the example above unless you have good reason to pursue something different.
Another possibility is some sort server/request context mix up but I'm pretty sure the above will fix - drop an update with your server setup code if your still having issues after the above changes
I was following some of the tutorial for creating bidirectional grpc client and server. Client will pass some value and when last maximum value changed on server it'll response client with current max. Finally I'd like to write down some of the test cases but I have no experience with testing scenarios that's why I'm not sure if I'm doing the correct thing or not.
func TestClientConnection(t *testing.T) {
creds, _ := credentials.NewClientTLSFromFile("../server-cert.pem", "")
conn, err := grpc.Dial(address, grpc.WithTransportCredentials(creds))
if err != nil {
t.Error("Had problem with connection, NOT PASSED")
}
defer conn.Close()
c := proto.NewHerdiusServerClient(conn)
stream, err := c.CheckMax(context.Background())
if err != nil {
t.Error("Had problem with stream, NOT PASSED")
return
}
err = stream.Send(&proto.MaxRequest{Val: int32(10)})
err = stream.Send(&proto.MaxRequest{Val: int32(12)})
err = stream.Send(&proto.MaxRequest{Val: int32(13)})
err = stream.Send(&proto.MaxRequest{Val: int32(9)})
if err != nil {
t.Error("Had problem with stream, NOT PASSED")
return
}
return
}
Right now when I test this scenario wiht go test it passes but I also want to test if something received from server side.
My second question was If I want to tear this test to different scenarios for example to check is server connected or is stream connected or it received response from server side, how can I do that? Should I create another class to retrieve connection and streaming and use on test functions?
Create a contest with timeout context.WithTimeout, and after sending your data call Recv on the stream. Check if you receive anything within the timeout.
The specifics depend on the protocol here - you may need a goroutine to Recv if you have to send server data at the same time.
As for your second question, the Go philosophy is to have clear, explicit, readable tests for each scenario. It's OK if some code is duplicated. It's much more important that each test in isolation is readable and understandable. In cases where the tests are very repetitive one should use table driven tests, but in the cases you describe that sounds like separate tests to me.
It's useful to have tests that "build up" functionality. One test to test connection, the other connection and sending, yet another connection and sending and receiving. This way when tests fail, by rerunning them individually you can very quickly isolate the problem even before you look at the tests' code.
I am trying to understand GoLang "Go" together with gRPC and to make a simple service scalable.
Lets say I have a client1 that calls a service1(adds numbers) that calls service2(determines if the result is prime), and service2 returns the result to service1 that returns the result to client1 all via gRPC.
When I use protocol buffers "proto3" and generate the Go code via protoc.
I get generated methods that call the service in one particular way.
I see no distinction to call the methods asynchronously "Go".
And the underlying call seems to be "Invoke" which I believe is synchronous,the call returns once a result is received.
How do I make service1 "performant", I know I can run this in a cluster and have copies, but that would mean I can only serve clients as per the amount of instances within the cluster.
I want a "single" service to be able to serve multiple clients(e.g. 1000) .
Here is a simple server and I am not sure if this is performant or not:
I do know that the getprime function does dial every time,
and this could probably be moved to make this dial persist and be re-used; But more importantly I want to make a simple performant scaleable service and get a good understanding.
(A)
Perhaps the whole design is incorrect and the service1 should just return
as soon as the instruction is received "ack", do the addition and send the next request to sercice2 which determines if the answer is prime or not; again service2 just responds with an acknowledgement of the request being received. Once prime is determined by the service2 a call is made to the client with an answer.
If (A) above is the better approach, then still please explain the bottlenecks below; what happens when multiple clients are processed?
The call to "Listen" does what, "blocks, or does not block", etc.
package main
import (
pb "demo/internal/pkg/proto_gen/calc"
"fmt"
"golang.org/x/net/context"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
"log"
"net"
)
const (
port = ":8080"
)
type service struct {
}
func (s *service) Calculate(ctx context.Context, req *pb.Instruction) (*pb.Response, error) {
var answer float64
answer = req.Number1 + req.Number2
// call service prime
p := getprime(int(answer))
pa := pb.PrimeAnswer{Prime: p}
return &pb.Response{Answer: answer, Prime: &pa}, nil
}
const (
primeAddress = "127.0.0.1:8089"
)
func getprime(number int) bool {
conn, err := grpc.Dial(primeAddress, grpc.WithInsecure())
if err != nil {
log.Fatalf("Did not connect to prime service: %v", err)
}
defer conn.Close()
client := pb.NewPrimeServiceClient(conn)
p := pb.PrimeMessage{"", float64(number)}
r, err := client.Prime(context.Background(), &p)
if err != nil {
log.Fatalf("Call to prime service failed: %v", err)
}
return r.Prime
}
func main() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterCalculatorServer(s, &service{})
reflection.Register(s)
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
Thanks for your question. It is true that gRPC-Go is sync only; that is your Unary RPC(the one in your example) will return only when the RPC has finished (got a response from the server).
About performance:
The Dial operation establishes an underlying connection which may be expensive. So it not wise to do it every time getprime is called. A better way is to create a client, keep it around and make calls to the prime server on it. This way only first RPC incurs the cost of connection.
For each RPC request a server gets we launch a goroutine to process that request. So in general, this should scale fairly well.
About (A): It is not uncommon for a service handler to make an RPC call to yet another server and wait for its response before returning back.
Note that there's no way for a server to make call to the client.
To phrase what JimB said as an answer: "Synchronous" means that the function that makes the remote call waits for a reply before continuing, not that the whole server or client does. The server is normally multithreaded, even when processing synchronous calls; it can accept and work on a second call while it's responding to the first.
And similarly, if a client has multiple concurrent tasks that each have a gRPC call running, that won't block the process. Clients like that could include net/http servers serving end users, or gRPC servers handling multiple RPCs.
Where you might add explicit go statements is if you want to do something else from the specific function making the RPC call. For example, if you want to issue several RPC calls at once then wait for all their results to come in, you could write code following the examples of fan-out calls.
I am using goroutines in a package where there is a tcp server. The response most of the time is very heavy, but when the routines end it is not cleared from the memory.
func Handle() {
service := ":7777"
tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
checkError(err)
listener, err := net.ListenTCP("tcp", tcpAddr)
checkError(err)
defer listener.Close()
for {
conn, err := listener.Accept()
checkError(err)
go handleRequest(conn, db)
}
}
func handleRequest(conn net.Conn, db *sql.DB) {
message := make([]byte, 0, 4096)
tmp := make([]byte, 256)
n, err := conn.Read(tmp)
if err != nil {
if err != io.EOF {
fmt.Println("read error:", err)
}
}
message = append(message, tmp[:n]...)
fmt.Println("Message Received:", string(message))
// do something to get resp
conn.Write(append(resp, []byte("\n")...))
conn.Close()
debug.FreeOSMemory()
return
}
So in this case the response is big, and a goroutine using 10% of the memory, thats okay, because I'm getting 170.000 users from the database and parse the result to JSON. But when the handleRequest and it is still in the memory, if I'm not using debug.FreeOsMemory(). I have doubts it is a good way to do it because it is in the debug pacakge so my question is it a good way to empty the memory what the goroutines are using? I tested it so it is not affected to the system and working very well. If not, what is the good way? I can't wait the GC for clear it up?! I read this and this is why I started to use it, in the first answer there is the last suggestion.
The Go runtime does not release free memory back to the OS "immediately", it would be inefficient. Read more about it here: Golang - Cannot free memory once occupied by bytes.Buffer.
You should let the Go runtime handle this. If your app is unstable without calling debug.FreeOsMemory(), there are bigger problems which you shouldn't cover up with this even if it "seemingly" helps. It may even make things worse, as if serving a request does require a large amount of memory (which is properly freed by the GC when done with the request), calling FreeOsMemory() will just return it to the OS which the runtime will have to ask for / allocate again when serving another request. Should you have not handed it back to the OS, it would be available for the next request...
Try to decrease the memory requirement of the request handler. If it is not possible (questionable), then limit the number of requests requiring large memory that may be served concurrently.
See this question+answer how to do that: Process Management for the Go Webserver
Also: Is this an idiomatic worker thread pool in Go?
As an alternative to occasionally invoking debug.FreeOsMemory(), you can control the aggressiveness of the garbage collector with the GOGC environment variable or with the debug.SetGCPercent() method.
Empirically, I've seen that with a value as low as 10, it does its job.
See Package runtime
GO lang clean all memory that is not in use, but is not inmediate. You can read the other question:
Golang - Cannot free memory once occupied by bytes.Buffer
That is crazy, I think is a bug for golang,My code include:
go func() {
for {
debug.FreeOSMemory()
log4sys.Warn("NumGoroutine:",runtime.NumGoroutine())
time.Sleep(1 * time.Minute)
}
}()
to solution this question