I'm looking to add what's essentially a "status" boolean field to my gRPC Server.
The idea is that the state of the server could be set when certain conditions are met, so when any new gRPC requests come in, the response would be tailored accordingly.
I have a server struct that I'm using, and for this example, I've added a HappyHour bool field.
I have a goroutine that runs when the server is started, and every minute it checks the time of day. If the time of day is between 5-7pm, HappyHour gets set to true. Otherwise, it gets set to false.
type server struct {
HappyHour bool
mu sync.RWMutex
}
func (s *server) OrderDrink(ctx context.Context, req *bar.DrinkRequest) (*flatbuffers.Builder, error) {
drink := string(req.drink())
qty := req.qty()
// Code here to look up the price of the drink in the database
total := price*qty
// Use a mutex to check HappyHour status
s.mu.RLock()
defer mu.RLock()
if s.HappyHour == true {
total=total/2
}
b := flatbuffers.NewBuilder(0)
// Serialize the response and send it to the client
}
func main() {
lis, err := net.Listen("tcp", "0.0.0.0:50051")
if err != nil {
log.Fatal(err)
}
s := grpc.NewServer(grpc.CustomCodec(flatbuffers.FlatbuffersCodec{}))
var srv server
bar.RegisterBARServer(s, &srv)
// Monitor happy hour
go happyHour(&srv)
}
Things seem to be working from my initial testing, but I'm quite new to gRPC, and am a bit worried about using a mutex on the server struct.
Will there be negative performance impacts if there are many clients connecting to the server? If I'm reading or updating HappyHour, and the lock has been acquired, would this impact new users connecting or sending requests? Or would this only impact the actual HappyHour field?
Related
I am trying to construct a receiver and sender pattern using two channels in Golang. I am doing a task (API call), and receiving back a Response struct. My goal is that when a response is received I'd like to send it to another channel (writeChan) for additional processing.
I'd like to continuously read/listen on that receiver channel (respChan) and process anything that comes through (such as a Response). Then I'd like to spin up a thread to go and do a further operation with that Response in another goroutine.
I'd like to understand how I can chain together this pattern to allow data to flow from the API calls and concurrently write it (each Response will be written to a separate file destination which the Write() func handles.
Essentially my current pattern is the following:
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
respChan := make(chan Response) // Response is a struct that contains API response metadata
defer close(respChan)
// requests is just a slice of requests to be made to an API
// This part is working well
for _, req := range requests {
wg.Add(1)
go func(r Request) {
defer wg.Done()
resp, _ := r.Get() // Make the API call and receive back a Response struct
respChan <- resp // Put the response into our channel
}(req)
}
// Now, I want to extract the responses as they become available and send them to another function to do some processing. This I am unsure of how to handle properly
writeChan := make(chan string)
defer close(writeChan)
select {
case resp := <-respChan: // receive from response channel
go func(response Response) {
signal, _ := Write(response) // Separate func to write the response to a file. Not important here in this context.
writeChan <- signal // Put the signal data into the channel which is a string file path of where the file was written (will be used for a later process)
}(resp)
case <-time.After(15 *time.Second):
fmt.Println("15 seconds have passed without receiving anything...")
}
wg.Wait()
}
Let me share with you a working example that you can benefit from. First, I'm gonna present the code, then, I'm gonna walk you through all the relevant sections.
package main
import (
"fmt"
"net/http"
"os"
"strings"
"time"
)
type Request struct {
Url string
DelayInSeconds time.Duration
}
type Response struct {
Url string
StatusCode int
}
func main() {
requests := []Request{
{"https://www.google.com", 0},
{"https://stackoverflow.com", 1},
{"https://www.wikipedia.com", 4},
}
respChan := make(chan Response)
defer close(respChan)
for _, req := range requests {
go func(r Request) {
fmt.Printf("%q - %v\n", r.Url, strings.Repeat("#", 30))
// simulate heavy work
time.Sleep(time.Second * r.DelayInSeconds)
resp, _ := http.Get(r.Url)
res := Response{r.Url, resp.StatusCode}
fmt.Println(time.Now())
respChan <- res
}(req)
}
writeChan := make(chan struct{})
defer close(writeChan)
for i := 0; i < len(requests); i++ {
select {
case res := <-respChan:
go func(r Response) {
f, err := os.Create(fmt.Sprintf("%v.txt", strings.Replace(r.Url, "https://", "", 1)))
if err != nil {
panic(err)
}
defer f.Close()
f.Write([]byte(fmt.Sprintf("%q OK with %d\n", r.Url, r.StatusCode)))
writeChan <- struct{}{}
}(res)
case <-time.After(time.Second * 2):
fmt.Println("Timeout")
}
}
}
Set up
First, I've defined the two structs that will be used in the example: Request and Response. In the former, I put a DelayInSeconds to mock some heavy loads and time-consuming operations. Then, I defined the requests variable that contains all the requests that have to be done.
The writing part
Here, I range over the requests variable. For each request, I'm gonna issue an HTTP request to the target URL. The time.Sleep emulate the heavy load. Then, I write the response to the respChan channel which is unbuffered.
The reading part
Here, the major change is to wrap the select construct into a for loop. Thanks to this, we'll make sure to iterate the right times (based on the length of the requests variable).
Final notes
First of all, bear in mind that the code is oversimplified just to show off the relevant parts. Due to this, a lot of error handling is missing and some inline functions could be extrapolated into named functions. You don't need to use sync.WaitGroup to achieve what you need, the usage of channels will be enough.
Feel free to play with delays and check which files are written!
Let me know if this helps you!
Edit
As requested, I'm gonna provide you with a more accurate solution based on your needs. The new reading part will be something like the following:
count := 0
for {
// this check is need to exit the for loop and not wait indefinitely
// it can be removed based on your needs
if count == 3 {
fmt.Println("all responses arrived...")
return
}
res := <-respChan
count++
go func(r Response) {
f, err := os.Create(fmt.Sprintf("%v.txt", strings.Replace(r.Url, "https://", "", 1)))
if err != nil {
panic(err)
}
defer f.Close()
f.Write([]byte(fmt.Sprintf("%q OK with %d\n", r.Url, r.StatusCode)))
writeChan <- struct{}{}
}(res)
}
Here, the execution is waiting indefinitely within the for loop. No matter how long each request takes to complete, it will be fetched as soon as it arrives. I put, at the top of the for loop, an if to exit after it processed the requests that we need. However, you can avoid it and let the code run till a cancellation signal comes in (it's up to you).
Let me know if this better meets your requirements, thanks!
I've been working on a sort-of pub-sub mechanism in the application we're building. The business logic basically generates a whack-ton of events, which in turn can be used to feed data to the client using API's, or persistent in storage if the application is running with that option enabled.
What we had, and observed:
Long story short, it turns out we were dropping data we really ought not to have been dropping. The "subscriber" had a channel with a large buffer, and essentially only read data from this channel, checked a few things, and appended it to a slice. The capacity of the slice is such that memory allocations were kept to a minimum. Simulating a scenario where the subscriber channel had a buffer of, say, 1000 data-sets, we noticed data could be dropping after only 10 sets being sent. The very first event was never dropped.
The code we had at this point looks something like this:
type broker struct {
ctx context.Context
subs []*sub
}
type sub struct {
ctx context.Context
mu *sync.Mutex
ch chan []interface{}
buf []interface{}
}
func (b *broker) Send(evts ...interface{}) {
rm := make([]int, 0, len(b.subs))
defer func() {
for i := len(rm) - 1; i >= 0; i-- {
// last element
if i == len(b.subs)-1 {
b.subs = b.subs[:i]
continue
}
b.subs = append(b.subs[:i], b.subs[i+1:]...)
}
}()
for i, s := range b.subs {
select {
case <-b.ctx.Done(): // is the app still running
return
case <-s.Stopped(): // is this sub still valid
rm = append(rm, i)
case s.C() <- evts: // can we write to the channel
continue
default: // app is running, sub is valid, but channel is presumably full, skip
fmt.Println("skipped some events")
}
}
}
func NewSub(ctx context.Context, buf int) *sub {
s := &sub{
ctx: ctx,
mu: &sync.Mutex{},
ch: make(chan []interface{}, buf),
buf: make([]interface{}, 0, buf),
}
go s.loop(ctx) // start routine to consume events
return s
}
func (s *sub) C() chan<- []interface{} {
return s.ch
}
func (s *sub) Stopped() <-chan struct{} {
return s.ctx.Done()
}
func (s *sub) loop(ctx context.Context) {
defer func() {
close(s.ch)
}()
for {
select {
case <-ctx.Done():
return
case data := <-s.ch:
// do some processing
s.mu.Lock()
s.buf = append(s.buf, data...)
s.mu.Unlock()
}
}
}
func (s *sub) GetProcessedData(amt int) []*wrappedT {
s.mu.Lock()
data := s.buf
if len(data) == amt {
s.buf = s.buf[:0]
} else if len(data) > amt {
data = data[:amt]
s.buf = s.buf[amt:]
} else {
s.buf = make([]interface{}, 0, cap(s.buf))
}
s.mu.Unlock()
ret := make([]*wrappedT, 0, len(data))
for _, v := range data {
// some processing
ret = append(ret, &wrappedT{v})
}
return ret
}
Now obviously, the buffers are there to ensure that events can still be consumed when we're calling things like GetProcessedData. That type of call is usually the result of an API request, or some internal flush/persist to storage mechanism. Because of the mutex lock, we might not be reading from the internal channel. Weirdly, the channel buffers never got backed up all the way through, but not all data made its way through to the subscribers. As mentioned, the first event always did, which made us even more suspicious.
What we eventually tried (to fix):
After a fair bit of debugging, hair pulling, looking at language specs, and fruitless googling I began to suspect the select statement to be the problem. Instead of sending to the channels directly, I changed it to the rather hacky:
func (b *broker) send(s *sub, evts []interface{}) {
ctx, cfunc := context.WithTimeout(b.ctx, 100 *time.Millisecond)
defer cfunc()
select {
case <-ctx:
return
case sub.C() <- evts:
return
case <-sub.Closed():
return
}
}
func (b *broker) Send(evts ...interface{}) {
for _, s := range b.subs {
go b.send(s, evts)
}
}
Instantly, all events were correctly propagated through the system. Calling Send on the broker wasn't blocking the part of the system that actually performs the heavy lifting (that was the reason for the use of channels after all), and things are performing reasonably well.
The actual question(s):
There's a couple of things still bugging me:
The way I read the specs, the default statement ought to be the last resort, solely as a way out to prevent blocking channel operations in a select statement. Elsewhere, I read that the runtime may not consider a case ready for communication if there is no routine consuming what you're about to write to the channel, irrespective of channel buffers. Is this indeed how it works?
For the time being, the context with timeout fixes the bigger issue of data not propagating properly. However, I do feel like there should be a better way.
Has anyone ever encountered something similar, and worked out exactly what's going on?
I'm happy to provide more details where needed. I've kept the code as minimal as possible, omitting a lot of complexities WRT the broker system we're using (different event types, different types of subscribers, etc...). We don't use the interface{} type anywhere in case anyone is worried about that :P
For now, though, I think this is plenty of text for a Friday.
I'm using a rate limiter to throttle the number of requests that are routed
The requests are sent to a channel, and I want to limit the number that are processed per second but i'm struggling to understand if i'm setting this correctly, I don't get an error, but i'm unsure if i'm even using the rate limiter
This is what is being added to the channel:
type processItem struct {
itemString string
}
Here's the channel and limiter:
itemChannel := make(chan processItem, 5)
itemThrottler := rate.NewLimiter(4, 1) //4 a second, no more per second (1)
var waitGroup sync.WaitGroup
Items are added to the channel:
case "newItem":
waitGroup.Add(1)
itemToExec := new(processItem)
itemToExec.itemString = "item string"
itemChannel <- *itemToExec
Then a go routine is used to process everything that is added to the channel:
go func() {
defer waitGroup.Done()
err := itemThrottler.Wait(context.Background())
if err != nil {
fmt.Printf("Error with limiter: %s", err)
return
}
for item := range itemChannel {
execItem(item.itemString) // the processing function
}
defer func() { <-itemChannel }()
}()
waitGroup.Wait()
Can someone confirm that the following occurs:
The execItem function is run on each member of the channel 4 times a second
I don't understand what "err := itemThrottler.Wait(context.Background())" is doing in the code, how is this being invoked?
... i'm unsure if i'm even using the rate limiter
Yes, you are using the rate-limiter. You are rate-limiting the case "newItem": branch of your code.
I don't understand what "err := itemThrottler.Wait(context.Background())" is doing in the code
itemThrottler.Wait(..) will just stagger requests (4/s i.e. every 0.25s) - it does not refuse requests if the rate is exceeded. So what does this mean? If you receive a glut of 1000 requests in 1 second:
4 requests will be handled immediately; but
996 requests will create a backlog of 996 go-routines that will block
The 996 will unblock at a rate of 4/s and thus the backlog of pending go-routines will not clear for another 4 minutes (or maybe longer if more requests come in). A backlog of go-routines may or may not be what you want. If not, you may want to use Limiter.Allow - and if false is returned, then refuse the request (i.e. don't create a go-routine) and return a 429 error (if this is a HTTP request).
Finally, if this is a HTTP request, you should use it's imbedded context when calling Wait e.g.
func (a *app) myHandler(w http.ResponseWriter, r *http.Request) {
// ...
err := a.ratelimiter(r.Context())
if err != nil {
// client http request most likely canceled (i.e. caller disconnected)
}
}
I am trying to create an intermediate layer between user and tcp, with Send and Receive functions. Currently, I am trying to integrate a context, so that the Send and Receive respects a context. However, I don't know how to make them respect the context's cancellation.
Until now, I got the following.
// c.underlying is a net.Conn
func (c *tcpConn) Receive(ctx context.Context) ([]byte, error) {
if deadline, ok := ctx.Deadline(); ok {
// Set the read deadline on the underlying connection according to the
// given context. This read deadline applies to the whole function, so
// we only set it once here. On the next read-call, it will be set
// again, or will be reset in the else block, to not keep an old
// deadline.
c.underlying.SetReadDeadline(deadline)
} else {
c.underlying.SetReadDeadline(time.Time{}) // remove the read deadline
}
// perform reads with
// c.underlying.Read(myBuffer)
return frameData, nil
}
However, as far as I understand that code, this only respects a context.WithTimeout or context.WithDeadline, and not a context.WithCancel.
If possible, I would like to pass that into the connection somehow, or actually abort the reading process.
How can I do that?
Note: If possible, I would like to avoid another function that reads in another goroutine and pushed a result back on a channel, because then, when calling cancel, and I am reading 2GB over the network, that doesn't actually cancel the read, and the resources are still used. If not possible in another way however, I would like to know if there is a better way of doing that than a function with two channels, one for a []byte result and one for an error.
EDIT:
With the following code, I can respect a cancel, but it doesn't abort the read.
// apply deadline ...
result := make(chan interface{})
defer close(result)
go c.receiveAsync(result)
select {
case res := <-result:
if err, ok := res.(error); ok {
return nil, err
}
return res.([]byte), nil
case <-ctx.Done():
return nil, ErrTimeout
}
}
func (c *tcpConn) receiveAsync(result chan interface{}) {
// perform the reads and push either an error or the
// read bytes to the result channel
If the connection can be closed on cancellation, you can setup a goroutine to shutdown the connection on cancellation within the Receive method. If the connection must be reused again later, then there is no way to cancel a Read in progress.
recvDone := make(chan struct{})
defer close(recvDone)
// setup the cancellation to abort reads in process
go func() {
select {
case <-ctx.Done():
c.underlying.CloseRead()
// Close() can be used if this isn't necessarily a TCP connection
case <-recvDone:
}
}()
It will be a little more work if you want to communicate the cancelation error back, but the CloseRead will provide a clean way to stop any pending TCP Read calls.
I am trying to understand GoLang "Go" together with gRPC and to make a simple service scalable.
Lets say I have a client1 that calls a service1(adds numbers) that calls service2(determines if the result is prime), and service2 returns the result to service1 that returns the result to client1 all via gRPC.
When I use protocol buffers "proto3" and generate the Go code via protoc.
I get generated methods that call the service in one particular way.
I see no distinction to call the methods asynchronously "Go".
And the underlying call seems to be "Invoke" which I believe is synchronous,the call returns once a result is received.
How do I make service1 "performant", I know I can run this in a cluster and have copies, but that would mean I can only serve clients as per the amount of instances within the cluster.
I want a "single" service to be able to serve multiple clients(e.g. 1000) .
Here is a simple server and I am not sure if this is performant or not:
I do know that the getprime function does dial every time,
and this could probably be moved to make this dial persist and be re-used; But more importantly I want to make a simple performant scaleable service and get a good understanding.
(A)
Perhaps the whole design is incorrect and the service1 should just return
as soon as the instruction is received "ack", do the addition and send the next request to sercice2 which determines if the answer is prime or not; again service2 just responds with an acknowledgement of the request being received. Once prime is determined by the service2 a call is made to the client with an answer.
If (A) above is the better approach, then still please explain the bottlenecks below; what happens when multiple clients are processed?
The call to "Listen" does what, "blocks, or does not block", etc.
package main
import (
pb "demo/internal/pkg/proto_gen/calc"
"fmt"
"golang.org/x/net/context"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
"log"
"net"
)
const (
port = ":8080"
)
type service struct {
}
func (s *service) Calculate(ctx context.Context, req *pb.Instruction) (*pb.Response, error) {
var answer float64
answer = req.Number1 + req.Number2
// call service prime
p := getprime(int(answer))
pa := pb.PrimeAnswer{Prime: p}
return &pb.Response{Answer: answer, Prime: &pa}, nil
}
const (
primeAddress = "127.0.0.1:8089"
)
func getprime(number int) bool {
conn, err := grpc.Dial(primeAddress, grpc.WithInsecure())
if err != nil {
log.Fatalf("Did not connect to prime service: %v", err)
}
defer conn.Close()
client := pb.NewPrimeServiceClient(conn)
p := pb.PrimeMessage{"", float64(number)}
r, err := client.Prime(context.Background(), &p)
if err != nil {
log.Fatalf("Call to prime service failed: %v", err)
}
return r.Prime
}
func main() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterCalculatorServer(s, &service{})
reflection.Register(s)
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
Thanks for your question. It is true that gRPC-Go is sync only; that is your Unary RPC(the one in your example) will return only when the RPC has finished (got a response from the server).
About performance:
The Dial operation establishes an underlying connection which may be expensive. So it not wise to do it every time getprime is called. A better way is to create a client, keep it around and make calls to the prime server on it. This way only first RPC incurs the cost of connection.
For each RPC request a server gets we launch a goroutine to process that request. So in general, this should scale fairly well.
About (A): It is not uncommon for a service handler to make an RPC call to yet another server and wait for its response before returning back.
Note that there's no way for a server to make call to the client.
To phrase what JimB said as an answer: "Synchronous" means that the function that makes the remote call waits for a reply before continuing, not that the whole server or client does. The server is normally multithreaded, even when processing synchronous calls; it can accept and work on a second call while it's responding to the first.
And similarly, if a client has multiple concurrent tasks that each have a gRPC call running, that won't block the process. Clients like that could include net/http servers serving end users, or gRPC servers handling multiple RPCs.
Where you might add explicit go statements is if you want to do something else from the specific function making the RPC call. For example, if you want to issue several RPC calls at once then wait for all their results to come in, you could write code following the examples of fan-out calls.