I am writing a web application where I have a long running goroutine.
I want to delegate all HTTP requests to this goroutine via channels.
The pattern that I have come across is:
// Internal long running goroutine
for{
select{
case e := <-event: //web request
req := e.req
// do something
....
select {
case <-ctx.Done():
//log
default:
e.replyTo <- result
}
}
}
// Web handler
http.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) {
//decode request etc
...
replyTo := make(chan interface{}, 1)
ctx, cancel := context.WithCancel(context.BackGround())
event <- Event{req: req, ctx: ctx, replyTo: replyTo}
select{
case <-time.After(time.Second):
cancel()
//return 500
case r := <-replyTo:
// return some response
}
})
I do see that there is one single go-routine at the end, so parallelism is lost but I am okay with it.
Is this pattern the right way of doing this?
What other approaches can be suggested?
Is this pattern the right way of doing this?
Assuming you are trying to manage where state in a single go routine, I would say no. I think it would be better to have some form of a state manager that is responsible for thread safety. Therefore the handler should take in something that can manage the state and simply expose a few methods to the handler.
type State interface{
Load() (string, error)
Save(something string) error
}
Decoupling the code will pay off for you later on. It will also allow unit tests for both the handler and the State that can be focused and readable.
Related
I am writing wrapper on top of nats client in golang, I want to take handler function which can be invoked from consumer once I get the message from nats server.
I want to hold custom subscribe method until it receives the message from nats.
Publish:
func (busConfig BusConfig) Publish(service string, data []byte) error {
pubErr := conn.Publish(service, data)
if pubErr != nil {
return pubErr
}
return nil
}
Subscribe:
func (busConfig BusConfig) Subscribe(subject string, handler func(msg []byte)) {
fmt.Println("Subscrbing on : ", subject)
//wg := sync.WaitGroup{}
//wg.Add(1)
subscription, err := conn.Subscribe(subject, func(msg *nats.Msg) {
go func() {
handler(msg.Data)
}()
//wg.Done()
})
if err != nil {
fmt.Println("Subscriber error : ", err)
}
//wg.Wait()
defer subscription.Unsubscribe()
}
test case:
func TestLifeCycleEvent(t *testing.T) {
busClient := GetBusClient()
busClient.Subscribe(SUBJECT, func(input []byte) {
fmt.Println("Life cycle event received :", string(input))
})
busClient.Publish(SUBJECT, []byte("complete notification"))
}
I am seeing message is published but not subscribed, I tried to hold subscribe method using waitgroup but I think this is not the correct solution.
You don't see the message being delivered because Subscribe is an async method that spawns a goroutine to handle the incoming messages and call the callback.
Straight after calling busClient.Publish() your application exits. It does not wait for anything to happen inside Subscribe().
When you use nats.Subscribe(), you usually have a long-running application that exits in specific conditions (like receiving a shutdown signal). WaitGroup can work here, but probably not for real applications, just for tests.
You should also call Flush() method on NATS connection to ensure all buffered messages have been sent before exiting the program.
If you want a synchronous method, you can use nats.SubscribeSync()
Check out examples here: https://natsbyexample.com/examples/messaging/pub-sub/go
For my understanding, I think in NATs we need to respond to the message even if we are not providing the reply address, so it can respond to the message.
func (busConfig BusConfig) Subscribe(subject string, handler func(msg []byte)) {
subscription, err := conn.Subscribe(subject, func(msg *nats.Msg) {
go func() {
handler(msg.Data)
msg.Respond(nil)
}()
})
}
I have written an API that makes DB calls and does some business logic. I am invoking a goroutine that must perform some operation in the background.
Since the API call should not wait for this background task to finish, I am returning 200 OK immediately after calling the goroutine (let us assume the background task will never give any error.)
I read that goroutine will be terminated once the goroutine has completed its task.
Is this fire and forget way safe from a goroutine leak?
Are goroutines terminated and cleaned up once they perform the job?
func DefaultHandler(w http.ResponseWriter, r *http.Request) {
// Some DB calls
// Some business logics
go func() {
// some Task taking 5 sec
}()
w.WriteHeader(http.StatusOK)
}
I would recommend always having your goroutines under control to avoid memory and system exhaustion.
If you are receiving a spike of requests and you start spawning goroutines without control, probably the system will go down soon or later.
In those cases where you need to return an immediate 200Ok the best approach is to create a message queue, so the server only needs to create a job in the queue and return the ok and forget. The rest will be handled by a consumer asynchronously.
Producer (HTTP server) >>> Queue >>> Consumer
Normally, the queue is an external resource (RabbitMQ, AWS SQS...) but for teaching purposes, you can achieve the same effect using a channel as a message queue.
In the example you'll see how we create a channel to communicate 2 processes.
Then we start the worker process that will read from the channel and later the server with a handler that will write to the channel.
Try to play with the buffer size and job time while sending curl requests.
package main
import (
"fmt"
"log"
"net/http"
"time"
)
/*
$ go run .
curl "http://localhost:8080?user_id=1"
curl "http://localhost:8080?user_id=2"
curl "http://localhost:8080?user_id=3"
curl "http://localhost:8080?user_id=....."
*/
func main() {
queueSize := 10
// This is our queue, a channel to communicate processes. Queue size is the number of items that can be stored in the channel
myJobQueue := make(chan string, queueSize) // Search for 'buffered channels'
// Starts a worker that will read continuously from our queue
go myBackgroundWorker(myJobQueue)
// We start our server with a handler that is receiving the queue to write to it
if err := http.ListenAndServe("localhost:8080", myAsyncHandler(myJobQueue)); err != nil {
panic(err)
}
}
func myAsyncHandler(myJobQueue chan<- string) http.HandlerFunc {
return func(rw http.ResponseWriter, r *http.Request) {
// We check that in the query string we have a 'user_id' query param
if userID := r.URL.Query().Get("user_id"); userID != "" {
select {
case myJobQueue <- userID: // We try to put the item into the queue ...
rw.WriteHeader(http.StatusOK)
rw.Write([]byte(fmt.Sprintf("queuing user process: %s", userID)))
default: // If we cannot write to the queue it's because is full!
rw.WriteHeader(http.StatusInternalServerError)
rw.Write([]byte(`our internal queue is full, try it later`))
}
return
}
rw.WriteHeader(http.StatusBadRequest)
rw.Write([]byte(`missing 'user_id' in query params`))
}
}
func myBackgroundWorker(myJobQueue <-chan string) {
const (
jobDuration = 10 * time.Second // simulation of a heavy background process
)
// We continuosly read from our queue and process the queue 1 by 1.
// In this loop we could spawn more goroutines in a controlled way to paralelize work and increase the read throughput, but i don't want to overcomplicate the example.
for userID := range myJobQueue {
// rate limiter here ...
// go func(u string){
log.Printf("processing user: %s, started", userID)
time.Sleep(jobDuration)
log.Printf("processing user: %s, finisehd", userID)
// }(userID)
}
}
There is no "goroutine cleaning" you have to handle, you just launch goroutines and they'll be cleaned when the function launched as a goroutine returns. Quoting from Spec: Go statements:
When the function terminates, its goroutine also terminates. If the function has any return values, they are discarded when the function completes.
So what you do is fine. Note however that your launched goroutine cannot use or assume anything about the request (r) and response writer (w), you may only use them before you return from the handler.
Also note that you don't have to write http.StatusOK, if you return from the handler without writing anything, that's assumed to be a success and HTTP 200 OK will be sent back automatically.
See related / possible duplicate: Webhook process run on another goroutine
#icza is absolutely right there is no "goroutine cleaning" you can use a webhook or a background job like gocraft. The only way I can think of using your solution is to use the sync package for learning purposes.
func DefaultHandler(w http.ResponseWriter, r *http.Request) {
// Some DB calls
// Some business logics
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
// some Task taking 5 sec
}()
w.WriteHeader(http.StatusOK)
wg.wait()
}
you can wait for a goroutine to finish using &sync.WaitGroup:
// BusyTask
func BusyTask(t interface{}) error {
var wg = &sync.WaitGroup{}
wg.Add(1)
go func() {
// busy doing stuff
time.Sleep(5 * time.Second)
wg.Done()
}()
wg.Wait() // wait for goroutine
return nil
}
// this will wait 5 second till goroutune finish
func main() {
fmt.Println("hello")
BusyTask("some task...")
fmt.Println("done")
}
Other way is to attach a context.Context to goroutine and time it out.
//
func BusyTaskContext(ctx context.Context, t string) error {
done := make(chan struct{}, 1)
//
go func() {
// time sleep 5 second
time.Sleep(5 * time.Second)
// do tasks and signle done
done <- struct{}{}
close(done)
}()
//
select {
case <-ctx.Done():
return errors.New("timeout")
case <-done:
return nil
}
}
//
func main() {
fmt.Println("hello")
ctx, cancel := context.WithTimeout(context.TODO(), 2*time.Second)
defer cancel()
if err := BusyTaskContext(ctx, "some task..."); err != nil {
fmt.Println(err)
return
}
fmt.Println("done")
}
I'm trying to stop all clients connected to a stream server from server side.
Actually I'm using GracefulStop method to handle it gracefully.
I am waiting for os.Interrupt signal on a channel to perform a graceful stop for gRPC. but it gets stuck on server.GracefulStop() when the client is connected.
func (s *Service) Subscribe(_ *empty.Empty, srv clientapi.ClientApi_SubscribeServer) error {
ctx := srv.Context()
updateCh := make(chan *clientapi.Update, 100)
stopCh := make(chan bool)
defer func() {
stopCh<-true
close(updateCh)
}
go func() {
ticker := time.NewTicker(1 * time.Second)
defer func() {
ticker.Stop()
close(stopCh)
}
for {
select {
case <-stopCh:
return
case <-ticker.C:
updateCh<- &clientapi.Update{Name: "notification": Payload: "sample notification every 1 second"}
}
}
}()
for {
select {
case <-ctx.Done():
return ctx.Err()
case notif := <-updateCh:
err := srv.Send(notif)
if err == io.EOF {
return nil
}
if err != nil {
s.logger.Named("Subscribe").Error("error", zap.Error(err))
continue
}
}
}
}
I expected the context in method ctx.Done() could handle it and break the for loop.
How to close all response streams like this one?
Create a global context for your gRPC service. So walking through the various pieces:
Each gRPC service request would use this context (along with the client context) to fulfill that request
os.Interrupt handler would cancel the global context; thus canceling any currently running requests
finally issue server.GracefulStop() - which should wait for all the active gRPC calls to finish up (if they haven't see the cancelation immediately)
So for example, when setting up the gRPC service:
pctx := context.Background()
globalCtx, globalCancel := context.WithCancel(pctx)
mysrv := MyService{
gctx: globalCtx
}
s := grpc.NewServer()
pb.RegisterMyService(s, mysrv)
os.Interrupt handler initiates and waits for shutdown:
globalCancel()
server.GracefulStop()
gRPC methods:
func(s *MyService) SomeRpcMethod(ctx context.Context, req *pb.Request) error {
// merge client and server contexts into one `mctx`
// (client context will cancel if client disconnects)
// (server context will cancel if service Ctrl-C'ed)
mctx, mcancel := mergeContext(ctx, s.gctx)
defer mcancel() // so we don't leak, if neither client or server context cancels
// RPC WORK GOES HERE
// RPC WORK GOES HERE
// RPC WORK GOES HERE
// pass mctx to any blocking calls:
// - http REST calls
// - SQL queries etc.
// - or if running a long loop; status check the context occasionally like so:
// Example long request (10s)
for i:=0; i<10*1000; i++ {
time.Sleep(1*time.Milliscond)
// poll merged context
select {
case <-mctx.Done():
return fmt.Errorf("request canceled: %s", mctx.Err())
default:
}
}
}
And:
func mergeContext(a, b context.Context) (context.Context, context.CancelFunc) {
mctx, mcancel := context.WithCancel(a) // will cancel if `a` cancels
go func() {
select {
case <-mctx.Done(): // don't leak go-routine on clean gRPC run
case <-b.Done():
mcancel() // b canceled, so cancel mctx
}
}()
return mctx, mcancel
}
Typically clients need to assume that RPCs can terminate (e.g. due to connection errors or server power failure) at any moment. So what we do is GracefulStop, sleep for a short time period to allow in-flight RPCs an opportunity to complete naturally, then hard-Stop the server. If you do need to use this termination signal to end your RPCs, then the answer by #colminator is probably the best choice. But this situation should be unusual, and you may want to spend some time analyzing your design if you do find it is necessary to manually end streaming RPCs at server shutdown.
I'm writing a test app with SSE, but my problem is that
ReadTimeout and WriteTimeout are closing the clients connection every 10 Seconds and because of this the main page are losing data.
How can I manage this Issue, serving SSE and webpages together without goroutines leak risk and SSE working done?
Server:
server := &http.Server{Addr: addr,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
Handler: s.mainHandler(),
}
Handler:
func sseHandler(w http.ResponseWriter, r *http.Requests) {
f, ok := w.(http.Flusher)
if !ok {
http.Error(w, "Streaming not supported!", http.StatusInternalServerError)
log.Println("Streaming not supported")
return
}
messageChannel := make(chan string)
hub.addClient <- messageChannel
notify := w.(http.CloseNotifier).CloseNotify()
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
for i := 0; i < 1440; {
select {
case msg := <-messageChannel:
jsonData, _ := json.Marshal(msg)
str := string(jsonData)
fmt.Fprintf(w, "data: {\"str\": %s, \"time\": \"%v\"}\n\n", str, time.Now())
f.Flush()
case <-time.After(time.Second * 60):
fmt.Fprintf(w, "data: {\"str\": \"No Data\"}\n\n")
f.Flush()
i++
case <-notify:
f.Flush()
i = 1440
hub.removeClient <- messageChannel
}
}
}
Both ReadTimeout and WriteTimeout define the duration within which the whole request must be read from or written back to the client. These timeouts are applied to the underlying connection (http://golang.org/src/pkg/net/http/server.go?s=15704:15902) and this is before any headers are received, so you cannot set different limits for separate handlers – all connections within the server will share the same timeout limits.
Saying this, if you need customised timeouts per request, you will need to implement them in your handler. In your code you're already using timeouts for your job, so this would be a matter of creating a time.After at the beginning of the handler, keep checking (in the handler itself or even pass it around) and stop the request when necessary. This actually gives you more precise control over the timeout, as WriteTimeout would only trigger when trying to write the response (e.g. if the timeout is set to 10 seconds and it takes a minute to prepare the response before any write, you won't get any error until the job is done so you'll waste resources for 50 seconds). From this perspective, I think WriteTimeout itself is good only when your response is ready quite quickly, but you want to drop the connection when network become very slow (or the client deliberately stops receiving data).
There is a little helper function in the library, http.TimeoutHandler, which behaves similarly to WriteTimeout, but sends back 503 error if the response takes longer than predefined time. You could use it to set up behaviour similar to WriteTimeout per handler, for example:
package main
import (
"log"
"net/http"
"time"
)
type Handler struct {
}
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
time.Sleep(3*time.Second)
// This will return http.ErrHandlerTimeout
log.Println(w.Write([]byte("body")))
}
func main() {
h := &Handler{}
http.Handle("/t1", http.TimeoutHandler(h, 1*time.Second, ""))
http.Handle("/t2", http.TimeoutHandler(h, 2*time.Second, ""))
http.ListenAndServe(":8080", nil)
}
This looks very handy, but I found one disadvantage that would affect your code: http.ResponseWriter passed from http.TimeoutHandler doesn't implement http.CloseNotifier. If it's required, you could dive into the implementation and see how they solved the timeout problem in order to come up with a similar, but enhanced solution.
A few months ago I was thinking how to implement a closable event loop in Go, for an RPC library. I managed to facilitate closing the server like so:
type Server struct {
listener net.Listener
closeChan chan bool
routines sync.WaitGroup
}
func (s *Server) Serve() {
s.routines.Add(1)
defer s.routines.Done()
defer s.listener.Close()
for {
select {
case <-s.closeChan:
// close server etc.
default:
s.listener.SetDeadline(time.Now().Add(2 * time.Second))
conn, _ := s.listener.Accept()
// handle conn routine
}
}
}
func (s *Server) Close() {
s.closeChan <- true // signal to close serve routine
s.routines.Wait()
}
The problem that I've found with this implementation is it involves a timeout, which means minimum close time is 2 seconds more than it could be. Is there a more idiomatic method of creating an event loop?
I don't think that event loops in Go need to be loops.
It would seem simpler to handle closing and connections in separate goroutines:
go func() {
<-s.closeChan
// close server, release resources, etc.
s.listener.Close()
}()
for {
conn, err := s.listener.Accept()
if err != nil {
// log, return
}
// handle conn routine
}
Note that you might also close the listener directly in your Close function without using a channel. What I have done here is used the error return value of Listener.Accept to facilitate inter-routine communication.
If at some point of the closing and connection handling implementations you need to protect some resources you're closing while you're answering, you may use a Mutex. But it's generally possible to avoid that.