How to add items from HTTP request to channel without waiting - go

I'm struggling a bit with understanding concurrency and could use a sanity check. I'm building a notification service and my goal is to have a endpoint in Gin Framework that receives textmessages that my service should send to different protocols (like SMS, email etc.)
My attached simplified code example works but after reading up on pitfalls and caveats with concurrency I'm very unconfident if it's even close to idiomatic.
In the code example Gin has one endpoint, notify, that receives POST request with a json payload. The payload is a json array containing notification objects (just a property with a message string).
Gin binds this array to a slice of pointers for type Notification in Go. Then a Goroutine adds these Notifications to a unbuffered channel Notifier.que. I do this in a Goroutine so that Gin can return status 200 to the HTTP request sender.
The Go application has a simple notification engine with a worker pool that receives Notifications on channel Notifier.que and then calls receiver function Notification.Send().
I have two questions:
1: Is it reasonable to add Notification to the channel Notifier.que using a Goroutine in this way? Can you see any danger with it?
2: Is it ok to have the worker pool inactive while the Notifier.que is empty. Would it be better to close the channel, kill the workers and create a new pool next time notifications is posted?
package main
import (
"fmt"
"net/http"
"time"
"github.com/gin-gonic/gin"
)
// A simple notification sender engine
type Notifier struct {
que chan *Notification // Que of notification to send
workers chan struct{} // Workerpool
}
// Start the notifier engine by generating workers for the pool
func (n *Notifier) Start() {
for i := 0; i < cap(n.workers); i++ {
go n.worker(i + 1)
}
}
// Defining a worker process
func (n *Notifier) worker(id int) {
for {
notification := <-n.que // Listens on channel que
notification.Send(id) // Processes a notification
}
}
// A simplified notification
type Notification struct {
Message string `json:"message"`
}
// Process a notification by calling it's send function
func (n *Notification) Send(workerId int) {
time.Sleep(500 * time.Millisecond)
fmt.Printf("Send '%v' from worker id %v\n", n.Message, workerId)
}
func main() {
// Initiare a notifier engine with four workers in pool
notifier := Notifier{
que: make(chan *Notification),
workers: make(chan struct{}, 4),
}
notifier.Start()
// Start gin engine
r := gin.Default()
r.POST("/notify", func(c *gin.Context) { // Route for POST to /notify
// Get notifications from json body in HTTP POST
var addToQue []*Notification
if err := c.ShouldBindJSON(&addToQue); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
// Generate new notification in the que, without waiting before sending HTTP response
go func() {
for _, notification := range addToQue {
notifier.que <- notification
}
}()
c.JSON(200, gin.H{"message": "OK"})
})
r.Run()
}
A curl request to send 25 notifications
curl --location --request POST '127.0.0.1:8080/notify' \
--header 'Content-Type: application/json' \
--data-raw '[
{"message":"Notice 1"},{"message":"Notice 2"},{"message":"Notice 3"},{"message":"Notice 4"},{"message":"Notice 5"},
{"message":"Notice 6"},{"message":"Notice 7"},{"message":"Notice 8"},{"message":"Notice 9"},{"message":"Notice 10"},
{"message":"Notice 11"},{"message":"Notice 12"},{"message":"Notice 13"},{"message":"Notice 14"},{"message":"Notice 15"},
{"message":"Notice 16"},{"message":"Notice 17"},{"message":"Notice 18"},{"message":"Notice 19"},{"message":"Notice 20"},
{"message":"Notice 21"},{"message":"Notice 22"},{"message":"Notice 23"},{"message":"Notice 24"},{"message":"Notice 25"}
]'

Related

Golang unexpected behaviour while implementing concurrency using channel and select statement

Below is my implementation of the Pub/Sub model in golang using concurrency. The code executes as expected sometimes but sometimes it gives output like this:
Output
message received on channel 2: Hello World
message received on channel 3: Hello World
message received on channel 1: Hello World
message received on channel 1:
subscriber 1's context cancelled
message received on channel 3: Only channels 2 and 3 should print this
message received on channel 2: Only channels 2 and 3 should print this
Key thing to note here is that subscriber 1 printed message received on channel 1: even after there is a call in the main function to remove it as a subscriber. Subscriber 1 should only receive one message. What seems to be happening is that subscriber receives a message from subscriber.out before the dctx is cancelled. How can I fix this. The subscriber's goroutine is launched in the AddSubscriber method.
Expected execution:
Channel 1 should only print once and then it's context should be cancelled and it's goroutine should exit.
main.go
package main
import (
"fmt"
"net/http"
)
func main() {
var err error
publisher := NewPublisher()
publisher.AddSubscriber()
publisher.AddSubscriber()
publisher.AddSubscriber()
publisher.Start()
err = publisher.Publish("Hello World")
if err != nil {
fmt.Printf("could not publish: %v\n", err)
}
err = publisher.RemoveSubscriber(1)
if err != nil {
fmt.Printf("could not remove subscriber: %v\n", err)
}
err = publisher.Publish("Only channels 2 and 3 should print this")
if err != nil {
fmt.Printf("could not publish: %v\n", err)
}
// this is merely to keep the server running
http.ListenAndServe(":8080", nil)
}
publisher.go
package main
import (
"context"
"errors"
"fmt"
"sync"
)
// Publisher sends messages received in its in channel
// to all the Subscribers listening to it.
type Publisher struct {
sequence uint
in chan string
subscribers map[uint]*Subscriber
sync.RWMutex
ctx context.Context
cancel *context.CancelFunc
}
// NewPublisher returns a new Publisher with zero subscribers.
// A Publishers needs to be started with the Publisher.Start()
// method before it can start publishing incoming messages.
func NewPublisher() *Publisher {
ctx, cancel := context.WithCancel(context.Background())
return &Publisher{subscribers: map[uint]*Subscriber{}, ctx: ctx, cancel: &cancel}
}
// AddSubscriber creates a new Subscriber that starts
// listening to the messages sent by this Publisher.
func (p *Publisher) AddSubscriber() {
dctx, cancel := context.WithCancel(p.ctx)
p.Lock()
nextId := p.sequence + 1
subscriber := NewSubscriber(nextId, dctx, &cancel)
p.subscribers[nextId] = subscriber
p.sequence = p.sequence + 1
p.Unlock()
go func() {
for {
select {
case <-p.ctx.Done():
fmt.Printf("parent context cancelled\n")
(*subscriber.cancel)()
return
case <-dctx.Done():
fmt.Printf("subscriber %d's context cancelled\n", subscriber.id)
return
case msg := <-subscriber.out:
fmt.Printf("message received on channel %d: %s\n", subscriber.id, msg)
}
}
}()
}
// Publish sends the msg to all the subscribers subscribed to it.
func (p *Publisher) Publish(msg string) error {
// publish only if Publisher has been started
if p.in == nil {
return errors.New("publisher not started yet")
}
// publish only if subscriber count > 0
p.RLock()
if len(p.subscribers) == 0 {
return errors.New("no subscribers to receive the message")
}
p.RUnlock()
// send message to in channel
// establish lock to ensure no modifications take place
// while the message is being sent
p.Lock()
p.in <- msg
p.Unlock()
return nil
}
// Start initializes the the publishers in channel
// and makes it ready to start publishing incoming messages.
func (p *Publisher) Start() {
in := make(chan string)
p.in = in
go func() {
for {
select {
case <-p.ctx.Done():
// using break here only breaks out of select statement
// instead of breaking out of the loop
fmt.Printf("done called on publisher\n")
return
case msg := <-p.in:
p.RLock()
for _, subscriber := range p.subscribers {
subscriber.out <- msg
}
p.RUnlock()
}
}
}()
}
// Stop prevents the Publisher from listening to any
// incoming messages by closing the in channel.
func (p *Publisher) Stop() {
// should I also remove all subscribers to prevent it from panicking?
(*p.cancel)()
}
// RemoveSubscriber removes the subscriber specified by the given id.
// It returns an error "could not find subscriber"
// if Subscriber with the given id is not subscribed to the Publisher.
func (p *Publisher) RemoveSubscriber(id uint) error {
p.Lock()
defer p.Unlock()
subscriber, ok := p.subscribers[id]
if !ok {
return errors.New("could not find subscriber")
}
(*subscriber.cancel)()
delete(p.subscribers, id)
close(subscriber.out)
return nil
}

How to communicate between a http request handler and websocket server using gin and gorilla/websocket?

I want to create a realtime api. Github Link of current code
There will be two types of users A and B.
Users A can connect to the service using a websocket to see realtime Updates.
Users B can only make http request to push data to mongodb. I have been following this tutorial. In this tutorial sqlitedb and redis pub/sub is used but i don't want to use that.
slotServer.go
...
func (server *WsServer) Run() {
gb := *models.GetInstanceGlobal()
for {
select {
case client := <-server.register:
server.registerClient(client)
case client := <-server.unregister:
server.unregisterClient(client)
case message := <-server.broadcast:
server.broadcastToClient(message)
case message := <-gb.Channel:
server.createRoomAndBroadCast(message)
}
}
}
...
func (server *WsServer) createRoom(name string) *Room {
room := NewRoom(name)
go room.RunRoom()
server.rooms[room] = true
return room
}
func (server *WsServer) createRoomAndBroadCast(name string) {
room := server.createRoom(name)
var roomList []string
for r := range server.rooms {
roomList = append(roomList, r.GetName())
}
addRoomReply := models.AddRoomReply{
RoomList: roomList,
AddedRoom: room.GetName(),
}
addReply := MessageAddRoom{
Data: addRoomReply,
Action: "room-add-success",
}
server.broadcast <- addReply.encode()
}
I am trying to a listen on channel global. If a string is pushed to it, a function createRoomAndBroadCast will be called.
models.go
....
type GlobalChannel struct {
Channel chan string
}
func GetInstanceGlobal() *GlobalChannel {
return &GlobalChannel{
Channel: make(chan string),
}
}
I am writing to this channel in POST message handler
room.go
...
//add room to mongo
if err := db.CreateRoom(&room); err != nil {
ctx.JSON(http.StatusBadGateway, gin.H{
"data": err,
})
return
}
//write it to channel
gb := models.GetInstanceGlobal()
gb.Channel <- *room.Name
// send reponse to user
ctx.JSON(http.StatusCreated, gin.H{
"data": "Room Created Successfully",
})
...
But my post request gets stuck at line gb := models.GetInstanceGlobal()
In the logs I see the following message
redirecting request 307: /api/v1/room/ --> /api/v1/room/
I don't understand whether I am doing something wrong or my logic is completely wrong.
I started reading about how channels work in golang and thanks to this post found that mentioning buffer limit in a channel is very important.
If ch is unbuffered, then ch <- msg will block until the message is consumed by a receiver
So I changed
models.go
//Channel: make(chan string),
Channel: make(chan string,100),
and it started transmitting the messages.
I'm only going to comment on the code you have shown here and not the overall design.
func GetInstanceGlobal() *GlobalChannel {
return &GlobalChannel{
Channel: make(chan string),
}
}
This is likely a misunderstanding of channels. While you have named it GetInstanceGlobal it is actually returning a brand new channel each time you call it. So if one side is calling it and then trying to receive messages and the other is calling it and pushing messages, the two sides will be different channels and never communicate. This explains why your push is blocking forever. And it also explains why when you add 100 to make it a buffered channel that appears to unblock it. But really all you have done is let it build up to 100 queued messages on the push side and eventually block again. The receiver is still on its own channel.
Likely what the code implies is that you wanted to create the global once, return it, and share it. I'm not going to get into the design issues with globals here. But it might look like this for your case:
var globalChannel *GlobalChannel
func GetInstanceGlobal() *GlobalChannel {
if globalChannel == nil {
globalChannel = &GlobalChannel{
Channel: make(chan string),
}
}
return globalChannel
}

Fire and forget goroutine golang

I have written an API that makes DB calls and does some business logic. I am invoking a goroutine that must perform some operation in the background.
Since the API call should not wait for this background task to finish, I am returning 200 OK immediately after calling the goroutine (let us assume the background task will never give any error.)
I read that goroutine will be terminated once the goroutine has completed its task.
Is this fire and forget way safe from a goroutine leak?
Are goroutines terminated and cleaned up once they perform the job?
func DefaultHandler(w http.ResponseWriter, r *http.Request) {
// Some DB calls
// Some business logics
go func() {
// some Task taking 5 sec
}()
w.WriteHeader(http.StatusOK)
}
I would recommend always having your goroutines under control to avoid memory and system exhaustion.
If you are receiving a spike of requests and you start spawning goroutines without control, probably the system will go down soon or later.
In those cases where you need to return an immediate 200Ok the best approach is to create a message queue, so the server only needs to create a job in the queue and return the ok and forget. The rest will be handled by a consumer asynchronously.
Producer (HTTP server) >>> Queue >>> Consumer
Normally, the queue is an external resource (RabbitMQ, AWS SQS...) but for teaching purposes, you can achieve the same effect using a channel as a message queue.
In the example you'll see how we create a channel to communicate 2 processes.
Then we start the worker process that will read from the channel and later the server with a handler that will write to the channel.
Try to play with the buffer size and job time while sending curl requests.
package main
import (
"fmt"
"log"
"net/http"
"time"
)
/*
$ go run .
curl "http://localhost:8080?user_id=1"
curl "http://localhost:8080?user_id=2"
curl "http://localhost:8080?user_id=3"
curl "http://localhost:8080?user_id=....."
*/
func main() {
queueSize := 10
// This is our queue, a channel to communicate processes. Queue size is the number of items that can be stored in the channel
myJobQueue := make(chan string, queueSize) // Search for 'buffered channels'
// Starts a worker that will read continuously from our queue
go myBackgroundWorker(myJobQueue)
// We start our server with a handler that is receiving the queue to write to it
if err := http.ListenAndServe("localhost:8080", myAsyncHandler(myJobQueue)); err != nil {
panic(err)
}
}
func myAsyncHandler(myJobQueue chan<- string) http.HandlerFunc {
return func(rw http.ResponseWriter, r *http.Request) {
// We check that in the query string we have a 'user_id' query param
if userID := r.URL.Query().Get("user_id"); userID != "" {
select {
case myJobQueue <- userID: // We try to put the item into the queue ...
rw.WriteHeader(http.StatusOK)
rw.Write([]byte(fmt.Sprintf("queuing user process: %s", userID)))
default: // If we cannot write to the queue it's because is full!
rw.WriteHeader(http.StatusInternalServerError)
rw.Write([]byte(`our internal queue is full, try it later`))
}
return
}
rw.WriteHeader(http.StatusBadRequest)
rw.Write([]byte(`missing 'user_id' in query params`))
}
}
func myBackgroundWorker(myJobQueue <-chan string) {
const (
jobDuration = 10 * time.Second // simulation of a heavy background process
)
// We continuosly read from our queue and process the queue 1 by 1.
// In this loop we could spawn more goroutines in a controlled way to paralelize work and increase the read throughput, but i don't want to overcomplicate the example.
for userID := range myJobQueue {
// rate limiter here ...
// go func(u string){
log.Printf("processing user: %s, started", userID)
time.Sleep(jobDuration)
log.Printf("processing user: %s, finisehd", userID)
// }(userID)
}
}
There is no "goroutine cleaning" you have to handle, you just launch goroutines and they'll be cleaned when the function launched as a goroutine returns. Quoting from Spec: Go statements:
When the function terminates, its goroutine also terminates. If the function has any return values, they are discarded when the function completes.
So what you do is fine. Note however that your launched goroutine cannot use or assume anything about the request (r) and response writer (w), you may only use them before you return from the handler.
Also note that you don't have to write http.StatusOK, if you return from the handler without writing anything, that's assumed to be a success and HTTP 200 OK will be sent back automatically.
See related / possible duplicate: Webhook process run on another goroutine
#icza is absolutely right there is no "goroutine cleaning" you can use a webhook or a background job like gocraft. The only way I can think of using your solution is to use the sync package for learning purposes.
func DefaultHandler(w http.ResponseWriter, r *http.Request) {
// Some DB calls
// Some business logics
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
// some Task taking 5 sec
}()
w.WriteHeader(http.StatusOK)
wg.wait()
}
you can wait for a goroutine to finish using &sync.WaitGroup:
// BusyTask
func BusyTask(t interface{}) error {
var wg = &sync.WaitGroup{}
wg.Add(1)
go func() {
// busy doing stuff
time.Sleep(5 * time.Second)
wg.Done()
}()
wg.Wait() // wait for goroutine
return nil
}
// this will wait 5 second till goroutune finish
func main() {
fmt.Println("hello")
BusyTask("some task...")
fmt.Println("done")
}
Other way is to attach a context.Context to goroutine and time it out.
//
func BusyTaskContext(ctx context.Context, t string) error {
done := make(chan struct{}, 1)
//
go func() {
// time sleep 5 second
time.Sleep(5 * time.Second)
// do tasks and signle done
done <- struct{}{}
close(done)
}()
//
select {
case <-ctx.Done():
return errors.New("timeout")
case <-done:
return nil
}
}
//
func main() {
fmt.Println("hello")
ctx, cancel := context.WithTimeout(context.TODO(), 2*time.Second)
defer cancel()
if err := BusyTaskContext(ctx, "some task..."); err != nil {
fmt.Println(err)
return
}
fmt.Println("done")
}

In a go worker/event system, should workers access the same struct (via pointer) to do work?

I'm a beginner gopher, and I wrote an event listener worker queue for a project I'm working on.
I've deployed it on a staging server. After around 100 events have been triggered the listeners stop getting called when events are published. The server hasn't crashed either.
Here is my implementation:
// Event struct
type Event struct {
Name string
Data interface{}
}
// Stream to publish events to
var stream = make(chan *Event, 100)
// Publish sends new event data to the stream by the event name
func Publish(name string, data interface{}) {
ev := &Event{name, data}
stream <- ev
}
// Handler provides the interface for all event handlers.
// The Work will be called with the event that it should process
type Handler interface {
Work(*Event)
}
type worker struct {
Handler
Listen chan *Event
Quit chan bool
}
// Stop shuts down the worker
func (w *worker) Stop() {
go func() {
w.Quit <- true
}()
}
// Queue of worker Listen channels
type workerQueue chan chan *Event
// registry of workers
var registry = make(map[string][]workerQueue)
// Register creates 20 workers, assigns them to a queue, and
// appends the resulting worker queue to an event on the handler registry
func Register(name string, handlers ...Handler) {
if _, ok := registry[name]; !ok {
registry[name] = make([]workerQueue, 0)
}
// Create workerQueues for each handler
for _, h := range handlers {
queue := make(workerQueue, numListeners)
// Create 20 workers
for i := 0; i < 20; i++ {
newWorker := worker{
Handler: h,
Listen: make(chan *Event),
Quit: make(chan bool),
}
go func() {
for {
select {
case ev := <-newWorker.Listen:
nl.Work(ev)
case <-newWorker.Quit:
return
}
}
}()
queue <- newWorker.Listen
}
registry[name] = append(registry[name], queue)
}
}
// Start begins listening for events on stream
func Start() {
go func() {
for {
select {
// listen for events
case ev := <-stream:
go func() {
// get registered queues for the event
queues, ok := registry[ev.Name]
if !ok {
return
}
// Get worker channel from queue and send the event
for _, queue := range queues {
worker := <-queue
worker <- ev
}
}()
}
}
}()
}
Here is an example usage.
// Usage
Start()
type demoHandler struct {
db *sql.DB
}
type eventData struct {}
func (h *demoHandler) Work(ev *Event) {
// Do something
return
}
// Register handler
Register('some-event', &demoHandler{r})
Publish('some-event', &eventData{})
I'm passing a pointer to a demoHandler as the event handler because they need access to the underlying sql instance. Is it a problem that each worker queue uses the same demoHandler?
I can't for the life of me figure out where I went wrong! Short of an error in the handler code, is there a mistake in my code which causes all of my workers to go down?
"In a go worker/event system, should workers access the same struct (via pointer) to do work?"
No, it's not a problem. It would be a problem if the code inside your handler access a critical section, but I think that's not causing your program to block.
Your server doesn't crash or block either because no panic is being triggered, and your program is listening and executing on separate goroutines, which are lightweight threads of execution.
It probably has to be with the channels you are using to send and receive events.
Sends and receives to a channel are blocking by default. This means that when you send or receive from a channel it will block its goroutine until the other side is ready.
In the case of buffered channels, sends block when the buffer is full, and receives block when the buffer is empty, as in your stream channel:
var stream = make(chan *Event, 100)
You said: "After around 100 events have been triggered the listeners stop getting called when events are published".
So if you call the Publish function and do stream <- ev when the "stream" channel buffer is full, it will block until the channel has place to receive another element.
I'd suggest reading a bit about non-blocking channel operations.
Maybe the block is occurring in some part of your real usage code.

Should things like an app's mailer system run in a separate channel as shown in this example?

Imagine a web service with a fair amount of different routes. Some of them trigger transactional emails being sent to the user. It seems weird to initialise a mailer instance, for instance something using github.com/aws/aws-sdk-go/service/sns every time a request wants to send something.
Instead I'd assume there's one mailer instance and everything happens on a separate channel to which a message gets posted to.
Example
I created a simple example illustrating the problem. A global Mailer instance gets configured once, the Index handler asks for a channel and passes a Message.
package main
import (
"fmt"
"log"
"net/http"
"os"
)
// Message is the custom type used to pass the channel
type Message struct {
To string
Subject string
Body string
}
// Mailer is responsible to send out emails
type Mailer struct{}
// send sends out the email
func (m *Mailer) send(message Message) {
fmt.Printf("Sending email to:`%s`\nSubject: %s\n%s\n\n", message.To, message.Subject, message.Body)
}
// Messages returns the channel to which messages can be passed
func (m *Mailer) Messages() chan<- Message {
cm := make(chan Message)
go func() {
msg := <-cm
m.send(msg)
close(cm)
}()
return cm
}
// mailer is a global var in this example, would probably be part of some
// sort of app context that's accessible from any handler.
//
// Note the mailer is NOT handler-scoped.
var mailer = Mailer{} // would this be thread-safe?
// Index handler
func Index(w http.ResponseWriter, r *http.Request) {
m := Message{"email#example.com", fmt.Sprintf("visited `%s`", r.URL.Path[1:]), "Lorem ipsum"}
mailer.Messages() <- m
fmt.Fprintf(w, "Sent out email with subject line `%s`\n", m.Subject)
}
func main() {
http.HandleFunc("/", Index)
port := os.Getenv("PORT")
if port == "" {
port = "8080"
}
if err := http.ListenAndServe(":"+port, nil); err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
Output
Visting http://localhost:8080/hello-world will render…
Sent out email with subject line `visited `hello-world``
… and log
sending email to `email#example.com`:
visited `hello-world`
Lorem ipsum
Questions
Is this the right approach?
Is it thread-safe – if not how to get it thread-safe?
You're not really doing anything in this example, but passing message over channels is always safe -- channels are one of the basic concurrency primitives in the language. You are leaving yourself open to the possibility of race conditions, depending on what send actually ends up doing. Another way to handle this is to have send receive from a single channel.
type Mailer struct{
Messages chan Message
}
func (m *Mailer) send() {
for message := range m.Messages {
fmt.Printf("Sending email to:`%s`\nSubject: %s\n%s\n\n", message.To, message.Subject, message.Body)
}
}
var mailer *Mailer
func Index(w http.ResponseWriter, r *http.Request) {
m := Message{"email#example.com", fmt.Sprintf("visited `%s`", r.URL.Path[1:]), "Lorem ipsum"}
mailer.Messages <- m
fmt.Fprintf(w, "Sent out email with subject line `%s`\n", m.Subject)
}
func main() {
mailer = &Mailer{
// buffer up to 100 message to be sent before blocking
Messages: make(chan Message, 100),
}
// start the mailer send loop
go mailer.send()
...

Resources