Broadcast server end messages to clients using Websockets - go

I am new to Golang, I am trying to a create a WebSocket server which will broadcast messages to the connected clients. The messages here will be generated from the server side(by creating a default client).
Here is my client.go
c, _, err := websocket.DefaultDialer.Dial(u.String(), nil)
if err != nil {
log.Fatal("dial:", err)
}
defer c.Close()
done := make(chan struct{})
new_chan := make(chan string)
//defer new_chan.Stop()
go func() {
for {
new_chan <- "my message"
}
}()
hub := newHub()
go hub.run()
client := &Client{hub: hub, ws: c, send: make(chan []byte, 256)}
for {
select {
case <-done:
return
case t := <-new_chan:
err := c.WriteMessage(websocket.TextMessage, []byte(t))
if err != nil {
log.Println("write:", err)
return
}
log.Printf(t)
client.hub.broadcast <- bytes.TrimSpace(bytes.Replace([]byte(t), newline, space, -1))
}
}
this function will generate the messages and try to broadcast to other clients.
server.go will add the clients to the hub
func echo(w http.ResponseWriter, r *http.Request) {
c, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Print("upgrade:", err)
return
}
hub := newHub()
go hub.run()
client := &Client{hub: hub, ws: c, send: make(chan []byte, 256)}
client.hub.register <- client
go client.writePump()
writePump() here will listen to the client.send channel and broadcast messages
Now the connected client's hub is different is from the hub of the client at the server. So when I try to send messages, I am not receiving anything.
How can I make it belong to the same hub(context)?

It looks like you are starting from the Gorilla chat example. In that example, use hub.broadcast <- message to broadcast a message to all clients where hub is the *Hub created in main().
There's no need to create a client connection to send message originated from the server.
Here's a modified version of the main() that broadcasts "hello" to all clients every second:
func broadcast(hub *Hub) {
m := []byte("hello")
for range time.NewTicker(time.Second).C {
hub.broadcast <- m
}
}
func main() {
flag.Parse()
hub := newHub()
go hub.run()
go broadcast(hub)
http.HandleFunc("/", serveHome)
http.HandleFunc("/ws", func(w http.ResponseWriter, r *http.Request) {
serveWs(hub, w, r)
})
err := http.ListenAndServe(*addr, nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}

Related

Running a consumer and api on port golang

I have a go api project where I also run a worker (RabbitMQ). I just discovered a problem that my worker and my http listen and serve do not work together. The moment I run the worker, the port of api is not reached.
Here is what my code looks like.
app.go
func (a *App) StartWorker() {
connection, err := amqp091.Dial(os.Getenv("AMQP_URL"))
if err != nil {
panic(err)
}
defer connection.Close()
consumer, err := events.NewConsumer(connection, database.GetDatabase(a.Database))
if err != nil {
panic(err)
}
consumer.Listen(os.Args[1:])
}
func (a *App) Run(addr string) {
logs := log.New(os.Stdout, "my-service", log.LstdFlags)
server := &http.Server{
Addr: addr,
Handler: a.Router,
ErrorLog: logs,
IdleTimeout: 120 * time.Second, // max time for connections using TCP Keep-Alive
ReadTimeout: 5 * time.Second,
WriteTimeout: 10 * time.Second,
}
go func() {
if err := server.ListenAndServe(); err != nil {
logs.Fatal(err)
}
}()
// trap sigterm or interrupt and gracefully shutdown the server
c := make(chan os.Signal)
signal.Notify(c, os.Interrupt)
signal.Notify(c, os.Kill)
sig := <-c
logs.Println("Recieved terminate, graceful shutdown", sig)
tc, _ := context.WithTimeout(context.Background(), 30*time.Second)
server.Shutdown(tc)
}
here is my
consumer.go
// NewConsumer returns a new Consumer
func NewConsumer(conn *amqp.Connection, db *mongo.Database) (Consumer, error) {
consumer := Consumer{
conn: conn,
db: db,
}
err := consumer.setup()
if err != nil {
return Consumer{}, err
}
return consumer, nil
}
// Listen will listen for all new Queue publications
// and print them to the console.
func (consumer *Consumer) Listen(topics []string) error {
ch, err := consumer.conn.Channel()
if err != nil {
return err
}
defer ch.Close()
if err != nil {
return err
}
msgs, err := ch.Consume("update.package.rating", "", true, false, false, false, nil)
if err != nil {
return err
}
forever := make(chan bool)
go func() {
for msg := range msgs {
switch msg.RoutingKey {
case "update.package.rating":
worker.RatePackage(packageRepo.NewPackagesRepository(consumer.db), msg.Body)
}
// acknowledege received event
log.Printf("Received a message: %s", msg.Body)
}
}()
log.Printf("[*] Waiting for message [Exchange, Queue][%s, %s]. To exit press CTRL+C", getExchangeName(), "update.package.rating")
<-forever
return nil
}
main.go
func main() {
start := app.App{}
start.StartApp()
start.StartWorker()
start.Run(":3006")
}
the port 3006 is not reached.
I am using gin-gonic to serve my http request.
Any help is welcomed.
I had a similar problem while using gin framework.Solved the issue by running my consumer inside a go routine.I invoked my consumer like below.
go notificationCallback.ConsumeBankTransaction()
and both the server and the rabbitmq consumer run seamlessly.Still monitoring performance to see if it is robust and resilient enough.

Golang: how to ensure redis subscribers receive all messages from redis pubsub?

I am trying to publish messages to dynamically generated channels in redis while subscribing all messages from the existing channels.
The following seems to work, but it fails to receive some messages depending on the timing of requests from the client (browser).
I tried "fan-in" for the two go channels in the select statement, but it did not work well.
package main
import (
...
"github.com/go-redis/redis/v8"
"github.com/gorilla/websocket"
)
var upgrader = websocket.Upgrader{}
var rd = redis.NewClient(&redis.Options{
Addr: "localhost:6379",
})
var ctx = context.Background()
func echo(w http.ResponseWriter, r *http.Request) {
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Println("websocket connection err:", err)
return
}
defer conn.Close()
room := make(chan string)
start := make(chan string)
go func() {
loop:
for {
sub := rd.Subscribe(ctx)
defer sub.Close()
channels := []string{}
for {
select {
//it seems that messages are not received when executing this case sometimes
case channel := <-room:
log.Println("channel", channel)
channels = append(channels, channel)
sub = rd.Subscribe(ctx, channels...)
start <- "ok"
case msg := <-sub.Channel():
log.Println("msg", msg)
err := conn.WriteMessage(websocket.TextMessage, []byte(msg.Payload))
if err != nil {
log.Println("websocket write err:", err)
break loop
}
}
}
}
}()
for {
_, msg, err := conn.ReadMessage()
if err != nil {
log.Println("websocket read err:", err)
break
}
log.Println(string(msg))
chPrefix := strings.Split(string(msg), ":")[0]
ch := chPrefix + "-channel"
if string(msg) == "test" || string(msg) == "yeah" {
room <- ch
log.Println(ch)
log.Println(<-start)
}
if err := rd.Publish(ctx, ch, msg).Err(); err != nil {
log.Println("redis publish err:", err)
break
}
}
}
func main() {
http.Handle("/", http.FileServer(http.Dir("./js")))
http.HandleFunc("/ws", echo)
log.Println("server starting...", "http://localhost:5000")
log.Fatal(http.ListenAndServe("localhost:5000", nil))
}
If by all messages, you mean you do not wish to lose any messages, I would recommend using Redis Streams instead of pub/sub. This will ensure you are not missing messages and can go back on the stream history if necessary.
This is an example of using Go, Streams and websockets that should get you started in that direction

Golang TCP Proxy (io.Copy) breaks when using io.Multiwriter

I'm currently dabbling in Go and I'm having trouble figuring out the following:
I am trying to proxy a MySQL connection through my application intending to do some introspection on both the requests coming from the client and responses coming from the server. I did the exact same thing for a Redis connection, which works well.
Now the issue is, if I simply do
go io.Copy(serverConnection, clientConnection)
go io.Copy(clientConnection, serverConnection)
This works fine. I can use a mysql client to connect to my proxy and issue queries to and receive responses from the MySQL server.
If, however, I try to "spy" on the connection by writing to an io.MultiWriter, the connection just hangs, waiting for the initial response.
The implementation (simplified) looks like this (and works this way):
package main
import (
"fmt"
"io"
"log"
"net"
)
func main () {
port := 21001
listenAddr, err := net.ResolveTCPAddr("tcp", fmt.Sprintf("127.0.0.1:%d", port))
if err != nil {
log.Fatalln(err)
}
targetAddr, err := net.ResolveTCPAddr("tcp", "127.0.0.1:11001")
if err != nil {
log.Fatalln(err)
}
listener, err := net.ListenTCP("tcp", listenAddr)
if err != nil {
log.Fatalln(err)
}
for {
clientConnection, err := listener.AcceptTCP()
if err != nil {
log.Fatalln(err)
}
go func () {
serverConnection, err := net.DialTCP("tcp", nil, targetAddr)
if err != nil {
return
}
connectionDone := make(chan bool)
go monitorClientConnection(clientConnection, serverConnection, connectionDone)
go monitorServerConnection(serverConnection, clientConnection, connectionDone)
_ = <-connectionDone
_ = <-connectionDone
fmt.Println("MySQL connection closed")
}()
}
}
func monitorClientConnection(clientConnection *net.TCPConn, serverConnection *net.TCPConn, connectionDone chan bool) {
go func() {
_, err := io.Copy(serverConnection, clientConnection)
fmt.Println(err)
_ = serverConnection.Close()
connectionDone <- true
}()
}
func monitorServerConnection(serverConnection *net.TCPConn, clientConnection *net.TCPConn, connectionDone chan bool) {
go func() {
_, err := io.Copy(clientConnection, serverConnection)
fmt.Println(err)
_ = clientConnection.Close()
connectionDone <- true
}()
}
Where it starts to fail
If I change the implementation of the monitoring functions from (client and server are identical, just reversed):
func monitorServerConnection(serverConnection *net.TCPConn, clientConnection *net.TCPConn, connectionDone chan bool) {
go func() {
_, err := io.Copy(clientConnection, serverConnection)
fmt.Println(err)
_ = clientConnection.Close()
connectionDone <- true
}()
}
to
func monitorServerConnection(serverConnection *net.TCPConn, clientConnection *net.TCPConn, connectionDone chan bool) {
_, w := io.Pipe() // The reader here would be used to consume the stream, left out for simplicity
go func() {
mw := io.MultiWriter(clientConnection, w)
_, err := io.Copy(mw, serverConnection)
fmt.Println(err)
_ = clientConnection.Close()
connectionDone <- true
}()
}
It just hangs. This is the exact same approach that works fine for the Redis proxy (the Redis Serialization Protocol is plaintext, newline delimited, could be a factor maybe?).
What I'd like to know
For my understanding, why does using the io.MultiWriter cause the stream to hang?
Is there a better way to "spy" on this stream, or tee it off into a separate goroutine that can read/parse the stream while it's also being forwarded to the upstream server?

Handling multiple websocket connections

I'm trying to create a program that will connect to several servers though gorilla web-sockets. I currently have a program that will iterate over a list of server addresses and create a new goroutine that will create its own Websocket.conn and handle reading and writing.
The problem is that every time a new goroutine is created the previous goroutines are blocked and only the last one can continue. I believe this is because the gorilla websocket library is blocking each gorotutine, but I might be mistaken.
I have tried putting a timer in the server list iterator and each goroutine will work perfectly but then the moment a new goroutine is made with another address the previous goroutine is blocked.
The relevant bits of my code:
In my main.go
for _, server := range servers {
go control(ctx, server, port)
}
In control()
func control(ctx context.Context, server, port string) {
url := url.URL{
Scheme: "ws",
Host: server + ":" + port,
Path: "",
}
conn, _, err := websocket.DefaultDialer.Dial(url.String(), nil)
if err != nil {
panic(err)
}
defer conn.Close()
go sendHandler(ctx, conn)
go readHandler(ctx, conn)
}
readHandler(ctx context.Context, conn *websocket.Con) {
for {
_, p, err := conn.ReadMessage(); if err != nil {
panic(err)
}
select {
case <-ctx.Done():
goto TERM
default:
// do nothing
}
}
TERM:
// do termination
}
sendHandler(ctx context.Context, conn *websocket.Con) {
for _, msg := range msges {
err = conn.WriteMessage(websocket.TextMessage, msg)
if err != nil {
panic(err)
}
}
<-ctx.Done()
}
I removed the parts where I add waitgroups and other unnecessary pieces of code.
So what I expect is for there to be 3n goroutines running (where n is the number of servers) without blocking but right now I see only 3 goroutines running which are the ones called by the last iteration of the server list.
Thanks!
EDIT 14/06/2019:
I spent some time making a small working example and in the example the bug did not occur - none of the threads blocked each other. I'm still unsure what was causing it but here is my small working example:
main.go
package main
import (
"context"
"fmt"
"os"
"time"
"os/signal"
"syscall"
"sync"
"net/url"
"github.com/gorilla/websocket"
)
func main() {
servers := []string{"5555","5556", "5557"}
comms := make(chan os.Signal, 1)
signal.Notify(comms, os.Interrupt, syscall.SIGTERM)
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
var wg sync.WaitGroup
for _, server := range servers {
wg.Add(1)
go control(server,
ctx,
&wg)
}
<-comms
cancel()
wg.Wait()
}
func control(server string, ctx context.Context, wg *sync.WaitGroup) {
fmt.Printf("Started control for %s\n", server)
url := url.URL {
Scheme: "ws",
Host: "0.0.0.0" + ":" + server,
Path: "",
}
conn, _, err := websocket.DefaultDialer.Dial(url.String(), nil)
if err != nil {
panic(err)
}
defer conn.Close()
var localwg sync.WaitGroup
localwg.Add(1)
go sendHandler(ctx, conn, &localwg, server)
localwg.Add(1)
go readHandler(ctx, conn, &localwg, server)
<- ctx.Done()
localwg.Wait()
wg.Done()
return
}
func sendHandler(ctx context.Context, conn *websocket.Conn, wg *sync.WaitGroup, server string) {
for i := 0; i < 50; i++ {
err := conn.WriteMessage(websocket.TextMessage, []byte("ping"))
if err != nil {
panic(err)
}
fmt.Printf("sent msg to %s\n", server)
time.Sleep(1 * time.Second)
}
<- ctx.Done()
wg.Done()
}
func readHandler(ctx context.Context, conn *websocket.Conn, wg *sync.WaitGroup, server string) {
for {
select {
case <- ctx.Done():
wg.Done()
return
default:
_, p, err := conn.ReadMessage()
if err != nil {
wg.Done()
fmt.Println("done")
}
fmt.Printf("Got [%s] from %s\n", string(p), server)
}
}
}
I tested it with dpallot's simple-websocket-server by a server on 5555, 5556 and 5557 respectively.
This part of your code is causing the problem:
conn, _, err := websocket.DefaultDialer.Dial(url.String(), nil)
if err != nil {
panic(err)
}
defer conn.Close()
go sendHandler(ctx, conn)
go readHandler(ctx, conn)
You create the connection, defer the close of it, start two other goroutines and then end the function. The function end closes the socket due to your defer.

GoLang TCP Connection - Remote Network is down check

I am creating a GoLang application and clients are android phones. I am able to handle connections. If user closes the android application connection is dropped with EOF
My problem is, if client just turn off wifi network connection is still alive.
Here is my code
func main() {
fmt.Println("Starting server...")
connection, err := net.Listen("tcp", ":4406")
if err != nil {
fmt.Println(err)
}
defer connection.Close()
manager := ClientManager{
clients: make(map[*Client]bool),
broadcast: make(chan []byte),
register: make(chan *Client),
}
go manager.start()
for {
connection, _ := connection.Accept()
if err != nil {
fmt.Println(err)
}
client := &Client{socket: connection, data: make(chan []byte), uuid: connection.RemoteAddr().String()}
manager.register <- client
go manager.receive(client)
go handleConnection(client)
}
}
Handeling connections
func handleConnection(client *Client) {
conn := client.socket
defer conn.Close()
notify := make(chan error)
go func() {
buf := make([]byte, 1024)
for {
n, err := conn.Read(buf)
if err != nil {
notify <- err
return
}
if n > 0 {
fmt.Println("unexpected data: %s", buf[:n])
}
}
}()
for {
select {
case err := <-notify:
if err != nil {
fmt.Println("connection dropped message", err)
return
}
case <-time.After(time.Second * 1):
fmt.Println("timeout 1, still alive")
}
}
}
When remote wifi is off (cable removed) I want to disconnect the user. I tried to read a byte and every second and it is reading it. I sent a byte and it is sent as well.

Resources