Golang TCP Proxy (io.Copy) breaks when using io.Multiwriter - go

I'm currently dabbling in Go and I'm having trouble figuring out the following:
I am trying to proxy a MySQL connection through my application intending to do some introspection on both the requests coming from the client and responses coming from the server. I did the exact same thing for a Redis connection, which works well.
Now the issue is, if I simply do
go io.Copy(serverConnection, clientConnection)
go io.Copy(clientConnection, serverConnection)
This works fine. I can use a mysql client to connect to my proxy and issue queries to and receive responses from the MySQL server.
If, however, I try to "spy" on the connection by writing to an io.MultiWriter, the connection just hangs, waiting for the initial response.
The implementation (simplified) looks like this (and works this way):
package main
import (
"fmt"
"io"
"log"
"net"
)
func main () {
port := 21001
listenAddr, err := net.ResolveTCPAddr("tcp", fmt.Sprintf("127.0.0.1:%d", port))
if err != nil {
log.Fatalln(err)
}
targetAddr, err := net.ResolveTCPAddr("tcp", "127.0.0.1:11001")
if err != nil {
log.Fatalln(err)
}
listener, err := net.ListenTCP("tcp", listenAddr)
if err != nil {
log.Fatalln(err)
}
for {
clientConnection, err := listener.AcceptTCP()
if err != nil {
log.Fatalln(err)
}
go func () {
serverConnection, err := net.DialTCP("tcp", nil, targetAddr)
if err != nil {
return
}
connectionDone := make(chan bool)
go monitorClientConnection(clientConnection, serverConnection, connectionDone)
go monitorServerConnection(serverConnection, clientConnection, connectionDone)
_ = <-connectionDone
_ = <-connectionDone
fmt.Println("MySQL connection closed")
}()
}
}
func monitorClientConnection(clientConnection *net.TCPConn, serverConnection *net.TCPConn, connectionDone chan bool) {
go func() {
_, err := io.Copy(serverConnection, clientConnection)
fmt.Println(err)
_ = serverConnection.Close()
connectionDone <- true
}()
}
func monitorServerConnection(serverConnection *net.TCPConn, clientConnection *net.TCPConn, connectionDone chan bool) {
go func() {
_, err := io.Copy(clientConnection, serverConnection)
fmt.Println(err)
_ = clientConnection.Close()
connectionDone <- true
}()
}
Where it starts to fail
If I change the implementation of the monitoring functions from (client and server are identical, just reversed):
func monitorServerConnection(serverConnection *net.TCPConn, clientConnection *net.TCPConn, connectionDone chan bool) {
go func() {
_, err := io.Copy(clientConnection, serverConnection)
fmt.Println(err)
_ = clientConnection.Close()
connectionDone <- true
}()
}
to
func monitorServerConnection(serverConnection *net.TCPConn, clientConnection *net.TCPConn, connectionDone chan bool) {
_, w := io.Pipe() // The reader here would be used to consume the stream, left out for simplicity
go func() {
mw := io.MultiWriter(clientConnection, w)
_, err := io.Copy(mw, serverConnection)
fmt.Println(err)
_ = clientConnection.Close()
connectionDone <- true
}()
}
It just hangs. This is the exact same approach that works fine for the Redis proxy (the Redis Serialization Protocol is plaintext, newline delimited, could be a factor maybe?).
What I'd like to know
For my understanding, why does using the io.MultiWriter cause the stream to hang?
Is there a better way to "spy" on this stream, or tee it off into a separate goroutine that can read/parse the stream while it's also being forwarded to the upstream server?

Related

Golang: how to ensure redis subscribers receive all messages from redis pubsub?

I am trying to publish messages to dynamically generated channels in redis while subscribing all messages from the existing channels.
The following seems to work, but it fails to receive some messages depending on the timing of requests from the client (browser).
I tried "fan-in" for the two go channels in the select statement, but it did not work well.
package main
import (
...
"github.com/go-redis/redis/v8"
"github.com/gorilla/websocket"
)
var upgrader = websocket.Upgrader{}
var rd = redis.NewClient(&redis.Options{
Addr: "localhost:6379",
})
var ctx = context.Background()
func echo(w http.ResponseWriter, r *http.Request) {
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Println("websocket connection err:", err)
return
}
defer conn.Close()
room := make(chan string)
start := make(chan string)
go func() {
loop:
for {
sub := rd.Subscribe(ctx)
defer sub.Close()
channels := []string{}
for {
select {
//it seems that messages are not received when executing this case sometimes
case channel := <-room:
log.Println("channel", channel)
channels = append(channels, channel)
sub = rd.Subscribe(ctx, channels...)
start <- "ok"
case msg := <-sub.Channel():
log.Println("msg", msg)
err := conn.WriteMessage(websocket.TextMessage, []byte(msg.Payload))
if err != nil {
log.Println("websocket write err:", err)
break loop
}
}
}
}
}()
for {
_, msg, err := conn.ReadMessage()
if err != nil {
log.Println("websocket read err:", err)
break
}
log.Println(string(msg))
chPrefix := strings.Split(string(msg), ":")[0]
ch := chPrefix + "-channel"
if string(msg) == "test" || string(msg) == "yeah" {
room <- ch
log.Println(ch)
log.Println(<-start)
}
if err := rd.Publish(ctx, ch, msg).Err(); err != nil {
log.Println("redis publish err:", err)
break
}
}
}
func main() {
http.Handle("/", http.FileServer(http.Dir("./js")))
http.HandleFunc("/ws", echo)
log.Println("server starting...", "http://localhost:5000")
log.Fatal(http.ListenAndServe("localhost:5000", nil))
}
If by all messages, you mean you do not wish to lose any messages, I would recommend using Redis Streams instead of pub/sub. This will ensure you are not missing messages and can go back on the stream history if necessary.
This is an example of using Go, Streams and websockets that should get you started in that direction

Handling multiple websocket connections

I'm trying to create a program that will connect to several servers though gorilla web-sockets. I currently have a program that will iterate over a list of server addresses and create a new goroutine that will create its own Websocket.conn and handle reading and writing.
The problem is that every time a new goroutine is created the previous goroutines are blocked and only the last one can continue. I believe this is because the gorilla websocket library is blocking each gorotutine, but I might be mistaken.
I have tried putting a timer in the server list iterator and each goroutine will work perfectly but then the moment a new goroutine is made with another address the previous goroutine is blocked.
The relevant bits of my code:
In my main.go
for _, server := range servers {
go control(ctx, server, port)
}
In control()
func control(ctx context.Context, server, port string) {
url := url.URL{
Scheme: "ws",
Host: server + ":" + port,
Path: "",
}
conn, _, err := websocket.DefaultDialer.Dial(url.String(), nil)
if err != nil {
panic(err)
}
defer conn.Close()
go sendHandler(ctx, conn)
go readHandler(ctx, conn)
}
readHandler(ctx context.Context, conn *websocket.Con) {
for {
_, p, err := conn.ReadMessage(); if err != nil {
panic(err)
}
select {
case <-ctx.Done():
goto TERM
default:
// do nothing
}
}
TERM:
// do termination
}
sendHandler(ctx context.Context, conn *websocket.Con) {
for _, msg := range msges {
err = conn.WriteMessage(websocket.TextMessage, msg)
if err != nil {
panic(err)
}
}
<-ctx.Done()
}
I removed the parts where I add waitgroups and other unnecessary pieces of code.
So what I expect is for there to be 3n goroutines running (where n is the number of servers) without blocking but right now I see only 3 goroutines running which are the ones called by the last iteration of the server list.
Thanks!
EDIT 14/06/2019:
I spent some time making a small working example and in the example the bug did not occur - none of the threads blocked each other. I'm still unsure what was causing it but here is my small working example:
main.go
package main
import (
"context"
"fmt"
"os"
"time"
"os/signal"
"syscall"
"sync"
"net/url"
"github.com/gorilla/websocket"
)
func main() {
servers := []string{"5555","5556", "5557"}
comms := make(chan os.Signal, 1)
signal.Notify(comms, os.Interrupt, syscall.SIGTERM)
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
var wg sync.WaitGroup
for _, server := range servers {
wg.Add(1)
go control(server,
ctx,
&wg)
}
<-comms
cancel()
wg.Wait()
}
func control(server string, ctx context.Context, wg *sync.WaitGroup) {
fmt.Printf("Started control for %s\n", server)
url := url.URL {
Scheme: "ws",
Host: "0.0.0.0" + ":" + server,
Path: "",
}
conn, _, err := websocket.DefaultDialer.Dial(url.String(), nil)
if err != nil {
panic(err)
}
defer conn.Close()
var localwg sync.WaitGroup
localwg.Add(1)
go sendHandler(ctx, conn, &localwg, server)
localwg.Add(1)
go readHandler(ctx, conn, &localwg, server)
<- ctx.Done()
localwg.Wait()
wg.Done()
return
}
func sendHandler(ctx context.Context, conn *websocket.Conn, wg *sync.WaitGroup, server string) {
for i := 0; i < 50; i++ {
err := conn.WriteMessage(websocket.TextMessage, []byte("ping"))
if err != nil {
panic(err)
}
fmt.Printf("sent msg to %s\n", server)
time.Sleep(1 * time.Second)
}
<- ctx.Done()
wg.Done()
}
func readHandler(ctx context.Context, conn *websocket.Conn, wg *sync.WaitGroup, server string) {
for {
select {
case <- ctx.Done():
wg.Done()
return
default:
_, p, err := conn.ReadMessage()
if err != nil {
wg.Done()
fmt.Println("done")
}
fmt.Printf("Got [%s] from %s\n", string(p), server)
}
}
}
I tested it with dpallot's simple-websocket-server by a server on 5555, 5556 and 5557 respectively.
This part of your code is causing the problem:
conn, _, err := websocket.DefaultDialer.Dial(url.String(), nil)
if err != nil {
panic(err)
}
defer conn.Close()
go sendHandler(ctx, conn)
go readHandler(ctx, conn)
You create the connection, defer the close of it, start two other goroutines and then end the function. The function end closes the socket due to your defer.

Broadcast server end messages to clients using Websockets

I am new to Golang, I am trying to a create a WebSocket server which will broadcast messages to the connected clients. The messages here will be generated from the server side(by creating a default client).
Here is my client.go
c, _, err := websocket.DefaultDialer.Dial(u.String(), nil)
if err != nil {
log.Fatal("dial:", err)
}
defer c.Close()
done := make(chan struct{})
new_chan := make(chan string)
//defer new_chan.Stop()
go func() {
for {
new_chan <- "my message"
}
}()
hub := newHub()
go hub.run()
client := &Client{hub: hub, ws: c, send: make(chan []byte, 256)}
for {
select {
case <-done:
return
case t := <-new_chan:
err := c.WriteMessage(websocket.TextMessage, []byte(t))
if err != nil {
log.Println("write:", err)
return
}
log.Printf(t)
client.hub.broadcast <- bytes.TrimSpace(bytes.Replace([]byte(t), newline, space, -1))
}
}
this function will generate the messages and try to broadcast to other clients.
server.go will add the clients to the hub
func echo(w http.ResponseWriter, r *http.Request) {
c, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Print("upgrade:", err)
return
}
hub := newHub()
go hub.run()
client := &Client{hub: hub, ws: c, send: make(chan []byte, 256)}
client.hub.register <- client
go client.writePump()
writePump() here will listen to the client.send channel and broadcast messages
Now the connected client's hub is different is from the hub of the client at the server. So when I try to send messages, I am not receiving anything.
How can I make it belong to the same hub(context)?
It looks like you are starting from the Gorilla chat example. In that example, use hub.broadcast <- message to broadcast a message to all clients where hub is the *Hub created in main().
There's no need to create a client connection to send message originated from the server.
Here's a modified version of the main() that broadcasts "hello" to all clients every second:
func broadcast(hub *Hub) {
m := []byte("hello")
for range time.NewTicker(time.Second).C {
hub.broadcast <- m
}
}
func main() {
flag.Parse()
hub := newHub()
go hub.run()
go broadcast(hub)
http.HandleFunc("/", serveHome)
http.HandleFunc("/ws", func(w http.ResponseWriter, r *http.Request) {
serveWs(hub, w, r)
})
err := http.ListenAndServe(*addr, nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}

How to catch panic when HTTP client make request to unavailable url?

I need to make request to some URLs. It is possible, that someone of them is unavailable. It is OK for me, and I want just ignore these URLs.
My problem is I cannot catch error, which is occurs. I get a error message:
2018/01/13 18:46:24 Get http://fakesite.com: dial tcp [::1]:8084: connectex: No connection could be made because the target machine actively refused it.
My sample code is:
package main
import (
"fmt"
"log"
"net/http"
"io/ioutil"
)
func main() {
c := make(chan string, 1)
go func() {
defer func() {
if r := recover(); r != nil {
fmt.Println("try to catch error1", r)
}
}()
resp, err := http.Get("http://fakesite.com")
if err != nil {
log.Fatal(err)
c <- ""
return
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
c <- ""
return
}
c <- string(body)
}()
defer func() {
if r := recover(); r != nil {
fmt.Println("try to catch error2", r)
}
}()
msg := <-c
fmt.Println(msg)
}
The problem is:
log.Fatal(err)
This function performs two things. Firstly, it logs. But this function exits too.
// Fatal is equivalent to Print() followed by a call to os.Exit(1).
func Fatal(v ...interface{}) {
std.Output(2, fmt.Sprint(v...))
os.Exit(1)
}

golang: net.Conn: check conn status

I encountered a strange behavior of the conn.Read:
let's presume that I have a couple of functions for testing net.Conn:
package example
import (
"io"
"log"
"net"
"os"
"time"
)
func CheckConn(conn net.Conn) (net.Conn, error) {
conn.SetReadDeadline(time.Now())
var one = []byte{}
_, err := conn.Read(one)
if err != nil {
log.Println("Net err: ", err)
}
if err == io.EOF {
return conn, err
}
var zero time.Time
conn.SetReadDeadline(zero)
return conn, nil
}
func CheckConnWithTimeout(conn net.Conn) (net.Conn, error) {
ch := make(chan bool, 1)
defer func() {
ch <- true
}()
go func() {
select {
case <-ch:
case <-time.After(1 * time.Second):
log.Println("It works too long")
os.Exit(1)
}
}()
return CheckConn(conn)
}
And I want to implement tests for it, lets start with this one:
package example
import (
"io"
"net"
"testing"
)
func TestClosedConn(t *testing.T) {
server, client := net.Pipe()
client.Close()
defer server.Close()
_, err := CheckConn(server)
if err != io.EOF {
t.Errorf("Not equal:\nExpected: %v\nactual: %v", io.EOF, err)
}
}
this works pretty well, we will receive io.EOF from CheckConn function, lets add one more test:
func TestClosedConnAfterWrite(t *testing.T) {
server, client := net.Pipe()
go func() {
client.Write([]byte{0xb})
}()
client.Close()
defer server.Close()
_, err := CheckConn(server)
err = nil
if err != io.EOF {
t.Errorf("Not equal:\nExpected: %v\nactual: %v", io.EOF, err)
}
}
looks like the first test, but we wrote to the client before(?) it was closed.
And this will not pass!
conn.Read will return &errors.errorString{s:"EOF"}, instead of io.EOF, so CheckConn will return error == nil,
It looks so weird!
But let's continue the tests, now I want to check unclosed connections:
func TestActiveConn(t *testing.T) {
server, client := net.Pipe()
defer client.Close()
defer server.Close()
_, err := CheckConnWithTimeout(server)
if err != nil {
t.Errorf("Not equal:\nExpected: %v\nactual: %v", nil, err)
}
}
I think you noticed that I use the function with a timeout just because SetReadDeadline will not work in this case(I have no idea why!)
So what is going wrong in last two test cases? Is there a normal way to test the connection? Why SetReadDeadline is not working in this case?

Resources