Golang TCP Client does not receive data from server, hangs/blocks on conn.Read() - go

I'm taking a dive into the networking side of Go, and I'd thought I'd start with a TCP Client and Server.
I am able to get the client to connect to the server and send a simple message ("Hello") successfully. However, I can not get the server to send back a response (or the get the client to read the response).
Here is the code.
Server
Address := "localhost:9999"
Addr, err := net.ResolveTCPAddr("tcp", Address)
if err != nil {
log.Fatal(err)
}
listener, err := net.ListenTCP("tcp", Addr)
if err != nil {
log.Fatal(err)
}
defer listener.Close()
//server loop
for {
conn, err := listener.Accept()
if err != nil {
continue
}
go handle(conn)
}
func handle(c net.Conn) {
totalBytes, message := connRead(c)
fmt.Println(c.RemoteAddr())
fmt.Println(string(message[:totalBytes]))
c.Write([]byte("Hi"))
fmt.Println("Replied")
c.Close()
}
func connRead(c net.Conn) (int, []byte) {
buffer := make([]byte, 4096)
totalBytes := 0
for {
n, err := c.Read(buffer)
totalBytes += n
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
return totalBytes, buffer
}
Client
tcpAddr, err := net.ResolveTCPAddr("tcp", "localhost:9999")
if err != nil {
log.Fatal(err)
}
conn, err := net.DialTCP("tcp", nil, tcpAddr)
if err != nil {
log.Fatal(err)
}
defer conn.Close()
_, err = conn.Write([]byte("Hello"))
if err != nil {
log.Fatal(err)
}
tBytes, resp := connRead(conn)
fmt.Println(tBytes)
fmt.Println(string(resp[:tBytes]))
func connRead(c net.Conn) (int, []byte) {
buffer := make([]byte, 4096)
totalBytes := 0
for {
fmt.Println("Stuck?")
n, err := c.Read(buffer)
fmt.Println("Stuck.")
totalBytes += n
fmt.Println(totalBytes)
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
return totalBytes, buffer
}
From what I can tell it's not a problem with the server. When I run the client, everything stops right after fmt.Println("Stuck?"). This leads me to belive that it's messing up in the n, err := c.Read(buffer) statement somehow. The server doesn't even print out the messeage length (5) and message ("Hello") untill after I Ctrl-C the client. If I comment out the read and printings in the client, then things run smoothly.
I've tried googling for answers, but nothing has come up.
What am I doing wrong? Am I using conn.Read() wrong in the client?
EDIT:
I actually do have access to Linux, so here are the SIGQUIT dumps for the pertinent functions.
Server
http://pastebin.com/itevngCq
Client
http://pastebin.com/XLiKqkvs

for {
n, err := c.Read(buffer)
totalBytes += n
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
It is because you are reading from connection till EOF error occurs
conn.Write([]byte("Hello"))
The above statement won't reach EOF at all until you actually closes the connection
On pressing ctrl+c client side the connection will be closed, So EOF occurs at server side, That is the reason why it is exiting server side for loop and printing these
127.0.0.1:****
Hello
Replied
If you want to make this work you should not read the connection till EOF
There are many other alternatives for this
Chose a delimiter and read at the server until the delimiter occurs and respond back after that. Check out this link
Send number of bytes to read from client side before sending the actual message, First read number of bytes to read from the server side and then read those many bytes from the connection

Related

Transfering file using tcp golang

I'm trying to make a music app that sends file through tcp protocol using go and microservice architecture. Now I'm creating a player service that should:
Get user token and get claims from it
Check is user exists using claims and user_service microservice
Get song from redis
Check is song exists using music_service
Read file by chunks and send it to client using tcp
Redis data looks like this:
{
"user_id": [{
"song_id": "<song_id>"
}]
}
But I faced with a small problem. My music files stored in a flac format and when I receive it on the client, my player doesn't play it. I don't really know what can be the problem. So here's my code:
SERVER
service_setup.go
//this function is called in main function
func setService() {
ln, err := net.Listen("tcp", config.TCPAddress)
if err != nil {
panic("couldn't start tcp server")
}
defer ln.Close()
for {
conn, err := ln.Accept()
if err != nil {
logger.ErrorLog(fmt.Sprintf("Error: couldn't accept connection. Details: %v", err))
return
}
service.DownloadSong(conn)
}
}
downloader_service.go
func DownloadSong(conn net.Conn) {
token, err := bufio.NewReader(conn).ReadString('\n')
if err != nil {
logger.ErrorLog(fmt.Sprintf("Error: couldn't get token. Details: %v", token))
conn.Close()
return
}
claims, err := jwt_funcs.DecodeJwt(token)
if err != nil {
conn.Close()
return
}
songs, err := redis_repo.Get(claims.Id)
if err != nil {
conn.Close()
return
}
for _, song := range songs {
download(song, conn)
}
}
func download(song models.SongsModel, conn net.Conn) {
filePath, err := filepath.Abs(fmt.Sprintf("./songs/%s.flac", song.SongId))
if err != nil {
logger.ErrorLog(fmt.Sprintf("Errror: couldn't create filepath. Details: %v", err))
conn.Close()
return
}
file, err := os.Open(filePath)
defer file.Close()
if err != nil {
logger.ErrorLog(fmt.Sprintf("Errror: couldn't open file. Details: %v", err))
conn.Close()
return
}
read(file, conn)
}
func read(file *os.File, conn net.Conn) {
reader := bufio.NewReader(file)
buf := make([]byte, 15)
defer conn.Close()
for {
_, err := reader.Read(buf)
if err != nil && err == io.EOF {
logger.InfoLog(fmt.Sprintf("Details: %v", err))
fmt.Println()
return
}
conn.Write(buf)
}
}
CLIENT
main.go
func main() {
conn, _ := net.Dial("tcp", "127.0.0.1:6060")
var glMessage []byte
text := "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjYzYzlhNmE1OWI3ZmQyNTQ2ZjA4ZWEyYSIsInVzZXJuYW1lIjoiMTIiLCJleHAiOjE2NzQyMTE5ODl9.aarSDhrFF1df3i2pIRyjNxTfSHKObqLU3kHJiPreredIhLNCzs7z7jMgRHQIcLaIvCOECN7bX0OaSvKdW7VKsQ\n"
fmt.Fprint(conn, text)
reader := bufio.NewReader(conn)
b := make([]byte, 15)
c := 0
for i, _ := reader.Read(b); int(i) != 0; i, _ = reader.Read(b) {
c += i
glMessage = append(glMessage, b...)
}
os.WriteFile("./test.flac", glMessage, 0644)
}
If you know what can be the problem, please tell me. I'd really appreciate it!
It looks like you're trying to send the music file over the network in 15 byte chunks, which is likely not enough to play the song on the client side.
You can try increasing the chunk size, for example, to 8192 bytes. To do this, replace buf := make([]byte, 15) with buf := make([]byte, 8192).
Also, it's better to write the received data directly to the file rather than storing it in memory. You can do this by creating a file and using os.Create to write the received data to it:
file, err := os.Create("./test.flac")
if err != nil {
fmt.Println("Error: couldn't create file")
return
}
defer file.Close()
for {
i, err := reader.Read(buf)
if err != nil && err == io.EOF {
break
}
file.Write(buf[:i])
}
I believe that this can solve the issue.

SSH proxy, bad packet length

Implementing an ssh proxy in Go, errors out with bad packet length, these are the errors with ssh in debug mode:
debug1: SSH2_MSG_KEXINIT sent
Bad packet length 1231976033.
ssh_dispatch_run_fatal: Connection to ::1 port 8080: message authentication code incorrect
Code:
func handleSSH(conn net.Conn, r *bufio.Reader, protocol string) {
target, err := url.Parse("ssh://localhost:3333")
if err != nil {
fmt.Println("Error parsing target", err)
conn.Close()
return
}
targetConn, err := net.Dial("tcp", target.Host)
if err != nil {
fmt.Println("error dialing SSH target:", err)
conn.Close()
return
}
defer targetConn.Close()
var wg sync.WaitGroup
wg.Add(2)
go func() {
_, err := io.Copy(targetConn, conn)
if err != nil {
fmt.Println("error copying data to target:", err)
}
wg.Done()
}()
go func() {
_, err := io.Copy(conn, targetConn)
if err != nil {
fmt.Println("error copying data from target:", err)
}
wg.Done()
}()
wg.Wait()
conn.Close()
}
// EDIT
func connection(conn net.Conn) {
r := bufio.NewReader(conn)
protocol, err := r.ReadString('\n')
if err != nil {
fmt.Println("Error reading first line", err)
conn.Close()
return
}
if protocol[0:3] == "SSH" {
handleSSH(conn, r, protocol)
}
}
func main() {
ln, err := net.Listen("tcp", ":8080")
if err != nil {
panic(err)
}
defer ln.Close()
for {
conn, err := ln.Accept()
if err != nil {
panic(err)
}
go connection(conn)
}
}
EDIT: added code for relevant information on how the connection is initiated and reproduce the error.
My best guess is the ssh negotiation process is being interrupted, and things goes out of sync.
The code is reading the first line from the client and checks the kind of protocol in order to call the appropriate handler:
protocol, err := r.ReadString('\n')
...
if protocol[0:3] == "SSH" {
handleSSH(conn, r, protocol)
}
}
But the code fails to forward the already read bytes to the connected server. These bytes are in protocol and are given to handleSSH. But it fails to send these bytes to the connected server once the connection is established. Instead it only copies new data between client and server.
This means the server does not get the first line from the client. It therefore likely complains about a protocol error with something like Invalid SSH identification string. which gets forwarded to the client and misinterpreted as valid data from an SSH connection.

Go net.Conn.Write sending EOF error on TCP Proxy

Overview
I'm currently working on a simple TCP proxy to port forward a connection to an echo server that I've placed on a server in my house. I'm running the TCP proxy and client on the same machine, the echo server is the only program on another device.
The client is just a simple python script that declares and initializes a TCP socket and invokes a socket.send(b'testing some random byte message\n')
If I direct connect the client to the server, everything works fine but it's when I utilize the client/proxy connection when the server never receives the msg. In fact, the server outputs that the proxy and initiated a connection but only when I try to send a packet through from the client is when the server detects an io.EOF error.
Connections
server listening on : 192.168.1.253:6969
proxy listening on : 0.0.0.0:6969
proxy forwarding to server listener
client connecting on proxy listener
I've reduced the functionality to only have the proxy read and send once, then exit. Please excuse any remaining artifacts...
Proxy
func main() {
dstDone := make(chan struct{})
msgChannel := make(chan []byte)
go func(msgs <-chan []byte) {
dst, err := net.DialTimeout("tcp", "192.168.1.253:6969", 750*time.Millisecond)
if err != nil {
fmt.Println("Error connecting to dst -", err)
os.Exit(2)
}
dstWriter := bufio.NewWriter(dst)
n, err := dstWriter.Write(<-msgs)
if err != nil {
fmt.Println("Error writing to dst -", err)
os.Exit(1)
}
fmt.Println("Wrote ", n, "B to dst")
dstDone <- struct{}{}
}(msgChannel)
srcListener, err := net.Listen("tcp", ":6969")
if err != nil {
fmt.Println("Error listening for source -", err)
os.Exit(1)
}
srcConnection, err := srcListener.Accept()
if err != nil {
fmt.Println("Error accepting connection -", err)
os.Exit(1)
}
srcReader := bufio.NewReader(srcConnection)
msg, merr := srcReader.ReadBytes('\n')
if merr != nil {
fmt.Println("Error reading socket info -", err)
os.Exit(1)
}
fmt.Println("Message received [", msg, "]")
fmt.Println("Sending to dst")
msgChannel <- msg
<-dstDone
}
Server
// Start() is called in another go file's main which only consists of this call and
// initializes a channel for indicating when to quit.
func Start(serverDone *chan struct{}) {
listener, err := net.Listen("tcp", ":6969")
if err != nil {
fmt.Println("Could not listen:", err)
*serverDone <- struct{}{}
}
go func(serverDone *chan struct{}) {
buffer := make([]byte, 4096)
fmt.Println("-- Server Running --\nEnter 'Q' to quit.")
for {
s, err := os.Stdin.Read(buffer)
if err != nil {
fmt.Printf("%s\n", err)
break
}
if s > 2 {
fmt.Printf("Invalid Command\n")
} else {
if buffer[0] == 81 {
break
}
}
}
*serverDone <- struct{}{}
}(serverDone)
for {
connection, err := listener.Accept()
if err != nil {
fmt.Println("Error accepting connection:", err)
*serverDone <- struct{}{}
break
}
fmt.Printf("%v connected\n", connection.RemoteAddr())
go customBufferEcho(&connection)
}
}
func customBufferEcho(connection *net.Conn) {
defer (*connection).Close()
inBuffer := make([]byte, 4096)
outBuffer := make([]byte, 4096)
for {
fmt.Println("Waiting for msg...")
n, err := (*connection).Read(inBuffer[0:]) // tried only 'inBuffer' and did not work
if err == io.EOF {
fmt.Println("Client disconnected") // Server prints this msg on any msg sent from proxy
return
}
if err != nil {
fmt.Println("Error reading from connection |", err)
return
}
fmt.Println("-- MSG Received [ ", n, "B ]--\n", Dump(inBuffer, 4, 4))
outBuffer = inBuffer
(*connection).Write(outBuffer[0:n])
}
}
Wireshark Inspection

program blanks at net.conn read attempt in client. Does not show and error it just goes blank and stops working

Here is my client
func ClientStart() {
var dissconnect bool //used for detecting dissconnect
var previousData []byte
var packets [][]byte
conn, err := net.Dial("tcp", "0.0.0.0:9999")
if err != nil {
log.Fatalln(err)
}
defer conn.Close()
//Create the hello message
//--------
//-----
//----->>>>> tracker
tracker = Newtrack(15 * time.Second) //set timer for 15 seconds
//----->>>>> tracker end
//The loop
for {
buffer := make([]byte, 1<<16) //create 1 mb buffer //make sure to zero out when finished
_,err := conn.Read(buffer) //read into the buffer
//This is for more gracefully hnadling the stuff
if err != nil {
if err != io.EOF {
fmt.Println(err)
}
break
}
packets,previousData, err = Messages.ExtractPackets(append(previousData,buffer...))//extract the packets
//This is where everything gets processed
for _, value := range packets {
irc, err := Messages.CreateIrc(value)//get irc
if err != nil {fmt.Println(err)}
parse, opcode, err := Messages.Parse(*irc)
if err != nil {fmt.Println(err)}
err = handleMessages(*parse,opcode)
if err != nil {fmt.Println(err)}
}
//-+-+-+++++++++++++++++++++++ dissconnect if timer expires
go func(dis *bool) {
for {
<-tracker.timer.C
fmt.Println("Keepalive not sent from function")
break
}
*dis = true
}(&dissconnect)
if dissconnect {
fmt.Println("client has not sent keepalive dissconnecting")
os.Exit(3)
}
//++++++++++++++++++++++++++++ timer
sendKeepAlive(conn)
}
}
and here is my server
func ServerStart() {
listener, err := net.Listen("tcp", "0.0.0.0:9999")
if err != nil {
log.Fatalln(err)
}
defer listener.Close()
for {
con, err := listener.Accept()
if err != nil {
log.Println(err)
continue
}
// If you want, you can increment a counter here and inject to handleClientRequest below as client identifier
go handleClientRequest(con)
}
}
func handleClientRequest(conn net.Conn) {
defer conn.Close()
var dissconnect bool //used to dissconnect if keep alive fails
var previousData []byte //holds previous data from for loop
var packets [][]byte //holds packets in a list
var label string
tracker = Newtrack(15 * time.Second) //create a new tracker and set the timer for 15 seconds
for {
buffer := make([]byte, 1<<16) // small buffer
_,err := conn.Read(buffer) //read into the buffer
//This is for more gracefully hnadling the stuff
if err != nil {
if err != io.EOF {
fmt.Println(err)
}
break
}
//extract the packets and any extra
packets,previousData, err = Messages.ExtractPackets(append(previousData,buffer...))//extract the packets
//print the error
if err != nil {
fmt.Println(err)
}
//turn the packets into IRC packets
IRCpackets := make([]Messages.IrcPacket, len(packets)) //create a list of IRC packets len of raw packets
for index,value := range packets {
IRC, err := Messages.CreateIrc(value)
if IRC != nil { //if IRC = nil then err likely does not = nil
IRCpackets[index] = *IRC //create IRCpackets's for every raw packet
}
if err != nil {
fmt.Println(err)
}
}
//This is where everything gets processed
for _, value := range packets {
irc, err := Messages.CreateIrc(value)//get irc
if err != nil {fmt.Println(err)}
parse, opcode, err := Messages.Parse(*irc)
if err != nil {fmt.Println(err)}
err = handleMessages(*parse,opcode)
if err != nil {fmt.Println(err)}
}
}
}
As stated in the title it just sits there and does not carry on. Can you use the same connection to read and write? my read on the server side works just fine and has no issues however I have not attempted to write on the server side yet so I'm unsure if that's the issue
Edit: By stops working I mean it does not do anything past that point in the debugger it just sits there and refuses to move on and I've never seen something like that before.
the client and server are run separately and aren't part of the same process activated by a command line argument. So I can't use the debugger with both.
Can you use the same connection to read and write?
Yes. A TCP connection supports two asynchronous streams of data.
It's too much code to read for me (try to reduce it to a minimal compilable example that demonstrates the issue), but I suppose that the problem is the following. If you don't conn.Write() on one side, the other side will wait on conn.Read() and vice versa.

Golang broken pipe

I have a code segment that is reading from a TCP connection and after the first few connections, the server is outputing broken pipe however no error is occurring in my go code. The server sending the messages is at its core the coda hale metrics library, more specifically the PickledGraphite class.
here is the Go code that is reading:
func handleConn(conn net.Conn, id int) {
fmt.Println("handleConn")
defer conn.Close()
buf := make([]byte, 0, 10240)
tmp := make([]byte, 256)
fmt.Printf("%v Reading...\n", id)
for {
n, err := conn.Read(tmp)
fmt.Printf("%v Read %v\n", id, n)
if err != nil {
fmt.Printf("%v Got err: %v\n", id, err)
if err != io.EOF {
fmt.Printf("%v read error: %v\n", id, err)
}
buf = append(buf, tmp[:n]...)
break
}
buf = append(buf, tmp[:n]...)
}
fmt.Printf("%v Done Reading\n", id)
// Do stuff with buf
}
func main() {
ln, err := net.Listen("tcp", ":5555")
if err != nil {
fmt.Println(err)
os.Exit(-1)
}
id := 1
for {
fmt.Println("getting connection\n")
conn, err := ln.Accept()
if err != nil {
fmt.Println(err)
break
}
conn.SetReadDeadline(time.Now().Add(20 * time.Second))
fmt.Println("Got connection")
go handleConn(conn, id)
id = id + 1
fmt.Println("sent handleConn\n")
}
}
My code is still running, I can still execute nc commands and see my code receive it, so I am not sure how I am losing the connection.
If I remove the conn.SetReadDeadline() line then what happens is my code no longer receives a EOF after the first message.
Thanks in advance

Resources