How I can receive data for ever from TCP server - go

I try to create TCP client to receive data from TCP server,
but after server sending data only I receive data one even if server send many data, and I want to receive data forever, and I don't know what is my problem,and
Client:
func main() {
tcpAddr := "localhost:3333"
conn, err := net.DialTimeout("tcp", tcpAddr, time.Second*7)
if err != nil {
log.Println(err)
}
defer conn.Close()
// conn.Write([]byte("Hello World"))
connBuf := bufio.NewReader(conn)
for {
bytes, err := connBuf.ReadBytes('\n')
if err != nil {
log.Println("Rrecv Error:", err)
}
if len(bytes) > 0 {
fmt.Println(string(bytes))
}
time.Sleep(time.Second * 2)
}
}
I'm following this example to create TCP test server
Server:
// Handles incoming requests.
func handleRequest(conn net.Conn) {
// Make a buffer to hold incoming data.
buf := make([]byte, 1024)
// Read the incoming connection into the buffer.
_, err := conn.Read(buf)
if err != nil {
fmt.Println("Error reading:", err.Error())
}
fmt.Println(buf)
// Send a response back to person contacting us.
var msg string
fmt.Scanln(&msg)
conn.Write([]byte(msg))
// Close the connection when you're done with it.
conn.Close()
}

Read requires a Write on the other side of the connection
want to receive data forever
Then you have to send data forever. There's a for loop on the receiving end, but no looping on the sending end. The server writes its message once and closes the connection.
Server expects to get msg from client but client doesn't send it
// conn.Write([]byte("Hello World"))
That's supposed to provide the msg value to the server
_, err := conn.Read(buf)
So those two lines don't match.
Client expects a newline but server isn't sending one
fmt.Scanln expects to put each whitespace separated value into the corresponding argument. It does not capture the whitespace. So:
Only up to the first whitespace of what you type into server's stdin will be stored in msg
Newline will not be stored in msg.
But your client is doing
bytes, err := connBuf.ReadBytes('\n')
The \n never comes. The client never gets done reading that first msg.
bufio.NewScanner would be a better way to collect data from stdin, since you're likely to want to capture whitespace as well. Don't forget to append the newline to each line of text you send, because the client expects it!
Working code
I put these changes together into a working example on the playground. To get it working in that context, I had to make a few other changes too.
Running server and client in the same process
Hard coded 3 clients so the program ended in limited amount of time
Hard coded 10 receives in the client so program can end
Hard coded 3 server connections handled so program can end
Removed fmt.Scanln and have server just return the original message sent (because playground provides no stdin mechanism)
Should be enough to get you started.

Related

go-libp2p - receiving bytes from stream

I'm building my first go-libp2p application and trying to modify the echo example to read a []byte instead of a string as in the example.
In my code, I changed the doEcho function to run io.ReadAll(s) instead of bufio.NewReader(s) followed by ReadString('\n'):
// doEcho reads a line of data a stream and writes it back
func doEcho(s network.Stream) error {
b, err := io.ReadAll(s)
if err != nil {
return err
}
log.Printf("Number of bytes received: %d", len(b))
_, err = s.Write([]byte("thanks for the bytes"))
return err
}
When I run this and send a message, I do see the listener received new stream log but the doEcho function gets stuck after the io.ReadAll(s) call and never executes the reply.
So my questions are:
Why does my code not work and how can I make it work?
How does io.ReadAll(s) and bufio's ReadString('\n') work under the hood so that they cause this difference in behavior?
Edit:
As per #Stephan Schlecht suggestion I changed my code to this, but it still remains blocked as before:
func doEcho(s network.Stream) error {
buf := bufio.NewReader(s)
var data []byte
for {
b, err := buf.ReadByte()
if err != nil {
break
}
data = append(data, b)
}
log.Printf("Number of bytes received: %d", len(data))
_, err := s.Write([]byte("thanks for the bytes"))
return err
}
Edit 2: I forgot to clarify this, but I don't want to use ReadString('\n') or ReadBytes('\n') because I don't know anything about the []byte I'm receiving, so it might not end with \n. I want to read any []byte from the stream and then write back to the stream.
ReadString('\n') reads until the first occurrence of \n in the input and returns the string.
io.ReadAll(s) reads until an error or EOF and returns the data it read. So unless an error or EOF occurs it does not return.
In principle, there is no natural size for a data structure to be received on stream-oriented connections.
It depends on the remote sender.
If the remote sender sends binary data and closes the stream after sending the last byte, then you can simply read all data up to the EOF on the receiver side.
If the stream is not to be closed immediately and the data size is variable, there are further possibilities: One first sends a header that has a defined size and in the simplest case simply transmits the length of the data. Once you have received the specified amount of data, you know that this round of reception is complete and you can continue.
Alternatively, you can define a special character that marks the end of the data structure to be transmitted. This will not work if you want to transmit arbitrary binary data without encoding.
There are other options that are a little more complicated, such as splitting the data into blocks.
In the example linked in the question, a \n is sent at the end of the data just sent, but this would not work if you want to send arbitrary binary data.
Adapted Echo Example
In order to minimally modify the echo example linked in the question to first send a 1-byte header with the length of the payload and only then the actual payload, it could look something like the following:
Sending
In the function runSender line one could replace the current sending of the payload from:
log.Println("sender saying hello")
_, err = s.Write([]byte("Hello, world!\n"))
if err != nil {
log.Println(err)
return
}
to
log.Println("sender saying hello")
payload := []byte("Hello, world!")
header := []byte{byte(len(payload))}
_, err = s.Write(header)
if err != nil {
log.Println(err)
return
}
_, err = s.Write(payload)
if err != nil {
log.Println(err)
return
}
So we send one byte with the length of the payload before the actual payload.
Echo
The doEcho would then read the header first and afterwards the payload. It uses ReadFull, which reads exactly len(payload) bytes.
func doEcho(s network.Stream) error {
buf := bufio.NewReader(s)
header, err := buf.ReadByte()
if err != nil {
return err
}
payload := make([]byte, header)
n, err := io.ReadFull(buf, payload)
log.Printf("payload has %d bytes", n)
if err != nil {
return err
}
log.Printf("read: %s", payload)
_, err = s.Write(payload)
return err
}
Test
Terminal 1
2022/11/06 09:59:38 I am /ip4/127.0.0.1/tcp/8088/p2p/QmVrjAX9QPqihfVFEPJ2apRSUxVCE9wnvqaWanBz2FLY1e
2022/11/06 09:59:38 listening for connections
2022/11/06 09:59:38 Now run "./echo -l 8089 -d /ip4/127.0.0.1/tcp/8088/p2p/QmVrjAX9QPqihfVFEPJ2apRSUxVCE9wnvqaWanBz2FLY1e" on a different terminal
2022/11/06 09:59:55 listener received new stream
2022/11/06 09:59:55 payload has 13 bytes
2022/11/06 09:59:55 read: Hello, world!
Terminal 2
stephan#mac echo % ./echo -l 8089 -d /ip4/127.0.0.1/tcp/8088/p2p/QmVrjAX9QPqihfVFEPJ2apRSUxVCE9wnvqaWanBz2FLY1e
2022/11/06 09:59:55 I am /ip4/127.0.0.1/tcp/8089/p2p/QmW6iSWiFBG5ugUUwBND14pDZzLDaqSNfxBG6yb8cmL3Di
2022/11/06 09:59:55 sender opening stream
2022/11/06 09:59:55 sender saying hello
2022/11/06 09:59:55 read reply: "Hello, world!"
s
This is certainly a fairly simple example and will certainly need to be customized to your actual requirements, but could perhaps be a first step in the right direction.

When the TCP connection close in this code?

When I read this opensource code.
I have two questions about the two functions:
func listenTCP() {
for {
conn, err := tcpListener.Accept()
if err != nil {
if netErr, ok := err.(net.Error); ok && netErr.Temporary() {
log.Printf("Temporary error while accepting connection: %s", netErr)
}
log.Fatalf("Unrecoverable error while accepting connection: %s", err)
return
}
go handleTCPConn(conn) // check below
}
}
func handleTCPConn(conn net.Conn) {
log.Printf("Accepting TCP connection from %s with destination of %s", conn.RemoteAddr().String(), conn.LocalAddr().String())
defer conn.Close()
remoteConn, err := conn.(*tproxy.Conn).DialOriginalDestination(false)
if err != nil {
log.Printf("Failed to connect to original destination [%s]: %s", conn.LocalAddr().String(), err)
return
}
defer remoteConn.Close()
var streamWait sync.WaitGroup
streamWait.Add(2)
streamConn := func(dst io.Writer, src io.Reader) {
io.Copy(dst, src)
streamWait.Done()
}
go streamConn(remoteConn, conn)
go streamConn(conn, remoteConn)
streamWait.Wait()
}
Based on my understanding, I draw this diagram:
You see, the handleTCPConn created two goroutines for transmitting two direction(left -> right; right -> left)'s traffic,
My questions are:
You see the code use sync.WaitGroup, if they only send left-> right traffic, there is no traffic in opposite direction, so the handleTCPConn will not end, right? if it is, the listenTCP for loop will create many of those handleTCPConn function calls, is there nothing wrong with this program?
Every time the handleTCPConn is used, it will create a TCP connection to the remote server.
remoteConn, err := conn.(*tproxy.Conn).DialOriginalDestination(false)
My question is still in question 1, you can see that the handleTCPConn transmit the traffic once in both directions, and then ends it, whether the TCP connection is closed when does handleTCPConn end?
if they only transmit part of the data of a file(as per the application layer view), whether it is closed too? (i mean, if A->B->C: part data , then C->B->A: ACK ) .
per the golang docs, https://pkg.go.dev/io#Copy
Copy copies from src to dst until either EOF is reached on src or an error occurs. It returns the number of bytes copied and the first error encountered while copying, if any.
So when you start this program up, it will sit there and wait for you to hit the 'proxy', and send your bytes from the source to the destination... when the destination responds it will copy all those bytes back. if the destination doesn't write any bytes and doesn't close the connection i believe it'll sit there forever, waiting for the far side to either close the socket or respond.
Same is true if you make this connection and the remote sides starts sending data (without a request first). If the "local" side never sends any bytes and doesn't close the connection this code would wait forever as well.
As long as the remote side closes the connection gracefully, this code should exit with "0" bytes received and no error. If the remote side sends a reset, you should get an error of some kind

when all I get is EOF from a net.TCPConn, how do I know if the connection has been dropped?

The godocs say:
"EOF is the error returned by Read when no more input is available. Functions should return EOF only to signal a graceful end of input. If the EOF occurs unexpectedly in a structured data stream, the appropriate error is either ErrUnexpectedEOF or some other error giving more detail."
and in my program I would love love to get ErrUnexpectedEOF, but I don't. I always get just EOF even when the client terminates in the middle of sending a file.
so, i'm stuck. Successful files that transfer 100% of the way thru send EOF at end. And files that quit 50% thru with error also send EOF as the error. I do not know the total size of the file before the transfer.
Is there any more information I could get from the net.TCPConn to know was this a full file or not?
TCP is a streaming protocol; a stream of bytes. That's why file transfer protocols invented/implemented upon TCP, like FTP (File Transfer Protocol).
But (not recommended/thought experiment) if you can send your files in chunks that does not contain a certain character/byte (like base64 + \n as that certain character), then you can use that certain character as delimiter.
Now if we handle our connections using a function similar to this:
func (srv *Server) handler(conn net.Conn) {
//...
defer srv.closeConnection(conn, nil)
reader := bufio.NewReader(conn)
writer := bufio.NewWriter(conn)
//...
conn.SetDeadline(time.Now().Add(conf.ClientTimeout))
for {
select {
case <-conf.QuitSignal:
return
default:
line, err := reader.ReadBytes('\n')
if err != nil {
return
}
//...
}
conn.SetDeadline(time.Now().Add(conf.ClientTimeout))
}
}
func (srv *Server) closeConnection(conn net.Conn, err error) {
conn.Close()
//...
}
From the documentation of Reader.ReadBytes we read ReadBytes returns err != nil if and only if the returned data does not end in delimiter. This way we can recognize a broken chunk.

Trouble programming a client that allows sending text messages to server

So I've programmed a server that receives text messages from a connecting client, reverses and capses them and sends them back.
Now I'm trying to program a client so that when I launch it it will keep running until I shut it down (ctrl + c) and allow me to input text lines and send them to the server.
I have a problem though - if I pass a, say, cyrillic symbol to the input, it will return a <nil> <nil> (type, value) error and will remain bugged unless I flush the memory somehow.
I also can't figure how to read the whole message (whole meaning the size of the slice (1024 bytes)) instead of each word separately.
Also, how do I figure out how to delay my 'enter your message' text? Depending on the length of the message I pass on to the server, it should wait longer or shorter. I don't want it popping all over the place if the message is split into a few messages, just once after the answer is received.
Here's the relevant code:
func client() {
// connect to the server
c, err := net.Dial("tcp", "127.0.0.1"+":"+port)
if err != nil {
log.Printf("Dial error: %T %+v", err, err)
return
}
// send the message
msg := ""
for {
fmt.Print("Enter your message:\n")
_, errs := fmt.Scan(&msg)
if errs != nil {
log.Printf("Scan error: %T %+v", errs, errs)
return
}
fmt.Println("Client sending:\n", msg)
_, errw := c.Write([]byte(msg))
if errw != nil {
log.Printf("Write error: %T %+v", errw, errw)
return
}
// handle the response
go handleServerResponse(c)
time.Sleep(1 * time.Second)
}
func main() {
port = "9999"
// launch client
done := make(chan bool)
go client()
<-done // Block forever
}
I've used the empty channel to block the main() from ending.
How should I approach the 2 problems explained above?
Question answered by #JimB:
You're using fmt.Scan which scans space separated values. Don't use
that if you don't want to read each value separately. You can use
Scanln to read a line, or just read directly from stdin.

WebSocket - Closing Handshake Gorilla

Snippet from WebSocket RFC:
To Start the WebSocket Closing Handshake with a status code (Section 7.4) /code/ and an optional close reason (Section 7.1.6) /reason/, an endpoint MUST send a Close control frame, as described in Section 5.5.1, whose status code is set to /code/ and whose close reason is set to /reason/. Once an endpoint has both sent and received a Close control frame, that endpoint SHOULD Close the WebSocket Connection as defined in Section 7.1.1.
I am trying to do the Close Handshake using Gorilla WebSocket package with the following code:
Server:
// Create upgrader function
conn, err := upgrader.Upgrade(w, r, nil)
// If there is an error stop everything.
if err != nil {
fmt.Println(err)
return
}
for {
// Read Messages
_, _, err := conn.ReadMessage()
// Client is programmed to send a close frame immediately...
// When reading close frame resend close frame with same
// reason and code
conn.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(1000, "woops"))
fmt.Println(err)
break
}
Client:
d := &websocket.Dialer{}
conn, _, err := d.Dial("ws://localhost:8080", nil)
if err != nil {
fmt.Println(err)
return
}
go func() {
for {
// Read Messages
_, _, err := conn.ReadMessage()
if c, k := err.(*websocket.CloseError); k {
if(c.Code == 1000) {
// Never entering since c.Code == 1005
fmt.Println(err)
break
}
}
}
}()
conn.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(1000, "woops"))
for {}
The server is reading the Close Frame as expected outputting the following:
websocket: close 1000 (normal): woops
However the client is like its stopping to read once it sends a close message. The ReadMessage continue to return error 1005. What am I doing wrong?
The server responds to a close frame with the code:
c.WriteControl(CloseMessage, []byte{}, time.Now().Add(writeWait))
This is translated to close code 1005 (no status received) by the client.
The 1000 oops close frame written by the server is not seen by the client application because the websocket connection stops reading from network after receiving the first close frame.
The client application should exit the loop when an error is returned from ReadMessage. There's no need to check for specific close codes.
for {
// Read Messages
_, _, err := conn.ReadMessage()
if err != nil {
break
}
}
Unrelated to the issue in the question, the server application should close the websocket connection after sending the close frame.
Also unrelated to the issue in the question, use select {} instead of for {} to block the main goroutine. The former simply blocks the goroutine. The latter spins using CPU time.

Resources