I'm writing a simple tcp server in golang that writes to "output.txt". All the "server part" works fine, but it does not write to file.(but it does create "output.txt")
func handleRequest(conn net.Conn) {
// Make a buffer to hold incoming data.
buf := make([]byte, 1024)
// Read the incoming connection into the buffer.
size, err := conn.Read(buf)
if err != nil {
fmt.Println("Error reading:", err.Error())
}
s := string(buf[:size])
fmt.Println(s)
perm := os.FileMode(0644)
err = ioutil.WriteFile("output.txt", buf, perm)
check(err)
// Close the connection when you're done with it.
conn.Close()
}
Why does it not write to file and how to fix this?
Related
I'm trying to do simple UNIX socket communication with a service using GO. For testing I've created a socket like so:
$ nc -vlU /tmp/sock
Bound on /tmp/sock
Listening on /tmp/sock
And in GO, I net.Dial and try to write something, then read. I see the written data in the nc console, so I know that is working. But the net.Conn.Read operation appears non-blocking and will return immediately with zero length. From everything I've read and the example I've seen, this operation should block.
buf := make([]byte, 0, 4096)
ctl, err := net.Dial("unix", "/tmp/sock")
for {
ctl.Write([]byte("test write\n"))
n, err := ctl.Read(buf)
fmt.Printf("Len:%v, Data:%v, err:%v", n, buf, err)
}
I see the connection come through and data written ...
Connection received on /tmp/sock
test write
test write
test write
test write
...
But the GO console loops without blocking, reporting a zero length and no error
Len:0, Data:[], err:<nil>
Len:0, Data:[], err:<nil>
Len:0, Data:[], err:<nil>
Len:0, Data:[], err:<nil>
...
And of course if I type anything into the nc console nothing interesting happens in the GO program output.
Any idea what I'm doing wrong?
short answer
buf := make([]byte, 0, 4096)
this code make a buffer with len=0!!!
do it like
buf := make([]byte, 4096)
example
this is some example i used, and they all works.
In practice, socket read and write should happen in two goroutines
package main
import (
"bufio"
"fmt"
"io"
"net"
"time"
)
func main() {
buf := make([]byte, 0, 4096)
fmt.Println(buf)
main03()
}
func main01() {
// nc -vlU /tmp/sock0120
ctl, err := net.Dial("unix", "/tmp/sock0120")
if err != nil {
fmt.Println(err)
return
}
reader := bufio.NewReader(ctl)
for {
time.Sleep(time.Second)
ctl.Write([]byte("test write\n"))
msg, err := reader.ReadString('\n')
fmt.Printf("Len:%v, Data:%v, err:%v\n", len(msg), msg, err)
}
}
func main02() {
// nc -vlU /tmp/sock0120
buf := make([]byte, 4)
ctl, err := net.Dial("unix", "/tmp/sock0120")
if err != nil {
fmt.Println(err)
return
}
reader := bufio.NewReader(ctl)
for {
time.Sleep(time.Second)
ctl.Write([]byte("test write\n"))
n, err := io.ReadFull(reader, buf)
fmt.Printf("Len:%v, Data:%v, err:%v\n", n, buf, err)
}
}
func main03() {
// nc -vlU /tmp/sock0120
buf := make([]byte, 4)
ctl, err := net.Dial("unix", "/tmp/sock0120")
if err != nil {
fmt.Println(err)
return
}
for {
time.Sleep(time.Second)
ctl.Write([]byte("test write\n"))
n, err := ctl.Read(buf)
fmt.Printf("Len:%v, Data:%v, err:%v\n", n, buf, err)
}
}
I'm writing a simple TLV style service using TCP in go, that should be able to handle multiple connections, and allow for multiple messages to be sent on the same connection.
I need to be able to ensure that a message with incorrect length doesn't block the connection indefinitely, but I need to ensure that subsequent messages can be sent on the same connection without timing out.
I am using io.ReadFull to read in a fixed number of bytes that contain the type and length, and then when I receive the length, I am calling io.ReadFull again and reading in length number of bytes.
If I am reading in 4 bytes for the type and length, but the client only sends 3 for whatever reason, io.ReadFull will hang. Similarly, if the client sends 300 for the length, but the length should only be 200, io.ReadFull will hang, blocking all communication on that channel.
I've tried using conn.SetReadDeadline() and setting it to 5 seconds for this example. This causes the connection to timeout if the improper length is sent, which is great. The issue is that the connection will timeout if the next request isn't sent until after >5 seconds.
// ...
for {
conn, err := lis.Accept()
if err != nil {
fmt.Println(err)
continue
}
fmt.Println("Connected")
go handleC(conn)
}
func handleC(conn net.Conn) {
for {
err := conn.SetReadDeadline(time.Now().Add(5 * time.Second))
if err != nil {
fmt.Println(err)
break
}
l, err := readTL(conn)
if err != nil {
if err, ok := err.(net.Error); ok && err.Timeout() {
fmt.Println("Timeout", err)
break
}
fmt.Println("Other error"), err
break
}
v, err := readV(conn, l)
if err != nil {
if err, ok := err.(net.Error); ok && err.Timeout() {
fmt.Println("Timeout", err)
break
}
fmt.Println("Other error"), err
break
}
// go doStuffWithv()
}
}
func readTL(conn net.Conn) (l int, err error) {
buf := make([]byte, 4)
n, err := io.ReadFull(conn, buf)
if err != nil {
return l, err
}
fmt.Printf("Read %d bytes\n", n)
// getLengthFromTL()
return l, err
}
func readV(conn net.Conn, l int) (v []byte, err error) {
buf := make([]byte, l)
n, err := io.ReadFull(conn, buf)
if err != nil {
return v, err
}
fmt.Printf("Read %d bytes\n", n)
return v, err
}
If a client sends one request with the proper TL, things work as intended.
However, if the same client doesn't send a second message for 10 seconds, the connection will timeout before then, with the error tls: use of closed connection
Is there a way to ensure that this doesn't occur?
One thing I've tried doing is in the event of a timeout, it simply continues, rather than breaking.
I added in another error check to see if it is EOF, and break if it is.
My first impression is that this works, but I'm not sure if there are instances where a connection timeout can mean that the connection is dead and shouldn't be used anymore or not, or if that would always return an EOF error.
I have a little problem. I want to receive and write udp packets with a DialUDP function. The problem is that it only can write and not receive udp packets (i think). How can i send a packet and let the program wait on a respond from the server?
my code:
if programPacketType == "udp"{
server,err := net.ResolveUDPAddr("udp4", programAddressInput)
checkError(err)
lServer, err := net.ResolveUDPAddr("udp4", programLocalAddressInput)
checkError(err)
conn, err := net.DialUDP("udp4", server, lServer)
checkError(err)
i := 0
for {
msg := strconv.Itoa(i)
i++
buf := []byte(msg)
_,err := conn.Write(buf)
if err != nil {
fmt.Println(msg, err)
}
fmt.Println("sended:", i, "to connected server")
time.Sleep(time.Second * 1)
}
}
thanks.
(Pls ignore grammatical mistakes,)
I'm creating a simple chat server as a personal project to learn net package and some concurrency in go. My 1st idea is to make the server print whatever is send using nc command echo -n "hello" | nc -w1 -4 localhost 2016 -p 61865. However after the 1st read my code ignores the subsequent messages.
func (s *Server) messageReader(conn net.Conn) {
defer conn.Close()
buffer := make([]byte, 1024)
for {
//read buff
blen, err := conn.Read(buffer)
if err != nil {
log.Fatal(err)
}
message := string(buffer[:blen])
if message == "/quit" {
fmt.Println("quit command received. Bye.")
return
}
if blen > 0 {
fmt.Println(message)
buffer = buffer[:0]
}
}
}
// Run Start up the server. Manages join and leave chat
func (s *Server) Run() {
// Listen on port TCP 2016
listener, err := net.Listen("tcp", ":2016")
if err != nil {
log.Fatal(err)
}
defer listener.Close()
for {
//wait for connection
conn, err := listener.Accept()
if err != nil {
log.Fatal(err)
}
go s.messageReader(conn)
}
}
If I send a new message from a new client it prints without problems but if I send another one it does nothing. What am I missing do I need to reset the Conn or close it and spawn a new one?
After printing your message, you slice buffer down to zero length. You can't read any data into a zero-length slice. There's no reason to re-slice your read buffer at all.
You also need to handle the read bytes before checking for errors, as io.EOF can be returned on a successful read.
You shouldn't use log.Fatal in the server's read loop, as that calls os.Exit
A working messageReader body might look like:
defer conn.Close()
buffer := make([]byte, 1024)
for {
n, err := conn.Read(buffer)
message := string(buffer[:n])
if message == "/quit" {
fmt.Println("quit command received. Bye.")
return
}
if n > 0 {
fmt.Println(message)
}
if err != nil {
log.Println(err)
return
}
}
You should note though that because you're not using any sort of framing protocol here, you can't guarantee that each conn.Read returns a complete or single message. You need to have some sort of higher-level protocol to delimit messages in your stream.
I am trying to stream out bytes of a zip file using io.Pipe() function in golang. I am using pipe reader to read the bytes of each file in the zip and then stream those out and use the pipe writer to write the bytes in the response object.
func main() {
r, w := io.Pipe()
// go routine to make the write/read non-blocking
go func() {
defer w.Close()
bytes, err := ReadBytesforEachFileFromTheZip()
err := json.NewEncoder(w).Encode(bytes)
handleErr(err)
}()
This is not a working implementation but a structure of what I am trying to achieve. I don't want to use ioutil.ReadAll since the file is going to be very large and Pipe() will help me avoid bringing all the data into memory. Can someone help with a working implementation using io.Pipe() ?
I made it work using golang io.Pipe().The Pipewriter writes byte to the pipe in chunks and the pipeReader reader from the other end. The reason for using a go-routine is to have a non-blocking write operation while simultaneous reads happen form the pipe.
Note: It's important to close the pipe writer (w.Close()) to send EOF on the stream otherwise it will not close the stream.
func DownloadZip() ([]byte, error) {
r, w := io.Pipe()
defer r.Close()
defer w.Close()
zip, err := os.Stat("temp.zip")
if err != nil{
return nil, err
}
go func(){
f, err := os.Open(zip.Name())
if err != nil {
return
}
buf := make([]byte, 1024)
for {
chunk, err := f.Read(buf)
if err != nil && err != io.EOF {
panic(err)
}
if chunk == 0 {
break
}
if _, err := w.Write(buf[:chunk]); err != nil{
return
}
}
w.Close()
}()
body, err := ioutil.ReadAll(r)
if err != nil {
return nil, err
}
return body, nil
}
Please let me know if someone has another way of doing it.