I have a Golang project that utilizes conn.SetDeadline(). If a an EOF error is thrown because a read timed out, does Go automatically close the connection?
I have a setup where I need to wait a certain amount of time on a network connection for output to arrive, and if the output doesn't arrive, then it has to send a QUIT command. I haven't designed the network application, so redesigning the protocol isn't an option
Ideally, when an EOF is thrown because of SetDeadline timing out, then I would want the goroutine to awaken, but not for the connection to close
Thank you for your help in advance!
Apparently it doesn't. My workaround (well, not workaround, but the correct way to do this) was like so
timeout := make(chan error)
buf := make([]byte, 32)
go func() {
_, err := conn.Read(buf)
timeout <- err
}()
select {
case time.After(time.Now() + 1000 * 1000 * 1000 * 5): // Wait for 5 seconds
// Timed out reading
go func() {
<-timeout // We have to read from the sem to prevent mem leaks
}()
case err := <-timeout:
// Successfully read
}
Related
I'm trying to develop a simple API for Slack and I want to return something to the user right away to avoid the 3000 ms timeout.
Here are my questions:
Why the This should be printed to Slack first doesn't get printed right away, instead I only got the last message which is The long and blocking process completed? But it appears in ngrok log though.
Why is my function still reaching the 3000 ms limit even though I'm already using a go routine? Is it because of the done channel?
func testFunc(w http.ResponseWriter, r *http.Request) {
// Return to the user ASAP to avoid 3000ms timeout.
// But this doesn't work. Nothing is returned but
// the message appeared in ngrok log.
fmt.Fprintln(w, "This should be printed to Slack first!")
// Get the response URL.
r.ParseForm()
responseURL := r.FormValue("response_url")
done := make(chan bool)
go func() {
fmt.Println("Warning! This is a long and blocking process.")
time.Sleep(5 * time.Second)
done <- true
}()
// This works! I received this message. But I still reached the 3000ms timeout.
func(d bool) {
if d == true {
payload := map[string]string{"text": "The long and blocking process completed!"}
j, err := json.Marshal(payload)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
}
http.Post(responseURL, "application/json", bytes.NewBuffer(j))
}
}(<-done)
}
http.ResponseWriter streams are buffered by default. If you want data to be sent to a client in realtime (e.g. HTTP SSE), you need to flush the stream after each 'event':
wf, ok := w.(http.Flusher)
if !ok {
http.Error(w, "Streaming unsupported!", http.StatusInternalServerError)
return
}
fmt.Fprintln(w, "This should be printed to Slack first!")
wf.Flush()
Flushing is expensive - so take advantage of go's buffering. There will always be an implicit flush once your handler finally exits (hence why you saw your output 'late').
I am learning stage of golang. Here is my understanding:
1. any operations with channel is blocking
2. You are writing on the channel in the
go func() {
fmt.Println("Warning! This is a long and blocking process.")
time.Sleep(5 * time.Second)
done <- true
}()
The scheduler is still moving in the main function and trying to read from the channel but it is waiting to see, something has written on the channel or not. So it got blocked, when the above function is done with writing on the channel, controls came back and main start executing again.
Note: Experts will be able to explain better.
I have a go program that uses a goroutine to read UDP packets.
I wanted to use a select clause and a "stopping" channel to close the goroutine to shut down as soon as it is not needed anymore.
Here is a simple code example for the goroutine:
func Run(c chan string, q chan bool, conn *net.UDPConn) {
defer close(c)
buf := make([]byte, 1024)
for {
select {
case <- q:
return
default:
n, _, err := conn.ReadFromUDP(buf)
c <- string(buf[0:n])
fmt.Println("Received ", string(buf[0:n]))
if err != nil {
fmt.Println("Error: ", err)
}
}
}
}
The connection is created as:
conn, err := net.ListenUDP("udp",addr.Addr)
And the goroutine is supposed to terminate using:
close(q)
After closing the "stopping" channel ("q") the goroutine does not immediately stop. I need to send one more string via the UDP connection. When doing so the goroutine stops.
I simply do not understand this behaviour and I would be grateful if somebody could enlighten me.
Thank you in advance!
Your program is likely stopped at this line when you close the channel:
n, _, err := conn.ReadFromUDP(buf)
Because execution is blocked at a ReadFrom method, the select statement is not being evaluated, therefore the close on channel q is not immediately detected. When you do another send on the UDP connection, ReadFrom unblocks and (once that loop iteration finishes) control moves to the select statement: at that point the close on q is detected.
You can close the connection to unblock ReadFrom, as was suggested in a comment. See the PacketConn documentation in the net package, especially "Any blocked ReadFrom or WriteTo operations will be unblocked and return errors":
// Close closes the connection.
// Any blocked ReadFrom or WriteTo operations will be unblocked and return errors.
Close() error
Depending on your needs a timeout might be an option as well, see again PacketConn documentation in the net package:
// ReadFrom reads a packet from the connection,
// copying the payload into b. It returns the number of
// bytes copied into b and the return address that
// was on the packet.
// ReadFrom can be made to time out and return
// an Error with Timeout() == true after a fixed time limit;
// see SetDeadline and SetReadDeadline.
ReadFrom(b []byte) (n int, addr Addr, err error)
When I close a browser I want to disconnect a websocket in 3 seconds instead of 1 minute. The following just keep writing into a void without error until the tcp ip timeout I guess, not the SetWriteDeadline.
f := func(ws *websocket.Conn) {
for {
select {
case msg := <-out:
ws.SetWriteDeadline(time.Now().Add(3 * time.Second))
if _, err := ws.Write([]byte(msg)); err != nil {
fmt.Println(err)
return
}
case <-time.After(3 * time.Second):
fmt.Println("timeout 3")
return
}
}
}
return websocket.Handler(f)
I need to wait for this err
write tcp [::1]:8080->[::1]:65459: write: broken pipe
before it finally closes the connection, which takes about a minute or more.
You are you using WriteDeadline correctly. The deadline specifies the time for writing data to the TCP stack's buffers, not the time that the peer receives the data (if it does at all).
To reliably detect closed connections, the application should send PINGs to the peer and wait for the expected PONGs. The package you are using does not support this functionality, but the Gorilla package does. The Gorilla chat application shows how use PING and PONG to detect closed connections.
http.Serve either returns an error as soon as it is called or blocks if successfully executing.
How can I make it so that if it blocks it does so in its own goroutine? I currently have the following code:
func serveOrErr(l net.Listener, handler http.Handler) error {
starting := make(chan struct{})
serveErr := make(chan error)
go func() {
starting <- struct{}{}
if err := http.Serve(l, handler); err != nil {
serveErr <- err
}
}()
<-starting
select {
case err := <-serveErr:
return err
default:
return nil
}
}
This seemed like a good start and works on my test machine but I believe that there are no guarantees that serveErr <- err would be called before case err := <-serveErr therefore leading to inconsistent results due to a data race if http.Serve were to produce an error.
http.Serve either returns an error as soon as it is called or blocks if successfully executing
This assumption is not correct. And I believe it rarely occurs. http.Serve calls net.Listener.Accept in the loop – an error can occur any time (socket closed, too many open file descriptors etc.). It's http.ListenAndServe, usually being used for running http servers, which often fails early while binding listening socket (no permissions, address already in use).
In my opinion what you're trying to do is wrong, unless really your net.Listener.Accept is failing on the first call for some reason. Is it? If you want to be 100% sure your server is working, you could try to connect to it (and maybe actually transmit something), but once you successfully bound the socket I don't see it really necessary.
You could use a timeout on your select statement, e.g.
timeout := time.After(5 * time.Millisecond) // TODO: ajust the value
select {
case err := <-serveErr:
return err
case _ := <- timeout:
return nil
}
This way your select will block until serveErr has a value or the specified timeout has elapsed. Note that the execution of your function will therefore block the calling goroutine for up to the duration of the specified timeout.
Rob Pike's excellent talk on go concurrency patterns might be helpful.
It doesn't seem possible to have two way communication via channels with a goroutine which is performing file operations, unless you block the channel communication on the file operations. How can I work around the limits this imposes?
Another way to phrase this question...
If I have a loop similar to the following running in a goroutine, how can I tell it to close the connection and exit without blocking on the next Read?
func readLines(response *http.Response, outgoing chan string) error {
defer response.Body.Close()
reader := bufio.NewReader(response.Body)
for {
line, err := reader.ReadString('\n')
if err != nil {
return err
}
outgoing <- line
}
}
It's not possible for it to read from a channel that tells it when to close down because it's blocking on the network reads (in my case, that can take hours).
It doesn't appear to be safe to simply call Close() from outside the goroutine, since the Read/Close methods don't appear to be fully thread safe.
I could simply put a lock around references to response.Body that used inside/outside the routine, but would cause the external code to block until a pending read completes, and I specifically want to be able to interrupt an in-progress read.
To address this scenario, several io.ReadCloser implementations in the standard library support concurrent calls to Read and Close where Close interrupts an active Read.
The response body reader created by net/http Transport is one of those implementations. It is safe to concurrently call Read and Close on the response body.
You can also interrupt an active Read on the response body by calling the Transport CancelRequest method.
Here's how implement cancel using close on the body:
func readLines(response *http.Response, outgoing chan string, done chan struct{}) error {
cancel := make(chan struct{})
go func() {
select {
case <-done:
response.Body.Close()
case <-cancel:
return
}()
defer response.Body.Close()
defer close(cancel) // ensure that goroutine exits
reader := bufio.NewReader(response.Body)
for {
line, err := reader.ReadString('\n')
if err != nil {
return err
}
outgoing <- line
}
}
Calling close(done) from another goroutine will cancel reads on the body.