I am currently studying network programming in Go.
Currently, a timeout test using the deadline of the context is in progress by myself.
package socket
import (
"context"
"net"
"syscall"
"testing"
"time"
)
func TestDialContext(t *testing.T) {
deadline := time.Now().Add(time.Second * 5)
ctx, cancelFunc := context.WithDeadline(context.Background(), deadline)
defer cancelFunc()
var dialer net.Dialer
dialer.Control = func(_, _ string, _ syscall.RawConn) error {
time.Sleep(4 * time.Second)
return nil
}
conn, err := dialer.DialContext(ctx, "tcp", "10.0.0.0:http")
if err == nil {
conn.Close()
t.Fatal("expected no connection")
}
if nErr, ok := err.(net.Error); ok && nErr.Timeout() {
t.Logf("expected timeout error: %v", err)
} else {
t.Errorf("expected timeout error; actual: %v", err)
}
if ctx.Err() != context.DeadlineExceeded {
t.Errorf("expected deadline exceeded; actual: %v", ctx.Err())
}
}
As you can see from the code, I tried to connect using a context with a 5-second deadline.
In addition, by using the dialer's control function when requested, the control function is called immediately after being connected (actually right before being connected), so that a nil error is returned after 4 seconds.
By my intent, this test should fail. The reason is that the control function returns a nil error before the deadline of 5 seconds is reached.
But the test succeeds as below.
=== RUN TestDialContext
dial_context_test.go:27: expected timeout error: dial tcp 10.0.0.0:80: i/o timeout
--- PASS: TestDialContext (5.01s)
PASS
What exactly is the purpose of the control function? Is the error returned by the control function unrelated to the error returned by the net.Dial() function? Is the test using the control function like the code I wrote meaningless?
Thanks for reading this long question!
What exactly is the purpose of the control function?
To give the dialing code a chance to call functions on the socket's underlying file descriptor — e.g. one of the syscall.Setsockopt functions — after the socket is created and before the connection is established.
Is the error returned by the control function unrelated to the error returned by the net.Dial() function?
If the control function returns an error, then dialing fails with that error. If the control function returns a nil error, then dialing continues as usual.
Is the test using the control function like the code I wrote meaningless?
Pretty much. Your control function sleeps 4 seconds but otherwise doesn't do anything.
Related
Helo All,
New to golang and was debugging timeout issues in a production environment. Before making a call to the server we add a timeout of 50ms to context and fire a server call. If the response is not received within 50 ms we expect the application to move on and not wait for the response.
But while debugging, I capture the duration between we fire a server call and the response received (or error out), to my surprise the value at the time is much higher than 50 ms.
Client syntax -
ctx, cancel := context.WithTimeout(ctx, e.opts.Timeout)
defer cancel()
fireServerCall(ctx)
..
..
def fireServerCall(ctx context:Context){
startTime:=time.Now()
//call to the server
res, err:=callToServer(ctx)
if err!=nil{
//capture failure latency
return ....
}
//capture success latency
return ....
}
Has anyone ever faced any similar issue? Is this expected behaviour? How did you handle such cases?
Am I doing something incorrectly? Suggestions are welcome :)
Edit:
I am passing context in my original code but forgot to mention it here, just added it. That mean, I am passing the same context on which my client is waiting for server to respond within 50 ms.
You should pass created context to fireServerCall and callToServer functions
callToServer should consider passed context and monitor ctx.Done() channel to stop its execution accordingly
Answering to comment by #Bishnu:
Don't think this is needed. Did a test and even without passing ctx to callToServer() it works. The behaviour is not as expected under high load. Can you kindly share some document/test what you have pointed here?
Context timeout just can't work without context passing and checking its Done() channel. Context is not some kind of magic — simplifying it is just a struct with done channel which is closed by calling cancel function or when timeout occurs. Monitoring this channel — is responsibility of the innermost function that accepts it.
Example:
package main
import (
"context"
"fmt"
"time"
)
func callToServer(ctx context.Context) {
now := time.Now()
select {
case <-ctx.Done(): // context cancelled via cancel or deadline
case <-time.After(1 * time.Second): // emulate external call
}
fmt.Printf("callToServer: %v\n", time.Since(now))
}
func callToServerContextAgnostic(ctx context.Context) {
now := time.Now()
select {
case <-time.After(2 * time.Second): // emulate external call
}
fmt.Printf("callToServerContextAgnostic: %v\n", time.Since(now))
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
callToServer(ctx)
ctx2, cancel2 := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel2()
callToServerContextAgnostic(ctx2)
}
Results:
callToServer: 100ms
callToServerContextAgnostic: 2s
You can launch it on Go Playground: https://go.dev/play/p/tIxjHxUzYfh
Note that many of the clients (from standard or third party libraries) monitors Done channel by themselves.
For example standard HTTP client:
c := &http.Client{} // client for all requests
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(time.Millisecond*100))
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://google.com", nil)
if err != nil {
log.Fatal(err)
}
resp, err := c.Do(req) // will monitor `Done` channel for you
Some docs and articles:
https://pkg.go.dev/context
https://www.digitalocean.com/community/tutorials/how-to-use-contexts-in-go
When I use shell it returns ok,
but when I use golang it returns err: context deadline exceeded.
Why?
package main
import (
"context"
"fmt"
"go.etcd.io/etcd/clientv3"
"time"
)
func main() {
cli,err := clientv3.New(clientv3.Config{
Endpoints: []string{"http://192.168.11.34:2379"},
DialTimeout: 5*time.Second,
})
if err != nil{
panic(err)
}
fmt.Println("connect etcd success!")
defer cli.Close()
ctx,cancel := context.WithTimeout(context.Background(),time.Second*5)
_,err = cli.Put(ctx,"mac","leave")
defer cancel()
if err !=nil{
fmt.Println("put etcd err:",err.Error())
return
}
}
connect etcd success!
{"level":"warn","ts":"2021-02-08T17:10:46.490+0800","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-5b14f6c4-2395-4a15-9ebb-cadf5985fb06/192.168.11.34:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 192.168.11.34:2379: connect: connection refused\""}
put etcd err: context deadline exceeded
Creation of client does not guarantee that the connection is established.
It might get delayed till the first request that actually fails.
Please make sure that your server is reachable.
Increase your timeout:
DialTimeout: 5*time.Second
Clearly this time is being exceeded if you are receiving a context deadline exceeded error.
Deferring the cancel() means that the context won't be canceled until after your Println and return (during function tear-down). If the Put() call is on the edge of the timeout period that will cause the error to be reported. But calling cancel() directly after the Put() call seems to also have issues (see my recent question)
Given the following function:
func main() {
l := createListener(8080)
r := ksws.CreateRouter()
if err := http.Serve(l, r); err != nil {
fmt.Println("An error occured.")
}
}
I'm wondering why I should catch the 'error' returned from the 'http.Serve' method?
It seems that an error is never returned here.
However, according to the documentation https://golang.org/pkg/net/http/#Serve the Serve method always returns a non-nill error.
Can someone provide me some guidance on this?
Simple case: when port 8080 already used you'll have error:
listen tcp :8080: bind: address already in use
Another case: http.Serve calls srv.trackListener which also may fail in case go didn't manage to add listener.
Also: http.Serve calls l.Accept() which also may fail...
So there are many possible cases...
And also it's idiomatic for go to check all errors returned by any function.
PS: It's way better to have redundant error check than to have silent not working program (imho)...
Take a look at the source code and it might shine some light into your question.
https://golang.org/src/net/http/server.go?s=75585:75634#L2838
// Serve always returns a non-nil error and closes l.
// After Shutdown or Close, the returned error is ErrServerClosed.
So the error will alway be return either with a real error if something went wrong or the ErrServerClosed in case of a shutdown or close, which happen for several reasons.
in your code when you ctrl-c, note that the prints does not occur because the program has ended. you should listen to signals to prevent that behavior.
package main
import (
"fmt"
"net/http"
"os"
"os/signal"
"time"
)
func main() {
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
r := http.NewServeMux()
srv := &http.Server{
Addr: ":0",
Handler: r,
}
go func() {
if err := srv.ListenAndServe(); err != nil {
fmt.Println("An error occured.")
}
}()
// Block until a signal is received.
<-c
srv.Close()
fmt.Println("rr")
<-time.After(time.Second)
}
I'm trying to start an HTTP server in Go, and when the server is started, a message should be printed, in case of an error, an error message should be printed.
Given the following code:
const (
HTTPServerPort = ":4000"
)
func main() {
var httpServerError = make(chan error)
var waitGroup sync.WaitGroup
setupHTTPHandlers()
waitGroup.Add(1)
go func() {
defer waitGroup.Done()
httpServerError <- http.ListenAndServe(HTTPServerPort, nil)
}()
if <-httpServerError != nil {
fmt.Println("The Logging API service could not be started.", <-httpServerError)
} else {
fmt.Println("Logging API Service running # http://localhost" + HTTPServerPort)
}
waitGroup.Wait()
}
When I do start the application, I don't see anything printed to the console, where I would like to see:
Logging API Service running # http://localhost:4000
When I change the port to an invalid one, the following output is printed to the console:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
...app.go:45 +0x107
exit status 2
Could anyone point me in the right direction so that I know what I'm doing wrong with this implementation?
You can't do this unless you change the logic in your code or use Listen and Serve separately. Because ListenAndServe is a blocking function. If there something unexpected happens, it will return you an error. Provided it is not, it will keep blocking running the server. There is neither an event that is triggered whenever a server is started.
Let's run Listen and Serve separately then.
l, err := net.Listen("tcp", ":8080")
if err != nil {
// handle error
}
// Signal that server is open for business.
if err := http.Serve(l, rootHandler); err != nil {
// handle error
}
See https://stackoverflow.com/a/44598343/4792552.
P.S. net.Listen doesn't block because it runs in background. In other means, it simply spawns a socket connection in OS level and returns you with the details/ID of it. Thus, you use that ID to proxy orders to that socket.
The issue is that your if statement will always read from the httpServerError channel. However the only time something writes to that is if the server fails.
Try a select statement:
select{
case err := <-httpServerError
fmt.Println("The Logging API service could not be started.", err)
default:
fmt.Println("Logging API Service running # http://localhost" + HTTPServerPort
}
The default case will be ran if the channel does not have anything on it.
Notice this does not read from the channel twice like your example. Once you read a value from a channel, its gone. Think of it as a queue.
http.Serve either returns an error as soon as it is called or blocks if successfully executing.
How can I make it so that if it blocks it does so in its own goroutine? I currently have the following code:
func serveOrErr(l net.Listener, handler http.Handler) error {
starting := make(chan struct{})
serveErr := make(chan error)
go func() {
starting <- struct{}{}
if err := http.Serve(l, handler); err != nil {
serveErr <- err
}
}()
<-starting
select {
case err := <-serveErr:
return err
default:
return nil
}
}
This seemed like a good start and works on my test machine but I believe that there are no guarantees that serveErr <- err would be called before case err := <-serveErr therefore leading to inconsistent results due to a data race if http.Serve were to produce an error.
http.Serve either returns an error as soon as it is called or blocks if successfully executing
This assumption is not correct. And I believe it rarely occurs. http.Serve calls net.Listener.Accept in the loop – an error can occur any time (socket closed, too many open file descriptors etc.). It's http.ListenAndServe, usually being used for running http servers, which often fails early while binding listening socket (no permissions, address already in use).
In my opinion what you're trying to do is wrong, unless really your net.Listener.Accept is failing on the first call for some reason. Is it? If you want to be 100% sure your server is working, you could try to connect to it (and maybe actually transmit something), but once you successfully bound the socket I don't see it really necessary.
You could use a timeout on your select statement, e.g.
timeout := time.After(5 * time.Millisecond) // TODO: ajust the value
select {
case err := <-serveErr:
return err
case _ := <- timeout:
return nil
}
This way your select will block until serveErr has a value or the specified timeout has elapsed. Note that the execution of your function will therefore block the calling goroutine for up to the duration of the specified timeout.
Rob Pike's excellent talk on go concurrency patterns might be helpful.