How to check if an error is "deadline exceeded" error? - go

I'm sending a request with a context which specified with a 10 seconds timeout:
ctx, cancel := context.WithTimeout(context.Background(), time.Second * 10)
defer cancel()
_, err := client.SendRequest(ctx)
if err != nil {
return 0, err
}
now when I hit that timeout the error message is confusing:
context deadline exceeded
Is it possible to check if the err is the timeout error so that I can print a nicer error message?
ctx, cancel := context.WithTimeout(context.Background(), time.Second * 10)
defer cancel()
_, err := client.SendRequest(ctx)
if err != nil {
if isTimeoutError(err) {
return nil, fmt.Errorf("the request is timeout after 10 seconds")
}
return nil, err
}
How to implement such isTimeoutError function?

The cleanest way to do this in Go 1.13+ is using the new errors.Is function.
// Create a context with a very short timeout
ctx, cancel := context.WithTimeout(context.Background(), time.Millisecond)
defer cancel()
// Create the request with it
r, _ := http.NewRequest("GET", "http://example.com", nil)
r = r.WithContext(ctx)
// Do it, it will fail because the request will take longer than 1ms
_, err := http.DefaultClient.Do(r)
log.Println(err) // Get http://example.com: context deadline exceeded
// This prints false, because the http client wraps the context.DeadlineExceeded
// error into another one with extra information.
log.Println(err == context.DeadlineExceeded)
// This prints true, because errors.Is checks all the errors in the wrap chain,
// and returns true if any of them matches.
log.Println(errors.Is(err, context.DeadlineExceeded))

You can determine if an error is the result of a context timeout by comparing the error to context.DeadlineExceeded:
if err == context.DeadlineExceeded {
// context deadline exceeded
}
You can determine if an error is any timeout error using the following function:
func isTimeoutError(err error) bool {
e, ok := err.(net.Error)
return ok && e.Timeout()
}
This function returns true all timeout errors including the value context.DeadlineExceeded. That value satisfies the net.Error interface and has a Timeout method that always returns true.

Related

Calling an executable with a timeout

I am trying to use the context package to run a binary with a 10 second timeout, as such:
func RunQL(file db.File, flag string) string {
// 10-second timeout for the binary to run
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "qltool", "run", "-f", file.Path, "--rootfs", file.Root, "--args", flag)
out, err := cmd.Output()
// check to see if our timeout was executed
if ctx.Err() == context.DeadlineExceeded {
return ""
}
// no timeout (either completed successfully or errored)
if err != nil {
return ""
}
return string(out)
}
But for some reason, it still hangs if the process lasts longer than 10 seconds. Not sure what would be causing this, I also noticed that the documentation for the CommandContext() function appears to be wrong/misleading? It shows the following code:
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
if err := exec.CommandContext(ctx, "sleep", "5").Run(); err != nil {
// This will fail after 100 milliseconds. The 5 second sleep
// will be interrupted.
}
}
But CommandContext() returns type *Cmd not error .

"context deadline exceeded" AFTER first client-to-server request & response

In a simple gRPC example, I connect to the server, request to send a text message, and in return a success message is sent. Once the server is live, the first client request is successful, with no errors.
However, after the first try, each subsequent request (identical to the first) return this error, and does not return a response as the results (the text message is still sent, but the generated ID is not sent back):
rpc error: code = DeadlineExceeded desc = context deadline exceeded
After debugging a little bit, I found that the error
func (c *messageSchedulerClient) SendText(ctx context.Context, in *TextRequest, opts ...grpc.CallOption) (*MessageReply, error) {
...
err := c.cc.Invoke(ctx, "/communication.MessageScheduler/SendText", in, out, opts...)
...
return nil, err
}
returns
rpc error: code = DeadlineExceeded desc = context deadline exceeded
Here is my client code:
func main() {
// Set up a connection to the server.
conn, err := grpc.Dial(address, grpc.WithInsecure(), grpc.WithBlock())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
c := pb.NewMessageSchedulerClient(conn)
var r *pb.MessageReply
r, err = pbSendText(c, false)
if err != nil {
log.Fatalf("could not greet: %v", err)
}
log.Printf("Greeting: %s", r.GetId())
err = conn.Close()
if err != nil {
log.Printf("Connection Close Error: %v", err)
}
return
}
func pbSendText(c pb.MessageSchedulerClient, instant bool) (*pb.MessageReply, error) {
ctx, cancel := context.WithTimeout(context.Background(), time.Second * 5)
minute := timestamppb.New(time.Now().Add(time.Second * 30))
r, err := c.SendText(ctx, &pb.TextRequest{})
if err != nil {
log.Printf("DEBUG MESSAGE: Error after c.SendText(ctx, in): %v", err)
}
cancel()
return r, err
}
and the server code is...
func (s *MessageServer) SendText(ctx context.Context, in *pb.TextRequest) (*pb.MessageReply, error) {
return &pb.MessageReply{Id: GeneratePublicId()}, nil
}
func GrpcServe() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterMessageSchedulerServer(s, &MessageServer{})
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
return
}
const Alphabet = "abcdefghijklmnopqrstuvwxyzABCDEFGHJKMNOPQRSTUVWXYZ0123457"
// executes instantly, O(n)
func GeneratePublicId() string {
return id.NewWithAlphabet(Alphabet)
}
I've tried changing the context to .TODO from .Background. Doesn't work. I am SURE it's something so simple that I'm missing.
Again, the first time I execute the application from the client side, it works. But after the first client application execution, meaning application execution and beyond -- until I restart the gRPC Server, it gives me the error.
The GeneratePublicId function executes almost instantly, as it is O(n) and uses a random number function.

golang: grpc call timeout

I am using the below code to connect to a grpc server and clientConn object is used for all subsequent rpc calls. maxDelay is set to 5 seconds. Now because of some issue at server, it is not responding for a grpc call. So my client is waiting for a long time for each rpc call. Do i need to set timeout in a different way?
b := grpc.BackoffConfig{
MaxDelay: maxDelay,
}
clientConn, err := grpc.Dial(serverAddress, grpc.WithBackoffConfig(b), grpc.WithInsecure())
if err != nil {
log.Println("Dial failed!")
return err
}
You can modify your code to add a timeout using grpc.WithTimeout(5 * time.Second) instead of using MaxDelay and grpc.WithBackoffConfig(b) which are for retries and retries delay.
clientConn, err := grpc.Dial(serverAddress, grpc.WithTimeout(5 * time.Second), grpc.WithInsecure())
if err != nil {
log.Println("Dial failed!")
return err
}
However the above is deprecated, alternatively you can use DialContext and context.WithTimeout
ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
clientConn, err := grpc.DialContext(ctx, serverAddress, grpc.WithInsecure())
if err != nil {
log.Println("Dial failed!")
return err
}
The context.WithTimeout is used in the grpc.DialContext to control the timeout of all RPC calls of the current DialContext. It is inconvenient to handle the different timeouts of one/some specific RPC call.
We could define one custom timeout callOption to handle the forced timeout value of some RPC calls in the clientInterceptor
Define the custom timeout callOption through EmptyCallOption
type TimeoutCallOption struct {
grpc.EmptyCallOption
forcedTimeout time.Duration
}
func WithForcedTimeout(forceTimeout time.Duration) TimeoutCallOption {
return TimeoutCallOption{forcedTimeout: forceTimeout}
}
Handle the forcedTimeout in the UnaryClientInterceptor
func getForcedTimeout(callOptions []grpc.CallOption) (time.Duration, bool) {
for _, opt := range callOptions {
if co, ok := opt.(TimeoutCallOption); ok {
return co.forcedTimeout, true
}
}
return 0, false
}
func TimeoutInterceptor(t time.Duration) grpc.UnaryClientInterceptor {
return func(ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn,
invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
timeout := t
if v, ok := getForcedTimeout(opts); ok {
timeout = v
}
if timeout <= 0 {
return invoker(ctx, method, req, reply, cc, opts...)
}
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
return invoker(ctx, method, req, reply, cc, opts...)
}
}
Usage samples
// The default timeout of RPC call of this conn is 3 seconds
conn, err := grpc.Dial(
address,
grpc.WithUnaryInterceptor(TimeoutInterceptor(time.Duration(3)*time.Second)), ...)
....
c := pb.NewGreeterClient(conn)
c.SayHello(context.Background(), &pb.HelloRequest{Name: "world"},
WithForcedTimeout(time.Duration(10)*time.Second))
// The timeout of SayHello RPC call is 10 seconds

How to timeout RabbitMQConsumer if it didn`t receive response

I got code that consumes messages from RabbitMQ queue. It should fail if it doesn`t received message for some time.
msgs, err := ch.Consume(
c.ResponseQueue, // queue
"", // consumer
false, // auto-ack
false, // exclusive
false, // no-local
false, // no-wait
nil, // args
)
failOnError(err, "Failed to register a consumer")
...
loop:
for timeout := time.After(time.Second); ; {
select {
case <-timeout:
log.Printf("Failed to receive response for action %+v\n Payload: %+v\nError: %+v\n", action, body, err)
return errors.New("Failed to receive response for action")
default:
for d := range msgs {
if corrID == d.CorrelationId {
err = json.Unmarshal([]byte(uncompress(d.Body)), &v)
if err != nil {
return err
}
ch.Ack(d.DeliveryTag, false)
break loop
}
}
}
}
I took consume code from RabbitMQ manual and tried some advices for implementing timeout. I know how to do it in Java, but can`t repeat it in Golang.
Thanks in advance.
Update:
Changed select to this:
c1 := make(chan error, 1)
go func() {
for d := range msgs {
if corrID == d.CorrelationId {
err = json.Unmarshal([]byte(uncompress(d.Body)), &v)
if err != nil {
c1 <- err
}
ch.Ack(d.DeliveryTag, false)
c1 <- nil
}
}
}()
select {
case <-time.After(defaultTimeout * time.Second):
log.Printf("Failed to receive response for action %+v\n Payload: %+v\nError: %+v\n", action, body, err)
return errors.New("Failed to receive response in time for action")
case err := <-c1:
failOnError(err, "Failed to process response")
}
return err
Now it works as expected - if it doesn`t receive message with proper corellationId it will fail with timeout. Thanks for help everyone.
Your loop has a select with 2 cases: a timeout and a default branch. Upon entering the loop the timeout will not fire, so the default branch is executed.
The default branch contains a for range over the msgs channel which keeps receiving from the channel until it is closed (and all values have been received from it). Normally this shouldn't happen, so the timeout case will not be revisited (only if some error occurs and msgs is closed).
Instead inside the loop use a select with 2 cases, one timeout and one that receives only a single value from msgs. If a message is received, restart the timeout. For a restartable timer use time.Timer.
timeout := time.Second
timer := time.NewTimer(timeout)
for {
select {
case <-timer.C:
fmt.Println("timeout, returning")
return
case msg := <-msgs:
fmt.Println("received message:", msg)
// Reset timer: it must be stopped first
// (and drain its channel if it reports false)
if !timer.Stop() {
<-timer.C
}
timer.Reset(timeout)
}
}
Check this Go Playground example to see it in action.
Note that if you don't need to reset the timer once a message is received, just comment out the resetter code. Also, if no reset is needed, time.After() is simpler:
timeout := time.After(time.Second)
for {
select {
case <-timeout:
fmt.Println("timeout, returning")
return
case msg := <-msgs:
fmt.Println("received message:", msg, time.Now())
}
}
Try this one on the Go Playground.
One final note: if you would break from the loop before the timeout happens, the timer in the background would not be freed immediately (only when the timeout happens). If you need this operation frequently, you may use context.WithTimeout() to obtain a context.Context and a cancel function which you may call immediately before returning to free up the timer resource (preferably as deferred).
This is how it would look like:
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
for {
select {
case <-ctx.Done():
fmt.Println("timeout, returning")
return
case msg := <-msgs:
fmt.Println("received message:", msg, time.Now())
}
}
Try this one on the Go Playground.
Changed select to this:
c1 := make(chan error, 1)
go func() {
for d := range msgs {
if corrID == d.CorrelationId {
err = json.Unmarshal([]byte(uncompress(d.Body)), &v)
if err != nil {
c1 <- err
}
ch.Ack(d.DeliveryTag, false)
c1 <- nil
}
}
}()
select {
case <-time.After(defaultTimeout * time.Second):
log.Printf("Failed to receive response for action %+v\n Payload: %+v\nError: %+v\n", action, body, err)
return errors.New("Failed to receive response in time for action")
case err := <-c1:
failOnError(err, "Failed to process response")
}
return err

Websocket waiting for message with Timeout

I want to create a Websocket connection via GO. This connection follows a clearly defined pattern: The client should "authenticate" (enter data) themself immediately after creating the connection. If the client does not do it, the connection will be closed after a short period.
My current code contains this initial timeout (initTimeout) and the maximum timeout for all connections. While those timers can easily be checked, i am not sure how i can combine the timers with waiting for a message which blocks the execution.
ws, err := upgrader.Upgrade(w, r, nil)
initTimeout := time.NewTicker(time.Duration(30) * time.Second)
maxTimeout := time.NewTicker(time.Duration(45) * time.Minute)
for {
select {
case <- initTimeout.C:
ws.WriteMessage(websocket.TextMessage, []byte("No input received"))
ws.Close()
case <- maxTimeout.C:
ws.WriteMessage(websocket.TextMessage, []byte("Maximum timeout"))
ws.Close()
default:
mt, message, err := c.ReadMessage()
// will this block the timers?
}
}
Use the read deadline to implement the timeouts:
ws, err := upgrader.Upgrade(w, r, nil)
if err != nil {
// handle error
}
// Read the initial message with deadline of 30 seconds
ws.SetReadDeadline(time.Now().Add(30 * time.Second))
mt, message, err := ws.ReadMessage()
if err != nil {
// Handle the error which might be a deadline exceeded error.
}
// process the initial message
// ...
for {
// Read next message with deadline of 45 minutes
ws.SetReadDeadline(time.Now().Add(45 * time.Minute))
mt, message, err = ws.ReadMessage()
if err != nil {
// Handle the error which might be a deadline exceeded error.
}
// process message
// ....
}

Resources