Handle multiple errors from a single channel - go

I have some trouble with go routines and channels regarding error handling.
Firstly I have a function that listen for messages (in a infinite for loop):
func main() {
messageChannel := make(chan messageHandler.MessageInfo)
for {
if token := client.Subscribe("#", 0, func(client MQTT.Client, msg MQTT.Message) {
go messageHandler.DecodeMessage(msg, messageChannel)
select {
case messageInfo := <-messageChannel:
//Handle
}
}); token.Wait() && token.Error() != nil {
fmt.Println(token.Error())
}
}
}
But in the DecodeMessage function, there could arise multiple errors.
func DecodeMessage(msg mqtt.Message, c1 chan MessageInfo) {
//do something, might result in error
//do another thing, might result in error
c1 <- MessageInfo{...}
}
Normally I would just return from the function. But seems a bit trickier with routines. I've looked at this post, but if both errors would occur, I would only see the last error message.
Example:
func DecodeMessage(msg mqtt.Message, c1 chan MessageInfo) {
var returnError error
if err != nil {
returnError = err
}
if err != nil {
returnError = err
}
c1 <- MessageInfo{
Error: returnError,
...
}
}
Should I have an array of some sort and append all errors? Is it bad practice to have multiple errors in one routine?
The best thing, for me, is that the routine would exit on an error and return that error like it would do "normally". Is that possible?

I'll start by saying that having to go through all the error checks for a function before returning even if they fail is a bit of a code smell. It might mean that you have something weird going on and there may be a better way to accomplish what you're trying to do.
However, assuming that you've boiled down your problem to this, then I see two options, depending on how the errors have to be handled.
If the errors can be handled one-by-one and don't really rely on each other, then you can just create an error channel and send them back one by one as you encounter the errors. See the following working example:
package main
import (
"errors"
"fmt"
"strings"
)
func main() {
errCh := make(chan error)
go HandleErrorsSerially("bad error", errCh)
for err := range errCh {
fmt.Printf("Found error serially: %v\n", err)
}
}
func HandleErrorsSerially(msg string, errCh chan<- error) {
if strings.Contains(msg, "error") {
errCh <- errors.New("message contained string 'error'")
}
if strings.Contains(msg, "bad") {
errCh <- errors.New("message contained string 'bad'")
}
close(errCh)
}
Alternatively, if you need to have a view of all the errors that occurred all at once (because two errors happening simultaneously may indicate some special circumstances) then you'd have to append them all to an array and then pass them through a channel. See the following working example:
package main
import (
"errors"
"fmt"
"strings"
)
func main() {
errArrCh := make(chan []error)
go HandleErrorsTogether("bad error", errArrCh)
errArr := <-errArrCh
fmt.Printf("Found the following errors together: %v\n", errArr)
}
func HandleErrorsTogether(msg string, errArrCh chan<- []error) {
errArr := make([]error, 0)
if strings.Contains(msg, "error") {
errArr = append(errArr, errors.New("message contained string 'error'"))
}
if strings.Contains(msg, "bad") {
errArr = append(errArr, errors.New("message contained string 'bad'"))
}
errArrCh <- errArr
close(errArrCh)
}

I can see cases where it's useful to return multiple errors, such as when you are parsing messages and there are multiple bad fields and you need to summarise these back to a client.
I think the best approach is to use a package like hashicorp's multierror which allows multiple errors to be collected with formatting in a structure type that implements error interface and so can still be sent on a chan error. The receive side can then either process as just a standard error or extract the information on each individual error.
The multierror documentation is pretty good, just read through the examples on the github page.

The best thing, for me, is that the routine would exit on an error and
return that error like it would do "normally".
Of course, you can, and get last error or all error are both pretty strange.
Get last error is unhelpful for debugging, compare with getting first error.
Get all error is same if the first error will cause the following failure, the following error message is unhelpful; Another situation is these errors do not have the association, I think this means you have to sperate they into different concurrency part for better controlling.
Ok, now back to original problem. Consider the code:
func foo(errTo chan error) {
defer close(errTo)
v, err := CouldFailOne()
if err != nil {
errTo <- err
return // Yp, just stop this routine, let it join back to Invoker
}
v2, err := CloudFailTwo()
if err != nil {
errTo <- err
return
}
// As the previous error handle until end of the function
}
If you want to return value from this kind of function. Just use a channel, and send the value into it only no error raise. I think this style will be more clear and like return style, just became using a channel to return the error.

Related

Mock/test basic http.get request

I am leaning to write unit tests and I was wondering the correct way to unit test a basic http.get request.
I found an API online that returns fake data and wrote a basic program that gets some user data and prints out an ID:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
)
type UserData struct {
Meta interface{} `json:"meta"`
Data struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Gender string `json:"gender"`
Status string `json:"status"`
} `json:"data"`
}
func main() {
resp := sendRequest()
body := readBody(resp)
id := unmarshallData(body)
fmt.Println(id)
}
func sendRequest() *http.Response {
resp, err := http.Get("https://gorest.co.in/public/v1/users/1841")
if err != nil {
log.Fatalln(err)
}
return resp
}
func readBody(resp *http.Response) []byte {
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
return body
}
func unmarshallData(body []byte) int {
var userData UserData
json.Unmarshal(body, &userData)
return userData.Data.ID
}
This works and prints out 1841. I then wanted to write some tests that validate that the code is behaving as expected, e.g. that it correctly fails if an error is returned, that the data returned can be unmarshalled. I have been reading online and looking at examples but they are all far more complex that what I feel I am trying to achieve.
I have started with the following test that ensures that the data passed to the unmarshallData function can be unmarshalled:
package main
import (
"testing"
)
func Test_unmarshallData(t *testing.T) {
type args struct {
body []byte
}
tests := []struct {
name string
args args
want int
}{
{name: "Unmarshall", args: struct{ body []byte }{body: []byte("{\"meta\":null,\"data\":{\"id\":1841,\"name\":\"Piya\",\"email\":\"priya#gmai.com\",\"gender\":\"female\",\"status\":\"active\"}}")}, want: 1841},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := unmarshallData(tt.args.body); got != tt.want {
t.Errorf("unmarshallData() = %v, want %v", got, tt.want)
}
})
}
}
Any advise on where to go from here would be appreciated.
before moving on to the testing, your code has a serious flow, which will become a problem if you don't take care about it in your future programming tasks.
https://pkg.go.dev/net/http See the second example
The client must close the response body when finished with it
Let's fix that now (we will have to come back on this subject later), two possibilities.
1/ within main, use defer to Close that resource after you have drained it;
func main() {
resp := sendRequest()
defer body.Close()
body := readBody(resp)
id := unmarshallData(body)
fmt.Println(id)
}
2/ Do that within readBody
func readBody(resp *http.Response) []byte {
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
return body
}
Using a defer is the expected manner to close the resource. It helps the reader to identify the lifetime span of the resource and improve readability.
Notes : I will not be using much of the table test driven pattern, but you should, like you did in your OP.
Moving on to the testing part.
Tests can be written under the same package or its fellow version with a trailing _test, such as [package target]_test. This has implications in two ways.
Using a separate package, they will be ignored in the final build. Which will help to produce smaller binaries.
Using a separate package, you test the API in a black box manner, you can access only the identifiers it explicitly exposes.
Your current tests are white boxed, meaning you can access any declaration of main, public or not.
About sendRequest, writing a test around this is not very interesting because it does too little, and your tests should not be written to test the std library.
But for the sake of the demonstration, and for good reasons we might want to not rely on external resources to execute our tests.
In order to achieve that we must make the global dependencies consumed within it, an injected dependency. So that later on, it is possible to replace the one thing it depends on to react, the http.Get method.
func sendRequest(client interface{Get() (*http.Response, error)}) *http.Response {
resp, err := client.Get("https://gorest.co.in/public/v1/users/1841")
if err != nil {
log.Fatalln(err)
}
return resp
}
Here i use an inlined interface declaration interface{Get() (*http.Response, error)}.
Now we can add a new test which injects a piece of code that will return exactly the values that will trigger the behavior we want to test within our code.
type fakeGetter struct {
resp *http.Response
err error
}
func (f fakeGetter) Get(u string) (*http.Response, error) {
return f.resp, f.err
}
func TestSendRequestReturnsNilResponseOnError(t *testing.T) {
c := fakeGetter{
err: fmt.Errorf("whatever error will do"),
}
resp := sendRequest(c)
if resp != nil {
t.Fatal("it should return a nil response when an error arises")
}
}
Now run this test and see the result. It is not conclusive because your function contains a call to log.Fatal, which in turns executes an os.Exit; We cannot test that.
If we try to change that, we might think we might call for panic instead because we can recover.
I don't recommend doing that, in my opinion, this is smelly and bad, but it exists, so we might consider. This is also the least possible change to the function signature. Returning an error would break even more the current signatures. I want to minimize this for that demonstration. But, as a rule of thumb, return an error and always check them.
In the sendRequest function, replace this call log.Fatalln(err) with panic(err) and update the test to capture the panic.
func TestSendRequestReturnsNilResponseOnError(t *testing.T) {
var hasPanicked bool
defer func() {
_ = recover() // if you capture the output value or recover, you get the error gave to the panic call. We have no use of it.
hasPanicked = true
}()
c := fakeGetter{
err: fmt.Errorf("whatever error will do"),
}
resp := sendRequest(c)
if resp != nil {
t.Fatal("it should return a nil response when an error arises")
}
if !hasPanicked {
t.Fatal("it should have panicked")
}
}
We can now move on to the other execution path, the non error return.
For that we forge the desired *http.Response instance we want to pass into our function, we will then check its properties to figure out if what the function does is inline with what we expect.
We will consider we want to ensure it is returned unmodified : /
Below test only sets two properties, and I will do it to demonstrate how to set the Body with a NopCloser and strings.NewReader as it is often needed later on using the Go language;
I also use reflect.DeepEqual as brute force equality checker, usually you can be more fine grained and get better tests. DeepEqual does the job in this case but it introduces complexity that does not justify systematic use of it.
func TestSendRequestReturnsUnmodifiedResponse(t *testing.T) {
c := fakeGetter{
err: nil,
resp: &http.Response{
Status: http.StatusOK,
Body: ioutil.NopCloser(strings.NewReader("some text")),
},
}
resp := sendRequest(c)
if !reflect.DeepEqual(resp, c.resp) {
t.Fatal("the response should not have been modified")
}
}
At that point you may have figured that this small function sendRequest is not good, if you did not I ensure you it is not. It does too little, it merely wraps the http.Get method and its testing is of little interest for the survival of the business logic.
Moving on to readBody function.
All remarks that applied for sendRequest apply here too.
it does too little
it os.Exits
One thing does not apply. As the call to ioutil.ReadAll does not rely on external resources, there is no point in attempting to inject that dependency. We can test around.
Though, for the sake of the demonstration, it is the time to talk about the missing call to defer resp.Body.Close().
Let us assume we go for the second proposition made in introduction and test for that.
The http.Response struct adequately exposes its Body recipient as an interface.
To ensure the code calls for the `Close, we can write a stub for it.
That stub will record if that call was made, the test can then check for that and trigger an error if it was not.
type closeCallRecorder struct {
hasClosed bool
}
func (c *closeCallRecorder) Close() error {
c.hasClosed = true
return nil
}
func (c *closeCallRecorder) Read(p []byte) (int, error) {
return 0, nil
}
func TestReadBodyCallsClose(t *testing.T) {
body := &closeCallRecorder{}
res := &http.Response{
Body: body,
}
_ = readBody(res)
if !body.hasClosed {
t.Fatal("the response body was not closed")
}
}
Similarly, and for the sake of the demonstration, we might want to test if the function has called for Read.
type readCallRecorder struct {
hasRead bool
}
func (c *readCallRecorder) Read(p []byte) (int, error) {
c.hasRead = true
return 0, nil
}
func TestReadBodyHasReadAnything(t *testing.T) {
body := &readCallRecorder{}
res := &http.Response{
Body: ioutil.NopCloser(body),
}
_ = readBody(res)
if !body.hasRead {
t.Fatal("the response body was not read")
}
}
We an also verify the body was not modified in betwen,
func TestReadBodyDidNotModifyTheResponse(t *testing.T) {
want := "this"
res := &http.Response{
Body: ioutil.NopCloser(strings.NewReader(want)),
}
resp := readBody(res)
if got := string(resp); want != got {
t.Fatal("invalid response, wanted=%q got %q", want, got)
}
}
We have almost done, lets move one to the unmarshallData function.
You have already wrote a test about it. It is okish, though, i would write it this way to make it leaner:
type UserData struct {
Meta interface{} `json:"meta"`
Data Data `json:"data"`
}
type Data struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Gender string `json:"gender"`
Status string `json:"status"`
}
func Test_unmarshallData(t *testing.T) {
type args struct {
body []byte
}
tests := []UserData{
UserData{Data: Data{ID: 1841}},
}
for _, u := range tests {
want := u.ID
b, _ := json.Marshal(u)
t.Run("Unmarshal", func(t *testing.T) {
if got := unmarshallData(b); got != want {
t.Errorf("unmarshallData() = %v, want %v", got, want)
}
})
}
}
Then, the usual apply :
don't log.Fatal
what are you testing ? the marshaller ?
Finally, now that we have gathered all those pieces, we can refactor to write a more sensible function and re use all those pieces to help us testing such code.
I won't do it, but here is a starter, which still panics, and I still don't recommend, but the previous demonstration has shown everything needed to test a version of it that returns an error.
type userFetcher struct {
Requester interface {
Get(u string) (*http.Response, error)
}
}
func (u userFetcher) Fetch() int {
resp, err := u.Requester.Get("https://gorest.co.in/public/v1/users/1841") // it does not really matter that this string is static, using the requester we can mock the response, its body and the error.
if err != nil {
panic(err)
}
defer resp.Body.Close() //always.
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
var userData UserData
err = json.Unmarshal(body, &userData)
if err != nil {
panic(err)
}
return userData.Data.ID
}

How to pass byte slice between go routines using channels

I have a function that reads data from a source and send them to destination. Source and destination could be anything, lets say for this example source is database (any MySQL, PostgreSQL...) and destination is distributed Q (any... ActiveMQ, Kafka). Messages are stored in bytes.
This is main function. idea is it will spin a new go routine and will wait for messages to be returned for future processing.
type Message []byte
func (p *ProcessorService) Continue(dictId int) {
level.Info(p.logger).Log("process", "message", "dictId", dictId)
retrieved := make(chan Message)
go func() {
err := p.src.Read(retrieved, strconv.Itoa(p.dictId))
if err != nil {
level.Error(p.logger).Log("process", "read", "message", "err", err)
}
}()
for r := range retrieved {
go func(message Message) {
level.Info(p.logger).Log("message", message)
if len(message) > 0 {
if err := p.dst.sendToQ(message); err != nil {
level.Error(p.logger).Log("failed", "during", "persist", "err", err)
}
} else {
level.Error(p.logger).Log("failed")
}
}(r)
}
}
and this is read function itself
func (s *Storage) Read(out chan<- Message, opt ...string) error {
// I just skip some basic database read operations here
// but idea is simple, read data from the table / file row by row and
//
for _, value := range dataFromDB {
message, err := value.row
if err == nil {
out <- message
} else {
errorf("Unable to get data %v", err)
out <- make([]byte, 0)
}
}
})
close(out)
if err != nil {
return err
}
return nil
}
As you can see communication done via out chan<- Message channel.
My concern in Continue function, specifically here
for r := range retrieved {
go func(message Message) {
// basically here message and r are pointing to the same underlying array
}
}
When data received var r is a type of slice byte. Then it passed to go func(message Message) everything passed by value in go, in this case var r will be passed as copy to anonymous func, however it will still have a pointer to underlying slice data. I am curious if it could be a problem during p.dst.sendToQ(message); execution and at the same time read function will send something to out channel causing slice data structure to be overridden with a new information. Should I copy byte slice r into the new byte slice before passing to anonymous function, so underlying arrays will be different? I tested it, but couldn't really cause this behavior. Not sure if I am paranoid or have to worry about it.
The message in p.dst.sendToQ(message) is the same slice as value.row when you get data from the db. So, as long as each value.row has a different underlying array, you should be good. So, I suggest you check the source and make sure it does not use a common byte array and keeps rewriting to it.

Combine multiple error strings

I am new to golang, my application needs to return multiple errors in a loop, later requires to be combined and returned as a single error string. I am not able to use the string functions to combine the error messages. What methods can be use to combine these errors into a single error before returning ?
package main
import (
"fmt"
"strings"
)
func Servreturn() (err error) {
err1 = fmt.Errorf("Something else occured")
err2 = fmt.Errorf("Something else occured again")
// concatenate both the error
return err3
}
UPDATE for Go 1.13:
As of Go version 1.13, the language's errors package now supports error wrapping directly.
You can wrap an error by using the %w verb in fmt.Errorf:
err := errors.New("Original error")
err = fmt.Errorf("%w; Second error", err)
Use Unwrap to remove the last error added, and return what remains: previousErrors := errors.Unwrap(err)
Playground Example for errors.Unwrap
Two more functions, errors.Is and errors.As provide ways to check for and retrieve a specific type of error.
Playground Example for errors.As and errors.Is
Dave Cheney's excellent errors package (https://github.com/pkg/errors) include a Wrap function for this purpose:
package main
import "fmt"
import "github.com/pkg/errors"
func main() {
err := errors.New("error")
err = errors.Wrap(err, "open failed")
err = errors.Wrap(err, "read config failed")
fmt.Println(err) // "read config failed: open failed: error"
}
This also allows additional functionality, such as unpacking the cause of the error:
package main
import "fmt"
import "github.com/pkg/errors"
func main() {
err := errors.New("original error")
err = errors.Wrap(err, "now this")
fmt.Println(errors.Cause(err)) // "original error"
}
As well as the option to output a stack trace when specifying fmt.Printf("%+v\n", err).
You can find additional information about the package on his blog: here and here.
String functions don't work on errors because error is really an interface that implements the function Error() string.
You can use string functions on err1.Error() and err2.Error()
but not on the "err1" reference itself.
Some errors are structs, like the ones you get from database drivers.
So there's no natural way to use string functions on errors since they may not actually be strings underneath.
As for combining two errors:
Easy, just use fmt.Errorf again.
fmt.Errorf("Combined error: %v %v", err1, err2)
Alternatively:
errors.New(err1.Error() + err2.Error())
You could use the strings.Join() and append() function to acheive this slice.
example: golang playgorund
package main
import (
"fmt"
"strings"
"syscall"
)
func main() {
// create a slice for the errors
var errstrings []string
// first error
err1 := fmt.Errorf("First error:server error")
errstrings = append(errstrings, err1.Error())
// do something
err2 := fmt.Errorf("Second error:%s", syscall.ENOPKG.Error())
errstrings = append(errstrings, err2.Error())
// do something else
err3 := fmt.Errorf("Third error:%s", syscall.ENOTCONN.Error())
errstrings = append(errstrings, err3.Error())
// combine and print all the error
fmt.Println(fmt.Errorf(strings.Join(errstrings, "\n")))
}
This would output a single string which you can send back to the client.
First error:server1
Second error:Package not installed
Third error:Socket is not connected
hope this helps!
To expand on what #WillC had mentioned in a comment it is possible to define your own error type as error is an interface type. Any type that implements a Error() string function implements the error interface. Therefore, you could create a CollectionError which aggregates errors and returns a concatenated error string.
type ErrorCollector []error
func (c *ErrorCollector) Collect(e error) { *c = append(*c, e) }
func (c *ErrorCollector) Error() (err string) {
err = "Collected errors:\n"
for i, e := range *c {
err += fmt.Sprintf("\tError %d: %s\n", i, e.Error())
}
return err
}
This provides a collection function that appends a given error to a slice. Upon calling Error() string it iterates over the slice and creates a concatenated error string.
func main() {
collector := new(ErrorCollector)
for i := 0; i < 10; i++ {
collector.Collect(errors.New(fmt.Sprintf("%d Error", i)))
}
fmt.Println(collector)
}
There is a great golang.org blog post going over errors in more detail. A full example of the example is available on The Go Playground.
Uber has a multierr package for this use case:
return multierr.Combine(err1, err2)
People may be interested in https://github.com/hashicorp/go-multierror which describes itself as "A Go (golang) package for representing a list of errors as a single error.".
As of Go 1.20, we'll be able to wrap multiple errors using errors.Join.
See this proposal for more details.
First Option
You can print the errors; they will be separated by a newline character when you do that.
var (
ErrIncorrectUsername = errors.New("incorrect username")
ErrIncorrectPassword = errors.New("incorrect password")
)
func main() {
err := validate("ruster", "4321")
// You can print multi-line errors
// Each will be separated by a newline character (\n).
if err != nil {
fmt.Println(err)
// incorrect username
// incorrect password
}
}
func validate(username, password string) error {
var errs []error
// errors.Join the errors into a single error
if username != "gopher" {
errs = append(errs, ErrIncorrectUsername)
}
if password != "1234" {
errs = append(errs, ErrIncorrectPassword)
}
// Join returns a single `error`.
// Underlying, the error contains all the errors we add.
return errors.Join(errs...)
}
Second Option
errors.Join returns an error that contains each error you add. So you can use errors.Is and errors.As to check for individual errors for finer granularity.
// ...
func main() {
err := validate("ruster", "4321")
// You can detect each one:
if errors.Is(err, ErrIncorrectUsername) {
// handle the error here
}
// Or detect the other one:
if errors.Is(err, ErrIncorrectPassword) {
// handle the error here
}
}
func validate(username, password string) error {
// ...
}
Note: This naive validate example is here to convey the idea. Instead of chaining errors, think errors like a tree. Join allows you to do that when combined with other Joins.
Run both on Go Playground.
This seems to work well for me (space separated errors):
Put all your errors in a error slice/list/array, ie: var errors [] error
fmt.Sprintf("%s", errors)
var errors []error
errors = append(errors, fmt.Errorf("error 1"))
errors = append(errors, fmt.Errorf("error 2"))
errors = append(errors, fmt.Errorf("and yet another error"))
s := fmt.Sprintf("%s", errors)
fmt.Println(s)
Sometimes i need the way to detect if there some error in the chain. the standard way, provided with https://pkg.go.dev/errors is pretty convenient:
someErr:=errors.New("my error")
fmt.Errorf("can't process request: %w",someErr)
...
err:=f()
if errors.Is(err,someErr){...}
But it could be applied only in the case of detection of the last error in the chain. You can't wrap someErr1 and then someErr2 and then get true from both of checks: errors.Is(err,someErr1) and errors.Is(err,someErr2)
I solved this problem with following type:
func NewJoinedErrors(err1 error, err2 error) JoinedErrors {
return JoinedErrors{err1: err1, err2: err2}
}
type JoinedErrors struct {
err1 error
err2 error
}
func (e JoinedErrors) Error() string {
return fmt.Sprintf("%s: %s", e.err1, e.err2)
}
func (e JoinedErrors) Unwrap() error {
return e.err2
}
func (e JoinedErrors) Is(target error) bool {
return errors.Is(e.err1, target)
}
It uses the fact, that
An error is considered to match a target if it is equal to that target
or if it implements a method Is(error) bool such that Is(target)
returns true.
So you can join two errors and get positive result on both checks:
someErr1:=errors.New("my error 1")
someErr2:=errors.New("my error 2")
err:=NewJoinedErrors(someErr1, someErr2)
// this will be true because
// (e JoinedErrors) Is(target error)
// will return true
if errors.Is(err, someErr1){...}
// this will be true because
// (e JoinedErrors) Unwrap() error
// will return err2
if errors.Is(err, someErr2){...}
you can check it out here: https://play.golang.org/p/W7NGyfvr0v_N
func condenseErrors(errs []error) error {
switch len(errs) {
case 0:
return nil
case 1:
return errs[0]
}
err := errs[0]
for _, e := range errs[1:] {
err = errors.Wrap(err, e.Error())
}
return err
}
Use this function:
func JoinErrs(errs ...error) error {
var joinErrsR func(string, int, ...error) error
joinErrsR = func(soFar string, count int, errs ...error) error {
if len(errs) == 0 {
if count == 0 {
return nil
}
return fmt.Errorf(soFar)
}
current := errs[0]
next := errs[1:]
if current == nil {
return joinErrsR(soFar, count, next...)
}
count++
if count == 1 {
return joinErrsR(fmt.Sprintf("%s", current), count, next...)
} else if count == 2 {
return joinErrsR(fmt.Sprintf("1: %s\n2: %s", soFar, current), count, next...)
}
return joinErrsR(fmt.Sprintf("%s\n%d: %s", soFar, count, current), count, next...)
}
return joinErrsR("", 0, errs...)
}
It will give you nil when all errors are nil, will give you the same error when just one non-nil error, will give you numbered list of non-nil errors when multiple non-nil errors
Try it out here: https://play.golang.org/p/WttztCr-xHG

How to handle HTTP timeout errors and accessing status codes in golang

I have some code (see below) written in Go which is supposed to "fan-out" HTTP requests, and collate/aggregate the details back.
I'm new to golang and so expect me to be a nOOb and my knowledge to be limited
The output of the program is currently something like:
{
"Status":"success",
"Components":[
{"Id":"foo","Status":200,"Body":"..."},
{"Id":"bar","Status":200,"Body":"..."},
{"Id":"baz","Status":404,"Body":"..."},
...
]
}
There is a local server running that is purposely slow (sleeps for 5 seconds and then returns a response). But I have other sites listed (see code below) that sometime trigger an error as well (if they error, then that's fine).
The problem I have at the moment is how best to handle these errors, and specifically the "timeout" related errors; in that I'm not sure how to recognise if a failure is a timeout or some other error?
At the moment I get a blanket error back all the time:
Get http://localhost:8080/pugs: read tcp 127.0.0.1:8080: use of closed network connection
Where http://localhost:8080/pugs will generally be the url that failed (hopefully by timeout!). But as you can see from the code (below), I'm not sure how to determine the error code is related to a timeout nor how to access the status code of the response (I'm currently just blanket setting it to 404 but obviously that's not right - if the server was to error I'd expect something like a 500 status code and obviously I'd like to reflect that in the aggregated response I send back).
The full code can be seen below. Any help appreciated.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"sync"
"time"
)
type Component struct {
Id string `json:"id"`
Url string `json:"url"`
}
type ComponentsList struct {
Components []Component `json:"components"`
}
type ComponentResponse struct {
Id string
Status int
Body string
}
type Result struct {
Status string
Components []ComponentResponse
}
var overallStatus string = "success"
func main() {
var cr []ComponentResponse
var c ComponentsList
b := []byte(`{"components":[{"id":"local","url":"http://localhost:8080/pugs"},{"id":"google","url":"http://google.com/"},{"id":"integralist","url":"http://integralist.co.uk/"},{"id":"sloooow","url":"http://stevesouders.com/cuzillion/?c0=hj1hfff30_5_f&t=1439194716962"}]}`)
json.Unmarshal(b, &c)
var wg sync.WaitGroup
timeout := time.Duration(1 * time.Second)
client := http.Client{
Timeout: timeout,
}
for i, v := range c.Components {
wg.Add(1)
go func(i int, v Component) {
defer wg.Done()
resp, err := client.Get(v.Url)
if err != nil {
fmt.Printf("Problem getting the response: %s\n", err)
cr = append(cr, ComponentResponse{
v.Id,
404,
err.Error(),
})
} else {
defer resp.Body.Close()
contents, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Printf("Problem reading the body: %s\n", err)
}
cr = append(cr, ComponentResponse{
v.Id,
resp.StatusCode,
string(contents),
})
}
}(i, v)
}
wg.Wait()
j, err := json.Marshal(Result{overallStatus, cr})
if err != nil {
fmt.Printf("Problem converting to JSON: %s\n", err)
return
}
fmt.Println(string(j))
}
If you want to fan out then aggregate results and you want specific timeout behavior the net/http package isn't giving you, then you may want to use goroutines and channels.
I just watched this video today and it will walk you through exactly those scenarios using the concurrency features of Go. Plus, the speaker Rob Pike is quite the authority -- he explains it much better than I could.
https://www.youtube.com/watch?v=f6kdp27TYZs
I am adding this for completes, as the correct answer was provided by Dave C in the comments of the accepted answer.
We can try to cast the error to a net.Error and check if it is a timeout.
resp, err := client.Get(url)
if err != nil {
// if there is an error check if its a timeout error
if e, ok := err.(net.Error); ok && e.Timeout() {
// handle timeout
return
}
// otherwise handle other types of error
}
The Go 1.5 release solved this issue by being more specific about the type of error it has handled.
So if you see this example https://github.com/Integralist/Go-Requester/blob/master/requester.go#L38 you'll see that I'm able to apply a regex pattern to the error message to decipher if the error was indeed a timeout or not
status := checkError(err.Error())
func checkError(msg string) int {
timeout, _ := regexp.MatchString("Timeout", msg)
if timeout {
return 408
}
return 500
}

Is it possible to "return" a function from another function?

Due the verbose error handling syntax I've created a function check as below which acts as a "global" error handler. If I want to panic instead to log I change only the check function. The issue is that now I want to simply return so that the other go routines can still run if one has an error. So the question is: How can I do that ? Is it possible?
func main() {
for k, v := range foo {
go func() {
err = doSomething()
check("this one failed", err)
}()
}
}
func check(errMsg string, err error) {
if err != nil {
log.Fatalf(errMsg, err)
}
}
However now I've found that I need return the anonymous function if there is any error rather than exit ( log.Fatal ) so I'm wondering if it's possible to return the anyonymou
You could make your check function returns a bool:
func checkIfFailed(errMsg string, err error) bool {
if err != nil {
log.Printf(errMsg, err)
return true
}
return false
}
That way, you can still call your check (and do all kind of checks in it), while returning early from the anonymous function:
err = doSomething()
if checkIfFailed("this one failed", err) {
return
}
There isn't a language feature that allows you to cause the parent function to automatically return in response to a simple function call.
However, there is a way to cause the current goroutine to exit, which might be what you are after: runtime.Goexit. Note that this has similar disruptive potential to calling os.Exit, so it would be bad to call it in the context of goroutines created by other packages or other unrelated code.

Resources