I've constructed a simple script which goes on a website, and does a few things on there (automating a checkout process on lego.com)
This is all done using the go http client with a cookiejar on it, but when I try to print out the cookies after all the activity on the site, nothing prints - not sure if I'm doing something wrong here.
type Program struct {
taskInfo task.Task
client http.Client
jar *cookiejar.Jar
// Task Specific Variables
maxOrderQty string
sessionID string
sku string
shipMethodUID string
}
The HTTP client is initialized below
func (pr *Program) initializeClient() {
pr.jar, _ = cookiejar.New(nil)
pr.client = http.Client{
Timeout: time.Second * 10,
Jar: pr.jar,
}
}
After that, i make a few calls to the same domain (mixture of GET and POST) using said client. When I try to print out the cookies, nothing prints.
func (pr *Program) getSessionCookies() {
log.Debug("Getting Cookies")
u, _ := url.Parse("https://www.lego.com/")
for _, cookie := range pr.jar.Cookies(u) {
fmt.Printf(" %s: %s\n", cookie.Name, cookie.Value)
}
}
Go's http package support redirects by default, but the cookie jar must be refreshed, specially if the domain changes.
I spent a great deal of time debugging it some time ago, the solution I found was to specify a redirect handler:
client.CheckRedirect = getRedirectHandler(req, &client)
Which had the following code:
func getRedirectHandler(req *Request, client *http.Client) redirectHandler {
fn := func(request *http.Request, via []*http.Request) error {
if req.FollowRedirect {
log.Debug(fmt.Sprintf("Redirecting to %s", request.URL.String()))
jar, err := getCookieJarWithURL(request.URL.String(), req.Cookies)
if err != nil {
return err
}
client.Jar = jar
return nil
}
return errors.New("Redirect not allowed")
}
return fn
}
The full code can be found in this gist:
I hope that saves you some time :)
Related
i receive a response body "bad request" with "httptest.Client().Postform"
type testServer struct {
*httptest.Server
}
func newTestServer(t *testing.T, h http.Handler) *testServer {
ts := httptest.NewTLSServer(h)
jar, err := cookiejar.New(nil)
if err != nil {
t.Fatal(err)
}
ts.Client().Jar = jar
ts.Client().CheckRedirect = func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
return &testServer{ts}
}
func (ts *testServer) postForm(t *testing.T, urlPath string, form url.Values) (int, http.Header, string) {
rs, err := ts.Client().PostForm(ts.URL+urlPath, form)
if err != nil {
t.Fatal(err)
}
defer rs.Body.Close()
body, err := io.ReadAll(rs.Body)
if err != nil {
t.Fatal(err)
}
bytes.TrimSpace(body)
return rs.StatusCode, rs.Header, string(body)
}
I don't know where is the problem, i have also verified the url it's correct.
Always badrequest with POST but with GET request it's works fine.
this is the handler object :
func (app *application) routes() http.Handler {
router := httprouter.New()
router.NotFound = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
app.notFound(w)
})
dynamic := alice.New(app.sessionManager.LoadAndSave, noSurf, app.Authenticated)
router.Handler(http.MethodGet, "/", dynamic.ThenFunc(app.home))
router.Handler(http.MethodGet, "/user/signup", dynamic.ThenFunc(app.userSignup))
router.Handler(http.MethodPost, "/user/signup", dynamic.ThenFunc(app.userSignupPost))
standart := alice.New(app.recoverPanic, app.logRequest, securityHeaders)
return standart.Then(router)
}
the test function :https://go.dev/play/p/k45-JYTYCOS
the app.userSignupPost:
func (app *application) userSignupPost(w http.ResponseWriter, r *http.Request) {
var form userSignupForm
err := app.decodPostForm(r, &form)
if err != nil {
app.clientError(w, http.StatusBadRequest)
return
}
form.CheckField(validator.NotBlank(form.Name), "name", "this field must not be blank")
form.CheckField(validator.NotBlank(form.Email), "email", "this field must not be blank")
form.CheckField(validator.Matches(form.Email, validator.EmailRX), "email", "this field must be a valid email address")
form.CheckField(validator.NotBlank(form.Password), "password", "this field must not be blank")
form.CheckField(validator.MinChars(form.Password, 8), "password", "password must bee at least 8 caracter long")
if !form.Valid() {
data := app.newTemplateData(r)
data.Form = form
app.render(w, http.StatusUnprocessableEntity, "signup.tmpl.html", data)
return
}
err = app.users.Insert(form.Name, form.Email, form.Password)
if err != nil {
if errors.Is(err, models.ErrDuplicateEmail) {
form.AddFieldError("email", "Email already exist")
data := app.newTemplateData(r)
data.Form = form
app.render(w, http.StatusUnprocessableEntity, "signup.tmpl.html", data)
} else {
fmt.Println("error user postform")
app.serverError(w, err)
}
return
}
app.sessionManager.Put(r.Context(), "flash", "Signup Successful. Please log in")
http.Redirect(w, r, "/user/login", http.StatusSeeOther)
}
It appears that you're using https://github.com/justinas/alice to register handlers - you don't want to do this. That package is for middleware chaining - e.g. "before all requests to this URL, first authenticate the request" - you'd put the authentication into a middleware function and then add it to the chain.
So every POST /user/signup request is getting passed first to app.userSignup() (what you are using to handle GET requests). This is because calling alice.ThenFunc() appends the passed handler to the chain and then returns the entire chain as a handler - you need to read this part of the Alice docs carefully before using it.
Replace this line:
router.Handler(http.MethodPost, "/user/signup", dynamic.ThenFunc(app.userSignupPost))
with
router.Handler(http.MethodPost, "/user/signup", http.HandlerFunc(app.userSignupPost))
You may not need the additional decoration of http.HandlerFunc() - try it with and without to see what works. I cannot say for sure without knowing what the body of app.userSignupPost() looks like (same for the other handler functions as well).
You'll then need to do the same for the other handler registration lines - you shouldn't be using middleware chaining for your end handlers. An http.Handler is used for saying, "send any request to path /PP/ppp with method XXXX to this function." Middleware chaining is for preprocessing (authentication, authorization, etc.) - a whole host of things can be done there, but end request handling shouldn't be one of them.
I'm still curious if your use of PostForm() is going to cause you issues for the reason I cited in my comment on your question - try a raw Post() and see if the behavior differs, but after refactoring to take out the alice goop (at least temporarily). When testing a handler, I'd start off with a much more minimal approach - test that the handler itself works before muddying the waters with both alice and what looks like this package.
I think i found the problem , the session cookie are not the same for get and post request. i don't know why it has changed.They use the same http.Client()
I'm building an API that scrapes some data off a webpage.
To do so, i need to send a GET request to a home page, scrape a 'RequestVerificationToken' from the HTML, then send another POST request to the same URL with a username, password, and the RequestVerificationToken.
I've been able to do this previously with Python:
session_requests = requests.session()
result = session_requests.get(LOGIN_URL)
parser = createBS4Parser(result.text)
return parser.find('input', attrs={'name': '__RequestVerificationToken'})["value"]
pageDOM = session_requests.post(
LOGIN_URL,
data=requestPayload, //RequestVerificationToken is in here
headers=requestHeaders
)
It seems like when i reuse the session_requests variable in Python, it's reusing the previous instance of the HTTP request.
However, when i try to do this in Go, I get an error due to an invalid token. I assume that this is because for the POST request, Go is using a new instance.
Is there any way I can get the same behavior from Go as I was with Python?
package main
import (
"fmt"
"log"
"github.com/gocolly/colly"
"github.com/gocolly/colly/proxy"
)
func main() {
//initiates the configuration
c := colly.NewCollector(colly.AllowURLRevisit())
//defining the proxy chain
revpro, err := proxy.RoundRobinProxySwitcher("socks5://127.0.0.1:9050", "socks5://127.0.0.1:9050")
if err != nil {
log.Fatal(err)
}
c.SetProxyFunc(revpro)
//parsing the required field from html we are extracting the csrf_token required for the login
c.OnHTML("form[role=form] input[type=hidden][name=CSRF_TOKEN]", func(e *colly.HTMLElement) {
csrftok := e.Attr("value")
fmt.Println(csrftok)
//posting the csrf value along with password
err := c.Post("https://www.something.com/login.jsp", map[string]string{"CSRF_TOKEN": csrftok, "username": "username", "password": "password"})
if err != nil {
log.Fatal(err)
}
return
})
//The website to visit
c.Visit("https://www.something.com/login.jsp")
//maintaining the connection using clone not initiating a callback request
d := c.Clone()
d.OnHTML("a[href]", func(e *colly.HTMLElement) {
link := e.Attr("href")
fmt.Printf("Link found: %q -> %s\n", e.Text, link)
})
d.Visit("https://skkskskskk.htm")
}
I am leaning to write unit tests and I was wondering the correct way to unit test a basic http.get request.
I found an API online that returns fake data and wrote a basic program that gets some user data and prints out an ID:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
)
type UserData struct {
Meta interface{} `json:"meta"`
Data struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Gender string `json:"gender"`
Status string `json:"status"`
} `json:"data"`
}
func main() {
resp := sendRequest()
body := readBody(resp)
id := unmarshallData(body)
fmt.Println(id)
}
func sendRequest() *http.Response {
resp, err := http.Get("https://gorest.co.in/public/v1/users/1841")
if err != nil {
log.Fatalln(err)
}
return resp
}
func readBody(resp *http.Response) []byte {
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
return body
}
func unmarshallData(body []byte) int {
var userData UserData
json.Unmarshal(body, &userData)
return userData.Data.ID
}
This works and prints out 1841. I then wanted to write some tests that validate that the code is behaving as expected, e.g. that it correctly fails if an error is returned, that the data returned can be unmarshalled. I have been reading online and looking at examples but they are all far more complex that what I feel I am trying to achieve.
I have started with the following test that ensures that the data passed to the unmarshallData function can be unmarshalled:
package main
import (
"testing"
)
func Test_unmarshallData(t *testing.T) {
type args struct {
body []byte
}
tests := []struct {
name string
args args
want int
}{
{name: "Unmarshall", args: struct{ body []byte }{body: []byte("{\"meta\":null,\"data\":{\"id\":1841,\"name\":\"Piya\",\"email\":\"priya#gmai.com\",\"gender\":\"female\",\"status\":\"active\"}}")}, want: 1841},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := unmarshallData(tt.args.body); got != tt.want {
t.Errorf("unmarshallData() = %v, want %v", got, tt.want)
}
})
}
}
Any advise on where to go from here would be appreciated.
before moving on to the testing, your code has a serious flow, which will become a problem if you don't take care about it in your future programming tasks.
https://pkg.go.dev/net/http See the second example
The client must close the response body when finished with it
Let's fix that now (we will have to come back on this subject later), two possibilities.
1/ within main, use defer to Close that resource after you have drained it;
func main() {
resp := sendRequest()
defer body.Close()
body := readBody(resp)
id := unmarshallData(body)
fmt.Println(id)
}
2/ Do that within readBody
func readBody(resp *http.Response) []byte {
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
return body
}
Using a defer is the expected manner to close the resource. It helps the reader to identify the lifetime span of the resource and improve readability.
Notes : I will not be using much of the table test driven pattern, but you should, like you did in your OP.
Moving on to the testing part.
Tests can be written under the same package or its fellow version with a trailing _test, such as [package target]_test. This has implications in two ways.
Using a separate package, they will be ignored in the final build. Which will help to produce smaller binaries.
Using a separate package, you test the API in a black box manner, you can access only the identifiers it explicitly exposes.
Your current tests are white boxed, meaning you can access any declaration of main, public or not.
About sendRequest, writing a test around this is not very interesting because it does too little, and your tests should not be written to test the std library.
But for the sake of the demonstration, and for good reasons we might want to not rely on external resources to execute our tests.
In order to achieve that we must make the global dependencies consumed within it, an injected dependency. So that later on, it is possible to replace the one thing it depends on to react, the http.Get method.
func sendRequest(client interface{Get() (*http.Response, error)}) *http.Response {
resp, err := client.Get("https://gorest.co.in/public/v1/users/1841")
if err != nil {
log.Fatalln(err)
}
return resp
}
Here i use an inlined interface declaration interface{Get() (*http.Response, error)}.
Now we can add a new test which injects a piece of code that will return exactly the values that will trigger the behavior we want to test within our code.
type fakeGetter struct {
resp *http.Response
err error
}
func (f fakeGetter) Get(u string) (*http.Response, error) {
return f.resp, f.err
}
func TestSendRequestReturnsNilResponseOnError(t *testing.T) {
c := fakeGetter{
err: fmt.Errorf("whatever error will do"),
}
resp := sendRequest(c)
if resp != nil {
t.Fatal("it should return a nil response when an error arises")
}
}
Now run this test and see the result. It is not conclusive because your function contains a call to log.Fatal, which in turns executes an os.Exit; We cannot test that.
If we try to change that, we might think we might call for panic instead because we can recover.
I don't recommend doing that, in my opinion, this is smelly and bad, but it exists, so we might consider. This is also the least possible change to the function signature. Returning an error would break even more the current signatures. I want to minimize this for that demonstration. But, as a rule of thumb, return an error and always check them.
In the sendRequest function, replace this call log.Fatalln(err) with panic(err) and update the test to capture the panic.
func TestSendRequestReturnsNilResponseOnError(t *testing.T) {
var hasPanicked bool
defer func() {
_ = recover() // if you capture the output value or recover, you get the error gave to the panic call. We have no use of it.
hasPanicked = true
}()
c := fakeGetter{
err: fmt.Errorf("whatever error will do"),
}
resp := sendRequest(c)
if resp != nil {
t.Fatal("it should return a nil response when an error arises")
}
if !hasPanicked {
t.Fatal("it should have panicked")
}
}
We can now move on to the other execution path, the non error return.
For that we forge the desired *http.Response instance we want to pass into our function, we will then check its properties to figure out if what the function does is inline with what we expect.
We will consider we want to ensure it is returned unmodified : /
Below test only sets two properties, and I will do it to demonstrate how to set the Body with a NopCloser and strings.NewReader as it is often needed later on using the Go language;
I also use reflect.DeepEqual as brute force equality checker, usually you can be more fine grained and get better tests. DeepEqual does the job in this case but it introduces complexity that does not justify systematic use of it.
func TestSendRequestReturnsUnmodifiedResponse(t *testing.T) {
c := fakeGetter{
err: nil,
resp: &http.Response{
Status: http.StatusOK,
Body: ioutil.NopCloser(strings.NewReader("some text")),
},
}
resp := sendRequest(c)
if !reflect.DeepEqual(resp, c.resp) {
t.Fatal("the response should not have been modified")
}
}
At that point you may have figured that this small function sendRequest is not good, if you did not I ensure you it is not. It does too little, it merely wraps the http.Get method and its testing is of little interest for the survival of the business logic.
Moving on to readBody function.
All remarks that applied for sendRequest apply here too.
it does too little
it os.Exits
One thing does not apply. As the call to ioutil.ReadAll does not rely on external resources, there is no point in attempting to inject that dependency. We can test around.
Though, for the sake of the demonstration, it is the time to talk about the missing call to defer resp.Body.Close().
Let us assume we go for the second proposition made in introduction and test for that.
The http.Response struct adequately exposes its Body recipient as an interface.
To ensure the code calls for the `Close, we can write a stub for it.
That stub will record if that call was made, the test can then check for that and trigger an error if it was not.
type closeCallRecorder struct {
hasClosed bool
}
func (c *closeCallRecorder) Close() error {
c.hasClosed = true
return nil
}
func (c *closeCallRecorder) Read(p []byte) (int, error) {
return 0, nil
}
func TestReadBodyCallsClose(t *testing.T) {
body := &closeCallRecorder{}
res := &http.Response{
Body: body,
}
_ = readBody(res)
if !body.hasClosed {
t.Fatal("the response body was not closed")
}
}
Similarly, and for the sake of the demonstration, we might want to test if the function has called for Read.
type readCallRecorder struct {
hasRead bool
}
func (c *readCallRecorder) Read(p []byte) (int, error) {
c.hasRead = true
return 0, nil
}
func TestReadBodyHasReadAnything(t *testing.T) {
body := &readCallRecorder{}
res := &http.Response{
Body: ioutil.NopCloser(body),
}
_ = readBody(res)
if !body.hasRead {
t.Fatal("the response body was not read")
}
}
We an also verify the body was not modified in betwen,
func TestReadBodyDidNotModifyTheResponse(t *testing.T) {
want := "this"
res := &http.Response{
Body: ioutil.NopCloser(strings.NewReader(want)),
}
resp := readBody(res)
if got := string(resp); want != got {
t.Fatal("invalid response, wanted=%q got %q", want, got)
}
}
We have almost done, lets move one to the unmarshallData function.
You have already wrote a test about it. It is okish, though, i would write it this way to make it leaner:
type UserData struct {
Meta interface{} `json:"meta"`
Data Data `json:"data"`
}
type Data struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Gender string `json:"gender"`
Status string `json:"status"`
}
func Test_unmarshallData(t *testing.T) {
type args struct {
body []byte
}
tests := []UserData{
UserData{Data: Data{ID: 1841}},
}
for _, u := range tests {
want := u.ID
b, _ := json.Marshal(u)
t.Run("Unmarshal", func(t *testing.T) {
if got := unmarshallData(b); got != want {
t.Errorf("unmarshallData() = %v, want %v", got, want)
}
})
}
}
Then, the usual apply :
don't log.Fatal
what are you testing ? the marshaller ?
Finally, now that we have gathered all those pieces, we can refactor to write a more sensible function and re use all those pieces to help us testing such code.
I won't do it, but here is a starter, which still panics, and I still don't recommend, but the previous demonstration has shown everything needed to test a version of it that returns an error.
type userFetcher struct {
Requester interface {
Get(u string) (*http.Response, error)
}
}
func (u userFetcher) Fetch() int {
resp, err := u.Requester.Get("https://gorest.co.in/public/v1/users/1841") // it does not really matter that this string is static, using the requester we can mock the response, its body and the error.
if err != nil {
panic(err)
}
defer resp.Body.Close() //always.
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
var userData UserData
err = json.Unmarshal(body, &userData)
if err != nil {
panic(err)
}
return userData.Data.ID
}
I have been trying to create a grpc client in Go and I have followed the correct instructions as shown in the official grpc site. When I start my grpc server written in node.js, the connection works well but upon compiling the protocol buffer in Go and creating a client interface with the correct grpc client configurations, I run into an error.
Here is my what I have in my identity.pb.go.
type IdentityServiceClient interface {
CreateUser(ctx context.Context, in *GoogleIdToken, opts ...grpc.CallOption) (error, *UserInfo)
}
type simpleServerClient struct {
connection *grpc.ClientConn
}
func NewSimpleServerClient(connection *grpc.ClientConn) IdentityServiceClient {
return &simpleServerClient{connection}
}
func (simpleClient *simpleServerClient) CreateUser(ctx context.Context, in *GoogleIdToken, opts ...grpc.CallOption) (error, *UserInfo) {
out := new(UserInfo)
err := simpleClient.connection.Invoke(ctx, "/protobuf.IdentityService/CreateUser", in, out, opts...)
if err != nil {
return err, nil
}
return nil, out
}
here is the identity.proto
syntax="proto3";
package protobuf;
service IdentityService {
rpc CreateUser (GoogleIdToken) returns (UserInfo) {}
}
message GoogleIdToken {
string token = 1;
}
message UserInfo {
string name = 1;
string email = 2;
message Profile {
string imageUrl = 1;
string lastUpdated = 2;
};
Profile profile = 3;
string token = 4;
}
here is my main.go
import pb "github.com/Duncanian/iam-gateway/server/protobuf"
func grpcConnection() {
conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure())
if err != nil {
log.Fatalf("Failed to start gRPC connection: %v", err)
}
defer conn.Close()
client := pb.NewSimpleServerClient(conn)
err, _ = client.CreateUser(context.Background(), &pb.GoogleIdToken{Token: "tevgvybububvvg"})
if err != nil {
log.Fatalf("Failed to create user: %v", err)
}
log.Println("Created user!")
}
I expect the output of passing a correct google auth token to get me the correct user details which are
name: user,
email: user.email#user.com,
profile: {
imageUrl: myimageUrl,
lastUpdated: mylastUpdatedTime,
},
token,
but got
rpc error: code = Unimplemented desc = RPC method not implemented /protobuf.IdentityService/CreateUser
Here are my github repos:
Go grpc client &
Node JS grpc server
I had the same problem.
Here is my solution:
After compiling .proto I created 2 files: client.go and server.go.
In client.go I implemented my methods (rpc in terms of protobuffers) and had a main function.
In server.go I defined a server struct server with one field: Unimplemented*ServiceName*Server. After that I also implemented the mentioned above methods, but those had a receiver type: func (s *server) Foo(ctx context.Context, *other params*)
That worked for me, hopefully it will help you!
The error indicates that the /protobuf.IdentityService/CreateUser method is not registered at the server side. And I didn't see any service registration code in your linked server code. Please take a look at the node.js guide here.
Using GUI of grpc try to send your protos to your both servers and check for upcoming errors/correct endpoints.
In my case java proto had a package inside which was added to the endpoint.
Had package com.example.grpc;
instead of
option java_package = "com.example.grpc";
I have some code (see below) written in Go which is supposed to "fan-out" HTTP requests, and collate/aggregate the details back.
I'm new to golang and so expect me to be a nOOb and my knowledge to be limited
The output of the program is currently something like:
{
"Status":"success",
"Components":[
{"Id":"foo","Status":200,"Body":"..."},
{"Id":"bar","Status":200,"Body":"..."},
{"Id":"baz","Status":404,"Body":"..."},
...
]
}
There is a local server running that is purposely slow (sleeps for 5 seconds and then returns a response). But I have other sites listed (see code below) that sometime trigger an error as well (if they error, then that's fine).
The problem I have at the moment is how best to handle these errors, and specifically the "timeout" related errors; in that I'm not sure how to recognise if a failure is a timeout or some other error?
At the moment I get a blanket error back all the time:
Get http://localhost:8080/pugs: read tcp 127.0.0.1:8080: use of closed network connection
Where http://localhost:8080/pugs will generally be the url that failed (hopefully by timeout!). But as you can see from the code (below), I'm not sure how to determine the error code is related to a timeout nor how to access the status code of the response (I'm currently just blanket setting it to 404 but obviously that's not right - if the server was to error I'd expect something like a 500 status code and obviously I'd like to reflect that in the aggregated response I send back).
The full code can be seen below. Any help appreciated.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"sync"
"time"
)
type Component struct {
Id string `json:"id"`
Url string `json:"url"`
}
type ComponentsList struct {
Components []Component `json:"components"`
}
type ComponentResponse struct {
Id string
Status int
Body string
}
type Result struct {
Status string
Components []ComponentResponse
}
var overallStatus string = "success"
func main() {
var cr []ComponentResponse
var c ComponentsList
b := []byte(`{"components":[{"id":"local","url":"http://localhost:8080/pugs"},{"id":"google","url":"http://google.com/"},{"id":"integralist","url":"http://integralist.co.uk/"},{"id":"sloooow","url":"http://stevesouders.com/cuzillion/?c0=hj1hfff30_5_f&t=1439194716962"}]}`)
json.Unmarshal(b, &c)
var wg sync.WaitGroup
timeout := time.Duration(1 * time.Second)
client := http.Client{
Timeout: timeout,
}
for i, v := range c.Components {
wg.Add(1)
go func(i int, v Component) {
defer wg.Done()
resp, err := client.Get(v.Url)
if err != nil {
fmt.Printf("Problem getting the response: %s\n", err)
cr = append(cr, ComponentResponse{
v.Id,
404,
err.Error(),
})
} else {
defer resp.Body.Close()
contents, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Printf("Problem reading the body: %s\n", err)
}
cr = append(cr, ComponentResponse{
v.Id,
resp.StatusCode,
string(contents),
})
}
}(i, v)
}
wg.Wait()
j, err := json.Marshal(Result{overallStatus, cr})
if err != nil {
fmt.Printf("Problem converting to JSON: %s\n", err)
return
}
fmt.Println(string(j))
}
If you want to fan out then aggregate results and you want specific timeout behavior the net/http package isn't giving you, then you may want to use goroutines and channels.
I just watched this video today and it will walk you through exactly those scenarios using the concurrency features of Go. Plus, the speaker Rob Pike is quite the authority -- he explains it much better than I could.
https://www.youtube.com/watch?v=f6kdp27TYZs
I am adding this for completes, as the correct answer was provided by Dave C in the comments of the accepted answer.
We can try to cast the error to a net.Error and check if it is a timeout.
resp, err := client.Get(url)
if err != nil {
// if there is an error check if its a timeout error
if e, ok := err.(net.Error); ok && e.Timeout() {
// handle timeout
return
}
// otherwise handle other types of error
}
The Go 1.5 release solved this issue by being more specific about the type of error it has handled.
So if you see this example https://github.com/Integralist/Go-Requester/blob/master/requester.go#L38 you'll see that I'm able to apply a regex pattern to the error message to decipher if the error was indeed a timeout or not
status := checkError(err.Error())
func checkError(msg string) int {
timeout, _ := regexp.MatchString("Timeout", msg)
if timeout {
return 408
}
return 500
}