I am playing wit golang and orientdb to test them. i have written a tiny web app which uppon a request fetches a single document from local orientdb instance and returns it. when i bench this app with apache bench, when concurrency is above 1 it get following error:
2015/04/08 19:24:07 http: panic serving [::1]:57346: Get http://localhost:2480/d
ocument/t1/9:1441: EOF
when i bench Orientdb itself, it runs perfectley OK with any cuncurrency factor.
also when i change the url to fetch from this document to anything (other program whritten in golang, some internet site etc) the app runs OK.
here is the code:
func main() {
fmt.Println("starting ....")
var aa interface{}
router := gin.New()
router.GET("/", func(c *gin.Context) {
ans := getdoc("http://localhost:2480/document/t1/9:1441")
json.Unmarshal(ans, &aa)
c.JSON(http.StatusOK, aa)
})
router.Run(":3000")
}
func getdoc(addr string) []byte {
client := new(http.Client)
req, err := http.NewRequest("GET", addr, nil)
req.SetBasicAuth("admin","admin")
resp, err := client.Do(req)
if err != nil {
fmt.Println("oops", resp, err)
panic(err)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
return body
}
thanks in advance
The keepalive connections are getting closed on you for some reason. You might be overwhelming the server, or going past the max number of connections the database can handle.
Also, the current http.Transport connection pool doesn't work well with synthetic benchmarks that make connections as fast as possible, and can quickly exhaust available file descriptors or ports (issue/6785).
To test this, I would set Request.Close = true to prevent the Transport from using the keepalive pool. If that works, one way to handle this with keepalive, is to specifically check for an io.EOF and retry that request, possibly with some backoff delay.
Related
I have a struct called Server and it has this method:
func (s *Server) Run(addr string) error {
return http.ListenAndServe(addr, s)
}
Here is one of the ways that I usually test this kind of functions:
func TestServer_Run(t *testing.T) {
srv := NewServer()
go srv.Run(":2376")
// Give server sometime to start
time.Sleep(20*time.Millisecond)
// Send some request to server and test the result.
}
I don't want to test the endpoints and I just want to see if server runs properly.
But I didn't find this way very nice because I wait for 20ms and the server might start earlier and I'm wasting some time in here.
Also server starts faster but I have to sleep for 20ms so I'm sure that server has started and now I can start using the server.
What is the best way of testing functions like this? Actually How to avoid time.Sleep()?
You need to start the TCP listener first, after that you can make the HTTP request while the server is starting. The request will be processed as soon as the server is ready.
Actually another issue is how to stop the server at the end of the test.
Below is an example of how to achieve both:
func TestServer_Run(t *testing.T) {
l, err := net.Listen("tcp", "localhost:3000")
require.NoError(t, err)
r := http.NewServeMux()
r.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte("Hello!"))
})
server := &http.Server{Handler: r}
defer server.Close()
go func() {
if err := server.Serve(l); err != http.ErrServerClosed {
panic(err)
}
}()
res, err := http.Get("http://localhost:3000")
require.NoError(t, err)
body, err := io.ReadAll(res.Body)
require.NoError(t, err)
require.Equal(t, "Hello!", string(body))
}
requestHandler := func(ctx *fasthttp.RequestCtx) {
time.Sleep(time.Second*time.Duration(10))
fmt.Fprintf(ctx, "Hello, world! Requested path is %q", ctx.Path())
}
s := &fasthttp.Server{
Handler: requestHandler
}
if err := s.ListenAndServe("127.0.0.1:82"); err != nil {
log.Fatalf("error in ListenAndServe: %s", err)
}
multiple request,and it cost time like X*10s.
fasthttp is single process?
after two days...
I am sorry for this question,i describe my question not well.My question is caused by the browser，the browser request the same url by synchronization, and it mislead me, it make think the fasthttp web server hanlde the request by synchronization.
I think instead of fasthttp is single process?, you're asking whether fasthttp handles client requests concurrently or not?
I'm pretty sure that any server (including fasthttp) package will handle client requests concurrently. You should write a test/benchmark instead of manually access the server through several browsers. The following is an example of such test code:
package main_test
import (
"io/ioutil"
"net/http"
"sync"
"testing"
"time"
)
func doRequest(uri string) error {
resp, err := http.Get(uri)
if err != nil {
return err
}
defer resp.Body.Close()
_, err = ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
return nil
}
func TestGet(t *testing.T) {
N := 1000
wg := sync.WaitGroup{}
wg.Add(N)
start := time.Now()
for i := 0; i < N; i++ {
go func() {
if err := doRequest("http://127.0.0.1:82"); err != nil {
t.Error(err)
}
wg.Done()
}()
}
wg.Wait()
t.Logf("Total duration for %d concurrent request(s) is %v", N, time.Since(start))
}
And the result (in my computer) is
fasthttp_test.go:42: Total duration for 1000 concurrent request(s) is 10.6066411s
You can see that the answer to your question is No, it handles the request concurrently.
UPDATE:
In case the requested URL is the same, your browser may perform the request sequentially. See Multiple Ajax requests for same URL. This explains why the response times are X*10s.
I am sorry for this question,i describe my question not well.My question is caused by the browser，the browser request the same url by synchronization, and it mislead me, it make think the fasthttp web server hanlde the request by synchronization.
From the official documentation https://golang.org/pkg/net/http/#Client.Do it seems that the RoundTripper may not be able to re-use TCP connection for the next "keep-alive" request if Body is not closed and not fully read. What is this may about?
From what I see Close does not necessarily need to be called, when the whole Body is read. So what is the necessary requirement for connection re-use?
Code snippet (note commented out defer resp.Body.Close()) which creates multiple connections in a loop and from analysing it with netstat it seems the same TCP connection is used for all connections:
for nextPage != "" {
req, err := http.NewRequest("GET", nextPage, nil)
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", *token))
if err != nil {
panic(err)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
// defer resp.Body.Close()
result := []*User{}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
panic(err)
}
nextPage = getNextPage(resp.Header.Get("Link"))
}
Document doesn't say Don't call Close() for keep-alive. Just say if you want to re-use connection, you MUST call Close() and fully read.
You can only count on what's in the documentation. In some circumstances (go version, OS, architecture, response content length, etc.) it may reuse the connection without fully reading it, or it may not. If you want to ensure the connection will be reused, you must fully read the body and close it.
I generally write a quick helper:
func cleanUpRequest(req *http.Request) {
if req != nil && req.Body != nil {
io.Copy(ioutil.Discard, req.Body)
req.Body.Close()
}
}
This is safe to defer even before error checking.
Connecting to Redigo and manipulating data inside a function is easy like butter, but the problem comes when you have to re-use its connection, obviously for performance/practicality reasons.
Doing it inside a function like this works:
func main() {
client, err := redis.Dial("tcp", ":6379")
if err != nil {
panic(err)
}
defer client.Close()
client.Do("GET", "test:1")
}
But bringing it outside doesn't:
var Client = redis.Dial("tcp", ":6379")
defer Client.Close()
func main() {
Client.Do("GET", "test:1")
}
With the following error(s) returned:
./main.go:1: multiple-value redis.Dial() in single-value context
./main.go:2: non-declaration statement outside function body
I've tried putting the connection as a const(ant), putting defer inside the main function to my dismay not working too.
This is an even bigger concern as I have many other functions that have to communicate to Redis, but recreating the connection to Redis everytime seems silly.
The Redigo API just shows how to create a Dial instance but doesn't go further by explaining how to re-use it.
You may've been lost in my talk, but I wanted to put a bit of context here, so my clear and concise question is: How do you go about re-using (not recreating everytime) a Redigo connection?
The best way turned out to be using Pools, which are briefly documented here: Redigo Pools.
A global variable won't eventually reuse a connection, so I ended up with something like this (using Pools as noted before):
func newPool() *redis.Pool {
return &redis.Pool{
MaxIdle: 80,
MaxActive: 12000, // max number of connections
Dial: func() (redis.Conn, error) {
c, err := redis.Dial("tcp", ":6379")
if err != nil {
panic(err.Error())
}
return c, err
},
}
}
var pool = newPool()
func main() {
c := pool.Get()
defer c.Close()
test,_:=c.Do("HGETALL", "test:1")
fmt.Println(test)
}
If for example you want to reuse a pool inside another function you do it like this:
func test() {
c := pool.Get()
defer c.Close()
test2,_:=c.Do("HGETALL", "test:2")
fmt.Println(test2)
}
The redis.Dial() method returns client error. To fix it, you should replace:
var Client = redis.Dial("tcp", ":6379")
with:
var Client, _ = redis.Dial("tcp", ":6379")
I am trying to debug a very unusual error I am receiving for a simple REST library I wrote.
I am using the standard net/http package to make Get, Post, Put, Delete requests but my tests occasionally fail when I make multiple requests successively. My test looks like this:
func TestGetObject(t *testing.T) {
firebaseRoot := New(firebase_url)
body, err := firebaseRoot.Get("1")
if err != nil {
t.Errorf("Error: %s", err)
}
t.Logf("%q", body)
}
func TestPushObject(t *testing.T) {
firebaseRoot := New(firebase_url)
msg := Message{"testing", "1..2..3"}
body, err := firebaseRoot.Push("/", msg)
if err != nil {
t.Errorf("Error: %s", err)
}
t.Logf("%q", body)
}
And I am making the request like this:
// Send HTTP Request, return data
func (f *firebaseRoot) SendRequest(method string, path string, body io.Reader) ([]byte, error) {
url := f.BuildURL(path)
// create a request
req, err := http.NewRequest(method, url, body)
if err != nil {
return nil, err
}
// send JSON to firebase
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("Bad HTTP Response: %v", resp.Status)
}
defer resp.Body.Close()
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return b, nil
}
Sometimes it works, but most of the time I get 1 or 2 failures:
--- FAIL: TestGetObject (0.00 seconds)
firebase_test.go:53: Error: Get https://go-firebase-test.firebaseio.com/1.json: EOF
firebase_test.go:55: ""
--- FAIL: TestPushObject (0.00 seconds)
firebase_test.go:63: Error: Post https://go-firebase-test.firebaseio.com/.json: EOF
firebase_test.go:65: ""
FAIL
exit status 1
FAIL github.com/chourobin/go.firebase 3.422s
The failures happen when I make more than 1 request. If I comment out everything except for the PUT request, the tests consistently pass. Once I include a second test, such as GET, one or the other fails (sometimes both pass).
I experienced this reliably. You need to set Req.Close to true (the defer on resp.Body.Close() syntax used in the examples is not enough). Like this:
client := &http.Client{}
req, err := http.NewRequest(method, url, httpBody)
// NOTE this !!
req.Close = true
req.Header.Set("Content-Type", "application/json")
req.SetBasicAuth("user", "pass")
resp, err := client.Do(req)
if err != nil {
// whatever
}
defer resp.Body.Close()
response, err = ioutil.ReadAll(resp.Body)
if err != nil {
// Whatever
}
I agree with the assertion that you shouldn't be hitting outside servers in your unit tests, why not just use the built-in http.Server and serve up the content that you want to test. (There is actually the httptest package to help with this)
I recently ran into this same problem while trying to crawl sitemaps, and this is what I have found so far:
Go by default will send requests with the header Connection: Keep-Alive and persist connections for re-use. The problem that I ran into is that the server is responding with Connection: Keep-Alive in the response header and then immediately closing the connection.
As a little background as to how go implements connections in this case (you can look at the full code in net/http/transport.go). There are two goroutines, one responsible for writing and one responsible for reading (readLoop and writeLoop) In most circumstances readLoop will detect a close on the socket, and close down the connection. The problem here occurs when you initiate another request before the readLoop actually detects the close, and the EOF that it reads get interpreted as an error for that new request rather than a close that occurred prior to the request.
Given that this is the case the reason why sleeping in between requests works is that it gives readLoop time to detect the close on the connection before your new request and shut it down, so that your new request will initiate a new connection. (And the reason why it would intermittently fail is because there is some amount code running between your requests and depending of scheduling of goroutines, sometimes the EOF will be properly handled before your next request, sometimes not). And the req.Close = true, solution works because it prevents the connection from being re-used.
There is a ticket related to this situation: https://code.google.com/p/go/issues/detail?id=4677 (and a dupe ticket that I created that allowed me to reliably reproduce this: https://code.google.com/p/go/issues/detail?id=8122)
I'm going to guess there is no problem with your code. The most likely cause of your problem is because the server is closing the connection. Rate limiting is one possible reason for this.
Your test shouldn't be relying on an external service that's very brittle and not hermetic. Instead you should think about spinning up a test server locally.
My experience with this error was when I entered absolutely empty input for my JSON API!
I should send {} as empty JSON, but I sent so this error happened
I encountered this issue while sending an invalid body to a GET request
I could reproduce by making a request similar to below:
var requestBody interface{}
requestData, _ := json.Marshal(requestBody)
payload = strings.NewReader(string(requestData))
req, err := http.NewRequest("GET", url, payload)
...
I was developing an image download app when this problem occurs.
Tried request.Close=true but not work.
60% requests resulted in a EOF error.
I thought maybe it is an image server problem, not my code.
But php code works fine.
Then I use
var client = &http.Client{
Transport: &http.Transport{},
}
client.Do(request)
to make request, instead of
http.DefaultClient.Do(request)
problem gone.
Not sure why,I guess something with RoundTripper