I am trying to test how golang can handle big loads to compare it with our current applications made with Java.
What I did is a simple echo rest service like that (I am adding just the important parts of my code):
// Return default message for root routing
func Index(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
}
// Main function
func main() {
router := mux.NewRouter() //.StrictSlash(true)
router.HandleFunc("/", Index).Methods("GET")
router.HandleFunc("/echo/{message}", echoHandler(calledServiceURL)).Methods("GET")
log.Println("Running server....")
log.Fatal(http.ListenAndServe(port, router))
}
I did a test by using ab tool and it worked well with -c 500 -n 500 but when I tried to test with a big load like this
ab -c 500 -n 50000 http://localhost:9596/echo/javier
The process works well for a couple of seconds but then seems it close the tcp connection as I receive the following error:
Benchmarking localhost (be patient)
apr_socket_recv: Connection reset by peer (54)
Total of 501 requests completed
Is is due to a OS limitations that my test reached or is it the limit that my golang app could handle?
Is there a better way to process requests and then avoid the program to close connections? (queue requests or something like that).
Thanks in advance
J
Do you happen to use OSX? I understand that ab is broken on OSX.
Another thing you could try is to use -k to use the keep alive flag, but that may not be something that you want.
50000 is also close to the maximum number of sockets on an interface, so maybe your sockets are exhausted. Sockets are not directly reusable, because they will be in TIME_WAIT state for a minute or two. Exact value can vary per OS and configuration.
However, the code looks fine too me.
The following code worked fine for me:
package main
import (
"fmt"
"github.com/gorilla/mux"
"log"
"net/http"
)
func Echo(w http.ResponseWriter, r *http.Request) {
v := mux.Vars(r)
fmt.Fprintf(w, "Echo %v", v["message"])
}
func main() {
router := mux.NewRouter() //.StrictSlash(true)
router.HandleFunc("/echo/{message}", Echo).Methods("GET")
log.Println("Running server....")
log.Fatal(http.ListenAndServe("localhost:8080", router))
}
And gives the following results:
ab -c 500 -n 50000 localhost:8080/echo/foobar
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Completed 50000 requests
Finished 50000 requests
Server Software:
Server Hostname: localhost
Server Port: 8080
Document Path: /echo/foobar
Document Length: 12 bytes
Concurrency Level: 500
Time taken for tests: 2.471 seconds
Complete requests: 50000
Failed requests: 0
Total transferred: 6450000 bytes
HTML transferred: 600000 bytes
Requests per second: 20233.39 [#/sec] (mean)
Time per request: 24.712 [ms] (mean)
Time per request: 0.049 [ms] (mean, across all concurrent requests)
Transfer rate: 2548.93 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 18 122.8 3 1034
Processing: 0 5 14.9 4 225
Waiting: 0 4 14.7 3 222
Total: 1 24 132.7 6 1245
Percentage of the requests served within a certain time (ms)
50% 6
66% 7
75% 7
80% 7
90% 12
95% 20
98% 30
99% 1040
100% 1245 (longest request)
This is executed on Ubuntu 18.04.
Related
I am having some problems with the concurrent HTTP connection in the golang. Kindly read the whole question, and as the actual code is quite long, I am using pseudocode
In short, I have to create a single API, which will internally call 5 other APIs, unify their response, and send them as a single response.
I am using goroutines to call those 5 internal APIs along with timeout, and using channels to ensure that every goroutine has been completed, then I unify their response, and return the same.
Things are going fine when I do local testing, my response time is around 300ms, which is pretty good.
The problem arises when I do the locust load testing of 200 users, then my response time go as high as 7 8 sec. I am thinking it has to do with the HTTP client waiting for the resources as we are running a high number of goroutines.
like 1 API spin up 5 go-routine, so if each of 200 users makes API requests at the rate of supposing 5 req/sec. Then a total number of goroutines goes way higher. Again this is my assumption only
p.s. normally the API I am building is pretty good in response time,
I am using all the caching and stuff and any response greater than
400ms should not be the case
So can anyone please tell me how can I tackle this problem of
increasing response time when number of concurrent users increases
Locust test report
pseudo code
simple route
group.POST("/test", controller.testHandler)
controller
type Worker struct {
NumWorker int
Data chan structures.Placement
}
e := Worker{
NumWorker: 5, // Number of worker goroutine(s)
Data: make(chan, 5) /* Buffer Size */),
}
//call the goroutines along with the
for i := 0; i < e.NumWorker; i++ {
// Do some fake work
wg.Add(1)
go ad.GetResponses(params ,chan , &wg) //making HHTP call and returning the response in the channel
}
for v := range resChan {
//unifying all the response, and return the same as our response
switch v.Tyoe{
case A :
finalResponse.A = v
case B
finalResponse.B = v
}
}
return finalResponse
Request HTTP client
//i am using a global http client with custom transport , so that i can effectively use the resources
var client *http.Client
func init() {
tr := &http.Transport{
MaxIdleConnsPerHost: 1024,
TLSHandshakeTimeout: 0 * time.Second,
}
tr.MaxIdleConns = 100
tr.MaxConnsPerHost = 100
tr.MaxIdleConnsPerHost = 100
client = &http.Client{Transport: tr, Timeout: 10 * time.Second}
}
func GetResponses(params , chan ,wg){
res = client.Do(req)
chan <- res
}
So I have done some debugging and span monitoring , and turns out redis was the culprit in this. You can see this https://stackoverflow.com/a/70902382/9928176
To get an idea how I solved it
I am trying to send a page response as soon as request is received, then process something, but I found the response does not get sent out "first" even though it is first in code sequence.In real life I have a page for uploading a excel sheet which gets saved into the database which takes time (50,0000+ rows) and would like to update to user progress. Here is a simplified example; (depending how much RAM you have you may need to add a couple zeros to counter to see result)
package main
import (
"fmt"
"net/http"
)
func writeAndCount(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Starting to count"))
for i := 0; i < 1000000; i++ {
if i%1000 == 0 {
fmt.Println(i)
}
}
w.Write([]byte("Finished counting"))
}
func main() {
http.HandleFunc("/", writeAndCount)
http.ListenAndServe(":8080", nil)
}
The original concept of the HTTP protocol is a simple request-response server-client computation model. There was no streaming or "continuous" client update support. It is (was) always the client who first contacted the server should it needed some kind of information.
Also since most web servers cache the response until it is fully ready (or a certain limit is reached–which is typically the buffer size), data you write (send) to the client won't be transmitted immediately.
Several techniques were "developed" to get around this "limitation" so that the server is able to notify the client about changes or progress, such as HTTP Long polling, HTTP Streaming, HTTP/2 Server Push or Websockets. You can read more about these in this answer: Is there a real server push over http?
So to achieve what you want, you have to step around the original "borders" of the HTTP protocol.
If you want to send data periodically, or stream data to the client, you have to tell this to the server. The easiest way is to check if the http.ResponseWriter handed to you implements the http.Flusher interface (using a type assertion), and if it does, calling its Flusher.Flush() method will send any buffered data to the client.
Using http.Flusher is only half of the solution. Since this is a non-standard usage of the HTTP protocol, usually client support is also needed to handle this properly.
First, you have to let the client know about the "streaming" nature of the response, by setting the ContentType=text/event-stream response header.
Next, to avoid clients caching the response, be sure to also set Cache-Control=no-cache.
And last, to let the client know that you might not send the response as a single unit (but rather as periodic updates or as a stream) and so that the client should keep the connection alive and wait for further data, set the Connection=keep-alive response header.
Once the response headers are set as the above, you may start your long work, and whenever you want to update the client about the progress, write some data and call Flusher.Flush().
Let's see a simple example that does everything "right":
func longHandler(w http.ResponseWriter, r *http.Request) {
flusher, ok := w.(http.Flusher)
if !ok {
http.Error(w, "Server does not support Flusher!",
http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
start := time.Now()
for rows, max := 0, 50*1000; rows < max; {
time.Sleep(time.Second) // Simulating work...
rows += 10 * 1000
fmt.Fprintf(w, "Rows done: %d (%d%%), elapsed: %v\n",
rows, rows*100/max, time.Since(start).Truncate(time.Millisecond))
flusher.Flush()
}
}
func main() {
http.HandleFunc("/long", longHandler)
panic(http.ListenAndServe("localhost:8080", nil))
}
Now if you open http://localhost:8080/long in your browser, you will see an output "growing" by every second:
Rows done: 10000 (20%), elapsed: 1s
Rows done: 20000 (40%), elapsed: 2s
Rows done: 30000 (60%), elapsed: 3s
Rows done: 40000 (80%), elapsed: 4.001s
Rows done: 50000 (100%), elapsed: 5.001s
Also note that when using SSE, you should "pack" updates into SSE frames, that is you should start them with "data:" prefix, and end each frame with 2 newline chars: "\n\n".
"Literature" and further reading / tutorials
Read more about Server-sent events on Wikipedia.
See a Golang HTML5 SSE example.
See Golang SSE server example with client codes using it.
See w3school.com's turorial on Server-Sent Events - One Way Messaging.
You can check if the ResponseWriter is a http.Flusher, and if so, force the flush to network:
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
However, bear in mind that this is a very unconventional HTTP handler. Streaming out progress messages to the response as if it were a terminal presents a few problems, particularly if the client is a web browser.
You might want to consider something more fitting with the nature of HTTP, such as returning a 202 Accepted response immediately, with a unique identifier the client can use to check on the status of processing using subsequent calls to your API.
I have a go program which takes requests processes them within 100ms and sends a response. I am using digitalocean Load Balancer and i noticed 'Average Request Duration' is pretty high around 33Kms.Below is the graph
https://i.stack.imgur.com/Uj9bN.png
Here is the code i am using...
count :="one"
n := func(ctx *fasthttp.RequestCtx) {
switch count {
case "one":
handle(ctx) //...dosomething for 100 ms function
default:
ctx.Error("not found", fasthttp.StatusNotFound)
}
}
fasthttp.ListenAndServe(":8082", n)
What am i doing wrong and why isn't my program closing requests.
I tried to get html source from Reddit with Golang:
package main
import (
"fmt"
"io/ioutil"
"net/http"
"time"
)
func main() {
timeout := time.Duration(5 * time.Second)
client := http.Client{
Timeout: timeout,
}
resp, _ := client.Get("https://www.reddit.com/")
bytes, _ := ioutil.ReadAll(resp.Body)
fmt.Println("HTML:\n\n", string(bytes))
defer resp.Body.Close()
var input string
fmt.Scanln(&input)
}
First attemp was good. But at the second time it ran into an error:
<p>we're sorry, but you appear to be a bot and we've seen too many requests
from you lately. we enforce a hard speed limit on requests that appear to come
from bots to prevent abuse.</p>
<p>if you are not a bot but are spoofing one via your browser's user agent
string: please change your user agent string to avoid seeing this message
again.</p>
<p>please wait 6 second(s) and try again.</p>
<p>as a reminder to developers, we recommend that clients make no
more than <a href="http://github.com/reddit/reddit/wiki/API">one
request every two seconds</a> to avoid seeing this message.</p>
I tried to set delay but it still not work.
Sorry about my bad English.
Reddit doesn't want automatic scanner\grabbers on their site and has a bot-protection mechanism.
Here's a recommendation from them:
one request every two seconds
Just add a delay between requests.
timeout serves a different purpose. timeout is an upper limit for a routine to run. What you need is sleep between subsequent requests.
time.Sleep(6 * time.Second)
i'm write a httpserver in Golang , but i find the http.HandleFunc will be block when multi request from the web browser. how can i do make the server handle multi request in the same time ? thanks.
my code is:
func DoQuery(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
fmt.Printf("%d path %s\n", time.Now().Unix(), r.URL.Path)
time.Sleep(10 * time.Second)
fmt.Fprintf(w, "hello...")
//why this function block when multi request ?
}
func main() {
fmt.Printf("server start working...\n")
http.HandleFunc("/query", DoQuery)
s := &http.Server{
Addr: ":9090",
ReadTimeout: 30 * time.Second,
WriteTimeout: 30 * time.Second,
//MaxHeaderBytes: 1 << 20,
}
log.Fatal(s.ListenAndServe())
fmt.Printf("server stop...")
}
I ran your code and everything worked as expected. I did two requests at the same time (curl localhost:9090/query) and they both finished 10 seconds later, together. Maybe the problem is elsewhere? Here's the command I used: time curl -s localhost:9090/query | echo $(curl -s localhost:9090/query) – tjameson
thakns
that's strange.
when i request same url from chrome ,send two request not handle in the same time, but use cur test can handle in the same time.
but when i send two request use different url, it's can be handle in the same time.
[root#localhost httpserver]# ./httpServer
server start working...
1374301593 path /query?form=chrome
1374301612 path /query?from=cur2
1374301614 path /query?from=cur1
1374301618 path /query?form=chrome
1374301640 path /query?form=chrome2
1374301643 path /query?form=chrome1
*1374301715 path /query?form=chrome
1374301725 path /query?form=chrome*
**1374301761 path /query?form=chrome1
1374301763 path /query?form=chrome2**
Yes, the standard HTTP server will start a new goroutine for each request. You should be able to do thousands of requests in parallel depending on the operating system settings.
Your browser might be limiting how many requests it will send to one server; be sure you are testing with a client that doesn't have that limitation/"optimization".
Reliably Go docs explaining Http Server creates a new gorotine for each request: http://golang.org/pkg/net/http/#Server.Serve