I activate HTTP/2 support on my server. Now i got the problem with AJAX/jQuery scipts like upload or Api handling.
After max_input_time of 60sec for php i got: [HTTP/2.0 504 Gateway Timeout 60034ms]
with HTTP/1 only a few connections where startet simultaneously and when one is finished a nother starts.
with HTTP/2 all starts at once.
when fore example 100 images would uploaded it takes to long for all.
I don't wish to change the max_input_time. I hope to limit the simultaneous connections in the scripts.
thank you
HTTP/2 intentionally allows multiple requests in parallel. This differs from HTTP/1.1 which only allowed one request at a time (but which browsers compensated for by opening 6 parallel connections). The downside to drastically increasing that limit is you can have more requests on the go at once, contending for bandwidth.
You’ve basically two choices to resolve this:
Change your application to throttle uploads rather than expecting the browser or the protocol to do this for you.
Limit the maximum number of concurrent streams allowed by your webserver. In Apache for example, this is controlled by the H2MaxSessionStreams Directive while in Nginx it is similarly controlled by the
http2_max_concurrent_streams config. Other streams will need to wait.
Related
I have a Go app which is deployed to two 8 core pods instances on Kubernetes.
From it, I receive a list of ids than later on I use to retrieve some data from another service by sending each id to a POST endpoint.
I am using a bounded concurrency pattern to have a maximum number of simulataneous goroutines (and therefore, of requests) to this external service.
I set the limit of concurrency as:
sem := make(chan struct{}, MAX_GO_ROUTINES)
With this setup I started playing around with the MAX_GO_ROUTINES number by increasing it. I usually receive around 20000 ids to check. So I have played around by setting MAX_GO_ROUTINES from anywhere between 100 and 20000.
One thing I notice is as I go higher and higher some requests start to fail with the message: connection reset from this external service.
So my questions are:
What is the blocker in this case?
What is the limit of concurrent HTTP POST requests a server with 8 cores and 4GB of ram can send? Is it a memory limit? or file descriptors limit?
Is the error I am getting coming from my server or from the external one?
What is the blocker in this case?
As the comment mentioned: HTTP "connection reset" generally means:
the connection was unexpectedly closed by the peer. The server appears
to have dropped the connection on the unsuspecting HTTP client before
sending back a response. This is most likely due to the high load.
Most webservers (like nginx) have a queue where they stage connections while they wait to be serviced. When the queue exceeds some limit the connections may be shed and "reset". So it's most likely this is your upstream service being saturated (i.e. your app sends more requests than it can service and overloads its queue)
What is the limit of concurrent HTTP POST requests a server with 8 cores and 4GB of ram can send? Is it a memory limit? or file descriptors limit?
All :) At some point your particularl workload will overload a logical limit (like file descriptors) or a "physical" limit like memory. Unfortunately the only way to truly understand which resource is going to be exhausted (and which constraints you hit up against) is to run tests and profile and benchmark your workload :(
Is the error I am getting coming from my server or from the external one?
HTTP Connection reset is most likely the external, it indicates the connection peer (the upstream service) reset the connection.
We have a system client <-> server working over HTTP1.1. The client is making hundreds (sometimes thousands) of concurrent requests to the server.
Because the default limitations of the browsers to HTTP1.1 connections the client is actually making these requests in batches of (6 ~ 8) concurrent requests, we think we can get some performance improvement if we can increase the number of concurrent requests.
We moved the system to work over HTTP2 and we see the client requesting all the requests simultaneously as we wanted.
The problem now is the opposite: the server can not handle so many concurrent requests.
How can we limit the number of concurrent request the Client is doing simultaneous to something more manageable for the server? let's say 50 ~ 100 concurrent requests.
We were assuming that HTTP2 can allow us to graduate the number of concurrent
connections:
https://developers.google.com/web/fundamentals/performance/http2/
With HTTP/2 the client remains in full control of how server push is
used. The client can limit the number of concurrently pushed streams;
adjust the initial flow control window to control how much data is
pushed when the stream is first opened; or disable server push
entirely. These preferences are communicated via the SETTINGS frames
at the beginning of the HTTP/2 connection and may be updated at any
time.
Also here:
https://stackoverflow.com/a/36847527/316700
O maybe, if possible, we can limit this in the server side (what I think is more maintainable).
But looks like these solutions are talking about Server Push and what we have is the Client Pulling.
In case help in any way our architecture looks like this:
Client ==[http 2]==> ALB(AWS Beanstalk) ==[http 1.1]==> nginx ==[http 1.0]==> Puma
There is special setting in SETTINGS frame
You can specify SETTINGS_MAX_CONCURRENT_STREAMS to 100 on the server side
Reference
Case/Assumption:
There is a server that is written by someone else.
This server has an endpoint GET /api/watch.
This endpoint is plain HTTP/1.1
This endpoint will write events like
{type:"foo", message:"bar"}
to the response stream once they appear (one event per line and then a flush).
Sometimes this server writes events every second to the output, sometimes every 15 minutes.
Between my client and this server there is a third-party Load Balancer which assumes a connection as staling if there is no action on the connection for more than 60 seconds and drops the connection without closing it.
The client is written in simple Golang and simply makes a GET request to this endpoint.
Once the connection is marked by the LB as staled the client (the same happens to curl, too) is not notified that the connection was dropped by the LB and is still waiting for stuff to receive in the response of the GET request.
So: What are my possibilities to deal with this situation?
What is not possible:
Modify the server.
Use another server.
Use something else than this endpoint and how it is written.
Modify the Load Balancer.
Use another LB.
Leave the LB out of the connection.
15 minutes is an incredibly long quiet period for basic HTTP - possibly better-suited to WebSockets. Short of a protocol change, you should be able to adjust the timeout period on the load balancer (hard to say since you didn't specify what the LB is) to better suit your use case, though not all load balancers will allow timeouts as high as 15 minutes. If you can't change the protocol and can't turn the timeout up high enough, you would have to send keepalive messages from the server every so often (just short of the timeout period, so maybe 55s with your current config, or a little less than the highest timeout period you're able to set on the LB). This would have to be something the client knew to discard, like {"type": "keepalive"} - something easily identifiable on the client side as a "fake" message for keepalive purposes.
What is the use of keep alive option in Jmeter and ow its working ?
I did a performance test using Jmeter 3.0
In my recorded script Keep alive option is checked.
So i use keep alive option checked in my real test script
If i use keep alive option i got error with in 75 concurrent VU's
Error message : XXX.XXXX.XXXX:XXX server refused to respond
if i un check keep alive option i can able to go up to 500 VU's without error.
In this case do we need to use Keep alive option or not ?
Keep-alive is an HTTP feature to keep a persistent connection between round trips, so that it does not initiate a new one on every single request. This feature has many benefits, but one of the trade-offs is that it holds resources on the server side and that can be an issue under heavy load.
In your case, I guess that you've simply consumed all resources on the server with 75 opened connections and that it can't serve further requests. This error does not necessarily mean your server can't serve more than 75 connections, because it all depends on your HTTP server config.
Example of an Apache config:
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 100
Keep alive on wikipedia
Ugh! Just ran into this. According to JMeter's documentation:
http://svn.apache.org/repos/asf/jmeter/tags/v5_1_RC2/docs/usermanual/component_reference.html
JMeter sets the Connection: keep-alive header. This does not work
properly with the default HTTP implementation, as connection re-use is
not under user-control. It does work with the Apache HttpComponents
HttpClient implementations.
In other words, JMeter will send the header, but with the default implementation it will not re-use connections.
To use KeepAlive effectively, you need to select 'HttpClient4' from Implementation dropdown on Advanced tab.
HTTP keep-alive i.e. HTTP persistent connection, is an instruction that allows a single TCP connection to remain open for multiple HTTP requests/responses.
i´ve read somewhere that you can just have 2 connections (eg. ajax requests) to the same server. is this correct?
so you can´t run 3 ajax requests simultaneously? what will happen to the 3rd one?
and if I´ve got one iframe, then i can just run 1 ajax request at the time?
what is the easiest way to get around this?
what keywords could i use to search for more information regarding this on google?
The 2 connection maximum pr server is mandated in the HTTP RFC 2616 section 8.1 http://www.ietf.org/rfc/rfc2616.txt
Clients that use persistent connections SHOULD limit the number of
simultaneous connections that they maintain to a given server. A
single-user client SHOULD NOT maintain more than 2 connections with
any server or proxy. A proxy SHOULD use up to 2*N connections to
another server or proxy, where N is the number of simultaneously
active users. These guidelines are intended to improve HTTP response
times and avoid congestion.
Q:what will happen to the 3rd one?
The third one will be queued untill one of the other HTTP calls return
Q:and if I´ve got one iframe, then i can just run 1 ajax request at the time?
The iFrame will be loaded through a HTTP connection, but once the HTML content has be returned the HTTP call has been completed and you again have 2 available HTTP connections
Q:what is the easiest way to get around this?
The most important is not to have long running HTTP requests, i.e. speed up processing on the server side. As long as HTTP requests are responded to in less than 100 ms, it is for normal apps not a problem.
You read it right, browsers limit simultaneous connection to the exact same domain to 2 for any type of requests (script src, image src, ajax etc.) originating from a given document, it can be changed in registry for IE and about:config in Firefox.
One way to get around this is to have additional CNAMEs to your host.