I made a few test downloads using the Jetty 9 server, where it is made multiple downloads of a single file with an approximate size of 80 MB. When smaller number of downloads and the time of 55 seconds is not reached to download, all usually end, however if any downloads in progress after 55 seconds the flow of the network simply to download and no more remains.
I tried already set the timeout and the buffer Jetty, though this has not worked. Has anyone had this problem or have any suggestions on how to solve? Tests on IIS and Apache Server work very well. Use JMeter for testing.
Marcus, maybe you are just hitting Jetty bug 472621?
Edit: The mentioned bug is a separate timeout in Jetty that applies to the total operation, not just idle time. So by setting the http.timeout property you essentially define a maximum time any download is allowed to take, which in turn may cause timeout errors for slow clients and/or large downloads.
Cheers,
momo
A timeout means your client isn't reading fast enough.
JMeter isn't reading the response data fast enough, so the connection sits idle long enough that it idle times out and disconnects.
We test with 800MB and 2GB files regularly.
On using HTTP/1.0, HTTP/1.1, and HTTP/2 protocols.
Using normal (plaintext) connections, and secured TLS connections.
With responses being delivered in as many Transfer-Encodings and Content-Encodings as we can think of (compressed, gzip, chunked, ranged, etc.).
We do all of these tests using our own test infrastructure, often spinning up many many Amazon EC2 nodes to perform a load test that can sufficiently test the server demands (a typical test is 20 client nodes to 1 server node)
When testing large responses, you'll need to be aware of the protocol (HTTP/1.x vs HTTP/2) and how persistence behavior of that protocol can change the request / response latency. In the real world you wont have multiple large requests after each other on the same persisted connection via HTTP/1 (on HTTP/2 the multiple requests would be parallel and be sent at the same time).
Be sure you setup your JMeter to use HTTP/1.1 and not use persisted connections. (see JMeter documentation for help on that)
Also be aware of your bandwidth for your testing, its very common to blame a server (any server) for not performing fast enough, when the test itself is sloppily setup and has expectations that far exceed the bandwidth of the network itself.
Next, don't test with the same machine, this sort of load test would need multiple machines (1 for the server, and 4+ for the client)
Lastly, when load testing, you'll want to become intimately aware of your networking configurations on your server (and to a lesser extent, your client test machines) to maximize your network configuration for high load. Default configurations for OS's are rarely sufficient to handle proper load testing.
Related
We are executing a test of Upload scenario where we are aware that the response time will be more than 5 minutes. Hence we have configured timeout in HTTP Request Defaults as well as in the Http request as 3600000 milliseconds. But still we are getting Socket Exception in Upload transaction . Could you please suggest how to handle this.
Thanks,
SocketException doesn't necessarily means "timeout", it indicates that JMeter is not able to create or access Socket connection, there are too many possible reasons, the most common are:
Network configuration of your server doesn't allow that many connections as you're trying to open, check the maximum number of open connections on your application server and operating system level.
Your application server is overloaded and cannot handle such a big load. Make sure it has enough headroom to operate in terms of CPU, RAM and especially Network metrics (these can be monitored using JMeter PerfMon Plugin)
You might be experiencing the behaviour described in JMeterSocketClosed article
Basically the same as points 1 and 2 but this time you need to check JMeter health, make sure you're following JMeter Best Practices and maybe even consider going for distributed testing
I am doing stress tests / load tests on a mobile application using Jmeter.
The problem is that when i perform tests using HTTP it works fine, but using HTTPS makes the server go down.
Is there a mechanism included in HTTPS that blocks a load of simultaneous queries ? What can be the problem please ?
EDIT : The question is about Jmeter and the HTTP/HTTPS.
Jmeter simulates a query and starts to re send it like 1000, 2000, etc... time for a specified interval. The use of Jmeter and HTTP works, but Jmeter and HTTPS makes the server go down. Is it possible that the HTTPS have an inside mechanism that could classify the test as DoS attack and block it ?
Notice that HTTP and HTTPS uses different ports
Different ports can route to different IPs and even application, for example go straight to tomcat without apache server
About performance it depends, but overall ~insignificant
HTTPS requires an initial handshake which can be very slow. The actual amount of data transferred as part of the handshake isn't huge (under 5 kB typically), but for very small requests, this can be quite a bit of overhead. However, once the handshake is done, a very fast form of symmetric encryption is used, so the overhead there is minimal. Bottom line: making lots of short requests over HTTPS will be quite a bit slower than HTTP, but if you transfer a lot of data in a single request, the difference will be insignificant.
I need to test 200.000 VU hitting an app in 10 seconds, so I started to make a test of 10.000 VU, running Jmeter in Non-GUI mode, to see the response of my computer, my internet connection and the site response, but I got 83.50% of Errors.
95% of the errors were these:
Non HTTP response code: java.net.ConnectException/Non HTTP response message: Connection timed out: connect
This means that the internet connection was not enough for the short time of the test?
Thanks.
Running 200K users
Generally speaking in traditional HTTP running 200.000 users from one machine is impossible: there isn't that many ports. I.e. if you maximize your port usage (and it's likely you need to change OS settings to do that, since usually OS will limit number of open ports to somehwere between 1000 and 10000), JMeter will have about 64500 ports to run requests on. Each JMeter HTTP sampler needs a separate port, so you need 200K ports. Thus you need to have at least 4 machines to run 200K requests concurrently.
But that may not be enough: if you have more than one request sequentially (like most performance tests do), you will be able to run even less concurrent requests, since ports are usually not closed right away after request is done, so next request has to use a different port.
Don't forget that server also must be able to receive similar load.
But even that may not be enough: JMeter needs to have enough memory to accommodate 10-30K threads. Size of thread in memory will depend on a few things, and how your script is designed among them.
Bottom line: with all the tweaking, realistically, port availability limits number of concurrent requests JMeter can run from one machine to 10-30K concurrent users. Thus to test 200K users, you need about 7-20 JMeter machines.
Running 10K users
If you were testing in a designated environment (where clients and servers are next to each other with optimized network between them), you should be able to run 10K users from one machine, if other limits, e.g. memory and max ports were properly tweaked. But sounds like you are trying to test them over the internet connection?
Well, 2 problems here:
Performance testing over internet connection is absolutely pointless. You don't know what is between you and servers, and how those things in between are changing the shape of the load. You won't know if it was 10K concurrent requests, or 10K sequential requests. And results will only tell you how fast your internet is.
Any ISP will have a limit on number of connections from one IP, and it will be well below 10K. Not to mention that some ISPs may flag / temporary ban your IP for such flood.
Bottom line: whoever asked you to test 10K or 200K concurrent users, should also provide a set of JMeter machines to run this test from. Those machines should be close to tested servers, preferably without any extra routing in between (or with well known and well configured routing)
I don't think that stressing your application by kicking off 200k users at once is a good idea (same applies to 10k users) as the results, even in case of success, won't tell the full story. Moreover, in case of error you will be able to state only that 10k users in 10 seconds is not possible, however you won't have the information like:
What was the number of users when errors start occurring
What is the correlation between number of concurrent users and response time and/or throughput
What is the saturation point (the maximum system performance)
So I would recommend re-running your test and increasing the load gradually from one virtual user to 10 000 and see when it breaks. The breaking point is called bottleneck and the cause can be determined like:
First of all make sure you're following JMeter Best Practices as default JMeter configuration is not suitable for high loads and if JMeter is not capable of sending requests fast enough you will not get accurate results. Most probably you will have to run JMeter in Distributed mode, it is highly unlikely you will be able to mimic 20k requests per second from a single machine (or it has to be a very powerful one)
Set up monitoring of the application under test in order to ensure that it has enough headroom in terms of CPU, RAM, Disk, etc. You can use JMeter PerfMon Plugin for this
Check your application infrastructure, like JMeter the majority of middleware components like web/application servers, load balancers, databases, etc. default configurations are suitable for development and debugging, they need to be tuned for high throughput.
Check your application code using profiler tools telemetry, the reason could be in i.e. slow DB query, inefficient algorithm, large object, heavy function, etc.
I have a browser plugin which will be installed on 40,000 dekstops.
This plugin will connect to a backend configuration file available via https, e.g. http://somesite/config_file.js.
The plugin is configured to poll this backend resource once/day.
But there is only one backend server. So if 40,000 endpoints start polling together the server might crash.
I could think of randomize the polling frenquency from the desktop plugins. But randomization still does not gurantee that there will not be a overload at the server.
Is using websocket in this scenario solves the scalability issue?
Polling once a day is very little.
I don't see any upside for Websockets unless you switch to Push and have more notifications.
However, staggering the polling does make a lot of sense, since syncing requests for the same time is like writing a DoS attack against your own server.
Staggering doesn't necessarily have to be random and IMHO, it probably shouldn't.
You could start with a fixed time and add a second per client ID, allowing for ~86K connections in 24 hours which should be easy for any server to handle.
As a side note, 40K concurrent connections might not as hard to achieve as you imagine.
EDIT (relating to the comments)
Websockets vs. Server Sent Events:
IMHO, when pushing data (vs. polling), I would prefer Websockets over Server Sent Events (SSE).
Websockets have a few advantages, such as client side communication which allows clients to ping the server and confirm that the connection is still alive.
The Specific Use-Case:
From the description in the question and the comments it seems that you're using browser clients with a custom plugin and that the updates you wish to install daily might require the browser to be active.
This raises different questions that effect the implementation (are the client browsers open all day? do you have any control over the client browsers and their environment? can you guarantee installation while the browser is closed?).
...
IMHO, you might consider having the client plugins test for an update each morning as they load for the first time during that day (first access).
People arrive at work in different times and they open their browsers for the first time at different schedules. So the 40K requests you're expecting will be naturally scattered across that timeline (probably a 20-30 minute timespan).
This approach makes sure that the browsers and computers are actually open (making the update possible) and that the update requests are staggered over a period of time (about 33.3 requests per second, if my assumption is correct).
If you're serving a pre-written static configuration file (perhaps updated by the server daily), avoiding dynamic content and minimizing any database calls, than 33 req/sec should be very easy to manage.
Which tools can be used to measure performance of a webserver?
To test a webserver, you can use Apache Jmeter.
To see where is the bottleneck you have to flood your server application.
ApacheBench (ab) can do this. Here is a tool to get the server HTTP response code (ab) just says there is an HTTP error, and to automate test runs:
dsec.com/source/ab.c.txt
This program also gives useful tips about how to configure Linux and Windows (TCP/IP system options) to get the best possible performances.
It always depends on the setup.
Depending on the application there can be different bottlenecks.
Sometimes its the CPU, sometimtes the database connections, sometimes the sockets, sometimes the hard disc etc...
Most common practice is to use siege (simple command line tool) and increase the concurrent connections and see how many transactions per second go through.
It will increase per connection until an optimum is reached, then it will slowly decrase.
You can produce a set of urls that are randomly accessed, maby biased and/or send random data, request random ids etc to simulate more "real" clients.
Completely depends on your application whether this is relevant.