Within my JMeter test plan I have transaction controller which contains multiple requests inside. There are no any timers between HTTP samplers and controller is configured to generate parent sample.
When I run that test most of the samples are OK but there are couple outliers for which response times for parent sample are enormous though HTTP response times inside controller are quite low. After checking I found that there are couple minute gaps between HTTP requests even no timers are configured.
E.g. first HTTP sampler started 04:08:34 and load time was 358 ms. Second sampler started 04:11:41 - so it took more than 3 minutes to start it. Then there were couple more similar requests and overall parent sample time is more than 6 minutes though sum of all HTTP sampler load times is less than 1 second.
Does anyone has an idea why it occasionally takes very long to start next HTTP request? Can it be caused by low resources (like memory) on the machines from which test is executed (it's distributed testing)?
If you don't have Timers or Flow Control Action sampler or Inter-Thread Communication Plugin JMeter executes samplers as fast as it can immediately one after another.
The only reasons I can think of is are:
lack of resources (CPU, RAM, Network, Disk) on JMeter side, I would recommend ensuring that JMeter has enough headroom to operate using i.e. JMeter PerfMon Plugin
the delay can be caused by PreProcessors, PostProcessors and Assertions so you could take a thread dump - it will provide information where exactly JMeter "hangs"
Related
I want to run basic load testing scenario on a mendix application.
The initial requirement is at least 10 users should be able to perform some activities like login , file upload concurrently.
I have used the webdriver sampler in Jmeter with selenium and written the JS script for the actions.
The issue is it launches 10 browser on the same machine.
Please suggest the proper way to handle this. Currently it seems that the errors which I am getting are because of I am running the test on my laptop instead of distributed setup.
Also the total script takes 60 seconds to finish for single user. What should be the ramp up time?
If not Jmeter, please suggest any other suitable tool for this scenario.
As per The WebDriver Sampler: Your Top 10 Questions Answered article
Q. How do I Allocate the WebDriver Sampler?
A. First of all, don’t use the WebDriver Sampler to create the load! You’ll need around 1 CPU core per virtual user to keep the JMeter resource consumption within an acceptable range. The WebDriver Sampler should be used in addition to HTTP Request Samplers. Here’s an example of how it should be used:
1,000 of users are simulated by HTTP Request Samplers
1 user is the WebDriver Sampler
You should use the WebDriver Sampler to evaluate a real-life user experience. It’s most commonly used to measure the load times of pages when the system is experiencing high loads. The HTTP Request doesn’t really “render” the page and it can’t execute JavaScript. That’s why the WebDriver Sampler is so valuable - JMeter isn’t a browser and it overcomes limitations posed by this fact.
So first of all I would recommend re-considering your approach and migrating to HTTP Request samplers, you might be able to execute 10 browsers but if you will need to scale your test to 100 or 1000 users you will definitely won't be able to use this "real browser" approach while HTTP Request samplers act on HTTP protocol level, given you properly configure JMeter it will be no difference for the application under test whether the requests are originating from JMeter or from the real browser and the resources footprint will be much less.
With regards to the ramp-up time: the overall idea is to increase the load gradually so you could correlate the changing number of virtual users with changing metrics like response time, throughput, etc. According to JMeter Documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
While understanding concept of Concurrent thread and ultimate thread group, i am confused to understand result of summary/aggregate report while running concurrent thread or ultimate thread group .For example if i have 200 user and ramp up time 60 sec then i didn't see all sampler request as 200 sample after completing execution successfully but only few sampler request have 200 sample. when i use normal thread group then i always got thread count same for each sampler request after completing execution .
for realistic load testing with more user , could you please suggest me which thread group should prefer.
Could you please provide valuable guidance with some valuable link/book and also share me standard performance bench mark criteria or key parameter detail while doing load testing.(if any bench load parameter value does not meet standard then we can say that there is a performance issue)
Thanks for giving valuable time in advance.
Thanks
amit
This is due to the fact that:
Your application response time is too high
Your test duration is too low
For example I can see response times > 80 seconds:
it means that if a single virtual user has cumulative response time for 2 samplers > 160 seconds and the test duration is 120 seconds it will not be able to execute all the requests. Just increase your test duration to be i.e. 10 minutes and you should see more virtual users capable of executing all the Samplers you defined in the test plan.
Also given first user is capable of executing all the requests successfully and in time it looks like that your application gets overloaded hence cannot respond fast enough when the number of concurrent users reaches some "critical threshold", you can add listeners like Active Threads Over Time and Response Times vs Threads, this way you will be able to correlate increasing load with the increasing response time.
If also makes sense to collect:
Baseline health metrics of your application (CPU, RAM, Network, Disk usage, etc.), it can be done using JMeter PerfMon Plugin.
Lower level details like slowest methods, largest objects, heaviest database queries, etc. This form of information can be obtained using profiling tools specific to your application programming language(s).
I have a very simple web service and I'm trying to measure its latency metrics under different throughputs -
If the system under test can't handle the target throughput, I want Jmeter to keep the same throughput, even if a lot of requests fail.
I tried implementing this using a large number of threads and a constant throughput timer, but Jmeter tended to simply not reach the target throughput instead of requests failing.
Any recommendations?
Constant Throughput Timer has 2 main limitations:
It is accurate enough on minute level so you need to ensure that your test is running long enough (minute or more). Also its effect is not immediate so during 1st minute of your load test you need to control virtual users arrival rate using Ramp-Up Period setting on Thread Group level
Constant Throughput Timer is only capable of pausing the threads to limit their execution rate to the "target throughput", it will not be able to kick off any extra threads so make sure you define enough virtual users again on Thread Group level
Given Web service == HTTP transport and HTTP being stateful protocol (each JMeter thread (virtual user) will wail for response from the previous HTTP Request sampler prior to starting next one you can speed things up by defining a timeout for your HTTP Requests so they would fail earlier. The best place to put the timeouts is HTTP Request Defaults, this way you will be able to do it in a single shot for all HTTP Request samplers in scope
You may also want to try Throughput Shaping Timer which is a little bit enhanced version of the Constant Throughput Timer.
I am new to JMeter and I have been tasked to do a POC where I need to load test the webserivce. I learnt the basics like adding the test plan, adding threads, adding SOAP/RPC Requestsampler and I got the response as well. But, I am not sure how to achieve the below scenario using JMeter.
I need 600 users to hit the service per request/second (this should run for 10 minutes) and the 2nd scenario is about 2000 users to hit the service at 5 request/second (again this should run for 10 minutes)
Also, would it be possible for JMeter to handle this many number of threads/users?
Any inputs would be deeply appreciated.
Given you properly configure JMeter it shouldn't be a problem to simulate 2k users, actually you may need more as if your web service response time will exceed 1 second - you won't be able to achieve 2k requests per second.
Configuring JMeter:
Run test in non-GUI mode
Disable all Listeners (if any)
Increase JVM Heap size
See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure for detailed explanation and instructions
Simulating 600 / 2000 requests per second.
Set "Loop Count" to Forever or -1 in Thread Group
Tick "Scheduler" box and set desired duration (600 seconds)
Add Constant Throughput Timer and specify the desired throughput in requests per minute
It is recommended to use HTTP Request sampler for web services testing, you can set Content-Type and SOAPAction headers using HTTP Header Manager
I'm trying to understand the basics of JMeter. I've got a "plus1" Java servlet that adds one to a request parameter and returns the result, so it's a fast test servlet just so I can understand load testing.
Here's my test plan:
Thread Group: 1 thread, ramp up 1 s, loop count 10000
HTTP Request to localhost
Graph Results
Summary Report
When I run this, the summary report shows a throughput number of 200/sec or so.
The key question, with no controllers in the test plan, is JMeter running the test plan (sending a single request) and waiting for the response before looping?
When I introduce a more computationally intensive page for the request, the throughput number goes down as I would expect.
In short, yes.
There is an argument for having a sampler that would make a request and not wait for the response but it's an edge case. In most cases you would want a testing tool to wait to see what happens and verify things. It's also more realistic, most users will wait for a response, in fact they generally have to, before making subsequent calls.
If you want to run a capacity test then the best approach, I think, is to spread the load over multiple threads and to actually throttle the throughput of each one - you can do this using a Constant Throughput Controller. Eg. You could have 500 threads each running at 60 requests per minute, this would give a total load of 500 reqs/sec. This way, your test load is predictable and stable - it won't be linked to the speed of response from the server. Note. with multiple threads you'll want a ramp up period and you might find you have to spread the test over multiple machines (known as 'distributed' testing if you're going to google it).