So I'm working on JMeter and I tried to run a test plan (with only one script: edureka.co that further has 6 scripts when viewed in Result Tree like https://www.edureka.co/-0 https://www.edureka.co/-1 till https://www.edureka.co/-5) with concurrency thread group with 100 threads, 200 sec Ramp-up time and 2 Ramp-up steps. I ran this test plan in Non-GUI Mode of JMeter and to my surprise I got 7880+ records in my CSV File. Max I should have gotten like 600 records.
Anyone can tell why I'm getting this much records in my CSV file and what does the scripts https://www.edureka.co/-0, https://www.edureka.co/-1, https://www.edureka.co/-2 till https://www.edureka.co/-5 mean. (they show up even with 1 thread)
For these -0, -1, etc. there is an explanation:
first couple is being caused by redirection to https://www.edureca.co
next requests stand for images, scripts and styles - so called "embedded resources"
this is absolutely normal as long as JMeter sends the same requests as the real browser does, the only piece of advice I can give is to add HTTP Cache Manager to your test plan as real browsers download these embedded resources only once.
Max I should have gotten like 600 records - this is not true, the number of "records" depends on the application response time and related metrics (connect time, latency, network bandwidth, etc). If you want to limit the number of results to 600 only either switch to normal Thread Group with 100 threads and 6 loops or use Throughput Controller to limit the number of requests to 600 only
Related
I am currently using Jmeter for API Performance Testing, but recently I started to look into Gatling as a potential replacement of Jmeter. Below is the PoC I'm doing for Gatling but I notice the performance result is very different.
Setup:
where we hit a https endpoint with a concurrent user of 10 for 60 seconds.
Results
Jmeter: 10 threads (no ramp up), 60 seconds
Result: 150 TPS
Gatling: 10 concurrent users, also 60 seconds
Result: 27 TPS(cnt/s?)
Question:
first I want to confirm the terminology of Gatling; in Gatling result chart, I see a column named "mean cnt/s" I hovered over it and it says "count of event per second", I imagine that's the same thing as Jmeter's TPS?
Jmeter:
summary + 2386 in 00:00:16 = 153.1/s Avg
Gatling:
Mean cnt/s: 26.652
if above assumption is correct, can someone share some insight on why Gatling's number is much lower than Jmeter's?
Thank You!
Gatling: 10 concurrent users, also 60 seconds
Do you understand what this does?
This is going to spawn a new user every time an existing one finishes, and hence create new connections. Assuming it takes 100ms for a virtual user to complete the scenario, you're going to spawn 101060 = 6,000 virtual users and as many connections.
Is that really what you want and is it the same thing as you do with JMeter?
If you actually want the same 10 users to loop for 60 seconds, you have to inject atOnceUsers(10) and add a during(60) loop in your scenario.
https://gatling.io/docs/gatling/reference/current/core/injection/#open-model
https://gatling.io/docs/gatling/reference/current/core/scenario/#loop-statements
Many things can cause deviations.
I assume you use the same setup for both in terms of load generator/target insance. You can start with fixed number of requests first.
Use loops in Jmeter and repeat in Gatling.
Sending for example 60 x 10 = 600 requests in total.
Gatling will be able to generate much higher load than Jmeter if properly used.
I have taken 200 users at a time and ramp up period is 100 seconds.I entered URLs in the CSV file, added path and variables for respective folder in the Jmeter. It also display the results for users but the ramp up period is continuously going on and on. Even though I take 100 seconds it goes up to 2 hours. So is there any alternating way to test the multiple URLs at a time or not?
Ramp-up period doesn't have anything in common with test duration, if you have 200 users and 100 seconds ramp-up it basically mean that JMeter will add 2 users each second for 100 seconds instead of starting all 200 users at the same moment. More information: JMeter Ramp-Up Period: The Ultimate Guide
Once kicked off each user will start executing Samplers upside down for the number of iterations defined in the Thread Group or for the duration specified in the Thread Group.
Default CSV Data Set Config setup means that:
each user reads next line from the CSV file on each iteration (controllable by "Sharing Mode")
when the end of file is reached it starts over from the beginning (controllable by "Recycle on EOF" and "Stop thread on EOF")
So depending on what you're trying to achieve:
If you want your test to stop after 100 seconds you need to amend your Thread Group configuration like:
If you want your test to visit all URLs from the CSV file and then stop - amend your CSV Data Set Configuration to look like:
I want to run basic load testing scenario on a mendix application.
The initial requirement is at least 10 users should be able to perform some activities like login , file upload concurrently.
I have used the webdriver sampler in Jmeter with selenium and written the JS script for the actions.
The issue is it launches 10 browser on the same machine.
Please suggest the proper way to handle this. Currently it seems that the errors which I am getting are because of I am running the test on my laptop instead of distributed setup.
Also the total script takes 60 seconds to finish for single user. What should be the ramp up time?
If not Jmeter, please suggest any other suitable tool for this scenario.
As per The WebDriver Sampler: Your Top 10 Questions Answered article
Q. How do I Allocate the WebDriver Sampler?
A. First of all, don’t use the WebDriver Sampler to create the load! You’ll need around 1 CPU core per virtual user to keep the JMeter resource consumption within an acceptable range. The WebDriver Sampler should be used in addition to HTTP Request Samplers. Here’s an example of how it should be used:
1,000 of users are simulated by HTTP Request Samplers
1 user is the WebDriver Sampler
You should use the WebDriver Sampler to evaluate a real-life user experience. It’s most commonly used to measure the load times of pages when the system is experiencing high loads. The HTTP Request doesn’t really “render” the page and it can’t execute JavaScript. That’s why the WebDriver Sampler is so valuable - JMeter isn’t a browser and it overcomes limitations posed by this fact.
So first of all I would recommend re-considering your approach and migrating to HTTP Request samplers, you might be able to execute 10 browsers but if you will need to scale your test to 100 or 1000 users you will definitely won't be able to use this "real browser" approach while HTTP Request samplers act on HTTP protocol level, given you properly configure JMeter it will be no difference for the application under test whether the requests are originating from JMeter or from the real browser and the resources footprint will be much less.
With regards to the ramp-up time: the overall idea is to increase the load gradually so you could correlate the changing number of virtual users with changing metrics like response time, throughput, etc. According to JMeter Documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
I have tried but have a doubt that whether the below-mentioned specification is equivalent to 4000load or not.
the number of threads-100,
ramp-up period-10 secs,
loop count- 40, then
which is equal to how much load??
You are loading 100 concurrent threads, the loops just adds more execution time.
So it isn't equivalent to 4000 concurrent threads hitting your server
I don't know what do you mean by 4000load, your test will send 4000 requests per each Sampler which is in your Thread Group as fast as it can. The actual test duration will depend on your application response time but will not be less than 10 seconds.
You might want to take a look at Transactions per Second and Server Hits per Second charts to see how many requests your configuration delivers, both charts can be installed using JMeter Plugins Manager
Also you can generate HTML Reporting Dashboard which will have consolidated aggregate view of your test results.
I want to put the load of 20000 user requests on my server to test can it handle this number of requests in a second.
I have put the Number of threads(user) - 20000
Ramp-up period (seconds) - 1
Loop count - 1
Output - It is showing 20 seconds Avg time after completing the script.
Interpreting the output:
It is stating that the average server response time is 20 sec.
To check whether your server can handle 20000 users per sec,
run the test plan from non-GUI mode (Generating HTML dashboard report). Because 20000 is a huge number.
Go get the best of the non-GUI mode add the JMeter plugin manager. Add the required reports needed by you.
If I am not wrong what you are trying to achieve is a throughput of 20000 per sec. The same can be found using transactions per sec graph from the HTML dashboard report generated in non-GUI mode.
This plugin might help your use case: Throughput Shaping Timer
To achieve the desired 20000 users per sec you need to experiment with ramp-up and loop counts based on the output. The end goal should be that your transactions per sec graph should show 20000 or more at one point in time and at the same time the error should be zero then your server is able to handle 20000 users per sec. But average server response time is also an important metric to watch out for. Because no user wants a slow response from the server.
You can also consider the Concurrency thread group. This thread group is specifically designed to generate the desired number of concurrent users.
If you tried to do a Spike Test, like access application with 20 000 users for 1 second - your configuration is fine and and application failed the test so you can raise an issue.
However I'm under impression that your test is kind of short and doesn't tell the full story
20 000 users will produce 20 000 requests per second only if your application response time is 1 second sharp. If your application response time is 0.5 seconds - the load will be 40 000 requests per second, if response time is 2 seconds - the load will be 10 000 requests per second, if response time is 20 seconds - the load will be 1000 requests per second, etc. See What is the Relationship Between Users and Hits Per Second? article for more details if needed.
Normally you should increase the load gradually, this way you will have possibility to correlate increasing load with other metrics (response time, throughput, number of errors, etc) so I would recommend going for ramp-up -> plateau -> ramp-down setting and check the number of requests per second during this "plateau" phase
20000 virtual users is quite a number so make sure to follow JMeter Best Practices and it might be the case you will have to go for Distributed Testing