Is that possible to execute 10000 concurrent user in jmeter?
If so how?
What should be the ram-up time for this scenario??
JMeter's limit of virtual users per Thread Group is very high, to be precise it's 2,147,483,647
The question is: do you have good enough hardware to simulate 10 000 users from a single machine. The process of checking it looks as follows:
Make sure to follow JMeter Best Practices
Set up monitoring of the CPU, RAM, Network, Swap, Disk usage on JMeter side, it can be done using JMeter PerfMon Plugin
Start with 1 virtual user and gradually increase the load at the same time looking into CPU, RAM, etc. usage. Here there could be 2 options:
you will be able to reach 10 000 users without issues, if this is the case you should be good to go
you will run out of resources earlier, in this case look into i.e. Active Threads Over Time listener to see how many users you were able to simulate from this machine and extrapolate the value to determine how many load generators you will need for 10 000 users using Distributed Testing approach
There is no golden rule for ramp-up time calculation, the good practice is to onboard the users gradually, this way you will be able to correlate increasing load with increasing response time, increasing number of errors, etc. Adding 3 users each second so in 1 hour you will have 10 000 seems a valid starting point to me.
Related
I'm using jmeter to generate a performance test, to keep things short and straight i read the initial data from a json file, i have a single thread group in which after reading the data i randomize certain values to prevent data duplication when i need it, then i'm passing the final data to the endpoint using variables, this will end up in a json body that is recieved by the endpoint and it will basically generate a new transaction in the database. Also i added a constant timer to add a 7 seconds delay between requests, with a test duration of 10 minutes and no ramp up, i calculated the requests per second like this:
1 minute has 60 seconds and i have a delay of 7 seconds per request then it's logical to say that every minute i'm sending approximately 8.5 requests per minute, this is my calculation (60/7) = 8.5 now if the test lasts for 10 minutes then i multiply (8.5*10) = 85 giving me a total of 85 transactions in 10 minutes, so i should be able to see that exact same amount of transactions created in the database after the test completes.
This is true when i'm running 10-20-40 users, after the load test run i query the db and i get the exact same number of transaction however, as i increase the users in the thread group this doesn't happen anymore, for example if i set 1000 users i should be able to generate 8500 transactions in 10 minutes, but this is not the case, the db only creates around 5.1k transactions.
What is happening, what is wrong? Why it initially works as expected and as i increase the users it doesn't? I can provide more information if needed. Please help.
There could be 2 possible reasons for this:
You discovered your application bottleneck. When you add more users the application response time increases therefore throughput decreases. There is a term called saturation point which stands for the maximum performance of the system, if you go beyond this point - the system will respond slower and you will get less TPS than initially. From the application under test side you should take a look into the following areas:
It might be the case your application simply lacks resources (CPU, RAM, Network, etc.), make sure that it has enough headroom to operate using i.e. JMeter PerfMon Plugin
Your application middleware (application server, database, load balancer, etc.) are not properly set up for the high loads. Identify your application infrastructure stack and make sure to follow performance tuning guidelines for each component
It is also possible that your application code needs optimization, you can detect the most time/resource consuming functions, largest objects, slowest DB queries, idle times, etc. using profiling tools
JMeter is not sending requests fast enough
Just like for the application under test check that JMeter machine(s) have enough resources (CPU, RAM, etc.)
Make sure to follow JMeter Best Practices
Consider going for Distributed Testing
Can you please check once CPU and Memory utilization(RAM and java heap utilization) of jmeter load generator while running jemter for 1000 users? If it is higher or reaching to max then it may affect requests/sec. Also just to confirm requests/sec from Jmeter side, can you please add listener in Jmeter script to track Hit/sec or TPS?
This will also be true(8.5K requests in 10 mins test duration) if your API response time is 1 second and also you have provided enough ramp-up time for those 1000 users.
So possible reason is:
You did not provide enough ramp-up time for 1000 users.
Your API average response time is more than 1 second while you performing tests for 1000 users.
Possible workarounds:
First, try to measure the API response time for 1 user.
Then calculate accordingly that how many users you need to reach 8500 requests in 10 mins. Use this formula:
TPS* max response time in second
Give proper ramp-up time for 1000 users. Check this thread to understand how you should calculate ramp-up time.
Check that your load generator is able to generate 1000 users without any memory or health (i.e CPU usage) issues. If requires, try to use distributed architecture.
I have a business requirement wherein I want to simulate bandwidth related load tests targeting different bandwidth and in a steady manner.Like we have ultimate thread group in JMeter to control the users providing a steady ramp up at given time. Do we have any such element/plugin to control enormous bandwidth like 70 Gigabit/sec in JMeter or BlazeMeter?
"Bandwidth" is basically the sum of sent and received bytes so it mostly depends on request and response time.
If you have 1 megabyte request/response - in order to simulate 70 gigabit you will need to send 8750 requests at the same moment.
So I would recommend executing your test plan with 1 virtual user and 1 iteration and look into last 2 columns of the Aggregate Report listener
This is how many (kilo/mega/giga)bytes your test produces for the test duration. Divide it by test duration in seconds to find out the bandwidth "per second" and divide 70 gigabits by that number to find out how many requests you will need to make.
Most probably you will need to go for distributed testing (unless you have a single machine with 100-gigabit network adapter capable of running required amount of virtual users)
Do we need to adjust Throughput given by jmeter, to find out the actual tps of the system
For eg : I am getting 100 tps for concurrent 250 users. This ran for 10 hrs. Can I come to a conclusion like my software can handle 100 transactions per second. Or else do I need to do some adjustment and need to get a value. Why i am asking this because when load started, system will take sometime to perform in adequate level (warm up time). If so how to do this. Please help me to understand this.
By default JMeter sends requests as fast as it can, the main factor which are affecting TPS rate are:
number of threads (virtual users) - this you can define in Thread Group
your application response time - this is not something you can control
Ideally when you increase number of threads the number of TPS should increase by the same factor, i.e. if you have 250 users and getting 100 tps you should get 200 tps for 500 users. If this is not the case - these 500 users are beyond saturation point and your application bottleneck is somewhere between 250 and 500 users (if not earlier).
With regards to "warm up" time - the recommended approach of conducting the load is doing it gradually, this way you will allow your application to get prepared to increasing load, warm up caches, let JIT compiler/optimizer to go their work, etc. Moreover this way you will be able to correlate the increasing load with increasing/decreasing throughput, response time, number of errors, etc. while having 250 users released at once doesn't tell the full story. See
The system warmup period varies from one system to the other. Warm up period is where configurations are cached, different libraries are initialized (eg. Builder.init()) and other initial functions that usually don't happen for subsequent calls. If you study results of the load test, there is a slow period at the very beginning. For most systems, it could be as small as 5 to 10 minutes. These values could be even negligible if the test is as long as 10 hours. But then again, average calculation can be effected if the results give extremely low values at the start (it always depend on the jump from initial warming up period to normal operations).
As per jmeter configurations this thread may explain the configuration. How to exclude warmup time from JMeter summary?
Hellow
Which is the maximun number of virtual users that can be testet at a Jmeter distributed test? Is it possible to reach one million of virtual users?
Thak you.
It depends on may factors, technically the limit on JMeter end is very high (I think it should be 231 − 1 or 2 147 483 647 virtual users)
Nature of your application: use cases, is it more about consuming ore creating content, average request and response size, response time, etc.
Nature of your test: again, request and response size, need to use pre/post processors and assertions
Hardware specifications of your load generators
Number of load generators
So I would recommend the following approach:
Start with a single JMeter instance
Make sure you have optimal JMeter configuration and amended your test according to JMeter best practices
Make sure you have monitoring of baseline OS health metrics on that machine
Start with 1 virtual users and gradually increase the number of running users until you start running out of hardware resources (CPU or RAM or Network or Disk IO will be close to maximum)
Mind the number of active users at this stage (you can use i.e. Active Threads Over Time listener) - this is how many users you can simulate for particularly that test scenario. Note, the number might be different for other application or other test scenario.
Multiply the number you get by the number of the load generators you have - if there is > 1M - you are good to go.
If you won't be able to simulate that many users there is a workaround, but personally I don't really like it. The idea is that real users don't hammer application non-stop, they need some time to "think" between actions. Normally you should be simulating these "think times"using JMeter Timers. But if you lack load generators you can consider the following:
Given 1 virtual user needs 15 seconds to think between operations and response time of your application is 5 seconds, it means that each user will be able to execute only 3 requests per minute. So 1M of users will execute 3M requests per minute which gives us 50 000 requests per second which is also high, but more likely to be achievable.
while performance testing an application, i was unable to proceed further of handling large number of threads using JMeter, so, i would like to know the max number of threads that are allowed in Jmeter, Is jmeter capable of handling 1,50,000 threads?
There is no upper limit, it strongly depends on what your test is doing, what is response size, etc.
Also keep in mind that real users don't hammer the application nonstop, they need some time to "think" between operations plus they have to wait for response before they start "thinking" about next action.
For example, given users "think" for 10 seconds and response time is 2 seconds it means that each virtual user will execute 5 requests per minute.
In above scenario 1 50 000 users will execute 7 50 000 requests per minute - which is 12 500 requests per second - > 10x times less users to simulate.
So:
first of all make sure that your JMeter configuration is optimal, default settings are good for tests development and debugging but not very good for the load test execution. You need to
tune Java parameters (Heap size, GC, etc.)
disable all listeners
make sure that you have only those assertions and post processors which are absolutely required
you store only those metrics you need and you don't save any excessive results, especially response data
See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article for above points comprehensive explanation and some more tips
Even given you apply above tweaks I don't think you'll be able to conduct the load of 1 50 000 users from a single host (unless you have a supercomputer in your QA Lab) so I expect you'll need to consider JMeter Distributed Testing when one master machine orchestrates multiple load generators aka "slaves" acting as a single instance - this way you will be able to increase the load by factor equal to number of slaves
Yes Of course, it depends a lot on the machine running Jmeter, but if mileage counts I can give you some hints.
JMeter allows you to run multiple processes in the same box, and it's usually pretty reliable generating up to 200-300 threads per JMeter instance. If you need more than that, I'd recommend using multiple JMeter instances
Use below link for better description how Jmeter can handle 1,50,000 threads of multiple instances
https://blazemeter.com/blog/how-run-load-test-50k-concurrent-users