I am completely new to Performance testing and JMeter and hence my question may sound silly to some people.
We have identified some flows of an application and they are like:- Login, SignUp, Perform Transaction. Basically, we are trying to test our API's performance so we have used HTTP Request Sampler heavily. If I have scripted all these flows in JMeter, how can achieve answers to following
How can we decide the benchmark of this system? There is no one in organisation who can help with numbers right now and we have to identify number of users beyond which our system can crash.
For Example, if we say that 1,00,000 users are expected to visit our website in one hour's time then how can we execute this in JMeter? Should Forever loop be used with 3600 seconds(60 mins) of RampUp OR should I go ahead with Number of Threads as 1,00,000 RampUp ask 3600 and Loop Count as 1? What is the ideal way to test this?
What has been done till now?
1. We use to run above mentioned flows with Loop Count as 1. However, as per my knowledge, it's completely based on how much RampUp time I give and JMeter will decide accordingly how many threads it require in parallel to complete the task. Results were not helpful in our case as there was not much load to system.
2. Then, we changed the approach and tried Loop Count as Forever for some 100 users and ran the test for a duration of 10 minutes. After continuing with such test for sometime, we got higher Standard Deviation in JMeter's Summary Report which was fixed by tuning our DB and applying some indexes. We continued this way but I am still confused whether this can really simulate realistic scenario.
Thanks in advance!
Please refer my answer and comments to the similar question below:
performance-testing-in-production-environment-using-jmeter
Related
Using JMeter I want to simulate 10000 users that will connect to a webservice. The users will not connect at the same time but they will access the service from 08:00am to 08:00pm.
I didn't find how to do that in JMeter.
In the real life the number of concurrent users will be random (we can't predict that)
So if i'll use Ramp-up period it will not work because the number of users will be increasing gradually, which i don't want to have.
Is there a way to do this scenario in JMeter ?
And another question, i'm using JMeter v5.2 and i can't find the option Scheduler in the ThreadGroup, while in many tutorials they show that option.
Given this statement:
In the real life the number of concurrent users will be random (we can't predict that)
I can only think about implementing a Stress Test.
"Normal" Load Testing is the process of putting the system under anticipated load and if you cannot figure out the current or expected load pattern you can only try to evenly distribute 10 000 users across 12 hours which gives 833 users per hour. There could be some "spikes" at the beginning of the day and after the lunch, but without knowing the business logic of your application it's hard to give a recommendation apart from generic Little's Law approach.
So you can try to identify the limits of your system you can start with 1 virtual user and gradually increase the load up to 10 000 or even more, until the errors start occurring or response time starts exceeding acceptable thresholds, whatever comes the first.
This way you will be able to tell what is the saturation point and where is the first bottleneck
while performance testing an application, i was unable to proceed further of handling large number of threads using JMeter, so, i would like to know the max number of threads that are allowed in Jmeter, Is jmeter capable of handling 1,50,000 threads?
There is no upper limit, it strongly depends on what your test is doing, what is response size, etc.
Also keep in mind that real users don't hammer the application nonstop, they need some time to "think" between operations plus they have to wait for response before they start "thinking" about next action.
For example, given users "think" for 10 seconds and response time is 2 seconds it means that each virtual user will execute 5 requests per minute.
In above scenario 1 50 000 users will execute 7 50 000 requests per minute - which is 12 500 requests per second - > 10x times less users to simulate.
So:
first of all make sure that your JMeter configuration is optimal, default settings are good for tests development and debugging but not very good for the load test execution. You need to
tune Java parameters (Heap size, GC, etc.)
disable all listeners
make sure that you have only those assertions and post processors which are absolutely required
you store only those metrics you need and you don't save any excessive results, especially response data
See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article for above points comprehensive explanation and some more tips
Even given you apply above tweaks I don't think you'll be able to conduct the load of 1 50 000 users from a single host (unless you have a supercomputer in your QA Lab) so I expect you'll need to consider JMeter Distributed Testing when one master machine orchestrates multiple load generators aka "slaves" acting as a single instance - this way you will be able to increase the load by factor equal to number of slaves
Yes Of course, it depends a lot on the machine running Jmeter, but if mileage counts I can give you some hints.
JMeter allows you to run multiple processes in the same box, and it's usually pretty reliable generating up to 200-300 threads per JMeter instance. If you need more than that, I'd recommend using multiple JMeter instances
Use below link for better description how Jmeter can handle 1,50,000 threads of multiple instances
https://blazemeter.com/blog/how-run-load-test-50k-concurrent-users
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have created a test plan using 100 threads. how can we conclude that the web server can handle load? which are the factors we can be taken for the load test.
I personally think you need to define your own metrics for your test plan to get a load test pass.
Typical metrics I would use.
Each response should come back in less than 250 ms. (Adjust to what your customer would expect)
All responses should come back with a non error response.
The server should be in a 'good state' after the load test. (Check memory, threads,database connection leaks etc)
To many resources being consumed is also a bad sign. Database connections, memory , hard disk for log files. Define your own metrics here.
Successive 'soak tests' to compliment your load tests would also be a good idea.
Basically run the a smaller amount of jmeter tests every two hours (So the DBA's etc. don't complain) over the weekend and check on the Monday.
I would recommend to you to first clarify your concepts about performance testing and its types (like load test, stress test, soak test etc). You can refer to following blog to get basic understanding about performance testing and its types:
Load vs. Stress testing
http://www.testerlogic.com/performance-testing-types-concepts-issues/
Once you have a better understanding of concepts, you will be in better position to ask the right question. For now, you can focus on following points..
what is expected load on your web server (in normal and extreme scenarios!)
what is your acceptable criteria for response time, load time etc
Once you know these numbers, you can create a jmeter test which runs for a specific time span (say 1 hour) and no. of threads increase step-by-step (100 user in first 10 minutes, 200 users from 10-20 mins, 300 users from 20-30 mins and so on). (hint: you can use ramp-up period to achieve this scenario).
You can perform these tests and check the reports and compare the response time and other performance factors during first 10 minutes (when load was 100 users) and in last 10 minutes when load was maximum.
This is just to give you a high level idea. As i said before it will better if you first clarify basic performance testing concepts and then design/perform the actual testing.
Like the rjdkolb said you have to define your metrics, check what you require from your service/app.
It all depends what service you are working with - do you have some stable load on the server, or some peaks, do you think there will be like 100 users online or 10000 at once, do you need fast answers or just proper answers in reasonable time. Maybe business foresee that the load will be building gradually through next year and it will start with just 100 requests per minute but will finish with 1000 per sec?
If you think that, like mentioned in other answer, you need an answer in less than 250 ms, then gradually increase load to check how many users/requests you can handle to still have responses on time. And maybe you need answers for 1000 users working simultaneously - then try load like this and check do they have they answers and how fast are they coming back? A lot to think about, do you think?
Try to read a bit about types of performance testing - maybe here on soapui or this explanation of some metrics. A lot of texts on the internet can guide you in your way.
Have fun.
I am running a 1000 user test. and some of the flows have 25 users with the expected throughput as 0.000011574 per second.
The client is suggesting that I run it with about 1800 second think time.
Using Little's law I am getting the Think Time value to be 2160000.
I am suggesting that we just use 1 user and give a 600 second think time, even though calculations give me 86400 seconds think time. Since the flow has to be tested while under load.
What would be the correct approach? Go with client or Go with my assumption?
Let me know your valuable thoughts.
0.000011574 of what per second?
This reads like a requirement from a server admin and not from "the business."
My customer give me a traffic figure is 600 request/second and the RX(Mbps) is 30.
Please help me what is the suitable scenario test plan for this issue.
My customer and me is in different countries, so is the network effect to the result?
Many thanks on your pointers.
First of all, 600 requests/second rate isn't something which is recommended to be run from a single node.
You need to consider JMeter Remote Testing which assumes running the test from multiple JMeter instances. Make sure that you're following JMeter Performance and Tuning Tips guidelines while developing your test.
In order to achieve 600 requests/second rate, not more, not less you need to use Constant Throughput Timer