I'm executing multiple scripts for 1 hr in Non GUI Mode. I'm having couple of questions here.
Test Scripts:-
Script1
Script2
Script3
Number of Samples are differing with respect to the scenarios. I need equal distribution for all 3 scenarios. How to achieve this?
I'm saving all 3 scripts in one .jmx file( keeping 3 thread groups and assigning 20 Users per script). Is it correct approach.
I have added assertions for each request to check the response is valid or not.In loadrunner we will keep outside of Transactions but in Jmeter I'm not sure. Do we need to keep them during execution window.
I'm really looking forward to your suggestions.
It is difficult to achieve equal number of samples in all 3 scripts in Jmeter as response times for all 3 requests will be different. And there is no such thing as pacing in Jmeter as was there in Loadrunner. Only think time u can add as constant timer according to ur perception.
One reason of discrepancy could be the Ramp up time. You should give same ramp up period in all 3 thread groups. if ramp up time is different discrepancy in number of samples is expected.
I would be able to help if you could provide some more info like:
1. How much is the RT for each req
2. How much think time u r giving for each.
3. How much ramp up time u giving for each thread group.
4. How much startup delay u r giving.
Related
I have created a performance test script as below. I am running 4 Thread Groups in parallel (Even though there are 14 Thread Groups, only 4 are enabled and I am running only those enabled 4 Thread Groups). I have used default Thread Groups.
I have used Flow Control Action to simulate the user think time and set it as 3 seconds.
My requirement is to achieve Throughput as 6.6/seconds. What is the best way to achieve it? Also, does user think time creates any impact on Throughput?
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So basically throughput is the number of requests which JMeter was able to within test duration time frame. If you introduce artificial delay of 3 seconds - it will make the overall throughput lower
Other (and the main factor) is your application response time because JMeter waits for previous request to finish before executing new one so there could be 2 options:
The amount of threads in your Thread Groups is not sufficient to create the desired throughput. If this is the case - just add more threads (virtual users) in Thread Group(s)
The amount of threads in your Thread Groups is too high and you're getting higher throughput that you expect. If this is the case you can pause JMeter even more by adding Constant Throughput Timer and specifying the desired number of the requests per minute you want to have. If you need 6.6 requests per second it will mean 396 requests per minute
Also remember that JMeter itself must be able to send requests fast enough so make sure to follow JMeter Best Practices
My suggestion is to consider using the Arrivals Thread Group. This TG will allow you to configure the desire average throughput (ATP); the TG will instantiate the required threads needed to achieve the ATP goal.
As for the Think Time (TT), it should be a business requirement that emulates the pauses that users take when using the application in the real world.
We have some outside consultants whom are using JMeter to load test our application.
Our application consists of a web site and two mobile apps. Our usual usage pattern is having 20 users and x number of mobile users whom we track in our application. Our pattern is usually steady but we have some times of the year where our load will be heavier. We do not get spikes from no activity to, say, 300 mobile users. It will rise fairly steadily and reliably. We want to make sure our scaling settings will be able to handle the next time when we start adding load.
Our partner has written scripts that go from 0 load (zero CPU %, zero bandwith use, zero SQL etc) to a massive load that then ends in about 15 minutes. THIS IS NOT OUR USAGE PATTERN. Our system cannot scale out without a little bit of time.
I told them our usage pattern but they are having trouble using JMeter to do a more steady and slow ramp up. I know very little about the tool but I imagine this is a pretty typical use case.
Any tips or ideas that could send us in the correct direction, or is JMeter just not suited to this task?
It shouldn't be rocket science, for normal JMeter Thread Group ramping up from 0 to 300 users in 15 minutes looks like:
So it's sufficient to change Ramp-up period to 1800 in order to decrease users arrival rate by factor of 2x.
For Custom Thread Groups it's even easier because you can see visually users arrival chart:
So it's a matter of simple arithmetic, it shouldn't be that hard.
Check out JMeter Ramp-Up - The Ultimate Guide for more details if needed.
I like to use the Throughput Shaping Timer JMeter plugin for this. You can specify a duration of the rampup, full load, rampdown, etc. It's pretty flexible. I like to make these values parameters so I can pass in different values from the commandline at for flexibility without having to open and edit the script between differing test runs.
https://jmeter-plugins.org/wiki/ThroughputShapingTimer/
I am completely new to Performance testing and JMeter and hence my question may sound silly to some people.
We have identified some flows of an application and they are like:- Login, SignUp, Perform Transaction. Basically, we are trying to test our API's performance so we have used HTTP Request Sampler heavily. If I have scripted all these flows in JMeter, how can achieve answers to following
How can we decide the benchmark of this system? There is no one in organisation who can help with numbers right now and we have to identify number of users beyond which our system can crash.
For Example, if we say that 1,00,000 users are expected to visit our website in one hour's time then how can we execute this in JMeter? Should Forever loop be used with 3600 seconds(60 mins) of RampUp OR should I go ahead with Number of Threads as 1,00,000 RampUp ask 3600 and Loop Count as 1? What is the ideal way to test this?
What has been done till now?
1. We use to run above mentioned flows with Loop Count as 1. However, as per my knowledge, it's completely based on how much RampUp time I give and JMeter will decide accordingly how many threads it require in parallel to complete the task. Results were not helpful in our case as there was not much load to system.
2. Then, we changed the approach and tried Loop Count as Forever for some 100 users and ran the test for a duration of 10 minutes. After continuing with such test for sometime, we got higher Standard Deviation in JMeter's Summary Report which was fixed by tuning our DB and applying some indexes. We continued this way but I am still confused whether this can really simulate realistic scenario.
Thanks in advance!
Please refer my answer and comments to the similar question below:
performance-testing-in-production-environment-using-jmeter
I'm trying to create a jmeter script that sends 2 http requests(each with a different path). I managed to get it to send the requests randomly, but i also need it to send each request exactly 50% of the time. any ideas?
Divide your requests into 2 separate Thread Groups. Set identical number of Threads and Loops in each Thread Group
Put 2 requests under the same Thread Group. Add Throughput Controller as a child of each request and either set the same "Total Executions" value for both Throughput Controllers or use 50.0 value in "Percent executions" mode.
See Running JMeter Samplers with Defined Percentage Probability article for detailed information on above approaches and for more complex distribution scenarios.
Option 1: Math
Run a large number of users or a large number of times, picking randomly. On average, it's 50% of the time. Easiest to do, but not exact.
Option 2: Alternate
Use a variable to alternate a single thread back and forth over the course of multiple loops. I assume you have some sort of If Controller that you're using to split them. In your condition, use "${alternating_variable}"=="1". Then use a Beanshell Postprocessor to switch its value: vars.put(alternating_variable,2);. Obviously, you'll need the reverse for the other HTTP Request (both the If and the Beanshell). A little involved, and requires a thread to loop multiple times.
Option 3: Determined by Thread Number
Inside your If, use ${__threadNum}%2!=0 and ${__threadNum}%2==0. This gets the number of the thread, divides by 2, and compares the remainder to 0. Any even-numbered thread will go into one If and any odd thread will go into the other. Easy now that it's generated, but requires multiple threads. Also not necessarily easy to understand.
Apply 2 throughput controller and place your first http request into first throughput controller and 2nd request in other controller. Now change the mode to Percent Executions and pass 50 in throughput textbox
Please refer this
link for more details
I am not able to find out the specific answer of how to calculate the number of threads for running a load test in JMeter?
How to identify the loop count ?
Is there any formula?
What are the parameters to consider for the calculation?
Say if you want to fire 100 request to server with 2 tps. then your threas properties should be like below:
Number of threads(users) :2.
Ramp up period: 100
Loop count :50
Based on above example.Please find below explaination.
• Number of Threads (N): Sets the number of threads the JMeter will use to execute our test plan. We must know that each thread will execute the whole test plan, which effectively utilizes the number of users that could use the tested service at any given time simultaneously.
• Ramp-Up Period R: Specifies how much time (in seconds) it will take for JMeter to start all the threads (simultaneous user connections). If the number of users is 5 and the ramp-up time is 10 seconds, then each thread will be started in a 2 second delayed interval. We need to be careful when setting this value, because if the value is too high the first thread will already finish processing the whole test plan before the second thread will even begin. This is important because that would effectively reduce the number of concurrent users using the testing server application at any given time. But the ramp-up period also needs to be high enough to avoid starting all of the thread at a single time, which could overload the target application.
• Loop Count (L): How many times each thread group will loop through all configured elements belonging to that thread group.
Hope it helps!