What is the difference between "threads" and "Samples" in jmeter? - jmeter

Since I am a new to Jmeter,I have a little confusion regarding the difference between "Threads" and "Samples" in the Jmeter.
For example
Sometimes we have set the "Number of thread" as "200" and run the test in jmeter . We have seen the "samples" as "200" in the summary report.But sometimes for the same threads we are seeing "samples" as "160" or "samples" as "240"in the summary report.Image shows how we have set the threads in jmeter
Can any please explain the difference between threads and samples?Whether samples may differ from the thread?

Threads and Samples are totally different.
Threads represent the number of virtual users.
Samples represent the number of total requests made by users.
So suppose if you have a test plan with 5 requests.
If you run it with 1 user : you will have 1 thread and 5 samples.
If you run it with 10 users : you will have 10 threads and 50 samples.
Note, that number of threads or samples could vary because of machine/network/server limitations etc.
For example we ask JMeter to create 1000 threads but our machine is able to create only 100 threads at maximum (because of low CPU/ram)
Similarly, number of samples could be less/more because of network failure, test scenario etc.

If you have recorded a workflow, and want to run with single user you set the number of Threads as 1.Threads are nothing but # users.
The Samplers are the individual requests. If you group the samplers using "Transaction Controller" by checking "Generate parent Sample" all samplers under that Transaction Controller is treated as Single.

Related

Unable to Run Multiple Sampler APIs using Ultimate Thread Group in Jmeter

I have Four Sampler APIs : 3 GET APIs and One Put API and want to do spike test using UltimateThread Group. So i created an ultimate thread Group and added the four APIS under them.I have given split 25 user within different hold time and ramp-up time as per below image. But whenever i am running it is only running the first two API sampler and the last two apis are not running at all.
another issue is that among the two running APIs, only one API is running with 25 threads and other one with only 5 threads ....but its should run 25 threads for each APIs .Kindly let me know whats the issue and as i am new to Jmeter ..trying to resolve the issue but unable to do troubleshoot it.
APIs Sampler
Ultimate thread Group Set-up : AGGregate Report : TREE REPORT
AS YOU CAN SEE THERE FOUR APIS BUT IT HAS EXECUTED ONLY TWO APIS, AMONG THE TWO OF THEM,
ONE HAS 25 THREADS
ONE HAS 5 THREADS
BUT as per my understanding it should run all four APIs with 25 threads for each API sampler...
Each user in the Ultimate Thread Group runs for 10 seconds maximum
Average Response Times for first 2 Samplers are 12 and 32 seconds correspondingly
The threads shut down times and reasons can be seen in jmeter.log file
Assuming above 2 factors the threads are simply being shut down before starting 3rd and further Samplers.
You need to either:
Set "Hold load for" to a bigger value, i.e. a couple of minutes
Or introduce reasonable response timeouts under "Advanced" tab of the HTTP Request Defaults so if the response time exceeds acceptable

Concurrent Thread and Ultimate Thread Group and Performance Bench mark

While understanding concept of Concurrent thread and ultimate thread group, i am confused to understand result of summary/aggregate report while running concurrent thread or ultimate thread group .For example if i have 200 user and ramp up time 60 sec then i didn't see all sampler request as 200 sample after completing execution successfully but only few sampler request have 200 sample. when i use normal thread group then i always got thread count same for each sampler request after completing execution .
for realistic load testing with more user , could you please suggest me which thread group should prefer.
Could you please provide valuable guidance with some valuable link/book and also share me standard performance bench mark criteria or key parameter detail while doing load testing.(if any bench load parameter value does not meet standard then we can say that there is a performance issue)
Thanks for giving valuable time in advance.
Thanks
amit
This is due to the fact that:
Your application response time is too high
Your test duration is too low
For example I can see response times > 80 seconds:
it means that if a single virtual user has cumulative response time for 2 samplers > 160 seconds and the test duration is 120 seconds it will not be able to execute all the requests. Just increase your test duration to be i.e. 10 minutes and you should see more virtual users capable of executing all the Samplers you defined in the test plan.
Also given first user is capable of executing all the requests successfully and in time it looks like that your application gets overloaded hence cannot respond fast enough when the number of concurrent users reaches some "critical threshold", you can add listeners like Active Threads Over Time and Response Times vs Threads, this way you will be able to correlate increasing load with the increasing response time.
If also makes sense to collect:
Baseline health metrics of your application (CPU, RAM, Network, Disk usage, etc.), it can be done using JMeter PerfMon Plugin.
Lower level details like slowest methods, largest objects, heaviest database queries, etc. This form of information can be obtained using profiling tools specific to your application programming language(s).

JMeter in distributed mode not all servers taken in consideration in the graphs

I'm doing a performance test with some instances of JMeter and something came up when I was analyzing the JMeter report.
To explain the context, I'm using 3 JMeter servers and each one launches 200 threads, the test runs for 1 minute on non-GUI mode. During the execution, I can clearly see that I have 600 active threads (200 per server).
At the end, my CSV file shows all samples from the 3 servers (I can see this by enabling "hostname" on the jmeter.properties file).
My problem:
On the report I have the Dashboard, that shows the data of my 3 servers (the samples number is the same that the CSV lines, which has all 3 servers).
At the same time, when I see my Active threads over time graph, it only shows 200 (which for me is just 1 server).
JMeter doc says that on the Times vs Threads graph:
In distributed mode, this graph shows a horizontal axis the number of threads for 1 server. It's a current limitation
And this only for this graph, which means that the other should display the information from all 3 servers...
Am I missing something? Why does the Active threads over time graph shows only 200 threads, even if I saw 600 on my console screen?
Follow Active Threads Over Time Listener notice you need to change thread group names:
If you are doing distributed tests, you should name the thread groups like this in your test plan:
${__machineName()}_My Threadgroup name

Test Duration when we execute JMeter Test with 1 user and more

I have concerns about the duration of executing JMeter Test Scenario.
In fact, if we execute a manual test with 1 user in 1 hour, we will find the same duration or little more with 3 user.
But, with JMeter test the duration with 1 user will be multiplied by the number of users.
During my scripting with this tool, I've noticed that JMeter always wait for the response of the request to pass to another request. It's like we have 1 user doing the work of multiple users.
Does any body have explanations about this issue?
Can we configure JMeter to perform like we have x users working in // ?
Depends how you are simulating users..
Each thread will wait for a response, but users are represented by different threads, and threads do not wait for each other to complete samples, they are independent.
If you are using threads in thread group to represent users (which is the intended usage), the threads will start concurrently and run independently, so you should see the 3 users complete in an hour, as with manual tests. If this is not happening, then you should check the resources used on the client running jmeter during single thread and multi thread runs. Or perhaps the bottleneck may be in the transport (ie bandwidth). All you have determined so far is that the bottleneck is prior to the server, you need to determine where it is.
It is also possible you have created a thread group for each simulated user. If this is the case you can set the groups to run consecutive or concurrent. The setting for this is on the Test Plan element at the very top of the tree. You want concurrent, so untick 'Run Thread Groups Consecutively'.
So it comes down to how you are simulating users.
In ideal world if you test your site with 1 user all set of actions completes in 1 hour. If you add another 100 users - still one hour, 1000 users - 1 hour.
In real world server response time increases with the number of concurrent threads. But it isn't something like
1 user - average response time 10ms
2 users - 20 ms
3 users - 30 ms
But
1 user - 10 ms
2 users - 10 ms
3 users - 10 ms
10 users - 11 ms
The whole idea of the performance testing is to determine the upper limit of server load and identify bottlenecks and issues which happen under the load so you could state something like: using this hardware and this configuration the application is capable of serving 500 concurrent users without serious delays. In case of 1000 users response time will be more than 15 seconds.
In regards to threads, JMeter thread starts, executes samples one by one for defined amount of loops and exits. It is applicable for each thread which are independent. Basic throughput control can be done via ramp-up and loops count (see Thread Group documentation on details)
More advanced load scenario definition can be done using following Test Elements:
Ultimate Thread Group - which provides easy visual way of defining ramp-up, ramp-down and time to hold the load
Constant Throughput Timer - to set desired load rate in requests per minute
Synchronizing Timer - to pause all the test threads and release them at the same moment to produce a "spike"

Simultaneous users for web load tests in JMeter?

I have made some PHP scripts and I want to test the response time for simultaneous users in JMeter. This scripts are run in a very short time (50 miliseconds).
What I would like to do is to simulate a load test from 1 to 50 of users where each user (thread) repeats the request for an unlimited period. So first we will have 1 user, after 2 simultaneous users, after 3 ... and so on.
I am trying to do it but with I have is response times where it is evident that there is no simultaneous request.
With HP loaddrunner we can define number of iterations for each thread, is this possible in JMeter?
You can possibly use these 2 thread group implementations
Stepping Thread Group
Ultimate Thread Group
from Jmeter Plugins package, that let you set load-increase for you scenario as you want.
As well you can also look onto Synchronizing Timer if you want "better" concurrency - but this is rather stress-testcase than load one.

Resources