I am running a Jmeter load test for my application. I can successfully run the test to login-search-logout for 500 users with a ramp-up of 150. I am unable to run a test for any higher number of users ( 800 users with 240 ramp-up or 1000 with 300) I tried increasing the ramp-up time too. I dont see any system errors nor do I see any connection pool errors. Even Jmeter log looks normal. However, my test gets killed after a number of users is reached ( for 800 user, the test kill at 800 ). Any suggestions as to what I can check for? Thank you
As mentioned by CodeChimp and tried by you already, first to check is the heap size.
And also verify with lesser number of users.
You could also try to start two jmeter instances (probably with different user id range?) instead of one.
If the problem still persists, there could be two possibilities:
1. Unable to send so many requests from client side.
2. Server(being tested) is unable to handle that many requests.
To verify if the problem is with client:
Check the ulimit value on the amazon instance. "ulimit -a".
Specifically, check if the nproc value (ulimit -n) is higher than the max number of threads(users) you are trying to connect from the amazon instance.
Check if the client has ramped up the required number of users:
(In the case of jmeter, the number of threads is same as number of users.)
ps uH p | wc -l (This should be equal to the number of users you have given in the script)
The jmeter script itself might be logging out the users by the time other users are ramped up. Better to verify by running the thread group in a loop. (May be loop forever?).
Make sure your script is configured to "continue" after an error occurs. I think this can be configured at thread group level.
Related
I am trying to perform load testing on NodeJS application, the application allows single login session for an user.
But, when i tried to perform load testing with 8 concurrent users (using the same credentials) it was working.
So, i tried to create 10 user IDs and added to the CSV file and configured "CSV Data Set Config" and "HTTP Cookie Manager" and set the "Number of Threads" to 80.
When I run the test plan, the credentials are not passed to each sessions equally. for example, userID1 is passed 3 times and userID2 is passed 7 times.
Can you clarify the reason for this behaviour and how to run the threads with each credentials (8 sessions for every credentials) from csv file ?
Make sure you have set the options for CSV Data Set Config as All Threads for Sharing mode and provide a minimum ramp-up period (say 1 second) so that each thread reads the CSV lines properly without clashes.
For the question why it worked 8 concurrent sessions , where you allow only single session from UI - My guess is you used the same credentials for all 8 users - so server might have considered it to be 8 parallel requests from the same user. It depends on how session is maintained. For eg., if it was with a session-cookie , then If you had 8 different cookies it might have failed. But this is only a guess, as I don't know how its done in your app.
JMeter acts as follows:
All threads are started within the bounds of the ramp-up period which you define in the Thread Group
Each thread starts executing Samplers upside down (or according to the Logic Controllers) as fast as it can
JMeter waits for previous request to finish before starting next Sampler
When the thread doesn't have any more Samplers to execute and loops to iterate - it's being shut down
If you observe the situation when this or that logic was used more than another, most probably the response time for the login was less.
You might want to check the size of the response as for successful requests it should be more or less the same and it might be the situation when the server returns HTTP Status code 200 but response body contains errors, to wit some logins may fail silently, if this is the case it makes sense to add a Response Assertion to add some extra checks, i.e. presence of "Welcome" text or "Logout" link or absence of "error" text or whatever.
What i could understand is you are trying to achieve 8 session each user.in order to do that you have to make 10 concurrent threads and 8 iterations that is how you will achieve 8 session for every credentials.
But of your requirement is to run 80 concurrent users then try to make 80 users and run the test.
I'm trying to test my https endpoint with JMeter. I want to make at least 10000 requests per second, but when I set the number of threads to 10000 I get way less rps, around 500.
I've tried setting the number of threads to 1000 and 100, surprisingly I get this same number of rps. I'm using HTTP Sampler and "use Keep-Alive" is set to true. When I look in the statistics I see that when using 100 threads, it makes use of Keep-Alive and connect_time is around 100 ms, but when the number of threads is higher connect_time grows, it's like it stops reusing the connections.
I know this isn't a server issue, because I've tried testing that same endpoint with Yandex.Tank and phantom and it can easily maintain 10 000 requests per second, the problem is it can't use response data to make furhter requests, that's why I have to use JMeter for this task.
This can be done by using "Stepping thread group". It will allow you to send 10000 request per second upto specified time. Refer below image.
Stepping Thread Group
Download jar from below link.
https://jmeter-plugins.org/wiki/SteppingThreadGroup/
I hope you are trying to achieve this using one machine. Try with multiple machine or jmeter distributed mode.
https://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf
https://www.blazemeter.com/blog/how-to-perform-distributed-testing-in-jmeter/
https://blazemeter.com/blog/3-common-issues-when-running-jmeter-scripts-and-how-solve-them/
I am assuming that it is the issue with machine which is not able to generate that much load. Usually, i have use max 300 threads per machine but it depend on the machine config. Just check if the machine is having issue and multiple machine is able to generate more load, considering server is not having any issue.
Hope this helps.
Update:-Usually 200-500 can be handled my modern machines.
Please check the below link to have some more info:-
1.How do threads and number of iterations impact test and what is JMeter’s max. thread limit
2.https://www.blazemeter.com/blog/what%e2%80%99s-the-max-number-of-users-you-can-test-on-jmeter/ .
I have recorded a script against the application that I want to test. Now, I am having a hard time arriving at the decision that what is that number that the application will run without any issue and to find out the max number of users. Here is what I have done-
I have run the Jmeter script for 10, 50, 100, 150 users
Until 50 users, it runs like a charm. After about 80 users the throughput starts to come down and some samples do not show up in Aggregate Report.
I see heap memory problems in my console for about 150 users over period of time. Is it the application problem or my machine problem?
Do you have an article where I could read about how to come to a conclusion about THE number?
UPDATE- after increasing the heap size, it is running smoothly for 100 users. I am even more confused now
Thank you
The problem can be anywhere!
Server Performance Metrics collector:
First you need an agent running in the application server to monitor the server performance while you are running the test.
This link will give you an idea about the set up.
JMeter Best Practices:
I think that you are running your test in GUI mode with listeners. Most likely the problem is with your machine/your test. Ensure that you follow this.
Samples not showing in aggregate Report:
You already asked a question on this in SO. Do not select 'Successes' in the Log/display only section of the listener while writing the results in the jtl. It will not write the failed requests details. You might need all the results. Once the jtl is created, you can always filter 'Success' only results as and when you want.
I was using non-GUI mode to perform a remote load testing with Jmeter from master server (Linux) to 5 slave servers (Linux). 5x "n" users have been run, "n" users on each server.
The results have been written to master server.
There are samples from all servers in results file but they relate to the number of active users from particular servers ("n") and not from all servers (5x "n").
There are no information in the result file about the real number of active users on all the servers.
As a result, a maximum number of active users is "n" on generated graphs which does not reflect the real load (5x "n" users).
Has anyone got a similar problem?
Is there anything I can do to correct the results already gathered?
Should I change any JMeter parameter to get the correct results in the next run?
Short Answer:
This is normal and no, there's nothing in JMeter you can do to fix it.
Long Answer:
Each Load Generator creates a number of threads n: the threads will be numbered 1-n. When the Controller collects all of the information, it sees 5 results for Thread 1, 5 results for Thread 2, ... Thread n. The Controller has no way of knowing that each of them are 5 separate concurrent threads and not just the same thread 5 sequential times.
Fixing it:
It depends on what you mean by a maximum number of active users is "n" on generated graphs. If this is something inside JMeter, then no, you can't fix it.
If it's a report-generator that you have created yourself, then yes, you can fix it by passing in the number of load generators.
I'm running a distributed test using 5 JMeter clients (slaves). Each client is set to run 50 users. At the conclusion of the test I generate a series of graphs from the resulting JTL along with a SynthesisReport. The SynthesisReport details 250 samples for each request, as you'd expect, however the TimeVsThreads and the ThreadsStateOverTime peak at 50 users, as if they were showing the results from just one of the clients.
I've confirmed that the jmeter.properties files for each client are the same as I suspected that it was possibly an issue with the each clients results file configuration and settings for saveservice.
I can't imagine this is by design, has anyone experienced something similar and if so how did was it solved?
As per documentation:
http://jmeter-plugins.org/wiki/ActiveThreadsOverTime/
Just name your thread groups using a unique id for each generator ( hostname or a property you pass to injector and use with function __P if you have more than 1 injector per host) and it will work fine.
This is normal for Jmeter Distributed Testing.
The reason this occurs is that each load generator separately starts User Threads 1-50, so when cmdrunner runs, it sees 1 responses from each User Thread 1 (5 total), but can't differentiate between them.
If you're using a custom reporter tool (that wraps cmdrunner), you can multiply your peak users by your load generators to display a more accurate number at the top of your Report. But as long as you're calling cmdrunner, you won't be able to see the actual number of users on your graphs.
This is normal behavior of JMeter. 5 clients will run each 50 threads.
Open JMeter and run on all remote hosts, and check Active threads each will run 50 threads.