I tried 500 users with 100 Ramp up/Hatch rate per seconds. The load generated was monitored for tomcat using JConsole.
For same set of request JMeter requests were using 100-80% CPU usage where as locust was around 60-30% CPU usage.
I wanted to get the bottle-neck for the server how many maximum request it can server.
Can i simulate same using locust I tried setting min-max-wait to 0 but no use.
I tried 1000 concurrent users in locust but the RPS was same. Using Jmeter i was able to find that the RPS more that this is supported by my server. RPS value by locust gives 160 where as JMeter gives 250 which is huge difference.
Locust can't generate load on my tomcat server.
Can any one please help me to explain what i am missing to simulate the load similar to JMeter.
Thanks in Advance
Looking into Locust Issue # 464 my expectation is that Locust doesn't consider so called "embedded resources", to wit images, scripts, styles, fonts, etc. which actually generate lots of extra asynchronous requests and produce the main load.
Check what exact requests are being sent by Locust and JMeter using either Tomcat access log or a sniffer tool like Wireshark - the requests should be the same in terms of amount/nature. Ideally they also should match the requests which are being sent by real user using real browser.
Follow #aldenpeterson-wf suggestion of setting min_wait and max_wait with 1000 ms to get rate per second
Have you looked at min and max wait? Setting them to both be 1000 (ms) would result in locust making 1 request/sec per client which seems to be exactly what you are looking to do.
Related
We are using Apache JMeter (latest version) for load tests of a high performance API. JMeter response time is too high in comparison to SoapUI (free version). SoapUI gets an average response time of 200 milliseconds while JMeter reports at least 1200 milliseconds for the same API, testing from the same machine.
This affects the calculated throughput in each application. On SoapUI I can get up to 850 tps for said API, no errors. On JMeter I'm getting 190 tps at most, no matter what.
Tests are running with 200 threads, for 120 seconds.
I have already tweaked jvm/jmeter heap mmemory, garbage collector, and so on to no avail. Nothing changes. Can anyone help with this? Thank you.
Edit: Added test setup sample images
JMeter Thread Group and Test Plan Overall Setup
SoapUI TestSuite and Overall Setup
Edit 2: test results
Jmeter test results 1105 ms
SoapUi test results 263 ms
I can only think of one possible reason: you're running JMeter in GUI mode and this is not something you should be doing as JMeter isn't capable of sending requests fast enough, JMeter's GUI mode is designed for tests development and debugging.
Your images don't tell the full story as JMeter test results are missing. It would be also beneficial to generate a HTML Reporting Dashboard and see the correlation between increasing number of virtual users and response time/throughput.
And last but not the least, JMeter can be executed in Distributed Mode and in SoapUI it's available only for LoadUI Pro
I have come across a situation where I can`t decide what scenario would be best. I have written my test in JMeter as follows:
I have one test plan that runs test in consecutive.
I have 4 thread groups and each thread group has the following properties:
No of threads: 8000
Ramp-up period: 60 sec
Loop count : 10
Same user on each iteration: true
I was having connection error, connection time out error.
So, to make it work , when I test from localhost (same machine), I have to enter a response time out of 1800000 ms, whereas when I do the same test on a remote server, I have to enter the response time out of 3600000 ms.
Can someone please advise :
Is it a good idea to include response time? Is there any other issue I should look for instead of including a response time? Is it an alert for other issue?
Can I improve the test without using response time?
First of all, a couple of questions to you:
Do you really think that the real user will really wait for 1 hour for getting the response from the application?
How did you come to this 8000 users?
Now recommendations:
Never have JMeter and the system under test on the same machine, they are both resource intensive and they will start struggling for CPU cycles, memory pages, etc. and you won't be able to tell whether JMeter is not capable of sending requests fast enough or your application cannot respond properly
Follow JMeter Best Practices
Although the number of virtual users you can simulate with JMeter is very high it's limited by the machine/operating system resources so make sure JMeter has enough headroom to operate in terms of CPU, RAM, Network and Disk IO. The metrics can be checked using JMeter PerfMon Plugin. Once you notice that any of monitored metrics start exceeding i.e. 80% of total available capacity - mention how many users are online and this is how many you can simulate from this machine for this test. If you need more - go for Distributed Testing
The same for the system under test, if you hit the resources consumption limits you need either to upgrade the hardware or to deploy your application in clustered mode (if it supports such a mode)
I am using JMeter to test performance of an API Endpoint.
The Number of Threads(users) applied is 100.
When I execute the test for the very first time the average response time is 35345 ms.
for all the following tests with the same number of threads on the same API Endpoint the average response time is somewhere around 4705 ms.
what is the reason for such a big difference in average response time ?
Does JMeter cache any files on the first test and use the same cached files on all following tests ?
If yes how do I avoid this ?
I am new to JMeter any help in this regards will be much appreciated.
JMeter doesn't cache anything when it comes to API testing, it has HTTP Cache Manager which can represent browser cache when it comes to handling embedded resources like images, scripts and styles when it comes to web testing (mimicking HTTP requests send by real browsers) but it is not your case.
So my expectation is that it is something on your application under test side, i.e. it needs to "warm up" its own caches and first few requests are processed longer due to lazy initialization of components on the first access.
So my recommendations are:
Make sure you use reasonable ramp-up period so the load increases gradually, this way you will be able to correlate main metrics like response time, hits per second, etc. with the increased load
Monitor your application under test health from OS perspective, i.e. CPU, RAM, Swap consumption, etc. You can use JMeter PerfMon Plugin for that
Use profiling tools which are relevant for your application to detect "heavy" methods, long-running DB queries, etc.
I am using JMeter 2.11 for simulate 10000 thread users. I use CSV data set config to simulate 10000 user load and take 2-3 HTTP request in my test plan. All 10000 user load applied successfully by Jmeter but main problem is when my last request goes idle (as I put my Jmeter http request on home page of my website) all sessions goes timed out after some time.
I also configure "app pool's Idle time out=0" and my application's web config's "session state timeout is 20" (I also increase this time from 20 min but no way). Also Keep Alive is checked on all http request.
Please suggest me what to do for keep my last request's session alive.
JMeter threads are never idle, if thread doesn't have any more samplers to execute or no more loops to iterate it's being shut down. 10 000 users is quite a high load, it might be a problem with your JMeter configuration, i.e. it is not capable of generating and sustaining 10 000 threads. In 99% of cases jmeter.log file has enough troubleshooting information.
In the meantime few recommendations:
Upgrade to the latest version of JMeter (for the moment it's Apache JMeter 2.13)
Make sure you use 64-bit server JRE or JDK
Provide JMeter enough Java Heap space, by default it comes with 512Mb only which is not enough to simulate 10k users
Run JMeter test in non-GUI mode
Disable all Listeners during test run
See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure for above points explained and few more tips and tricks.
I am hitting two straightforward application URLs from jmeter using 100 threads via HTTP requests and using response assertion (200 OK). No cookie handler or nothing else at all used in jmeter.
Then using the same machines, I use apache wrk tool (developer asked so) with parameters set to -c100 and -t100. I get throughput difference. The throughput of my application is almost 60 to 80% shown higher via apache wrk compared to jmeter. How do I troubleshoot the difference? Developer is sticking to wrk but I want to resolve the difference. I can provide more info if required but I am not using anything special in jmeter as such.
From my experience wrk is faster, it can deliver as much as up to million requests from single node, quite beefy one though (32 cores with 2 network cards), and it's not apache BTW (while jmeter is). -t100 is probably overkill since your test machine probably has not that much cores. And regarding both throughput and latencies there is wrk2 tool which gives more correct results for general purpose web services (where the load is not coordinated)