I've managed to make a simple test with JMeter and a maximum of 400 virtual threads before my laptop gets frozen. It has 8 gigabytes of RAM and a processor Intel core i5 with 2.4 GHz.
Then I´ve made two slave nodes by using Oracle Virtual Box, and a master node, all in my laptop. I´ve run the test in non-GUI mode and the maximum number of virtual threads I can run before my laptop gets frozen is also 400, by setting 200 as number of threads on the JMX file (200 for each slave makes 400 threads in all)
So if the maximum number of threads I can run is the same with a single machine and with two slaves and one master configuration Which is the sense of the node slave configuration? It has no advantages. What am I doing wrong? May it be due to the fact that the slave nodes are virtual instead of real ones?
By another side, the .jtl file I get for the 400 virtual threads weights 18 megabytes, and JMeter´s listeners can not read all its steps. The error message is
jmeter.save.CSVSaveService: Error parsing field 'bytes' at line 67515 java.lang.NumberFormatException: For input string: "text" "
So How can I see the complete results of a test with a big number of threads? Is it a problem of RAM lacking or a JMeter´s limitation?
There is no sense in using master-slave configuration given you run everything on a single host. Given you can simulate 400 virtual users using one machine you should be able to simulate 400 more by adding one more machine (given it will have the same or similar specifications) so it should be like this:
1 machine - 400
2 machines - 800
3 machines - 1200
etc.
But again, distributed testing goal is to scale JMeter when you have reached the limits of a single load generator machine and haven't delivered the required load. Running more than one JMeter Server (slave) on a single machine doesn't add any value.
Also make sure you're following recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article for tuning JMeter engines for maximum performance, I would expect much more than 400 virtual users out of Core i5 with 8 GB of RAM.
Related
We are attempting to stress test our system using Jmeter and we hit a wall with inconsistent throughput, so we installed the Jmeter plugin Concurrency Thread Group with the Throughput Shaping Timer. We're currently running it on a single endpoint, to make sure we can achieve a consistent throughput. we set:
Throughput Shaping Timer to 30 RPS for 60 seconds, which is scoped directly under the Concurrency Thread Group
Target Concurrency using the tstFeedback function as tstFeedback(tst,13,50,10), since the max response time was around 1000
we initially achieved the expected RPS, it would reach close to 30 RPS every time we attempted the test:
However, after a couple of hours with no change to the configuration or the system, the RPS no longer reached the expected, we would consistently get lower RPS (ranging between 20 and 27 RPS):
We tried increasing the Target Concurrency, but it made no difference to the RPS and it was constantly being less than the expected. We attempted this using the GUI and CLI from a local machine and CLI on a remote instance with Jmeter installed, we are running the latest Jmeter version (5.5 at this time) locally and on the remote instance but it made no difference:
Increasing the HEAP didn’t make a difference either
When we increased the logging level on the local machine to diagnose the problem, this error would appear frequently:
o.a.j.p.h.s.HTTPHC4Impl: IOException
java.net.SocketException: Socket closed
Test instance specs
The remote instance is set up on an AWS m6i.8xlarge instance
Local machine has 16GB Memory with intel i7 8th gen processor
All tests were run on Ubuntu machines
Problem/Question
We need to simulate our customer behavior as accurately as possible, so it's critical that we're able to achieve consistent throughput in our requests. How can we do that?
Looking at your Throughput Shaping Timer configuration I can see that it's set up to kick off 50 maximum threads.
You can reach 30 requests per second with 50 threads only if your response time will be less than 1.6 seconds or less.
Looking into response times from your reports I can see that average response time is above 2 seconds and this perfectly explains the fact you're not able to reach the throughput.
Things to check:
Make sure to follow JMeter Best Practices
Make sure that JMeter has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc. It can be done using JMeter PerfMon Plugin
If you cannot conduct required load from the single JMeter instance - consider going for Distributed Testing
And finally it might be your application which cannot handle that many requests per second so it worth checking its logs, telemetry, and so on.
can one share some inputs/links to JMeter(Master & Slave) on Azure VM. What is the VM configuration(Master & Slave) to be used to trigger 3000 concurrent users.
We don't know as it depends on the nature of your test plan,number of requests per second, size of requests/responses, number of pre/post processors, assertions, etc.
You need to measure it, i.e.
Implement your test plan
Run the test starting from 1 user and gradually increasing the load up to 3000 at the same time looking at resources consumption via Azure Monitor or JMeter PerfMon Plugin
When any of monitored metrics like CPU, RAM or Network usage starts exceeding reasonable threshold, i.e. 80% of total available capacity take a look how many virtual users were online at this stage using i.e. Active Threads Over Time plugin
If the number of users is 3000 - you're good to go with this VM
If the number of users is less than 3000 - consider either increasing the size of the VM or adding a new VM of the current size as the slave (or several machines, depending on how many users you were able to kick off)
This question already has answers here:
How do threads and number of iterations impact test and what is JMeter’s max. thread limit
(6 answers)
Closed 5 years ago.
Can anybody explain about how many concurrent users one Jmeter will handle?
I want to run 2000 concurrent users for my project.
No one can "explain" this to you, you can only measure it.
The number of virtual users which can be simulated by JMeter depends on several factors:
machine hardware specifications (CPU, RAM, NIC, etc)
software specifications and versions (OS, JVM and JMeter version and architecture)
the nature of your test (number of requests, size of request/response, number of pre/post processors, assertions, etc)
So your actions should look like:
Make sure you're following JMeter Best Practices
Set up monitoring of baseline OS health metrics (CPU, RAM, disk and network usage). This can be done using i.e. JMeter PerfMon Plugin
Start with 1 virtual user and gradually increase the load until resource consumption won't exceed some reasonable threshold (i.e. 90% of maximum capacity)
Once you start running out of resources - mention the number of virtual users which were active at this moment - this will be the maximum you can simulate on particular this machine for particular this test. This can be done using i.e. Active Threads Over Time listener
If the number is 2000 or more - you're good to go, if it's less - you will have to go for Distributed Testing
See What’s the Max Number of Users You Can Test on JMeter? article for more detailed explanation of the above points and few more hints.
Well, it is the same as with any other software: One CPU-Core can handle exactly one operation (-step) at the same time. What JMeter does is ramping up x threads and then starts them. No "magic" there. So in order to give you good coverage in respect to collision you will want to dedicate a machine with a decent number of CPU cores (Server, not your local machine. Your Machine will occasionally be distracted by other tasks) and make sure your processes takes a decent amount of runtime themselves. Additionally, run the same Test several times to fade out warm up times. How many concurrent users you will get (to answer the question) depends on your environment and "it all depends". Basically, it is not limited by JMeter but by the System you execute it on. You will need to "try it out" and "finetune" your test.
I want to use above 5000 thread in Jmeter, SO what should be the system configuration. Now Am using 4GB RAM , 2.3 GHZ, Now it shows only 3000 threads results only in Summary Report.
In jmeter have any maximum number of limitation?
There is no limitation except:
Your machine CPU power
Your memory and JVM size
Your network interfaces
Answer depends on a lot of factor, You should read:
What is the highest number of threads that is reasonable to simultaneously run in Jmeter? (The accepted answer is not correct anymore)
JMeter max. thread limit
So you will have to benchmark your machine with your plan and see to where you can go, then different options:
1 machine is ok, that's it
You need more, switch to distributed testing or use a Cloud solution
I'm trying to stress-test my Spring RESTful Web Service.
I run my Tomcat server on a Intel Core 2 Duo notebook, 4 GB of RAM. I know it's not a real server machine, but i've only this and it's only for study purpose.
For the test, I run JMeter on a remote machine and communication is through a private WLAN with a central wireless router. I prefer to test this from wireless connection because it would be accessed from mobile clients. With JMeter i run a group of 50 threads, starting one thread per second, then after 50 seconds all threads are running. Each thread sends repeatedly an HTTP request to the server, containing a small JSON object to be processed, and sleeping on each iteration for an amount of time equals to the sum of a 100 milliseconds constant delay and a random value of gaussian distribution with standard deviation of 100 milliseconds. I use some JMeter plugins for graphs.
Here are the results:
I can't figure out why mi hits per seconds doesn't pass the 100 threshold (in the graph they are multiplied per 10), beacuse with this configuration it should have been higher than this value (50 thread sending at least three times would generate 150 hit/sec). I don't get any error message from server, and all seems to work well. I've tried even more and more configurations, but i can't get more than 100 hit/sec.
Why?
[EDIT] Many time I notice a substantial performance degradation from some point on without any visible cause: no error response messages on client, only ok http response messages, and all seems to work well on the server too, but looking at the reports:
As you can notice, something happens between 01:54 and 02:14: hits per sec decreases, and response time increase, okay it could be a server overload, but what about the cpu decreasing? This is not compatible with the congestion hypothesis.
I want to notice that you've chosen very well which rows to display on Composite Graph. It's enough to make some conclusions:
Make note that Hits Per Second perfectly correlates with CPU usage. This means you have "CPU-bound" system, and the maximum performance is mostly limited by CPU. This is very important to remember: server resources spent by Hits, not active users. You may disable your sleep timers at all and still will receive the same 80-90 Hits/s.
The maximum level of CPU is somewhere at 80%, so I assume you run Windows OS (Win7?) on your machine. I used to see that it's impossible to achieve 100% CPU utilization on Windows machine, it just does not allow to spend the last 20%. And if you achieved the maximum, then you see your installation's capacity limit. It just has not enough CPU resources to serve more requests. To fight this bottleneck you should either give more CPU (use another server with higher level CPU hardware), or configure OS to let you use up to 100% (I don't know if it is applicable), or optimize your system (code, OS settings) to spend less CPU to serve single request.
For the second graph I'd suppose something is downloaded via the router, or something happens on JMeter machine. "Something happens" means some task is running. This may be your friend who just wanted to do some "grep error.log", or some scheduled task is running. To pin this down you should look at the router resources and jmeter machine resources at the degradation situation. There must be a process that swallows CPU/DISK/Network.