My client has asked me to add below parameters in my load test result:
"How are we doing on performance testing? Do we have any benchmarks - Eg. PPR 8 Core 32 GB RAM 250
Concurrent Users - 50
Reports: min & min report size - xx kb)
If we haven’t carried out any such performance tests yet, can we plan for."
So, presently I'm only showing Response/elapsed time graph and Latency Graph in MS Excel, for up to 50 concurrent users.
enter image description here
My question is how may I add the Size parameter in the results?
Also how shall I test on the Hardware benchmarks for Load test (eg. 8 Core 32 GB RAM)?
What more things shall I do in performance testing, my client doesn't looks satisfied with the performance test I'm running currently.
(I'm a newbie to performance testing, 3 months only with this domain)
If you're downloading some report and the client wants to see correlation of response time and response size you can check out the following charts:
Bytes Throughput Over Time
Response Times Over Time
With regards to measuring the hardware impact, you can collect it using JMeter PerfMon Plugin, it can gather and plot more than 70 various metrics (CPU, Memory, Disk, Network, TCP, JMX and custom ones)
Related
When i run a test in gui mode with 500 threads and 10s ramp up time, the cpu usage was %80 - 85 and memory was 25gb. (This cpu is the one on which Jmeter is used, not the server cpu.)
When i run the same test in non gui mode the usages were the same.
Note1 : in gui mode "jp gc perfmon metrics collector" listener is active. in non gui mode, the same. another listener is not used.
Note2 : cpu specs : 96gb ram, logical processors = 40
The related listeners reports are attached.GuiModeActiveThread, NonGuiModeActiveThread, GuiModePerfMon, NonGuiModePerfMon, GuiModeAggregate, NonGuiModeAggregate.
You ran your test for how long? If test duration is 10 seconds, number of Samplers is low and response time is low as well the actual concurrency was not 400, it was rather around 40, you can double check it using Active Threads Over Timer listener. See JMeter Test Results: Why the Actual Users Number is Lower than Expected for more detailed explanation.
I wouldn't expect much higher CPU and RAM consumption in non-GUI mode, I would expect higher Throughput - the number of requests per second (as long as you don't have any artificial pauses introduced by Timers). JMeter GUI per se doesn't consume a lot of resources, but it can greatly slow JMeter down because refreshing/re-drawing the GUI is happening in the single thread
I have a client requirement for generating a load of 100 users. Those 100 users have an average internet speed of 20 Mbps.
The machine on which I am planning to run JMeter in non-GUI mode has an internet bandwidth of 100 Mbps.
So, is this feasible to generate a load of 100 users/threads with each having 20 Mbps internet speed considering, we have a total bandwidth of 100Mbps only.
Or
Please suggest an alternative solution for this problem statement.
If an average user has bandwidth of 20 Mbps and your machine's maximum is 100 Mbps - you will be able to only simulate 100 / 20 = 5 virtual users given your load test fully utilizes bandwidth of the user/machine
It might be the case you can still conduct the load testing for more virtual users using this machine as:
Real users don't hammer the application under test "non-stop", they need some time to "think" between operations so consider adding reasonable Timers to introduce realistic delays between the requests
The absolute majority of bandwidth is consumed via so called "embedded resources": images, scripts, styles, sounds, fonts, etc. Real browsers download these embedded resources, but they do this only once so you need to add HTTP Cache Manager to properly simulate this behaviour.
Once you come up with the realistic version of your test scenario you can measure the bandwidth consumed by 1 virtual user via Bytes Throughput Over Time and see how much spare bandwidth is left using PerfMon Plugin
if there will be sufficient bandwidth left for 100 users - you should be good to go
if not - you will have to find another machine(s) and run your JMeter test in Distributed Mode
My scenario,
Step1: I have set my thread group for 1000:threads & 500:seconds
Step2:Configure heep space : HEAP=-Xms1024m -Xmx1024m
Step3:Now, running jmeter for non gui mode.
In this scenario,"Uncaught Exception java.lang.OutOfMemoryError: unable to create new native thread" error occuring in my system.
My system configuration
Processor:Intel® Pentium(R) CPU G2010 # 2.80GHz × 2
OS Type:32 bit
Disc:252.6GB
Memory:3.4 GiB
kindly give me a solution for this scenario.
Thanks,
Vairamuthu.
You don't have enough memory in your machine to consume 1000 threads. It is clearly visible from the error that your machine can not create 1000 threads. You should tweak your machine to resolve this situation.
You have to consider these points:
JMeter is a Java tool it runs with JVM. To obtain maximum capability, we need to provide maximum resources to JMeter during execution.First, we need to increase heap size (Inside JMeter bin directory, we get jmeter.bat/sh)
HEAP=-Xms512m –Xmx512m
It means default allocated heap size is minimum 512MB, maximum 512MB. Configure it as per you own PC configuration. Keep in mind, OS also need some amount of memory, so don't allocate all of you physical RAM.
Then, add memory allocation rate
NEW=-XX:NewSize=128m -XX:MaxNewSize=512m
This means memory will be increased at this rate. You should be careful, because, if your load generation is very high at the beginning, this might need to increase. Keep in mind, it will fragment your heap space inside JVM if the range too broad. If so Garbage Collector needs to work harder to clean up.
JMeter is Java GUI application. It also has the non-GUI edition which is very resource intensive(CPU/RAM). If we run Jmeter in non-GUI mode , it will consume less resource and we can run more thread.
Disable all the Listeners during the Test Run. They are only for debugging and use them to design your desired script.
Listeners should be disabled during load tests. Enabling them causes additional overheads, which consume valuable resources (more memory) that are needed by more important elements of your test.
Always try to use the Up-to-date software. Keep your Java and JMeter updated.
Don’t forget that when it comes to storing requests and response headers, assertion results and response data can consume a lot of memory! So try not to store these values on JMeter unless it’s absolutely necessary.
Also, you need to monitor whether your machine's Memory consumption, CPU usages are running below 80 % or not. If these usages exceed 80 % consider those tests as unreliable as report.
After all of these, if you can't generate 1000 threads from your machine, then you must try with the Distributed Load Testing.
Here is a document for JMeter Distributed Testing Step-by-step.
For better and more elaborated understanding these two blogs How many users JMeter can support? and 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure must help.
I have also found this article very helpful to understand and how to handle them.
The error is due to lack of free RAM.
Looking into your hardware, it doesn't seem you will be able to produce the load of 1k users so I would recommend reconsidering your approach.
For example, you anticipate 1000 simultaneous users working with your application. However it doesn't necessarily mean 100 concurrent users as:
real users don't hammer application non-stop, they need some time to "think" between operations, this "think time" differs depending on application nature, but you should keep it as close to reality as possible
application response time should be added to think time
So given you have 1000 users, each of them "thinks" 10 seconds between operations and application response time is 2 seconds, each user will be able to send 5 requests per minute (60 / (10 + 2)).
Assuming above scenario 1000 users will send 5000 requests per minute which gives us ~83 requests per second and it seems to be achievable with your current hardware.
So if you are not in position to get more powerful hardware or more similar machines to use JMeter in distributed more, the options are in:
Add "think times" between operations using i.e. Constant Timer or Uniform Random Timer
Change your test scenario logic to simulate "requests per second" rather than "concurrent users". You can do it using Constant Throughput Timer or Throughput Shaping Timer.
Your issue is due to using a 32 bit OS, in this mode you are limited both in what you can allocate as Heap (depending on OS you will not be able to exceed 1.6 to 2.1 g) and native threads creation.
I'd suggest switching to 64 Bits OS + 64 bits Jdk.
But if you don't have any other option try setting in jmeter.sh in JVM_ARGS:
-Xss128k
Or if too low:
-Xss256k
I am load testing an IIS based webservice
I need to find out what max. throughput it can support
both the server and the load generators are setup in AWS
The problem is that throughput of the webservice is not going beyond 1500 req/sec even on increasing the users from 500 to 3000, only response time increases (PS: i am using 15GB ram 8 core AWS machines for load generation).
Eore wierd part is CPU usage is not 100%, it is merely30-40%
Even the memory utilization is not high it is 20%.
I tried many counters in PerfMon and did not see anything which could show possible bottleneck
When I use a single machine to generate load it shows ~1500 throughput, if I add one more load generator then the throughput visibly drops to half on the original machine, still giving me a combined total of ~1500 requests/sec.
WHat am I missing here?
Thanks for your help in advance
Check the IIS configurations and thread pool settings in IIS. This is quite
known issue. If available threads are less and even if CPU or memory
is available throughput wont grow as the requests are queued up waiting.
Also check processor queue length counter in perfmon. It could be
some IO issue if the queue is long throughout the test
We are trying to implement a web application with Apache Storm.
Applicationreceives a huge load of ad-requests (100 TPS - a hundred transactions / second ),makes some simple calculation on them and then stores the result in a NoSQL database with a maximum latency of 10 ms.
We are using Cassandra as a sink for its writing capabilities.
However, we have already overpassed the 8 ms requirement, we are in 100ms.
We tried to minimize the size of buffers (Disruptor buffers) and to well balance the topology, using the parallelism of bolts.
But we still in 20ms.
With 4 worker ( 8 cores / 16GB ) we are at 20k TPS which is still very low.
Is there any suggestions for optimization orare we just reaching the limits of Apache Storm(limits of Java)?
I don't know the platform you're using, but in C++ 10ms is eternity. I would think you are using the wrong tools for the job.
Using C++, serving some local query should take under a microsecond.
Non-local queries that touch multiple memory locations and/or have to wait for disk or network I/O, have no choice but taking more time. In this case parallelism is your best friend.
You have to find the bottleneck.
Is it I/O?
Is it CPU?
Is it memory bandwidth?
Is it memory access time?
After you've found the bottleneck, you can either improve it, async it and/or multiply (=parallelize) it.
There's a trade-off between low latency and high throughput.
If you really need to have high throughput, you should rely on batching adjusting size of buffers bigger, or using Trident.
Trying to avoid transmitting tuples to other workers helps low latency. (localOrShuffleGrouping)
Please don't forget to monitor GC which causes stop-the-world. If you need low-latency, it should be minimized.