I have a client requirement for generating a load of 100 users. Those 100 users have an average internet speed of 20 Mbps.
The machine on which I am planning to run JMeter in non-GUI mode has an internet bandwidth of 100 Mbps.
So, is this feasible to generate a load of 100 users/threads with each having 20 Mbps internet speed considering, we have a total bandwidth of 100Mbps only.
Or
Please suggest an alternative solution for this problem statement.
If an average user has bandwidth of 20 Mbps and your machine's maximum is 100 Mbps - you will be able to only simulate 100 / 20 = 5 virtual users given your load test fully utilizes bandwidth of the user/machine
It might be the case you can still conduct the load testing for more virtual users using this machine as:
Real users don't hammer the application under test "non-stop", they need some time to "think" between operations so consider adding reasonable Timers to introduce realistic delays between the requests
The absolute majority of bandwidth is consumed via so called "embedded resources": images, scripts, styles, sounds, fonts, etc. Real browsers download these embedded resources, but they do this only once so you need to add HTTP Cache Manager to properly simulate this behaviour.
Once you come up with the realistic version of your test scenario you can measure the bandwidth consumed by 1 virtual user via Bytes Throughput Over Time and see how much spare bandwidth is left using PerfMon Plugin
if there will be sufficient bandwidth left for 100 users - you should be good to go
if not - you will have to find another machine(s) and run your JMeter test in Distributed Mode
Related
My client has asked me to add below parameters in my load test result:
"How are we doing on performance testing? Do we have any benchmarks - Eg. PPR 8 Core 32 GB RAM 250
Concurrent Users - 50
Reports: min & min report size - xx kb)
If we haven’t carried out any such performance tests yet, can we plan for."
So, presently I'm only showing Response/elapsed time graph and Latency Graph in MS Excel, for up to 50 concurrent users.
enter image description here
My question is how may I add the Size parameter in the results?
Also how shall I test on the Hardware benchmarks for Load test (eg. 8 Core 32 GB RAM)?
What more things shall I do in performance testing, my client doesn't looks satisfied with the performance test I'm running currently.
(I'm a newbie to performance testing, 3 months only with this domain)
If you're downloading some report and the client wants to see correlation of response time and response size you can check out the following charts:
Bytes Throughput Over Time
Response Times Over Time
With regards to measuring the hardware impact, you can collect it using JMeter PerfMon Plugin, it can gather and plot more than 70 various metrics (CPU, Memory, Disk, Network, TCP, JMX and custom ones)
Till now I was doing load/performance testing (load runner & jMeter) on my local instance(connected to LAN not over wifi) and I was sure about the results. But now I have to do it Virtual desktop infrastructure (VDI). Does it provide the same result as the local instance? Or is it good practice to perform the tests over the VDI?
LoadRunner or JMeter don't care about underlying hardware as you have at least 2 abstraction layers: operating system and C and/or Java runtime.
So given your VDI has the same hardware specifications - you should be getting the same results for the same test in terms of delivered load. I would however recommend monitoring your VDI main health metrics, like CPU load, RAM and Pagefile usage, Network and Disk IO, etc.
In the majority of cases VDIs don't have fully dedicated resources, i.e. if you see 64GB of RAM it is not guaranteed you can allocate them all as the RAM may be shared between other VDIs on hypervisor level.
So monitor your load generator(s) system resources usage and make sure you have enough headroom for running your load tests. See How to Monitor Your Server Health & Performance During a JMeter Load Test guide for more details.
Use a physical load generator as a control element. Run single virtual users of each type on the physical box. If you see that your control group begins to take on different performance characteristics (slower, higher degrees of variance as measured by standard deviation, higher averages, 90th percentiles and maximums) then you have a case for maintaining some physical infrastructure for testing.
The biggest issue directly attacking timing record integrity inside of virtualized load generators is clock jump. The system clock in the virtualized host floats slower with respect to the physical clock on the hardware. Occasionally this needs to be re-synched, causing time to jump. Inevitably this happens while the timing record is open and causes what appears to be a long timing record. Some people suggest that this doesn't happen until you start to see backups in the CPU queue length, which is somewhere in the 75-80% CPU range. I have seen it at even as low as 10% of CPU, because under those light loads the hypervisor can make decisions to parse resources to higher need virtualized instances and then when you get the CPU token back it is time to jump the clock.
The control load generator provides a check against this behavior. If necessary you can even use the control load generator in a statistical sampling model along the lines of manufacturing quality control. You can also show objectively to anyone demanding you move to a virtualized model what the impact of this change will be on the integrity of the response time samples collected.
It depends on the VDI. Is it on same network as your localhost?
If it is, the result would be almost same (depends on configuration of VDI too, very minimal overheads that we might not even notice though)
If it is not, the result would depend on how good is that network.
My scenario,
Step1: I have set my thread group for 1000:threads & 500:seconds
Step2:Configure heep space : HEAP=-Xms1024m -Xmx1024m
Step3:Now, running jmeter for non gui mode.
In this scenario,"Uncaught Exception java.lang.OutOfMemoryError: unable to create new native thread" error occuring in my system.
My system configuration
Processor:Intel® Pentium(R) CPU G2010 # 2.80GHz × 2
OS Type:32 bit
Disc:252.6GB
Memory:3.4 GiB
kindly give me a solution for this scenario.
Thanks,
Vairamuthu.
You don't have enough memory in your machine to consume 1000 threads. It is clearly visible from the error that your machine can not create 1000 threads. You should tweak your machine to resolve this situation.
You have to consider these points:
JMeter is a Java tool it runs with JVM. To obtain maximum capability, we need to provide maximum resources to JMeter during execution.First, we need to increase heap size (Inside JMeter bin directory, we get jmeter.bat/sh)
HEAP=-Xms512m –Xmx512m
It means default allocated heap size is minimum 512MB, maximum 512MB. Configure it as per you own PC configuration. Keep in mind, OS also need some amount of memory, so don't allocate all of you physical RAM.
Then, add memory allocation rate
NEW=-XX:NewSize=128m -XX:MaxNewSize=512m
This means memory will be increased at this rate. You should be careful, because, if your load generation is very high at the beginning, this might need to increase. Keep in mind, it will fragment your heap space inside JVM if the range too broad. If so Garbage Collector needs to work harder to clean up.
JMeter is Java GUI application. It also has the non-GUI edition which is very resource intensive(CPU/RAM). If we run Jmeter in non-GUI mode , it will consume less resource and we can run more thread.
Disable all the Listeners during the Test Run. They are only for debugging and use them to design your desired script.
Listeners should be disabled during load tests. Enabling them causes additional overheads, which consume valuable resources (more memory) that are needed by more important elements of your test.
Always try to use the Up-to-date software. Keep your Java and JMeter updated.
Don’t forget that when it comes to storing requests and response headers, assertion results and response data can consume a lot of memory! So try not to store these values on JMeter unless it’s absolutely necessary.
Also, you need to monitor whether your machine's Memory consumption, CPU usages are running below 80 % or not. If these usages exceed 80 % consider those tests as unreliable as report.
After all of these, if you can't generate 1000 threads from your machine, then you must try with the Distributed Load Testing.
Here is a document for JMeter Distributed Testing Step-by-step.
For better and more elaborated understanding these two blogs How many users JMeter can support? and 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure must help.
I have also found this article very helpful to understand and how to handle them.
The error is due to lack of free RAM.
Looking into your hardware, it doesn't seem you will be able to produce the load of 1k users so I would recommend reconsidering your approach.
For example, you anticipate 1000 simultaneous users working with your application. However it doesn't necessarily mean 100 concurrent users as:
real users don't hammer application non-stop, they need some time to "think" between operations, this "think time" differs depending on application nature, but you should keep it as close to reality as possible
application response time should be added to think time
So given you have 1000 users, each of them "thinks" 10 seconds between operations and application response time is 2 seconds, each user will be able to send 5 requests per minute (60 / (10 + 2)).
Assuming above scenario 1000 users will send 5000 requests per minute which gives us ~83 requests per second and it seems to be achievable with your current hardware.
So if you are not in position to get more powerful hardware or more similar machines to use JMeter in distributed more, the options are in:
Add "think times" between operations using i.e. Constant Timer or Uniform Random Timer
Change your test scenario logic to simulate "requests per second" rather than "concurrent users". You can do it using Constant Throughput Timer or Throughput Shaping Timer.
Your issue is due to using a 32 bit OS, in this mode you are limited both in what you can allocate as Heap (depending on OS you will not be able to exceed 1.6 to 2.1 g) and native threads creation.
I'd suggest switching to 64 Bits OS + 64 bits Jdk.
But if you don't have any other option try setting in jmeter.sh in JVM_ARGS:
-Xss128k
Or if too low:
-Xss256k
I am load testing an IIS based webservice
I need to find out what max. throughput it can support
both the server and the load generators are setup in AWS
The problem is that throughput of the webservice is not going beyond 1500 req/sec even on increasing the users from 500 to 3000, only response time increases (PS: i am using 15GB ram 8 core AWS machines for load generation).
Eore wierd part is CPU usage is not 100%, it is merely30-40%
Even the memory utilization is not high it is 20%.
I tried many counters in PerfMon and did not see anything which could show possible bottleneck
When I use a single machine to generate load it shows ~1500 throughput, if I add one more load generator then the throughput visibly drops to half on the original machine, still giving me a combined total of ~1500 requests/sec.
WHat am I missing here?
Thanks for your help in advance
Check the IIS configurations and thread pool settings in IIS. This is quite
known issue. If available threads are less and even if CPU or memory
is available throughput wont grow as the requests are queued up waiting.
Also check processor queue length counter in perfmon. It could be
some IO issue if the queue is long throughout the test
I'm a web designer and will make a document to train my sites clients how to know if the internet bandwidth speed is slow or quick for each visitor of their sites, depending on the images or site weight (in bytes or in megabytes) and on their internet speed (in kilobit or megabit), for taking how long seconds or time by 250, 500 or 750kb and 1, 2, 4 or 10MB.
I guess there are a few different ways to look at that question. If you are just looking to calculate how quickly the data can be downloaded in a perfect world, Jim's answer explains doing that calculation. But simply downloading files is a small part of the overall performance of a site. There are a few tools out there which will help you understand the performance of your site at different connection speeds.
One of those tools is Webpagetest.org. Enter your URL, choose a location and a browser then expand the Advanced Settings section. Choose a connection type and enter the number of times to run the test. It should be more than 1, but you are limited to only 10 through this UI. Its probably a good idea to go over to the Video tab and choose to capture a video as well so that you can demonstrate what its like on each size pipe. Then repeat for each of the connection types you want to test. This will give you a pretty good idea of how long it will take to view this site at different connection types.
The downside here is that its a bit of work to create a bunch of tests, then collate that info. An alternative to this is the Site Monitor feature at http://yottaa.com. You can setup a single test which will look at the site from different locations, or different browsers, or different connection types, then let you easily graph out the results. To get access you have to register for a free account. An alternative to that is at http://websitetest.com which requires no registration, but you get none of the management of test results.
Webpagetest.org is run by Google in cooperation with a lot of other partners.
Full Disclosure: I am the Technical Evangelist at Yottaa.
If you're asking how long it takes to download, say, 54 megabytes on a 10 megabit (or whatever) connection, the simple solution is pretty easy.
For the sake of simplicity, and to take into account the overhead involved in TCP communications, we typically say that 10 megabits per second is the same as one megabyte per second (that is, we assume one byte is 10 bits rather than 8 bits).
So if you want to know how long it takes to download 54 megabytes on a 10 megabit (1 megabyte per second) connection, the answer is 54/1, or 54 seconds. If you have a 250 Kb connection, that's 25 kilobytes per second, or 0.025 megabytes per second. 54/0.025 = 2,160 seconds, or 36 minutes.
In general, this is a best case estimate. It assumes that the connection between client and server can support the sustained maximum rate. The numbers don't take into account other things the server might be doing that would reduce the bandwidth it can dedicate to a particular connection, nor does it include stalls that could occur in the many hops between client and server.
In practice, the number will most likely be larger. That is, if the above calculation says that the entire site can be downloaded in 27 seconds, it could easily take twice that long. There's no way to reliably predict, based only on the client's bandwidth.
It's also possible that it could take less time, if the client's "10 megabits" is an average, and bursts of higher bandwidth are available. Especially during times of lower overall network usage.