JMeter search result is different in the Local machine and remote server machine.
The JMeter batch file runs both environments separately.
Run one website in JMeter but the local machine and remote server load time are different.
Both machine internet speed is proper.
Several factors can influence that:
1) Network path from running JMeter to your server. You should always consider that.
If, say, you're testing microservice based on Amazon Cloud (AWS), and the downstream consumers of its data are also running in the same cloud - it doesn't make a lot of sense to run JMeter at your local machine, you have to run it at AWS as well (as your consumers do) to get realistic timings.
The travel over network path there and back would add hundreds of milliseconds, moreover, it's pretty unpredictable, deviations may be huge.
2) GUI vs Non-GUI mode, the rule of thumb: GUI is for development/debug only. It takes quite a toll on the performance.
3) Available resources on the machine - you didn't mention that at all, though mind that Java Runtime Environment is kind of far from being very lean, so if the machine is not dedicated for running especially JMeter, and especially if machine is not very powerful, the results may vary.
4) Addition to resource scope: by default, the scripts running JMeter are quite restrictive in resource allocation, and if you overwhelm the instance with a lot of threads to run, the timings may get distorted.
These are general factors, if you want it more specific to your case - show how & where (means, in what type of machine) you run your tests, and where's your target in terms of network path.
Related
I have an application running in environment PRODUCT. I want to test it's performance by using JMeter but I can not do it directly to environment PRODUCT, so my idea is test the application in an environment DEV then I can use the result to calculate values in PRODUCT. Can I do that? If it can, how can I do?
If DEV environment is identical to PRODUCT environment - you should be getting the same results.
If DEV environment is different - I don't think you will be able to precisely calculate the anticipated performance of the PRODUCT environment as there are too many factors to consider and it does not work like: "If my app is serving X requests per second being deployed on a system with Y GB of RAM then it will process X * 2 requests per second on a system with Y * 2 GB of RAM"
There are several things you still can check on a scaled down environment, i.e.
You can run some form of integration test but under the load, it will allow you to identify any deployment problems i.e. misconfiguration. If there is a problematic long-running slow DB query - it will replicate in production as well
You can test load allocation and distribution. If your application scales - you can perform scalability testing. Load allocation and scaling algorithms will be the same on both environments
You can perform monitoring of your system using i.e. PerfMon Plugin to identify the most consumed resource - the situation will be the same in production
You can run a Soak Test - put your system under a prolonged load. This way you will be able to catch memory leaks
You can use profiling tools while your system is under the load to identify the most resource consuming logic, largest objects, slowest functions, etc.
I want to simulate up to 100,000 requests per second and I know that tools like Jmeter and Locust can run in distributed mode to generate load.
But since there are cloud VMs with up to 64 vCPUs and 240GB of RAM on a single VM, is it necessary to run in a cluster of smaller machines, or can I just use 1 large VM?
Will I be able to achieve more "concurrency" with more machines due to a network bottleneck coming from the 1 large machine?
If I just use one big machine, would I be limited by the number of ports there are?
In the load generator, does every simulated "user" that sends a request also require a port on the machine to receive a 200 response? (Sorry, my understanding of how TCP ports work is a bit weak.)
Also, we use Kubernetes pretty heavily, but with Jmeter or Locust, I feel like it'd be easier to run it on bare VM, without containerizing (even in distributed mode) while still maintaining reproducibility. Should I be trying to containerize Jmeter or Locust and running in Kubernetes instead?
According to KISS principle it is better to go for a single machine assuming it is capable of conducting the required load.
Make sure you're following JMeter Best Practices
Make sure you have monitoring of baseline OS health metrics (CPU, RAM, swap, network and disk IO, JVM statistics, etc.)
Start with low number of users and gradually increase the load until you reach the desired throughput or limit of any of the monitored metrics, whatever comes the first. If there will be a lack of CPU or RAM or something - see what could be done to overcome the limitation.
More information: What’s the Max Number of Users You Can Test on JMeter?
I have limited number of machines (3 machines). I want to simulate 5000 concurrent users for my website. I want to know if I can run multiple instance on jmeter-server on one host. something like this
host1:
192.168.1.1:3000
192.168.1.1:3001
192.168.1.1:3002
host2:
192.168.1.2:3000
192.168.1.2:3001
192.168.1.2:3002
I dont want to run independent jmeter instances.
I haven't found multiple remotes on one machine to be any better than a single jmeter on the machine.
I have even found the opposite, since there are a lot more overheads.
I have found on some tests that one jmeter master can generate more samples than two or more slaves running in distributed mode.
To do more samples, you need to be using less local resources for other stuff. VMs, jmeter-server, etc all add overheads, unless you are running on a high power server that a single JVM can't make the most of. Even then, the least overhead method is run another jmeter jvm.
Straight to the point,
I have used jmeter to load test a website that hosted in apache web server.
When i load test with a single computer (2000 user), the error rate, connect time, and response time is higher than when i load test with 4 computer (500 user each computer, distributed testing).
My question is, what cause distributed testing gave a different and better result than single testing?
So many factors related to:
Your machine configuration is it powerful enough ?, is it correctly configured (OS, network stack, memory ...)
Your JMeter software configuration, is it running with default ? did you change things for 2000 run ?
You should read this:
http://www.dzone.com/links/see_how_to_make_jmeter_run_thousands_of_threads_w.html
http://jmeter.apache.org/usermanual/best-practices.html
Recently, I received a PC installed LoadRunner 11.03(perhaps patch 3) from my client and watched a web performance with it by long-run test.
In multiple user test, it seems not to work on proper performance because my web's performance monitor couldn't reach any limitation, usage of CPU, network bands, disk usage per minute, usage of Memory. Only waiting threads was little bad, but it was not obvious.
It seems a sequential behavior rather than a parallel access.
(No error occured.)
So I though it was not problem of servers, but the client have some problem having prevent to be acting parallel access for some reasons.
I don't have proper HP passport ID, I can't access the LoadRunner patches' website.
Please notice me if not LoadRunner patches, especially patch 4 or higher , let it show such the above behavior or not.
Ok, it sounds like you are just running a script in VUGen. If that is the case I am guessing (based on what you wrote, correct me if I'm wrong) you are running a script in the Virtual User Generator and not in the Controller. LoadRunner is actually a suite of multiple applications. The Virtual User Generator is the script development application, a development environment like Eclipse. It is single threaded and running a script there is meant only to test the script individually.
To run a multi-threaded test you need to use the Controller app and develop a test scenario, assign multiple virtual users (the LR term for concurrent threads) to each script you want to run and execute the test from the Controller. You can configure machines to be the Load Generators (another app set up to run as a process or service) and push out the test from the Controller to the Generators.