I use MS Test Manager to run coded UI tests on test machines, and need to run single test on completely all machines at time.
But http://msdn.microsoft.com/en-us/library/ff950318.aspx says that tests are distributed into machines sequentially.
If it's impossible to do so, I guess, this limitation is a great disadvantage of MTM.
So is there a way to run one test on all machines through MTM?
Thanks.
There is no way to do that that I am aware of.
The distribution MTM is doing is for load balancing, so if you have 100 tests and 2 servers, 50 will run in server A and 50 in server B.
You could however create different plans and assign tests to different roles.
Related
I ran a few tests using k6(OSS) by load impact and found it great in terms of usability compared to JMeter
I am doing a feasibility study to choose a load testing tool that should help me do API testing. I am inclined towards using K6 because I believe it is developer friendly but could not find resources that advise regarding maximum load I can simulate using K6.
Would it be possible to simulate 1 million rps(requests per second) using K6? If yes, how should I go about achieving this?
In theory, yes, if you use multiple k6 instances, you can achieve however many requests per second you want. A single k6 instance can produce anywhere from thousands to tens of thousands requests per second, depending on a lot of different factors - machine specs, script complexity, VUs, sleep times, network conditions, etc.
Right now k6 doesn't have a native distributed execution mode though, so you'd have to schedule the different instances yourself. There's a REST API (https://docs.k6.io/docs/rest-api) and you can output metrics to a centralized collector like InfluxDB (https://docs.k6.io/docs/results-output), but it'd take some work to execute a single test on multiple machines. A native k6 distributed execution mode is planned, but work on it hasn't started yet.
You can run k6 on the Load Impact (https://loadimpact.com) cloud (Cloud Execution mode) to access multiple k6 instances executing in parallel. Then, as noted, you can generate a large number of requests per second with the specific RPS being highly dependent on your script and other factors.
I'm using JMeter on development environment and I think of executing sanity tests on production servers.
Sanity of web sites login and other actions.
Is it reasonable to use JMeter on production servers? How to limit JMeter so it won't impact real users? I found only tutorial which doesn't advice it.
Do not run these tests against your production servers unless you know they can handle the load, or you may negatively impact your server's performance.
From JMeter's point of view it doesn't really matter where you run your tests. Running load tests against production environment is very useful as this way you can discover "real" limitations, bottlenecks, integration and interoperabitity problems opposite to load testing in scaled down environments where you can only guess or calculate the anticipated production metrics.
Ideally you should have some form of "staging environment" which is an exact replica of production environment in terms of hardware, software and data.
If you cannot afford having "staging" environment to play with you can run your tests on production, however you need to keep in mind several important constraints to avoid "surprises"
Run your tests in "dead" time when your application real life usage is minimal, i.e. over night or during weekends.
Make sure JMeter test leaves the system at the same state as it was before test, i.e. if you create users, content, data, etc. - make sure you clean it up after the test so your system is not filled with "junk" data used for load testing. So consider using setUp Thread Group for setting up all the necessary test data and tearDown Thread Group to clean up after yourself
Make sure you monitor your servers health so you will be notified when (if) your system is overloaded. You can use JMeter PerfMon Plugin for this.
It would be also good to have AutoStop Listener enabled so JMeter test would stop automatically
Consider adding SMTP Sampler to your test plan so you would be informed in case of unexpected errors.
As a engineering manager I would say: not in my life time ;-)
So what do you want to hear: that it is not a problem?
Only you can tell whether it would be an issue if something behaves different from what you expect.
My advise would be the same as what you are quoting: don't do it. Unless you know what you are doing, and even then...
I have limited number of machines (3 machines). I want to simulate 5000 concurrent users for my website. I want to know if I can run multiple instance on jmeter-server on one host. something like this
host1:
192.168.1.1:3000
192.168.1.1:3001
192.168.1.1:3002
host2:
192.168.1.2:3000
192.168.1.2:3001
192.168.1.2:3002
I dont want to run independent jmeter instances.
I haven't found multiple remotes on one machine to be any better than a single jmeter on the machine.
I have even found the opposite, since there are a lot more overheads.
I have found on some tests that one jmeter master can generate more samples than two or more slaves running in distributed mode.
To do more samples, you need to be using less local resources for other stuff. VMs, jmeter-server, etc all add overheads, unless you are running on a high power server that a single JVM can't make the most of. Even then, the least overhead method is run another jmeter jvm.
Recently, I received a PC installed LoadRunner 11.03(perhaps patch 3) from my client and watched a web performance with it by long-run test.
In multiple user test, it seems not to work on proper performance because my web's performance monitor couldn't reach any limitation, usage of CPU, network bands, disk usage per minute, usage of Memory. Only waiting threads was little bad, but it was not obvious.
It seems a sequential behavior rather than a parallel access.
(No error occured.)
So I though it was not problem of servers, but the client have some problem having prevent to be acting parallel access for some reasons.
I don't have proper HP passport ID, I can't access the LoadRunner patches' website.
Please notice me if not LoadRunner patches, especially patch 4 or higher , let it show such the above behavior or not.
Ok, it sounds like you are just running a script in VUGen. If that is the case I am guessing (based on what you wrote, correct me if I'm wrong) you are running a script in the Virtual User Generator and not in the Controller. LoadRunner is actually a suite of multiple applications. The Virtual User Generator is the script development application, a development environment like Eclipse. It is single threaded and running a script there is meant only to test the script individually.
To run a multi-threaded test you need to use the Controller app and develop a test scenario, assign multiple virtual users (the LR term for concurrent threads) to each script you want to run and execute the test from the Controller. You can configure machines to be the Load Generators (another app set up to run as a process or service) and push out the test from the Controller to the Generators.
I'm trying to create a load test in VS Test Project. I have one empty test method which is called multiple times. At best my test runs 600 times per second.
How to increase the number of running tests per sec?
At best my test runs 600 times per second.
I think this is limited by the machine where the Load test is executing. Perhaps, you can increase this number by:
adding more Virtual Users to your Load Test. Your test iterations will be shared to the virtual users and they will be executing them in parallel.
adding more Test Agents. If you are using a test rig for your Load Tests you can define more VM (if possible) as Test Agents.
See the following references for more details:
Configuring Test Controllers and Test Agents for Load Testing
About the Mix Control for Load Tests
How to: Create a Test Setting for a Distributed Load Test
**If you add functionality to your empty test method I am afraid the runs/sec number will decreased again.