We are setting up a load testing platform and developping some tools in go.
We'd like to remotely check a JMeter load generator and (if possible) get some statistics or infos from a JMeter daemon. This will be from a utility developed in go (the language isn't the concern here) so outside from JMeter GUI. The idea is to display the status of the slaves machine (daemon up or down, latest errors, etc.).
At first, I thought to try to open a network connection to the JMeter process, but I don't know if it would interfere with a load test in progress or if there is an API that would expose more information (errors, load genrator currently in use, etc.).
Thank you for any help or pointers.
You could have a look at this although it does not exactly fit your needs:
http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server
If you're willing to contribute, then you might want to open a discussion on JMeter Dev Mailing list with your ideas and get dev team advices.
Edit (18 June 2015) :
See:
http://mail-archives.apache.org/mod_mbox/jmeter-user/201506.mbox/%3CCAH9fUpajsEz8KEMkEVrU4UBYXxdJC2JvJ%3Dw%3DTpmn2ucgoFDX7w%40mail.gmail.com%3E
This feature would be interesting in JMeter core.
Related
I am new to load testing.
So please help in learning gatling and Apache Jmeter for stress testing.
Please help in installing both on Windows and Linux.
How to implement them in my application?
Which one is better for stress testing?
You are asking very generic questions in terms of Stress/Load testing. I think it would be best if you take a look at their documentation then formulate a more specific question.
Installation documentation is best served from the creators of the software.
Implementing these load/stress testing tools into your application isn't really a thing. If you are looking for unit testing (test to utilize in validating your functions/classes/etc work then look at your languages specific go-to libraries - ie. Java is junit/jboss, Nodejs is Karma/Protractor, Python is TestCase/Nose, etc). These tools (jmeter/gatling) are used for stressing your application outside of your build process so they should be treated as end-users (meaning you run the stress testing from remote machines if it is a web service).
Either are best for the right scenario. I think jmeter clusters easier (built-in, where gatling is more manual) but gatling is more programatic and can be manipulated more.
These are opinions and shouldn't be taken as fact or the best so your milage may vary
I strongly doubt that you need them both, if you want a piece of advice in regards which one to choose take a look at Open Source Load Testing Tools: Which One Should You Use? guide.
Once you have clear vision on what tool better suits the needs - you could start ramping up on the selected tool and ask questions in its community communication channels.
I have a scenario where I need to do performance test on a web application to check when multiple users login to the app and use, then the UIs are getting rendered in time (fast response). I don't want to use record & replay. Is there a way i can run my existing selenium UI tests from Loadrunner with multiple users ?
Thanks in advance
Amit
It may be possible.
First hurdle is language. Loadrunner supports a few languages. The two I know for which there are also Selenium bindings are Java and C#. If your Selenium scripts are in either one or can be packaged and invoked from JVM or CLR (e.g. Python) it could be possible.
Second hurdle is hardware to support user load. Running the browsers will take a lot of resources. I think you could feasibly run three browsers (Chrome, FF and IE) on a single box. That limits you to three users per agent. If you have hardware to run the number of agents to meet your user load it could be possible. UI rendering times will be meaningless.
The loadrunner script will be a C# or Java project. Add whatever references you need to run your Selenium tests and invoke from there.
However, my gut reaction is this may be rickety. Error handling will add complexity. These alternatives may give a better outcome:
Ditch the idea of using Loadrunner and use a service like Neustar to run multiple Selenium tests. If user load is low, Selenium Grid may also be an option.
Ditch the idea of using Selenium and code in the Loadunner web API.
There is an Eclipse addin for LoadRunner.
You can install this addin, import LoadRunner references and add transaction to your selenium scripts.
Here is a link, that has detailed description about above procedure.
http://www.joecolantonio.com/2014/05/08/how-to-run-a-selenium-script-in-loadrunner/
But yes as earlier said you have to take memory usage into considerations.
No. If you wish to run GUI virtual users in a LoadRunner context then your model is through QuickTest Professional.
The act of rendering (actual drawing of pixels on the screen after an item is downloaded or rasterized) will remain the same for a given speed of video card independent of load on the system as the actual draw to the screen is a local client side activity.
I suspect you mean something different than rendering (drawing on the screen).
I want to set up an automation system to run nightly regression tests for a command line tool on a Linux environment. There are some automation scripts which are written in Ruby, but I'd like a complete system which includes:
Build the tool from code base.
Run the Ruby automation scripts which I currently have.
Generate a test report and send through email.
The nature of this command-tool is low-level which deals with system hardware. Some of the features of the tool could crash the system it is being run on. I want this kind of cases to be reported as well. So, an ideal solution might be to have two computers linked through a network like (like ssh, etc). One system could be for logging and reporting the test results, the other system where the tool is actually run.
Appreciate any ideas you might have.
The Jenkins CI server should be able to do what you need. You might have to do some custom work to set up test reports to your liking, but Jenkins is well suited out of the box for the rest of your listed requirements.
As a quick intro I would suggesting checking out some screencasts or other videos on Jenkins - this talk is an example.
Is anybody know a good testing tool that can produce a graph containing the CPU cycle and RAM usage?
What I will do for ex. is I will run an application and while the application is running the testing tool will record CPU cycle and RAM Usage and it will make a graph as an output.
Basically what I'm trying to test is how much heavy load an application put on RAM and CPU.
Thanks in advance.
In case this is Windows the easiest way is probably Performance Monitor (perfmon.exe).
You can configure the counters you are interested in (Such as Processor Time/Commited Bytes/et) and create a Data Collector Set that measures these counters at the desired interval. There are even templates for basic System Performance Report or you can add counters for the particular process you are interested in.
You can schedule the time where you want to execute the sampling and you will be able to see the result using PerfMon or export to a file for further processing.
Video tutorial for the basics: http://www.youtube.com/watch?v=591kfPROYbs
Good Sample where it shows how to monitor SQL:
http://www.brentozar.com/archive/2006/12/dba-101-using-perfmon-for-sql-performance-tuning/
Loadrunner is the best I can think of ; but its very expensive too ! Depending on what you are trying to do, there might be cheaper alternatives.
Any tool which can either hook to the standard Windows or 'NIX system utilities can do this. This has been a defacto feature set on just about every commercial tool for the past 15 years (HP, IBM, Microfocus, etc). Some of the web only commercial tools (but not all) and the hosted services offer this as wekll. For the hosted services you will generally need to punch a hole through your firewall for them to get access to the hosts for monitoring purposes.
On the open source fron this is a totally mixed bag. Some have it, some don't. Some support one platform, but not others (i.e. support Windows, but not 'NIX or vice-versa).
What tools are you using? It is unfortunately common for people to have performance tools in use and not be aware of their existing toolset's monitoring capabilities.
All of the major commercial performance testing tools have this capability, as well as a fair number of the open source ones. The ability to integrate monitor data with response time data is key to the identification of bottlenecks in the system.
If you have a commercial tool and your staff is telling you that it cannot be done then what they are really telling you is that they don't know how to do this with the tool that you have.
It can be done using jmeter, once you install the agent in the target machine you just need to add the perfmon monitor to your test plan.
It will produce 2 result files, the pefmon file and the requests log.
You could also build a plot that compares the resource compsumtion to the load, and througput. The throughput stops increasing when some resource capacity is exceeded. As you can see in the image CPU time increases as the load increases.
JMeter perfmon plugin: http://jmeter-plugins.org/wiki/PerfMon/
I know this is an old thread but I was looking for the same thing today and as I did not found something that was simple to use and produced graphs I made this helper program for apachebench:
https://github.com/juanluisbaptiste/apachebench-graphs
It will run apachebench and plot the results and percentile files using gnuplot.
I hope it helps someone.
Any suggestions for tools to monitor page load times/errors and other performance metrics for a PHP application?
I am aware of the FireBug and YSlow tools, but this is for more server monitoring.
There is the classic 'ab' (apachebench) program. More power comes from JMmeter. For server health, I recommend Munin, which can painlessly capture data from several systems and aggregate it on one page.
Try Nagios, it's the default tool to monitor servers. You can write plugins to report just about any data.
For profiling your code, there's Xdebug. Doing regression testing with Siege can also be quite useful.
You can also try httperf. It's a very flexible tool and if you want to test how your application and webserver can deal with various traffic loads you should definitely give it a go.