I want to be able to run a JMeter test for thousands of users and plot the results dynamically using a JQuery based charting library like HighCharts i.e. the response from every virtual user must be plotted in near real time to show a stock ticker like chart which gets updated dynamically. I am OK running the test in Non-GUI mode.
I have tried the following,
- Run the JMeter test in non-GUI mode and write the response to a file. What I notice is that the results get written to the file in a buffered manner which means even if I have a program monitoring the file for new records, I wont get it in real time.
I am looking for suggestions on how this can be achieved
1. Do I need to write a custom JMeter plugin? In this case how will it work?
2. Is there some listener which can give me the desired data
3. Can this be done via post processor?
I have seen real time reporting being done on some cloud based load testing websites which use JMeter, so I'm sure it can be done, but how?
There is some buffering when writing to a file, but it shouldn't be more than a few seconds worth of data.
I'd go with the route of reading the log file into something like statsD using something like logstash.net and from there you can probably find an existing solution that pushes it to a chart.
You can disable buffering by adding this in user.properties file:
jmeter.save.saveservice.autoflush=true
This impacts slightly performances for test that have low or no pauses.
To do what you want you could use this kind of library:
http://www.chartjs.org/
Related
I'm looking for a way to display pie chart/table after running 100 tests, but all available built-in reports seem to accumulate data on time spent per sample, controller and about performance metrics.
Although tests mostly check performance and some metrics are usefull, we also need statistics on actual response data.
Each http request queries service for items availability per product.
After tests finish we also would like pie chart appear with 3 sections:
Available
Low on stock
Unavailable
Now, I found "Save Responses to a file" listener but it generates separate files which isn't very good. Also with "View Results Tree" we can specify filename where responses will be dumped.
We don't need the whole response object and preferably not even write anything to disk.
And than, how to actually visualize that data in JMeter after tests complete? Would it be Aggregate Graph?
So to recap: while threads run, each value from json response object (parsed with JPath) should be remembered somewhere and after tests complete these variables should be grouped and displayed as a pie chart.
I can think only of Sample Variables property, if you add the next line to user.properties file:
sample_variables=value1,value2,etc.
next time when you run JMeter test in command-line non-GUI mode the .jtl result file will contain as many extra columns as you have Sample Variables and each cell will contain respective value of the variable for each Sampler.
You will be able to use Excel or equivalent to build your charts. Alternatively you could use Backend Listener and come up with a Grafana dashboard showing what you need.
I just want to get the aggregate report include a column for latency.
I have already tried uncommenting jmeter.save.saveservice.latency=true in bin/jmeter.properties.
I am currently using JMeter 3.3
Any help would be greatly appreciated. Thanks!
UPDATE:
A little hack I tried that kind of worked was adding a JSR223 PostProcessor with the following 3 lines of code:
long startTime = prev.getStartTime();
long latency = prev.getLatency();
prev.setEndTime(startTime + latency);
I say kind of worked because it is a hack and even though it was able to correctly reset the average column from load time to latency, it started throwing off the throughput values.
The end goal in my case is just to get the latency along with the other values by running the CMDRunner.jar on the JTL file to output the summary results CSV file.
You can't .
Alternatively why not use the Web Report that exists since 3.0 and that provides a lot of rich information much better than all existing listeners.
See:
https://jmeter.apache.org/usermanual/generating-dashboard.html
When I run the test in GUI, i see the Average, Min, Max in GUI. But when I run in console, is there a way to add these to the csv file?
These values are being calculated so you will be able to see the values only when you open .jtl results file after test finishes in the listener of your choice, i.e. Aggregate Report or Summary Report.
If you want to see the interim statistics while your test is being executed you have the following choices:
JMeter Summarizer output. JMeter reports some numbers into stdout while your test is being running
You can get some extended information if you run your JMeter test using Taurus tool as a wrapper
Both console and web interface options are available, in order to see current test execution stats in browser start your test like:
bzt yourtest.jmx -report
And finally you can use Backend Listener to send your results into database, message queue or web service and use custom plotting application to print out either raw or parsed statistics, here you are limited only by your fantasy:
More information:
JMeter: Real Time Results
How to Use Grafana to Monitor JMeter Non-GUI Results
JMeter produces some basic fields/result_field. JMeter doesn't create everything you see in different types of Listeners.
You can give this a try.
Create a plan
Generate atleast 100 samples (As large amount of data is required for some listeners), using a single sampler (request)
Use as many Listeners as you want of different types (say 15 types of listeners)
Run the plan....
Now in the filename field of all the listeners give series of names of files like a1.jtl a2.jtl
and so on....
see the screen shot
enter image description here
Now again run the plan. Go to the files and open them in some good editors like notepad++.
For your surprise you will find the same data in all the files irrespective of the type of listener generating the file.
Crux of the matter is : JMeter gathers only handful of information from the run, the rest information which is shown in different Listeners is COMPUTED by the JMeter.
So you can read the *.jtl file into any of the listener.
In JMeter, the new way since 3.0 to have results is to use the Web report generated at end of test:
http://jmeter.apache.org/usermanual/generating-dashboard.html
I have a system that should be able to handle millions of users requests concurrently. In order to check how the system handles the load, I setup a cluster of JMeter servers (slaves), and one controller (client).
I have a database of all users (~10M), and I need each request sent to be from a different user.
I am wondering how I can implement such a thing in JMeter. Basically, I thought about dividing a range of users (let's say 100,000) per each slave, and then within a given slave, each request should read a new user from the local 100,000 list, and delete it. Thus, I will eventually send a request from every user.
The thing is while this idea sounds logical theoretically, I do not exactly know how to implement it using the JMeter terms. Also, I am not sure how to read from database in the test, although I could theoretically read it in advance into a text file, and have each slave contain the text file with its 100,000 users portion.
I can setup a very large cluster of machines, so scale will not be the issue here. Just how to set it all up.
The best way to provide Jmeter with a list of parameters is to use a CSV file:
http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config
You can configure the CSV dataset config to do make every thread use a different line in the CSV. Each engine will need to have it’s own unique CSV file, because the sharing mode does nto work between engines in distributed testing (you can try to automate this part, this can be interesting to do :) ).
This is how your script should look like:
1. Thread Group
1.1 HTTP sampler (login)
1.1.1 CSV dataset config
1.2 second http sampler
etc...
The login sampler will use the parameters loaded by from the CSV file, so for every ’login’ it will use a different line.
Distributed testing is pretty simple:
http://jmeter.apache.org/usermanual/remote-test.html
Keep in mind that running 100K concurrent users on a single Jmeter load engine will be hard (Jmeter consumes resources on the server, so you will need lots of CPU and memory). So you should also monitor the engines.
Also 1M users will cause a lot of data that the engines will send back to the console, so you might need to start a bunch of distributed tests in parallel, and at the end aggregate the results.
Cheers,
This is can be implemented by doing the following steps:
Taking the user credential dump and saving it to a csv file
split the csv file. copy 1 file to each Jmeter slave, in the same location on all the
machines e.g. "C:\Loadtest\"
from the controller, give the path of your csv file in "CSV Data Set Config".
Run the Test.
By doing the above steps, Jmeter controller will start execution of the test by pointing all the Jmeter Slave nodes to use the CSV file in the same location "C:\Loadtest\".
But the trick here is that all the machine will be using different set of users.
Hope this will help.
I'm familiarising myself with JMeter and I've thought of something that's either pretty cool or a very dumb idea.
Whilst reading about Listeners I noticed the following:
Note that all Listeners save the same data; the only difference is in the way the data is presented on the screen.
And this:
Graph Results MUST NOT BE USED during load test as it consumes a lot of resources (memory and CPU). Use it only for either functional testing or during Test Plan debugging and Validation.
So I was wondering: if all listeners receive the same data; why not save that data in a CSV or even XML file, and feed that to a listener afterwards? It would be very resource friendly to have the Graph Results Listener display a graph after the tests are done, instead of while testing.
Am I missing something, or is this a good possiblity?
Yes you can do that and i think most guys use it that way only. Instead of CSV and XML files use JTL file format to save the results. In normal scenario one uses command line to run the test and save the data in a file(preferably JTL). After the test is done you can use the JTL file to generate reports with JMeter UI or using other tools like this.