I have run the JMeter testing with 400 concurrent users with a 125 loop count for the following GET request.
https://example.com/rest_api?from_pt={from_point}
I have run the same request in locust with 500 concurrent users and 100 spawn rate per second and get the following charts
Locust file containing code
import csv
from locust import HttpUser, task
class WebsiteUser(HttpUser):
def on_start(self):
pass
#task(1)
def request_testing(self):
with open("test.csv", 'rt') as f:
reader = csv.DictReader(f)
for row in reader:
from_point = row.get('from_point')
to_point = row.get('to_point')
headers = {'content-type': 'application/json','q':'{{api_key_value}}'}
self.client.get("rest_api?from_pt=%s" %(from_point),headers=headers)
I need to find where I was wrong in doing locust testing.
I fail to see any JMeter test results, you're looking at PerfMon metrics (usage of CPU, RAM, etc.). Moreover it's not clear how did you configure and run JMeter test (Thread Group settings, etc.)
Generate HTML Reporting Dashboard from the JMeter .jtl test results file and compare it with Locust results.
In general comparing 2 load testing tools output doesn't make a lot of sense, if you're running the same test (in terms of number of concurrent users,sending the same request payload at the same rate, etc.) you should be getting the same results
Related
I am new in load testing , For understanding the statistics generated by locust , I give very low load to an API, See my below snapshot, there is only 1 user run against given API and it finish its process within milliseconds but my RPS is showing 0 . I am expecting RPS value as 1 because the request able to finish within one second.
Test Result
See my code below and please note that I am not giving any wait_time in my code and it is a very simple code which I wrote.
Main code
config file
I'm creating a small test using JMeter. So far I have one Thread Group that executes an HTTP request, waits for 10 seconds, then executes an other HTTP Request and checks what was returned. If I start 100 such threads with 1 second ramp-up period from the JMeter GUI, it works fine, I get the expected values and the whole test finishes in 22 seconds. However, when I start the very same jmx file from the command line, the test runs for more than 120 seconds and some threads (at the last run, 36 out of the 100) don't get the expected value. This might indicate a bug in the system I test, but I don't understand why the test takes that long time from the CLI and why I get errors from the CLI. What is the difference between running the test from the GUI and from the CLI? Does the CLI run the tests "more parallel"? By the way, this is the command line I'm using:
/home/nar/apache-jmeter-3.3/bin/jmeter -n -t test_transactions.jmx -l test_transactions.out
I'm afraid I cannot share the test plan, but I can share the "outline":
+ Thread Group
+ CSV Data Set Config
+ HTTP Request
| + JSON Extractor
+ Constant timer
+ HTTP Request
| + JSON Extractor
| + Response Assertion
+ View Results Tree
+ Save Responses to a file
+ View Results in Table
+ Summary Report
The Constant timer waits for 10 seconds. The first HTTP Request sends in some data and initiates a computation, the second checks the result.
I think you should disable the following listeners in non gui test:
View Results Tree
Save Responses to a file
View Results in Table
Summary Report
After disable you still have result using -l test_transactions.out which you can later view using GUI mode with Browse button in your Listener
In non GUI you can also generate dashboard report if you want by adding -e -o /path/dashboardfolder
It actually does indicate the bug in the system under test. The reason is that you must run JMeter in non-GUI mode as GUI creates huge overhead in terms of resources consumption, especially when you're using Listeners, especially if one of them is View Results Tree.
So my expectation is that in non-GUI mode you're basically creating more immense load which your application cannot handle. You can check this out using i.e. Active Threads Over Time and Transactions Per Second listeners.
How to generte csv file and load csv using response time graph listener?
Can any one help me in detail that how we find response time in jmeter ?
If you run JMeter in command-line non-GUI mode as follows:
jmeter -n -t /path/to/your/test_plan.jmx -l /path/to/results_file.jtl
your results_file.jtl content will look like:
1409124780902,182,Logon,200,OK,Thread Group 1-1,text,true,214,0
1409124781219,153,Logout,200,OK,Thread Group 1-1,text,true,110,0
where second column is page response time in milliseconds.
Other values are:
"1409124780902" - current time stamp in ms
"182" - page response time
"Logon" - sampler name
"200" - Response Code
"OK" - Response Message
"Thread Group 1-1" - Parent Thread Group name, thread number and iteration.
"text" - response data type
"214" - response data size in bytes
"0" - latency
Once your test run is done you can open JMeter GUI and load this results_file.jtl into the listener of your choice.
You might also be interested in JMeter Plugins Extras Set which is capable of generating nice looking and easy understandable response-time related graphs to wit:
Response Times vs Threads
Response Times Distribution
Response Times Percentiles
You can get it by adding Reporters.
Please keep in mind Reporters is cpu and memory intensive components and thus should not be used while actual load test.
But for sample testing you can use it and for load test run you can get response time, average,throughput etc by saving output to jtl file in JMeter.
For normal/sample run
Aggregate report gives Average response time, min, max, median etc.
Summary report also gives the same with less details,
While performing actual run you can save output of these reporters in a jtl file. After the test results can be analyzed from jtl files.
I am using JMeter for load testing. I have noticed that the response time it shows keeps increasing until the test plan has finished running.
I have 3 thread groups with the following settings:
Number of threads: 900, 180, 180
Rampup: 0
Loop count: 20
Each of the threads has a constant throughput controller with the following settings:
Throughput: 900, 180, 180
Jmeter Test Plan http://cl.ly/UPhC/jmeter_test_plan.png
I don't understand why the response time keeps increasing from the beginning until the end of the test plan execution.
Maybe the system under test is creating objects and provoking garbage collections. That won't be a problem at first, but will get worst as the test progresses. Do a profiling of the JVM of the system under test with Visual VM or similar.
The problem could be the target system.
But if you are running your plan in GUI mode then it can explain your issue particularly View Results Tree listener which exists for DEBUGGING Test plan and absolutely not for GUI Load Test.
Read this:
http://www.ubik-ingenierie.com/blog/jmeter_performance_tuning_tips/
So fix is:
- Run your test in NON GUI mode and keep only Summary report
You can after test reload result file.
You may also see this:
http://www.ubik-ingenierie.com/blog/automatically-generating-nice-graphs-at-end-of-your-load-test-with-apache-jmeter-and-jmeter-plugins/
I am new into Apache Jmeter. Basically I want to load test our couple of thrift APIs but have no clue where to start with. It is in java where api takes 2 parameter and then send java object as response.
Any pointer would be very helpful.
JMeter isn't especially for it but it's flexible enough to support your use case.
There is an extensibility mechanism which uses BeanShell. JMeter provides BeanShell Sampler which is capable of invoking Java code, including using external jars.
Simple usage:
Start with empty JMeter project
Create a Thread Group with all defaults (you can play with number of threads, ramp-up, etc)
Add a BeanShell Sampler with following code:
Thread.sleep(2000L);
Add View Results Tree listener
Save and run
You should see a green triangle (or triangles) basing on your number of threads and loops) with output like following:
Thread Name: Thread Group 1-1
Sample Start: 2013-11-02 14:48:11 GMT+03:00
Load time: 5030
Latency: 0
Size in bytes: 0
Headers size in bytes: 0
Body size in bytes: 0
Sample Count: 1
Error Count: 0
Response code: 200
Response message: OK
If you use any of techniques to analyze results, i.e.
JMeter embedded listeners like Aggregate Report, Summary Report, Graph Resuls, etc.
Storing results to CSV file and opening them with Excel or equivalent (see jmeter.properties file under /bin directory of your JMeter installation. Properties prefix is "jmeter.save.saveservice."
JMeter Ant Task (see Test.jmx and build.xml in /extras folder under your JMeter installation)
JMeter Results Analysis Plugin
You'll see your request(s) success rate, min/max/average times (something like 2 seconds I guess) and some more information (depending on your configuration).
Particular your use case assumes
IMPORTANT Placing thrift (or whatever) jars under lib/ext folder (or you won't be able to access your APIs
importing classes you need to test somewhere in BeanShell Sampler
import yourpackage.YourClass;
Invoking methods you want to test from BeanShell Sampler
(optional) do some assertions on responses. i.e.
if (yourresponse != yourexpectedresponse){
IsSuccess=false;
ResponseMessage= "Test Failed";
}
Hope this helps
You can use JSR223 Sampler + Groovy (add groovy-all.jar in jmeter/lib) and look at this client example, see NonblockingClient code for an example:
http://www.javacodegeeks.com/2012/03/apache-thrift-quickstart-tutorial.html
Make your groovy code call a least the following at end:
SampleResult.setSuccessful(true/false)
SampleResult.setResponseCode("code")
SampleResult.setResponseMessage("message")
See:
http://jmeter.apache.org/usermanual/component_reference.html#JSR223_Sampler
And of course, ensure you add the required dependencies in jmeter/lib.
I have writtena CustomThriftSampler for JMeter to load test HBase through thrift service. You can get the details about it at my blog - http://1-st.blogspot.in/2013/12/load-testing-thrift-services-custom.html . Couldn't create a generalized code. Anyway its simple and starightforward java code. Anyone could try it. If time permit I shall write a generalised code and commit to github!!