We recently updated apache jmeter from v4.0 to the latest v5.4.3.
The tests are running fine, but we have strange results.
The requests appear with 3 different labels (without suffix or -0 / -1 appended).
timeStamp, label, responseCode, threadName
1643834640785, API call, 200, Load Group 1 1-1
1643834640785, API call-0, 302, Load Group 1 1-1
1643834641189, API call-1, 200, Load Group 1 1-1
1643834640785, API call, 200, Load Group 1 1-2
....
It seems to me this happens, when the same thread calls the API multiple times.
I am not a jmeter and I am not sure why this happens and how to fix it. Also I don't know what information is needed to analyze the problem correctly.
Thanks in advance!
I'm seeing HTTP Status 302 which means redirection (for example from HTTP to HTTPS or from global website to country-specific website), in that case JMeter generates additional Sub-Result.
The "strange" labels is how JMeter calculates throughput for embedded resources and the cumulative execution time of the parent sampler in case of redirects as well.
The question is what do you want to do with this.
The options are in:
Take it for granted given the above explanation
If you want the "strange" labels to be resolved into real URLs - tick "Functional Testing" box in the Test Plan
or add the next line to user.properties file:
subresults.disable_renaming=true
If you want to get rid of these subresults completely - add the next line to user.properties file:
jmeter.save.saveservice.subresults=false
More information:
Configuring JMeter
JMeter Properties Reference
Related
I'm creating a small test using JMeter. So far I have one Thread Group that executes an HTTP request, waits for 10 seconds, then executes an other HTTP Request and checks what was returned. If I start 100 such threads with 1 second ramp-up period from the JMeter GUI, it works fine, I get the expected values and the whole test finishes in 22 seconds. However, when I start the very same jmx file from the command line, the test runs for more than 120 seconds and some threads (at the last run, 36 out of the 100) don't get the expected value. This might indicate a bug in the system I test, but I don't understand why the test takes that long time from the CLI and why I get errors from the CLI. What is the difference between running the test from the GUI and from the CLI? Does the CLI run the tests "more parallel"? By the way, this is the command line I'm using:
/home/nar/apache-jmeter-3.3/bin/jmeter -n -t test_transactions.jmx -l test_transactions.out
I'm afraid I cannot share the test plan, but I can share the "outline":
+ Thread Group
+ CSV Data Set Config
+ HTTP Request
| + JSON Extractor
+ Constant timer
+ HTTP Request
| + JSON Extractor
| + Response Assertion
+ View Results Tree
+ Save Responses to a file
+ View Results in Table
+ Summary Report
The Constant timer waits for 10 seconds. The first HTTP Request sends in some data and initiates a computation, the second checks the result.
I think you should disable the following listeners in non gui test:
View Results Tree
Save Responses to a file
View Results in Table
Summary Report
After disable you still have result using -l test_transactions.out which you can later view using GUI mode with Browse button in your Listener
In non GUI you can also generate dashboard report if you want by adding -e -o /path/dashboardfolder
It actually does indicate the bug in the system under test. The reason is that you must run JMeter in non-GUI mode as GUI creates huge overhead in terms of resources consumption, especially when you're using Listeners, especially if one of them is View Results Tree.
So my expectation is that in non-GUI mode you're basically creating more immense load which your application cannot handle. You can check this out using i.e. Active Threads Over Time and Transactions Per Second listeners.
My JMeter test:
Iterate over a CSV file (logins.csv) with login credentials, and their unique identifier user a CsvDataSetConfig
Sign in
Based on the login credentials (unique identifier from logins.csv), identify and load a second file in the format of <user_identifier>_invoices.csv which then has the necessary path to view an invoice for that user.
Simplified test setup:
ThreadGroup
> CsvDataSetConfig - file: logins.csv, variables: user_identifier,email,password, sharing_mode: all threads
> `SignIn` TransactionController using email and password from above CSV file to login via series of HTTP Requests
> UserParameters - USER_IDENTIFIER,INVOICE_CSV_FOR_USER
> BeanShellSampler
props.setProperty("USER_IDENTIFIER", vars.get("user_identifier"));
props.setProperty("INVOICE_CSV_FOR_USER","${__P(USER_IDENTIFIER)}_invoices.csv");
> WhileController - condition: ${__javaScript("${invoice-id}" != "<EOF>",)}
> CsvDataSetConfig - file: ${__P(INVOICE_CSV_FOR_USER,)}, variables: invoice-id, sharing_mode: current thread
> `ViewInvoice` TransactionController with HTTP Request to url `../${invoice-id}`
# logins.csv
c7beaa99c6d99fa7754fc2213f9b17b8,foo#example.com,password321
9c8466bee65e39c9d3cf715e42933c3b,bar#example.com,password456
# c7beaa99c6d99fa7754fc2213f9b17b8_invoices.csv
f54eca1cbbba4a97c1dc459e0ba64970
0024f2cdf28dd7ebf3606988fd229afd
# 9c8466bee65e39c9d3cf715e42933c3b_invoices.csv
64f725fdeb2980b28bdf5e02076a55cd
60ac45a12ea3d6b59c2cb82f27da1722
Problem:
In local JMeter, seeing requests to invoice urls being made with incorrect invoice-id for the business. So seems the parameters are not being treat correctly between threads.
In BlazeMeter, seeing the content of the while controller never being hit.
I've tried loop controllers, having 50 rows per _invoices.csv file, but not got anywhere with that either. I also originally started off with
User Defined Variables rather than User Parameters, but the latter seems to be what I should be using for this use case.
Threads are running at the same time and sharing JMeter properties.
In your test plan each thread sets the property USER_IDENTIFIER. So this and other property can/will be overriden by different thread(s) and create inconsistency.
I suggest you save (and get) in variables which aren't shared by threads:
vars.put("USER_IDENTIFIER", vars.get("user_identifier"));
vars.put("INVOICE_CSV_FOR_USER"," ${USER_IDENTIFIER}_invoices.csv");
Also about beanshell, JMeter advice to change to JSR223
Since JMeter 3.1, we advise switching from BeanShell to JSR223 Test Elements (see JSR223 section below for more details), and switching from __Beanshell function to __groovy function.
I am using Apache JMeter 2.13 on Windows 7 and Java 1.8.0_73
I have a JMeter file set up with a thread group with 1 thread, 0 ramp-up period and 5 loop count. I've recorded the log in process for a web application using the recording controller, the HTTP(S) test script recorder and Firefox. I have added the other elements you can see in the screenshot below
I've then parameterized several of the requests to extract login tokens and moved the root URLs out into user variables (so that I can run it against my local build or any of the server builds) including a beanshell script for dynamic authentication headers in some of the requests.
When I clear results and then run the test script I see that the first request in the first iteration has a sample time of around 10 seconds, the same is true of the 4th request (the next major page).
The second iteration these same request takes ~600ms, the third iteration 350
e.g.
10318, 605, 350, 641, 310
I then clear results, run again with 10 loops
10366, 1012, 600, 300, 293, 632, 324,292, 331, 291
So my questions are: Is this expected JMeter behaviour?
Is there anyway to change it
Usually you have to have some warmup method to be run first. Just to set up all the stuff, to reach the db, start some threads, etc.
The results of the warmup are not taken into account when analysing the performance.
How to generte csv file and load csv using response time graph listener?
Can any one help me in detail that how we find response time in jmeter ?
If you run JMeter in command-line non-GUI mode as follows:
jmeter -n -t /path/to/your/test_plan.jmx -l /path/to/results_file.jtl
your results_file.jtl content will look like:
1409124780902,182,Logon,200,OK,Thread Group 1-1,text,true,214,0
1409124781219,153,Logout,200,OK,Thread Group 1-1,text,true,110,0
where second column is page response time in milliseconds.
Other values are:
"1409124780902" - current time stamp in ms
"182" - page response time
"Logon" - sampler name
"200" - Response Code
"OK" - Response Message
"Thread Group 1-1" - Parent Thread Group name, thread number and iteration.
"text" - response data type
"214" - response data size in bytes
"0" - latency
Once your test run is done you can open JMeter GUI and load this results_file.jtl into the listener of your choice.
You might also be interested in JMeter Plugins Extras Set which is capable of generating nice looking and easy understandable response-time related graphs to wit:
Response Times vs Threads
Response Times Distribution
Response Times Percentiles
You can get it by adding Reporters.
Please keep in mind Reporters is cpu and memory intensive components and thus should not be used while actual load test.
But for sample testing you can use it and for load test run you can get response time, average,throughput etc by saving output to jtl file in JMeter.
For normal/sample run
Aggregate report gives Average response time, min, max, median etc.
Summary report also gives the same with less details,
While performing actual run you can save output of these reporters in a jtl file. After the test results can be analyzed from jtl files.
I am new into Apache Jmeter. Basically I want to load test our couple of thrift APIs but have no clue where to start with. It is in java where api takes 2 parameter and then send java object as response.
Any pointer would be very helpful.
JMeter isn't especially for it but it's flexible enough to support your use case.
There is an extensibility mechanism which uses BeanShell. JMeter provides BeanShell Sampler which is capable of invoking Java code, including using external jars.
Simple usage:
Start with empty JMeter project
Create a Thread Group with all defaults (you can play with number of threads, ramp-up, etc)
Add a BeanShell Sampler with following code:
Thread.sleep(2000L);
Add View Results Tree listener
Save and run
You should see a green triangle (or triangles) basing on your number of threads and loops) with output like following:
Thread Name: Thread Group 1-1
Sample Start: 2013-11-02 14:48:11 GMT+03:00
Load time: 5030
Latency: 0
Size in bytes: 0
Headers size in bytes: 0
Body size in bytes: 0
Sample Count: 1
Error Count: 0
Response code: 200
Response message: OK
If you use any of techniques to analyze results, i.e.
JMeter embedded listeners like Aggregate Report, Summary Report, Graph Resuls, etc.
Storing results to CSV file and opening them with Excel or equivalent (see jmeter.properties file under /bin directory of your JMeter installation. Properties prefix is "jmeter.save.saveservice."
JMeter Ant Task (see Test.jmx and build.xml in /extras folder under your JMeter installation)
JMeter Results Analysis Plugin
You'll see your request(s) success rate, min/max/average times (something like 2 seconds I guess) and some more information (depending on your configuration).
Particular your use case assumes
IMPORTANT Placing thrift (or whatever) jars under lib/ext folder (or you won't be able to access your APIs
importing classes you need to test somewhere in BeanShell Sampler
import yourpackage.YourClass;
Invoking methods you want to test from BeanShell Sampler
(optional) do some assertions on responses. i.e.
if (yourresponse != yourexpectedresponse){
IsSuccess=false;
ResponseMessage= "Test Failed";
}
Hope this helps
You can use JSR223 Sampler + Groovy (add groovy-all.jar in jmeter/lib) and look at this client example, see NonblockingClient code for an example:
http://www.javacodegeeks.com/2012/03/apache-thrift-quickstart-tutorial.html
Make your groovy code call a least the following at end:
SampleResult.setSuccessful(true/false)
SampleResult.setResponseCode("code")
SampleResult.setResponseMessage("message")
See:
http://jmeter.apache.org/usermanual/component_reference.html#JSR223_Sampler
And of course, ensure you add the required dependencies in jmeter/lib.
I have writtena CustomThriftSampler for JMeter to load test HBase through thrift service. You can get the details about it at my blog - http://1-st.blogspot.in/2013/12/load-testing-thrift-services-custom.html . Couldn't create a generalized code. Anyway its simple and starightforward java code. Anyone could try it. If time permit I shall write a generalised code and commit to github!!