Automatically running the same test plan multiple times with different variables - jmeter

I've created a test plan to do some load testing against HTTP endpoints. I want to run the same test plan against multiple endpoints, and find myself having to do the following each time:
Change variable that determines which endpoint I'm hitting.
Run the test.
When complete, record results.
Clear all results.
Back to step 1.
I want to automate this. Is the following possible?
Define a list of endpoints as a variable.
For each endpoint in endpoints
Run test against endpoint
When complete, save results
Clear all results.
The things in particular that I don't know how to automate are:
Starting a test
Defining an list-style variable
Clearing all results
Any help would be greatly appreciated!

Please see answers below:
Starting a test
Apart from running JMeter from GUI there are several execution options such as:
Command line non-GUI mode - you can wrap JMeter execution line in operating system batch file
JMeter Ant Task - integration with Apache Ant build system, more powerful if you're comfortable with it
JMeter Maven Plugin - the same as Ant but for Apache Maven
Option to run JMeter with CI systems like Jenkins
Defining an list-style variable
JMeter offers "properties" which can be set via .properties files (see user.properties in /bin folder of your JMeter installation for example) or passed as name/value pairs to JMeter startup script. For instance instead of hard-coding your host in .jmx file you can use read a property function.
Set "Server Name or IP" field value to ${__P(myhost,)}
Add myhost=example.com line to user.properties file in /bin folder of your JMeter installation, restart JMeter and run the test. Request will go to example.com.
Alternatively you can pass myhost property to JMeter in command-line mode as follows
jmeter -Jmyhost=example.com
Multiple properties can be passed this way:
jmeter -Jproperty1=value1 -Jproperty2=value2 ....
And even files via -p or --propfile JMeter command line argument
See Apache JMeter Properties Customization Guide for more options and information.
Clearing all results
It depends on unattended execution option you'll choose, all of them provide at least one way to delete file and/or folder. Personally I wouldn't delete any results and would rather keep them for future reference. To distinguish different endpoints test runs and to avoid overwriting I would use timestamps like:
jmeter -Jhttp.endpoint=my.test.host -n -t your-script.jmx -l my-test-host-results-${date}.jtl
where
-J - sets endpoint to my.test.host
-n - tells JMeter to run in non-GUI mode
-t - specifies .jmx file to use
-l - specifies name and location of results file

Related

Additional property files to load in JMeter

My performance test setup
Jmeter 5.3
1 JMX File that contains my test plan.
1 default properties file. This is named 'user.properties'.
1 custom properties file. I pass this when I run my test plan headless.
I have a test plan that I run in the following ways,
headless
jmeter -p "customProp.properties" -n -t "myTestPlan.jmx"
GUI
Open > myTestPlan.jmx and like above I want the customProp.properties also to come into effect with myTestPlan.jmx in play.
What I've done so far?
I've used the 'user.properties' attribute within the 'user.properties' file in the bin directory as follows,
user.properties=customProp.properties
I know how to do add properties files when running headless. Just now sure how to do it with GUI. Any pointers?
Just proceed as for Non GUI, but note it’s better to make -p point to jmeter.properties and -q to user.properties and your additional properties file:
jmeter -p path_to_jmeter.properties -q path_to_user.properties -q path_to_custom.properties
See:
https://jmeter.apache.org/usermanual/get-started.html#options
If you don't have any customizations under user.properties file and don't plan to have them in the future - locate the following line in the jmeter.properties file:
user.properties=user.properties
and change it to point to your "custom" properties file:
user.properties=customProp.properties
it will make the change permanent and you won't have to play with command-line arguments no matter whether you run JMeter in GUI or non-GUI modes (JMeter restart will be required to pick the properties up)
More information:
Configuring JMeter
Apache JMeter Properties Customization Guide

Running "create-rmi-keystore.bat" file in Jmeter bin folder not working

I'm trying to set up distributed test in Jmeter however whenever I tried to run the "create-rmi-keystore.bat" file in bin folder doesn't seem to work.. It's not opening...Can someone enlighten me what should be the problem and solution.
Nobody can "enlighten" you about the solution unless you run this command in the terminal and share the output with us, the possible reasons are:
You already have rmi_keystore.jks file in the "bin" folder of your JMeter installation
You don't have keytool program in your operating system PATH
I also believe that 99.99999% of users don't need secure communication between JMeter master and slaves as it doesn't add any value and just consuming valuable resources, so I tend to disable secure RMI communication when I run JMeter tests in distributed mode.
Add the next line to user.properties file (lives in "bin" folder of your JMeter installation)
server.rmi.ssl.disable=true
Restart JMeter to pick the property up
Repeat it for all machines which are involved in distributed test
That's it, you don't need this rmi-keystore.jks anymore
References:
Remote hosts and RMI configuration
JMeter Distributed Testing with Docker
Remote Testing
On Unix system use create-rmi-keystore.sh script instead
for Windows systems (called bin/create-rmi-keystore.bat) and Unix like systems (called bin/create-rmi-keystore.sh)
$ ./create-rmi-keystore.sh

java.Nullpointer when read csv file performance test jmeter

I have issue, which is relate to read csv file. When i run jmx file with gui, everything is ok, but when run with non-gui always show error java.lang.Nullpointer exception.
Does anyone know how to fix it?
Have you tried to "See the log file for details"? The file is called jmeter.log and it's being generated in "bin" folder of your JMeter installation or in the folder where you launch JMeter from (if it differs from the above)
You can try troubleshooting the issue yourself by comparing JMeter log files from GUI and from non-GUI execution. Another suggestion is using full path to the CSV file instead of the relative one.
References:
JMeter Hints and Tips: Enabling debug logging
How to Configure JMeter Logging

No jmeter.properties file when installed with brew?

I installed jmeter with brew and built a test plan which I hoped to be able to run on a distributed set of nodes. But I cannot find where the jmeter.properties file is in order to tell my local client where to look for the helper nodes. Any ideas? I tried adding it manually, but it did not take.
The /usr/local/Cellar/jmeter/5.0/bin folder only contains jmeter and jmeter.log
Thanks!
Looking into JMeter formula my expectation is that you should see jmeter.properties file under /usr/local/Cellar/jmeter/5.0/libexec/bin/ folder (this is applicable for JMeter 5.0, for other versions location should be different)
PS1. You can install JMeter without using Homebrew, if you execute the following commands:
cd ~
wget https://www-eu.apache.org/dist//jmeter/binaries/apache-jmeter-5.0.tgz
tar xf apache-jmeter-5.0.tgz
then you will have "normal" JMeter installation as apache-jmeter-5.0 in your home folder
PS2. According to JMeter Best Practices you should be rather using user.properties than jmeter.properties so consider placing your configuration into /usr/local/Cellar/jmeter/5.0/libexec/bin/user.properties file instead
PS3. You can also pass any JMeter property via -J command-line argument so you can do something like
jmeter -JpropertyName=propertyValue
i.e.
jmeter -Jremote.hosts=10.10.10.1,10.10.10.2 -n -t test.jmx -l result.jtl
It's also not in my /usr/local/Cellar/jmeter/5.0/bin folder.
However, if your intention is simply to have jmeter load your user.properties file, then
create a user.properties file inside
/usr/local/Cellar/jmeter/5.0/bin
then execute jmeter by running $ /usr/local/Cellar/jmeter/5.0/bin/jmeter
The user.properties file would be loaded by jmeter.

How to run Jmeter tests from bamboo?

How to run the Jmeter tests from bamboo?
I tried in Jenkins which has the 'Execute windows batch command' option and I just pasted these lines and worked fine
jmeter -n -t C:\apache-jmeter-2.13\apache-jmeter-2.13\bin\test.jmx
I need to work the same with bamboo as well. So I tried with the script task, selected Inline and in the script body I pasted the above lines
It failed with this error
/opt/bamboo/current/temp/TEST-NEW-NEW-33-ScriptBuildTask-6294682377804302931.sh: line 2: C:Jmeterapache-jmeter-2.13bin: command not found
/opt/bamboo/current/temp/TEST-NEW-NEW-33-ScriptBuildTask-6294682377804302931.sh: line 3: jmeter: command not found
I tried with the Command task also in bamboo but no luck.
I'm struggling how to run Jmeter tests from bamboo? Anyone please advise. I see that we can do it with maven I believe, but I don't know much about it, but if anyone has step by step instructions that I can follow that helps as well.
Thanks in advance
Define the jmeter executable as a remote capabiliety.
Then add a new command to your task and select jmeter as executable. Add your arguments and you are done.
It looks like that you need to either need to escape backslashes like:
C:\\apache-jmeter-2.13\\apache-jmeter-2.13\\bin\\test.jmx
or change them to forward slashes as:
C:/apache-jmeter-2.13/apache-jmeter-2.13/bin/test.jmx
Other options on how to kick off a JMeter test from Bamboo build step are:
bamboo-jmeter-plugin
JMeter Ant Task
JMeter Maven Plugin
existing JMeter tests can be executed from Java code, moreover, it is possible to create a JMeter test purely in Java.
See 5 Ways To Launch a JMeter Test without Using the JMeter GUI guide for detailed information on aforementioned options.
I assume your Bamboo is in a linux environment.
From Bamboo on Linux, create a new task and select script:
cd /<your path>/apache-jmeter-2.13/bin
./jmeter -n -t <your path>/test.jmx
If Windows, set environment variable JMETER_HOME to C:\apache-jmeter-2.13\apache-jmeter-2.13\ and then on Bamboo's command line:
cd C:\apache-jmeter-2.13\apache-jmeter-2.13\bin\
jmeter -n -t test.jmx

Resources