JMeter : Run all tests in a file - jmeter

I have around 4 files named:
Test_1.csv
Test_2.csv
...
Each line in each test file has the following format:
method;request
Where, method is the URL that I call and request is the request I make to it. Everything is configured to pick up these values and form URLs.
However, first Test_1.csv must run, then Test_2.csv must run and so on. To do that, I have created 5 thread groups in the following hierarchy:
Test Plan
|
+- Step 1
|
+- HTTP request
+- CSV Data Set Config <- Reads from Test_1.csv
+- Uniform Random Timer
+- Step 2
|
+- HTTP request
+- CSV Data Set Config <- Reads from Test_2.csv
+- Uniform Random Timer
And I have also selected the Run thread groups consecutively option in my test plan. Each thread group is configured for 20 threads. Now, what I want it to do is that, Step 1 should run each and every test in Test_1.csv, then Step 2 should execute and run each and every line in Test_2.csv. However, what is happening is that, Step 1 runs the first 20 lines from Test_1.csv and then Step 2 starts, runs 20 tests and continues. Then, after all steps are done, Step 1 runs again and runs the very same 20 lines from Test1.csv. I want it to just run once, loop through all the lines in the test file and then quit and hand over control to the next thread.
How do I go about this?

Try to use the following schema:
Test Plan
Thread Group #1
Number of Threads: N
. . .
While Controller
Condition: ${__javaScript("${request}"!="<EOF>",)} - until the EOF
CSV Data Set Config
Filename: [path to your file with test-data] - Test_1.csv in this case
Variable Names: method,request
HTTP Request
Uniform Random Timer
. . .
Thread Group #2
. . .
[same for Test_2.csv]
Thread Group #3
. . .
[same for Test_3.csv]

jMeter Plugins' ParameterizedController can help you call parts of your test plan like you call functions in a regular programming language. More importantly, you can parametrise those calls with variables.

Related

How to set -JlopCount = <Forever> in Jmeter non-GUI mode?

Goal is to set the following and run a test plan with a "Thread Group" and "CSV Data Set Config" with 1000 lines/user accounts. So in 20 mins i want 10 threads to go through my 1000 line long csv file.
I set these settings in GUI mode and it does what i want:
Thread Group .
num of threads - 10 .
LoopCount=Forever .
scheduler=on .
duration=7200 .
"CSV Data Set Config"
Recycle on EOF? - False
Stop thread of EOF? - False
Sharing mode=All Threads
The problem is that i can't reproduce these setting from non_gui mode. I run it as follows and it only goes through # of csv lines equaling the # of threads set. So if i set 20 threads it will go through 20 lines of a file and exit.
-Jseconds=1200
-JthreadCount=20
-JcsvFile=../../user_files/j2kUsers.csv
-JloopCount=???
Use -1 as the property value, Thread Group has a Loop Controller under the hood and as per documentation:
The value -1 is equivalent to checking the Forever toggle.
You can also put the values to user.properties file like:
seconds=1200
threadCound=20
etc.
to avoid entering the values each time you run the script, the values van be overriden via -J command line argument at any time.
More information: Apache JMeter Properties Customization Guide

Adding Job Array elements in Slurm after submission

I'm trying to use a Slurm-operated cluster to run LS-Dyna (a finite-element simulation program with a limited number of licenses available on my cluster). I am trying to write my batch scripts so that I do not waste processing time due to this license limit (as well as to improve legibility when running 'squeue' commands) by using job arrays -but I'm having trouble making that work.
I want to run identical Bash scripts in a variety of FEM meshes, each of which I have organized into different subfolders.
Given this folder structure on my cluster...
cluster root
|
...
|
|-+ my scratch space's root
|
|-+ this project
|
|--+ lat_-5mm
| |- runCurrentLine.bash
| |- other files
|
|--+ lat_-4.75mm
| |- runCurrentLine.bash
| |- other files
|
|--+ lat_-4.5mm
| |- runCurrentLine.bash
| |- other files
|
...
|
|--+ lat_5mm
| |- runCurrentLine.bash
| |- other files
|
|
|-sendDynaRuns.bash
|-other dependencies
...I'm trying to submit "runCurrentLine.bash" in each folder by running the following script in my login node.
#!/bin/bash
iter=0
for foldernow in */; do
# change to subdirectory for current line iteration
cd "./${foldernow}";
# make Slurm and user happy
echo "sending LS Dyna simulation for ${pos}mm line..."
sleep 1
# first line only: send batch, and get job ID
if [ "${iter}" == 0 ];then
# send the batch...
jobID=$(sbatch -J "Dyna" --array="${iter}"%15 runCurrentLine.bash)
# ...ensure that Slurm's output shows on console (which includes the job ID)...
echo "${jobID}"
# ...and extract the job ID and save as a variable
jobID=$(echo "${jobID}" | grep -Eo '[+-]?[0-9]+([.][0-9]+)?')
# subsequent lines: add current line to job array
else
scontrol update --jobid="${jobID}" --array="${iter}"%15 runCurrentLine.bash
fi
# prepare to move onto next position
iter=$((iter+1))
cd ../
done
This setup properly sends the batch job for the first line, at -0.25mm*. However, for the second line onwards, it doesn't seem to do the same thing... This is what I end up getting on my console:
*: I intended the "lat_xmm" folders to be numerically ordered, but Unix doesn't seem to recognize that
$ ./sendDynaRuns.bash
sending LS Dyna simulation for -0.25mm line...
Submitted batch job 1081040
sending LS Dyna simulation for 0.25mm line...
sbatch: error: Batch job submission failed: Invalid job id specified
sending LS Dyna simulation for -0.5mm line...
sbatch: error: Batch job submission failed: Invalid job id specified
I know that runCurrentLine.bash runs just fine if I manually send it as a batch (and it runs to completion within the time limit I specified in-file, mainly since it doesn't have to compete with other lines for open licenses). What should I do to be able to get my code to work?
Thank you in advance!
As state by #Poshi, you cannot add jobs to an existing array.
I would create a submission script like this one:
#!/bin/bash
#SBATCH --array=1-<nb of folders>%15
# ALL OTHER SLURM SBATCH DIRECTIVES HERE
folders=(lat_*)
foldernow=${folders[$SLURM_TASK_ARRAY_ID]}
cd $foldernow && ./runCurrentLine.bash
The only drawback is that you need setup explicitly the number of jobs the array based on the number of folders.

how to pass specific arguments in batch commands during jenkins builds

I'm trying to use Jenkins to automate performance testing with JMeter,
each build is a single JMeter test and I want to increase the number of users(threads) for each Jenkins build if the previous was successful.
I have configured most of the build, with SSH plugin I can restart Tomcat, copy catalina.out, with publishing performance I can open the .jtl file and determine if the build was successful.
What I want is to execute a different batch command for the next build(to increase the number of users(threads) and user id's)
For example:
jmeter -Jthreads=10 -n -t C:\TestScripts\script.jmx -l C:\TestScripts\Jenkins.jtl
jmeter -Jthreads=20 -n -t C:\TestScripts\script.jmx -l C:\TestScripts\Jenkins.jtl
jmeter -Jthreads=30 -n -t C:\TestScripts\script.jmx -l C:\TestScripts\Jenkins.jtl...
Is there some good jmeter plugin some counter that i can use to increase some variable by 10 each time:
jmeter -Jthreads=%variable1%...
I have tried by setting environmental variables and then incrementing that variable by:
"SET /A thread+=10"
but it doesn't change that variable because jenkins opens its own CMD, a new process :
("cmd /c call C:\WINDOWS\TEMP\jenkins556482303577128680.bat")
Use the following SET command to increase threads variable by 10:
SET /A threads=threads+10
Or inside double quotes:
SET /A "threads+=10"
Not knowing your Jenkins configuration, and which plugins you have installed and how do you run the test it is quite hard to come up with the best solution.
The only "universal" workaround I can think of is writing the current number of threads into a file in Jenkins workspace and reading the value of threads from the file on next execution.
Add setUp Thread Group to your Test Plan
Add JSR223 Sampler to your Thread Group
Put the following Groovy code into "Script" area:
import org.apache.jmeter.threads.ThreadGroup
import org.apache.jorphan.collections.SearchByClass
import org.apache.commons.io.FileUtils
SampleResult.setIgnore()
def file = new File(System.getenv('WORKSPACE') + System.getProperty('file.separator') + 'threads.number')
if (file.exists()) {
def newThreadNum = (FileUtils.readFileToString(file, 'UTF-8') as int) + 10
FileUtils.writeStringToFile(file, newThreadNum as String)
def engine = ctx.getEngine()
def test = org.apache.commons.lang.reflect.FieldUtils.getField(engine.getClass(), 'test', true)
def testPlanTree = test.get(engine)
SearchByClass<ThreadGroup> threadGroupSearch = new SearchByClass<>(ThreadGroup.class)
testPlanTree.traverse(threadGroupSearch)
def threadGroups = threadGroupSearch.getSearchResults()
threadGroups.each {
it.setNumThreads(newThreadNum)
}
} else {
FileUtils.writeStringToFile(file, props.get('threads'))
}
The code will write down the current number of threads in all Thread Groups into a file called threads.number in Jenkins Workspace and on subsequent runs it reads the value from it, adds 10 and writes it back.
For now i am creating 20 .jmx files (1.jmx, 2.jmx , 3.jmx ...) each whith a different number of users. and calling them whit this command :
jmeter -n -t C:\TestScripts\%BUILD_NUMBER%.jmx -l C:\TestScripts\%BUILD_NUMBER%.jtl
the first billd will call 1.jmx the second 2.jmx ...
it isn't the best method but it works for now. I will try your advice over the weekend when i have more time.
i have found the a solution that works for me, it inst pretty. I created a python script which changes a .CVS fil from which JMeter reads the number of threads and the starting user id. This python script incremets the starting user id by the number of threads in the previous bild and the number of threads by 10
file = open('C:\\Users\\mp\\AppData\\Local\\Programs\\Python\\Python37-32\\eggs.csv', 'r')
a,b=file.readlines()[0].split(",")
print(a,b)
b=int(b)
a=int(a)
b=a+b
a=a+10
print(a,b)
f = open("C:\\Users\\mp\\AppData\\Local\\Programs\\Python\\Python37-32\\eggs2.csv", "a")
f.write(str(a)+","+str(b))
f.close()
I have python on my pc and a i am calling the script in Jenkins as a windows Bach command
C:\Users\mp\AppData\Local\Programs\Python\Python37-32\python.exe C:\Users\mp\AppData\Local\Programs\Python\Python37-32\rename_write_file.py
I am much better in python than Java so I implemented this in Python.
So for each new test,the CSV file from which jmeter reads values is changed.

For loop to check for a xpath in Robot Framework

I am a newbie in Robot Framework. I want to implement a For loop to check for a xpath on the page. It should wait for 10 seconds for the xpath, if the xpath is still not there then wait again for 10 seconds, if the xpath has appeared then exit the loop and move ahead. A total of 10 iterations are required to wait for the element.
I am trying the below:
|| ${count}= | Get Matching Xpath Count | xpath=//*[.='Continue'] |
|| :FOR | ${loopIndex} | IN RANGE | 10
| Wait Until Page Contains Element | xpath=//*[.='Continue'] | 10
| Exit For Loop If | ${count}>0
|| Log | Got out of loop
I am getting the error right now as:
FAIL : Element 'xpath=//*[.='Continue']' did not appear in 10 seconds.
Let me know if I have missed some information. I am using RIDE for editing.
EDIT:
I have to do it in a loop. I am asking for help regarding this so that I can use this loop example in other places of my work.
You can achieve this either with a FOR loop or with a "Wait until keyword succeeds".
See both examples:
*** Test Cases ***
test with for loop
:FOR ${loopIndex} IN RANGE 10
\ ${element_is_here} = Run Keyword and return status Page should contain element xpath=//*[.='Continue']
\ Exit For Loop If ${element_is_here}
\ sleep 10s
Log I found the element
test with WUKS
wait until keyword succeeds 10s 10s Page should contain element xpath=//*[.='Continue']
Log I found the element
The keyword is failing, so the test case is failing. What you need to do instead is either not use a loop and simply wait for 100 seconds, or use something like Run keyword and ignore error or Run keyword and continue on failure (eg: Run keyword and ignore error | Wait until page contains element | ...)

Run thread groups concurrently but evenly distributed over time

I have a Test plan with several Thread Groups (TG) which have a different Loop Count.
Test Plan
|--TG 1: loop count=5
|--TG 2: loop count=10
|--TG 3: loop count=3
These thread groups are executed concurrently. However because the loop count of threads differ, some threads are finished while others are still running. What I would like is for JMeter to manage the execution of each Thread Group based on a ratio of the the highest Loop Count in the Test plan in order to create a more even time distribution and ensure all Thread Groups roughly end at the same time. For instance:
Test Plan
|--TG 1: loop count=5, execute every 2 occurrences of TG 2
|--TG 2: loop count=10
|--TG 3: loop count=3, execute every 3.3 occurrences of TG 2
In terms of execution, things would look like this:
[round 1] TG1: run | TG2: run | TG3: run
[round 2] TG1: wait | TG2: run | TG3: wait
[round 3] TG1: run | TG2: run | TG3: wait
[round 4] TG1: wait | TG2: run | TG3: run
etc...
Can I configure JMeter to acheive this, if so how?

Resources