I am looking for a way to tag Jmeter test cases.
We are using Jmeter for functional test , so we have a lot of test cases , not every test cases are run for every application configuration.
So based on configuration we need tag our test cases and run the test set accordingly from command line.
(Some thing we can do in testng and other framework , where you tag TC and during run you provide the tag so that TC only with that tag are executed)
If there is no tagging available then i feel that i will need to create multiple test set as per configuration and run them accordingly.
In most cases the Test overlap between this test sets and this will result in duplication and require quite a good maintenance.
Please suggest if you all see any solution to this problem.
One solution that I can think of is explained below:
- Declare a property for every test cases
- Process that property to execute your test cases. Use If Controller to check whether that property is passed or not
- Control that property via command line or via GUI. If you want to run some of them pass that property only.
Practical example is shown below:
Test Plan will look like this:
Test Case steps will be inside If controller, and if controller will decide whether to run that Test Case or not, depending upon what you are passing in that property.
Property declaration for all you test cases.
I have designed this in such a way that you can execute that via GUI as well as via command prompt.
If Controller logic
${__BeanShell("${TC1}"=="ON",)}
Execution via command line
jmeter -n -t <>.jmx -JTC1=ON -JTC3=ON -j sample.log
Here I am running TC1 and TC3, depending upon your requirement you can pass whichever scenario you need to execute.
Related
I am trying to update the user defined variable set at a test plan level from a thread group. This is my scenario:
Test plan> user defined variable (variable name/value: fBurst=0)
Test plan> Threadgroup1 > Once only controller> JSR223 test plan (inside the test plan I have the following code)
log.info("fBurst user defined value : " + ${fBurst})
vars.put("fBurst", Integer.toString(111))
log.info("fBurst user defined value vars.put' : " + ${fBurst})
props.put("fBurst", 222)
log.info("fBurst user defined value props.put' : " + ${fBurst})
What I am trying to do here is to update the fBurst user defined value from inside the once only controller sampler and so far not been able to do it using the vars.put or props.put. Is there another way to do this?
Take a look at JSR223 Sampler documentation:
The JSR223 test elements have a feature (compilation) that can significantly increase performance. To benefit from this feature:
Use Script files instead of inlining them. This will make JMeter compile them if this feature is available on ScriptEngine and cache them.
Or Use Script Text and check Cache compiled script if available property.
When using this feature, ensure your script code does not use JMeter variables or JMeter function calls directly in script code as caching would only cache first replacement. Instead use script parameters.
So replace ${fBurst} with vars.get('fBurst") or props.get('fBurst') correspondingly and it should start working as expected
More information: Top 8 JMeter Java Classes You Should Be Using with Groovy
Cypress tests with mocha multi reports don't show results from all the tests
My test structure looks like so:
cypress
integration
module1
module1test1_spec.js
module1test1_spec.js
module2
module2test1_spec.js
module2test1_spec.js
I have set up Cypress to use mocha-multi-reports like in instruction provided under https://docs.cypress.io/guides/tooling/reporters.html#Multiple-Reporters
My config.json looks exactly like here: https://github.com/cypress-io/cypress-example-docker-circle#spec--xml-reports
When Cypress finishes testing, results.xml file shows results from last test spec ONLY; module2test1_spec.js
How to configure this to get the aggregated results from all test spec?
You can use [hash].xml in your path.
e.g. ./path_to_your/test-results.[hash].xml. [hash] is replaced by MD5 hash of test results XML. This enables support of parallel execution of multiple mocha-junit-reporter's writing test results in separate files.
https://www.npmjs.com/package/mocha-junit-reporter#results-report
I solved this problem with this way.
my config.json file seems like this:
"reporterEnabled": "spec,json, mocha-junit-reporter",
"mochaJunitReporterReporterOptions": {
"mochaFile": "multiple-results/[hash].xml",
Just to add to the answer above, if you would like to merge all these multiple [hash].xml created into a single mergedreport.xml report, then you can use this package junit-report-merger which is useful for running it on CI pipeline which usually looks for a single reporter with a command like the one below:
jrm ./cypress/reports/mergedreport.xml "./cypress/reports/*.xml"
My JMeter test receives a parameter to specify test environment like PROD, DEV.
Results from both test environment will be sent to a certain GraphiteHost. But I need to separate the results of each environment by using rootMetricsPrefix.
For example, results from PROD will use prefix global.myapp.performance.prod. while the results from DEV will use prefix global.myapp.performance.dev..
So I set the rootMetricsPrefix in my Backend Listener as global.myapp.performance.${__groovy($__P(env).toLowerCase())}..
Unfortunately, It doesn't work.
Data in Graphite doesn't contains the environment name.
Can anyone tell me how to solve this?
You can use new changeCase function to lower case your value:
${__changeCase(${__P(env)},LOWER,)}
It will read the property and then execute lower case on value
There is an error in your expression it should be:
global.myapp.performance.${__groovy("${__P(env)}".toLowerCase())}
This will also work:
global.myapp.performance.${__groovy(props.get("env").toLowerCase())}
But for performances, it is better to use the solution provided by #user7294900
I want to set the value of 'teamcity.build.branch' dynamically according to the result of another TC build configuration part of the build pipeline.
Is that even possible? It looks like the value is evaluated and used at the start of the build pipeline.
UseCase:
I am executing a TC build configuration that will generate a unique number
in the connected TC build configuration part of the same pipeline I want the number to be used in the 'teamcity.build.branch' - just for visualization purposes
I am already using message service to overwrite the parameter, but the change is not taken into account. It looks like the value is read in the very early stage of the build process.
Check below reference containing build number and git branch name
https://octopus.com/blog/teamcity-version-numbers-based-on-branches
You could overwrite the value of the parameter by using a simple script that emits a "set parameter" service message.
By using a dedicated service message in your build script, you can dynamically update build parameters of the build right from a build step (...)
With that approach, here are the steps that you need to perform:
In the first build config, define a custom build parameter and set its value to the unique number you're generating. Do this directly from the script that generates the unique number by writing something like this to STDOUT:
##teamcity[setParameter name='magicNumber' value='1234']
In the dependent build config, you now have access to that parameter. Using a second build script, you can overwrite the teamcity.build.branch with the same mechanism:
##teamcity[setParameter name='teamcity.build.branch' value='the new value']
Note 1: I recommend against overwriting the built-in parameters, because this might have strange side-effects. Rather, define a custom parameter in the second build config and use that for your visualization purposes.
Note 2: In case you decide to ignore Note 1, it may be necessary to overwrite the build parameters by setting the dependency property as outlined in the docs in section "Overriding Dependencies Properties":
##teamcity[setParameter name='reverse.dep.*.teamcity.build.branch' value='the new value']
I've got a set of parametrized builds in Jenkins that to build I have to click 'Build Now' and then enter a value for the parameter (in this case called GIT_TAG_NAME). I'd like to trigger a set of these parametrized builds that all use the same parameter without typing it multiple times.
I'm trying to get this working with the Build Flow Plugin (https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin), by making a master build flow that triggers all the other builds, but I'm not understanding the plugin syntax, or maybe this just isn't possible.
My DSL looks like:
out.println "-------------------------"
out.println 'Building all OTA builds at tag: '
out.println params["GIT_TAG_NAME"]
out.println "-------------------------"
build( "SomeOTA-Build-1", param1: params["GIT_TAG_NAME"] )
build( "SomeOTA-Build-2", param1: params["GIT_TAG_NAME"] )
The print statement prints the parameter correctly, but the child builds don't seem to get the parameter passed in to them.
Try using the Parameterized Trigger Plugin -
Set Job-A with a parameter GIT_TAG_NAME - this is your "front-end"
Set Job-B1, Job-B2 and Job-B3 with the same parameter GIT_TAG_NAME - those do the actual work
Set Job-A to Trigger parameterized build on other projects (in Post-build Actions) and pass Current build parameters to the triggered jobs(need a trigger per derived job - either with same or different conditions)
Now, running Job-A will rigger the other jobs, while passing them the value of GIT_TAG_NAME.
EDIT:
There is a Plugin from TIKAL that uses a different approach:
Multijob Plugin tries to squeeze all job-steps into one big job
(have not tried it, so cannot comment on this approach).
If taking the first approach, you would probably want to take a look at the Join Plugin -
this plugin allows a job to be run after all the immediate downstream jobs have ended.