Question: how to execute different tests from the same test suite, depending on parameter sent from Jenkins (version)?
Background: have application and 5 versions of it.
The difference between them is small, but exist and it is implemented in testcases.
So I have group of testcases which are the same from version to version and I have groups of test cases which are different from version to version.
What by Boss wants: we should be able to select application version number in Jenkins right in the same place where we selecting against which server to execute and what browser to use.
So I have in my text.xml
<suite name="aaaa">
<parameter name="SiteAddress" value="${SiteAddress}"/>
<parameter name="Version" value="${version}"/>
in POM.xml
<systemPropertyVariables>
<SiteAddress>${serv_url}</SiteAddress>
<browser>${browser}</browser>
<Version>${version}</Password>
</systemPropertyVariables>
Question again:
what is pattern to switch test cases depending of parameter value?
I just can't figure it out how to implement it.
Thank you!
This how I implemented a worflow in Jenkins by creating testng xml programmatically and use the command line args for filtering tests for execution.
Jenkins calls Ant build.xml which in turn calls the Java class, where I create testng xml programmatically and the command line args are passed from jenkins > ant > java class. Let me know if you need more help on this.
Related
I am trying to execute my JMeter scripts through Maven. I am using Jmeter 5.3 and I am using below pom.xml.
When I execute it through command prompt mvn verify -Dusers=1 -DrampUp=1 -Dloopcount=1 -Durl=1 -DSmokeDemoTest=1, it shows me the build success message but seems it is not executing my JMeter test script that's why I am not getting JTL report file in target folder. I am attaching command prompt execution screenshot. Could you please suggest and help me out on this issue.
POMXML:
CommandPromptExecution Image:
Your test is running fine, the only problem is that there are 0 samplers executed therefore there are no results.
From your very beautiful screenshot of the pom.xml file it's not very clear how do you define the users property, you should have something like:
<properties>
<users>1</users>
<rampUp>1</rampUp>
<loopcount>1</loopcount>
<url>http://example.com</url>
<suite>foo</suite>
</properties>
in the beginning of your pom.xml file.
Once done the properties will be declared and you will be able to override them via -D command-line argument.
Also make sure to properly reference the properties in JMeter test plan using __P() function
If you will still be experiencing problems check the jmeter.log file(s) under target/jmeter/logs folder for any suspicious entries
More information:
How to create user defined properties in Maven
How to Use the JMeter Maven Plugin
I know how to configure Jenkins to run a particular test class using mvn -Dtest = nameoftest test in my windows batch command. However, how can I run a xml script that contains many tests?
for anyone who stumbles upon this, you run an xml file via jenkins like this..
mvn test -DsuiteXmlFile = src/whatever/path/my_file.xml
We have execute the different test scenarios in different feature file using cucumber with maven in eclipse.
EX:
FeatureFile1:Login1
Scenario:UserLogin
Given:the userName is "aaa"
When:the login button is clicked
Then:the user login as successfully
FeatureFile2:Login2
Scenario:UserLogin01
Given:the userName is "bbb"
When:the login button is clicked
Then:the user login as successfully
In above example how to run in parallels and also how to configure in pom.xml file.
Regards,
Gopal
I have no idea how to solve it using maven only. I'm using jbehave and run tests in parallel with testng. It provides xml which operates test running and there you could specify amount of parallel threads and what exactly should be paralleled (metods, tests, suits). On build server use maven command:
mvn test -Dtestng.xml.file=suite.xml
Also you could use surefire plugin with any test provider.
http://testng.org/doc/index.html
http://maven.apache.org/surefire/maven-surefire-plugin/index.html
I am running gradle to run the tests from Windows command line. What I do to run a single test is:
gradlew.bat chromeTest -DchromeTest.single=test1Spec
or for all the tests:
gradlew.bat chromeTest
If I try to run only two test classes like this:
gradlew.bat chromeTest -DchromeTest=test1Spec,test2Spec--info
then gradle starts to run all the tests.
What I need: is to run only 2 or 3 Groovy classes. To be specific, neither one nor all. Any help would be really beneficial! Sorry, for reposting this question again.
-DtestTaskName supports wildcards such as Test*Spec or foo.bar.*Spec, but is limited to a single pattern. If you need support for multiple patterns, you'll have to implement your own command line parameter (which in the simplest case means reading a system property) and use that to configure Test#include or Test#getFilter. (See Test in the Gradle Build Language Reference for details.)
Gradle 1.10 introduced --tests as a replacement for -DtestTaskName, but again, only one pattern is supported.
I need to setup performance tests which are run automatically triggered by a CI system. For that I want to use JMeter due to some scripts and experience already exist and I want to combine it with Maven.
During my research for a reasonable plugin I found that two plugins are existing:
jmeter-maven-plugin:
http://wiki.apache.org/jmeter/JMeterMavenPlugin
chronos-jmeter-maven-plugin:
http://mojo.codehaus.org/chronos/chronos-jmeter-maven-plugin/usage.html
Which one is better to be used? Both seem to be currently maintained and under development. Is there any experience on this? Even the configuration is similar.
I would be happy to get some hints to help me descide without playing around with both plugins for some days.
I haven't yet used the .jmx files with maven and specifically those plugins you mention.
But I can think of a way how to do it if I needed that.
So consider this, you can execute jmeter test in no gui mode.
Create a shell script wrapper that will execute the jmeter test in no gui mode, example (jmeter_exe.sh):
$JMETER_HOME/bin/jmeter.sh -n -t MY_LOAD_TEST.jmx -l resultFile.jtl
So this will execute the given script and store results in the .jtl file, you can use that to display your test results maybe this post will be useful to you, it's off topic for now.
With step one done.
2.You could then create directory scripts in your project root. Than you can put this in your pom.xml :
<plugin>
<artifactId>exec-maven-plugin</artifactId>
<groupId>org.codehaus.mojo</groupId>
<executions>
<execution>
<id>Run load Test</id>
<phase>generate-sources</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>${basedir}/scripts/jmeter_exe.sh</executable>
</configuration>
</execution>
</executions>
</plugin>
And voila your test is executed during generate-sources phase. This might have been easier with the plugins you mentioned but I have no knowledge of those, this is what just came to my mind.
Use jmeter-maven-plugin: http://wiki.apache.org/jmeter/JMeterMavenPlugin.
It's the de-facto one and (as #Ardesco mentioned above) it doesn't require anything to be installed, which gives you abstraction on where JMeter executable is installed and all those kind of problems...
Word(s) of warning on the apache plugin (lazerycode):
It suppresses JMeter output by default, add the following configuration settings to prevent that:
<configuration>
<suppressJMeterOutput>false</suppressJMeterOutput>
<!-- to override debug logging from the plugin (although also in jmeter.properties) -->
<overrideRootLogLevel>debug</overrideRootLogLevel>
<jmeterLogLevel>DEBUG</jmeterLogLevel>
</configuration>
Looking at the source (of version 1.8.1), it seems the -Xms and Xmx are limited to 512
The plugin swallows exceptions so your tests may fail but you don't know why. It looks like they've just completed but not provided results.
The jmeter mojo kicks off jmeter as a new java process but does not provide the capacity to provide any arguments to this execution. So if exceptions are swallowed (See above), and logging isn't sufficient (which it may not be) it's not easy to debug the process to fing out what's wrong. We (my colleague) added the debug args to the process execution and debugged the jmeter call to find out.
you get informative output running jmeter directly for dev purposes. I'd say it's even more informative in the jmeter UI output.
I've not used chronos mind.
JMeter Maven Plugin by #Ardesco is updated every time JMeter version is released.
It is very well documented and works perfectly.
It is easily setup and allow easy addition of plugins like JMeter-Plugins or commercial plugins as long as required libraries.
You can read a full blog showing the setup for old version 1.1.10:
http://www.ubik-ingenierie.com/blog/integrate-load-testing-in-build-process-with-jmeter-ubikloadpack-maven/
For more recent version 2.5.1 (as of November 2017) ensure you read documentation:
https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/wiki