How to Pass the features folder dynamically to a Jenkins->Maven->Testng Runner File - maven

In my jenkins pipeline I am invoking my test like
"mvn test -Drun_location=US"
And my pom.xml file looks like this
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<systemPropertyVariables>
<run.location>${run_location}</run.location>
</systemPropertyVariables>
<suiteXmlFiles>
<suiteXmlFile>${basedir}/testng.xml</suiteXmlFile>
</suiteXmlFiles>
<skipTests>false</skipTests>
<testFailureIgnore>false</testFailureIgnore>
</configuration>
</plugin>
And my testng.xml file looks like this
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite>
<test>
<parameter name="location" value="${run_location}"/>
<classes>
<class name="com.test.Runner_Jenkins"/>
</classes>
</test>
</suite>
I need to pass this parameter to my runner file to set the feature directory dynamically. How do I accomplish this.
#CucumberOptions (
features = {"features/${run_location}"},
glue = "StepDefinitions"
)

There are two ways of achieving your requirement.
You could use the build parameters of Jenkins and pass your run_location. Then using the Execute Shell script option that runs before the maven build, you could use the sed command to add this parameter to your pom.xml file, feature file, and testng.xml file. sed command replaces the existing text with new text. You could search google about the usage of sed command.
The second option is to pass the run-location as a Jenkins build parameter. Then receive the build parameter inside your java code. Then add this run-location as a parameter to your testng xml file dynamically. This is called programmatic execution of testng. The below code will do this. The below code reads an existing testng XML file and adds a parameter based on your Jenkins input. In this way, you could receive this value inside any of your tests using the code mentioned below.
But this approach will not modify your pom.xml file and feature file. During the execution you have to figure out way to get this value to your code using java.
Programmatic execution of testng with dynamic jenkins parameter:
String run_location = System.getProperty("run_location");
TestNG tng = new TestNG();
File initialFile = new File("testng.xml");
InputStream inputStream = FileUtils.openInputStream(initialFile);
Parser p = new Parser(inputStream);
List<XmlSuite> suites = p.parseToList();
List<XmlSuite> modifiedSuites = new ArrayList<>();
for (XmlSuite suite : suites) {
XmlSuite modifiedSuite = new XmlSuite();
modifiedSuite.setParallel(suite.getParallel());
modifiedSuite.setThreadCount(deviceNames.size());
modifiedSuite.setName(suite.getName());
modifiedSuite.setListeners(suite.getListeners());
List<XmlTest> tests = suite.getTests();
for (XmlTest test : tests) {
for (int i = 0; i < deviceNames.size(); i++) {
XmlTest modifedtest = new XmlTest(modifiedSuite);
HashMap<String, String> parametersMap = new HashMap<>();
parametersMap.put("run_location", run_location);
modifedtest.setParameters(parametersMap);
modifedtest.setXmlClasses(test.getXmlClasses());
}
}
modifiedSuites.add(modifiedSuite);
}
inputStream.close();
tng.setXmlSuites(modifiedSuites);
tng.run();
Accessing the Parameter value inside your tests
#Test
#Parameters(value={"run_location"})
public void executeBeforeTest(String runlocation){
/**Your code goes her**/
}

Related

testng_failed.xml does not get refreshed before running and run older failed testcases

Actually question related to testng-failed.xml has already been asked many times but my problem is little different. I want to run all the failed test cases together so what i did is in my pom I passed testng-failed.xml.
But the problem I am facing is first my testng.xml runs then testng-failed.xml and then it testng-failed.xml gets overridden. Due to this , suppose if i give a second time fresh run to my testcases, testng.xml runs, then my testng-failed.xml has previously failed test cases so it runs the previously failed cases and then updates testng-failed.xml with this time failed cases.
I dont knoe which listener to add to handle this issue that whenever i run first testng.xml should run , then it should override testng-failed.xml and then testng-failed.xml should run.
I am using Maven, selenium, testng.
I just eneterd testng-failed.xml in my pom as shown below. Please let me know which listner to use
<suiteXmlFiles>
<suiteXmlFile>src/resources/testng/testng.xml</suiteXmlFile>
<suiteXmlFile>test-output/testng-failed.xml</suiteXmlFile>
</suiteXmlFiles>
Create class 'RetryListener' by implementing 'IAnnotationTransformer'.
public class RetryListener implements IAnnotationTransformer {
#Override
public void transform(ITestAnnotation testannotation, Class testClass,
Constructor testConstructor, Method testMethod) {
IRetryAnalyzer retry = testannotation.getRetryAnalyzer();
if (retry == null) {
testannotation.setRetryAnalyzer(Retry.class);
}
}
}
Now Create another class.
public class Retry implements IRetryAnalyzer {
private int retryCount = 0;
private int maxRetryCount = 1;
// Below method returns 'true' if the test method has to be retried
else 'false'
//and it takes the 'Result' as parameter of the test method that just
ran
public boolean retry(ITestResult result) {
if (retryCount < maxRetryCount) {
System.out.println("Retrying test " + result.getName() + " with status "
+ getResultStatusName(result.getStatus()) + " for the " + (retryCount+1) + " time(s).");
retryCount++;
return true;
}
return false;
}
public String getResultStatusName(int status) {
String resultName = null;
if(status==1)
resultName = "SUCCESS";
if(status==2)
resultName = "FAILURE";
if(status==3)
resultName = "SKIP";
return resultName;
}
}
And Now Add below lines in your testNG xml file
<listeners>
<listener class-name="com.pack.test.RetryListener"/>
</listeners>
and Do not pass Xml file in pom.xml
Hope it will works
Thanks
Why are you running the testng xml and failed test xml in the same testng task. You should have to separate build task, first that runs testng xml and generates the failed tests xml and then another task running the failed test xml. It will work.
I implemented run one time and rerun three times only the newly failed tests.
mvn $par1=$pSuiteXmlFile test > $test1log
mvn $par1=$failedRelPath test > $failed1log
mvn $par1=$failedRelPath test > $failed2log
mvn $par1=$failedRelPath test > $failed3log
It works, but with small test-cases-count. I have a suite with 300 tests in it and somehow the testng-failed.xml is not created by surefire/testng after the main (first) run. When the suite is smaller, the testng-failed.xml is created as required.

How to make CodeNarc force maven build to fail

I'm trying to integrate CodeNarc into a maven based project and I've been running into problems.
I want to use a custom ruleset, and when a rule is violated, I'd like my maven build to fail.
How can I configure codenarc so that violations of rules lead to a failure when I run the following?
mvn clean install
Also, the documentation for configuring CodeNarc in a POM doesn't explain how to reference where my custom ruleset is. Any advice for how to set that up? Thanks!
When I run mvn clean install with the configurations below (I have a groovy file with blatant violations in accordance with my ruleset)
My build succeeds. :(
I tried referencing my own ruleset and no violations were being produced.
I took away a rulesetfiles property in the POM and it started producing violations.
(But I don't get to choose my own)
Anyone know how to make it actually read a custom rule set file? I tried with both xml and groovy.
Here's my ruleset and plugin config from my POM:
<ruleset xmlns="http://codenarc.org/ruleset/1.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://codenarc.org/ruleset/1.0 http://codenarc.org/ruleset-schema.xsd";
xsi:noNamespaceSchemaLocation="http://codenarc.org/ruleset-schema.xsd">;
<description>Dummy rule set</description>
<rule class='org.codenarc.rule.formatting.SpaceAfterIf'>
<property name='priority' value='1'/>
</rule>
<rule class='org.codenarc.rule.basic.EmptyIfStatement'>
<property name='priority' value='1'/>
</rule>
</ruleset>
I referenced this ruleset in my POM like this:
<groupId>org.codehaus.mojo</groupId>
<artifactId>codenarc-maven-plugin</artifactId>
<version>0.18-1</version>
<configuration>
<sourceDirectory>${basedir}/src/test/groovy</sourceDirectory>
<maxPriority1Violations>0</maxPriority1Violations>
<maxPriority2Violations>0</maxPriority2Violations>
<maxPriority3Violations>0</maxPriority3Violations>
<rulesetfiles>${basedir}/rulesets/ruleset.xml</rulesetfiles>
<xmlOutputDirectory>${basedir}/</xmlOutputDirectory>
</configuration>
<executions>
<execution>
<id>execution1</id>
<phase>install</phase>
<goals>
<goal>codenarc</goal>
</goals>
</execution>
</executions>
I was struggling with the same some time ago. I remember it was possible to run with maven properly but I don't have this config. Why? Because CodeNarc needs to compile your sources for purpuse of some rules execution. But codenarc maven plugin doesn't pass classpath and compilation was failing.
So I went for different approach which is running CodeNarc as a test source with ant task. It looks like:
import spock.lang.Specification
class GroovyCodeNarcStaticAnalysisRunner extends Specification {
private static final GROOVY_FILES = '**/*.groovy'
private static final ANALYSIS_SCOPE = 'src/main/groovy'
private static final RULESET_LOCATION = 'file:tools/static-analysis/codenarc.xml'
private static final HTML_REPORT_FILE = 'target/codenarc-result.html'
private static final XML_REPORT_FILE = 'target/codenarc-result.xml'
def 'Groovy code should meet coding standards'() {
given:
def ant = new AntBuilder()
ant.taskdef(name: 'codenarc', classname: 'org.codenarc.ant.CodeNarcTask')
expect:
ant.codenarc(
ruleSetFiles: RULESET_LOCATION,
maxPriority1Violations: 0,
maxPriority2Violations: 0,
maxPriority3Violations: 0)
{
fileset(dir: ANALYSIS_SCOPE) {
include(name: GROOVY_FILES)
}
report(type: 'text') {
option(name: 'writeToStandardOut', value: true)
}
report(type: 'xml') {
option(name: 'outputFile', value: XML_REPORT_FILE)
}
report(type: 'html') {
option(name: 'outputFile', value: HTML_REPORT_FILE)
}
}
}
}
You don't need to use Spock's Specification for that. Any test runner will do. On the maven side it's enough to make CodeNarc dependency configured with scope test.

How can I use Jenkins to run my integration tests in parallel?

Right now we've got a project that builds in two jobs. 1) Is the standard build with unit tests. 2) is the integration tests. They work like this:
build the whole project, run unit tests, start integration test job
build the whole project, deploy it to the integration server, run client side integration tests against integration server
The problem is step 2) now takes over an hour to run and I'd like to parallelize the integration tests so that they take less time. But I'm not exactly sure how I can/should do this. My first thought is that I could have two step 2)s like this:
build the whole project, run unit tests, start integration test job
build the whole project, deploy it to the integration server1, run client side integration tests against integration server1
build the whole project, deploy it to the integration server2, run client side integration tests against integration server2
But then, how do I run half the integration tests on integration server1, and the other half on integration server2? I am using maven, so I could probably figure out something with failsafe and a complex includes/excludes pattern. But that sounds like something that would take a lot of effort to maintain. EG: when someone adds a new integration test class, how do I ensure that it gets run on one of the two servers? Does the developer have to modify the maven patterns?
I found this great article on how to do this, but it gives a way to do it in Groovy code. I pretty much followed these steps, but I haven't written the code to distribute the tests evenly by duration. But this is still a useful tool so I'll share it.
import junit.framework.JUnit4TestAdapter;
import junit.framework.TestSuite;
import org.junit.Ignore;
import org.junit.extensions.cpsuite.ClassesFinder;
import org.junit.extensions.cpsuite.ClasspathFinderFactory;
import org.junit.extensions.cpsuite.SuiteType;
import org.junit.runner.RunWith;
import org.junit.runners.AllTests;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
#RunWith(AllTests.class)
public class DistributedIntegrationTestRunner {
private static Logger log = LoggerFactory.getLogger(DistributedIntegrationTestRunner.class);
public static TestSuite suite() {
TestSuite suite = new TestSuite();
ClassesFinder classesFinder = new ClasspathFinderFactory().create(true,
new String[]{".*IntegrationTest.*"},
new SuiteType[]{SuiteType.TEST_CLASSES},
new Class[]{Object.class},
new Class[]{},
"java.class.path");
int nodeNumber = systemPropertyInteger("node.number", "0");
int totalNodes = systemPropertyInteger("total.nodes", "1");
List<Class<?>> allTestsSorted = getAllTestsSorted(classesFinder);
allTestsSorted = filterIgnoredTests(allTestsSorted);
List<Class<?>> myTests = getMyTests(allTestsSorted, nodeNumber, totalNodes);
log.info("There are " + allTestsSorted.size() + " tests to choose from and I'm going to run " + myTests.size() + " of them.");
for (Class<?> myTest : myTests) {
log.info("I will run " + myTest.getName());
suite.addTest(new JUnit4TestAdapter(myTest));
}
return suite;
}
private static int systemPropertyInteger(String propertyKey, String defaultValue) {
String slaveNumberString = System.getProperty(propertyKey, defaultValue);
return Integer.parseInt(slaveNumberString);
}
private static List<Class<?>> filterIgnoredTests(List<Class<?>> allTestsSorted) {
ArrayList<Class<?>> filteredTests = new ArrayList<Class<?>>();
for (Class<?> aTest : allTestsSorted) {
if (aTest.getAnnotation(Ignore.class) == null) {
filteredTests.add(aTest);
}
}
return filteredTests;
}
/*
TODO: make this algorithm less naive. Sort each test by run duration as described here: http://blog.tradeshift.com/just-add-servers/
*/
private static List<Class<?>> getAllTestsSorted(ClassesFinder classesFinder) {
List<Class<?>> allTests = classesFinder.find();
Collections.sort(allTests, new Comparator<Class<?>>() {
#Override
public int compare(Class<?> o1, Class<?> o2) {
return o1.getSimpleName().compareTo(o2.getSimpleName());
}
});
return allTests;
}
private static List<Class<?>> getMyTests(List<Class<?>> allTests, int nodeNumber, int totalNodes) {
List<Class<?>> myTests = new ArrayList<Class<?>>();
for (int i = 0; i < allTests.size(); i++) {
Class<?> thisTest = allTests.get(i);
if (i % totalNodes == nodeNumber) {
myTests.add(thisTest);
}
}
return myTests;
}
}
The ClasspathFinderFactory is used to find all test classes that match the .*IntegrationTest pattern.
I make N jobs and they all run this Runner but they all use different values for the node.number system property, so each job runs a different set of tests. This is how the failsafe plugin looks:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/DistributedIntegrationTestRunner.java</include>
</includes>
<skipITs>${skipITs}</skipITs>
</configuration>
</plugin>
The ClasspathFinderFactory comes from
<dependency>
<groupId>cpsuite</groupId>
<artifactId>cpsuite</artifactId>
<version>1.2.5</version>
<scope>test</scope>
</dependency>
I think there should be some Jenkins plugin for this, but I haven't been able to find one. Something that's close is the Parallel Test Executor, but I don't think this does the same thing I need. It looks like it runs all the tests on a single job/server instead of multiple servers. It doesn't provide an obvious way to say, "run these tests here, and those tests there".
I believe you already found a solution by now, but I'll leave a path for the others who'll open this page asking the same question:
Parallel test executor plugin:
"This plugin adds a new builder that lets you easily execute tests defined in a separate job in parallel. This is achieved by having Jenkins look at the test execution time of the last run, split tests into multiple units of roughly equal size, then execute them in parallel."
https://wiki.jenkins-ci.org/display/JENKINS/Parallel+Test+Executor+Plugin
Yes, Parallel Test Executor is a cool plugin if you've got 2 slave or one slave with 8 executor because the this plugin based on "tests splitting" so e.g: you split your junit tests into 4 different array, these arrays will run on 4 different executor on that slave what you specified. I hope you got it :D, it depends on the number of executors on that slave where you want to run parallel testing or you should decrease split tests count to 2 from 4.

How can I provide custom logic in a Maven archetype?

I'm interested in creating a Maven archetype, and I think I have most of the basics down. However, one thing I'm stuck on is that sometimes I want to use custom logic to fill in a template. For example, if somebody generates my archetype and specifies the artifactId as hello-world, I'd like to generate a class named HelloWorld that simply prints out "Hello World!" to the console. If another person generates it with artifactId = howdy-there, the genned class would be HowdyThere and it would print out "Howdy There!".
I know that under the covers, Maven's archetype mechanism leverages the Velocity Template Engine, so I read this article on creating custom directives. This seemed to be what I was looking for, so I created a class called HyphenatedToCamelCaseDirective that extends org.apache.velocity.runtime.directive.Directive. In that class, my getName() implementation returns "hyphenatedCamelCase". In my archetype-metadata.xml file, I have the following...
<requiredProperties>
<requiredProperty key="userdirective">
<defaultValue>com.jlarge.HyphenatedToCamelCaseDirective</defaultValue>
</requiredProperty>
</requiredProperties>
My template class looks like this...
package ${package};
public class #hyphenatedToCamelCase('$artifactId') {
// userdirective = $userdirective
public static void main(String[] args) {
System.out.println("#hyphenatedToCamelCase('$artifactId')"));
}
}
After I install my archetype and then do an archetype:generate by specifying artifactId = howdy-there and groupId = f1.f2, the resulting class looks like this...
package f1.f2;
public class #hyphenatedToCamelCase('howdy-there') {
// userdirective = com.jlarge.HyphenatedToCamelCaseDirective
public static void main(String[] args) {
System.out.println("#hyphenatedToCamelCase('howdy-there')"));
}
}
The result shows that even though userdirective is being set the way I expected it to, It's not evaulating the #hyphenatedToCamelCase directives like I was hoping. In the directive class, I have the render method logging a message to System.out, but that message doesn't show up in the console, so that leads me to believe that the method never got executed during archetype:generate.
Am I missing something simple here, or is this approach just not the way to go?
The required properties section of the archetype-metatadata xml is used to pass additional properties to the velocity context, it is not meant for passing velocity engine configuration. So setting a property called userDirective will only make the variable $userDirective availble and not add a custom directive to the velocity engine.
If you see the source code, the velocity engine used by maven-archetype plugin does not depend on any external property source for its configuration. The code that generates the project relies on an autowired (by the plexus container) implementation of VelocityComponent.
This is the code where the velocity engine is initialized:
public void initialize()
throws InitializationException
{
engine = new VelocityEngine();
// avoid "unable to find resource 'VM_global_library.vm' in any resource loader."
engine.setProperty( "velocimacro.library", "" );
engine.setProperty( RuntimeConstants.RUNTIME_LOG_LOGSYSTEM, this );
if ( properties != null )
{
for ( Enumeration e = properties.propertyNames(); e.hasMoreElements(); )
{
String key = e.nextElement().toString();
String value = properties.getProperty( key );
engine.setProperty( key, value );
getLogger().debug( "Setting property: " + key + " => '" + value + "'." );
}
}
try
{
engine.init();
}
catch ( Exception e )
{
throw new InitializationException( "Cannot start the velocity engine: ", e );
}
}
There is a hacky way of adding your custom directive. The properties you see above are read from the components.xml file in the plexus-velocity-1.1.8.jar. So open this file and add your configuration property
<component-set>
<components>
<component>
<role>org.codehaus.plexus.velocity.VelocityComponent</role>
<role-hint>default</role-hint>
<implementation>org.codehaus.plexus.velocity.DefaultVelocityComponent</implementation>
<configuration>
<properties>
<property>
<name>resource.loader</name>
<value>classpath,site</value>
</property>
...
<property>
<name>userdirective</name>
<value>com.jlarge.HyphenatedToCamelCaseDirective</value>
</property>
</properties>
</configuration>
</component>
</components>
</component-set>
Next add your custom directive class file to this jar and run archetype:generate.
As you see this is very fraglie and you will need to figure a way to distribute this hacked plexus-velocity jar. Depending on what you are planning to use this archetype for it might be worth the effort.

how to get the super pom basedir in a child module pom?

I want to define a local repository in my maven project.
I've got a super pom and several child modules. My file structure is :
/root
/repository
/child
pom.xml
pom.xml
in my super pom I define :
<repository>
<id>my-local-repo</id>
<url>file://${basedir}/repository</url>
</repository>
The problem is that in my child pom, the repository defined in my super pom refers to /root/child/repository and so, dependencies cannot be found...
Is there a way to define a path always relative to the super pom ?
If not, what's the best way to solve the problem ?
In this case, at first you could try ${project.parent.basedir}.
As it seems it doesn't work, the simple(and native) way is use complete path (/root/...) or try relative path (../) instead of using ${basedir} variable.
But for me, a great solution would be externalize this configuration into a properties file.
You can use properties-maven-plugin ( http://mojo.codehaus.org/properties-maven-plugin/plugin-info.html ).
With this plugin, properties defined on the properties file can be read just like properties defined inside pom.xml.
From the plugin site:
If you have a properties file called teams.properties with this content:
toronto=raptors
miami=heat
Would be the same as declaring the following in your pom.xml:
<properties>
<toronto>raptors</toronto>
<miami>heat</miami>
</properties>
${project.parent.basedir} should do the job.
Or you can set the basedir-path of the root in a property, so it will be inherited. Something like this in the Parent
<properties>
<rootPath>${basedir}</rootPath>
</properties>
And in the Child
<repository>
<id>my-local-repo</id>
<url>file://${rootPath}/repository</url>
</repository>
I solved this many times with groovy plugin. Add a file called "basepath_marker" to your super pom's directory and add the following to your pom. You can access the property like this: ${base-path}. Read this blog post for more details.
Example:
...
<build>
<plugins>
<plugin>
<groupId>org.codehaus.gmaven</groupId>
<artifactId>groovy-maven-plugin</artifactId>
<executions>
<!-- set absolute base path from super pom -->
<execution>
<id>find-basepath</id>
<phase>validate</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
<source>
<![CDATA[
import java.io.File;
log.info('## define projects super pom absolute path through basepath_marker')
String p = "basepath_marker";
File f = null;
if( p != null ) {
def _max_child_poms = 0
while( _max_child_poms++ < 5 ) {
f = new File( p );
if( f.exists() ) {
break;
}
p = "../" + p;
}
}
if( f != null ) {
String basePath = f.getCanonicalPath();
basePath = basePath.substring( 0, basePath.lastIndexOf( File.separator ) );
project.properties['base-path'] = basePath.replace( '\\' , '/');
log.info(' - used base path = ' + project.properties['base-path'] );
} else {
log.error( 'Could not find basepath_marker marker file!' );
System.stop( 0 );
}
]]>
</source>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
...
I try ${basedir}/../ in child pom and it works.
${project.parent.basedir} can not be interpreted.
Also the solution as follow not work, seems ${basedir} is dynamic decided.
define a properties <rootPath> ${basedir} </rootPath> int your parent pom
use ${rootPath} in your child pom
In Parent pom -
Try to use relative path (../) instead of using ${basedir}

Resources