Execute only some subset of integration tests with maven-failsafe-plugin - maven

We have a set of integration tests, which all end with IT. Out of those, there is some specific subset, which we would like to execute separately. Let's say their names end with SpecialIT. So what we want to achieve is two configurations of the failsafe plugin:
To execute all tests, ending with IT, but not with SpecialIT -> this is easy. Just normal inclusion and exclusion by those names.
To execute all tests, ending with SpecialIT, but not all others ...IT.
I thought it would be natural to create some dedicated profile and use a separate failsafe configuration with a negative lookbehind regex for that, so ended up with this configuration (had to use < instead of <, as that one is not allowed there):
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<excludes>
<exclude>%regex[.*(?<!Special)IT.*]</exclude>
</excludes>
<includes>
<include>**/*SpecialIT.*</include>
</includes>
</configuration>
</plugin>
But when I try to run this - I'm getting the following error:
Exclamation mark not expected in 'exclusion': %regex[.*(?<!Special)IT.*]
Reading the documentation of the failsafe plugin - I see this:
The syntax in parameter excludes and excludesFile should not use (!).
So the question is: is there any other way to achieve this, without, let's say, renaming all our integration tests into ...StandardIT and ...SpecialIT?
I was thinking in the direction of tags, test suit names or smth., but in our project we currently have a mix of JUnit5, JUnit4 and Spock (Groovy) tests, so it becomes not so straightforward.
P.S. If I just use this configuration - all IT tests are getting disabled and nothing is executed:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/*IT.*</exclude>
</excludes>
<includes>
<include>**/*SpecialIT.*</include>
</includes>
</configuration>
</plugin>

Thanks to #andrey-b-panfilov, I realized that my problem was that I somehow was under an impression (based on this) that all ...IT tests are always executed by default... But this is, of course, only until the <includes> is overwritten. So, the solution is eventually to just define the following two configurations:
To execute all but SpecialIT:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<includes>
<include>**/*IT.*</include>
</includes>
<excludes>
<exclude>**/*SpecialIT.*</exclude>
</excludes>
</configuration>
</plugin>
To execute only SpecialIT:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<includes>
<include>**/*SpecialIT.*</include>
</includes>
</configuration>
</plugin>

Related

Maven(Surefire) Exclude all Unit Test Cases but run selected

My company is in the middle of revamping the java code that has been written till now. Earlier there were some test cases written, but they were not maintained. As a result they are failing.
I have inherited this project, but currently have to make do with skipping all the tests.
I want to start by picking some unit test cases within a module fix them and add it to the suite of test cases that are being run. Over a period of time, I want to cover all the tests in my project.
I have tried the following plugin configuration in my POM file, but it is not working. Is there something I am missing ?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<skipTests>${skipTests}</skipTests>
<excludes>
<exclude>**/*Test.java</exclude>
</excludes>
<includes>
<include>**/ParticularSubModuleTest.java</include>
</includes>
</configuration>
</plugin>

For Maven, is there a way to skip all the tests in a package?

When I running unit tests, I want to skip all the tests in a package, but I need to make sure that tests in other packages work properly.
In short, there are three package now:
com.felix.a, com.felix.b and com.felix.c,
I don't want the test class in com.felix.c to be run.
I tried to filter the package through the maven-surefire- plugin, but it didn't take effect.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<excludes>
<exclude>${basedir}/src/test/java/com/felix/c/*</exclude>
</excludes>
</configuration>
</plugin>
Thanks!
Just remove the leading ${basedir}/src/test/java/ from your exclude, and it will work, as that is implied by the plugin already. From the documentation, it's also possible to use the fully qualified package name for the (arguably) more legible:
<configuration>
<excludes>
<exclude>com.felix.c.*</exclude>
</excludes>
</configuration>

TestSuite Execution Order in Maven

My unit tests are incremental. That is, each test tests one particular function but there is dependency on the functioning of previous tests....perhaps not orthodox, but it is robust.
Within a test suite, tests are ordered by the order provided in the #Suite.SuiteClasses annotation.
Maven (surefire plugin), seems to order test suites in alphabetical order:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<includes>
<include>**/ModelTestSuite.java</include>
<include>**/ServiceTestSuite.java</include>
<include>**/ZFlowTestSuite.java</include>
</includes>
<additionalClasspathElements>
<additionalClasspathElement>${webinf.dir}</additionalClasspathElement>
</additionalClasspathElements>
<systemPropertyVariables>
<log4j.configuration>file:${l4j.test}/log4j.test.properties</log4j.configuration>
</systemPropertyVariables>
</configuration>
</plugin>
If I want FlowTestSuite to run after ServiceTestSuite, then I rename it ZFlowTestSuite.
This works but it feels like a hack. Is there an alternative approach?

How to tell Maven to execute testng tests one by one each in new JVM instance?

Is it possible to tell Maven to execute every testng test in new JVM instance (fork) in serial mode, i.e. one by one.
The configuration below works for junit, but not works for testng
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.9</version>
<configuration>
<forkMode>always</forkMode>
</configuration>
Does anybody know how to set for testng?
#ben75, Thanks for your answer, But it doesn't work for me.
The key point is we use suiteXmlFiles to specify which case to run, and if we use suiteXmlFiles, the forkMode or (forkCount resuseForks) don't work.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<includes>
<include>**/Test*.java</include>
</includes>
<forkMode>always</forkMode>
</configuration>
</plugin>
This configuration works for me.
You can use this
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<forkCount>1</forkCount>
<reuseForks>false</reuseForks>
...
It's a very expensive configuration (i.e. it will take a long time to run your tests... so it's better to not use it, but you probably have some (good ?) reasons to use it).
This solution is the way to go since version 2.14, so I suggest you to upgrade your surefire-plugin version.
the reference is here

How to create different jar files within one build block in maven?

I have a maven project that compiles two different projects and then creates classes in this dir: ${project.build.directory}/classes
Where ${project.build.directory} points to the dir that pom.xml exists.
I’m using maven-jar-plugin with different “execution” blocks to make the jar files out of related directories/classes for each project. I’m very new to maven and have difficulty to define the right “include” and “exclude” directories.
This is the structure that my classes reside:
\target\classes\com
\target\classes\com\microsoft
\target\classes\com\google
\target\classes\org
The first jar file needs to be created out of these classes:
\target\classes\com\microsoft
\target\classes\org
And the second jar needs to be created out of these classes:
\target\classes\com\google
Following is the part of “build” block that has “execution” blocks to create these jars. The first jar is called: msn-prod and the other is called: google. As you see, I’ve tried all different combinations to create these jars and none worked - they exist in the following build block as the parts which are commented.
Can somebody please help me on this? Any help is greatly appreciated.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.3.2</version>
<executions>
<execution>
<id>msn-prod</id>
<goals><goal>jar</goal></goals>
<phase>generate-resources</phase>
<configuration>
<classifier>msn-prod</classifier>
<!-- <classesDirectory>${project.build.directory}/classes/com/microsoft</classesDirectory>
<includes>
<include>**/*.class</include>
</includes>
<classesDirectory>${project.build.directory}/classes/org</classesDirectory>
<includes>
<include>**/*.class</include>
</includes>-->
<classesDirectory>${project.build.directory}/classes</classesDirectory>
<!-- <includes>
<include>**/*.class</include>
</includes>-->
<!-- <excludes>
<exclude>**/com/google/*</exclude>
</excludes>-->
<!-- <excludes>
<exclude>**/google/*.class</exclude>
</excludes>-->
<includes>
<include>**/com/microsoft/*.class</include>
<include>**/org/*.class</include>
</includes>
<finalName>${msn.prod}-${msn.api.version}</finalName>
</configuration>
</execution>
<execution>
<id>google</id>
<goals><goal>jar</goal></goals>
<phase>generate-resources</phase>
<configuration>
<classifier>google</classifier>
<!-- <classesDirectory>${project.build.directory}/classes</classesDirectory>
<includes>
<include>**/com/google/*.class</include>
</includes>-->
<classesDirectory>${project.build.directory}/classes/com/google</classesDirectory>
<includes>
<include>**/*.class</include>
</includes>
<finalName>${google}-${google.api.version}</finalName>
</configuration>
</execution>
</executions>
</plugin>
You are violating the Maven best practice of one build artifact per module and therefore running into trouble. Just break it up into multiple projects and it will be easy.

Resources