My company is in the middle of revamping the java code that has been written till now. Earlier there were some test cases written, but they were not maintained. As a result they are failing.
I have inherited this project, but currently have to make do with skipping all the tests.
I want to start by picking some unit test cases within a module fix them and add it to the suite of test cases that are being run. Over a period of time, I want to cover all the tests in my project.
I have tried the following plugin configuration in my POM file, but it is not working. Is there something I am missing ?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<skipTests>${skipTests}</skipTests>
<excludes>
<exclude>**/*Test.java</exclude>
</excludes>
<includes>
<include>**/ParticularSubModuleTest.java</include>
</includes>
</configuration>
</plugin>
Related
We have a set of integration tests, which all end with IT. Out of those, there is some specific subset, which we would like to execute separately. Let's say their names end with SpecialIT. So what we want to achieve is two configurations of the failsafe plugin:
To execute all tests, ending with IT, but not with SpecialIT -> this is easy. Just normal inclusion and exclusion by those names.
To execute all tests, ending with SpecialIT, but not all others ...IT.
I thought it would be natural to create some dedicated profile and use a separate failsafe configuration with a negative lookbehind regex for that, so ended up with this configuration (had to use < instead of <, as that one is not allowed there):
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<excludes>
<exclude>%regex[.*(?<!Special)IT.*]</exclude>
</excludes>
<includes>
<include>**/*SpecialIT.*</include>
</includes>
</configuration>
</plugin>
But when I try to run this - I'm getting the following error:
Exclamation mark not expected in 'exclusion': %regex[.*(?<!Special)IT.*]
Reading the documentation of the failsafe plugin - I see this:
The syntax in parameter excludes and excludesFile should not use (!).
So the question is: is there any other way to achieve this, without, let's say, renaming all our integration tests into ...StandardIT and ...SpecialIT?
I was thinking in the direction of tags, test suit names or smth., but in our project we currently have a mix of JUnit5, JUnit4 and Spock (Groovy) tests, so it becomes not so straightforward.
P.S. If I just use this configuration - all IT tests are getting disabled and nothing is executed:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/*IT.*</exclude>
</excludes>
<includes>
<include>**/*SpecialIT.*</include>
</includes>
</configuration>
</plugin>
Thanks to #andrey-b-panfilov, I realized that my problem was that I somehow was under an impression (based on this) that all ...IT tests are always executed by default... But this is, of course, only until the <includes> is overwritten. So, the solution is eventually to just define the following two configurations:
To execute all but SpecialIT:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<includes>
<include>**/*IT.*</include>
</includes>
<excludes>
<exclude>**/*SpecialIT.*</exclude>
</excludes>
</configuration>
</plugin>
To execute only SpecialIT:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<includes>
<include>**/*SpecialIT.*</include>
</includes>
</configuration>
</plugin>
I am facing an issue with Maven which nowhere on internet I could find an answer for it. appreciate if anyone can help me with it. I aiming to test 2 testng files sent as parameter to POM as:
mvn clean test -DsuiteXmlFiles=1.xml,2.xml
and the POM file is:
<plugins>
<!-- Following plugin executes the testng tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<!-- Suite testng xml file to consider for test execution -->
<suiteXmlFiles>
<suiteXmlFile>${suiteXmlFile}</suiteXmlFile>
</suiteXmlFiles>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
<!-- Compiler plugin configures the java version to be usedfor compiling
the code -->
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
but, when i run, it ONLY runs the 2.xml file and runs it twice!!!
tried many options but no matter what is the second xml file it totally ignores the first and runs the second one twice.
can anyone please help?
thanks
Your surefire plugin should look like below
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.15</version>
<configuration>
<suiteXmlFiles>${file}</suiteXmlFiles>
<skipTests>false</skipTests>
</configuration>
</plugin>
Pay close attention to <suiteXmlFiles>${file}</suiteXmlFiles>
So now you can pass in multiple suite files via
mvn clean test -Dfile=src/test/resources/suite-one-with-execution.xml, src/test/resources/suite-two-with-execution.xml
For more details you can refer to my blog post here.
finally understood where the issue was, for some reason i still dont know why, the Suite name in xml files should be not same as each other!!
When I running unit tests, I want to skip all the tests in a package, but I need to make sure that tests in other packages work properly.
In short, there are three package now:
com.felix.a, com.felix.b and com.felix.c,
I don't want the test class in com.felix.c to be run.
I tried to filter the package through the maven-surefire- plugin, but it didn't take effect.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<excludes>
<exclude>${basedir}/src/test/java/com/felix/c/*</exclude>
</excludes>
</configuration>
</plugin>
Thanks!
Just remove the leading ${basedir}/src/test/java/ from your exclude, and it will work, as that is implied by the plugin already. From the documentation, it's also possible to use the fully qualified package name for the (arguably) more legible:
<configuration>
<excludes>
<exclude>com.felix.c.*</exclude>
</excludes>
</configuration>
My unit tests are incremental. That is, each test tests one particular function but there is dependency on the functioning of previous tests....perhaps not orthodox, but it is robust.
Within a test suite, tests are ordered by the order provided in the #Suite.SuiteClasses annotation.
Maven (surefire plugin), seems to order test suites in alphabetical order:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<includes>
<include>**/ModelTestSuite.java</include>
<include>**/ServiceTestSuite.java</include>
<include>**/ZFlowTestSuite.java</include>
</includes>
<additionalClasspathElements>
<additionalClasspathElement>${webinf.dir}</additionalClasspathElement>
</additionalClasspathElements>
<systemPropertyVariables>
<log4j.configuration>file:${l4j.test}/log4j.test.properties</log4j.configuration>
</systemPropertyVariables>
</configuration>
</plugin>
If I want FlowTestSuite to run after ServiceTestSuite, then I rename it ZFlowTestSuite.
This works but it feels like a hack. Is there an alternative approach?
Is it possible to tell Maven to execute every testng test in new JVM instance (fork) in serial mode, i.e. one by one.
The configuration below works for junit, but not works for testng
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.9</version>
<configuration>
<forkMode>always</forkMode>
</configuration>
Does anybody know how to set for testng?
#ben75, Thanks for your answer, But it doesn't work for me.
The key point is we use suiteXmlFiles to specify which case to run, and if we use suiteXmlFiles, the forkMode or (forkCount resuseForks) don't work.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<includes>
<include>**/Test*.java</include>
</includes>
<forkMode>always</forkMode>
</configuration>
</plugin>
This configuration works for me.
You can use this
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<forkCount>1</forkCount>
<reuseForks>false</reuseForks>
...
It's a very expensive configuration (i.e. it will take a long time to run your tests... so it's better to not use it, but you probably have some (good ?) reasons to use it).
This solution is the way to go since version 2.14, so I suggest you to upgrade your surefire-plugin version.
the reference is here