Stop Tests on first Failure with Maven/JUnit/Spring - spring

I'd like Maven to stop trying to run my JUnit Spring tests when it encounters the first error. Is this possible?
My test classes look like the following, and I run them just as a standard Maven target.
#ContextConfiguration(locations = {"classpath:/spring-config/store-persistence.xml","classpath:/spring-config/store-security.xml","classpath:/spring-config/store-service.xml", "classpath:/spring-config/store-servlet.xml" })
#RunWith(SpringJUnit4ClassRunner.class)
#Transactional
public class SkuLicenceServiceIntegrationTest
{
...
If there's an error in the Spring config, then each test will try to restart the Spring context, which takes 20 seconds a go. This means we don't find out for ages that any tests have failed, as it'll try to run the whole lot before concluding that the build was a failure!

Answering ten years later, since I needed exactly the same.
The failsafe plugin can be configure to "skip tests after failure", using the skipAfterFailureCount parameter.
Official documentation:
https://maven.apache.org/surefire/maven-failsafe-plugin/examples/skip-after-failure.html

This is more a remark, than an answer, but still, maybe you'll find it useful.
I'd recommend separating your integration tests into a separate phase, and running them with Failsafe, rather than Surefire. This way you can decide whether you need to run only fast unit tests, or the complete set with long-running integration tests:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<!-- Uncomment/comment this in order to fail the build if any integration test fail -->
<execution>
<id>verify</id>
<goals><goal>verify</goal></goals>
</execution>
</executions>
</plugin>
</plugins>
A workaround for your problem might be singling out a test into a separate execution and run it first; this way the execution would fail and subsequent surefire/failsafe executions will not be launched. See how to configure the plugin to do it.

Related

Tests run slower with Maven than with IDE

I'm facing some test time execution issue related to local test execution with Maven. I have around 250 tests which run about 15 min on the IDE test runner and about an hour (55min to be exact) when executing them with maven locally. I tried a lot of configurations to make test execution in parallel but neither of them work for me, the time is still the same...probably I'm doing something wrong. Can anyone help on this? The last maven surefire plugin configuration that I tried is the following:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<parallel>methods</parallel>
<threadCount>10</threadCount>
</configuration>
</plugin>
And for test execution command I've used:
mvn test
mvn surefire:test

why does my maven tests fails when run as a whole and passes when ran individually?

Hi I am having a strange scenario.
I have a spring boot java project . I have lots of Junit tests as well.
when i run the tests from the terminal by issuing the command mvn clean test, 2 tests belong to a class fails
however if run the those tests belonging to that class from the eclipse Run as >> Junit tests it passes .
any idea why does this happens ? and how can i fix it ?
the following is my sure fire plugin configuration
<maven-surefire-plugin.version>2.18.1</maven-surefire-plugin.version>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<!-- Force alphabetical order to have a reproducible build -->
<runOrder>alphabetical</runOrder>
</configuration>
</plugin>
thank you

How to invoke a custom method/cleanUp after all tests? (Geb and Spock)

To set up the environment before running tests I use the GebConfig.groovy class which is triggered as first component when runnig tests. But I also need to restore the environment to the initial state after all tests are finished.
I tried to overwtire the method cleanUpSpec() in class which extended GebReportingSpec class, but it is invoked after each test.
Is there a way to invoke a method after all tests are completed to clean up the environment to the initial state?
I am using maven in this project.
I don't know any mechanism in Geb or Spock. Generally my experience is that you will be in a better place when every test sets the stage before it starts instead of relying on some cleanup mechanism, which might not have been executed.
However, if you really need this, I'd suggest to use something like Maven Exec Plugin with execution phase post-(integration-)test:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<id>cleanup</id>
<phase>post-test</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>cleanup.groovy</executable>
</configuration>
</plugin>
You can use any executable file instead of cleanup.groovy but keep in mind that is has to be executable (chmod +x ...).

What is the recommended way to execute code at the end of the test phase in Maven?

I have some JUnit tests that execute in parallel in the test phase and output one .json file per test, and I want to call a custom Java method to do some aggregation and post-processing on those files after all of the tests have finished executing.
The integration-test phase is followed by the post-integration-test phase in the default Maven lifecycle, but the test phase is not followed by a post-test phase, and I would prefer not to abuse some other phase for this purpose.
Question: What is the recommended way to post-process the results at the end of the test phase?
As well described in another SO post, there is no post-test phase in Maven for good reasons (mainly, unit test is unit test).
However, in your case you don't need to create an additional Maven plugin, which would probably solve the issue but also add an additional layer of complexity in terms of maintenance, testing, sharing.
Since you already have the required code in a Java method - as mentioned in the question - it would probably make more sense to use the Exec Maven Plugin and its java goal.
You could hence simply add to your POM:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.1.1</version>
<executions>
<execution>
<phase>test</phase> <!-- executed as post-test, that is, after Surefire default execution -->
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass>com.sample.JsonFileAggregator</mainClass> <!-- your existing Java code, wrapped in a main -->
<arguments>
<argument>${project.build.directory}</argument> <!-- directory from where to load json files -->
</arguments>
<classpathScope>test</classpathScope> <!-- if your Java code is in test scope -->
</configuration>
</execution>
</executions>
</plugin>
That is, binding its execution to the test phase, Maven will executed it after any default binding (hence after the default Maven Surefire execution) and as such executed as a post-test.
Your existing Java code can then be invoked via a crafted (if not already existing) Java main, potentially passing to it arguments (like the directory from where to load the .json files, in the snippet above to the target folder, via its standard property ${project.build.directory}, as an example). Moreover, as mentioned in the snippet, your Java code may be located in test scope (that is, under src/test/java), hence to make it visible you would need to also configure the classpathScope accordingly.

Maven + jarsigner + test classes = error

I have a Maven project that includes some test cases. Today I tried adding the jarsigner plugin and now the tests fail with:
java.lang.SecurityException: class "types.AccountType"'s signer information does not match signer information of other classes in the same package
The test classes are in the same package to have access to package-private methods etc. I believe that this error is happening because the junit test classes are not being signed while the classes being tested are.
Does anyone have a suggestion on how to avoid this problem? I had some ideas but don't know how to implement them:
Cause the testing phase to use the classes instead of the jar file.
Put the test classes into their own jar file and sign it.
I ran into this issue today and the problem is as you guessed it many years ago (signing order issue). This was the fix for me (change the phase to install):
<plugin>
<artifactId>maven-jarsigner-plugin</artifactId>
<version>${maven-jarsigner-plugin.version}</version>
<executions>
<execution>
<id>sign</id>
<!-- note: this needs to be bound after integration tests or they will fail re: not signed -->
<phase>install</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
<configuration>
<tsa>http://sha256timestamp.ws.symantec.com/sha256/timestamp</tsa>
<keystore>${project.basedir}/.conf/Keystore</keystore>
<alias>Alias</alias>
<storepass>{1234}</storepass>
</configuration>
</plugin>
Jars are still signed and integration tests once again work.

Resources