Spring Boot 2.3.1, JUnit 5, Maven 3.6.3 - Maven lifecycle "test" does not run test suite - spring-boot

pom.xml:
...
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-runner</artifactId>
<version>1.7.0-M1</version>
<scope>test</scope>
</dependency>
...
In <build> only is used:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
Not in use in <build> are the plugins maven-surefire-plugin and maven-compiler-plugin
There are a few test classes (MyTest1Test, MyTest2Test, etc.), testing REST controllers. Some test methods are tagged with "myTag", to be included in a test suite, like:
#SpringBootTest
#AutoConfigureMockMvc
public class MyTest1Test {
...
#Test
#WithMockUser
public void test1() {...}
#Test
#WithMockUser
#Tag("myTag")
public void test2() {...}
...
}
Also there is a test suite, defined like:
#RunWith(JUnitPlatform.class)
#SelectClasses({MyTest1Test.class, MyTest3Test.class})
#IncludeTags({"myTag"})
public class MyTestSuiteTest {
...
}
When I select Maven > test, all test classes are run, only the test suite is not run. Why not? What kind of configuration has to be done?
When I run the test suite itself by right-click > Run 'MyTestSuiteTest' within the test class, it is run correctly.
Trying replacing #RunWith(JUnitPlatform.class with #ExtendWith(SpringExtension.class) doesn't solve it. Maven > "test" doesn't run the test suite, and worth, trying to run the test suite with IDE (right-click on the class > Run 'MyTestSuiteTest") shows and error:
org.junit.runners.model.InvalidTestClassError: Invalid test class '...MyTestSuiteTest':
1. No runnable methods
at org.junit.runners.ParentRunner.validate(ParentRunner.java:525)
at org.junit.runners.ParentRunner.<init>(ParentRunner.java:102)
at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:84)
at org.junit.runners.JUnit4.<init>(JUnit4.java:23)
at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:37)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at org.junit.internal.requests.ClassRequest.createRunner(ClassRequest.java:28)
at org.junit.internal.requests.MemoizingRequest.getRunner(MemoizingRequest.java:19)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:49)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
The IDE tries to run the test suite with JUnit 4, but why? Because of #SelectClasses, #IncludeTags? I have to include the dependency
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-runner</artifactId>
<version>1.7.0-M1</version>
<scope>test</scope>
</dependency>
to be able to use #SelectClasses and #IncludeTags, but also I excluded the vintage engine. Btw, do I have to remove the exclusion of vintage engine, if running a test suite with #RunWith?
Observation:
Including vintage engine:
Using #RunWith, Maven > "test", shows the following in the console output:
org.junit.vintage.engine.discovery.DefensiveAllDefaultPossibilitiesBuilder$DefensiveAnnotatedBuilder buildRunner
WARNING: Ignoring test class using JUnitPlatform runner: ....MyTestSuiteTest
Running the test suite directly with right-click > Run 'MyTestSuiteTest" via IDE shows an additional entry in the unit test view:
MyTestSuiteTest
JUnit Jupiter
MyTest1Test
test2
MyTest3Test
test1
JUnit Vintage
<no name>
Although there are just two annotated methods to run with the test suite (one from MyTest1Test class, and one from MyTest3Test class), it says 3 test were run successfully. JUnit Vintage didn't show up before when vintage engine was exluded.
Using #ExtendWith with vintage engine included:
Maven > "test" doesn't run the test suite, no warning or similar in the console output. "right-click > Run 'MyTestSuiteTest" gives the reported error.
Excluding vintage engine:
#RunWith behaves as described initially in the question.
#ExtendWith Maven > "test" doesn't run the test suite, no warning or similar in the console output. "right-click > Run 'MyTestSuiteTest" gives the reported error.

Related

Maven runs JUnit 5 tests but does not run TestFactory method

I have a number of tests in a project which I am able to run using Maven (mvn test), all of these are JUnit 5 using the org.junit.jupiter.api.Test annotation.
I have one other class which has a method using the org.junit.jupiter.api.TestFactory annotation. When I run the tests in IDEA all the tests run and the TestFactory is run as expected (it is used as a factory to generate a number of tests, each of which runs and then the results are produced).
When I run in Maven it says it is running the class but it doesn't run any methods within it and the test factory method code isn't run.
Here's some example code showing the import and how it is being used on the method.
import org.junit.jupiter.api.TestFactory;
...class definition here etc...
#TestFactory
public Collection<DynamicTest> runTests() throws Exception {
System.out.println("[Test] TestFactory test running");
... other code here...
}
Maven will say it is running the class but the printout above will not occur.
I've tried adding another empty test method which uses the #Test annotation but this also doesn't run.
If I have maven specifically run this test using -Dtest=xxx Maven will even print out that it is running the test but then show that no tests have been run:
Running tests.FactoryTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Update: changing the name of a method to start with 'test' means that Maven does run the method, and if I change that method to start with something else (e.g. 'run') maven does not run it. This is also a difference between IDEA and Maven it seems (IDEA will run whatever is marked as a #Test but Maven seems to require some sort of naming convention). But that said, altering the name of the #TestFactory method still doesn't get Maven to run it.
Is there a simple way to get maven to handle the depencies etc when running the tests but just use the same mechanism as IDEA or just use a more standard JUnit way to run the tests rather than applying its own interpretations?
Pom file as it relates to JUnit:
<properties>
<junit-platform.version>5.9.1</junit-platform.version>
</properties>
...
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M7</version>
<type>maven-plugin</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.9.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<version>5.9.1</version>
<scope>test</scope>
</dependency>
mvn -v output:
Apache Maven 3.8.5 (3599d3414f046de2324203b78ddcf9b5e4388aa0)
Maven home: /usr/local/Cellar/maven/3.8.5/libexec
Java version: 1.8.0_161, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "10.16", arch: "x86_64", family: "mac"
You have not configured Maven Surefire correctly.
Thus, I assume that your tests are "running" using POJO Tests.
See the official JUnit 5 - Maven sample application for a working example with JUnit Jupiter and Maven Surefire.

Spock test won't run with a new spring-boot project created by start.spring.io

Spock tests run fine in the IDE but will not execute in the maven build.
To recreate the issue
Go to https://start.spring.io/ and create a new Groovy / Maven project, keep all other defaults.
Add the spock dependency to the pom:
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>
<version>1.3-groovy-2.5</version>
<scope>test</scope>
</dependency>
Add a failing spock test named HelloSpec.groovy next to the existing test:
import spock.lang.Specification
class HelloSpec extends Specification {
def "length of Spock's and his friends' names"() {
expect:
name.size() == length
where:
name | length
"Spock" | 5
"Kirk" | 4
"Scotty" | 6
"Dave" | 911
}
}
add the maven surefire plugin to the build so it knows to pick up the spock test. I understand this is supposed to be automatic for files ending in Spec but I haven't found that to be so. I also understand the file doesn't end in .java, that it is in fact a .groovy file, just go with it.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/*Spec.java</include>
</includes>
</configuration>
</plugin>
Run mvn clean install from the command line. You will find you are on false greenbar and the spock test does not run.
Remove the exclusion of the junit-vintage-engine that Spring added to your spring-boot-starter-tests dependency
<!-- <exclusion>-->
<!-- <groupId>org.junit.vintage</groupId>-->
<!-- <artifactId>junit-vintage-engine</artifactId>-->
<!-- </exclusion>-->
Run mvn clean install from the command line. Spock tests are picked up and run and your build correctly fails with the failing test

Glue code is not loaded when running with cucumber-spring back-end from jar file

I have been trying to get spring-based cucumber tests to run using a combination of Junit(4.12), Cucumber-Java(4.1.1), Cucumber-Spring(4.1.1) and Cucumber-Junit(4.1.1).
I have no issues loading glue code when running the tests from inside the IDE (IntelliJ 2018.3.4) but it seems that for some reason when I try running from the a compiled jar file (which is a requirement in this case) cucumber doesn't find the step definitions.
I've already tried multiple glue code formats such as:
"classpath:com.a.b.c.stepdefs"
"com.a.b.c.stepdefs"
"classpath:com/a/b/c/stepdefs"
I've also tried providing relative paths from the runner class up to the step definitions class (nested just one level below)
"stepdefs"
Also gave a try running using both JUnit and the cucumber.cli.Main and attempted to use different style of step definitions (both cucumber expression - which the missing step snippets are pointing me to - and regex)
I am using the spring-boot-maven-plugin so I am aware that that generally changes the jar structure
All of the above variations fully work when running from the IDE, but not from the jar file
Main Class:
#SpringBootApplication(exclude = {DataSourceAutoConfiguration.class, HibernateJpaAutoConfiguration.class})
#ComponentScan(basePackages = {"com.a.b.test.core.data",
"com.a.b.c",
"com.a.b.c.stepdefs"}
)
public class CucumberApplication {
public static void main(String[] args) throws IOException, InterruptedException {
SpringApplication.run(CucumberApplication.class, args);
Result result = JUnitCore.runClasses(RunnerCentral.class);
System.exit(result.wasSuccessful() ? 0 : 1);
}
}
Runner Class:
package com.a.b.c;
#RunWith(Cucumber.class)
#CucumberOptions(features = "classpath:BOOT-INF/classes/features",
glue = "classpath:com/a/b/c/stepdefs",
plugin = "json:target/cucumber-html-reports/cucumber.json")
public class RunnerCentral {
}
POM config of spring-boot-maven-plugin:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>2.1.0.RELEASE</version>
<configuration>
<fork>true</fork>
<mainClass>${start-class}</mainClass>
<requiresUnpack>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-spring</artifactId>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
</dependency>
</requiresUnpack>
</configuration>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
I am expecting the behavior to be consistent between running from IDE and running from a packaged source although I may be missing something
Another thing I want to mention is that when swapping the backend with cucumber-picocontainer everything seems to work (spring is a requirement so a swap isn't possible)
This is the kind of issue that can have you launching your hot coffee at the nearest colleague.
Have you seen this post about using a custom ResourceLoader https://github.com/cucumber/cucumber-jvm/issues/1320
I think you'd have to copy and paste the Cucumber.java class, providing the resource loader to the runtime from the Application Context, and change your RunnerCentral class to RunWith the new class.
FWIW in my case, I placed the raw project in a docker container, that on startup ran ./mvnw test which is the Maven Wrapper supplied in Spring Boot projects. You can do ./mvnw test -s /path/to/maven/settings.xml if using a corporate repository, and if your container host can't access the corporate repository, run the image first on the Jenkins box (or wherever the image is being built) which will cause the dependency jars to be downloaded inside, then commit the docker image, and push that image out.
That way, the container can run the cucumber test phase using the local .m2 directory inside it, with the dependencies it needs already there.

Springboot test with coverage in Intellij

I have a spring boot 2 project with maven pom in Intellij
I thought maven uses different internal coverage tools compared to jacoco or something similar
If I click on - Run All Feature in Test with coverage
I get
Error: Could not find or load main class cucumber.api.cli.Main
If I add cucumber dependency (I dont have cucumber based tests or want it)
Exception in thread "main" cucumber.runtime.CucumberException:
No backends were found. Please make sure you have a
backend module on your CLASSPATH.
I just want to run simple spring boot rest based tests with coverage
What setup do I need?
Edit:
I had
Caused by: java.lang.ClassNotFoundException: org.jetbrains.plugins.cucumber.java.run.CucumberJvm3SMFormatter
I needed this in the pom
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-java</artifactId>
<version>1.2.2</version>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-junit</artifactId>
<version>1.2.2</version>
</dependency>
and needed cucumber for java plugin installed
Now everything runs, no errors
BUT no code coverage, it is blank
It's a known issue that JUnit is not suggested from contex menu of project root: https://youtrack.jetbrains.com/issue/IDEA-198762
It appears you have the Cucumber integration installed in IDEA. This adds the option to run all feature files in the root of your project. If you want to run JUnit tests you have to drill down to src/main/test/java and select "Run All Tests".

Run nested JUnit 5 tests with Maven Surefire

I'm trying to use JUnit 5 in my side project as a trial before migrating my main project. I'd like to use #Nested tests to make my test classes cleaner.
Everything is fine when I ran my test suite as a whole. However, as soon as I try running just a single test, #Nested ones are not executed.
mvn -Dtest=com.mycompany.test.MyTest surefire:test
Is there any way of getting it to run the selected class and all #Nested ones?
Using JUnit 5.1.0, JUnit platform 1.1.0
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>${org.junit.platform.version}</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${org.junit.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
Test class:
public class MyTest {
#Test
public void thisTestExecutes() { }
#Nested
public class NestedTests {
#Test
public void thisTestDoesnt() { }
}
}
To run all nested classes you just need to add an "*" at end of class name. Something like:
mvn -Dtest=com.mycompany.test.MyTest\* surefire:test
Facing with same issue, then realized my parent test class name and .java file name are different. I changed my test class name to my .java file name with right click > Refactor > Rename (for possible reference issue). Lastly run my test with following command:
mvn -Dtest=com.mycompany.test.MyTest*
By the way I'm using maven-surefire-plugin 2.22.2 version.
Whole problem is that nested tests are classes compiled the same way as the anonymous classes with a name containing $. The Surefire and Failsafe excludes these by the default pattern
**/*$*
If you use lambda then these exclusions become more and more important.
This should work as well:
mvn test -Dexcludes=nonetest

Resources