I am trying to re run my failed cucumber tests. I use maven and TestNG in my framework. How can I achieve this?
I have tried maven-surefire-plugin (version: 3.0.0-M3) in my POM with <rerunFailingTestsCount>2</rerunFailingTestsCount>. But this does not rerun the test automatically when a test fails.
POM snippet
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<!-- <suiteXmlFiles>testNG.xml</suiteXmlFiles> -->
<classpathScope>test</classpathScope>
<skipTests>false</skipTests>
<!-- <testFailureIgnore>true</testFailureIgnore> -->
<testSourceDirectory>/src/test/java/testRunners</testSourceDirectory>
<useTestNG>false</useTestNG>
<rerunFailingTestsCount>2</rerunFailingTestsCount>
</configuration>
</plugin>
Runner class
#RunWith(Cucumber.class)
#CucumberOptions(
features = "src/test/java/features",
glue= {"stepDefinitions"},
tags= {"#licensing01"},
dryRun = false,
plugin = {"com.cucumber.listener.ExtentCucumberFormatter:TestExecutionReports/TestResults.html"
+ ",rerun:TestExecutionReports/failed_scenarios.txt"
+ ",json:target/cucumber.json"},
monochrome = true
)
public class TestRunnerBAU extends AbstractTestNGCucumberTests{
}
I expect the failed tests to be rerun automatically. But currently the failed tests are not rerun.
As Per Maven Guideline
This feature is supported only for JUnit 4.x (Not TestNG)
Since of 2.21.0 the provider surefire-junit47 can rerun scenarios created by cucumber-jvm 2.0.0 and higher.
Source : https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html
Related
Spock tests run fine in the IDE but will not execute in the maven build.
To recreate the issue
Go to https://start.spring.io/ and create a new Groovy / Maven project, keep all other defaults.
Add the spock dependency to the pom:
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>
<version>1.3-groovy-2.5</version>
<scope>test</scope>
</dependency>
Add a failing spock test named HelloSpec.groovy next to the existing test:
import spock.lang.Specification
class HelloSpec extends Specification {
def "length of Spock's and his friends' names"() {
expect:
name.size() == length
where:
name | length
"Spock" | 5
"Kirk" | 4
"Scotty" | 6
"Dave" | 911
}
}
add the maven surefire plugin to the build so it knows to pick up the spock test. I understand this is supposed to be automatic for files ending in Spec but I haven't found that to be so. I also understand the file doesn't end in .java, that it is in fact a .groovy file, just go with it.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/*Spec.java</include>
</includes>
</configuration>
</plugin>
Run mvn clean install from the command line. You will find you are on false greenbar and the spock test does not run.
Remove the exclusion of the junit-vintage-engine that Spring added to your spring-boot-starter-tests dependency
<!-- <exclusion>-->
<!-- <groupId>org.junit.vintage</groupId>-->
<!-- <artifactId>junit-vintage-engine</artifactId>-->
<!-- </exclusion>-->
Run mvn clean install from the command line. Spock tests are picked up and run and your build correctly fails with the failing test
I am trying to run test cases of spring boot application.
For that I tried using mvn test -Dspring.profiles.active=test.
But it did not work.
Do I need to add profile in pom.xml
Could you please help.
The java arguments come before the maven lifecycle parameter:
mvn -Dspring.profiles.active=test test
You can also add <argLine> property to maven surefire plugin to apply to all tests.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables>
<testEnvironment>true</testEnvironment>
</systemPropertyVariables>
<argLine>-Dspring.profiles.active=test</argLine>
</configuration>
</plugin>
The best way to run a test with a profile is using the ActiveProfiles annotation (assuming JUnit 5):
#SpringBootTest
#ActiveProfiles("test")
class MyApplicationTest {
#Test
void test() {
// implement actual test here
}
}
This will start your Spring Application with the test profile active.
I want to run selenium tests only if a special flag is provided e.g.
mvn -DrunUiTests=true test
I am using Junit5. I tried already:
An env varible. But the tests will be instantieted anyway.
An home made annotation
#Target({ ElementType.TYPE, ElementType.METHOD })
#Retention(RetentionPolicy.RUNTIME)
#ExtendWith(DisableUITests.class)
public #interface UITest { }
Where DisableUITests returns ConditionEvaluationResult.enabled if a env variable is set. But maven run on console ignores that annotationn.
Furthermore it tryied the surefire plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<excludes>
<exclude>**/*UiTest.java</exclude>
</excludes>
</configuration>
</plugin>
But then I am not able to start the tests my own :-)
I'm afraid the only condition that surefire plugin can consider is file/test name patterns. However you can use the approach described in Conditionally ignoring tests in JUnit 4 where you would test required system property as Assumption
Hi I am having a strange scenario.
I have a spring boot java project . I have lots of Junit tests as well.
when i run the tests from the terminal by issuing the command mvn clean test, 2 tests belong to a class fails
however if run the those tests belonging to that class from the eclipse Run as >> Junit tests it passes .
any idea why does this happens ? and how can i fix it ?
the following is my sure fire plugin configuration
<maven-surefire-plugin.version>2.18.1</maven-surefire-plugin.version>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<!-- Force alphabetical order to have a reproducible build -->
<runOrder>alphabetical</runOrder>
</configuration>
</plugin>
thank you
Background info:
Maven version: 3.2.5
Java version: 1.8
Cucumber reporting version: 3.8.0
cucumber-java, cucumber-spring, and cucumber-junit: all version 1.2.5
I have a maven java project. I have cucumber tests and the maven-cucumber-reporting plugin running and working for the project.
Here is how I am running the Cucumber tests:
package com.util.test;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
#CucumberOptions(strict = false, features = "src/test/resources/features", format = {"pretty", "json:src/test/report/cucumber.json"}, tags={"#test, #util"})
public class UtilCucumberTest {
}
And here is my pom with the maven-cucumber-reporting plugin:
<plugin>
<groupId>net.masterthought</groupId>
<artifactId>maven-cucumber-reporting</artifactId>
<version>${cucumber.reporting.version}</version>
<executions>
<execution>
<id>testing</id>
<phase>test</phase>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<projectName>${project.artifactId}</projectName>
<outputDirectory>${project.basedir}/src/test/report/cucumber-reports</outputDirectory>
<cucumberOutput>${project.basedir}/src/test/report/cucumber.json</cucumberOutput>
<skippedFails>true</skippedFails>
<enableFlashCharts>false</enableFlashCharts>
<buildNumber>${project.version}</buildNumber>
</configuration>
</execution>
</executions>
</plugin>
Each time I run a build the cucumber.json file is overwritten and I lose previous build info. I would like to keep a copy of each report (or one per build, as in --report.json). I currently do not see the Trends page either and would like to see this information.
Questions:
What do I need to do to see the Trends information in the pretty HTML report that is generated?
How can I get cucumber to generate a uniquely named .json report file each time I run a build (or one unique report per version build)? And will this allow me to see the trends section comparing previous reports?