I have cucumber tests with rest-Assured. I tried to use #Skip tags to exclude some tests. But Jenkins integration test stage in the pipeline shows as unstable(Amber).
Code that enables skip
#RunWith(CucumberWithSerenity.class)
#CucumberOptions(
plugin = {"pretty", "html:target/reports/cucumber-html-report",
"html:target/cucumber-reports/cucumber-pretty",
"json:target/cucumber.json"},
tags = {"not #skip"},
glue = {"myproject.stepdefinitions"},
features = "src/it/resources/features/")
public class MyCucumberTests {
}
Serenity reports correctly shows that particular test as skipped
How to make the test Jenkins pipeline stage green as no tests failed.
Related
I have been at this for a few days now, let's say I have a build.gradle as follows:
ext.createRunIntegrationTestsTask = { String taskName , String buildPlatform ->
return project.tasks.create(taskName, Test.class) {
dependsOn buildPlatform
[...]
}
}
def Platforms = ["A", "B"]
for (platform in Platforms) {
println "Creating task: runIntegrationTestsPlatform$platform"
createRunIntegrationTestsTask("runIntegrationTestsPlatform$platform", "buildPlatform$platform")
}
task runIntegrationTest(type: Test) {
dependsOn 'runIntegrationTestPlatformA'
dependsOn 'runIntegrationTestPlatformB'
}
I want to be able to run the following command with the tests filter:
gradlew :runIntegrationTest --tests "com.somestuff.methodTest"
For some reason when the above command runs, it does not pass the tests filter to runIntegrationTestPlatformA and runIntegrationTestPlatformB task. Both the runIntegrationTestPlatform... tasks run the whole suite of integration tests.
The current behavior I get is for runIntegrationTestPlatform... tasks to run the whole integration test suite for the specified platform and runIntegrationTest to run the integration test suite for all platforms.
But, when I pass the --tests filter to runIntegrationTest task, it does not get propagated to the chained tasks, i.e. runIntegrationTestPlatformA and runIntegrationTestPlatformB. Instead, it runs the whole integration test suite for both platforms. I want runIntegrationTestPlatformA and runIntegrationTestPlatformB to use the tests filter passed to runIntegrationTest to only run the specified integration tests.
Is there a way to get runIntegrationTestPlatformA and runIntegrationTestPlatformB to run from runIntegrationTest with the --tests filter.
Any help would be appreciated. Thank you.
Additional Sources:
Refactor duplicated code in gradle task "type: Copy"
I am using Jenkins Ver:2.176.3 for declarative multibranch pipeline.
I have junit test in my application code base and using maven surefire plugin to run unit test using maven command.
Multibranch job page shows 'Test Result' link and 'Test Result Trend' graph also.
I think this is being displayed/published here due to plugin 'test-results-analyzer'.
In declarative pipeline we have two stages as shown in code sample and we use maven commands.
Now my problem is that this test result count same unit test for each stage of pipeline so the count of unit tests are being double on this Test Result of Jenkins job page.
I tried skipping unit test in stage 'package-IT' using maven option -DskipTests and as per log it does skip unit testing but still see the duplicate test results.
if know please suggest
stages{
stage('compile-N-test') {
agent {label 'agent'}
steps {
script {
// mvn clean test
}
}
}
stage('packaging-IT') {
agent {label 'agent'}
steps {
script {
//mvn verify
}
}
}
Finally i got the solution: in pipeline we are using withMaven() which invoke test result and keep adding it so increasing count.
So I have a test suite written in JUnit, like below, basic mantra is that all the test cases which are listed in #Suite.SuiteClasses are going to be included in for running of the tests.
package bddDemo.TestSuite;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
#RunWith(Suite.class)
#Suite.SuiteClasses({
//serenityBDDDemo.features.calculator.MathsUsingAnAngularCalculator.class,
//serenityBDDDemo.features.API.automateRESTAPI.class
})
public class TestSuite {
}
This runs perfectly well in maven, I can do mvn clean verify and what not, but when I tried to convert this project to a Gradle project, suite is not run, instead all the tests are run, weather are not marked for inclusion for running in the suite(which is kinda embarrassing).
Can you suggest a way to mark few tests cases for running and make a test suite in a gradle?
Is there already a possibility to generate an HTML report when JUnit tests were started via Gradle? Any hint or comment is appreciated.
UPDATE
Gradle 4.6 provides built-in support for the JUnit Platform which allows you to run JUnit Jupiter tests using the standard Gradle test task which generates HTML reports out of the box.
Answer for Gradle versions prior to 4.6
The JUnit Platform Gradle Plugin generates JUnit 4 style XML test reports.
These XML files are output to build/test-results/junit-platform by default.
So, if your build server knows how to parse JUnit 4 style XML reports, you can just point it to the XML files in that directory and let the build server generate the HTML report for you.
However, if you are asking if Gradle can generate an HTML report for your tests run via the junitPlatformTest task, then the answer is "No, unfortunately not." The reason is that the standard Gradle test task only generates HTML reports based on its own proprietary "binary" report format. Since the junitPlatformTest task does not generate reports in Gradle's binary format, Gradle itself cannot generate HTML reports for JUnit Platform tests.
Having said that, however, there is in fact a work around: you can use Ant within your Gradle build. Ant has a task for aggregating JUnit 4 based XML reports and generating an HTML report from those aggregated reports. The output is not very modern, but it is at least human readable. The downside is that the default XSLT stylesheet does not display the test class names for tests run via the JUnit Platform.
In any case, you can configure Ant's JUnitReport task in Gradle as follows.
junitPlatform {
// configure as normal
}
configurations {
junitXmlToHtml
}
task generateHtmlTestReports << {
def reportsDir = new File(buildDir, 'test-reports')
reportsDir.mkdirs()
ant.taskdef(
name: 'junitReport',
classname: 'org.apache.tools.ant.taskdefs.optional.junit.XMLResultAggregator',
classpath: configurations.junitXmlToHtml.asPath
)
ant.junitReport(todir: "$buildDir/test-results/junit-platform", tofile: "aggregated-test-results.xml") {
fileset(dir: "$buildDir/test-results/junit-platform")
report(format: 'frames', todir: reportsDir)
}
}
afterEvaluate {
def junitPlatformTestTask = tasks.getByName('junitPlatformTest')
generateHtmlTestReports.dependsOn(junitPlatformTestTask)
check.dependsOn(generateHtmlTestReports)
}
dependencies {
// configure as normal ...
junitXmlToHtml 'org.apache.ant:ant-junit:1.9.7'
}
Then, executing gradle check will generate an HTML report in build/test-reports/index.html.
Regards,
Sam (Core JUnit 5 committer)
Adding below line to my java command created TEST-junit-jupiter.xml in my target/test-result folder. This xml file has all info about number of testcases run, number of tests passed/failed etc
--reports-dir target/test-result
Yes, you can using Jacoco plugin.
Here is an example:
apply plugin: 'war' or apply plugin: 'java'
apply plugin: "jacoco"
test {
reports.junitXml.destination="build/test-results"
jacoco {
destinationFile = file("build/jacoco/jacoco.exec")
append=true
}
}
jacocoTestReport {
reports {
xml.enabled false
csv.enabled false
html.destination "${buildDir}/jacocoHtml"
}
}
Regards.
I have a junit test suite that runs 4 test classes. When I run the test suite using gradle it creates 4 html reports, 1 for each test class in the suite. I'm new to gradle, is there a way to have gradle combine the results into a single html report?
Here is my test suite.
#RunWith(Suite.class)
#Suite.SuiteClasses([
TestClass1.class,
TestClass2.class,
TestClass3.class,
TestClass4.class,
])
class MyTestSuite {
}
In my gradle.build file I the following test method.
test {
include("com/geb/tests/MyTestSuite.class")
jvmArgs '-Dsomevariabe=someValue'
}
I run my test suite with the gradle command: gradle :web-tests:test
Then when the test suite has completed running there are 4 html files created for each class. Saved in web-tests\build\reports\tests\classes
testClass1.html
testClass2.html
testClass3.html
testClass4.html
Each testClass.html file lists each test cases pass or fail status.
I'd like to have a single html file containing a combined list of all the test cases pass or fail statuses. Is this possible?
This plugin can help you. Add this to your build.gradle:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'com.kncept.junit.reporter:junit-reporter:2.1.0'
}
}
plugins {
id 'com.kncept.junit.reporter' version '2.1.0'
}
apply plugin: 'com.kncept.junit.reporter'
junitHtmlReport {
// The maximum depth to traverse from the results dir.
// Any eligible reports will be included
maxDepth = 5
// Directory where to search (exact or relative to build path)
testResultsDir = '/path/to/root/of/project'
// Where to output
testReportsDir = 'reports/junit'
// Fail build when no XML files to process
failOnEmpty = true
}
Now build the report with:
./gradlew junitHtmlReport
Creates an HTML report of all JUnit XML files it finds to ./reports/junit. Note that this combines files like TEST-*.xml and not the html files.