I am using TestNG and Gradle, what I am trying to achieve is, if the task that runs the tests fails, run another task which is also of type test which actually sends the test report. If all tests pass, don't do anything, I know about finalizedBy, but that sends the test report either way.
I tried something like this, but no luck.
task uiTest(type: Test) {
useTestNG() {
environment "DOMAIN", "${DOMAIN}"
useDefaultListeners = true
suites "src/test/resources/ui-tests/ThemeA/chrome67.xml"
}
reports {
html {
enabled true
}
reports.html.destination = "build/reports/TestReports/uiTestThemeA"
}
testLogging {
showStandardStreams = true
exceptionFormat "full"
events = ["started", "passed", "skipped", "failed"] //, "standard_out", "standard_error"]
showExceptions = true
showStackTraces = true
}
}
task testing(dependsOn: uiTest, type: Test) {
boolean state = uiTestThemeA.state.executed
if(!state) {
println name
useTestNG() {
suites "src/test/resources/ui-tests/sendReport.xml"
}
}
}
If I understand you correctly, then the following should do what you need. Replace your testing task with the following one and configure your uiTest task to be finalized by the testing task:
task testing(type: Test) {
onlyIf { uiTest.state.failure }
useTestNG() {
suites "src/test/resources/ui-tests/sendReport.xml"
}
}
uiTest {
finalizedBy testing
}
Some notes:
the onlyIf statement does the magic you are looking for
this assumes you execute uiTest
Related
I have a Spring Project where I wrote some Unit and Integration Tests. Now I want to create custom tasks for running all unit tests, all integration tests and one task for running both. But how can I do this?
I would suggest separating integration tests into separate source set. By default you already have 2 source sets, one for production code and one for tests. To create a new source set (create new directory under src/integrationTest/java) and add a following configuration using Junit5:
test {
useJUnitPlatform()
}
sourceSets {
integrationTest {
java.srcDir file("src/integrationTest/java")
resources.srcDir file("src/integrationTest/resources")
compileClasspath += sourceSets.main.output + configurations.testRuntime
runtimeClasspath += output + compileClasspath
}
}
For separate task:
task integrationTest(type: Test) {
description = 'Runs the integration tests.'
group = 'verification'
testClassesDirs = sourceSets.integrationTest.output.classesDirs
classpath = sourceSets.integrationTest.runtimeClasspath
useJUnitPlatform()
reports {
html.enabled true
junitXml.enabled = true
}
}
Now you have 3 tasks available:
gradlew test
gradlew integrationTest
gradlew check - runs both as it depends on Test task which both extend
If you also are using jacoco and want to merge the test results then you can have following tasks:
task coverageMerge(type: JacocoMerge) {
destinationFile file("${rootProject.buildDir}/jacoco/test.exec")
executionData fileTree(project.rootDir.absolutePath).include("**/build/jacoco/*.exec")
}
// aggregate all coverage data at root level
if (tasks.findByName("test")) {
tasks.findByName("test").finalizedBy coverageMerge
}
if (tasks.findByName("integrationTest")) {
tasks.findByName("integrationTest").finalizedBy coverageMerge
}
task codeCoverageReport(type: JacocoReport) {
executionData fileTree(project.rootDir.absolutePath).include("**/build/jacoco/*.exec")
subprojects.each {
if (!it.name.contains('generated')) {
sourceSets it.sourceSets.main
}
}
reports {
xml.enabled true
html.enabled true
html.setDestination(new File("${buildDir}/reports/jacoco"))
csv.enabled false
}
}
To run coverage report just execute
gradlew codeCoverageReport
After executing test task in Gradle build I want to always execute additional task which should have access to test result. Something like:
task('afterTest') {
doLast {
if (testSuccessful) {
//
} else {
//
}
}
}
There are two parts in your question: 1) make a custom task executed always after test task is executed, 2) make the test "result" available in this custom task.
1) First part is very simple, you just have to use the dedicated Task.finalizedBy method to create a "finalized by" dependency between test task and your custom task. (see Finalizer tasks)
2) Second part is a bit tricky, as there is no simple way provided by Gradle, as far as I know, to get the "result" (SUCCESS or FAILURE) of test task. But you could use API exposed by the Test Task to store the number of test in error into a variable and test this counter in your custom task : here is a working example:
ext{
// counter of test cases in error
nbTestOnError = 0
}
test {
// use "afterTest" hook to increment nbTestOnError counter
afterTest { desc , result ->
if (result.getResultType().equals(TestResult.ResultType.FAILURE)){
nbTestOnError++
}
}
}
task('afterTest') {
doLast {
// implement your logic depending on counter value
if (nbTestOnError > 0) {
// do something if test failed
} else{
// do something when all tests passed
}
}
}
// create "finalized by" dependency
test.finalizedBy afterTest
EDIT : based on important remark in comment from #lance-java : in order to support up-to-date check , you could configure your custom task to be "skipped" if test task is not executed. A simple way would be to use Task upToDateWhen feature (see here) :
task('afterTest') {
// consider this task "UP TO DATE" if `test` task did not execute
outputs.upToDateWhen {
!test.didWork
}
doLast {
//...
}
}
As I said in the other thread, it's best to use the file system to pass values from one task to another. This way the value will be available even if the test task was up to date and skipped. Eg:
test {
outputs.file "$buildDir/test.properties"
ext.errorCount = 0
afterTest { desc , result ->
if (result.resultType.name() == "FAILURE") {
errorCount++
}
}
doLast {
file("$buildDir/test.properties").text = "errorCount=$errorCount"
}
finalizedBy 'afterTest'
ignoreFailures = true
}
task afterTest {
dependsOn 'test'
inputs.file "$buildDir/test.properties"
doLast {
def props = new Properties()
props.load(file("$buildDir/test.properties"))
def errorCount = Integer.parseInt(props['errorCount'])
if (errorCount) {
// doStuff
throw new TaskExecutionException("$errorCount tests failed")
}
}
}
Building on the existing answers, use org.gradle.api.tasks.testing.Test.afterSuite to save the test result to a file:
test {
afterSuite { descriptor, result ->
if (descriptor.parent == null) {
file("$buildDir/test.result").text = result.resultType
}
}
}
afterTest.onlyIf {
file("$buildDir/test.result").text == 'SUCCESS'
}
where the afterTest task is executed only when the test suite succeeds.
I am using Gradle with TestNG. I have this build.gradle:
useTestNG() {
useDefaultListeners = true
suites "src/test/resources/tests1.xml"
suites "src/test/resources/tests2.xml"
}
}
How can I run for example only tests1.xml from command line?
you can use project properties to add/change/... different suites. In you example you are probably running
gradlew test
which run both suites. If you modify test task in your build.gradle
def suite1 = project.hasProperty("suite1")
def suite2 = project.hasProperty("suite2")
test {
useTestNG() {
dependsOn cleanTest
useDefaultListeners = true
if(suite1) {
suites "src/test/resources/simpleSuite.xml"
}
if(suite2) {
suites "src/test/resources/advancedSuite.xml"
}
}
}
you can choose suite in this way
gradlew test -Psuite1
gradlew test -Psuite2
gradlew test -Psuite1 -Psuite2
you can specify variable let say suiteFile with default value and use it in testNG section. For example:
ext{
set suiteFile, default is 'testrun_config.xml'
if (!project.hasProperty('suiteFile')) {
suiteFile = 'testrun_config.xml'
}
}
test {
useTestNG() {
dependsOn cleanTest
useDefaultListeners = true
suites "src/test/resources/"+suiteFile
}
}
Refer qaf gradle build file
If you want to pass through command line
gradlew test -PsuiteFile=test.xml
In build.gradle update:
test {
useTestNG() {
if (project.hasProperty('suite1')) { suites './src/test/suite1.xml' }
if (project.hasProperty('suite2')) { suites './src/test/suite2.xml' }
}
}
Use command gradlew test -Psuite1 to run suite1 and similarly update for suite2 as well.
build.gradle
test {
jvmArgs "-DisParallel=true"
useTestNG(){
suites testngxml
}
}
when i am trying to run gradlew clean test -Dgroups='sanityTests' will not run tests in that specific group but will run all tests. Can you please provide some insight on this.
test {
jvmArgs "-DisParallel=true"
useTestNG(){
suites testngxml
includeGroups System.properties['groups']
}
}
We can pass by adding includeGroups inside test which is shown below:
test {
useTestNG() {
dependsOn cleanTest
useDefaultListeners = true
includeGroups groupName
}
}
I'm setting up some tests. I have the following entries in my build.gradle file:
integTest {
useTestNG() {
}
}
integTest2 (type: Test){
useTestNG() {
include 'Group2'
}
}
And in my test I have the following annotation:
#Test (groups={"Group2"})
public void testMethod() {
// Test code here
}
When I run the second task (integTest2) with gradle it only builds my directory and doesn't actually pick up any tests to run (integTest works fine and runs all the tests but I want to be able to run separate suites individually too).
Is there something obvious I have missed?
Figured out the answer. I was using our company's internal version of gradle which only recognized "integTest" as the main task. So as a workaround I had to do something like this:
def allTestGroups = ['all', 'Group1', 'Group2']
def testGroup = project.hasProperty("testGroup") ? project.testGroup : 'all'
integTest {
useTestNG() {
includeGroups "${testGroup}"
excludeGroups allTestGroups.findAll { it != "${testGroup}" }.collect { "'${it}'" }.join(',')
}
}
Just try to change include 'Group2' to includeGroups 'Group2', like this:
integTest2 (type: Test) {
useTestNG {
includeGroups 'Group2'
}
}