I have defined a test Task that uses the TestNG to run testcases and am passing a list of testNG suites file dynamically for it to run. However it does not seem to pick the file list dynamically.
task runTestNG(type: Test) {
doFirst {
if(project(':testNG').hasProperty("testJerseyFiles")) {
ext.ngFiles = project.property("testJerseyFiles")
List testFiles = ngFiles as List
if(testFiles != null && testFiles.size() != 0 ) {
println "++++++++If : runTestNG++++++++++++++++++++++++++++"
testFiles.each { test ->
println "${test}"
}
}
} else {
println "++++++++Else : runTestNG++++++++++++++++++++++++++++"
}
}
useTestNG() {
if(project(':testNG').hasProperty("testJerseyFiles")) {
ext.ngFiles = project.property("testJerseyFiles")
List testFiles = ngFiles as List
if(testFiles != null && testFiles.size() != 0 ) {
listeners << 'org.testng.reporters.XMLReporter'
ignoreFailures = true
testFiles.each { test ->
println "******* ${test} ***************"
suites new File ("${test}")
//options.suites(new File ("${test}"))
}
}
} else {
// println "++++++++Else : runTestNG++++++++++++++++++++++++++++"
}
}
When I run this test from another task which is setting the property project(':testNG').ext.testJerseyFiles than I get the following
++++++++If : runTestNG++++++++++++++++++++++++++++
output/suiteDir/Jersey_setNo1.xml
Running test: test method init(test1.Test1)
However the setNo1.xml has a list of a completely different class. It does not seem to pick the file in the "useTestNG" suite option however it is clearly printing out the variable in the "doFirst" closure. Not sure what I am doing wrong.
The more research I do it seems that the suite parameter for the useTestNG is setup at configuration time and since the project.ext.property is being set at execution time it always find the ".hasProperty" to be empty and skips the if clause. Which causes it to go to the test list whatever it finds in the classpath.
Any way to make this suite pick up a list of suite dynamically?
Related
I am trying to configure Gradle to fail a test upon detecting a particular console output, but only if that test did not already fail.
For better context, I am trying to fail a test whenever the word "LEAK" is logged to the console, which Netty will automatically log.
My current solution looks like this:
ext {
testsWithLeak = [:]
}
test {
useJUnitPlatform()
onOutput { test, output ->
if (output.message.contains("LEAK")) {
testsWithLeak[test] = output.message
}
}
afterTest { test, result ->
if (result.getResultType().equals(TestResult.ResultType.FAILURE)) {
testsWithLeak.remove(test);
}
}
finalizedBy 'checkLeaks'
}
task checkLeaks {
dependsOn 'test'
doLast {
testsWithLeak.each{entry ->
throw new GradleException("ERROR: $entry.key produced leak: $entry.value")
}
}
}
This mostly works, and will correctly fail a build upon detecting a "LEAK" message. The problem is that since the test itself is considered a SUCCESS, it appears to be cached and not run again of subsequent builds. I tried calling cleanTest as part of throwing a GradleException, but that did not help.
Note that I use the testsWithLeak variable because I only want to throw this exception if the test did not already fail. A test may fail due to other assertions in which case I do not want to check for leaks (as the GradleException may mask the underlying assertion failure).
Also note that throwing the GradleException as part of the afterTest closure does not appear to fail the build.
How can I:
Fail a build when a test logs this message
Only throw this exception when the test did not otherwise fail
Ensure the build will correctly fail on subsequent attempts (without relying on commandline options)
The test is considered up to date because the test inputs (the class files) and the test outputs (the xml report) have not changed since the previous run. You can do the following so that tests are never up to date
test {
outputs.upToDateWhen {false}
...
}
See
https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskInputs.html
https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskOutputs.html
Your testsWithLeak map will not persist between gradle invocations. I suggest you write to a file under $buildDir and add a TaskOutputs.upToDateWhen {...} predicate to check the file exists. Your checkLeaks tasks should also be driven by the file and not a project variable.
Eg:
test {
ext {
testsWithLeak = new java.util.Properties()
leakFile = file("$buildDir/testsWithLeak.props")
}
outputs.upToDateWhen {!leakFile.exists()}
onOutput {...}
afterTest {...}
doFirst {
delete leakFile
}
doLast {
if (testsWithLeak) {
leakFile.withWriter { testsWithLeak.store(it)}
}
}
}
After executing test task in Gradle build I want to always execute additional task which should have access to test result. Something like:
task('afterTest') {
doLast {
if (testSuccessful) {
//
} else {
//
}
}
}
There are two parts in your question: 1) make a custom task executed always after test task is executed, 2) make the test "result" available in this custom task.
1) First part is very simple, you just have to use the dedicated Task.finalizedBy method to create a "finalized by" dependency between test task and your custom task. (see Finalizer tasks)
2) Second part is a bit tricky, as there is no simple way provided by Gradle, as far as I know, to get the "result" (SUCCESS or FAILURE) of test task. But you could use API exposed by the Test Task to store the number of test in error into a variable and test this counter in your custom task : here is a working example:
ext{
// counter of test cases in error
nbTestOnError = 0
}
test {
// use "afterTest" hook to increment nbTestOnError counter
afterTest { desc , result ->
if (result.getResultType().equals(TestResult.ResultType.FAILURE)){
nbTestOnError++
}
}
}
task('afterTest') {
doLast {
// implement your logic depending on counter value
if (nbTestOnError > 0) {
// do something if test failed
} else{
// do something when all tests passed
}
}
}
// create "finalized by" dependency
test.finalizedBy afterTest
EDIT : based on important remark in comment from #lance-java : in order to support up-to-date check , you could configure your custom task to be "skipped" if test task is not executed. A simple way would be to use Task upToDateWhen feature (see here) :
task('afterTest') {
// consider this task "UP TO DATE" if `test` task did not execute
outputs.upToDateWhen {
!test.didWork
}
doLast {
//...
}
}
As I said in the other thread, it's best to use the file system to pass values from one task to another. This way the value will be available even if the test task was up to date and skipped. Eg:
test {
outputs.file "$buildDir/test.properties"
ext.errorCount = 0
afterTest { desc , result ->
if (result.resultType.name() == "FAILURE") {
errorCount++
}
}
doLast {
file("$buildDir/test.properties").text = "errorCount=$errorCount"
}
finalizedBy 'afterTest'
ignoreFailures = true
}
task afterTest {
dependsOn 'test'
inputs.file "$buildDir/test.properties"
doLast {
def props = new Properties()
props.load(file("$buildDir/test.properties"))
def errorCount = Integer.parseInt(props['errorCount'])
if (errorCount) {
// doStuff
throw new TaskExecutionException("$errorCount tests failed")
}
}
}
Building on the existing answers, use org.gradle.api.tasks.testing.Test.afterSuite to save the test result to a file:
test {
afterSuite { descriptor, result ->
if (descriptor.parent == null) {
file("$buildDir/test.result").text = result.resultType
}
}
}
afterTest.onlyIf {
file("$buildDir/test.result").text == 'SUCCESS'
}
where the afterTest task is executed only when the test suite succeeds.
I am using TestNG and Gradle, what I am trying to achieve is, if the task that runs the tests fails, run another task which is also of type test which actually sends the test report. If all tests pass, don't do anything, I know about finalizedBy, but that sends the test report either way.
I tried something like this, but no luck.
task uiTest(type: Test) {
useTestNG() {
environment "DOMAIN", "${DOMAIN}"
useDefaultListeners = true
suites "src/test/resources/ui-tests/ThemeA/chrome67.xml"
}
reports {
html {
enabled true
}
reports.html.destination = "build/reports/TestReports/uiTestThemeA"
}
testLogging {
showStandardStreams = true
exceptionFormat "full"
events = ["started", "passed", "skipped", "failed"] //, "standard_out", "standard_error"]
showExceptions = true
showStackTraces = true
}
}
task testing(dependsOn: uiTest, type: Test) {
boolean state = uiTestThemeA.state.executed
if(!state) {
println name
useTestNG() {
suites "src/test/resources/ui-tests/sendReport.xml"
}
}
}
If I understand you correctly, then the following should do what you need. Replace your testing task with the following one and configure your uiTest task to be finalized by the testing task:
task testing(type: Test) {
onlyIf { uiTest.state.failure }
useTestNG() {
suites "src/test/resources/ui-tests/sendReport.xml"
}
}
uiTest {
finalizedBy testing
}
Some notes:
the onlyIf statement does the magic you are looking for
this assumes you execute uiTest
Our build has some generic logic to create tasks from configuration data. There are also several Gradle files on whose execution order I do not want to depend.
Now I need to add a task dependency to a task without knowing if that task will be defined before my part of the build script runs (at configuration time) or after my part.
[Edit: as #canillas writes I can say myTask.dependsOn "otherTask" and "lazily" depend on yet undefined "otherTask", but I need it the other way around.]
Since I can't write tasks["otherTask"].dependsOn myNewTask before "otherTask" is defined, I made the following helper function:
void safelyDoWithTask(String taskName, Closure func) {
def task = tasks.findByName(taskName)
if (task != null) {
func(task)
} else {
project.tasks.whenTaskAdded { task_ ->
if (taskName.equals(task_.getName())) {
func(task_)
}
}
}
}
safelyDoWithTask('otherTask', { it.dependsOn myNewTask })
Now I wonder if there is more canonical way to achieve this?
Consider the following:
// section 1
def dynamicDependencies = [:]
dynamicDependencies['otherTask'] = 'myNewTask'
// section 2
afterEvaluate { project ->
// taskA dependsOn taskB
dynamicDependencies.each { taskA, taskB ->
def task = project.tasks.findByName(taskA)
if (task) { task.dependsOn "${taskB}" }
}
}
// section 3
task myNewTask() {
doLast {
println 'myNewTask !'
}
}
task otherTask() {
doLast {
println 'otherTask !'
}
}
The gist is:
section 1 defines our dependency info in a custom map
section 2 uses the afterEvaluate hook to process the map
because the above is decoupled from the task definitions, we can simply put them in section 3 (or wherever)
Given the following task in gradle, the dev would start a new module by:
gradle initModule -PmoduleName=derp
task initModule(type: Copy) {
description "Initialize an empty module based on the template. Usage: gradle initModule -P moduleName=derp"
onlyIf { project.hasProperty("moduleName") }
from "$rootDir/modules/template"
into "$rootDir/modules/$moduleName"
}
I am unable to run gradle since moduleName is not defined during evaluation, although I was hoping that "onlyIf" would do so.
To solve this I assign it to a locally defined variable in a guard block:
def modName = ""
if (!project.hasProperty("moduleName")) {
logger.error("Invalid moduleName : It can't be")
} else {
modName = moduleName
}
and finally use the new variable to survive the configuration phase.
Is this the right way to do this? It just doesn't feel right.
Additionally if I was using a rule to make this a tad more elegant:
tasks.addRule("Pattern: initModule_<mod name>") { String taskName ->
if (taskName.startsWith("initModule_")) {
def params = taskName.split('_');
task(taskName) {
doLast {
project.ext.moduleName = params.tail().head()
tasks.initModule.execute()
}
}
}
}
The moduleName is not passed around (even if I change it to finalizedBy).
Any help is appreciated.
As you already figured out the property is not available during the configuration phase.
But can postpone the look-up to the execution phase by using
into "$rootDir/modules/" + project.property("moduleName")
instead of
into "$rootDir/modules/$moduleName"