I am trying to configure Gradle to fail a test upon detecting a particular console output, but only if that test did not already fail.
For better context, I am trying to fail a test whenever the word "LEAK" is logged to the console, which Netty will automatically log.
My current solution looks like this:
ext {
testsWithLeak = [:]
}
test {
useJUnitPlatform()
onOutput { test, output ->
if (output.message.contains("LEAK")) {
testsWithLeak[test] = output.message
}
}
afterTest { test, result ->
if (result.getResultType().equals(TestResult.ResultType.FAILURE)) {
testsWithLeak.remove(test);
}
}
finalizedBy 'checkLeaks'
}
task checkLeaks {
dependsOn 'test'
doLast {
testsWithLeak.each{entry ->
throw new GradleException("ERROR: $entry.key produced leak: $entry.value")
}
}
}
This mostly works, and will correctly fail a build upon detecting a "LEAK" message. The problem is that since the test itself is considered a SUCCESS, it appears to be cached and not run again of subsequent builds. I tried calling cleanTest as part of throwing a GradleException, but that did not help.
Note that I use the testsWithLeak variable because I only want to throw this exception if the test did not already fail. A test may fail due to other assertions in which case I do not want to check for leaks (as the GradleException may mask the underlying assertion failure).
Also note that throwing the GradleException as part of the afterTest closure does not appear to fail the build.
How can I:
Fail a build when a test logs this message
Only throw this exception when the test did not otherwise fail
Ensure the build will correctly fail on subsequent attempts (without relying on commandline options)
The test is considered up to date because the test inputs (the class files) and the test outputs (the xml report) have not changed since the previous run. You can do the following so that tests are never up to date
test {
outputs.upToDateWhen {false}
...
}
See
https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskInputs.html
https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskOutputs.html
Your testsWithLeak map will not persist between gradle invocations. I suggest you write to a file under $buildDir and add a TaskOutputs.upToDateWhen {...} predicate to check the file exists. Your checkLeaks tasks should also be driven by the file and not a project variable.
Eg:
test {
ext {
testsWithLeak = new java.util.Properties()
leakFile = file("$buildDir/testsWithLeak.props")
}
outputs.upToDateWhen {!leakFile.exists()}
onOutput {...}
afterTest {...}
doFirst {
delete leakFile
}
doLast {
if (testsWithLeak) {
leakFile.withWriter { testsWithLeak.store(it)}
}
}
}
Related
I have the following Gradle task:
task deploy(type: Exec) {
doFirst {
println 'Performing deployment'
}
final Map properties = project.getProperties()
if (!properties['tag']) {
throw new StopExecutionException("Need to pass a tag parameter")
}
commandLine "command", tag
}
If I run another task I get an error because properties['tag'] is undefined, I guess because Gradle is executing everything in the task except commandLine. Is there a better way to do this or a way to stop Gradle from executing parts of the task even when I'm running another task?
Using Gradle 6.6.1
I use this pattern:
// Groovy DSL
tasks.register("exec1", Exec) {
def tag = project.findProperty("tag") ?: ""
commandLine "echo", tag
doFirst {
if (!tag) {
throw new GradleException("Need to pass a tag parameter")
}
}
}
It adds the tag property if it exists.
If it does not exist, it adds an empty string but checks for this before it actually runs.
It would be great if the Exec task accepted providers as arguments so you could just give it providers.gradleProperty("tag"), but unfortunately it doesn't.
After executing test task in Gradle build I want to always execute additional task which should have access to test result. Something like:
task('afterTest') {
doLast {
if (testSuccessful) {
//
} else {
//
}
}
}
There are two parts in your question: 1) make a custom task executed always after test task is executed, 2) make the test "result" available in this custom task.
1) First part is very simple, you just have to use the dedicated Task.finalizedBy method to create a "finalized by" dependency between test task and your custom task. (see Finalizer tasks)
2) Second part is a bit tricky, as there is no simple way provided by Gradle, as far as I know, to get the "result" (SUCCESS or FAILURE) of test task. But you could use API exposed by the Test Task to store the number of test in error into a variable and test this counter in your custom task : here is a working example:
ext{
// counter of test cases in error
nbTestOnError = 0
}
test {
// use "afterTest" hook to increment nbTestOnError counter
afterTest { desc , result ->
if (result.getResultType().equals(TestResult.ResultType.FAILURE)){
nbTestOnError++
}
}
}
task('afterTest') {
doLast {
// implement your logic depending on counter value
if (nbTestOnError > 0) {
// do something if test failed
} else{
// do something when all tests passed
}
}
}
// create "finalized by" dependency
test.finalizedBy afterTest
EDIT : based on important remark in comment from #lance-java : in order to support up-to-date check , you could configure your custom task to be "skipped" if test task is not executed. A simple way would be to use Task upToDateWhen feature (see here) :
task('afterTest') {
// consider this task "UP TO DATE" if `test` task did not execute
outputs.upToDateWhen {
!test.didWork
}
doLast {
//...
}
}
As I said in the other thread, it's best to use the file system to pass values from one task to another. This way the value will be available even if the test task was up to date and skipped. Eg:
test {
outputs.file "$buildDir/test.properties"
ext.errorCount = 0
afterTest { desc , result ->
if (result.resultType.name() == "FAILURE") {
errorCount++
}
}
doLast {
file("$buildDir/test.properties").text = "errorCount=$errorCount"
}
finalizedBy 'afterTest'
ignoreFailures = true
}
task afterTest {
dependsOn 'test'
inputs.file "$buildDir/test.properties"
doLast {
def props = new Properties()
props.load(file("$buildDir/test.properties"))
def errorCount = Integer.parseInt(props['errorCount'])
if (errorCount) {
// doStuff
throw new TaskExecutionException("$errorCount tests failed")
}
}
}
Building on the existing answers, use org.gradle.api.tasks.testing.Test.afterSuite to save the test result to a file:
test {
afterSuite { descriptor, result ->
if (descriptor.parent == null) {
file("$buildDir/test.result").text = result.resultType
}
}
}
afterTest.onlyIf {
file("$buildDir/test.result").text == 'SUCCESS'
}
where the afterTest task is executed only when the test suite succeeds.
I created a new Groovy project of integration tests and want to be able to be able to group tests together for different components. For example, some tests are only valid for a docker image, while others are for testing a webapp running locally. I'd also like to get reports for the results of the tests and also be able to run the tests over and over if needed.
The first step to grouping tests in JUnit is to create an interface to name the group. Create this in a package your test classes can reach.
If your Test Class looks like this:
package com.example.test.api.get.cert
class ByIssuedBeforeTest {
//awesome tests
}
Then create an empty interface like this:
package com.example.test
interface DockerTest {}
Now change your Test Class to look like this:
package com.example.test.api.get.cert
import org.junit.experimental.categories.Category
#Category(DockerTest.class)
class ByIssuedBeforeTest {
//awesome tests
}
You can also annotate individual tests instead of the whole class.
Now in your build.gradle file:
apply plugin: 'java'
sourceSets {
main {
java {
srcDirs = ["src/main/java", "src/main/groovy"]
}
}
test {
java {
srcDirs = ["src/test/java", "src/test/groovy"]
}
}
}
test {
outputs.upToDateWhen { false }
reports {
junitXml.enabled=true
html.enabled=true
}
}
task nondockerTest(type: Test) {
outputs.upToDateWhen { false }
useJUnit {
excludeCategories 'com.example.test.DockerTest'
}
}
task dockerTest(type: Test) {
outputs.upToDateWhen { false }
useJUnit {
includeCategories 'com.example.test.DockerTest'
}
}
The java plugin for gradle gives you the test task. In the test task we add the outputs line in order to keep the test task from ever being labeled as "UP-TO-DATE", in essence forcing gradle to always run the task when we call it instead of caching the result.
The report block enables JUnit report files to be created. They will be placed in the build/reports/tests directory. If you ran the nonDockerTest it would be build/reports/tests/nondockerTest/index.html.
The useJUnit block tells Gradle to use JUnit to run the tests as well as which category of tests to run (if using includeCategories) or category to not run (if using excludeCategories).
gradle test to run all the test.
gradle nondockerTest to run the tests not labeled as with the DockerTest category.
gradle dockerTest to run just tests labeled with the DockerTest category.
Each of these tasks will create reports in the /build/reports/tests directory.
I have folowing code in biuld.gradle:
task generateReport(type: Exec){
onlyIf{
project.getState().failure
}
doLast{
executable "generateReport.bat"
}
}
tasks.withType(Test)*.finalizedBy generateReport
I've tried before:
task generateReport(type: Exec){
executable "generateReport.bat"
}
tasks.withType(Test)*.finalizedBy generateReport
gradle.afterProject {project, projectState ->
if (projectState.failure) {
doLast{
generateReport
}
}
}
And others examples, but all was useless..
What I've done incorrectly?
You need to hook into the buildFinished lifecycle event, like this:
gradle.buildFinished { buildResult ->
if (buildResult.failure) {
println "Build Failed!"
} else {
println "Build Succeeded!"
}
}
First of all, you have to use a BuildListener as it was mentioned in other answers already. But one more note, you can't call some task the way you did it, with generateReport. So you have rather to use an exec right in the closure of the listener, as:
gradle.buildFinished { buildResult ->
if (buildResult.failure) {
exec {
executable "generateReport.bat"
}
}
}
Project.state is the evaluation state of the project, not some execution state. If you want to do something when the build failed, you should do gradle.addBuildListener() and in the BuildListener implement the method buildFinished() where you can check result.failure to see whether the build failed. If you want to perform some action for each failed test tast, you should instead use gradle.addListener() and give it an implementation of TestListener where you can act on failed tests or test suites. Alternatively you can also add a test listener only to specific test tasks with testTask.addTestListener(). Or to have it even nicer, you can do testTask.afterTest { ... } or testTask.afterSuite { ... }.
I'd like to hook into the compileJava target and spit out an additional custom message on failure. We've got a really common case setup case that a lot of folks overlook and it'd be useful, only on failure, to be able to do something like:
compileJava.onFailure {
println "Did you configure your wampfrankle to talk to the slackometer?"
}
My Google skills have not yet led to an answer.
The error is a dependency error and as Rene points out that needs to be detected after the build has executed, not after the project has been evaluated.
Here I've added a call to buildFinished with a closure that detects if a failure has happened and prints out an error message.
project.gradle.buildFinished { buildResult ->
if (buildResult.getFailure() != null) {
println "Did you configure your wampfrankle to talk to the slackometer?"
}
}
To test this I force a dependency resolution failure with this bogus dependency:
dependencies {
compile 'foo:bar:baz'
}
Use finalizedBy on a task that executes onlyIf the first task fails. For example:
tasks.compileJava.finalizedBy('compileJavaOnFailure')
task compileJavaOnFailure {
doLast {
println 'Did you configure your wampfrankle to talk to the slackometer?'
}
onlyIf {
tasks.compileJava.state.failure != null
}
}