We have a Azure Pipeline with Gradle Task which runs almost 1700 unit tests. There are some flaky tests (2-3)which is causing the build to fail or partially succeed. Is there a way that I can bypass these flaky tests and have the build run successfully? Thanks in Advance.
In build.gradle, add this:
test {
ignoreFailures = true
}
You can find the documentation here.
Better still, if you know exactly what are the flaky tests, you can exclude them as follows:
test {
// explicitly exclude tests
exclude 'org/boo/**'
}
And if there is no choice, or you are in a rush, you can skip all the tests:
gradlew build -x test
Related
I have a multi-project gradle build with four Kotlin Multiplatform modules, two of which have tests. When I run gradle check, if any of the tests from one of the modules fails, the tests for the other module do not get executed.
I'm using Gradle 7.3, Java 17 and kotlin.test. Tests for both projects are located in the commonTest source set. Also tried Gradle 7.1 and Java 11 with the same behavior.
Excerpt from settings.gradle.kts:
include(":ProjectA")
include(":ProjectB") // B has tests and depends on D, its tests are run
include(":ProjectC")
include(":ProjectD") // D has tests but are not run
Excerpt from ProjectB build.gradle.kts:
sourceSets {
val commonMain by getting {
dependencies {
api(compose.runtime)
api(compose.foundation)
api(compose.material)
implementation(project(":ProjectD"))
}
}
val commonTest by getting {
dependencies {
implementation(kotlin("test"))
}
}
From the output of gradle check I can see that :ProjectB:allTests gets executed and fails but :ProjectB:allTests never gets executed. This is an excerpt from the gradle output:
> Task :ProjectB:desktopTest
com.mylibrary.AppTest[desktop] > helloTestNg[desktop] FAILED
java.lang.AssertionError at AppTest.kt:8
2 tests completed, 1 failed
There were failing tests
> Task :ProjectB:allTests FAILED
FAILURE: Build failed with an exception.
If I do gradle -p ProjectD check tests for ProjectD are executed correctly.
Default Gradle behavior is to stop if any task fails, and Gradle considers a failing test as failing the check task. As a consequence, if any test fails in a certain project, the tests for the projects that have not yet been executed will not get executed.
The --continue can be useful in this case, it changes the default behavior and forces Gradle to continue executing all tasks even if some of them failed.
In this issue it is very well explained https://youtrack.jetbrains.com/issue/KT-49858p
I have jacoco set up for my project.
I can do:
gradle cleanTest test
followed by
gradle jacocoTestReport
and get code coverage
This means two steps. Is there any way, I can just pass a switch to gradle test and get it
https://docs.gradle.org/current/userguide/jacoco_plugin.html
If the Java plugin is also applied to your project, a new task named
jacocoTestReport is created that depends on the test task.
So just call gradle jacocoTestReport, the test will also be invoked first.
edit : if you really want to call the test task, just add this in your build.gradle :
test.finalizedBy jacocoTestReport
Gradle 4.6 added support for JUnit5.
This works for me as long as I don't have another sourceset for e.g. integration tests: I do not know how to enable useJUnitPlatform() in my integration tests.
What I was able to do is to have test task working with new JUnit5 support, but my testInt task was using JUnit5 console and running tests as it would run from command line. At the end I ditch JUnit5 support in gradle and rollback to using JUnit5 console for both tests.
How to enable Gradle 4.6 JUnit5 support on other tasks then test?
If your integration test task is also a Test task, you may configure all test tasks via:
tasks.withType(Test) {
useJUnitPlatform()
}
Or configure it explicitly:
task testInt(type: Test) {
useJUnitPlatform()
...
}
Is it possible to run a TestNG test suite that is embedded in a JAR file via a Gradle test task?
My project includes JARed bundles of TestNG tests that have an embedded testng.xml file defining which tests should be run in the JAR. Is it possible for Gradle to refer to this embedded XML when running the TestNG tests?
From the command line I use the xmlpathinjar option.
I dont think it can be done using the Gradle TestNG task. I couldn't find any such support in the TestNGOptions
Instead of using
test{
useTestNG()
}
you could try going through this post on SO How can I tell Gradle to use my testng.xml file for Test Classes and Ordering? and maybe employ the approach detailed here https://stackoverflow.com/a/28868416
But when you are using a custom Gradle task to run your TestNG tests, please make sure that you add a reference to the ExitCodeListener
Here's a sample
task ('myTask', type: JavaExec) {
main = 'org.testng.TestNG'
classpath = sourceSets.main.runtimeClasspath + sourceSets.test.runtimeClasspath
args = ["-xmlpathinjar", "suites/mysuite.xml", "-listener", "org.testng.TestNG\$ExitCodeListener"]
}
More details on why the ExitCodeListener needs to be referred, can be found here
I have a gradle project in scala to which I am trying to add sonarRunner. Things are working fine, but compilation and test is running twice on executing task sonarRunner as SonarRunner adds test as a dependency. However, scoverage always recompiles the code for instrumentation and then runs the tests.
Is there a way to avoid running the tests twice. I tried ignoring tests in sonarRunner task, but then test task won't run at all. I tried guarding ignore test in sonarRunner task by something like below mentioned code and that basically does not do anything.
task sonarRunner {
test {
gradle.taskGraph.whenReady { graph ->
if (graph.hasTask('sonarRunner')) {
enabled = false
}
}
}
dependsOn 'reportScoverage'
}
Any suggestion will be helpful.
You can remove the dependency to the test task by overriding the sonarRunner task's dependencies.
tasks.sonarRunner {
dependsOn = []
}