Handling multi project builds with Grade, Jacoco and Sonarqube - gradle

This is working so far (at least it looks like it works), but I don't feel that it is idiomatic or would stand the test of time even for a few months. Is there any way to do it better or more "by the book"?
We have a few multi project builds in Gradle where a project's test could touch another one's code, so it is important to see the coverage even if it wasn't in the same project, but it was in the same multiproject. I might be solving non existing problems, but in earlier SonarQube versions I had to "jacocoMerge" coverage results.
jacocoTestReport {
reports {
executionData (tasks.withType(Test).findAll { it.state.upToDate || it.state.executed })
xml.enabled true
}
}
if(!project.ext.has('jacocoXmlReportPathsForSonar')) {
project.ext.set('jacocoXmlReportPathsForSonar', [] as Set<File>)
}
task setJacocoXmlReportPaths {
dependsOn('jacocoTestReport')
doLast {
project.sonarqube {
properties {
property 'sonar.coverage.jacoco.xmlReportPaths', project.
ext.
jacocoXmlReportPathsForSonar.
findAll { d -> d.exists() }.
collect{ f -> f.path}.
join(',')
}
}
}
}
project.rootProject.tasks.getByName('sonarqube').dependsOn(setJacocoXmlReportPaths)
sonarqube {
properties {
property "sonar.java.coveragePlugin", "jacoco"
property "sonar.tests", []
property "sonar.junit.reportPaths", []
}
}
afterEvaluate { Project evaldProject ->
JacocoReport jacocoTestReportTask = (JacocoReport) evaldProject.tasks.getByName('jacocoTestReport')
evaldProject.ext.jacocoXmlReportPathsForSonar += jacocoTestReportTask.reports.xml.destination
Set<Project> dependentProjects = [] as Set<Project>
List<String> configsToCheck = [
'Runtime',
'RuntimeOnly',
'RuntimeClasspath',
'Compile',
'CompileClasspath',
'CompileOnly',
'Implementation'
]
evaldProject.tasks.withType(Test).findAll{ it.state.upToDate || it.state.executed }.each { Test task ->
logger.debug "JACOCO ${evaldProject.path} test task: ${task.path}"
sonarqube {
properties {
properties["sonar.junit.reportPaths"] += task.reports.junitXml.destination.path
properties["sonar.tests"] += task.testClassesDirs.findAll { d -> d.exists() }
}
}
configsToCheck.each { c ->
try {
Configuration cfg = evaldProject.configurations.getByName("${task.name}${c}")
logger.debug "JACOCO ${evaldProject.path} process config: ${cfg.name}"
def projectDependencies = cfg.getAllDependencies().withType(ProjectDependency.class)
projectDependencies.each { projectDependency ->
Project depProj = projectDependency.dependencyProject
dependentProjects.add(depProj)
}
} catch (UnknownConfigurationException uc) {
logger.debug("JACOCO ${evaldProject.path} unknown configuration: ${task.name}Runtime", uc)
}
}
}
dependentProjects.each { p ->
p.plugins.withType(JacocoPlugin) {
if (!p.ext.has('jacocoXmlReportPathsForSonar')) {
p.ext.set('jacocoXmlReportPathsForSonar', [] as Set<File>)
}
p.ext.jacocoXmlReportPathsForSonar += jacocoTestReportTask.reports.xml.destination
JacocoReport dependentJacocoTestReportTask = (JacocoReport) p.tasks.getByName('jacocoTestReport')
dependentJacocoTestReportTask.dependsOn(jacocoTestReportTask)
setJacocoXmlReportPaths.dependsOn(dependentJacocoTestReportTask)
}
}
}

Related

eachFile in Gradle Copy-task is not working

I have a Gradle rule like:
tasks.addRule("Pattern: updatelight<Path> (copies files to ../<Path>).") { String taskName ->
if (taskName.startsWith("updatelight")) {
task([ "type": Copy ], taskName) {
def projectGroups = (sub + root)
def testEnvPath = taskName - 'updatelight'
into ("../${testEnvPath}/")
logger.info("Copies user.xml")
projectGroups.each { project ->
if (!project.isEmpty()) {
from (project.output) {
into "cfg/${project.newPath}"
eachFile { file ->
println " ${file.sourcePath} -> ${file.path}"
println '----------------------------------------------'
}
}
}
}
[...]
My problem is, eachFile { ... } is not printing to console. Any hint? I am a Gradle-newbie and just try to add some logging to the existing task.
There was no problem with the code. Because of cache there were to actions taken therefore no logging.

Jacoco Kotlin Dsl Multimodule exclude

I am trying to exclude some files from the merged jacoco report. I am using:
(root gradle)
tasks.register<JacocoReport>("codeCoverageReport") {
subprojects {
val subProject = this
subProject.plugins.withType<JacocoPlugin>().configureEach {
subProject.tasks.matching { it.extensions.findByType<JacocoTaskExtension>() != null }.configureEach {
val testTask = this
sourceSets(subProject.sourceSets.main.get())
executionData(testTask)
}
subProject.tasks.matching { it.extensions.findByType<JacocoTaskExtension>() != null }.forEach {
rootProject.tasks["codeCoverageReport"].dependsOn(it)
}
}
}
reports {
xml.isEnabled = false
html.isEnabled = true
csv.isEnabled = false
}
}
And for the every module exclusion jacoco report (e.g. for common module):
tasks.withType<JacocoReport> {
classDirectories.setFrom(
sourceSets.main.get().output.asFileTree.matching {
exclude(JacocoExcludes.commonModule)
}
)
}
For each module this is working but when trying to interact with root gradle task either the gradle sync fails or it only add the files from the last module. Any help ?
Thanks
I had the same problem and used the following code :
tasks.jacocoTestReport {
// tests are required to run before generating the report
dependsOn(tasks.test)
// print the report url for easier access
doLast {
println("file://${project.rootDir}/build/reports/jacoco/test/html/index.html")
}
classDirectories.setFrom(
files(classDirectories.files.map {
fileTree(it) {
exclude("**/generated/**", "**/other-excluded/**")
}
})
)
}
as suggested here : https://github.com/gradle/kotlin-dsl-samples/issues/1176#issuecomment-610643709

Integration tests with Gradle Kotlin DSL

I'm using this blog post to configure integration tests for a Spring Boot project, but I'm pretty stuck on declaring the source sets. I also found this post on StackOverflow, but I think I'm a bit further already.
My project structure is
project
|_ src
|_ main
| |_ kotlin
| |_ resources
|_ testIntegration
| |_ kotlin
| |_ resources
|_ test
| |_ kotlin
| |_ resources
|_ build.gradle.kts
|_ ... other files
And build.gradle.kts
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
plugins {
idea
kotlin("jvm")
id("org.springframework.boot") version "2.0.5.RELEASE"
id("org.jetbrains.kotlin.plugin.spring") version "1.2.71"
}
fun DependencyHandlerScope.springBoot(module: String) = this.compile("org.springframework.boot:spring-boot-$module:2.0.5.RELEASE")
fun DependencyHandlerScope.springBootStarter(module: String) = this.springBoot("starter-$module")
dependencies {
springBoot("devtools")
springBootStarter("batch")
springBootStarter("... spring boot dependencies")
compile("... more dependencies")
testCompile("... more test dependencies")
}
val test by tasks.getting(Test::class) {
useJUnitPlatform { }
}
kotlin {
sourceSets {
val integrationTest by creating {
kotlin.srcDir("src/testIntegration/kotlin")
resources.srcDir("src/testIntegration/resources")
}
}
}
val integrationTestCompile by configurations.creating {
extendsFrom(configurations["testCompile"])
}
val integrationTestRuntime by configurations.creating {
extendsFrom(configurations["testRuntime"])
}
val testIntegration by tasks.creating(Test::class) {
group = "verification"
testClassesDirs = kotlin.sourceSets["integrationTest"].kotlin
}
idea {
module {
testSourceDirs.addAll(kotlin.sourceSets["integrationTest"].kotlin.srcDirs)
testSourceDirs.addAll(kotlin.sourceSets["integrationTest"].resources.srcDirs)
}
}
I think I'm pretty much in the right direction. At least it doesn't throw an exception any more :)
When I run the testIntegration task, I get the following output:
Testing started at 12:08 ...
12:08:49: Executing task 'testIntegration'...
> Task :project:compileKotlin UP-TO-DATE
> Task :project:compileJava NO-SOURCE
> Task :project:processResources UP-TO-DATE
> Task :project:classes UP-TO-DATE
> Task :project:compileTestKotlin UP-TO-DATE
> Task :project:compileTestJava NO-SOURCE
> Task :project:processTestResources UP-TO-DATE
> Task :project:testClasses UP-TO-DATE
> Task :project:testIntegration
BUILD SUCCESSFUL in 2s
5 actionable tasks: 1 executed, 4 up-to-date
12:08:51: Task execution finished 'testIntegration'.
Also, IntelliJ doesn't recognise the testIntegration directories as Kotlin packages.
I was finally able to figure it out thanks to some help on the Kotlin Slack channel. First of all I had to upgrade to Gradle version 4.10.2.
For more info have a look at these two pages from Gradle:
https://docs.gradle.org/release-nightly/userguide/organizing_gradle_projects.html#sec:separate_test_type_source_files
https://docs.gradle.org/release-nightly/userguide/organizing_gradle_projects.html#sec:separate_test_type_source_files
Then I just had to create the sourceSets for the integrationTests
sourceSets {
create("integrationTest") {
kotlin.srcDir("src/integrationTest/kotlin")
resources.srcDir("src/integrationTest/resources")
compileClasspath += sourceSets["main"].output + configurations["testRuntimeClasspath"]
runtimeClasspath += output + compileClasspath + sourceSets["test"].runtimeClasspath
}
}
This would work just fine for Java, but since I'm working with Kotlin I had to add an extra withConvention wrapper
sourceSets {
create("integrationTest") {
withConvention(KotlinSourceSet::class) {
kotlin.srcDir("src/integrationTest/kotlin")
resources.srcDir("src/integrationTest/resources")
compileClasspath += sourceSets["main"].output + configurations["testRuntimeClasspath"]
runtimeClasspath += output + compileClasspath + sourceSets["test"].runtimeClasspath
}
}
}
In the docs they only put runtimeClasspath += output + compileClasspath, but I added sourceSets["test"].runtimeClasspath so I can directly use the test dependencies instead of declaring new dependencies for the integrationTest task.
Once the sourceSets were created it was a matter of declaring a new task
task<Test>("integrationTest") {
description = "Runs the integration tests"
group = "verification"
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
classpath = sourceSets["integrationTest"].runtimeClasspath
mustRunAfter(tasks["test"])
}
After this the tests still didn't run, but that was because I'm using JUnit4. So I just had to add useJUnitPlatform() which makes this the final code
task<Test>("integrationTest") {
description = "Runs the integration tests"
group = "verification"
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
classpath = sourceSets["integrationTest"].runtimeClasspath
mustRunAfter(tasks["test"])
useJUnitPlatform()
}
I didnt like the use of withConvention and how the kotlin src dir was set. So after check out both gradle docs here and here, I came up with this:
sourceSets {
create("integrationTest") {
kotlin {
compileClasspath += main.get().output + configurations.testRuntimeClasspath
runtimeClasspath += output + compileClasspath
}
}
}
val integrationTest = task<Test>("integrationTest") {
description = "Runs the integration tests"
group = "verification"
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
classpath = sourceSets["integrationTest"].runtimeClasspath
mustRunAfter(tasks["test"])
}
tasks.check {
dependsOn(integrationTest)
}
I preferr the less verbose style when using kotlin { and the use of variable for the new integrationTestTask.
As of Gradle 5.2.1 see https://docs.gradle.org/current/userguide/java_testing.html#sec:configuring_java_integration_tests
sourceSets {
create("intTest") {
compileClasspath += sourceSets.main.get().output
runtimeClasspath += sourceSets.main.get().output
}
}
val intTestImplementation by configurations.getting {
extendsFrom(configurations.testImplementation.get())
}
configurations["intTestRuntimeOnly"].extendsFrom(configurations.runtimeOnly.get())
dependencies {
intTestImplementation("junit:junit:4.12")
}
val integrationTest = task<Test>("integrationTest") {
description = "Runs integration tests."
group = "verification"
testClassesDirs = sourceSets["intTest"].output.classesDirs
classpath = sourceSets["intTest"].runtimeClasspath
shouldRunAfter("test")
}
tasks.check { dependsOn(integrationTest) }
Here is git repo that you can refer to: enter link description here
import org.gradle.api.tasks.testing.logging.TestExceptionFormat
import org.gradle.api.tasks.testing.logging.TestLogEvent
plugins {
application
kotlin("jvm") version "1.3.72"
id("com.diffplug.gradle.spotless") version "3.24.2"
id("org.jmailen.kotlinter") version "1.26.0"
checkstyle
}
version = "1.0.2"
group = "org.sample"
application {
mainClass.set("org.sample.MainKt")
}
repositories {
mavenCentral()
jcenter()
}
tasks.checkstyleMain { group = "verification" }
tasks.checkstyleTest { group = "verification" }
spotless {
kotlin {
ktlint()
}
kotlinGradle {
target(fileTree(projectDir).apply {
include("*.gradle.kts")
} + fileTree("src").apply {
include("**/*.gradle.kts")
})
ktlint()
}
}
tasks.withType<Test> {
useJUnitPlatform()
testLogging {
lifecycle {
events = mutableSetOf(TestLogEvent.FAILED, TestLogEvent.PASSED, TestLogEvent.SKIPPED)
exceptionFormat = TestExceptionFormat.FULL
showExceptions = true
showCauses = true
showStackTraces = true
showStandardStreams = true
}
info.events = lifecycle.events
info.exceptionFormat = lifecycle.exceptionFormat
}
val failedTests = mutableListOf<TestDescriptor>()
val skippedTests = mutableListOf<TestDescriptor>()
addTestListener(object : TestListener {
override fun beforeSuite(suite: TestDescriptor) {}
override fun beforeTest(testDescriptor: TestDescriptor) {}
override fun afterTest(testDescriptor: TestDescriptor, result: TestResult) {
when (result.resultType) {
TestResult.ResultType.FAILURE -> failedTests.add(testDescriptor)
TestResult.ResultType.SKIPPED -> skippedTests.add(testDescriptor)
else -> Unit
}
}
override fun afterSuite(suite: TestDescriptor, result: TestResult) {
if (suite.parent == null) { // root suite
logger.lifecycle("----")
logger.lifecycle("Test result: ${result.resultType}")
logger.lifecycle(
"Test summary: ${result.testCount} tests, " +
"${result.successfulTestCount} succeeded, " +
"${result.failedTestCount} failed, " +
"${result.skippedTestCount} skipped")
failedTests.takeIf { it.isNotEmpty() }?.prefixedSummary("\tFailed Tests")
skippedTests.takeIf { it.isNotEmpty() }?.prefixedSummary("\tSkipped Tests:")
}
}
private infix fun List<TestDescriptor>.prefixedSummary(subject: String) {
logger.lifecycle(subject)
forEach { test -> logger.lifecycle("\t\t${test.displayName()}") }
}
private fun TestDescriptor.displayName() = parent?.let { "${it.name} - $name" } ?: "$name"
})
}
dependencies {
implementation(kotlin("stdlib"))
implementation("com.sparkjava:spark-core:2.5.4")
implementation("org.slf4j:slf4j-simple:1.7.30")
testImplementation("com.squareup.okhttp:okhttp:2.5.0")
testImplementation("io.kotest:kotest-runner-junit5-jvm:4.0.5")
testImplementation("io.kotest:kotest-assertions-core-jvm:4.0.5") // for kotest core jvm assertions
testImplementation("io.kotest:kotest-property-jvm:4.0.5")
}
sourceSets {
create("integTest") {
kotlin {
compileClasspath += main.get().output + configurations.testRuntimeClasspath
runtimeClasspath += output + compileClasspath
}
}
}
val integTest = task<Test>("integTest") {
description = "Runs the integTest tests"
group = "verification"
testClassesDirs = sourceSets["integTest"].output.classesDirs
classpath = sourceSets["integTest"].runtimeClasspath
mustRunAfter(tasks["test"])
}
tasks.check {
dependsOn(integTest)
}
sourceSets {
create("journeyTest") {
kotlin {
compileClasspath += main.get().output + configurations.testRuntimeClasspath
runtimeClasspath += output + compileClasspath
}
}
}
val journeyTest = task<Test>("journeyTest") {
description = "Runs the JourneyTest tests"
group = "verification"
testClassesDirs = sourceSets["journeyTest"].output.classesDirs
classpath = sourceSets["journeyTest"].runtimeClasspath
mustRunAfter(tasks["integTest"])
}
tasks.check {
dependsOn(journeyTest)
}
I hope this helps. :)
There is a dedicated Gradle feature called Declarative Test Suite that supports this case:
testing {
suites {
val test by getting(JvmTestSuite::class) {
useJUnitJupiter()
}
register("integrationTest", JvmTestSuite::class) {
dependencies {
implementation(project())
}
targets {
all {
testTask.configure {
shouldRunAfter(test)
}
}
}
}
}
}
More:
https://docs.gradle.org/current/userguide/java_testing.html#sec:configuring_java_integration_tests

How to re-run only failed JUnit test classes using Gradle?

Inspired by this neat TestNG task, and this SO question I thought I'd whip up something quick for re-running of only failed JUnit tests from Gradle.
But after searching around for awhile, I couldn't find anything analogous which was quite as convenient.
I came up with the following, which seems to work pretty well and adds a <testTaskName>Rerun task for each task of type Test in my project.
import static groovy.io.FileType.FILES
import java.nio.file.Files
import java.nio.file.Paths
// And add a task for each test task to rerun just the failing tests
subprojects {
afterEvaluate { subproject ->
// Need to store tasks in static temp collection, else new tasks will be picked up by live collection leading to StackOverflow
def testTasks = subproject.tasks.withType(Test)
testTasks.each { testTask ->
task "${testTask.name}Rerun"(type: Test) {
group = 'Verification'
description = "Re-run ONLY the failing tests from the previous run of ${testTask.name}."
// Depend on anything the existing test task depended on
dependsOn testTask.dependsOn
// Copy runtime setup from existing test task
testClassesDirs = testTask.testClassesDirs
classpath = testTask.classpath
// Check the output directory for failing tests
File textXMLDir = subproject.file(testTask.reports.junitXml.destination)
logger.info("Scanning: $textXMLDir for failed tests.")
// Find all failed classes
Set<String> allFailedClasses = [] as Set
if (textXMLDir.exists()) {
textXMLDir.eachFileRecurse(FILES) { f ->
// See: http://marxsoftware.blogspot.com/2015/02/determining-file-types-in-java.html
String fileType
try {
fileType = Files.probeContentType(f.toPath())
} catch (IOException e) {
logger.debug("Exception when probing content type of: $f.")
logger.debug(e)
// Couldn't determine this to be an XML file. That's fine, skip this one.
return
}
logger.debug("Filetype of: $f is $fileType.")
if (['text/xml', 'application/xml'].contains(fileType)) {
logger.debug("Found testsuite file: $f.")
def testSuite = new XmlSlurper().parse(f)
def failedTestCases = testSuite.testcase.findAll { testCase ->
testCase.children().find { it.name() == 'failure' }
}
if (!failedTestCases.isEmpty()) {
logger.info("Found failures in file: $f.")
failedTestCases.each { failedTestCase ->
def className = failedTestCase['#classname']
logger.info("Failure: $className")
allFailedClasses << className.toString()
}
}
}
}
}
if (!allFailedClasses.isEmpty()) {
// Re-run all tests in any class with any failures
allFailedClasses.each { c ->
def testPath = c.replaceAll('\\.', '/') + '.class'
include testPath
}
doFirst {
logger.warn('Re-running the following tests:')
allFailedClasses.each { c ->
logger.warn(c)
}
}
}
outputs.upToDateWhen { false } // Always attempt to re-run failing tests
// Only re-run if there were any failing tests, else just print warning
onlyIf {
def shouldRun = !allFailedClasses.isEmpty()
if (!shouldRun) {
logger.warn("No failed tests found for previous run of task: ${subproject.path}:${testTask.name}.")
}
return shouldRun
}
}
}
}
}
Is there any easier way to do this from Gradle? Is there any way to get JUnit to output a consolidated list of failures somehow so I don't have to slurp the XML reports?
I'm using JUnit 4.12 and Gradle 4.5.
Here is one way to do it. The full file will be listed at the end, and is available here.
Part one is to write a small file (called failures) for every failed test:
test {
// `failures` is defined elsewhere, see below
afterTest { desc, result ->
if ("FAILURE" == result.resultType as String) {
failures.withWriterAppend {
it.write("${desc.className},${desc.name}\n")
}
}
}
}
In part two, we use a test filter (doc here) to restrict the tests to any that are present in the failures file:
def failures = new File("${projectDir}/failures.log")
def failedTests = []
if (failures.exists()) {
failures.eachLine { line ->
def tokens = line.split(",")
failedTests << tokens[0]
}
}
failures.delete()
test {
filter {
failedTests.each {
includeTestsMatching "${it}"
}
}
// ...
}
The full file is:
apply plugin: 'java'
repositories {
jcenter()
}
dependencies {
testCompile('junit:junit:4.12')
}
def failures = new File("${projectDir}/failures.log")
def failedTests = []
if (failures.exists()) {
failures.eachLine { line ->
def tokens = line.split(",")
failedTests << tokens[0]
}
}
failures.delete()
test {
filter {
failedTests.each {
includeTestsMatching "${it}"
}
}
afterTest { desc, result ->
if ("FAILURE" == result.resultType as String) {
failures.withWriterAppend {
it.write("${desc.className},${desc.name}\n")
}
}
}
}
The Test Retry Gradle plugin is designed to do exactly this. It will rerun each failed test a certain number of times, with the option of failing the build if too many failures have occurred overall.
plugins {
id 'org.gradle.test-retry' version '1.2.1'
}
test {
retry {
maxRetries = 3
maxFailures = 20 // Optional attribute
}
}

How to create sourceset in custom plugin

I'm writing custom plugin which should create number of sourcesets depends on plugin extension.
How can I do it in apply method?
Here's my code snippets (both don't work), labels - list from extension:
1.
project.sourceSets {
labels.each { info ->
"${info.lower}Src" {
java.srcDirs = ['src'] + info.srcPostfix.collect { postfix -> "src_custom/${postfix}" }
}
}
main { java.srcDirs = ['src'] + labels.collect { info -> "src_custom/${info.lower}" } }
test { java.srcDirs = ['test'] + labels.collect { info -> "test_custom/${info.lower}" } }
}
2.
labels.each { info ->
SourceSet modelSrc = project.sourceSets.create("${info.lower}Src")
modelSrc.getJava().setSrcDirs(['src'] + info.srcPostfix.collect { postfix -> "src_custom/${postfix}" })
}
SourceSetContainer sourceSets = project.convention.getPlugin(JavaPluginConvention).sourceSets
SourceSet mainSourceSet = sourceSets.getByName(MAIN_SOURCE_SET_NAME)
mainSourceSet.getJava().setSrcDirs(['src'] + labels.collect { info -> "src_custom/${info.lower}" })
SourceSet testSourceSet = sourceSets.getByName(TEST_SOURCE_SET_NAME)
testSourceSet.getJava().setSrcDirs(['test'] + labels.collect { info -> "test_custom/${info.lower}" })
I know this is old, but here's what I ran into. I needed to create a plugin that would create a sourceSet. What I had before was this in my build.gradle
sourceSets {
myNewSet {
groovy.srcDir file("${project.myNewSet_src}")
resources.srcDir file("${project.myNewSet_resources}")
}
}
So when I created my plugin, in the apply method, I had this:
project.getSourceSets().create("myNewSet", {
groovy.srcDir new File("${project.getProjectDir()}/${project.myNewSet_src}")
resources.srcDir new File("${project.getProjectDir()}/${project.myNewSet_resources}")
});
Hope this helps someone.
Here is an example what does it look like in Kotlin Gradle DSL, configuring Spring Boot's BootRun task project-wide in multiproject build:
tasks {
withType<BootRun> {
val createGitRevisionReport: GitRevisionReportTask by this#tasks
dependsOn(createGitRevisionReport)
val sourceSets = project.convention.getPlugin(JavaPluginConvention::class.java).sourceSets
val additionalSourceSet = sourceSets.create("bootRunAdditionalSourceSet") {
resources.srcDir(createGitRevisionReport.gitRevisionReportDir)
}
sourceResources(additionalSourceSet)
}
}

Resources