Generate Jacoco report for integration tests - gradle

I've created a new test suite called integrationTest using the jvm-test-suite plugin.
Now I want to generate a jacoco report just for the integration tests.
I create a new gradle task like this:
tasks.create<JacocoReport>("jacocoIntegrationTestReport") {
group = "verification"
description = "Generates code coverage report for the integrationTest task."
executionData.setFrom(fileTree(buildDir).include("/jacoco/integrationTest.exec"))
reports {
xml.required.set(true)
html.required.set(true)
}
}
But the generated HTML/XML report is empty. I have run the integration tests before executing the task and the file integrationTest.exec exists.
Thanks

It seems the important part of the new JaCoCo report task configuration is to wire the execution data via the integrationTest task instead of the exec-file path. The official docs (see last example here) also imply that the source set must be wired as well.
Here is the full build script (Gradle 7.6) that produces a report with command:
./gradlew :app:integrationTest :app:jacocoIntegrationTestReport
// build.gradle.kts
plugins {
application
jacoco
}
repositories {
mavenCentral()
}
testing {
suites {
val integrationTest by creating(JvmTestSuite::class) {
dependencies {
implementation(project(":app"))
}
}
}
}
tasks.register<JacocoReport>("jacocoIntegrationTestReport") {
executionData(tasks.named("integrationTest").get())
sourceSets(sourceSets["integrationTest"])
reports {
xml.required.set(true)
html.required.set(true)
}
}

Related

How to duplicate/clone a gradle task?

I am using the new JVM Test Suite plugin to create a regression test suite (including a dedicated source set and dedicated dependencies).
testing {
suites {
regressionTest(JvmTestSuite) {
dependencies {
implementation project
implementation 'org.assertj:assertj-core:3.22.0'
}
targets {
all {
testTask.configure {
sourceCompatibility = '17'
targetCompatibility = '17'
useJUnitPlatform {
includeTags 'check'
}
}
}
}
}
}
}
This generates a 'regressionTest' task that runs the tests on my new source set. So far so good.
Now I need to be able to run that task with a different includeTags for JUnit, but I can't find a good way to do so.
I found https://github.com/gradle/gradle/issues/6172 and JUnit5 tag-specific gradle task, but those only work for the duplicating the standard test task. If I do so, it would not run with the correct dependencies and the correct source set.
What's a good way to create a new test task with specific JUnit5 tags when using the JVM Test Suite plugin ?

How to get Jacoco reports for the Karate test feature files using gradle

How to get Jacoco reports for the Karate test feature files using Gradle.
My project is a Gradle project and I am trying to integrate jacoco report feature in my project for the karate tests. The server is running in my local on 8080 port.
I am doing the following way to generate jacoco report and please let me know is my approach correct and also give me a solution to get the jacoco report for the gradle project.
1) First I am trying to generate jacoco execution data with the help of jacocoagent.jar as follows with a Gradle task:
java -javaagent:/pathtojacocojar/jacocoagent.jar=destfile=/pathtojocofile/jacoco.exec -jar my-app.jar
2) Next, I am running a Gradle task to generate the report
project.task ('jacocoAPIReport',type: org.gradle.testing.jacoco.tasks.JacocoReport) {
additionalSourceDirs = files(project.sourceSets.main.allSource.srcDirs)
sourceDirectories = files(project.sourceSets.main.allSource.srcDirs)
classDirectories = files(project.sourceSets.main.output)
executionData = fileTree(dir: project.projectDir, includes: ["**/*.exec", "**/*.ec"])
reports {
html.enabled = true
xml.enabled = true
csv.enabled = false
}
onlyIf = {
true
}
doFirst {
executionData = files(executionData.findAll {
it.exists()
})
}
}
project.task('apiTest', type: Test) {
description = 'Runs the api tests'
group = 'verification'
testClassesDirs = project.sourceSets.apiTest.output.classesDirs
classpath =
project.sourceSets.apiTest.runtimeClasspath
useJUnitPlatform()
outputs.upToDateWhen { false }
finalizedBy jacocoAPIReport
}
I don't see any of my application's classes in the jococo.exec file. I think, bcz of that I am always getting the coverage report as 0%.
The server is running in my local on 8080 port.
I don't think that is going to work. Depending on how your code is structured you need to instrument the code of the server.
I suggest trying to get a simple unit test of a Java method to work with Gradle. If that works, then use the same approach for the server-side code and it will work.

Gradle coverage report with jacoco for a custom test task

I've split my tests into unit & DB test suites using Junit5 tags.
I created a new test task of called dbTest to run only those test tagged with the Db tag.
Looks like this (kotlin-dsl):
val dbTest by creating(Test::class) {
useJUnitPlatform {
includeTags("db")
}
}
Now I'm trying to calculate the coverage of the DB tests but the jacoco report is always empty.
Jacoco creates a non-empty exec file in the build/jacoco dir called dbTest.exec.
But when running the jacoco test report the resulting report is empty.
I generate the DB report using a custom jacoco report task that looks like this:
val jacocoDbTestReport: JacocoReport by creating(JacocoReport::class) {
reports {
xml.isEnabled = true
html.isEnabled = true
}
executionData(file("$buildDir/jacoco/dbTest.exec"))
}

Grouping JUnit tests with Gradle

I created a new Groovy project of integration tests and want to be able to be able to group tests together for different components. For example, some tests are only valid for a docker image, while others are for testing a webapp running locally. I'd also like to get reports for the results of the tests and also be able to run the tests over and over if needed.
The first step to grouping tests in JUnit is to create an interface to name the group. Create this in a package your test classes can reach.
If your Test Class looks like this:
package com.example.test.api.get.cert
class ByIssuedBeforeTest {
//awesome tests
}
Then create an empty interface like this:
package com.example.test
interface DockerTest {}
Now change your Test Class to look like this:
package com.example.test.api.get.cert
import org.junit.experimental.categories.Category
#Category(DockerTest.class)
class ByIssuedBeforeTest {
//awesome tests
}
You can also annotate individual tests instead of the whole class.
Now in your build.gradle file:
apply plugin: 'java'
sourceSets {
main {
java {
srcDirs = ["src/main/java", "src/main/groovy"]
}
}
test {
java {
srcDirs = ["src/test/java", "src/test/groovy"]
}
}
}
test {
outputs.upToDateWhen { false }
reports {
junitXml.enabled=true
html.enabled=true
}
}
task nondockerTest(type: Test) {
outputs.upToDateWhen { false }
useJUnit {
excludeCategories 'com.example.test.DockerTest'
}
}
task dockerTest(type: Test) {
outputs.upToDateWhen { false }
useJUnit {
includeCategories 'com.example.test.DockerTest'
}
}
The java plugin for gradle gives you the test task. In the test task we add the outputs line in order to keep the test task from ever being labeled as "UP-TO-DATE", in essence forcing gradle to always run the task when we call it instead of caching the result.
The report block enables JUnit report files to be created. They will be placed in the build/reports/tests directory. If you ran the nonDockerTest it would be build/reports/tests/nondockerTest/index.html.
The useJUnit block tells Gradle to use JUnit to run the tests as well as which category of tests to run (if using includeCategories) or category to not run (if using excludeCategories).
gradle test to run all the test.
gradle nondockerTest to run the tests not labeled as with the DockerTest category.
gradle dockerTest to run just tests labeled with the DockerTest category.
Each of these tasks will create reports in the /build/reports/tests directory.

JUnit5 tag-specific gradle task

I use the following annotation to tag my integration tests:
#Target({ ElementType.TYPE, ElementType.METHOD })
#Retention(RetentionPolicy.RUNTIME)
#Tag("integration-test")
public #interface IntegrationTest {
}
This is the filter I use in build.gradle to exclude these tests from gradle build:
junitPlatform {
filters {
tags {
exclude 'integration-test'
}
}
}
So far, so good.
Now I would like to offer a Gradle task which specifically runs my integration tests – what's the recommended approach?
Based on https://github.com/gradle/gradle/issues/6172#issuecomment-409883128
Amended in 2020 to take lazy task configuration and Gradle 5 into account. See answer's history for older versions.
plugins {
id "java"
}
def test = tasks.named("test") {
useJUnitPlatform {
excludeTags "integration"
}
}
def integrationTest = tasks.register("integrationTest2", Test) {
useJUnitPlatform {
includeTags "integration"
}
shouldRunAfter test
}
tasks.named("check") {
dependsOn integrationTest
}
Running
gradlew test will run tests without integration
gradlew integrationTest will run only integration test
gradlew check will run test followed by integrationTest
gradlew integrationTest test will run test followed by integrationTest
note: order is swapped because of shouldRunAfter
History
Gradle 4.6+ supports JUnit 5 natively
JUnit 5 deprecated their plugin: https://github.com/junit-team/junit5/issues/1317
JUnit 5 deleted plugin: 'org.junit.platform.gradle.plugin'
JUnit 5 closed junit5#579 (same as OP's question) as won't-fix (due to decommissioning their plugin)
Gradle supports the above feature: https://github.com/gradle/gradle/issues/6172
Tip
Note: while the above works, IntelliJ IDEA has a hard time inferring stuff, so I suggest to use this more explicit version where everything is typed and code completion is fully supported:
... { Test task ->
task.useJUnitPlatform { org.gradle.api.tasks.testing.junitplatform.JUnitPlatformOptions options ->
options.includeTags 'integration'
}
}
build.gradle.kts
Root project Kotlin DSL drop-in for configuring integration tests in all modules in Gradle 5.6.4
allprojects {
plugins.withId("java") {
#Suppress("UnstableApiUsage")
this#allprojects.tasks {
val test = "test"(Test::class) {
useJUnitPlatform {
excludeTags("integration")
}
}
val integrationTest = register<Test>("integrationTest") {
useJUnitPlatform {
includeTags("integration")
}
shouldRunAfter(test)
}
"check" {
dependsOn(integrationTest)
}
}
}
}
I filed an issue: https://github.com/junit-team/junit5/issues/579 (as suggested by Sam Brannen).
Meanwhile, I am using a project property as a workaround:
junitPlatform {
filters {
tags {
exclude project.hasProperty('runIntegrationTests') ? '' : 'integration-test'
}
}
}
Consequently, integrations tests will be skipped with:
gradle test
but will be included with:
gradle test -PrunIntegrationTests
Gradle 6
I am not sure if it is because Gradle behavior has changed, but the highest voted answer did not work for me in Gradle. 6.8.3. I was seeing the integrationTests task run along with the main test task. This simplified version worked for me:
test {
useJUnitPlatform {
excludeTags "integration"
}
}
tasks.register("integrationTests", Test) {
useJUnitPlatform {
includeTags "integration"
}
mustRunAfter check
}
Commands:
./gradlew test or ./gradlew clean build - Runs tests without
'integration' tag.
./gradlew integrationTests - Only runs test with
'integration' tag.
According to me, the best, current working code to solve this, is the one presented by: TWiStErRob found here.
Note that tests must be tagged with ui in the example below (Junit5 tagging):
task uiTest(type: Test) {
useJUnitPlatform {
includeTags 'ui'
excludeTags 'integration'
}
}
Howe ever, I did not managed to get the Junit5 test-suit-thing to be run from gradle directly witch I would think would be an even nicer solution. But I think the solution buy TWiStErRob, is good enough. The down side is that the gradle.build file now also will be bloated with test-suit-things.
Please note that it is fine to create multiple test suites in the gradle file like this:
task firstTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-first-test-suite'
}
}
task secondTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-second-test-suite'
}
}
Then then all could be run separately like this:
gradlew firstTestSuite
gradlew secondTestSuite
gradlew ui
Solution run with Gradle 6.6.1
A similar approach to Rahel Lüthy avoiding the usage of empty strings, in this case to run all tests or just some tags:
test {
useJUnitPlatform() {
if (project.hasProperty("includes")) {
includeTags(project.property("includes") as String)
}
}
}

Resources