My project structure is as follows: :app, :core. :app is the Android application project, and it depends on :core which has all the business logic.
I have Espresso tests for :app, and I am able to run and get coverage report thanks to all the questions and guides out there. But the coverage is only for code in :app.
How do I get coverage for all projects (:app and :core) resulting from my Espresso instrumentation tests? Is this even possible?
Any help is greatly appreciated.
Even if you did not mention Jacoco in your question, it is listed in the tags, so I assume you want to create coverage reports by using it. The Gradle Jacoco plugin (apply plugin: 'jacoco') allows you to create custom tasks of the type JacocoReport.
Defining such a task, you can specify source and class directories as well as the execution data:
task jacocoTestReport(type: JacocoReport) {
dependsOn // all your test tasks
reports {
xml.enabled = true
html.enabled = true
}
sourceDirectories = // files() or fileTree() to your source files
classDirectories = // files() or fileTree() to your class files
executionData = // files() or fileTree() including all your test results
}
For Espresso tests, the execution data should include all generated .ec files. You could define this task in the Gradle root project and link to both the files and the tasks of your subprojects.
There is an example for a custom Jacoco report task, even if it aims on creating an unified report for multiple test types, it can be transfered for this multi-project problem.
By default the JacocoReport task only reports coverage on sources within the project. You'll need to set the additionalSourceDirs property on the JacocoReport task.
app/build.gradle
apply plugin: 'java'
apply plugin: 'jacoco'
// tell gradle that :core must be evaluated before :app (so the sourceSets are configured)
evaluationDependsOn(':core')
jacocoTestReport {
def coreSourceSet = project(':core').sourceSets.main
additionalSourceDirs.from coreSourceSet.allJava
additionalClassDirs.from coreSourceSet.output
}
Related
I want to upload my Jacoco test report to Coveralls automatically after my Travis build finishes. It works for Java, but how to configure it for Kotlin?
Error message
I can generate a Jacoco test report locally and on Travis, but when Travis tries to submit to coveralls it fails with message
> Task :coveralls
No source file found on the project: "kotlin-template-project"
With coverage file: /home/travis/build/myname/myreponame/build/reports/jacoco/test/jacocoTestReport.xml
Google links me to the Gradle plugin implementation which shows where it throws this message, which tells me (I think) that the Jacoco report file is found but not the source files which coveralls apparently needs.
What I tried
Hence, I tried pointing the coveralls task to my source files, in all of these ways:
coveralls {
sourceDirs += allprojects.sourceSets.main.allSource.srcDirs.flatten()
sourceDirs += files(sourceSets.main.kotlin.srcDirs).files.absolutePath
project.extensions.coveralls.sourceDirs += project.sourceSets.main.kotlin.srcDirs
sourceDirs += ['src/main/kotlin']
jacocoReportPath = 'build/reports/jacoco/test/jacocoTestReport.xml'
sourceDirs += ['src/test/kotlin']
sourceDirs += ["${projectDir}/src/main/kotlin"]
}
I also tried adding sourceSets project.sourceSets.main to the jacocoTestReport task.
Project setup
My minimal build.gradle file:
plugins {
id 'org.jetbrains.kotlin.jvm' version '1.2.50'
id 'java' // Required by at least JUnit.
// Test coverage
id 'jacoco'
// Upload jacoco coverage reports to coveralls
id 'com.github.kt3k.coveralls' version '2.8.2'
}
dependencies {
compile 'org.jetbrains.kotlin:kotlin-stdlib-jdk8'
// JUnit 5
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.2.0'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.2.0'
testRuntime 'org.junit.platform:junit-platform-console:1.2.0'
// Kotlintest
testCompile 'io.kotlintest:kotlintest-core:3.1.6'
testCompile 'io.kotlintest:kotlintest-assertions:3.1.6'
testCompile 'io.kotlintest:kotlintest-runner-junit5:3.1.6'
// Spek
testCompile 'org.jetbrains.spek:spek-api:1.1.5'
testRuntime 'org.jetbrains.spek:spek-junit-platform-engine:1.1.5'
}
repositories {
mavenCentral()
mavenLocal()
jcenter()
}
test {
// Enable JUnit 5 (Gradle 4.6+).
useJUnitPlatform()
// Always run tests, even when nothing changed.
dependsOn 'cleanTest'
// Show test results.
testLogging {
events "passed", "skipped", "failed"
}
}
// Test coverage reporting
jacocoTestReport {
// Enable xml for coveralls.
reports {
html.enabled = true
xml.enabled = true
xml.setDestination(file("${buildDir}/reports/jacoco/test/jacocoTestReport.xml"))
}
}
coveralls {
sourceDirs += ['src/main/kotlin']
jacocoReportPath = 'build/reports/jacoco/test/jacocoTestReport.xml'
}
Related issues
Essentially the same issue is on https://github.com/kt3k/coveralls-gradle-plugin/issues/32 but the solution is to set sourceDirs and jacocoReportPath which I already have tried.
At https://github.com/kt3k/coveralls-gradle-plugin/issues/39 and https://github.com/kt3k/coveralls-gradle-plugin/issues/63 it is suggested to add sourceDirs += ['src/main/kotlin'] which sounds sensible but doesn't help. Same for, from the first link, sourceDirs = files(sourceSets.main.kotlin.srcDirs).files.absolutePath.
From https://github.com/kt3k/coveralls-gradle-plugin/issues/77 the solution is project.extensions.coveralls.sourceDirs += project.sourceSets.main.kotlin.srcDirs which I tried.
The question Kotlin code coverage in CI pipeline is phrased rather generally, but a comment points to discuss.kotlinlang.org where someone shows a way to improve the Jacoco result regarding kotlin, and the answer links to the Jacoco Gradle plugin which I use and works: when I run the jacocoTestReport task a report is generated in build/reports/jacoco/test/, both in xml and html.
The question Kotlin Test Coverage is also phrased general and answered with an unnecessarily complex build file from which I learned nothing new.
The question Measure test coverage for Kotlin code? claims that the Jacoco report does not work, but for me this is not the case as I said.
There are similar questions for Java, like Tool for java code coverage on GitHub but for me when I use Java it all works fine.
PS Actually I want to use the Gradle Kotlin DSL, but since nobody seems to use it I'm asking this question for Gradle. But in the end I want this question solved for the Kotlin DSL as well.
[edit August 2020] #nbaztec wrote a plugin to support Kotlin, please see his answer.
Old answer:
Kotlin is not supported by Coveralls, see for example this open isse that was mentioned in the question as well (in the question it was also mentioned that the workaround presented there does not work): https://github.com/kt3k/coveralls-gradle-plugin/issues/77
Solution: try Codecov.io instead.
Install it to GitHub using the Marketplace and add to your .travis.yml
after_success:
- bash <(curl -s https://codecov.io/bash)
Then commit and push, done!
You can view the result (after the build finished) at https://codecov.io/gh/githubaccountname/reponame
Had a similar experience with a variety of QA products not supporting or only partially supporting Kotlin codebases. Tried submitting support PRs to a couple of projects to no avail.
In the end ended up going with Coveralls and contributed a Kotlin focused plugin for the platform
https://github.com/nbaztec/coveralls-jacoco-gradle-plugin
Usage
Include the plugin in your build.gradle.kts (similar for build.gradle files):
plugins {
jacoco
id("com.github.nbaztec.coveralls-jacoco")
}
Then set the environment variable COVERALLS_REPO_TOKEN to the token from your Coveralls page.
Now you can use the coverallsJacoco task to publish a coverage report.
For more information and usage in CI, see
https://github.com/nbaztec/coveralls-jacoco-gradle-plugin
Not an answer, but in case anyone else is struggling with nbaztec like me, I want to give an alternative that worked for me: https://github.com/kt3k/coveralls-gradle-plugin
And besides what is in README.md, I needed this detail in build.gradle:
coveralls {
sourceDirs += ['src/main/kotlin']
jacocoReportPath "${buildDir}/reports/jacoco/report.xml"
}
Added JACOCO HTML report in my Gradle build for java project. But Report was not getting added as part of build phase, need to execute gradle task jacocoTestReport to get report.
Pseudo Gradle build file :
apply plugin : 'java'
apply plugin : 'jacoco'
repositories {
jcenter()
maven { maveno repo url }
}
jacocoTestReport {
reports {
xml.enabled false
csv.enabled false
html.enabled true
}
}
dependencies {
}
jacocoTestReport.mustRunAfter test
How to include Jacoco task in build phase of the project? So that no separate task needs to be executed.
The methods mustRunAfter and shouldRunAfter only affect the execution order of tasks that will be executed anyway (e.g. when passed via command line).
To cause a task to be executed when another task is executed, use either dependsOn or finalizedBy.
In your case, you could use finalizedBy on the test task:
test.finalizedBy jacocoTestReport
I am building a SonarQube 6.2 server which is already analyzing my Java 8/Gradle 3.3 projects. When adding JaCoCo to a multimodule gradle project, I realized that SonarQube is measuring code coverage on a "per-module" basis:
If a class is located in module A and a test for this class is located in module B, SonarQube figures the class is not covered.
I want to measure code coverage across all modules, not on a per module basis. How do I achieve this?
There are lots of similar questions but no helpful answers, although the situation seems quite common to me. Jenkins for example does that per default.
I decided to build a blueprint on github to clarify the issue.
The main build.gradle consists of
plugins { id "org.sonarqube" version "2.2.1" }
subprojects {
apply plugin: 'java'
apply plugin: 'jacoco'
repositories { mavenCentral() }
dependencies { testCompile "junit:junit:4.12" }
}
modA/build.gradleis empty.
It contains 3 classes: TestedInModA, TestedInModATest and TestedViaModB.
modB/build.gradlejust declares a dependency to modA:
dependencies { compile project(':modA') }
It contains just one class: TestedViaModBTest, testing the class TestedViaModB located in modA.
My (private) Jenkins instance shows 100% coverage for the two classes included while SonarQube says only the class TestedInModA (which is tested in its own module) is covered.
How can I modify my build process to see "cross-module coverage" in SonarQube?
I would love to update my project so future visitors to this question can find a working example.
My working solution (thanks #Godin)
add the following to the subprojects closure
tasks.withType(Test) {
// redirect all coverage data to one file
// ... needs cleaning the data prior to the build to avoid accumulating coverage data of different runs.
// see `task cleanJacoco`
jacoco {
destinationFile = file("$rootProject.buildDir/jacoco/test.exec")
}
}
add
task cleanJacoco(dependsOn: 'clean') { delete "$buildDir/jacoco" }
outside the subprojects closure.
When you perform build JaCoCo Gradle Plugin will produce modA/build/jacoco/test.exec and modB/build/jacoco/test.exe that contain information about execution of tests in modA and modB respectively. SonarQube performs analysis of modules separately, so during analysis of modA for the file TestedViaModB it sees only modA/build/jacoco/test.exec.
Most common trick to cross boundaries - is to collect all coverage information into single location. This can be done with
JaCoCo Gralde Plugin
either by changing location - see destinationFile and destPath ( since information is appended to exec file, don't forget to remove this single location prior to build, otherwise it will be accumulating information from different builds and not just from different modules ),
either by merging all files into single one - see JacocoMerge task. And then specify this single location to SonarQube as sonar.jacoco.reportPath.
Another trick: SonarQube 6.2 with Java Plugin 4.4 supports property sonar.jacoco.reportPaths allowing to specify multiple locations.
If you are interested in the solution with sonar.jacoco.reportPaths (see answer of Godin), have looke at this gradle code:
tasks.getByName('sonarqube') {
doFirst {
// lazy initialize the property to collect all coverage files
def jacocoFilesFromSubprojects = subprojects.findAll {it.plugins.hasPlugin('jacoco')}
.collect {it.tasks.withType(Test)}.flatten().collect {it.jacoco.destinationFile}
sonarqube.properties {
property "sonar.jacoco.reportPaths", jacocoFilesFromSubprojects.join(',')
}
}
}
This will collect all coverage binary files and set them as comma-separated list to the sonar property. Considers all test tasks of sub projects where jacoco is applied and their jacoco destination file configured.
I'm trying to integrate the sonarqube gradle plugin with the jacoco plugin:
classpath 'org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.1'
apply plugin: 'org.sonarqube'
apply plugin: 'jacoco'
My build/jacoco folder contains:
integrationTest.exec
test.exec
However, the sonarqube plugin only recognises the 'test.exec' file:
18:20:45.130 INFO - JaCoCoItSensor: JaCoCo IT report not found: C:\Users\abc3\Downloads\sme-letter\target\jacoco-it.exec
:
18:05:55.609 INFO - Analysing C:\Users\abc3\Downloads\sme-letter\build\jacoco\test.exec
How do I get the sonarqube plugin to recognise 'integrationTest.exec'
Thanks
Mark
I'm not really sure, whether this will work for Gradle plugun, but you may try.
Sonar has a property to specify the name of the integration tests JaCoCo report. This property is called sonar.jacoco.itReportPath (sonar.jacoco.reportPath for unit tests report).
And as far as I know, gradle sonar plugin let you add custom properties to it. So you can change IT report name via properties as follows:
sonarqube {
properties {
property "sonar.jacoco.itReportPath", "build/jacoco/ integrationTest.exec"
}
}
It works for me, but only after merging all jacoco reports into one file AND (important) deleting all other reports
property "sonar.java.coveragePlugin", "jacoco"
property "sonar.junit.reportsPath", "$projectDir/build/test-results"
property 'sonar.jacoco.reportPaths', "${rootProject.buildDir}/jacoco/mergeJacoco.exec"
And you need a merge jacoco task
Task mergeJacoco = tasks.create('mergeJacoco', JacocoMerge, {
doFirst {
delete "$buildDir/jacoco/mergeJacoco.exec"
}
destinationFile(new File(buildDir, 'jacoco/mergeJacoco.exec'))
executionData fileTree('./').include('**/*.exec-partial')
doLast {
delete fileTree('./').include('**/test.exec-partial')
}
})
mergeJacoco.outputs.upToDateWhen { false }
project.tasks["sonarqube"].dependsOn mergeJacoco
mergeJacoco.mustRunAfter ":myProject:test"
And setup jacoco to use those "partial" files
subprojects {
....
jacoco {
destinationFile file("$buildDir/jacoco/test.exec-partial")
append = false
includes = ['com.mypackage.*']
}
}
You'll get unit and IT reports mixed in one, but that's the best I could get
I have a custom task to do a Websphere EJBDeploy. I have defined the inputs and ouputs, and get successful incremental compilation, but I can't get the automatically generated clean task to work properly.
According to docs, for a custom task named "ejbDeploy" with a defined output, a cleanEjbDeploy task should be automatically generated.
Pattern: clean<TaskName>: Cleans the output files of a task.
So here's my custom task:
task ejbDeploy(dependsOn: 'jar'){
srcFile = file(jar.archivePath)
destDir = new File("build/ejbDeploy")
inputs.file srcFile
outputs.dir destDir
def cp = project.files(
project.sourceSets.main.output.classesDir,
project.sourceSets.main.resources,
project.configurations.runtime
).getAsPath()
doLast{
destDir.mkdirs()
exec{
executable = wasEjbDeploy
workingDir = destDir
args = [
jar.archivePath,
".",
jar.archiveName,
"-cp",
cp
]
}
}
}
Anyone have any ideas as to why the clean rule isn't working?
[Edit]
Here's the full (anonymized) file contents (this has changed since initial question post):
version = '1.0-SNAPSHOT'
group = 'com.company'
buildscript {
repositories {
mavenLocal()
mavenCentral()
}
dependencies {
classpath group: 'name.benjaminAbbitt', name: 'WASEjbDeploy', version: '1.0-SNAPSHOT'
}
}
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'base'
repositories {
mavenCentral()
mavenLocal()
}
dependencies {
compile 'javax.mail:mail:1.4.5'
compile 'log4j:log4j:1.2.16'
compile files(fileTree(dir: 'lib', includes: ['*.jar']) )
}
task ejbDeploy(type:name.benjaminAbbitt.WASEjbDeploy, dependsOn: 'jar'){
wasEjbDeployPath = wasEjbDeploy
}
Here's the relevant chunk of "$gradle tasks"
build - Assembles and tests this project.
buildDependents - Assembles and tests this project and all projects that depend
on it.
buildNeeded - Assembles and tests this project and all projects it depends on.
classes - Assembles the main classes.
clean - Deletes the build directory.
jar - Assembles a jar archive containing the main classes.
testClasses - Assembles the test classes.
Documentation tasks
-------------------
javadoc - Generates Javadoc API documentation for the main source code.
Help tasks
----------
dependencies - Displays the dependencies of project ':project'.
help - Displays a help message
projects - Displays the sub-projects of project ':project'.
properties - Displays the properties of project ':project'.
tasks - Displays the tasks runnable from project ':project' (some of the
displayed tasks may belong to subprojects).
IDE tasks
---------
cleanEclipse - Cleans all Eclipse files.
eclipse - Generates all Eclipse files.
Verification tasks
------------------
check - Runs all checks.
test - Runs the unit tests.
Other tasks
-----------
ejbDeploy
Rules
-----
Pattern: build<ConfigurationName>: Assembles the artifacts of a configuration.
Pattern: upload<ConfigurationName>: Assembles and uploads the artifacts belongin
g to a configuration.
Pattern: clean<TaskName>: Cleans the output files of a task.
To see all tasks and more detail, run with --all.
BUILD SUCCESSFUL
Total time: 3.321 secs
The clean rule is provided by the base plugin. Since many other plugins (e.g. java) already apply the base plugin, you typically don't have to apply the base plugin yourself. But in case:
apply plugin: "base"