How to upload test reports of Kotlin sources to Coveralls? - gradle

I want to upload my Jacoco test report to Coveralls automatically after my Travis build finishes. It works for Java, but how to configure it for Kotlin?
Error message
I can generate a Jacoco test report locally and on Travis, but when Travis tries to submit to coveralls it fails with message
> Task :coveralls
No source file found on the project: "kotlin-template-project"
With coverage file: /home/travis/build/myname/myreponame/build/reports/jacoco/test/jacocoTestReport.xml
Google links me to the Gradle plugin implementation which shows where it throws this message, which tells me (I think) that the Jacoco report file is found but not the source files which coveralls apparently needs.
What I tried
Hence, I tried pointing the coveralls task to my source files, in all of these ways:
coveralls {
sourceDirs += allprojects.sourceSets.main.allSource.srcDirs.flatten()
sourceDirs += files(sourceSets.main.kotlin.srcDirs).files.absolutePath
project.extensions.coveralls.sourceDirs += project.sourceSets.main.kotlin.srcDirs
sourceDirs += ['src/main/kotlin']
jacocoReportPath = 'build/reports/jacoco/test/jacocoTestReport.xml'
sourceDirs += ['src/test/kotlin']
sourceDirs += ["${projectDir}/src/main/kotlin"]
}
I also tried adding sourceSets project.sourceSets.main to the jacocoTestReport task.
Project setup
My minimal build.gradle file:
plugins {
id 'org.jetbrains.kotlin.jvm' version '1.2.50'
id 'java' // Required by at least JUnit.
// Test coverage
id 'jacoco'
// Upload jacoco coverage reports to coveralls
id 'com.github.kt3k.coveralls' version '2.8.2'
}
dependencies {
compile 'org.jetbrains.kotlin:kotlin-stdlib-jdk8'
// JUnit 5
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.2.0'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.2.0'
testRuntime 'org.junit.platform:junit-platform-console:1.2.0'
// Kotlintest
testCompile 'io.kotlintest:kotlintest-core:3.1.6'
testCompile 'io.kotlintest:kotlintest-assertions:3.1.6'
testCompile 'io.kotlintest:kotlintest-runner-junit5:3.1.6'
// Spek
testCompile 'org.jetbrains.spek:spek-api:1.1.5'
testRuntime 'org.jetbrains.spek:spek-junit-platform-engine:1.1.5'
}
repositories {
mavenCentral()
mavenLocal()
jcenter()
}
test {
// Enable JUnit 5 (Gradle 4.6+).
useJUnitPlatform()
// Always run tests, even when nothing changed.
dependsOn 'cleanTest'
// Show test results.
testLogging {
events "passed", "skipped", "failed"
}
}
// Test coverage reporting
jacocoTestReport {
// Enable xml for coveralls.
reports {
html.enabled = true
xml.enabled = true
xml.setDestination(file("${buildDir}/reports/jacoco/test/jacocoTestReport.xml"))
}
}
coveralls {
sourceDirs += ['src/main/kotlin']
jacocoReportPath = 'build/reports/jacoco/test/jacocoTestReport.xml'
}
Related issues
Essentially the same issue is on https://github.com/kt3k/coveralls-gradle-plugin/issues/32 but the solution is to set sourceDirs and jacocoReportPath which I already have tried.
At https://github.com/kt3k/coveralls-gradle-plugin/issues/39 and https://github.com/kt3k/coveralls-gradle-plugin/issues/63 it is suggested to add sourceDirs += ['src/main/kotlin'] which sounds sensible but doesn't help. Same for, from the first link, sourceDirs = files(sourceSets.main.kotlin.srcDirs).files.absolutePath.
From https://github.com/kt3k/coveralls-gradle-plugin/issues/77 the solution is project.extensions.coveralls.sourceDirs += project.sourceSets.main.kotlin.srcDirs which I tried.
The question Kotlin code coverage in CI pipeline is phrased rather generally, but a comment points to discuss.kotlinlang.org where someone shows a way to improve the Jacoco result regarding kotlin, and the answer links to the Jacoco Gradle plugin which I use and works: when I run the jacocoTestReport task a report is generated in build/reports/jacoco/test/, both in xml and html.
The question Kotlin Test Coverage is also phrased general and answered with an unnecessarily complex build file from which I learned nothing new.
The question Measure test coverage for Kotlin code? claims that the Jacoco report does not work, but for me this is not the case as I said.
There are similar questions for Java, like Tool for java code coverage on GitHub but for me when I use Java it all works fine.
PS Actually I want to use the Gradle Kotlin DSL, but since nobody seems to use it I'm asking this question for Gradle. But in the end I want this question solved for the Kotlin DSL as well.

[edit August 2020] #nbaztec wrote a plugin to support Kotlin, please see his answer.
Old answer:
Kotlin is not supported by Coveralls, see for example this open isse that was mentioned in the question as well (in the question it was also mentioned that the workaround presented there does not work): https://github.com/kt3k/coveralls-gradle-plugin/issues/77
Solution: try Codecov.io instead.
Install it to GitHub using the Marketplace and add to your .travis.yml
after_success:
- bash <(curl -s https://codecov.io/bash)
Then commit and push, done!
You can view the result (after the build finished) at https://codecov.io/gh/githubaccountname/reponame

Had a similar experience with a variety of QA products not supporting or only partially supporting Kotlin codebases. Tried submitting support PRs to a couple of projects to no avail.
In the end ended up going with Coveralls and contributed a Kotlin focused plugin for the platform
https://github.com/nbaztec/coveralls-jacoco-gradle-plugin
Usage
Include the plugin in your build.gradle.kts (similar for build.gradle files):
plugins {
jacoco
id("com.github.nbaztec.coveralls-jacoco")
}
Then set the environment variable COVERALLS_REPO_TOKEN to the token from your Coveralls page.
Now you can use the coverallsJacoco task to publish a coverage report.
For more information and usage in CI, see
https://github.com/nbaztec/coveralls-jacoco-gradle-plugin

Not an answer, but in case anyone else is struggling with nbaztec like me, I want to give an alternative that worked for me: https://github.com/kt3k/coveralls-gradle-plugin
And besides what is in README.md, I needed this detail in build.gradle:
coveralls {
sourceDirs += ['src/main/kotlin']
jacocoReportPath "${buildDir}/reports/jacoco/report.xml"
}

Related

How to use JUnit 5 with Gradle?

I am trying to use JUnit 5 with Gradle after I succeeded in running a JUnit 4 test.
Expected result: Tthe JUnit 4 test gave a nice 'passed' in the output and an html report in build/reports/tests.
Actual result: The JUnit 5 test as below does not output anything besides (...) build succesful, while I know the test is not actually run since there is no test log output passed/skipped/failed, and putting a fail in the test keeps the build successful.
Running gradle test --info yields Skipping task ':testClasses' as it has no actions. among a lot of I think mostly unrelevant output.
Surprisingly, it also says Executing task ':test' and Generating HTML test report... Finished generating test html results and similar for the xml in build/test-results/test, while the xml is not generated, the html shows no tests run and no errors, and the test is indeed not run.
What I also think very interesting, is that gradle test --debug yields
[TestEventLogger] Gradle Test Run :test STARTED
[org.gradle.api.internal.tasks.testing.junit.JUnitDetector] test-class-
scan : failed to scan parent class java/lang/Object, could not find the class file
[TestEventLogger]
[TestEventLogger] Gradle Test Run :test PASSED
while my only test contains
fail("test fails");
which I think is very strange!
My build file is
apply plugin: 'java'
test {
dependsOn 'cleanTest' // run tests every time
}
sourceSets {
main {
java {
srcDirs 'src'
}
}
test {
java {
srcDirs 'test'
}
}
}
repositories {
mavenCentral()
}
dependencies {
// when using this, it worked with a junit 4 test
// testCompile 'junit:junit:4.10'
// this should be needed for junit 5 (using M4 is required since IJ 2017.1.2
testCompile("org.junit.jupiter:junit-jupiter-api:5.0.0-M4")
}
test {
testLogging {
events "passed", "skipped", "failed"
}
}
My test is
package mypackage;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class HelloWorldTest {
#Test
public void testHelloWorld(){
assertEquals(2, 1+1, "message");
}
}
My folder structure is, using package mypackage,
java-template-project
--- src
--- mypackage
--- HelloWorld.java
--- test
--- mypackage
--- HelloWorldTest.java
and in IntelliJ 2017.1.3, which I am using, the module structure looks like this
java-template-project
--- java-template-project_main
--- src/mypackage
--- HelloWorld(.java)
--- java-template-project_test
--- test/mypackage
--- HelloWorldTest(.java)
because Gradle nowadays wants the source and tests in their own package.
What I tried
Obviously this is not the first question about this topic, all the relevant questions I found are
Gradle project running jUnit 5 tests in IntelliJ
But as you can see this is for older versions of IntelliJ, and I am already using the syntax for IJ 2016.3.3 and higher according to one of the answers, in in the one JUnit dependency line, so that should be okay.
Upgrade from JUnit 4 to JUnit 5 in intellij with gradle
Links back to above question, and links to this Jetbrains blog which uses the same line as above question. Also links to:
Integrate JUnit 5 tests results with Intellij test report
This one shows, in the question, as dependency also
testRuntime("org.junit.vintage:junit-vintage-engine:5.0.0-M1")
which is explained in Why were JUnit Jupiter and JUnit Vintage separated When I Running TestCase in IntelliJ?
Well, when I ran it, the output showed it couldn't find this version but according to the Maven Repository this one is for JUnit 5:
testRuntime("org.junit.vintage:junit-vintage-engine:4.12.0-M4")
The answers there note that you can just run the tests within IntelliJ since the later versions have JUnit 5 support. I know, and the test runs fine when I run from within IntelliJ. But I want to use Gradle (and Travis, which needs dependency management).
How to capture stdout/stderr in junit 5 gradle test report?
I tried using
testCompile("org.junit.platform:junit-platform-gradle-plugin:1.0.0-M3")
testCompile("org.junit.jupiter:junit-jupiter-engine:5.0.0-M3")
but results didn't change.
My template project is located on https://github.com/PHPirates/java-template-project but this question should contain all information necessary.
New: JUnit 5 support in Gradle 4.6
As pointed out in this GitHub issue from Gradle 4.6 onwards JUnit 5 is supported!
Official release notes of 4.6 (at the moment of editing the latest, but check the GitHub releases page to make sure you use the latest version) at docs.gradle.org. The old setup will still work, but using this makes the build file a lot cleaner.
[Edit May 2019] As #deFreitas pointed out in his answer, the JUnit documentation has improved and now they provide a complete example at https://github.com/junit-team/junit5-samples/tree/r5.4.0/junit5-jupiter-starter-gradle, see especially the build.gradle there. Fortunately it turns out to be effectively the same as the one from this answer.
Update Gradle
First, make sure you are using the latest Gradle version, check latest releases at their GitHub releases. If that is for example 4.6, run in a terminal in your project location gradlew wrapper --gradle-version=4.6 or make sure to update this line in your gradle/wrapper/gradle-wrapper.properties file: distributionUrl=https\://services.gradle.org/distributions/gradle-4.6-all.zip.
How to use the built-in JUnit 5
Then with the java files, directory structure etc. from the question the build.gradle file will be (using the new plugins block)
plugins {
id 'java'
}
repositories {
mavenCentral()
mavenLocal()
}
dependencies {
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.0.3'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.0.3'
}
// These lines can be removed when you use the default directories src/main/kotlin and src/test/kotlin
sourceSets {
main.java.srcDirs += 'src'
main.resources.srcDirs += 'src'
test.java.srcDirs += 'test'
test.resources.srcDirs += 'test'
}
// Java target version
sourceCompatibility = 1.8
test {
// Enable JUnit 5 (Gradle 4.6+).
useJUnitPlatform()
// Always run tests, even when nothing changed.
dependsOn 'cleanTest'
// Show test results.
testLogging {
events "passed", "skipped", "failed"
}
}
PS For the absolute minimal version, see Ray's answer.
Android (See this post: JUnit 5 for Android testing)
On Android I managed to run the JUnit 5 test from the question by adding the following to my app module build file. As you can see the dependencies are the same, but I didn't need useJUnitPlatform() and the test configuration block is slightly different.
apply plugin: 'com.android.application'
// In fact I am not sure you need this, but I had it included to run Spek tests anyway
apply plugin: 'de.mannodermaus.android-junit5'
repositories {
mavenCentral()
jcenter()
}
dependencies {
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.3.1'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.3.1'
}
android {
// I'm omitting your other configurations like compileSdkVersion, buildTypes etc.
testOptions {
unitTests.all {
// Always run tests, even when nothing changed.
dependsOn 'clean'
// Show test results.
testLogging {
events "passed", "skipped", "failed"
}
}
}
}
however, it only works for me when I execute the Gradle test task, not when I run the check task. As usual, I test this by creating a failing test and then I try if the Gradle task passes or fails.
You need the engines for both JUnit versions, and you need to apply the JUnit platform gradle plugin. I do not see that in your gradle file. Here is a working gradle build executing both JUnit 4 and 5:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath ("org.junit.platform:junit-platform-gradle-plugin:1.0.0-M4")
}
}
apply plugin: 'org.junit.platform.gradle.plugin'
...
dependencies {
...
testCompile("junit:junit:4.12")
testRuntime("org.junit.vintage:junit-vintage-engine:4.12.0-M4")
testCompile("org.junit.jupiter:junit-jupiter-api:5.0.0-M4")
testRuntime("org.junit.jupiter:junit-jupiter-engine:5.0.0-M4")
// Enable use of the JUnitPlatform Runner within the IDE
testCompile("org.junit.platform:junit-platform-runner:1.0.0-M4")
}
junitPlatform {
details 'tree'
}
See the JUnit doc form more information on that.
just adding to the knowledge base, i just got the following to work with gradle 4.7:
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.1.1'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.1.1'
}
test {
useJUnitPlatform()
}
Due to github issue built-in support for JUnit 5, scheduled for Gradle 4.6
Thus since gradle 4.6 your expected result have to be the same as actual result.
Expected result: Tthe JUnit 4 test gave a nice 'passed' in the output
and an html report in build/reports/tests.
UPD:
gradle 4.6-rc-1 was released on 16th of February 2018 and this version provides the built-in support for junit 5.
To enable junit 5 support you need to update gradle wrapper:
gradle wrapper --gradle-version=4.6-rc-1
and add just one line to build.gradle:
test {
useJUnitPlatform()
}
Checkout junit official documentation of how to use junit 5 with gradle.
build.gradle
plugins {
id 'java'
}
repositories {
mavenCentral()
}
dependencies {
testImplementation('org.junit.jupiter:junit-jupiter:5.4.0')
}
test {
useJUnitPlatform()
testLogging {
events "passed", "skipped", "failed"
}
}
Maybe something helpful for those who were struck with this problem when trying to integrate JUnit5 with gradle version 4.10.
Could not find method test() for arguments [build_dzas89s5z18l3bfyn6b3q0dxv$_run_closure2$_closure9#8e60c6] on project ':app' of type org.gradle.api.Project.
Actually, with 4.10 you don't need to add this test configuration block in build.gradle to enable JUnit5.
test {
useJUnitPlatform()
}
It should work fine just by adding the necessary dependencies of jupitor-api and jupitor-engine.
I tried to explore release notes of 4.10 but couldn't find anything about this change. If someone knows more about the "Why" behind it then please englighten me as well.

Generate coverage for other modules

My project structure is as follows: :app, :core. :app is the Android application project, and it depends on :core which has all the business logic.
I have Espresso tests for :app, and I am able to run and get coverage report thanks to all the questions and guides out there. But the coverage is only for code in :app.
How do I get coverage for all projects (:app and :core) resulting from my Espresso instrumentation tests? Is this even possible?
Any help is greatly appreciated.
Even if you did not mention Jacoco in your question, it is listed in the tags, so I assume you want to create coverage reports by using it. The Gradle Jacoco plugin (apply plugin: 'jacoco') allows you to create custom tasks of the type JacocoReport.
Defining such a task, you can specify source and class directories as well as the execution data:
task jacocoTestReport(type: JacocoReport) {
dependsOn // all your test tasks
reports {
xml.enabled = true
html.enabled = true
}
sourceDirectories = // files() or fileTree() to your source files
classDirectories = // files() or fileTree() to your class files
executionData = // files() or fileTree() including all your test results
}
For Espresso tests, the execution data should include all generated .ec files. You could define this task in the Gradle root project and link to both the files and the tasks of your subprojects.
There is an example for a custom Jacoco report task, even if it aims on creating an unified report for multiple test types, it can be transfered for this multi-project problem.
By default the JacocoReport task only reports coverage on sources within the project. You'll need to set the additionalSourceDirs property on the JacocoReport task.
app/build.gradle
apply plugin: 'java'
apply plugin: 'jacoco'
// tell gradle that :core must be evaluated before :app (so the sourceSets are configured)
evaluationDependsOn(':core')
jacocoTestReport {
def coreSourceSet = project(':core').sourceSets.main
additionalSourceDirs.from coreSourceSet.allJava
additionalClassDirs.from coreSourceSet.output
}

SonarQube: Coverage incomplete on multimodule gradle project with JaCoCo

I am building a SonarQube 6.2 server which is already analyzing my Java 8/Gradle 3.3 projects. When adding JaCoCo to a multimodule gradle project, I realized that SonarQube is measuring code coverage on a "per-module" basis:
If a class is located in module A and a test for this class is located in module B, SonarQube figures the class is not covered.
I want to measure code coverage across all modules, not on a per module basis. How do I achieve this?
There are lots of similar questions but no helpful answers, although the situation seems quite common to me. Jenkins for example does that per default.
I decided to build a blueprint on github to clarify the issue.
The main build.gradle consists of
plugins { id "org.sonarqube" version "2.2.1" }
subprojects {
apply plugin: 'java'
apply plugin: 'jacoco'
repositories { mavenCentral() }
dependencies { testCompile "junit:junit:4.12" }
}
modA/build.gradleis empty.
It contains 3 classes: TestedInModA, TestedInModATest and TestedViaModB.
modB/build.gradlejust declares a dependency to modA:
dependencies { compile project(':modA') }
It contains just one class: TestedViaModBTest, testing the class TestedViaModB located in modA.
My (private) Jenkins instance shows 100% coverage for the two classes included while SonarQube says only the class TestedInModA (which is tested in its own module) is covered.
How can I modify my build process to see "cross-module coverage" in SonarQube?
I would love to update my project so future visitors to this question can find a working example.
My working solution (thanks #Godin)
add the following to the subprojects closure
tasks.withType(Test) {
// redirect all coverage data to one file
// ... needs cleaning the data prior to the build to avoid accumulating coverage data of different runs.
// see `task cleanJacoco`
jacoco {
destinationFile = file("$rootProject.buildDir/jacoco/test.exec")
}
}
add
task cleanJacoco(dependsOn: 'clean') { delete "$buildDir/jacoco" }
outside the subprojects closure.
When you perform build JaCoCo Gradle Plugin will produce modA/build/jacoco/test.exec and modB/build/jacoco/test.exe that contain information about execution of tests in modA and modB respectively. SonarQube performs analysis of modules separately, so during analysis of modA for the file TestedViaModB it sees only modA/build/jacoco/test.exec.
Most common trick to cross boundaries - is to collect all coverage information into single location. This can be done with
JaCoCo Gralde Plugin
either by changing location - see destinationFile and destPath ( since information is appended to exec file, don't forget to remove this single location prior to build, otherwise it will be accumulating information from different builds and not just from different modules ),
either by merging all files into single one - see JacocoMerge task. And then specify this single location to SonarQube as sonar.jacoco.reportPath.
Another trick: SonarQube 6.2 with Java Plugin 4.4 supports property sonar.jacoco.reportPaths allowing to specify multiple locations.
If you are interested in the solution with sonar.jacoco.reportPaths (see answer of Godin), have looke at this gradle code:
tasks.getByName('sonarqube') {
doFirst {
// lazy initialize the property to collect all coverage files
def jacocoFilesFromSubprojects = subprojects.findAll {it.plugins.hasPlugin('jacoco')}
.collect {it.tasks.withType(Test)}.flatten().collect {it.jacoco.destinationFile}
sonarqube.properties {
property "sonar.jacoco.reportPaths", jacocoFilesFromSubprojects.join(',')
}
}
}
This will collect all coverage binary files and set them as comma-separated list to the sonar property. Considers all test tasks of sub projects where jacoco is applied and their jacoco destination file configured.

Code coverage for EJB's Using Wildfly-Arquillian-Gradle-Jacoco

I am trying to set-up code coverage for my project. I am using Wildfly 8.2 server, gradle as a build tool, and JUnit and Arquillian for testing. In gradle I have configured jacoco plugin to generate code coverage. I have a task called jacocoTestReport which allows me to generate an html report.
Something about running the tests:
I am working on a multi module project, each sub-project has a Deployments class in which we have two methods - one for creating a shrinkwrap archive of REST classes and other for non-REST classes. In arqullian.xml we are configuring this as REST_CONTAINER and NON_REST_CONTAINER and giving path to WildFly installation directory. When we run gradle build test , It will run the whole tests by deploying the REST.ear and non-REST.ear and generate the coverage reports.
The issue is code coverage for EJB's and other server managed classes are showing 0% (From primary ananlysis of coverage report). Also I analysed the jacoco.exec, there I found the classes which are showing 0% coverage are not listed in the file (Mostly bean classes).
Can someone provide me the correct configuration which works for the combination: Wildfly-Arquillian-Gradle-Jacoco
Note: I am ok to use tools other than jacoco, tried cobertura but same result.
This worked for me (but I used jboss7 should not be a problem) source:https://developer.jboss.org/thread/241883
apply plugin: 'jacoco'
jacoco {
toolVersion = '0.7.4.201502262128'
reportsDir = file("$buildDir/jacoco")
}
dependencies {
testCompile 'YOUR_ARQUILLIAN_ADAPTER'
testCompile 'org.jboss.arquillian.junit:arquillian-junit-container:1.1.5.Final'
testCompile 'org.jboss.arquillian.extension:arquillian-jacoco:1.0.0.Alpha7'
}
// Important: add the Jacoco libs on the test classpath (required for the Jacoco Arquillian extension to work).
sourceSets {
test.runtimeClasspath += configurations.jacocoAnt
}

How to use JMH with gradle?

I want to use JMH, an OpenJDK microbenchmark tool, with gradle. However, Im getting the NPE on compilation. On the other hand, JMH works when using from maven.
I am not posting any build.gradle as it is basic - apply java plugin and add dependency on JHM tool (org.openjdk.jmh:jmh-core:0.2).
I have tried whats written here without success.
What else I have to do? I think something with setting the agent, but I still didnt figure it out.
Exception:
:compileJava
java.lang.NullPointerException
at org.openjdk.jmh.processor.internal.GenerateMicroBenchmarkProcessor.validMethodSignature(GenerateMicroBenchmarkProcessor.java:502)
Just finished my "masterpiece". No uber-jars, no plugins, code base separated from main & test, benchmarks compilation hooked to main, but does not run automatically in the mainstream lifecycle. Simple, explicit, and hackable, vanilla gradle.
I run it directly from IntelliJ, to run on a box you probably will need the uber-jar back :-)
Before doing it I have spent a fair amount of time trying to get that plugin work, but it's way too clunky for my taste.
Step-by-step breakdown below.
Define a new sourceSet called jmh with classpath hooked to that of the main sourceSet
sourceSets {
jmh {
java.srcDirs = ['src/jmh/java']
scala.srcDirs = ['src/jmh/scala']
resources.srcDirs = ['src/jmh/resources']
compileClasspath += sourceSets.main.runtimeClasspath
}
}
Define dependencies for it (at minimum JMH and its annotation processor).
dependencies {
...
jmhImplementation 'org.openjdk.jmh:jmh-core:1.35'
jmhImplementation 'org.openjdk.jmh:jmh-generator-annprocess:1.35'
}
Define a task jmh of type JavaExec
task jmh(type: JavaExec, dependsOn: jmhClasses) {
main = 'org.openjdk.jmh.Main'
classpath = sourceSets.jmh.compileClasspath + sourceSets.jmh.runtimeClasspath
}
Hook jmhClasses task to run after classes to make sure benchmarks are compiled with the rest of the code
classes.finalizedBy(jmhClasses)
Currently you can just use dedicated plugin jmh-gradle-plugin
It requires minimal configuration and allows you to run JMH benchmarks as well as build benchmarks artifact
My bad, I was trying to benchmark a method that has an argument - of course JMH will not know what to pass :) Once when I created a void method with no arguments, everything worked.
My build.gradle:
defaultTasks 'build'
apply plugin: 'java'
apply plugin: 'shadow'
buildscript {
repositories {
mavenCentral()
maven {
name 'Shadow'
url 'http://dl.bintray.com/content/johnrengelman/gradle-plugins'
}
}
dependencies {
classpath 'org.gradle.plugins:shadow:0.7.4'
}
}
jar {
manifest {
attributes 'Main-Class': 'org.openjdk.jmh.Main'
}
}
repositories {
mavenCentral()
}
build.doLast {
tasks.shadow.execute()
}
shadow {
outputFile = new File('build/libs/microbenchmarks.jar')
}
ext {
lib = [
... other dependencies...
jmh: 'org.openjdk.jmh:jmh-core:0.2'
]
}
dependencies {
compile lib... other dependencies...
compile lib.jmh
}
sourceCompatibility = 1.7
Build tests and jar:
gw clean build
and then run them with:
java -jar build/libs/microbenchmarks.jar ".*" -wi 2 -i 10 -f 2 -t 16
UPDATE
From recent versions of JMH, you would also need to add dependency to:
org.openjdk.jmh:jmh-generator-annprocess:0.5.4
and you can use shadow 0.8.
I made a very small example project to clone and modify as you like. It's a full working example:
https://gitlab.com/barfuin/jmh-gradle-example
It requires no shadow Jars and no plugins, while still running the benchmark in a dedicated JVM. The project also includes some extra Gradle tasks for printing the classpath, the JMH options, etc., stuff that may help to understand what's going on.
If you're an IntelliJ user, perhaps the easiest way to make it work, without all that workarounds, is to use the IDE plugin:
https://github.com/artyushov/idea-jmh-plugin
Add the dependencies
Create your benchmark
be happy :)

Resources