Gradle / Jacoco / SonarQube issues - gradle

I'm going to put a fat bounty on this as soon as the system permits.
What I'm specifically having trouble with is getting coverage and getting integration tests working. For these I see the uninformative error:
Resource not found: com.something.somethingelse.SomeITCase
Uninformative because there's nothing to relate it to, nor does it mean much to someone who has none of the context.
Here's other weirdness I'm seeing. In cases where there's no integration tests for a subproject, I see this:
JaCoCoItSensor: JaCoCo IT report not found: /dev/build/dilithium/target/jacoco-it.exec
Why would I see target? This isn't a Maven project. A global search shows there's no target directory mentioned anywhere in the code base.
Then, there's this section of the documentation:
sonarqube {
properties {
properties["sonar.sources"] += sourceSets.custom.allSource.srcDirs
properties["sonar.tests"] += sourceSets.integTest.allSource.srcDirs
}
}
Near as I can tell sourceSets.integTest.allSource.srcDirs returns Files, not Strings. Also it should be:
sonarqube {
properties {
property "sonar.tests", "comma,separated,file,paths"
}
Note that you get an error if you have a directory in there that doesn't exist. Of course there's apparently no standard for what directory to put integration tests in and for some sub-projects they may not even exist. The Gradle standard would be to simply ignore non-existent directories. Your code ends up looking like:
sonarqube {
StringBuilder builder = new StringBuilder()
sourceSets.integrationTest.allSource.srcDirs.each { File dir ->
if ( dir.exists() ) {
builder.append(dir.getAbsolutePath())
builder.append(",")
}
}
if (builder.size() > 1) builder.deleteCharAt(builder.size() -1 )
if (builder.size() > 1 )
properties["sonar.tests"] += builder.toString()
properties["sonar.jacoco.reportPath"] +=
"$project.buildDir/jacoco/test.exec,$project.buildDir/jacoco/integrationTest.exec"
}
Sonar is reporting no coverage at all. If I search for the *.exec files, I see what I would expect. That being:
./build/jacoco/test.exec
./build/jacoco/integrationTest.exec
...but weirdly, I also see this:
./build/sonar/com.proj_name_component_management-component_proj-recordstate/jacoco-overall.exec
What is that? Why is it in such a non-standard location?
OK, I've added this code:
properties {
println "Before: " + properties.get("sonar.tests")
println "Before: " + properties.get("sonar.jacoco.reportPath")
property "sonar.tests", builder.toString()
property "sonar.jacoco.reportPath", "$project.buildDir/jacoco/test.exec,$project.buildDir/jacoco/integrationTest.exec"
println "After: " + properties.get("sonar.tests")
println "After: " + properties.get("sonar.jacoco.reportPath")
}
...which results in:
[still running]

I don't want any bounty or any points.
Just a suggestion.
Can you get ANY Jacoco reports at all?
Personally I would separate the 2: namely Jacoco report generation and Sonar.
I would first try to simply generate Jacoco THEN I would look at why Sonar can not get a hold of them.

Related

How to add multiple tasks of the same type in Gradle?

I need to generate FlatBuffers files from *.fbs file before the build.
So i'm using gradle.plugin.io.netifi:gradle-flatbuffers-plugin:1.0.7 to do it for me.
It works as expected for 1 task:
def generatedSourcePathJava = "$buildDir/generated/source/flatbuffers/java"
def generatedSourcePathCpp = "$buildDir/generated/source/flatbuffers/cpp"
...
task createFlatBuffersJava(type: io.netifi.flatbuffers.plugin.tasks.FlatBuffers) {
outputDir = file(generatedSourcePathJava)
language = "kotlin"
}
build.dependsOn createFlatBuffersJava
But if i add the 2nd one (to generate C++ files for JNI):
task createFlatBuffersJava(type: io.netifi.flatbuffers.plugin.tasks.FlatBuffers) {
outputDir = file(generatedSourcePathJava)
language = "kotlin"
}
task createFlatBuffersCpp(type: io.netifi.flatbuffers.plugin.tasks.FlatBuffers) {
outputDir = file(generatedSourcePathCpp)
language = "cpp"
}
assemble.dependsOn createFlatBuffersJava, createFlatBuffersCpp
Gradle build (../gradlew :engine-flatbuffers:clean :engine-flatbuffers:build) fails with the following:
What went wrong:
A problem occurred configuring project ':engine-flatbuffers'.
java.util.ConcurrentModificationException (no error message)
I think the question can be generalized to "How to add multiple tasks of the same type in Gradle?".
PS. "gradle-5.6-all"
That's a known plugin bug/feature reported at https://github.com/gregwhitaker/gradle-flatbuffers-plugin/issues/7.
It works in 1.0.5 but kotlin [argument] is sadly not supported there at that point. java works and it's compatible.

Passing a gradle task arguments in command line

I'm trying to find the best way to pass a gradle task arguments from the command line.
I have this task. I want to unpack solutions from student exercises and copy them into the right place in the project to evaulate them. I call this task like this:
> gradle swapSolution -Pstudent=MyStudent -Pexercise=ex05
One Problem i have with this while doing this in IntelliJ while having the Gradle plugin enabled is that i get this error message when build the project. What could be a solution to this?
A problem occurred evaluating root project 'kprog-2020-ws'.
> Could not get unknown property 'student' for root project 'kprog-2020-ws' of type org.gradle.api.Project.
This is the gradle task:
task swapSolution(type: Copy) {
new File("${rootDir}/Abgaben").eachDir { file ->
if (file.name.toString().matches("(.*)" + project.property("student") + "(.*)")) {
def exDir = new File("/src/main/java/prog/" + project.property("exercise"))
if (!exDir.exists()) {
delete exDir
}
new File(file.path).eachFile { zipSolution ->
//def zipFile = new File("./Abgaben/" + file.name.toString() + "/" + project.property("exercise") + "Solution.zip")
from zipTree(zipSolution)
into "/src/main/java/"
}
}
}
}
Do you have any suggestions to optimize this process?
-P denotes the Gradle Project Property. If you need to use project properties you can specify it as a system property in gradle.properties file in project root directory.
If your task is of type JavaExec you can use --args switch and pass it in Arguments text field of the Gradle Task Run Configuration togenther with the task name like swapSolution -args="-student=MyStudent -exercise=ex05". See also
https://stackoverflow.com/a/48370451/2000323

Gradle - Configure tests includes from property file

I've got a Java project build with Gradle and a property file that contains custom configuration for my testing framework (amount of thread to use, test environment url, custom username & password for those environments, etc...).
I'm facing an issue related to using properties from that file that I can't figure out:
if my Test task include '**/*Test.class', all tests are running as expected.
if my Test task include '**/MyTest.class', only that test is running as expected.
if my Test task include readProperty(), the task is skipped as NO-SOURCE. <- this is the part I can't understand - as the readProperty return the correct value.
Let's get into details:
This is how the property is defined in a my.property file:
testng.class.includes='**/MyTest.class'
This is what the build.gradle file looks like:
Properties props = new Properties()
props.load(new FileInputStream(projectDir.toString() + '/common.properties'))
def testsToRunWorking(p) {
String t = 'MyTest.class'
println "Tests = $t"
return t ? t : '**/*Test.class'
}
def testsToRunNotWorking(p) {
String t = getProperty(p, "testng.class.includes")
println "Tests = $t"
return t ? t : '**/*Test.class'
}
task testCustom(type: Test) {
outputs.upToDateWhen { false }
testLogging.showStandardStreams = true
classpath = configurations.customTest + sourceSets.customTest.output
include testsToRunNotWorking(props) ///< Does not work!
// include testsToRunWorking(props) ///< Works!
useTestNG()
}
In terms of debugging:
The println properly return the value I expect, even when the testCustom task doesn't do what I would expect.
I tried adding a dependsOn task just to print the content of testCustom.configure { println $includes } which looks correct as well.
--info
Tests = '**/MyTest.class'
:clean
:compileCustomTestJava - is not incremental (e.g. outputs have changed, no previous execution, etc.).
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:processCustomTestResources
:customTestClasses
:testCustom NO-SOURCE
The core of the issue seems to be coming from the fact that I'm reading that value from property. I hard coded inside the build.gradle everything works as expected. If read from a property file - build stops with a NO-SOURCE statement.
Any idea?
Thanks!
You are using quotation marks in the values of your property files. Everything that comes after the assignment sign in a property file is used as value, so the quotation marks remain in the string. They are printed in your output Tests = '**/MyTest.class'. On the other hand, if you define a string in your (Groovy) code with quotation marks, they are not included in the string. Therefor, the passed strings are not the same.
Remove the quotation marks from your property file(s) and everything should work, since the class files will match your string without the quotation marks.

Is it possible to override gradle from prompting error because ("." and "-") in the name of a variable? eg; name.dir (.dir not found) in task

The error message:
* What went wrong:
A problem occurred evaluating root project 'telescope-master'.
> Cannot get property 'dir' on null object
gradle.properties file
classes.dir = WebContent/WEB-INF/classes
webContent.dir = WebContent
template.dir = hdm/template
javascript.dir = hdm/function
javascript4.0.2.dir = hdm/function/4.0.2
datamodel.dir = hdm/datamodel
certificate.dir = certificate
build.gradle file
Properties extFile = new Properties()
extFile.load(new FileInputStream('gradle.properties'))
task FirmwareMatch(type: Zip) {
from ("${extFile.javascript.dir}")
include 'factoryResetOnFirmwareMatch.*'
archiveName 'factoryResetOnFirmwareMatch.zip'
destinationDir file('dist/hdm/function')
}
So basically if I remove the "." from .dir on both files it would work. But is there any way to over ride it?
Also how can I display actual date when using ${TODAY} in gradle.
So your problematic expression is:
extFile.javascript.dir
If we break that into how Groovy will interpret it:
extFile.getProperty('javascript').getProperty('dir')
You want Groovy to interpret it as:
extFile.getProperty('javascript.dir')
Besides directly calling getProperty, here are a couple Groovy options:
extFile.'javascript.dir'
extFile['javascript.dir']
Additionally, assuming your gradle.properties file is either in your project root (generally as a sibling to the build.gradle) or in your GRADLE_HOME directory (i.e. ~/.gradle/gradle.properties) it will be automatically loaded by Gradle and all properties available as project properties.
So you can remove all of your properties parsing code and just do the following:
project.getProperty('javascript.dir')
// or
project.'javascript.dir'
// or
project['javascript.dir']
If you want to protect against those properties not being set, and are on Gradle 2.13 or higher, you can use findProperty instead of getProperty which will return null instead of throwing an exception.

Gradle with Eclipse - incomplete .classpath when multiple sourcesets

I have a gradle build script with a handful of source sets that all have various dependencies defined (some common, some not), and I'm trying to use the Eclipse plugin to let Gradle generate .project and .classpath files for Eclipse, but I can't figure out how to get all the dependency entries into .classpath; for some reason, quite few of the external dependencies are actually added to .classpath, and as a result the Eclipse build fails with 1400 errors (building with gradle works fine).
I've defined my source sets like so:
sourceSets {
setOne
setTwo {
compileClasspath += setOne.runtimeClasspath
}
test {
compileClasspath += setOne.runtimeClasspath
compileClasspath += setTwo.runtimeClasspath
}
}
dependencies {
setOne 'external:dependency:1.0'
setTwo 'other:dependency:2.0'
}
Since I'm not using the main source-set, I thought this might have something to do with it, so I added
sourceSets.each { ss ->
sourceSets.main {
compileClasspath += ss.runtimeClasspath
}
}
but that didn't help.
I haven't been able to figure out any common properties of the libraries that are included, or of those that are not, but I can't find anything that I'm sure of (although of course there has to be something). I have a feeling that all included libraries are dependencies of the test source-set, either directly or indirectly, but I haven't been able to verify that more than noting that all of test's dependencies are there.
How do I ensure that the dependencies of all source-sets are put in .classpath?
This was solved in a way that was closely related to a similar question I asked yesterday:
// Create a list of all the configuration names for my source sets
def ssConfigNames = sourceSets.findAll { ss -> ss.name != "main" }.collect { ss -> "${ss.name}Compile".toString() }
// Find configurations matching those of my source sets
configurations.findAll { conf -> "${conf.name}".toString() in ssConfigNames }.each { conf ->
// Add matching configurations to Eclipse classpath
eclipse.classpath {
plusConfigurations += conf
}
}
Update:
I also asked the same question in the Gradle forums, and got an even better solution:
eclipseClasspath.plusConfigurations = configurations.findAll { it.name.endsWith("Runtime") }
It is not as precise, in that it adds other stuff than just the things from my source sets, but it guarantees that it will work. And it's much easier on the eyes =)
I agree with Tomas Lycken, it is better to use second option, but might need small correction:
eclipse.classpath.plusConfigurations = configurations.findAll { it.name.endsWith("Runtime") }
This is what worked for me with Gradle 2.2.1:
eclipse.classpath.plusConfigurations = [configurations.compile]

Resources