Gradle - Configure tests includes from property file - gradle

I've got a Java project build with Gradle and a property file that contains custom configuration for my testing framework (amount of thread to use, test environment url, custom username & password for those environments, etc...).
I'm facing an issue related to using properties from that file that I can't figure out:
if my Test task include '**/*Test.class', all tests are running as expected.
if my Test task include '**/MyTest.class', only that test is running as expected.
if my Test task include readProperty(), the task is skipped as NO-SOURCE. <- this is the part I can't understand - as the readProperty return the correct value.
Let's get into details:
This is how the property is defined in a my.property file:
testng.class.includes='**/MyTest.class'
This is what the build.gradle file looks like:
Properties props = new Properties()
props.load(new FileInputStream(projectDir.toString() + '/common.properties'))
def testsToRunWorking(p) {
String t = 'MyTest.class'
println "Tests = $t"
return t ? t : '**/*Test.class'
}
def testsToRunNotWorking(p) {
String t = getProperty(p, "testng.class.includes")
println "Tests = $t"
return t ? t : '**/*Test.class'
}
task testCustom(type: Test) {
outputs.upToDateWhen { false }
testLogging.showStandardStreams = true
classpath = configurations.customTest + sourceSets.customTest.output
include testsToRunNotWorking(props) ///< Does not work!
// include testsToRunWorking(props) ///< Works!
useTestNG()
}
In terms of debugging:
The println properly return the value I expect, even when the testCustom task doesn't do what I would expect.
I tried adding a dependsOn task just to print the content of testCustom.configure { println $includes } which looks correct as well.
--info
Tests = '**/MyTest.class'
:clean
:compileCustomTestJava - is not incremental (e.g. outputs have changed, no previous execution, etc.).
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:processCustomTestResources
:customTestClasses
:testCustom NO-SOURCE
The core of the issue seems to be coming from the fact that I'm reading that value from property. I hard coded inside the build.gradle everything works as expected. If read from a property file - build stops with a NO-SOURCE statement.
Any idea?
Thanks!

You are using quotation marks in the values of your property files. Everything that comes after the assignment sign in a property file is used as value, so the quotation marks remain in the string. They are printed in your output Tests = '**/MyTest.class'. On the other hand, if you define a string in your (Groovy) code with quotation marks, they are not included in the string. Therefor, the passed strings are not the same.
Remove the quotation marks from your property file(s) and everything should work, since the class files will match your string without the quotation marks.

Related

gradle: how to use result of task in configuration phase (plugin ospackage)

I have a gradle script in which I configure a plugin (in my case ospackage but I guess the same would apply to another plugin) using a variable as per:
ospackage {
...
version project.ext.my_version
...
}
This variable is first initialized and then is updated using a task that I call first in my build script:
ext {
...
my_version = "XXX"
...
}
task init{
group 'ho'
description 'get HO Version'
doLast {
...
project.ext.my_version = getParameter("VERSION")
...
}
}
the problem is that the plugin (in my case ospackage) always consider the initial value "XXX" and not the correct one that was set by executing the init task.
I know it has something to do with configuration and execution phase but still I cannot find a workaround to do what I want.
For info, I also tried to create a task like the one below but it also fail as it seems that buildDeb task does not overwrite ospackage version parameter
buildDeb {
doLast {
...
version project.ext.my_version
link('/usr/bin/aa', '/usr/bin/aa.sh')
...
}
}
I also tried to put at the end of my file something like:
ospackage.dependsOn("init")
but the problem is that ospackage is not recognized as a task
Thank you in advance for your help.
It looks to me like the essence of your question revolves around on-demand values. My understanding is that you would like to set a version number during the configuration phase and use that value during the execution phase to set a package version using the ospackage plugin.
The issue is that the ospackage documentation only provides examples (to date) that setup the package constants during the configuration phase. Obviously that won't work because it is the same time you are setting your version (something that must be able to be done in parallel). You have the right idea with doLast. I found that some things from the ospackage cannot go in "doLast" blocks like packageName (if you have more than one of the same package/task type), so I just put the things that require on-demand evaluation in that block (the version, because we need its evaluation delayed until the execution phase).
My solution was to create a variable that holds the function that resolves the version.
def versionForRpm = { -> project.version }
Create a configuration block
configurations.ext {
version = versionForRpm
...
}
This is an example of an on-demand value (aka lazily-evaluated value).
task someRpmBuild(type: Rpm) {
// all package configs that require evaluation during execution phase (lazy)
doLast {
version = configurations.ext.version
requires("someotherpackageinthisbuild", configurations.ext.version(), 0)
}
// all package configs that may be evaluated during the configuration phase
release = configurations.ext.release
packageGroup = configurations.ext.packageGroup
license = configurations.ext.license
packager = configurations.ext.packager
user = configurations.ext.user
distribution = configurations.ext.distribution
vendor = configurations.ext.vendor
url = configurations.ext.url
os = configurations.ext.os
buildHost = configurations.ext.buildHost
epoch = configurations.ext.epoch
arch = configurations.ext.arch
}
Note that configurations.ext.version will be "called" automatically in the execution phase. I needed to explicitly call it when used as an argument in requires, however.
according to the documentation, the task type is Deb:
task fooDeb(type: Deb) {
packageName // Default to project.name
packageDescription // Defaults to project.description
version // Version field, defaults to project.version
arch // Architecture, defaults to "all". E.g. "amd64", "all"
multiArch // Configure multi-arch behavior: NONE (default), SAME, FOREIGN, ALLOWED (see: https://wiki.ubuntu.com/MultiarchSpec )
release // DEB Release
epoch // Epoch, defaults to 0
user // Default user to permission files to
permissionGroup // Default group to permission files to, "group" is used by Gradle for the display of tasks
packageGroup
buildHost
license
packager
distribution
vendor
url
signingKeyId
signingKeyPassphrase
signingKeyRingFile
sourcePackage
provides
uid // Default uid of files
gid // Default gid of files
createDirectoryEntry // [Boolean]
maintainer // Defaults to packager
uploaders // Defaults to packager
priority
summary
conflicts
recommends
suggests
enhances
preDepends
breaks
replaces
}
where:
version Version field, defaults to project.version
might give the RPM plugin a try.
I was able to solve the issue i had, setting the destination for the ospackage copy destination to a calculated value by using
configurations.ext{
mydestdir = ""
rpmVersion = "1"
releaseNumber = "1"
}
task init{
group 'ho'
description 'get HO Version'
doLast {
...
configurations.ext.mydestdir = "/store/tmp/+getSubDir()"
configurations.ext.rpmVersion = "123"
configurations.ext.releaseNumber = "456"
...
}
}
task fooRpm(type: Rpm) {
dependsOn init
...
doLast(){
version = configurations.rpmVersion
release = configurations.releaseNumber
}
from(project.tempDir) {
into configurations.mydestdir
fileMode = 0644
user = "nobody"
permissionGroup = "nobody"
}
}
I think you'll have use type Deb, and make some changes, but this should speed up your build, and you can verify results by adding --scan before and after making these changes.

How do I specify optional inputs on a Gradle task without an #Optional annotation?

I want to use an optional input if it exists and just move on if it does not exist.
When I run gradle -Dorg.gradle.warning.mode=all I got a deprecation warning for just specifying the input:
A problem was found with the configuration of task ':addWorkingCopyInfo'. Registering invalid inputs and outputs via TaskInputs and TaskOutputs methods has been deprecated and is scheduled to be removed in Gradle 5.0.
- File '/Users/robert/test/special-build-tag' specified for property '$1' does not exist.
This is the task in the build script:
task addWorkingCopyInfo(type: Exec) {
inputs.file file("tagFile") // deprecated if the file does not exist
outputs.file file("generated/taginfo")
executable "perl" args "..."
}
I've seen that I could add an #Optional annotation if I had a custom task class, but that's not the case here.
My best solution was to add a check for the file and only make it an input if it exists. This seems to work.
task addWorkingCopyInfo(type: Exec) {
def tagFile = new File("tagFile");
if (tagFile.exists()) {
inputs.file tagFile
}
outputs.file file("generated/taginfo")
executable "perl" args "..."
}
Is there a better / more Gradle-ish way to do this?
The method inputs.files(...) returns a TaskInputFilePropertyBuilder that provides the methods optional() and optional(boolean).
Just try:
inputs.files('my-file').optional()

Is it possible to override gradle from prompting error because ("." and "-") in the name of a variable? eg; name.dir (.dir not found) in task

The error message:
* What went wrong:
A problem occurred evaluating root project 'telescope-master'.
> Cannot get property 'dir' on null object
gradle.properties file
classes.dir = WebContent/WEB-INF/classes
webContent.dir = WebContent
template.dir = hdm/template
javascript.dir = hdm/function
javascript4.0.2.dir = hdm/function/4.0.2
datamodel.dir = hdm/datamodel
certificate.dir = certificate
build.gradle file
Properties extFile = new Properties()
extFile.load(new FileInputStream('gradle.properties'))
task FirmwareMatch(type: Zip) {
from ("${extFile.javascript.dir}")
include 'factoryResetOnFirmwareMatch.*'
archiveName 'factoryResetOnFirmwareMatch.zip'
destinationDir file('dist/hdm/function')
}
So basically if I remove the "." from .dir on both files it would work. But is there any way to over ride it?
Also how can I display actual date when using ${TODAY} in gradle.
So your problematic expression is:
extFile.javascript.dir
If we break that into how Groovy will interpret it:
extFile.getProperty('javascript').getProperty('dir')
You want Groovy to interpret it as:
extFile.getProperty('javascript.dir')
Besides directly calling getProperty, here are a couple Groovy options:
extFile.'javascript.dir'
extFile['javascript.dir']
Additionally, assuming your gradle.properties file is either in your project root (generally as a sibling to the build.gradle) or in your GRADLE_HOME directory (i.e. ~/.gradle/gradle.properties) it will be automatically loaded by Gradle and all properties available as project properties.
So you can remove all of your properties parsing code and just do the following:
project.getProperty('javascript.dir')
// or
project.'javascript.dir'
// or
project['javascript.dir']
If you want to protect against those properties not being set, and are on Gradle 2.13 or higher, you can use findProperty instead of getProperty which will return null instead of throwing an exception.

Gradle / Jacoco / SonarQube issues

I'm going to put a fat bounty on this as soon as the system permits.
What I'm specifically having trouble with is getting coverage and getting integration tests working. For these I see the uninformative error:
Resource not found: com.something.somethingelse.SomeITCase
Uninformative because there's nothing to relate it to, nor does it mean much to someone who has none of the context.
Here's other weirdness I'm seeing. In cases where there's no integration tests for a subproject, I see this:
JaCoCoItSensor: JaCoCo IT report not found: /dev/build/dilithium/target/jacoco-it.exec
Why would I see target? This isn't a Maven project. A global search shows there's no target directory mentioned anywhere in the code base.
Then, there's this section of the documentation:
sonarqube {
properties {
properties["sonar.sources"] += sourceSets.custom.allSource.srcDirs
properties["sonar.tests"] += sourceSets.integTest.allSource.srcDirs
}
}
Near as I can tell sourceSets.integTest.allSource.srcDirs returns Files, not Strings. Also it should be:
sonarqube {
properties {
property "sonar.tests", "comma,separated,file,paths"
}
Note that you get an error if you have a directory in there that doesn't exist. Of course there's apparently no standard for what directory to put integration tests in and for some sub-projects they may not even exist. The Gradle standard would be to simply ignore non-existent directories. Your code ends up looking like:
sonarqube {
StringBuilder builder = new StringBuilder()
sourceSets.integrationTest.allSource.srcDirs.each { File dir ->
if ( dir.exists() ) {
builder.append(dir.getAbsolutePath())
builder.append(",")
}
}
if (builder.size() > 1) builder.deleteCharAt(builder.size() -1 )
if (builder.size() > 1 )
properties["sonar.tests"] += builder.toString()
properties["sonar.jacoco.reportPath"] +=
"$project.buildDir/jacoco/test.exec,$project.buildDir/jacoco/integrationTest.exec"
}
Sonar is reporting no coverage at all. If I search for the *.exec files, I see what I would expect. That being:
./build/jacoco/test.exec
./build/jacoco/integrationTest.exec
...but weirdly, I also see this:
./build/sonar/com.proj_name_component_management-component_proj-recordstate/jacoco-overall.exec
What is that? Why is it in such a non-standard location?
OK, I've added this code:
properties {
println "Before: " + properties.get("sonar.tests")
println "Before: " + properties.get("sonar.jacoco.reportPath")
property "sonar.tests", builder.toString()
property "sonar.jacoco.reportPath", "$project.buildDir/jacoco/test.exec,$project.buildDir/jacoco/integrationTest.exec"
println "After: " + properties.get("sonar.tests")
println "After: " + properties.get("sonar.jacoco.reportPath")
}
...which results in:
[still running]
I don't want any bounty or any points.
Just a suggestion.
Can you get ANY Jacoco reports at all?
Personally I would separate the 2: namely Jacoco report generation and Sonar.
I would first try to simply generate Jacoco THEN I would look at why Sonar can not get a hold of them.

Issue with Gradle task to write file

I am developing an Android application where I have a directory of JSON files and I want to create a gradle task that will combine all these files into a single JSON file.
This is the gradle task I have so far but does not create the file:
// Task that combines all JSON files in ../libraries into src/main/res/raw/libraries.json
task combineJSonFiles {
String content = ""
FileTree tree = fileTree(dir: '../libraries', include: '**/*.json')
tree.each {File file ->
content += file.getText()
}
println "[" + content.substring(0, content.length()-1) + "]" // prints out the correct contents
File libraries = file("../app/src/main/res/raw/libraries.json")
println libraries.getProperties()
}
I print out the properties and I am not sure why these are the property values:
{directory=false, canonicalFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, file=false, freeSpace=0, canonicalPath=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, usableSpace=0, hidden=false, totalSpace=0, path=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, name=libraries.json, prefixLength=1, absolute=true, class=class java.io.File, parentFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw, absolutePath=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, absoluteFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, parent=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw}
Any help is appreciated as I have not seemed to figure this out even after reading the documentation. http://www.gradle.org/docs/current/userguide/working_with_files.html
I am just posting the code for the task that now works:
task combineJSonFiles {
String content = ""
FileTree tree = fileTree(dir: '../libraries', include: '**/*.json')
tree.each {File file ->
content += file.getText()
}
def libraries = new File("app/src/main/res/raw/libraries.json")
libraries.text = "[" + content.substring(0, content.length()-1) + "]"
}
My issue was trying to use Java.io.File and having the wrong directory path set for my file.
Creating an instance of java.io.File in Groovy/Java does not create the file on disk. You will need to write something to it. Check out this tutorial for working with files in Groovy.
Also you have put your task implementation in a task configuration block, rather than a task action. This means your code will not be running when you are expecting - it will run every time you run gradle rather than when you run this task. You need to put your code in a doLast block

Resources