Gradle CustomTask not considering InputFiles for dependencies - gradle

I'm trying to write a custom Gradle task that takes a specific set of input files, which may be in different directories, an output file and call a javaexec with the inputs.
I currently have it where it will skip the #TaskAction if it sees that the #OutputFile exists but it pays no consideration to the InputFiles list and if they are newer than the output file.
Here are the relevant parts of the buildfile:
class JavaCPPTask extends DefaultTask {
def className = []
String outputDir = ""
String localClasspath = "bin/classes"
String runIfExists = ""
String javacpp = "libs/javacpp.jar"
String outputName = ""
//#InputDirectory
//File inputFile = new File("bin/classes/com/package/ve")
#InputFiles
List<File> getInputFiles() {
ArrayList<File> inputFiles = new ArrayList<File>()
for ( int i = 0; i < className.size(); i++ ) {
def inputFileStr = localClasspath + '/' + className.get(i).replace('.','/') + '.class'
println("InputFileStr: " + inputFileStr )
File inputFile = new File(inputFileStr)
println("Input file exists? " + inputFile.exists() )
inputFiles.add(inputFile)
}
return inputFiles
}
#OutputFile
File getTargetFile() {
if ( outputName.length() == 0 ) {
def nameWithPackage = className.get(0)
def name = nameWithPackage.substring(nameWithPackage.lastIndexOf(".")+1)
outputName = 'jni' + name;
}
File outputFile = new File( outputDir + outputName + '.cpp');
println("Output Name is: " + outputFile )
println("Output file exists? " + outputFile.exists() )
return outputFile
}
JavaCPPTask() {
}
#TaskAction
def javacpp() {
def haveFile = new File(runIfExists)
if ( runIfExists.length() > 0 && ! haveFile.exists() ) {
println("Skipping class " + className + " makefile does not exist: " + runIfExists );
}
else {
def useOutputName = []
if ( outputName.length() > 0 ) {
useOutputName = [ '-o', outputName ]
}
def optionArray = [ '-nocompile', '-classpath', localClasspath, '-d', outputDir ]
def allOptions = [javacpp] + useOutputName + optionArray + className.findAll()
project.javaexec { main="-jar"; args allOptions }
}
}
}
def codecOpus = ['com.package.ve.CodecOpus']
task javaCPP_CodecOpus(type:JavaCPPTask) {
className = codecOpus
outputDir = 'jni/VECodecOpus/'
runIfExists = outputDir + 'Android.mk'
}
And the output, which may or may not help. The output file has already been created by a previous run, but the .class file has been updated so I'd expect that the target would run again.
> c:\java\gradle-2.2.1\bin\gradle -b build
JavaCPP.gradle javaCPP_CodecOpus
Incremental java compilation is an incubating feature.
InputFileStr: bin/classes/com/package/ve/CodecOpus.class
Input file exists? true
:javaCPP_CodecOpus
InputFileStr: bin/classes/com/package/ve/CodecOpus.class
Input file exists? true
Output Name is: jni\VECodecOpus\jniCodecOpus.cpp
Output file exists? true
Output Name is: jni\VECodecOpus\jniCodecOpus.cpp
Output file exists? true
Output Name is: jni\VECodecOpus\jniCodecOpus.cpp
Output file exists? true
Output Name is: jni\VECodecOpus\jniCodecOpus.cpp
Output file exists? true
InputFileStr: bin/classes/com/package/ve/CodecOpus.class
Input file exists? true
:javaCPP_CodecOpus UP-TO-DATE
BUILD SUCCESSFUL
What am I doing wrong?

Turns out after debugging through the Gradle source, the script is working. I had been forcing an update and a recompile on the .java file which created a new .class file, but Gradle was smart enough to know that the contents of the class file had not changed. Once I changed the source in the .java file and recompiled, Gradle correctly identified the source as changed and rebuilt the target.
Guess I've been using Make for far too long.
There's still and issue with the script since i was forced to run a clean build before I started down this rabbit hole, but the out of date input file wasn't it.

Related

Incremental build support for output directory considering added files

The Gradle documentation tells this:
Note that if a task has an output directory specified, any files added to that directory since the last time it was executed are ignored and will NOT cause the task to be out of date. This is so unrelated tasks may share an output directory without interfering with each other. If this is not the behaviour you want for some reason, consider using TaskOutputs.upToDateWhen(groovy.lang.Closure)
Question: How does the solution with upToDateWhen look like (so that added files are considered). The main problem is that one has to access the build cache to retrieve the output directory content hash the last time the task ran.
Not sure if I understand the question correctly or why you mention the build cache. I assume you are not aware that the predicates added with upToDateWhen() are considered in addition to any other up-to-date checks like the ones added with TaskOutputs.dir()?
Take the following sample task:
task foo {
def outDir = file('out')
outputs.dir(outDir)
outputs.upToDateWhen { outDir.listFiles().length == 1 }
doLast {
new File(outDir, 'foo.txt') << 'whatever'
}
}
As long as there is only a single file in the output directory (as configured via upToDateWhen) and the file produced by the task (out/foo.txt) isn’t changed after the task has run, the task will be up-to-date. If you change/remove the file created by the task in the output directory or if you add further files to the output directory, then the task will run again.
Updated answer as per the updated question from the comments:
task foo {
def outDir = file('out')
/* sample task action: */
doFirst {
def numOutFiles = new Random().nextInt(5)
for (int i = 1; i <= numOutFiles; i++) {
new File(outDir, "foo${i}.txt") << 'whatever'
}
}
/* up-to-date checking configuration: */
def counterFile = new File(buildDir, 'counterFile.txt')
outputs.dir(outDir)
outputs.upToDateWhen {
counterFile.isFile() \
&& counterFile.text as Integer == countFiles(outDir)
}
doLast {
counterFile.text = countFiles(outDir)
}
}
def countFiles(def dir) {
def result = 0
def files = dir.listFiles()
if (files != null) {
files.each {
result++
if (it.isDirectory()) {
result += countFiles(it)
}
}
}
result
}

Version increment using gradle task

I want to increase the version number of my project from 1.0.0. to 1.0.1 automatically whenever a new build is made through bash command. I only need to increase path number and others i will be increasing manually during manual build.
i want to change
this :
version=1.0.0
to
This:
version=1.0.1
using gradle task.
any help that how can i do this .
Is there any way to update this using regex or using substring function.
Here is an example task:
version='1.0.0' //version we need to change
task increment<<{
def v=buildFile.getText().find(version) //get this build file's text and extract the version value
String minor=v.substring(v.lastIndexOf('.')+1) //get last digit
int m=minor.toInteger()+1 //increment
String major=v.substring(0,v.length()-1) //get the beginning
//println m
String s=buildFile.getText().replaceFirst("version='$version'","version='"+major+m+"'")
//println s
buildFile.setText(s) //replace the build file's text
}
Run this task several times and you should see the version change.
A variant:
version='1.0.0'
task incrementVersion<<{
String minor=version.substring(version.lastIndexOf('.')+1)
int m=minor.toInteger()+1
String major=version.substring(0,version.length()-1)
String s=buildFile.getText().replaceFirst("version='$version'","version='"+major+m+"'")
buildFile.setText(s)
}
Here's a custom task for version bumps in Gradle (Android project):
class Version {
private int major
private int minor
private int patch
private int code
Version(int code, String version) {
this.code = code
def (major, minor, patch) = version.tokenize('.')
this.major = major.toInteger()
this.minor = minor.toInteger()
this.patch = patch.toInteger()
}
#SuppressWarnings("unused")
void bumpMajor() {
major += 1
minor = 0
patch = 0
code += 1
}
#SuppressWarnings("unused")
void bumpMinor() {
minor += 1
patch = 0
code += 1
}
#SuppressWarnings("unused")
void bumpPatch() {
patch += 1
code += 1
}
String getName() { "$major.$minor.$patch" }
int getCode() { code }
}
tasks.addRule("Pattern: bump<TYPE>Version") { String taskName ->
if (taskName.matches("bump(Major|Minor|Patch)Version")) {
task(taskName) {
doLast {
String type = (taskName - 'bump' - 'Version')
println "Bumping ${type.toLowerCase()} version…"
int oldVersionCode = android.defaultConfig.versionCode
String oldVersionName = android.defaultConfig.versionName
version = new Version(oldVersionCode, oldVersionName)
version."bump$type"()
String newVersionName = version.getName()
String newVersionCode = version.getCode()
println "$oldVersionName ($oldVersionCode) → $newVersionName ($newVersionCode)"
def updated = buildFile.getText()
updated = updated.replaceFirst("versionName '$oldVersionName'", "versionName '$newVersionName'")
updated = updated.replaceFirst("versionCode $oldVersionCode", "versionCode $newVersionCode")
buildFile.setText(updated)
}
}
}
}
See this Kanji learning Android app for completeness.
Prerequisites
Following format required (note the single quotes):
android {
defaultConfig {
versionCode 3
versionName '0.3.13'
}
}
Usage
$ ./gradlew bumpPatchVersion
> Task :app:bumpPatchVersion
Bumping patch version…
0.3.13 (3) → 0.3.14 (4)
BUILD SUCCESSFUL in 0s
1 actionable task: 1 executed
$ ./gradlew bumpMinorVersion
> Task :app:bumpMinorVersion
Bumping minor version…
0.3.14 (4) → 0.4.0 (5)
BUILD SUCCESSFUL in 0s
1 actionable task: 1 executed
$ ./gradlew bumpMajorVersion
> Task :app:bumpMajorVersion
Bumping major version…
0.4.0 (5) → 1.0.0 (6)
BUILD SUCCESSFUL in 0s
1 actionable task: 1 executed
You could also use split with a increment-matrix, that could be changed depending on the amount of changes:
def version = '1.0.0'
def incstep = '0.0.1'.split(/\./).collect{it.toInteger()}
def indexedVersionList = version.split(/\./).toList().withIndex()
def updatedVersionList = indexedVersionList.collect{num, idx -> num.toInteger()+incstep[idx]}
def updatedVersion = updatedVersionList.join(".")
This is how I did it with Kotlin DSL (build.gradle.kts):
tasks.create("incrementVersion") {
group = "my tasks"
description = "Increments the version in this build file everywhere it is used."
fun generateVersion(): String {
val updateMode = properties["mode"] ?: "minor" // By default, update the minor
val (oldMajor, oldMinor, oldPatch) = version.split(".").map(String::toInt)
var (newMajor, newMinor, newPatch) = arrayOf(oldMajor, oldMinor, 0)
when (updateMode) {
"major" -> newMajor = (oldMajor + 1).also { newMinor = 0 }
"minor" -> newMinor = oldMinor + 1
else -> newPatch = oldPatch + 1
}
return "$newMajor.$newMinor.$newPatch"
}
doLast {
val newVersion = properties["overrideVersion"] as String? ?: generateVersion()
val oldContent = buildFile.readText()
val newContent = oldContent.replace("""= "$version"""", """= "$newVersion"""")
buildFile.writeText(newContent)
}
}
Usage:
./gradlew incrementVersion [-P[mode=major|minor|patch]|[overrideVersion=x]]
Examples:
./gradlew incrementVersion -Pmode=minor
./gradlew incrementVersion -PoverrideVersion=2.5.11
That is given that you have something like this in your build script:
version = "1.2.3"
... and the patch part of the version is just a number (not containing letters like alpha, rc, etc.).
Below solution will not create an issue evern last number exceed from 9-10 and so on
version='1.0.11.1001'
task incrementrevsion{
def v = version
println v
String minor=v.substring(v.lastIndexOf('.')+1) //get last digit
int m=minor.toInteger()+1 //increment
println m
String major=v.substring(0,v.lastIndexOf(".")); //get the beginning
println major
String s=buildFile.getText().replaceFirst("version='$version'","version='"+major+ "." +m+"'")
//println s
buildFile.setText(s) //replace the build file's text
}
def patch = version.substring(version.lastIndexOf('.') + 1)
def p = patch.toInteger() + 1
def major = version.substring(0, version.length() - p.toString().length())
def s = buildFile.getText().replaceFirst("version = '$version'", "version = '" + major + p + "'")
buildFile.setText(s)
The only difference with Alexiy's answer that line 3 contains m.toString().length() as if minor version > 10, i.e 1.0.12 and you will use that approach it will change it to 1.0.113. We need to calculate the length of minor version instead of chopping off only 1 symbol.
And one more thing, usually the last number is called patch, minor is a middle one :)
My Solution where the version will be set by a Parameter.
version = '4.0.0' // I have whitespaces between the equals-sign
task setVersion << {
group = 'Versioning'
description = "Sets the version to the new number specified with -PnewVersion=\'x.x.x\'"
println version
if(project.hasProperty('newVersion')) {
println 'Set Project to new Version '+newVersion
String s=buildFile.getText().replaceFirst("version = '$version'","version = '"+newVersion+"'")
buildFile.setText(s)
}
}
Call Gradle Task with:
gradle setVersion -PnewVersion='5.1.1'
As for me work this solution.
You need add this code into build.gradle file:
version='1.0.1'
tasks.register("incrementVersion") {
doLast {
def ver = version
println ver
String lastNumber = ver.substring(ver.lastIndexOf('.') + 1)
int increment = lastNumber.toInteger() + 1
println increment
String firstNumber = ver.substring(0, ver.lastIndexOf("."))
println firstNumber
String result = buildFile.getText().replaceFirst("version='$version'","version='" + firstNumber + "." + increment + "'")
buildFile.setText(result)
}
}
I know I am posting this quite late, but the answers mentioned above work well upto '1.0.99'. After which it starts misbehaving.
If any one is still interested, I found a different approach.
task increment {
def v = buildFile.getText().find(version)
def (major, minor, patch) = v.tokenize('.')
int newPatch = patch.toInteger() + 1
String newVersion = major + "." + minor + "." + newPatch
String updatedVersion = buildFile.getText().replaceFirst("version='"+v+"'","version='"+newVersion+"'")
buildFile.setText(updatedVersion)
}
This is an example of same think but with KTS(Kotlin Script).
val newVersion: String? by project
tasks.register("bumpVersion") {
this.doFirst {
println("Old version $version")
val newVersion = takeIf { newVersion.isNullOrBlank() }?.let {
val versionArray = version.toString().split(".")
"${versionArray[0]}.${versionArray[1]}.${versionArray.last().toInt().plus(1)}"
} ?: newVersion
buildFile.readText().apply {
println("Bump to $newVersion")
val content = this.replaceFirst("version = \"$version\"", "version = \"$newVersion\"")
buildFile.writeText(content)
}
}
}
Increment automatic the patch or it is possible to send new version as property.
./gradlew bumpVersion -PnewVersion=0.2.0
Kotlin dsl:
tasks.create("incrementPatch") {
group = "version"
description = "Автоматически поднять патч версию в файле VERSION"
doFirst {
incrementVersion("patch")
}
}
tasks.create("incrementMinor") {
group = "version"
description = "Автоматически поднять минор версию в файле VERSION"
doFirst {
incrementVersion("minor")
}
}
tasks.create("incrementMajor") {
group = "version"
description = "Автоматически поднять мажор версию в файле VERSION"
doFirst {
incrementVersion("major")
}
}
tasks.named("compileKotlin") {
dependsOn(":incrementPatch")
}
fun incrementVersion(updateMode: String){
val versions = file("version").readText().trim()
println("read version = $versions")
val (oldMajor, oldMinor, oldPatch) = versions.substringBefore("-").split(".").map(String::toInt)
var (newMajor, newMinor, newPatch) = arrayOf(oldMajor, oldMinor, 0)
when (updateMode) {
"major" -> newMajor = (oldMajor + 1).also { newMinor = 0 }
"minor" -> newMinor = oldMinor + 1
else -> newPatch = oldPatch + 1
}
val newVersion ="$newMajor.$newMinor.$newPatch-SNAPSHOT"
println("new version = $newVersion")
file("version").writeText(newVersion)
}

Grooviest way in Gradle to list subdirectories matching a certain pattern

In my Gradle script, I would like to include directories matching a certain pattern in my source set. I need to search and scan for all subdirectories and include all subdirectories:
Any second level directories that have Java or Groovy files in them (could be deeply nested inside the package folder structure).
And/or any sub/directories that have a certain name e.g. api or source
The problem is that srcDirs under java sourceSet only accepts directories and not files.
What I've tried so far:
def javaSourceDirs = []
project.rootDir.eachFileRecurse(FileType.DIRECTORIES) { file ->
if ((file.getName().equalsIgnoreCase("api") || file.getName().equalsIgnoreCase("sources")))
javaSourceDirs << file
}
sourceSets {
main {
java {
srcDirs = javaSourceDirs
};
}
}
Just wondering if there's a better/cleaner way using fileTree or something similar?
This should work:
def findAllUnder = { File f, String fileOrDir, String pattern ->
def result = [ ]
def recursiveCall
switch ( fileOrDir ) {
case 'file': recursiveCall = f.&eachFileRecurse
break
case 'dir': recursiveCall = f.&eachDirRecurse
break
default: throw new RuntimeException( "Must be file or dir: $fileOrDir" )
}
recursiveCall { File path -> if ( path.name ==~ pattern ) result << path }
return result
}
def hasSources = { File f -> f.directory && findAllUnder( f, 'file', /.*\.(groovy|java)/ ) }
def sourceDirs = findAllUnder( project.rootDir, 'dir', '(?i)api|sources' ).findAll( hasSources )
It will return a List<File> with all dirs matching the names api or sources (case-insensitive) which have at least one groovy or java file under it.
It does not check for nested dirs, so if you're unlucky and have a dir with sources called groovy/api/com/api/, both groovy/api and groovy/api/com/api will be returned.

Spock to provide temporary files for the Gradle task method within expect: and where: blocks

I would like to use Spock's Data Driven Testing.
My task has a method with a File argument to process. Result is an Array with processed lines. If there is nothing valid to process mentioned array will have 0 size. I used a JUnit rule for creating a temporary folder and I suspect it's the problem.
How can I solve this ?
class H2JSpec extends Specification {
#Rule
TemporaryFolder temporaryFolder
#Shared
private ArrayList<File> tempFiles = []
def "let's build the mappings for template"() {
setup:
Project project = ProjectBuilder.builder().build()
H2JTask task = project.task('h2j', type: H2JTask)
def inputs = ["""#dsfs
""",
"""#this file defines the mapping of h files declarations into java errorcodes: autoenum=off prefix=ec_ class=test.framework.base.MsgErrorCodes
"""]
tempFiles.add(temporaryFolder.newFile('1.txt'))
tempFiles.add(temporaryFolder.newFile('2.txt'))
tempFiles[0].withWriter { it << inputs[0] }
tempFiles[1].withWriter { it << inputs[1] }
expect:
task.prepareCommandList(a).size() == b
where:
a || b
tempFiles[0] || 0
tempFiles[1] || 1
}
}
And the result is the java.io.FileNotFoundException.
To be honest I'd eliminate use of Rule at all and rewrite the test in the following way:
#Grab(group='org.spockframework', module='spock-core', version='1.0-groovy-2.4')
import spock.lang.*
class H2JSpec extends Specification {
def "let's build the mappings for template"() {
setup:
def task = new A()
when:
def f = File.createTempFile('aaa', 'bbb')
f.write(a)
then:
task.prepareCommandList(f).size() == b
where:
a || b
"""#dsfs
""" || 3
"""#this file defines the mapping of h files declarations into java errorcodes: autoenum=off prefix=ec_ class=test.framework.base.MsgErrorCodes
""" || 2
}
}
class A {
List<String> prepareCommandList(File f) {
f.readLines()
}
}

How do I concatenate multiple files in Gradle?

Is there an easy way to concatenate multiple text files into a single one in Gradle? The build script should look something like this:
FileCollection jsDeps = files(
'file1.js',
'file2.js'
// other files here
)
task concatenate << {
// concatenate the files to file.js
}
I am using Gradle 2.3.
leftShift / "<<" is deprecated in gradle 3.4 You may use something like:
task concatenate {
doLast {
def toConcatenate = files("filename1", "filename2", ...)
def outputFileName = "output.txt"
def output = new File(outputFileName)
output.write('') // truncate output if needed
toConcatenate.each { f -> output << f.text }
}
You can also register the files as inputs/outputs to help with incremental builds. It's especially helpful with larger files.
something like this:
task 'concatenateFiles', {
inputs.files( fileTree( "path/to/dir/with/files" ) ).skipWhenEmpty()
outputs.file( "$project.buildDir/tmp/concatinated.js" )
doLast {
outputs.files.singleFile.withOutputStream { out ->
for ( file in inputs.files ) file.withInputStream { out << it << '\n' }
}
}
}
Instead of the fileTree, it can also be replaced with sourceset/sourceset output, specific files, outputs from a different task, etc.
Gradle doc on task inputs/output
Concatenating files in groovy
The following task should do the job:
task concatenate << {
def toConcatenate = files('f1', 'f2', 'f3')
def output = new File('output')
toConcatenate.each { f -> output << f.text }
}
(new File('test.js')).text = file('test1.js').getText() + file('test2.js').getText()
UPDATE:
For collections.
(new File('test.js')).text = files('test1.js', 'test2.js').collect{it.getText()}.join("\n")

Resources