copy tree with gradle and change structure? - gradle

Can gradle alter the structure of the tree while copying?
original
mod/a/src
mod/b/src
desired
dest/mod-a/source
dest/mod-b/source
dest/mod-c/source
I'm not sure where I should create a closure and override the copy tree logic
I'd like to do the gradle equivalent of ant's globmapper functionality
<property name="from.dir" location=".."/>
<property name="to.dir" location="dbutil"/>
<copy>
<fileset dir="${from.dir}" ... />
<globmapper from="${from.dir}/*/db" to="${to.dir}"/>
</copy>
Thanks
Peter

When changing file name, rename seems a good approach. When changing path you can override eachFile and modify the destination path.
This works pretty well.
copy {
from("${sourceDir}") {
include 'modules/**/**'
}
into(destDir)
eachFile {details ->
// Top Level Modules
def targetPath = rawPathToModulesPath(details.path)
details.path = targetPath
}
}
....
def rawPathToModulesPath(def path) {
// Standard case modules/name/src -> module-name/src
def modified=path.replaceAll('modules/([^/]+)/.*src/(java/)?(.*)', {"module-${it[1]}/src/${it[3]}"})
return modified
}

Please see sample below. Gradle 4.3 does not have rename/move methods, so we can do renaming on the fly.
What was happened:
Load file tree into the memory. I used zip file from dependencies in my example
Filter items, which are in the target folder
All result items will have the same prefix: if we filter files from directory "A/B/C/", then all files will be like "A/B/C/file.txt" or "A/B/C/D/file.txt". E.g. all of them will start with the same words
In the last statement eachFile we will change final name by cutting the directory prefix (e.g. we will cut "A/B/C").
Important: use type of task "Copy", which has optimizations for incremental compilation. Gradle will not do file copy if all of items below are true:
Input is the same (for my case - all dependencies of scope "nativeDependenciesScope") with previous build
Your function returned the same items with the previous build
Destination folder has the same file hashes, with the previous build
task copyNativeDependencies(type: Copy) {
includeEmptyDirs = false
def subfolderToUse = "win32Subfolder"
def nativePack = configurations.nativeDependenciesScope.singleFile // result - single dependency file
def nativeFiles = zipTree(nativePack).matching { include subfolderToUse + "/*" } // result - filtered file tree
from nativeFiles
into 'build/native_libs'
eachFile {
print(it.path)
// we filtered this folder above, e.g. all files will start from the same folder name
it.path = it.path.replaceFirst("$subfolderToUse/", "")
}
}
// and don't forget to link this task for something mandatory
test.dependsOn(copyNativeDependencies)
run.dependsOn(copyNativeDependencies)

The following works, but is there a more gradle-ish way to do this?
ant.copy(todir: destDir) {
fileset( dir: "${srcDir}/module", includes: '**/src/**')
regexpmapper(from: '^(.*)/src/(.*)$', to: /module-\1\/src\/\2/)
}

Related

Gradle task to create a zip archive

I need to create a gradle task to create a zip where I need to include all the json files that contain text "type":"customer" how can i include the check for containing text ?
is there a way I can do it:
task createZip(type: Zip) {
archiveName = "customer.zip"
destinationDir = file(testDir)
from(files(customer))
}
You could use a FileTree together with Gradle/Groovy filtering capabilities:
Let's say you have your source Customer JSON files under src/customers: you can define a Task like that:
task createZip(type: Zip) {
archiveName = "customer.zip"
destinationDir = buildDir
from fileTree('src/customers').filter { f -> f.text.contains('"type": "customer"')}
}
Note that this uses Groovy's File.getText() method to read the whole file content into memory to match the "type": "customer" expression. Be careful with performances if you have plenty or huge files to filter.

Setting file permissions dynamically in a gradle Copy task

I'm trying to get around a problem where a dependency in my build is a zip file that contains some read only files. When I extract that zip as part of my build I end out with read only files in a staging folder and they prevent the task running in the future since they cannot be overwritten.
Until there's a way to force overwrite in a gradle copy task I've been trying to find a way to change the file mode of the read-only files in a way that doesn't remove the execute bit from those files that need it.
I've come up with this:
task stageZip(type: Copy) {
from({ zipTree(zipFile) })
into stagingFolder
eachFile {
println "${it.name}, oldMode: ${Integer.toOctalString(it.mode)}, newMode: ${Integer.toOctalString(it.mode | 0200)}"
fileMode it.mode | 0200
}
}
But this doesn't work. If I comment out the fileMode line then the println correctly lists the old and new file modes with the write bit enabled for all files. If I leave the code as is, then all the files in the zip get extracted with the newMode of the first file.
This doesn't seem like this is an unreasonable thing to try and do, but I'm obviously doing something wrong. Any suggestions?
Based on this thread, consider the Sync task. Specifically:
task stageZip(type: Sync) {
from zipTree('data/data.zip')
into 'staging'
fileMode 0644
}
I've put a working example (as I understand the question) here.
Here is a method that answers the question about file permissions. The example is posted to GitHub here.
First, consider a method to add w to a file:
import java.nio.file.*
import java.nio.file.attribute.PosixFilePermission
def addWritePerm = { file ->
println "TRACER adding 'w' to : " + file.absolutePath
def path = Paths.get(file.absolutePath)
def perms = Files.getPosixFilePermissions(path)
perms << PosixFilePermission.OWNER_WRITE
Files.setPosixFilePermissions(path, perms)
}
then, the Gradle task could be as follows:
project.ext.stagingFolder = 'staging'
project.ext.zipFile = 'data/data.zip'
task stageZip(type: Copy) {
from({ zipTree(project.ext.zipFile) })
into project.ext.stagingFolder
doLast {
new File(project.ext.stagingFolder).eachFileRecurse { def file ->
if (! file.canWrite()) {
addWritePerm(file)
}
}
}
}
eachFile {
file -> file.setMode(file.getMode() | 0200)
}
Worked for me in an rpm task which works with copyspec

Calling the same task multiple times in a single build.gradle file

I have a custom Gradle plugin that will generate Java files from a template file. I have several such template files in different locations, and I need to "compile" all of them to generate the Java files I need. Once I have the files, I want to package them into a .jar.
One way I thought I could do this was to call the "compile template" task multiple times from within the same build file. I'd call it once in a task that compiles template files in location A, again from a task that compiles template files from location B... etc., until I have all the Java files I need.
Something like this:
task compileFromLocationA <<{
compileTemplate.execute(A)...
}
task compileFromLocationB
compileTemplate.execute(B)...
...
packageJar(depends: compileFromLocationA, compileFromLocationB, ...)
...
However, you can't programmatically call a task from within another task. I suppose I could break each compileFromLocation_ task into it's own build.gradle file, but that seems like overkill. What's the "best practice" in a case like this?
This code seems to work in build.gradle by using tasks.register() - e.g. to perform multiple source code generating steps - in my case I needed to load different pairs of files (XML schema and generation options) in two different steps:
plugins {
id 'java'
id "com.plugin" version "1.0"
}
sourceSets.main.java.srcDirs += file("${buildDir}/genSrc")
sourceSets.test.java.srcDirs += file("${buildDir}/testGenSrc")
tasks.compileJava {
dependsOn tasks.named("genMessage")
}
genMessage {
codesFile = "${projectDir}/src/main/resources/file.xml"
}
def testGenModel1 = tasks.register("testGenModel1", com.plugin.TestGenModelTask.class) {
schema = "${projectDir}/src/test/resources/file.xsd"
options = "${projectDir}/src/test/resources/file.xml"
}
def testGenModel2 = tasks.register("testGenModel2", com.plugin.TestGenModelTask.class) {
schema = "${projectDir}/src/test/resources/file2.xsd"
options = "${projectDir}/src/test/resources/file2.xml"
}
tasks.compileTestJava {
dependsOn tasks.named("testGenModel1"), tasks.named("testGenModel2")
}

Reading includes from idl file in custom task

I want to make my gradle build inteligent when building my model.
To acquire this I was planning to read schema files, acquire what is included and then build firstly included models (if they are not present).
I'm pretty new to Groovy and Gradle, so please that into account.
What I have:
build.gradle file on root directory, including n subdirectories (subprojects added to settings.gradle). I have only one gradle build file, because I defined tasks like:
subprojects {
task init
task includeDependencies(type: checkDependencies)
task build
task dist
(...)
}
I will return to checkDependencies shortly.
Schema files located externally, which I can see.
Each of them have from 0 to 3 lines of code, that say about dependencies and looks like that:
#include "ModelDir/ModelName.idl"
In my build.gradle I created task that should open, and read those dependencies, preferably return them:
class parsingIDL extends DefaultTask{
String idlFileName="*def file name*"
def regex = ~/#include .*\/(\w*).idl/
#Task Action
def checkDependencies(){
File idlFile= new File(idlFileName)
if(!idlFile.exists()){
logger.error("File not found)
} else {
idlFile.eachLine{ line ->
def dep = []
def matcher = regex.matcher(line)
(...)*
}
}
}
}
What should I have in (...)* to find all dependencies and how should I define, that for example
subprojectA::build.dependsOn([subprojectB::dist, subprojectC::dist])?
All I could find on internet created dep, that outputted given:
[]
[]
[modelName]
[]
[]
(...)

Issue with Gradle task to write file

I am developing an Android application where I have a directory of JSON files and I want to create a gradle task that will combine all these files into a single JSON file.
This is the gradle task I have so far but does not create the file:
// Task that combines all JSON files in ../libraries into src/main/res/raw/libraries.json
task combineJSonFiles {
String content = ""
FileTree tree = fileTree(dir: '../libraries', include: '**/*.json')
tree.each {File file ->
content += file.getText()
}
println "[" + content.substring(0, content.length()-1) + "]" // prints out the correct contents
File libraries = file("../app/src/main/res/raw/libraries.json")
println libraries.getProperties()
}
I print out the properties and I am not sure why these are the property values:
{directory=false, canonicalFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, file=false, freeSpace=0, canonicalPath=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, usableSpace=0, hidden=false, totalSpace=0, path=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, name=libraries.json, prefixLength=1, absolute=true, class=class java.io.File, parentFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw, absolutePath=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, absoluteFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, parent=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw}
Any help is appreciated as I have not seemed to figure this out even after reading the documentation. http://www.gradle.org/docs/current/userguide/working_with_files.html
I am just posting the code for the task that now works:
task combineJSonFiles {
String content = ""
FileTree tree = fileTree(dir: '../libraries', include: '**/*.json')
tree.each {File file ->
content += file.getText()
}
def libraries = new File("app/src/main/res/raw/libraries.json")
libraries.text = "[" + content.substring(0, content.length()-1) + "]"
}
My issue was trying to use Java.io.File and having the wrong directory path set for my file.
Creating an instance of java.io.File in Groovy/Java does not create the file on disk. You will need to write something to it. Check out this tutorial for working with files in Groovy.
Also you have put your task implementation in a task configuration block, rather than a task action. This means your code will not be running when you are expecting - it will run every time you run gradle rather than when you run this task. You need to put your code in a doLast block

Resources