I'm trying to get around a problem where a dependency in my build is a zip file that contains some read only files. When I extract that zip as part of my build I end out with read only files in a staging folder and they prevent the task running in the future since they cannot be overwritten.
Until there's a way to force overwrite in a gradle copy task I've been trying to find a way to change the file mode of the read-only files in a way that doesn't remove the execute bit from those files that need it.
I've come up with this:
task stageZip(type: Copy) {
from({ zipTree(zipFile) })
into stagingFolder
eachFile {
println "${it.name}, oldMode: ${Integer.toOctalString(it.mode)}, newMode: ${Integer.toOctalString(it.mode | 0200)}"
fileMode it.mode | 0200
}
}
But this doesn't work. If I comment out the fileMode line then the println correctly lists the old and new file modes with the write bit enabled for all files. If I leave the code as is, then all the files in the zip get extracted with the newMode of the first file.
This doesn't seem like this is an unreasonable thing to try and do, but I'm obviously doing something wrong. Any suggestions?
Based on this thread, consider the Sync task. Specifically:
task stageZip(type: Sync) {
from zipTree('data/data.zip')
into 'staging'
fileMode 0644
}
I've put a working example (as I understand the question) here.
Here is a method that answers the question about file permissions. The example is posted to GitHub here.
First, consider a method to add w to a file:
import java.nio.file.*
import java.nio.file.attribute.PosixFilePermission
def addWritePerm = { file ->
println "TRACER adding 'w' to : " + file.absolutePath
def path = Paths.get(file.absolutePath)
def perms = Files.getPosixFilePermissions(path)
perms << PosixFilePermission.OWNER_WRITE
Files.setPosixFilePermissions(path, perms)
}
then, the Gradle task could be as follows:
project.ext.stagingFolder = 'staging'
project.ext.zipFile = 'data/data.zip'
task stageZip(type: Copy) {
from({ zipTree(project.ext.zipFile) })
into project.ext.stagingFolder
doLast {
new File(project.ext.stagingFolder).eachFileRecurse { def file ->
if (! file.canWrite()) {
addWritePerm(file)
}
}
}
}
eachFile {
file -> file.setMode(file.getMode() | 0200)
}
Worked for me in an rpm task which works with copyspec
Related
I have the following simple task in my build:
task generateFile << {
def file = new File("$buildDir/setclasspath.sh")
file.text = "sample"
outputs.file(file)
}
task createDistro(type: Zip, dependsOn: ['copyDependencies','packageEnvironments','jar', 'generateFile']) <<{
from generateClasspathScript {
fileMode = 0755
into 'bin'
}
}
When I run gradle clean build I see the following output:
Cannot call TaskOutputs.file(Object) on task ':generateFile' after task has started execution. Check the configuration of task ':generateFile' as you may have misused '<<' at task declaration
How do I declare the task file creation outputs as an input to the zip task while also ensuring they happen in the execution phase?
If I leave off the << then the clean task wipes the generated file before the ZIP can use it. If I keep them, I get the above error.
It's the opposite as what is being suggested in the comments. You are trying to set the outputs in execution phase. The correct way to do what you are probably trying to do is for example:
task generateFile {
def file = new File("$buildDir/setclasspath.sh")
outputs.file(file)
doLast {
file.text = "sample"
}
}
I want to make my gradle build inteligent when building my model.
To acquire this I was planning to read schema files, acquire what is included and then build firstly included models (if they are not present).
I'm pretty new to Groovy and Gradle, so please that into account.
What I have:
build.gradle file on root directory, including n subdirectories (subprojects added to settings.gradle). I have only one gradle build file, because I defined tasks like:
subprojects {
task init
task includeDependencies(type: checkDependencies)
task build
task dist
(...)
}
I will return to checkDependencies shortly.
Schema files located externally, which I can see.
Each of them have from 0 to 3 lines of code, that say about dependencies and looks like that:
#include "ModelDir/ModelName.idl"
In my build.gradle I created task that should open, and read those dependencies, preferably return them:
class parsingIDL extends DefaultTask{
String idlFileName="*def file name*"
def regex = ~/#include .*\/(\w*).idl/
#Task Action
def checkDependencies(){
File idlFile= new File(idlFileName)
if(!idlFile.exists()){
logger.error("File not found)
} else {
idlFile.eachLine{ line ->
def dep = []
def matcher = regex.matcher(line)
(...)*
}
}
}
}
What should I have in (...)* to find all dependencies and how should I define, that for example
subprojectA::build.dependsOn([subprojectB::dist, subprojectC::dist])?
All I could find on internet created dep, that outputted given:
[]
[]
[modelName]
[]
[]
(...)
I am developing an Android application where I have a directory of JSON files and I want to create a gradle task that will combine all these files into a single JSON file.
This is the gradle task I have so far but does not create the file:
// Task that combines all JSON files in ../libraries into src/main/res/raw/libraries.json
task combineJSonFiles {
String content = ""
FileTree tree = fileTree(dir: '../libraries', include: '**/*.json')
tree.each {File file ->
content += file.getText()
}
println "[" + content.substring(0, content.length()-1) + "]" // prints out the correct contents
File libraries = file("../app/src/main/res/raw/libraries.json")
println libraries.getProperties()
}
I print out the properties and I am not sure why these are the property values:
{directory=false, canonicalFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, file=false, freeSpace=0, canonicalPath=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, usableSpace=0, hidden=false, totalSpace=0, path=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, name=libraries.json, prefixLength=1, absolute=true, class=class java.io.File, parentFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw, absolutePath=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, absoluteFile=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw/libraries.json, parent=/Users/michaelcarrano/AndroidStudioProjects/detective_droid/app/src/main/res/raw}
Any help is appreciated as I have not seemed to figure this out even after reading the documentation. http://www.gradle.org/docs/current/userguide/working_with_files.html
I am just posting the code for the task that now works:
task combineJSonFiles {
String content = ""
FileTree tree = fileTree(dir: '../libraries', include: '**/*.json')
tree.each {File file ->
content += file.getText()
}
def libraries = new File("app/src/main/res/raw/libraries.json")
libraries.text = "[" + content.substring(0, content.length()-1) + "]"
}
My issue was trying to use Java.io.File and having the wrong directory path set for my file.
Creating an instance of java.io.File in Groovy/Java does not create the file on disk. You will need to write something to it. Check out this tutorial for working with files in Groovy.
Also you have put your task implementation in a task configuration block, rather than a task action. This means your code will not be running when you are expecting - it will run every time you run gradle rather than when you run this task. You need to put your code in a doLast block
Can gradle alter the structure of the tree while copying?
original
mod/a/src
mod/b/src
desired
dest/mod-a/source
dest/mod-b/source
dest/mod-c/source
I'm not sure where I should create a closure and override the copy tree logic
I'd like to do the gradle equivalent of ant's globmapper functionality
<property name="from.dir" location=".."/>
<property name="to.dir" location="dbutil"/>
<copy>
<fileset dir="${from.dir}" ... />
<globmapper from="${from.dir}/*/db" to="${to.dir}"/>
</copy>
Thanks
Peter
When changing file name, rename seems a good approach. When changing path you can override eachFile and modify the destination path.
This works pretty well.
copy {
from("${sourceDir}") {
include 'modules/**/**'
}
into(destDir)
eachFile {details ->
// Top Level Modules
def targetPath = rawPathToModulesPath(details.path)
details.path = targetPath
}
}
....
def rawPathToModulesPath(def path) {
// Standard case modules/name/src -> module-name/src
def modified=path.replaceAll('modules/([^/]+)/.*src/(java/)?(.*)', {"module-${it[1]}/src/${it[3]}"})
return modified
}
Please see sample below. Gradle 4.3 does not have rename/move methods, so we can do renaming on the fly.
What was happened:
Load file tree into the memory. I used zip file from dependencies in my example
Filter items, which are in the target folder
All result items will have the same prefix: if we filter files from directory "A/B/C/", then all files will be like "A/B/C/file.txt" or "A/B/C/D/file.txt". E.g. all of them will start with the same words
In the last statement eachFile we will change final name by cutting the directory prefix (e.g. we will cut "A/B/C").
Important: use type of task "Copy", which has optimizations for incremental compilation. Gradle will not do file copy if all of items below are true:
Input is the same (for my case - all dependencies of scope "nativeDependenciesScope") with previous build
Your function returned the same items with the previous build
Destination folder has the same file hashes, with the previous build
task copyNativeDependencies(type: Copy) {
includeEmptyDirs = false
def subfolderToUse = "win32Subfolder"
def nativePack = configurations.nativeDependenciesScope.singleFile // result - single dependency file
def nativeFiles = zipTree(nativePack).matching { include subfolderToUse + "/*" } // result - filtered file tree
from nativeFiles
into 'build/native_libs'
eachFile {
print(it.path)
// we filtered this folder above, e.g. all files will start from the same folder name
it.path = it.path.replaceFirst("$subfolderToUse/", "")
}
}
// and don't forget to link this task for something mandatory
test.dependsOn(copyNativeDependencies)
run.dependsOn(copyNativeDependencies)
The following works, but is there a more gradle-ish way to do this?
ant.copy(todir: destDir) {
fileset( dir: "${srcDir}/module", includes: '**/src/**')
regexpmapper(from: '^(.*)/src/(.*)$', to: /module-\1\/src\/\2/)
}
I am writing a Gradle build for a non-java project for assembling existing directories and tar archives into a .tar.gz The tar task skips if I use the definition like so:
task archive(dependsOn: 'initArchive',type: Tar) << {
baseName = project.Name
destinationDir = new File(project.buildDir.path+'/installer')
compression = Compression.GZIP
from (archiveDir)
doLast{
checksum(archivePath)
}
}
here's the console output
:jenkins-maven-sonar:archive
Skipping task ':jenkins-maven-sonar:archive' as it has no source files.
:jenkins-maven-sonar:archive UP-TO-DATE
BUILD SUCCESSFUL
Total time: 9.056 secs
When I try to use tar task as a method it fails complaining cannot find method
task archive(dependsOn: 'initArchive') << {
tar{
baseName = project.Name
destinationDir = new File(project.buildDir.path+'/installer')
compression = Compression.GZIP
from (archiveDir)
doLast{
checksum(archivePath)
}
}
}
FAILURE: Build failed with an exception.
* Where:
Build file '/home/anadi/Code/da-ci-installers/build.gradle' line: 29
* What went wrong:
Execution failed for task ':jenkins-maven-sonar:archive'.
> Could not find method tar() for arguments [build_6a2bckppv2tk8qodr6lkg5tqft$_run_closure3_closure5_closure7#4a5f634c] on task ':jenkins-maven-sonar:archive'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.
BUILD FAILED
Total time: 8.749 secs
Can we run the tar task in same way as Gradle allows running copy? In the same build I have a block like follows and I want to know if tar can used in the same way
copy {
project.logger.info("Copying bundle :: "+bundle[x])
from(rootProject.projectDir.path+"/3rd-party-tools/"+bundle[x]) {
include '**/*.*'
}
into(archiveDir)
}
if not how to make sure my build does not "skip tar" task if using the first form described above.
You have fallen for the classical mistake of configuring a task in the execution phase rather than the configuration phase. The solution is to remove the << in the first code snippet.
If you find the << (and the difference it makes) confusing, a good solution is to never use << but always the more explicit doLast {}.
There is no tar method, but it's usually better to make these things a separate task anyway. (Methods like copy should only be preferred over the corresponding task if there is a strong reason.)
I had a funny situation where I got hit by this when using doLast{} on a tar task.
It was because of a multi-project build:
build.gradle
--> sub-project
--> build.gradle
In this case if you try to have a tar or a copy task in the main build file that references something from that project(":sub-project") uses it will tempt the developer to wrap it in doLast.
For example, main build.gradle file has:
task distTar(type: Tar, dependsOn: "buildDist") {
description "Package ProjName into a Tar file"
group "Build"
baseName = 'outbasename'
archiveName = baseName + '.tar.gz'
compression = Compression.GZIP
destinationDir = file(project(":packaging").buildDir.path)
extension = 'tar.gz'
into('outdir') {
from project(":sub-project").war
}
}
So they got an error that project(":sub-project").war doesn't exist. So to get around it someone put doLast {} the task and errors went away. BAD!!
task distTar(type: Tar, dependsOn: "buildDist") {
doLast { // <-- BAD!!
description "Package ProjName into a Tar file"
group "Build"
baseName = 'outbasename'
archiveName = baseName + '.tar.gz'
compression = Compression.GZIP
destinationDir = file(project(":packaging").buildDir.path)
extension = 'tar.gz'
into('outdir') {
from project(":sub-project").war
}
}
}
Then i was left to fix it. So the correct was to do it was to add
evaluationDependsOn ":sub-project"
In the main build.gradle file. Now it knows to evaluate it. I remove the incorrect doLast{} block and now the task is no longer ignored.