How to copy files into flat directory in Gradle - gradle

I'm trying to write a Gradle task to copy specific files from a deep tree into a flat folder.
First Try:
task exportProperties << {
copy {
from "."
into "c:/temp/properties"
include "**/src/main/resources/i18n/*.properties"
}
}
This copies the correct files, but does not flatten the structure, so I end up with every single folder from my original project, and most of them are empty.
Second try, based on answers I saw here and here:
task exportProperties << {
copy {
from fileTree(".").files
into "c:/temp/properties"
include "**/src/main/resources/i18n/*.properties"
}
}
This time, it is not copying anything.
Third Try:
task exportProperties << {
copy {
from fileTree(".").files
into "c:/temp/properties"
include "*.properties"
}
}
Almost works, except it is copying every *.properties file when I only want the files in particular paths.

I solved the issue in a way similar to this:
task exportProperties << {
copy {
into "c:/temp/properties"
include "**/src/main/resources/i18n/*.properties"
// Flatten the hierarchy by setting the path
// of all files to their respective basename
eachFile {
path = name
}
// Flattening the hierarchy leaves empty directories,
// do not copy those
includeEmptyDirs = false
}
}

I got it to work like this:
task exportProperties << {
copy {
from fileTree(".").include("**/src/main/resources/i18n/*.properties").files
into "c:/temp/properties"
}
}

You can modify a number of aspects of copied files on the fly by feeding a closure into the Copy.eachFile method including target file path:
copy {
from 'source_dir'
into 'dest_dir'
eachFile { details ->
details.setRelativePath new RelativePath(true, details.name)
}
}
This copies all files directly into the specified destination directory, though it also replicates the original directory structure without the files.

I was able to solve this in a similar way to Kip but inverted:
distributions {
main {
// other distribution configurations here...
contents {
into('config') {
exclude(['server.crt', 'spotbugs-exclusion-filters.xml'])
from fileTree('src/main/resources').files
}
}
}
}
There are no issues with empty directories when the CopySpec is configured this way.

Related

Programmatically create a file in Gradle build script

I'm sure it is trivial, but I cannot find a way to do it...
In my build.gradle I want processResources task to create (not e.g. copy or fill some tempate) a resource file to be loaded by Java program.
I achieved the following:
processResources {
...
// This is a collection of files I want to copy into resources.
def extra = configurations.extra.filter { file -> file.isFile () }
// This actually copies them to 'classes/extra'. It works.
into ('extra') {
from extra
}
doLast {
// I want to write this string (list of filenames, one per
// line) to 'classes/extra/list.txt'.
println extra.files.collect { file -> file.name }.join ("\n")
}
}
You can see above a println that prints exactly what I need. But how do I write this string to a file instead of the console?
You can use the following code
task writeToFile {
// sample list.(you already have it as extra.files.collect { file -> file.name })
List<String> sample = [ 'line1','line2','line3' ] as String[]
// create the folders if it does not exist.(otherwise it will throw exception)
File extraFolder = new File( "${project.buildDir}/classes/extra")
if( !extraFolder.exists() ) {
extraFolder.mkdirs()
}
// create the file and write text to it.
new File("${project.buildDir}/classes/extra/list.txt").text = sample.join ("\n")
}
One way to implement this would be to define a custom task that will generate this "index" file from the extra configuration, and make the existing processResources task depend on this custom task.
Something like that would work:
// Task that creates the index file which lists all extra libs
task createExtraFilesIndex(){
// destination directory for the index file
def indexFileDir = "$buildDir/resources/main"
// index filename
def indexFileName = "extra-libs.index"
doLast{
file(indexFileDir).mkdirs()
def extraFiles = configurations.extra.filter { file -> file.isFile () }
// Groovy concise syntax for writing into file; maybe you want to delete this file first.
file( "$indexFileDir/$indexFileName") << extraFiles.files.collect { file -> file.name }.join ("\n")
}
}
// make processResources depends on createExtraFilesIndex task
processResources.dependsOn createExtraFilesIndex
def dirA = "${rootDir}/dirA" //creates dirA directory in root project directory
new File("${dirA}/yourFile.txt").append("\n") //creates yourFile.txt file in above created directory

How can I iterate over CopySpec in gradle, or even get the destination pathname of a file from the copyspec

I have a Zip task which uses the CopySpec like DSL.Like:-
task1 {
from(src/main/pig).exclude(<some file pattern>) {
into "pigscripts"
}
}
What I want to do is iterate over the pig scripts, as in if copySpec is the CopySpec object associated with it,
Map<String,String map> = new HashMap<String,String>()
copySpec.eachfile {
map.put(it.originalpath,it.destinationpath)
}
However in this gradle discussion forum https://discuss.gradle.org/t/iterate-over-a-copyspesc/13030/6 it seems this is not straight forward.
It will also suffice if I can get the destination path of a single file since I can get a FileTree of all the source files as
FileTree filetree = (copySpec as CopySpecInternal).buildRootResolver().allSource
then i would like to do something like
filetree.each{
//map.put(it,copySpec.findDestinationPath(it))
}

Gradle copy and filter task not executed

We are filtering an xml file replacing some tokens with gradle properties.
But the filtering (i.e. copy task) is not executed when we just change the properties in our build.gradle file.
How should we modify our script so that the filtering is performed each time or at least when the template and/or the build.gradle has been modified.
This we have:
war.doFirst {
delete 'src/main/webapp/WEB-INF/appengine-web.xml'
copy {
from 'build.gradle'
from 'src/main/webapp/WEB-INF/'
into 'src/main/webapp/WEB-INF/'
include '*-template.*'
rename { String fileName ->
fileName.replace('-template', '')
}
expand(gaeApp: "$gaeApp", gaeAppVersion: "$gaeAppVersion")
}
}
I just ran some test where the filtering worked. I am confused... I am sure that it sometimes does not!
So after good input from Vampire we tried this
war {
inputs.file "build.gradle"
exclude 'src/main/webapp/WEB-INF/appengine-web.xml'
// filesMatching('src/main/webapp/WEB-INF/**/*-template.*') {
filesMatching('**/*-template.*') {
println "WAR template: $it"
rename { it.replace '-template', '' }
expand gaeApp: gaeApp, gaeAppVersion: gaeAppVersion
}
}
A dollar and a dime to anyone who can explain why the filesMatching('src/main/webapp/WEB-INF/**/*-template.*')does not work!
BUT the biggest problem is that even if the filesMatching locates the template file the appengine-web.xml that is placed inside the WAR is not a processed version of appengine-web-template.xml.
You need to add those properties to the inputs of the task like
war.inputs.property 'gaeApp', gaeApp
war.inputs.property 'gaeAppVersion', gaeAppVersion
so that gradle knows the input changed, otherwise it cannot know when the input is different.
But besides that, you should not (need not) use a copy { } block in there.
The war task itself is an implicit copy spec, so you should be able just doing something like
war {
inputs.property 'gaeApp', gaeApp
inputs.property 'gaeAppVersion', gaeAppVersion
exclude 'src/main/webapp/WEB-INF/appengine-web.xml'
filesMatching('src/main/webapp/WEB-INF/**/*-template.*') {
rename { it.replace '-template', '' }
expand gaeApp: gaeApp, gaeAppVersion: gaeAppVersion
}
}
This is what worked for us in the end.
We moved the template to 'src/template/webapp' and removed the "-template" suffix,
war {
inputs.file "build.gradle"
with copySpec {
from 'src/template/webapp'
expand gaeApp: gaeApp, gaeAppVersion: gaeAppVersion
}
}
Our problem with Vampire's solution must be related to the fact that the template file was in same directory as the file it was to replace.

Gradle distribution - create an empty directory

Is there a way to add empty directories (e.g, "logs") when creating a distribution with the gradle distribution plugin?
I saw this JIRA, describing the exact same thing. It is still open https://issues.gradle.org/browse/GRADLE-1671
I wonder if there are any workarounds I can use. I don't quite understand the workarounds described in the jira.
Thank you.
So I managed to work around this by following the suggestion in the mentioned JIRA to create a dummy empty directory and then copy it to the distribution location.
It's ugly but works. I'm sure it can be written more efficiently though. This is the Copy block from inside distributions/main/contents:
into('') {
//create an empty 'logs' directory in distribution root
def logDirBase = new File('/tmp/app-dummy-dir')
logDirBase.mkdirs()
def logDir = new File(logDirBase.absolutePath + '/logs')
logDir.mkdirs()
from {logDirBase}
}
Based on Logato's own answer I've come up with the following code, which is more elegant and also closes the file pointer correctly (using the with context):
distributions {
main {
baseName = 'app'
contents {
into('') {
File.createTempDir().with {
def tmpLog = new File(absolutePath, 'logs')
println tmpLog.absolutePath
tmpLog.mkdirs()
from (absolutePath) {
includeEmptyDirs = true
}
}
// ...
}
// ...
}
}
}
This seems odd answering this so late. But, there are 2 issues here.
We should really avoid creating empty directories. But, if we must...we must.
The previous examples attempt to create empty directories outside of the current project, which seems to go against the goal of most builds. We can avoid this and work more naturally within gradle by adding a custom task.
plugins {
id 'java'
id 'distribution'
}
group 'org.example'
version '1.0-SNAPSHOT'
task createEmptyLogDir() {
ext {
buildDir = layout.buildDirectory.dir('empty_dirs')
}
doFirst {
File.createTempDir().with {
def dir = new File(buildDir.get().asFile, 'logs')
dir.mkdirs()
}
}
outputs.dir(buildDir)
}
tasks.assembleDist.configure {
dependsOn('createEmptyLogDir')
}
distributions {
main {
distributionBaseName = 'app'
contents {
into('lib') {
from tasks.jar
}
from tasks.createEmptyLogDir {
includeEmptyDirs = true
}
}
}
}
This has the advantage of building within the build directory, using task inputs/outputs for up-to-date checks, and cleaning up.
An important note is that we cannot just create the distribution with empty directories, alone. This will be seen as no source and up-to-date. So, I added the jar in this example. Tested with gradle 7.1.

gradle tar include nested files at top level

In gradle (1.9), I have multiple subprojects. Each one uses the application plugin to create a tar and cli. I am trying to get all these tars into a unified tar, but I am having a lot of trouble.
Here is the tar format I am looking for:
${project.name}/${subproject.name}.tar
I have tried using both the Tar task and the distribution plugin, but for each one, I am not able to find a clean way to just get the generated tars (or any tar), and put them at top level, excluding everything else.
Here is a sample using the distirbution pluging, but its not giving the output I like
apply plugin: 'distribution'
distributions {
testing {
contents {
from(".")
exclude "*src*"
exclude "*idea*"
exclude "*.jar"
exclude ".MF"
filesMatching("**/build/distributions/*.tar") {
if(file.name == "${project.name}-testing.tar") {
exclude()
} else {
name file.name
}
}
}
}
}
Here is what I would like (but not working):
apply plugin: 'distribution'
distributions {
testing {
contents {
include "**/*.tar" // shows up at top level
}
}
}
EDIT:
Getting closer.
distributions {
testing {
contents {
from subprojects.buildDir
includeEmptyDirs false
include "**/*.tar"
exclude "**/${project.name}-testing.tar"
}
}
}
This will give me ${project.name}/distribution/${subproject.name}.tar
Here is the solution for your problem. Put the following to the root project:
task distTar(type: Tar) {
destinationDir = new File("$buildDir/distributions")
baseName = 'unifiedTar'
}
subprojects {
// definitions common to subprojects...
afterEvaluate {
def distTar = tasks.findByName('distTar')
if(distTar) {
rootProject.distTar.dependsOn distTar
rootProject.distTar.inputs.file distTar.archivePath
rootProject.distTar.from distTar.archivePath
}
}
}
then invoke "build distTar" on the root project - it will assemble "unifiedTar.tar" in "build/distributions" subfolder (of the root project).
How it works:
"task distTar(...)" declares a new task of type Tar in the root project.
"subprojects" applies the specified closure to each subproject.
"afterEvaluate" ensures that the specified closure is called AFTER the current project (in this case subproject) is evaluated. This is very important, because we are going to use properties of the subproject which are defined only after it's evaluation.
"tasks.findByName" allows us to determine, whether the given task is defined for given project. If not, it returns null and the following code is not performed. This way we stay agnostic regarding the nature of the subproject.
"dependsOn" ensures that distTar of the root project depends on distTar of the given project (and, therefore, is executed later than it).
"inputs.file" ensures that distTar on root project is not executed, if none of the constituent tars has changed.
"from" adds constituent tar to unified tar.

Resources