I'm trying to include a ZIP file inside a TAR file in a gradle build. I'm not insane, this is to replicate an existing ant script and I cannot change the distribution layout for various business reasons.
I'm having to use a whenReady closure to collect dependencies without problems
whenReady means that the ZIP file is not built until after the buildTar task has completed, even though buildTar depends on buildZip.
I cannot call tar {} directly as gradle does not support this
Gradle does not appear to support calling tasks directly.
This is the general layout I have
task buildZip(type: Zip) {
gradle.taskGraph.whenReady {
// build zip file usual way with from blocks
from(...) {
}
from(...) {
}
}
doLast {
println "ZIP ready"
// could I call tar task from here??
}
}
task buildTar(type: Tar, dependsOn: buildZip) {
println "Building TAR"
from (buildZip.archivePath) {
}
... more stuff, installer script etc.
}
Output I see with gradle :buildTar, i.e. the TAR builds before the ZIP is built.
Building TAR
ZIP ready
Update.
Perryn Fowler comment below identifies the issue correctly, it is based on my misunderstanding of execution vs configuration in gradle.
The Tar is not being built before the Zip, the Tar task is being
configured before the Zip task is executed
Update.
This question is no longer necessary as the option duplicatesStrategy can be used in the ZIP task to avoid the problem being 'fixed' with gradle.taskGraph.whenReady
The answer to this question was actually provided by Perryn Fowler in the top comments, it was was based on my misunderstanding of execution vs configuration in gradle. I've created this answer so the question is marked as answered. The other answer simply paraphrases the original question with a link to the userguide.
The Tar is not being built before the Zip, the Tar task is being
configured before the Zip task is executed
i.e. any nested commands within a special task, e.g. Zip, Tar etc. are run and configuration time, the from blocks are executed later.
Here You have a sample working solution:
build.gradle:
task buildZip(type: Zip) {
from 'dir'
destinationDir project.file('build/zip')
archiveName 'lol.zip'
}
task buildTar(type: Tar, dependsOn: buildZip) {
from 'build/zip'
include '*.zip'
destinationDir project.file('build/tar')
archiveName 'lol.tar'
}
Is that clear for You?
P.S. I think that's a good idea for You to read userguide.
Related
I have a tar file that contains multiple tar.gz files (a Docker image), and I want to extract a single file from it. Doing this is in a shell script is trivial, but it seems a bit tricky when using Gradle.
This is what I have so far:
task extractOuter(type: Copy) {
dependsOn jibBuildTar
from tarTree(file("${buildDir}/jib-image.tar"))
include "*.tar.gz"
into "${buildDir}/tgz"
}
task extractInner(type: Copy) {
dependsOn extractOuter
from (fileTree(dir: "${buildDir}/tgz").collect { tarTree(it) }) {
include "**/filename"
includeEmptyDirs = false
}
into "${buildDir}/files"
}
It seemed to work at first, but it turned out that it fails occasionally: the extractInner task does not find the file. Maybe I don't use Gradle's lazy evaluation correctlty.
How to make it work? Or is there totally different, more elegant way?
Doing this is in a shell script is trivial
You can continue using the shell script by using the Exec task type.
but it seems a bit tricky when using Gradle.
What you have so far is how you do it with Gradle. The advantage with Gradle is that it won't perform work that has already happened. See Build Cache for more details.
It seemed to work at first, but it turned out that it fails occasionally: the extractInner task does not find the file. Maybe I don't use Gradle's lazy evaluation correctlty.
This is called out in the above linked docs (emphasis mine):
Some tasks, like Copy or Jar, usually do not make sense to make cacheable because Gradle is only copying files from one location to another. It also doesn’t make sense to make tasks cacheable that do not produce outputs or have no task actions.
So you've declared your tasks, but you haven't configured them to produce any outputs which may or may not contribute to the problem since you expect the output to be present for a task dependency.
Since Copy extends DefaultTask, you can use the outputs to set the task output.
task extractOuter(type: Copy) {
dependsOn jibBuildTar
outputs.dir(file("$buildDir/tgz")
from tarTree(file("${buildDir}/jib-image.tar"))
include "*.tar.gz"
into "${buildDir}/tgz"
}
task extractInner(type: Copy) {
dependsOn extractOuter
outputs.dir(file("$buildDir/files")
from (fileTree(dir: "${buildDir}/tgz").collect { tarTree(it) }) {
include "**/filename"
includeEmptyDirs = false
}
into "${buildDir}/files"
}
I have a multi-module project in Gradle
root
-- ProjectA
-- ProjectB
Both ProjectA and ProjectB use the application plugin to create a zip in "ProjectA/build/distributions" and "ProjectB/build/distributions" respectively.
Now I want to copy the two zip files into "root/build/distributions".
I have tried various approaches, e.g. adding this in the build.gradle of the root project:
subprojects {
task copyFiles(type: Copy) {
includeEmptyDirs = true
from "$buildDir/distributions"
include '*.zip'
into "$parent.buildDir/distributions"
}
copyFiles.dependsOn(build)
}
or just adding a task to the root project:
task copyFiles(type: Copy) {
from "ProjectA/build/distributions"
from "ProjectB/build/distributions"
include "*.zip"
into "$buildDir/distributions"
}
build.dependsOn(copyFiles)
However, in both cases, nothing happens. No file gets copied.
What am I doing wrong?
I can see two things you are doing wrong:
You have relative paths to the subprojects. This is discouraged as it means you will always have to invoke Gradle from the root folder. And if a Gradle daemon was started from somehere else, it will fail. You could fix it by using the rootDir property (e.g. `from "$rootDir/ProjectA/...") but there is a better way...
The other problem is that you have no dependencies from your copyFiles task in your root project to the required distZip tasks in the sub-projects. So if the distributions have not already been built previously, there are no guarantees that it will work (which it apparently doesn't).
To fix it, you can have a look at the question "Referencing the outputs of a task in another project in Gradle", which covers the more general use case of what you ask. There are currently two answers, both of which are good.
So in your case, you can probably do either this:
task copyFiles(type: Copy) {
from tasks.getByPath(":ProjectA:distZip").outputs
from tasks.getByPath(":ProjectB:distZip").outputs
into "$buildDir/distributions"
}
or this:
task copyFiles(type: Copy) {
subprojects {
from(tasks.withType(Zip)) {
include "*.zip"
}
}
into "$buildDir/distributions"
}
Gradle will implicitly make the copy task depend on the other tasks automatically, so you don't need to do that yourself.
Also note that the currently accepted answer to the question I referenced is about configuration variants, and this is probably the most correct way to model the relationships (see here for more documentation on the topic). But I prefer the simplicity of the direct access to the tasks over the verbose and arguably more complex way to model it through configurations. Let's hope it gets simpler in a later release.
I am working in an old project which was using Gradle Wrapper v3.2.1, but I want to update it to the latest version (currently v5.4.1).
I have tried updating it to v4.10.2 first but it fails too, so I guess it is something that was not backwards compatible between v3->v4.
The code we have in our build.gradle is:
task buildZip(type: Zip) {
group 'Build'
description 'Assembles a zip archive containing the main classes.'
baseName = "someName"
from compileJava
from processResources
into('lib') {
from configurations.runtime
}
}
Using gradle v3 it included all the libraries (as .jar files) in the .zip file under "lib/" folder, but if I use v4 or later, it does not fail, but it does not include the libraries neither. I have achieved to get the .class files of the dependencies, but that does not work for what I need (AWS Lambda function).
Any idea on how to get the .jar dependencies into the .zip file?
Cheers!
Francisco Robles Martin
So, thanks to Opal comment, I kept looking for a bit more and got a solution, but it seems to not be very correct as I am forcing implementation to allow be resolved:
configurations.implementation.setCanBeResolved(true)
task buildZip(type: Zip) {
group 'Build'
description 'Assembles a zip archive containing the main classes.'
baseName = "someName"
from compileJava
from processResources
into('lib') {
from configurations.implementation
}
}
It works, but I guess there should be a better way to do it without the first line.
I’m getting the following error whenever I attempt to use a Copy task to copy a file into the root of a project (the same folder I’m running gradle from):
Failed to create MD5 hash for file content.
I thought this was related to the artifacts I was pulling from Artifactory, but that seems to be unrelated. I was able to get the same results with a minimal script.
Is there something obviously wrong with what I’m doing, or does Gradle intentionally disallow such things?
task fails(type:Copy) {
from 'build/someFile.txt'
into new File('.').absolutePath
}
task works(type:Copy) {
from 'build/someFile.txt'
into new File('.').absolutePath + '/output'
}
Short Answer: Don't copy into the project directory, you are best to use into "$buildDir/someFolder" so that the folder is isolated to this single task, and also so that it will be cleaned by gradle clean
Long Answer: At it's core, Gradle has the concept of an "UP-TO-DATE" check for every single task. If Gradle sees that nothing has changed since last time a task was executed it will use the old result instead of executing again.
UP-TO-DATE checking is implemented by taking a "hash" of the task inputs and task outputs. Since you are using into '.' that means that the entire contents of the project directory is considered a task output (bad)
Gradle uses the .gradle folder for temp files (eg task hashes) It's likely some of these files are locked for writing as Gradle is trying to also read the same files (to calculate the "hash" of the task outputs) causing the error you are seeing
* EDIT *
If you need to copy into the project directory for legacy reasons, you might use Project.copy(...) directly instead of a Copy task. You could manually manage the task inputs/outputs in this case
Eg
task customCopy {
inputs.file "$buildDir/someFile.txt"
outputs.file 'someFile.txt'
doLast {
copy {
from "$buildDir/someFile.txt"
into '.'
}
}
}
Can you believe it, the following works
task myCopy(type: Copy) {
from "$rootDir/app1/src/main/resources/db"
into "$rootDir/app2/src/test/resources/db"
}
test.dependsOn myCopy
and the following doesn't 🤦
task myCopy(type: Copy) {
from '$rootDir/app1/src/main/resources'
into '$rootDir/app2/src/test/resources'
}
test.dependsOn myCopy
I'm adding a task to deploy war files to Tomcat .. the only thing that the task needs to do is copy the war file to the TOMCAT location.
There 2 ways that I can think of implementing this .. but being new to gradle, I'm not quite sure what's more conventional/right (or if it even matters).
task myCopy(type: Copy)
myCopy.configure {
from('source')
into('target')
include('*.war')
}
or
task myCopy{
doLast{
copy {
from 'source'
into 'target'
include '*.war'
}
}
}
In most cases (including this one), the Copy task is the better choice. Among other things, it will give you automatic up-to-date checking. The copy method is meant for situations where (for some reason) you have to bolt on to an existing task and cannot use a separate task for copying.
The code for your Copy task can be simplified to:
task myCopy(type: Copy) {
from('source')
into('target')
include('*.war')
}
UP-TO-DATE only verifies the file is in place but not if the files has changed
to avoid being cached with an old file use
outputs.upToDateWhen { false }