I am trying to use the zip in gradle and my build worked fine until I added this to my section for the "webserver" project (maybe my question should be why does the copy work without task in front of it?????...I am just really confused by the difference between copy and zip if you need task in front of zip but not copy below)
assemble << {
zip {
from '.'
includes ['app/*','conf/*','public/*','play-1.2.4/*','run*.sh']
}
}
The error is
What went wrong:
Execution failed for task ':webserver:assemble'.
Could not find method zip() for arguments [build_3nljqgmljp29v06751h102sq8b$_run_closure3_closure16_closure18#7cc8e407] on task ':webserver:assemble'.
I don't understand as I am using copy successfully like so
copy { from fixedLibDir into genLibDir }
Also, I am getting really really confused by the documentation as in the documentation, they almost always have
task myZip(type: Zip) {
}
I just want to call a zip task not create a new one every time, so the documentation examples seem to be very bad...they should be examples of using zip task not creating a new one(after all, who wants to create a new zip task when one already exists??) OR am I missing something here? At any rate, that confused me when getting started with gradle alot and I am guessing it will confuse others. It might be nice to show both using it and defining a new one(though I still don't get why I would define a new one).
MORE INFO/UPDATE:
I also tried this code which runs and I see the print out message but I see no zip file in my webserver/output/libs directory as I would expect???
assemble << {
println "I'm zipping up now"
task zip(type: Zip) {
from('.') {
fileMode = 0755
include 'run*.sh'
include 'app/*'
}
}
}
later,
Dean
The reason copy works, is because it's a global utility function defined in Project (see here). So you can call copy from anywhere, and it will work.
On the other hand there is no equivalent zip method. Perhaps, because zipping usually need more configuration than copying, like specifying zip archive name, or maybe they just missed it out. This means that you have to use the task zip(type: Zip){ from ... into ... } syntax for zip.
In contrast to Copy, copy is not a task; it is just a method on the Project class. Gradle doesn't have any information about, and doesn't control the execution of, methods like it does for tasks. Therefore, methods have some drawbacks over the corresponding tasks (e.g. no up-to-date check) and should only be used when a task is not an option (which is less often than you think).
As for zip, there is simply no such method, in particular not on the Project class. The best place to look up such information is the Gradle DSL reference.
after all, who wants to create a new zip task when one already exists
Not sure what you mean by that; maybe you are confusing tasks and task types. It's perfectly normal to use a separate task for every zip file to be created.
PS: I recommend to take a step back and learn more about Gradle before tackling real-world builds.
I think the correct way to do this now is
task myZip(type: Zip) {
from('.') {
include 'run*.sh'
include 'app/*'
}
}
assemble.dependsOn('myZip')
//This last line is important of course!!!!
I am not sure what the following does to be honest now as it doesn't do anything except println
assemble << {
println "I'm zipping up now"
task zip(type: Zip) {
from('.') {
fileMode = 0755
include 'run*.sh'
include 'app/*'
}
}
}
Related
I am trying to do something like this:
jar {
doLast{
from "build/libs/TheJar.jar"
into "."
}
}
So far, I have tried various tutorials including all forms from this answer but non have worked. The only thing that works is calling a separate task but I'd like to know why my construction is wrong and why can't I run something after the jar or shadowJar tasks.
It looks like you took some parts of the answers in the linked post and somehow mixed them without knowing what your final code actually does.
Tasks in Gradle may have a type (e.g. Copy, Jar, ...). This type defines what the task will do once it gets executed (so called task actions). A task without a type won't do anything when its executed, unless you add task actions manually. Using doFirst will add the passed action (also called closure) to the start of the list of task actions, using doLast will add it to the end of the list.
Everything outside of doFirst and doLast closures is not part of the execution of the task, but can be used to configure the task:
task example {
doLast {
println "second action"
}
doFirst {
println "first action"
}
println "configuration"
}
Run the code above with gradle example and you will see the order of the log messages as configuration, first action, second action.
Task configuration will run, even if the task won't be executed later on. Even if you call gradle (without any task names as arguments), the console will still print configuration. This was the actual problem in the linked question.
Lets come to the real question: How to copy a file?
Well, Gradle offers two ways to copy a file. The first one is the task type Copy. You can create a task based on this type, apply your configuration and then either call it directly from the command line or define task dependencies using dependsOn or finalizedBy:
task copySomeFiles(type: Copy) {
from '...'
into '...'
}
However, you don't want to create an additional task. Gradle also has a method called copy that may be called anywhere in your script and will instantly copy the files:
copy {
from '...'
into '...'
}
The code above will copy your files, but it will copy your files every time Gradle executes (which may be often, e.g. when using an IDE). To solve this problem, you may move your code with copy to a task action, e.g. inside a doFirst or doLast closure:
jar {
doLast {
copy {
from "build/libs/TheJar.jar"
into "."
}
}
}
As you can see, your code was missing the copy statement. Now, whenever your task jar gets executed, its last task action will copy the files as intended.
Bonus question: Why is there no error?
The "problem" in your case is that your code is perfectly valid Gradle code. The task jar is of type Jar. Every task of type Jar may be configured using methods called from and into. The method from adds files to the JAR file and the method into sets the destination directory inside the JAR. So instead of copying, your code configures the underlying task jar. However, this has no negative consequences, as this configuration gets applied inside doLast, which only runs once the JAR file has already been created. Something that already happened cannot be configured.
I have a tar file that contains multiple tar.gz files (a Docker image), and I want to extract a single file from it. Doing this is in a shell script is trivial, but it seems a bit tricky when using Gradle.
This is what I have so far:
task extractOuter(type: Copy) {
dependsOn jibBuildTar
from tarTree(file("${buildDir}/jib-image.tar"))
include "*.tar.gz"
into "${buildDir}/tgz"
}
task extractInner(type: Copy) {
dependsOn extractOuter
from (fileTree(dir: "${buildDir}/tgz").collect { tarTree(it) }) {
include "**/filename"
includeEmptyDirs = false
}
into "${buildDir}/files"
}
It seemed to work at first, but it turned out that it fails occasionally: the extractInner task does not find the file. Maybe I don't use Gradle's lazy evaluation correctlty.
How to make it work? Or is there totally different, more elegant way?
Doing this is in a shell script is trivial
You can continue using the shell script by using the Exec task type.
but it seems a bit tricky when using Gradle.
What you have so far is how you do it with Gradle. The advantage with Gradle is that it won't perform work that has already happened. See Build Cache for more details.
It seemed to work at first, but it turned out that it fails occasionally: the extractInner task does not find the file. Maybe I don't use Gradle's lazy evaluation correctlty.
This is called out in the above linked docs (emphasis mine):
Some tasks, like Copy or Jar, usually do not make sense to make cacheable because Gradle is only copying files from one location to another. It also doesn’t make sense to make tasks cacheable that do not produce outputs or have no task actions.
So you've declared your tasks, but you haven't configured them to produce any outputs which may or may not contribute to the problem since you expect the output to be present for a task dependency.
Since Copy extends DefaultTask, you can use the outputs to set the task output.
task extractOuter(type: Copy) {
dependsOn jibBuildTar
outputs.dir(file("$buildDir/tgz")
from tarTree(file("${buildDir}/jib-image.tar"))
include "*.tar.gz"
into "${buildDir}/tgz"
}
task extractInner(type: Copy) {
dependsOn extractOuter
outputs.dir(file("$buildDir/files")
from (fileTree(dir: "${buildDir}/tgz").collect { tarTree(it) }) {
include "**/filename"
includeEmptyDirs = false
}
into "${buildDir}/files"
}
This is my task
task copyDeps(type: Copy) {
from configurations.compile
into 'build/lib'
eachFile {
if (it.relativePath.getFile(destinationDir).exists() && !it.relativePath.getFile(destinationDir).getName().contains("SNAPSHOT")) {
it.exclude()
}
}
}
And it's supposed to download all the dependencies into the build/lib directory, except when there's already a file present with the same name, unless it's a snapshot.
But when all files are present, the snapshots don't get overwritten at all, which they should.
What's inconsistent is that when at least 1 file is missing (doesn't matter if it's a snapshot or not), then suddenly all the snapshots get overwritten, but not the non-snapshots.
So when at least one file is missing, the script works as intended, but when all files are present it just ignores half the expression of the if clause.
It's almost like it short circuits the "exists()" expression for the whole set of files.
Using gradlew which points to gradle-5.0-bin.zip
It seems it's related to this: gradle issue 4663
The Copy task has a concept of inputs and outputs and when neither the inputs or the outputs have changed since the last run (same list of dependencies, same list of copied files in the target directory) the Copy task is considered UP-TO-DATE by gradle.
Found alternative:
task copyDeps() {
project.copy {
from configurations.compile
into 'build/lib'
eachFile {
if (it.relativePath.getFile(project.file('build/lib')).exists() && !it.relativePath.getFile(project.file('build/lib')).getName().contains("SNAPSHOT")) {
it.exclude()
}
}
}
}
Had to make some edits because this construct doesn't know a destinationDir. Also had to move the task to below the list of dependencies as it gave configuration errors I don't yet understand.
I’m getting the following error whenever I attempt to use a Copy task to copy a file into the root of a project (the same folder I’m running gradle from):
Failed to create MD5 hash for file content.
I thought this was related to the artifacts I was pulling from Artifactory, but that seems to be unrelated. I was able to get the same results with a minimal script.
Is there something obviously wrong with what I’m doing, or does Gradle intentionally disallow such things?
task fails(type:Copy) {
from 'build/someFile.txt'
into new File('.').absolutePath
}
task works(type:Copy) {
from 'build/someFile.txt'
into new File('.').absolutePath + '/output'
}
Short Answer: Don't copy into the project directory, you are best to use into "$buildDir/someFolder" so that the folder is isolated to this single task, and also so that it will be cleaned by gradle clean
Long Answer: At it's core, Gradle has the concept of an "UP-TO-DATE" check for every single task. If Gradle sees that nothing has changed since last time a task was executed it will use the old result instead of executing again.
UP-TO-DATE checking is implemented by taking a "hash" of the task inputs and task outputs. Since you are using into '.' that means that the entire contents of the project directory is considered a task output (bad)
Gradle uses the .gradle folder for temp files (eg task hashes) It's likely some of these files are locked for writing as Gradle is trying to also read the same files (to calculate the "hash" of the task outputs) causing the error you are seeing
* EDIT *
If you need to copy into the project directory for legacy reasons, you might use Project.copy(...) directly instead of a Copy task. You could manually manage the task inputs/outputs in this case
Eg
task customCopy {
inputs.file "$buildDir/someFile.txt"
outputs.file 'someFile.txt'
doLast {
copy {
from "$buildDir/someFile.txt"
into '.'
}
}
}
Can you believe it, the following works
task myCopy(type: Copy) {
from "$rootDir/app1/src/main/resources/db"
into "$rootDir/app2/src/test/resources/db"
}
test.dependsOn myCopy
and the following doesn't 🤦
task myCopy(type: Copy) {
from '$rootDir/app1/src/main/resources'
into '$rootDir/app2/src/test/resources'
}
test.dependsOn myCopy
I have a Gradle copy task set-up to publish undecorated JAR file(s) for testing and debugging, viz.
Task definition:
task copyJarToStaging( type: Copy ) {
from jar // shortcut for createJar.outputs.files
into ( "${rootProject.rootDir}/dist/" )
rename( '-.*\\.jar', ".jar" )
}
Which works, to put a JAR file into the one directory. What's really needed is to drop the JAR into one or more different folders under "dist/".
Following many trials (and errors) I found this version worked for me.
Invoke the copy task:
// build.gradle (module)
assemble.dependsOn copyJarToStaging {
println "into ==> ${destinationDir}/support"
into "${destinationDir}/support/"
}
However, it doesn't really smell right.
Is there a cleaner alternative way? I would have liked a closure for instance to just append to the into attribute -- But it didn't go.
If I wanted the same file in different places, it would be better if I can do something like take the into string and yield each value back.
Is part or all of that possible? Or, am I dreaming???
Typically you'd create multiple copy tasks
['dev', 'staging', 'uat', 'prod'].each { String dir ->
Task task = tasks.create("copyJarTo${dir.capitalize()}", type: Copy) {
from jar
into "dist/$dir"
}
assemble.dependsOn task
}