I have a Gradle task which simply copies the files from one folder to another.
When the copy 'from' 'into' methods are called during configuration phase it works but when they are called in the execution phase it doesn't.
Doing it this way it works
task copyServerConfig(type:Copy) {
from "${projectDir}" + File.separator + 'server_config'
into localServer
}
gradle copyServerConfig
:copyServerConfig
BUILD SUCCESSFUL
Total time: 0.686 secs
With the '<<' it doesn't show any error but the files are not copied:
task copyServerConfig(type:Copy) << {
from "${projectDir}" + File.separator + 'server_config'
into localServer
}
gradle copyServerConfig
:copyServerConfig UP-TO-DATE
BUILD SUCCESSFUL
Total time: 0.654 secs
Same thing happen if I put it inside a doFirst{} block.
Another thing I don't understand is the following:
If I execute this task:
task task1(type:Exec) << {
commandLine 'echo', ' TEST'
}
I get this error:
gradle task1
:task1 FAILED
FAILURE: Build failed with an exception.
What went wrong:
Execution failed for task ':task1'.
execCommand == null!
But if I change to this:
task task1(type:Exec) {
commandLine 'echo', ' TEST'
}
gradle task1
:task1
TEST
BUILD SUCCESSFUL
Total time: 0.655 secs
I read in the documentation about the about build lifecycle and I understand commands inside doFirst{} and doLast{} blocks execute during the Execution phase and commands outside those blocks execute in the Configuration phase.
But I think there is some important concept I am missing here. Why is this happening ? What is the difference of having a command being executed in the Configuration phase vs Execution phase ?
The first variant is a correct way to declare a Copy task
task copyServerConfig(type:Copy) {
from "${projectDir}" + File.separator + 'server_config'
into localServer
}
Here you say "this is a task of type Copy with name copyServerConfig, this is from param and this is into param". The code inside the brackets is used to configure a task.
<< operator adds a doAfter closure to the task, that is executed after it.
task copyServerConfig(type:Copy) << {
from "${projectDir}" + File.separator + 'server_config'
into localServer
}
This declaration means "create empty (not configured) Copy task and execute these two lines after it is done. Here you did not configure a task, it does nothing and is always UP-TO-DATE. The two lines in doAfter closure do nothing as well, because they are not related to Copy task, so from and into keywords have no effect.
The similar thing happens with Exec task
task task1(type:Exec) << {
commandLine 'echo', ' TEST'
}
Here you have empty Exec task that fails because it is not configured.
Related
I have the following gradle task in a subproject's build.gradle file to copy file.txt from a directory called from_dir/ to a directory to_dir/ and rename it to fileRenamed.txt:
task copyRenameFile(type: Copy) {
System.out.println("copyRenameFile begin")
from('from_dir')
into('to_dir')
include('file.txt')
rename('file.txt', 'fileRenamed.txt')
System.out.println("copyRenameFile end")
}
build.finalizedBy(copyRenameFile)
When I run gradlew :subprojectname:build, this task performs the copy as expected after the build task is finalized, but it executes the printlns during the configuration phase, before the build task.
In an attempt to make the printlns appear after the build phase when the copy is actually being executed, I tried using the << operator like so:
task copyRenameFile(type: Copy) << {
But this results in the task beings skipped with the following message:
[INFO] [org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter] Skipping task ':subprojectname:copyRenameFile' as it has no source files.
Does this mean file.txt could not be found during the execution phase? Why?
EDIT: After reading this answer, I now understand that my usage of the << operator is causing the configuration phase to skip this task and this is the reason the source files are not found. So I guess I can't use the << operator, but how else can I make the printlns occur when the task is being executed and not when it is being configured?
I figured it out:
task copyRenameFile(type: Copy) {
doFirst{
System.out.println("copyRenameFile begin")
}
from('from_dir')
into('to_dir')
include('file.txt')
rename('file.txt', 'fileRenamed.txt')
doLast{
System.out.println("copyRenameFile end")
}
}
build.finalizedBy(copyRenameFile)
I got rid of the << and instead used doFirst and doLast to ensure my printlns occured during the execution phase.
I have two gradle tasks in my build.gradle file, one to archive a folder and another to push it to a remote server.
task tarTask(type: Exec) {
commandLine 'tar', '-czf', 'javadocs.tgz', 'javadocs/'
}
If I execute tarTask alone with gradle tarTask and with the publish task commented out, the build succeeds.
I am using this task as a dependency in the publish task.
task publish(dependsOn: tarTask) {
ssh.run {
settings {
knownHosts = allowAnyHosts
fileTransfer = 'scp'
}
session(remotes.webServer) {
from: 'javadocs.tgz', into: 'publishingHouse/'
}
}
}
But when i execute gradle publish it fails saying that it is not able to find the tgz file which should have been created if the previous task is executed.
java.io.FileNotFoundException: javadocs.tgz
Being new to gradle i am not really sure what I am missing here. Any ideas on what I can do?
I suppose the reason is within the phase when tasks are executed. tarTask is configured at the configuration phase and will be executed at the execution phase.
And at the same time publish task doesn't have any behavior to execute at the execution phase, but has ssh.run to be executed during configuration.
This mean, that when you run gradle publish your logic to copy tar-archive is executed at the configuration phase, while tar-archive is not yet exists (it will be created later at the execution phase).
To make a copy execution at the execution phase you can simply add << to the publish task declaration as follows:
task publish(dependsOn: tarTask) << {
ssh.run {
settings {
knownHosts = allowAnyHosts
fileTransfer = 'scp'
}
session(remotes.webServer) {
from: 'javadocs.tgz', into: 'publishingHouse/'
}
}
}
Note, that << is the same as doLast and the closure will be excuted at the execution phase. You can read about Gradle build lifecycle here
Please find below init.gradle file that has a common task for all my projects :-
rootProject { apply plugin: "eclipse" }
gradle.allprojects{
ext.commonTaskForEveryBuild = {
println "Common task for every build starts here..."
println "Common task for every build ends here..."
}
}
Also find below sample build.gradle for one of my project :-
apply plugin : 'java'
clean.doFirst{
println "Before invoking commonTaskForEveryBuild"
//This is invocation of commonTaskForEveryBuild
commonTaskForEveryBuild
println "After invoking commonTaskForEveryBuild"
}
Below are the logs for the execution of "gradle clean" task for the sample build.gradle file :-
:clean
Before invoking commonTaskForEveryBuild
After invoking
commonTaskForEveryBuild
:clean UP-TO-DATE
BUILD SUCCESSFUL
Total time: 3.722 secs
Looks like clean.doFirst is called, but it's not invoking definition of commonTaskForEveryBuild. Although, there is no compile time or runtime error.
Please suggest.
It's all right, that your task is not invoked, since your script doesn't attempt to do it and it's not even a task. You just getting your commonTaskForEveryBuild instance, but do nothing with it.
You may need to read the official user guide to find out, how to call one task from another. In short - this is usually done via task dependencies. You have to make your clean task depending on commonTaskForEveryBuild task to run it before the clean task execution. This could be done like so:
clean.dependsOn commonTaskForEveryBuild
Furthermore, it's all about tasks, but in your case ext.commonTaskForEveryBuild = {...} is not even a task, but just a closure and it must be executed as commonTaskForEveryBuild()
Let's say that I have a task "main" that depends on another task "dependency." I would like "main" to be rerun any time its dependency (or its dependency's dependencies) is rebuilt because "main" relies on the artifacts produced by "dependency" (or the dependencies of "dependency").
A build.gradle file containing an example of what I'm dealing with is the following:
defaultTasks 'main'
task baseDependency {
outputs.file 'deps.out'
outputs.upToDateWhen { false }
doLast {
exec {
commandLine 'bash', '-c', 'echo hello world > deps.out'
}
}
}
task dependency(dependsOn: baseDependency)
task main(dependsOn: dependency) {
outputs.file 'main.out'
doLast {
exec {
commandLine 'bash', '-c', 'echo hello world > main.out'
}
}
}
Executing gradle the first time:
:baseDependency
:dependency
:main
BUILD SUCCESSFUL
Total time: 0.623 secs
Executing it a second time:
:baseDependency
:dependency
:main UP-TO-DATE
BUILD SUCCESSFUL
Total time: 0.709 secs
I would really love if "main" were not marked "UP-TO-DATE" if its dependencies had to be rebuilt. That seems essential. How do you make sure that's the case?
The standard way to specify dependency between tasks is via task's inputs and outputs. This can be any file, fileset or directory.
In your case you should modify main task and add inputs.file 'deps.out' to its definition.
Note that gradle has an optimization that may lead to unexpected behavior in a simplistic example that you provided.
Before a task is executed for the first time, Gradle takes a snapshot
of the inputs. This snapshot contains the set of input files and a
hash of the contents of each file. Gradle then executes the task. If
the task completes successfully, Gradle takes a snapshot of the
outputs. This snapshot contains the set of output files and a hash of
the contents of each file. Gradle persists both snapshots for the next
time the task is executed.
Each time after that, before the task is executed, Gradle takes a new
snapshot of the inputs and outputs. If the new snapshots are the same
as the previous snapshots, Gradle assumes that the outputs are up to
date and skips the task. If they are not the same, Gradle executes the
task. Gradle persists both snapshots for the next time the task is
executed.
So even if you specify correct inputs in a simple example where the same file is generated the dependent task will be marked as up to date on the second and subsequent runs.
If you do not want or cannot hardcode dependency on the file you can override upToDateWhen for dependent task and calculate the condition if the task is up to date based on dependencies of this task and their state like this:
outputs.upToDateWhen { task ->
task.taskDependencies.inject(true) { r, dep ->
r && dep.values.inject(true) { res, v ->
res && (!(v instanceof Task) || v?.state.getSkipped())
}
}
}
upToDateWhen should return true if this task should not be run at all (because its output are already up-to-date). And this is the case when no dependent task was run (the gradle documentation is a bit vague about this I must admit but getSkipped seems work as expected).
I've prepared a very simple script, that illustrates the problem I see using Gradle 1.7 (need to stick with it because of some plugins not yet supporting newer versions).
I'm trying to dynamically create tasks each of which corresponds to a file in the project directory. This works fine, but the tasks I create never get executed as soon as I assign them type 'Copy'.
Here is my problem build.gradle:
file('templates').listFiles().each { File f ->
// THIS LINE DOES NOT WORK
task "myDist-${f.name}" (type: Copy) {
// NEXT LINE WORKS
//task "myDist-${f.name}" {
doLast {
println "MYDIST-" + f.name
}
}
}
task distAll(dependsOn: tasks.matching { Task task -> task.name.startsWith("myDist")}) {
println "MYDISTALL"
}
defaultTasks 'distAll'
in this way my tasks do not get executed when I call default task calling simply gradle:
MYDISTALL
:myDist-template1 UP-TO-DATE
:myDist-template2 UP-TO-DATE
:distAll UP-TO-DATE
BUILD SUCCESSFUL
If I remove type Copy from my dynamic task (uncommenting the line above), my tasks get executed:
MYDISTALL
:myDist-template1
MYDIST-template1
:myDist-template2
MYDIST-template2
:distAll
BUILD SUCCESSFUL
(You'll need to create a folder name templates in the same directory where build.gradle is located and put couple of empty files into there in order to run the test)
According to the debug output:
Skipping task ':myDist-template1' as it has no source files.
Skipping task ':myDist-template2' as it has no source files.
So how can I specify source files and make my Copy tasks execute?
I've tried adding
from( '/absolute/path/to/existing/file' ) {
into 'myfolder'
}
to the task body, I've tried assigning task's inputs.source file('/my/existing/file') with no success.
Could you please advise on how to modify my simple script leaving dynamic task creation and keeping my dynamic tasks of type Copy?
Thank you!
Edit:
All right, this way the task gets called:
file('templates').listFiles().each { File f ->
task "myDist-${f.name}" (type: Copy) {
from f
into 'dist'
doLast {
println "MYDIST-" + f.name
}
}
}
but it looks I must always specify from/into. It doesn't suffice to do that in the doLast{} body.
A Copy task only gets executed if it has something to copy. Telling it what to copy is part of configuring the task, and therefore needs to be done in the configuration phase, rather than the execution phase. These are very important concepts to understand, and you can read up on them in the Gradle User Guide or on the Gradle Forums.
doFirst and doLast blocks get executed in the execution phase, as part of executing the task. Both are too late to tell the task what to copy: doFirst gets executed immediately before the main task action (which in this case is the copying), but (shortly) after the skipped and up-to-date checks (which are based on the task's configuration). doLast gets executed after the main task action, and is therefore clearly too late.
I think the following Gradle User Guide quote answers my question the best:
Secondly, the copy() method can not honor task dependencies when a task is used as a copy source (i.e. as an argument to from()) because it's a method and not a task. As such, if you are using the copy() method as part of a task action, you must explicitly declare all inputs and outputs in order to get the correct behavior.
Having read most of the answers to "UP-TO-DATE" Copy tasks in gradle, it appears that the missing part is 'include' keyword:
task copy3rdPartyLibs(type: Copy) {
from 'src/main/jni/libs/'
into 'src/main/libs/armeabi/'
include '**/*.so'
}
Putting from and into as part of the doLast section does not work. An example of a working task definitions is:
task copyMyFile(type: Copy) {
def dockerFile = 'src/main/docker/Dockerfile'
def copyTo = 'build/docker'
from dockerFile
into copyTo
doLast {
println "Copied Docker file [$dockerFile] to [$copyTo]"
}
}
Not the behavior I was expecting.
Using gradle 3.2.1