I have had problems with that the Exec task in gradle have problems with the up-to-date check if the output files is captured from standardOutput.
I have tried to simplify the example as much as possible:
task printToOutputFile(type: Exec) {
inputs.file file("file1") // not relevant for this example
outputs.file file("file2")
commandLine = ["echo", "1234"]
standardOutput = new FileOutputStream("file2")
}
When rerunning this task I expect it to be UP-TO-DATE but it is not.
How can I make the UP-TO-DATE check work when using the standardOutput as outputs?
What I have tried:
Closing and/or Flushing the stream in an doLast block.
the problem is, that the line
standardOutput = new FileOutputStream("file2")
changes the lastModified attribute of file2. To get the up-to-date check working, you have to move this assignment to the execution phase. You can do this by putting this assignment in a doFirst block. The following snippet should do the trick:
task printToOutputFile(type: Exec) {
inputs.file file("file1") // not relevant for this example
outputs.file file("file2")
commandLine = ["echo", "1234"]
doFirst{
standardOutput = new FileOutputStream("file2")
}
}
cheers,
René
Related
I have 2 gradle task, one depends on the other.
The first task get db dump from remote server using sh script. The second task parse dump from previous task.
The problem is that second task starts early than dump script from the first task finishes. So there is no file yet, when second task start to work.
Tasks:
task getDumpFromRemotePostgres(type: Exec) {
executable = "/bin/sh"
println 'started task getDumpFromRemotePostgres'
def dumpScript = './createSqliteDb/scripts/dump_db_script.sh'
def dbDumpFileName = 'dump.sql'
args += [dumpScript]
println 'finished task getDumpFromRemotePostgres'
}
task patchPostgresDumpFile(type: Exec) {
dependsOn getDumpFromRemotePostgres
executable = "/bin/sh"
println 'started task patchPostgresDumpFile'
def dbDumpFileName = 'dump.sql'
File dumpFile = file(dbDumpFileName)
def line
dumpFile.withReader { reader ->
while ((line = reader.readLine()) != null) {
//parse and modify
}
println 'finished task patchPostgresDumpFile'
}
Script dump_db_script.sh:
echo "dump script started"
pg_dump --data-only --inserts --dbname=postgresql://user:pas#server/base_name > dump.sql
echo "dump script finished"
Console log is the follow (if i delete access file lines):
started task getDumpFromRemotePostgres
finished task getDumpFromRemotePostgres
started task patchPostgresDumpFile
finished task patchPostgresDumpFile
> Task :getDumpFromRemotePostgres
dump script started
dump script finished
Is any idea how to solve the problem?
Tried with doLast{ ... } but came to nothing
It seems, it's all due to the different lifecycle phases. You can read more about it here.
First of all, when you createe a task of type Exec or any kind without <<, everything in it's body is a task configuration and is getting executed at the configuration pahse of the build. That's why you have messages like started task getDumpFromRemotePostgres first in your output.
The second thing is that an executable you are running, is executed at the execution phase, after all task's configuration is done already. And that is why dump script started appears after all the configurations are done.
In your case, you don't need to declare a patchPostgresDumpFile as an Exec task, because you don't actually call any executable, but need to run some logic. For that, you have to move this logic to the doLast closure, to run it in the execution phase. Something like this:
task patchPostgresDumpFile() {
dependsOn getDumpFromRemotePostgres
doLast {
println 'started task patchPostgresDumpFile'
def dbDumpFileName = 'dump.sql'
File dumpFile = file(dbDumpFileName)
def line
dumpFile.withReader { reader ->
while ((line = reader.readLine()) != null) {
//parse and modify
}
println 'finished task patchPostgresDumpFile'
}
}
And note, that your message started task getDumpFromRemotePostgres doesn't actually mean that this task is running, but that it's configuring. If you want to have a messages before and after the execution, move them into the doFirst and doLast closures within a task configuration closure.
I'm trying to create a custom Gradle 4.3.1 task that will:
Run ./gradlew build which produces a build/libs/myapp.jar artifact; then
Creates a myapp-1.0.zip ZIP file whose contents include:
build/libs/myapp.jar; and
./AppGuide.md; and
./app-config.json
Here's my best attempt:
task zipMeUp(type: Zip) {
String zipName = 'myapp-1.0.zip'
doFirst {
tasks.build
}
from 'build/libs/myapp.jar'
from 'AppGuide.md'
from 'app-config.json'
into zipName
}
When I run this (./gradlew zipMeUp) I get the following output:
HarveyZ:myapp myuser$ ./gradlew zipMeUp
BUILD SUCCESSFUL in 2s
1 actionable task: 1 executed
But nothing actually seems to happen (no myapp-1.0.zip file in the directory). Any idea what the fix/solution is?
Don't use doFirst, use dependsOn
task zipMeUp(type:Zip, dependsOn :[build]) {
String zipName = 'myapp-1.0.zip'
from 'build/libs/myapp.jar'
from 'AppGuide.md'
from 'app-config.json'
version = "1.0"
baseName = "myapp"
}
I have the following:
task cloneProtobuf(type: Exec) {
workingDir "${rootProject.buildDir}/github.com/google"
commandLine 'git', 'clone', 'https://github.com/google/protobuf.git'
enabled = { -> !new File(workingDir, "protobuf/.git").isDirectory() }()
doFirst {
mkdir workingDir
}
}
Rather than explicitly setting enabled and having gradle indicate that the task was SKIPPED, I would rather have gradle consider that if the protobuf directory already exists, the task is UP-TO-DATE. How can this be done?
Instead of enable/disable a task, please register a task output, then gradle will be aware itself if a task is up-to-date or not. Please have a look at the example below which you can find helpful:
task mk(type: Exec) {
def output = project.file('mk')
outputs.dir output
commandLine 'mkdir', output
workingDir '.'
}
If you run gradle mk twice, the task will execute only once.
I have the following simple task in my build:
task generateFile << {
def file = new File("$buildDir/setclasspath.sh")
file.text = "sample"
outputs.file(file)
}
task createDistro(type: Zip, dependsOn: ['copyDependencies','packageEnvironments','jar', 'generateFile']) <<{
from generateClasspathScript {
fileMode = 0755
into 'bin'
}
}
When I run gradle clean build I see the following output:
Cannot call TaskOutputs.file(Object) on task ':generateFile' after task has started execution. Check the configuration of task ':generateFile' as you may have misused '<<' at task declaration
How do I declare the task file creation outputs as an input to the zip task while also ensuring they happen in the execution phase?
If I leave off the << then the clean task wipes the generated file before the ZIP can use it. If I keep them, I get the above error.
It's the opposite as what is being suggested in the comments. You are trying to set the outputs in execution phase. The correct way to do what you are probably trying to do is for example:
task generateFile {
def file = new File("$buildDir/setclasspath.sh")
outputs.file(file)
doLast {
file.text = "sample"
}
}
I am writing a Gradle build for a non-java project for assembling existing directories and tar archives into a .tar.gz The tar task skips if I use the definition like so:
task archive(dependsOn: 'initArchive',type: Tar) << {
baseName = project.Name
destinationDir = new File(project.buildDir.path+'/installer')
compression = Compression.GZIP
from (archiveDir)
doLast{
checksum(archivePath)
}
}
here's the console output
:jenkins-maven-sonar:archive
Skipping task ':jenkins-maven-sonar:archive' as it has no source files.
:jenkins-maven-sonar:archive UP-TO-DATE
BUILD SUCCESSFUL
Total time: 9.056 secs
When I try to use tar task as a method it fails complaining cannot find method
task archive(dependsOn: 'initArchive') << {
tar{
baseName = project.Name
destinationDir = new File(project.buildDir.path+'/installer')
compression = Compression.GZIP
from (archiveDir)
doLast{
checksum(archivePath)
}
}
}
FAILURE: Build failed with an exception.
* Where:
Build file '/home/anadi/Code/da-ci-installers/build.gradle' line: 29
* What went wrong:
Execution failed for task ':jenkins-maven-sonar:archive'.
> Could not find method tar() for arguments [build_6a2bckppv2tk8qodr6lkg5tqft$_run_closure3_closure5_closure7#4a5f634c] on task ':jenkins-maven-sonar:archive'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.
BUILD FAILED
Total time: 8.749 secs
Can we run the tar task in same way as Gradle allows running copy? In the same build I have a block like follows and I want to know if tar can used in the same way
copy {
project.logger.info("Copying bundle :: "+bundle[x])
from(rootProject.projectDir.path+"/3rd-party-tools/"+bundle[x]) {
include '**/*.*'
}
into(archiveDir)
}
if not how to make sure my build does not "skip tar" task if using the first form described above.
You have fallen for the classical mistake of configuring a task in the execution phase rather than the configuration phase. The solution is to remove the << in the first code snippet.
If you find the << (and the difference it makes) confusing, a good solution is to never use << but always the more explicit doLast {}.
There is no tar method, but it's usually better to make these things a separate task anyway. (Methods like copy should only be preferred over the corresponding task if there is a strong reason.)
I had a funny situation where I got hit by this when using doLast{} on a tar task.
It was because of a multi-project build:
build.gradle
--> sub-project
--> build.gradle
In this case if you try to have a tar or a copy task in the main build file that references something from that project(":sub-project") uses it will tempt the developer to wrap it in doLast.
For example, main build.gradle file has:
task distTar(type: Tar, dependsOn: "buildDist") {
description "Package ProjName into a Tar file"
group "Build"
baseName = 'outbasename'
archiveName = baseName + '.tar.gz'
compression = Compression.GZIP
destinationDir = file(project(":packaging").buildDir.path)
extension = 'tar.gz'
into('outdir') {
from project(":sub-project").war
}
}
So they got an error that project(":sub-project").war doesn't exist. So to get around it someone put doLast {} the task and errors went away. BAD!!
task distTar(type: Tar, dependsOn: "buildDist") {
doLast { // <-- BAD!!
description "Package ProjName into a Tar file"
group "Build"
baseName = 'outbasename'
archiveName = baseName + '.tar.gz'
compression = Compression.GZIP
destinationDir = file(project(":packaging").buildDir.path)
extension = 'tar.gz'
into('outdir') {
from project(":sub-project").war
}
}
}
Then i was left to fix it. So the correct was to do it was to add
evaluationDependsOn ":sub-project"
In the main build.gradle file. Now it knows to evaluate it. I remove the incorrect doLast{} block and now the task is no longer ignored.