I want to analyze Gradle cache misses for my custom NpmTask.
But the only difference I see in the log is in the hash of the NpmTask:
Appending implementation to build cache key: com.github.gradle.node.npm.task.NpmTask_Decorated#0da0ee70c0950afa6ed5b5d984cb5c41
Appending additional implementation to build cache key: com.github.gradle.node.npm.task.NpmTask_Decorated#0da0ee70c0950afa6ed5b5d984cb5c41
All other fingerprints are identical, i.e. for
args
environment
ignoreExitValue
npmCommand
$1
$2
My task is defined like this:
task taskName(type: NpmTask) {
mustRunAfter(otherTask)
inputs.dir(srcDir)
inputs.files(srcFiles)
args = ['run-script', 'script']
outputs.dir (outDir)
outputs.files file(outFile)
outputs.cacheIf { true }
}
How can I find out what causes the cache miss?
Related
I have a gradle script in which I configure a plugin (in my case ospackage but I guess the same would apply to another plugin) using a variable as per:
ospackage {
...
version project.ext.my_version
...
}
This variable is first initialized and then is updated using a task that I call first in my build script:
ext {
...
my_version = "XXX"
...
}
task init{
group 'ho'
description 'get HO Version'
doLast {
...
project.ext.my_version = getParameter("VERSION")
...
}
}
the problem is that the plugin (in my case ospackage) always consider the initial value "XXX" and not the correct one that was set by executing the init task.
I know it has something to do with configuration and execution phase but still I cannot find a workaround to do what I want.
For info, I also tried to create a task like the one below but it also fail as it seems that buildDeb task does not overwrite ospackage version parameter
buildDeb {
doLast {
...
version project.ext.my_version
link('/usr/bin/aa', '/usr/bin/aa.sh')
...
}
}
I also tried to put at the end of my file something like:
ospackage.dependsOn("init")
but the problem is that ospackage is not recognized as a task
Thank you in advance for your help.
It looks to me like the essence of your question revolves around on-demand values. My understanding is that you would like to set a version number during the configuration phase and use that value during the execution phase to set a package version using the ospackage plugin.
The issue is that the ospackage documentation only provides examples (to date) that setup the package constants during the configuration phase. Obviously that won't work because it is the same time you are setting your version (something that must be able to be done in parallel). You have the right idea with doLast. I found that some things from the ospackage cannot go in "doLast" blocks like packageName (if you have more than one of the same package/task type), so I just put the things that require on-demand evaluation in that block (the version, because we need its evaluation delayed until the execution phase).
My solution was to create a variable that holds the function that resolves the version.
def versionForRpm = { -> project.version }
Create a configuration block
configurations.ext {
version = versionForRpm
...
}
This is an example of an on-demand value (aka lazily-evaluated value).
task someRpmBuild(type: Rpm) {
// all package configs that require evaluation during execution phase (lazy)
doLast {
version = configurations.ext.version
requires("someotherpackageinthisbuild", configurations.ext.version(), 0)
}
// all package configs that may be evaluated during the configuration phase
release = configurations.ext.release
packageGroup = configurations.ext.packageGroup
license = configurations.ext.license
packager = configurations.ext.packager
user = configurations.ext.user
distribution = configurations.ext.distribution
vendor = configurations.ext.vendor
url = configurations.ext.url
os = configurations.ext.os
buildHost = configurations.ext.buildHost
epoch = configurations.ext.epoch
arch = configurations.ext.arch
}
Note that configurations.ext.version will be "called" automatically in the execution phase. I needed to explicitly call it when used as an argument in requires, however.
according to the documentation, the task type is Deb:
task fooDeb(type: Deb) {
packageName // Default to project.name
packageDescription // Defaults to project.description
version // Version field, defaults to project.version
arch // Architecture, defaults to "all". E.g. "amd64", "all"
multiArch // Configure multi-arch behavior: NONE (default), SAME, FOREIGN, ALLOWED (see: https://wiki.ubuntu.com/MultiarchSpec )
release // DEB Release
epoch // Epoch, defaults to 0
user // Default user to permission files to
permissionGroup // Default group to permission files to, "group" is used by Gradle for the display of tasks
packageGroup
buildHost
license
packager
distribution
vendor
url
signingKeyId
signingKeyPassphrase
signingKeyRingFile
sourcePackage
provides
uid // Default uid of files
gid // Default gid of files
createDirectoryEntry // [Boolean]
maintainer // Defaults to packager
uploaders // Defaults to packager
priority
summary
conflicts
recommends
suggests
enhances
preDepends
breaks
replaces
}
where:
version Version field, defaults to project.version
might give the RPM plugin a try.
I was able to solve the issue i had, setting the destination for the ospackage copy destination to a calculated value by using
configurations.ext{
mydestdir = ""
rpmVersion = "1"
releaseNumber = "1"
}
task init{
group 'ho'
description 'get HO Version'
doLast {
...
configurations.ext.mydestdir = "/store/tmp/+getSubDir()"
configurations.ext.rpmVersion = "123"
configurations.ext.releaseNumber = "456"
...
}
}
task fooRpm(type: Rpm) {
dependsOn init
...
doLast(){
version = configurations.rpmVersion
release = configurations.releaseNumber
}
from(project.tempDir) {
into configurations.mydestdir
fileMode = 0644
user = "nobody"
permissionGroup = "nobody"
}
}
I think you'll have use type Deb, and make some changes, but this should speed up your build, and you can verify results by adding --scan before and after making these changes.
I'm trying to configure a Zip task based on one of the property inside sub-projects, but the property is not yet accessible at the time of configuring the task. For instance, I want to exclude all my projects that has toexclude = true from my zip file. So, the build.gradle of the sub-projects that I want to exclude starts with this:
ext.toexclude = true;
...
And my main build.gradle has this task:
task zipContent (type: Zip){
def excludedProjects = allprojects.findAll{Project p -> p.toexclude == true}.collect{it.name}
println excludedProjects
destinationDir = "/some/path"
baseName = "myFile.zip"
exclude excludedProjects
from "/some/other/path"
}
The problem is that excludedProjects is always empty. Indeed, when I am executing the task, I can see []. I believe this is due to the fact that the property that I set in the subproject's build.gradle is not available at the moment the task is configured. As a proof, if I replace the first line of the task by this:
def excludedProjects = allprojects.collect{it.name}
The task prints out all of my project's name, and the zip contains nothing (which means the problem is in the p.toexclude == true).
Also, if I try this:
task zipContent (type: Zip){
def excludedProjects = []
doFirst{
excludedProjects = allprojects.findAll{Project p -> p.toexclude == true}.collect{it.name}
println "IN DOFIRST"
println excludedProjects
}
println "IN TASK CONFIG"
println excludedProjects
destinationDir = "/some/path"
baseName = "myFile.zip"
exclude excludedProjects
from "/some/other/path"
}
The task prints out IN TASK CONFIG followed by an empty array, then IN DOFIRST with the array containing only the subprojects that I set ext.toexclude == true.
So, is there a way to get the properties of the sub-projects at configuration time?
Well, the crucial question is: At which point of the build is all necessary information available?
Since we want to know each project in the build, where the extra property toexclude is set to true and it is possible (and by design) that the property is set via the build script, we need each build script to be evaluated.
Now, we have two options:
By default, subprojects are evaluated after the parent (root) project. To ensure the evaluation of each project, we need to wait for the point of the build, where all projects are evaluated. Gradle provides a listener for that point:
gradle.addListener(new BuildAdapter() {
#Override
void projectsEvaluated(Gradle gradle) {
tasks.getByPath('zipContent').with {
exclude allprojects.findAll { it.toexclude }.collect{ it.name }
}
}
})
Gradle provides the method evaluationDependsOnChildren(), to turn the evaluation order around. It may be possible to use your original approach by calling this method before querying the excluded projects. Since this method only applies on child projects, you may try to call evaluationDependsOn(String) for each project in the build to also apply for 'sibling' projects. Since this solution breaks Gradle default behavior, it may have undesired side effects.
Just define excludedProjects outside the task
def excludedProjects = allprojects.findAll{Project p -> p.toexclude == true}.collect{it.name}
task zipContent (type: Zip){
destinationDir = file("/some/path")
baseName = "myFile.zip"
exclude excludedProjects
from "/some/other/path"
}
You can call evaluationDependsOnChildren() in the root project so that child projects are evaluated before the root
Eg
evaluationDependsOnChildren()
task zipContent (type: Zip) { ... }
Another option is to use an afterEvaluate { ... } closure to delay evaluation
Eg:
afterEvaluate {
task zipContent (type: Zip) { ... }
}
I created task which depends on build.
task packageJar(dependsOn: build, type: JavaExec) {
main = 'com.xxxx.util.KiePackageCreator'
classpath = sourceSets.main.runtimeClasspath
}
But build task invokes other tasks like checkstyle, test and etc.
How to exclude them?
I can do it through console -x but how to do it inside task?
You can simply disable the tasks, by setting it's enabled property to false in the root of the script:
test.enabled = false
But in that case, those tasks won't ever run. If you just need them to not running if some other task is called, then you have to use execution graph:
gradle.taskGraph.whenReady {
taskGraph ->
if (taskGraph.hasTask(packageJar)) {
test.enabled = false
}
}
But not sure at the moment, whether it is possible to change this property when the graph is ready. If not, then you can make a variable and in the tasks, you want to exclude, add the doFirst block, which will throw the StopExecutionException according to this variable value.
We have an optional gradle task docker that depends on task war, which if executed, needs a war file generated with an extra file in it. This extra file can be added to the resources within the processResources task (or potentially directly in the war task). However, the corresponding code block must not run if task docker has not been requested and will not be run.
We need a correct condition in the following block checking if task docker is in the pipeline:
processResources {
if (/* CONDITION HERE: task docker is requested */) {
from ("${projectDir}/docker") {
include "app.properties"
}
}
}
task docker(type: Dockerfile) {
dependsOn build
...
Clarification: processResources is a standard dependency of the war task and the latter is a standard dependency of the build task. processResources is always executed on build, with or without the docker task to collect resources for assembling the war and may not be fully disabled in this case. One could move the code in question to a separate task dependent on docker and working on the output directory of processResources, yet before war is run, however, such a construct will result in much less clarity for such a simple thing.
You can simply add additional dependency to your docker task, to make it relying not only on build task, but also on processResources. In that case, your processResources task will be called only if docker should be executed.
Another solution is to use TaskExecutionGraph. This let you initialize some variable, which could tell you, whether or not some task will be executed. But you have to understand, that graph is prepared only after all the configuration is done and you can rely on it only during the execution phase. Here is a short example, how it could be used:
//some flag, whether or not some task will be executed
def variable = false
//task with some logic executed during the execution phase
task test1 << {
if (variable) {
println 'task2 will be executed'
} else {
println 'task2 will not be executed'
}
}
//according to whether or not this task will run or not,
//will differs test1 task behavior
task test2(dependsOn: test1) {
}
//add some logic according to task execution graph
gradle.taskGraph.whenReady {
taskGraph ->
//check the flag and execute only if it's true
if (taskGraph.hasTask(test2)) {
variable = true
println 'task test2 will be executed'
}
}
Furthermore, you can try to configure your custom task to make it disabled by setting is enabled property to false, if docker task is not in the execution graph. In that case you don't have to provide some flags and logic in execution phase. Like:
task test1 {
//some logic for execution
doLast {
println "execute some logic"
}
}
task test2(dependsOn: test1) {
}
gradle.taskGraph.whenReady {
taskGraph ->
if (!taskGraph.hasTask(test2)) {
//Disable test1 task if test2 will not run
test1.enabled = false
}
}
But it'll be impossible to run this custom task separately without some additional configuration.
This Android example build file contains the snippet
buildTypes {
release {
minifyEnabled true
}
But since Gradle always executes all configure statements in the build script, won't this always set minifyEnabled to true even for a debug build?
I wish Gradle would let me set a variable to true in one task and false in another and configure things differently, but the only way I've been able to do it is by waiting for taskGraph.whenReady. This is how it normally works:
def myBool = false
task runs {
myBool = true
}
task doesNotRun {
myBool = false
}
task whoWins(dependsOn: runs) {
doLast { println "myBool is ${myBool}" }
}
gradle whoWins
:runs
:whoWins
myBool is false
The configuration step is to help Gradle build the task execution graph (what will actually be run), which is why all the configuration code is executed. What you are describing sounds like a circular task dependency, where a task depends on a variable being set by a task that in turn depends on the first task.
Task A -> depends on Task B -> depends on variable from Task A
What you are probably looking for is the ability to configure a task based on the execution graph. See the Gradle user guide, section 6.13 Configure by DAG. Using this allows you to break your circular dependency.
Task A -> depends on Task B -> depends on gradle.taskGraph.whenReady
or alternatively you can create a new task, Task C, that handles setting the variable based on the task graph.
def myBool = false
task runs {}
task doesNotRun {}
task whoWins(dependsOn: runs) {
doLast { println "myBool is ${myBool}" }
}
task taskC << {
if (gradle.taskGraph.hasTask(runs)) {
myBool = true
} else if (gradle.taskGraph.hasTask(doesNotRun)) {
myBool = false
}
}
runs.dependsOn(taskC)
doesNotRun.dependsOn(taskC)
Results:
$ gradle whois
:taskC
:runs
:whoWins
myBool is true