Or in other words: How do I gather outputs of multiple tasks?
I have a project with a number of tasks whose output I want to include in the distribution.
I also have a task that depends on all of them.
How can I avoid listing all the tasks?
Example build.gradle:
plugins {
id 'distribution'
}
task taskA0 {
...
}
task taskA1 {
...
}
task taskA2 {
...
}
task allA {
dependsOn (taskA0, taskA1, taskA2)
}
distributions {
main {
contents {
// this works but is tedious
from taskA0
from taskA1
from taskA2
// this doesn't work, as allA doesn't have any output
from allA
}
}
}
How's this?
task allA {
dependsOn (taskA0, taskA1, taskA2)
outputs.files(taskA0, taskA1, taskA2)
}
distributions {
main {
contents {
from allA
}
}
}
Related
Consider this toy build.gradle file:
plugins {
id "java"
}
import org.gradle.internal.os.OperatingSystem;
apply plugin: 'java'
repositories {
mavenLocal()
mavenCentral()
}
def pathExists(pathname) {
// Returns true iff pathame is an existing file or directory
try {
// This may throw an error for a Windows pathname, c:/path/to/thing
if (file(pathname).exists()) {
return true;
}
} catch (GradleException e) {
// I don't care
}
if (OperatingSystem.current().isWindows()) {
try {
// If we're on Windows, try to make c:/path/to/thing work
if (file("file:///${pathname}").exists()) {
return true;
}
} catch (GradleException e) {
// I don't care
}
}
return false
}
def someVariable = "absent"
if (pathExists("/tmp")) {
someVariable = "present"
}
task someCommonTask() {
doLast {
println("Did some setup stuff: ${someVariable}")
}
}
task someATask(dependsOn: ["someCommonTask"]) {
doLast {
println("A: ${someVariable}")
}
}
task someBTask(dependsOn: ["someCommonTask"]) {
def otherVariable = someVariable == "absent" || pathExists("/etc")
doLast {
println("B: ${otherVariable}")
}
}
Is it possible to reorganize this build file so that someATask is in a.gradle and someBTask is in b.gradle? I've made some attempts to use apply from: without success and, if the answer is on this page: https://docs.gradle.org/current/userguide/organizing_gradle_projects.html it eludes me.
In the real project, there would be a couple of dozen tasks in each of those subordinate files and the goal of dividing up the build.gradle file isn't to have them be subprojcts, per se, they're just logical groupings of tasks.
This split can be done using apply from: ... to separate the files.
E.g.
build.gradle:
plugins {
id "java"
}
apply plugin: 'java'
repositories {
mavenLocal()
mavenCentral()
}
ext {
someVariable = "absent" // XXX
}
apply from: "setup.gradle" // XXX
apply from: "common.gradle" // XXX
task someATask(dependsOn: ["someCommonTask"]) {
doLast {
println("A: ${someVariable}")
}
}
setup.gradle:
task setup() {
someVariable = "present"
}
common.gradle:
task someCommonTask(dependsOn: ["setup"]) {
doLast {
println("Did some setup stuff: ${someVariable}")
}
}
# gradle someATask
> Task :someCommonTask
Did some setup stuff: present
> Task :someATask
A: present
BUILD SUCCESSFUL in 403ms
My guess, the problems you are facing are with toplevel code and not
using properties.
I have a Gradle build file which uses ProtoBuffer plugin and runs some tasks. At some point some tasks are run for some files, which are inputs to tasks.
I want to modify the set of files which is the input to those tasks. Say, I want the tasks to be run with files which are listed, one per line, in a particular file. How can I do that?
EDIT: Here is a part of rather big build.gradle which provides some context.
configure(protobufProjects) {
apply plugin: 'java'
ext {
protobufVersion = '3.9.1'
}
dependencies {
...
}
protobuf {
generatedFilesBaseDir = "$projectDir/gen"
protoc {
if (project.hasProperty('protocPath')) {
path = "$protocPath"
}
else {
artifact = "com.google.protobuf:protoc:$protobufVersion"
}
}
plugins {
...
}
generateProtoTasks {
all().each { task ->
...
}
}
sourceSets {
main {
java {
srcDirs 'gen/main/java'
}
}
}
}
clean {
delete protobuf.generatedFilesBaseDir
}
compileJava {
File generatedSourceDir = project.file("gen")
project.mkdir(generatedSourceDir)
options.annotationProcessorGeneratedSourcesDirectory = generatedSourceDir
}
}
The question is, how to modify the input file set for existing task (which already does something with them), not how to create a new task.
EDIT 2: According to How do I modify a list of files in a Gradle copy task? , it's a bad idea in general, as Gradle makes assumptions about inputs and outputs dependencies, which can be broken by this approach.
If you would have added the gradle file and more specific that would have been very helpful. I will try to give an example from what I have understood:
fun listFiles(fileName: String): List<String> {
val file = file(fileName).absoluteFile
val listOfFiles = mutableListOf<String>()
file.readLines().forEach {
listOfFiles.add(it)
}
return listOfFiles
}
tasks.register("readFiles") {
val inputFile: String by project
val listOfFiles = listFiles(inputFile)
listOfFiles.forEach {
val file = file(it).absoluteFile
file.readLines().forEach { println(it) }
}
}
Then run the gradle like this: gradle -PinputFile=<path_to_the_file_that_contains_list_of_files> readFiles
I defined two tasks in my build.gradle
task a(type: JavaExec) {
}
task b(type: JavaExec) {
}
When I execute task a, b also runs. Is this normal?
gradle a
I'm sharing my own experience for others.
I am a newbie on groovy/gradle.
What I tried to achieve was using a shared function for getting project properties.
def projectProperty = {
if (!project.hasProperty(it)) {
throw new Exception...
}
return project.getProperty(it);
}
task a(type: JavaExec) {
do some with projectProperty(a);
}
task b(type: JavaExec) {
do some with projectProperty(b);
}
And I changed like this.
task a(type: JavaExec) {
if (project.hasProperty('a')) {
do some with projectProperty('a');
}
}
task b(type: JavaExec) {
if (project.hasProperty('b')) {
do some with projectProperty('b');
}
}
I have the following in a groovy-based build script. How do I do the same in a kotlin-based script?
processResources {
filesMatching('application.properties'){
expand(project.properties)
}
}
Why not to just use "withType" ?
I just mean (IMHO)
tasks {
withType<ProcessResources> {
..
}
looks much better than
tasks {
"processResources"(ProcessResources::class) {
..
}
So,
tasks.withType<ProcessResources> {
//from("${project.projectDir}src/main/resources")
//into("${project.buildDir}/whatever/")
filesMatching("*.cfg") {
expand(project.properties)
}
}
EDIT:
With newer release you can just do:
tasks.processResources {}
or
tasks { processResources {} }
generated accessors are "lazy" so it has all the benefits and no downsides.
I think task should look like:
Edit: According this comment in gradle/kotlin-dsl repository. Task configuration should work this way:
import org.gradle.language.jvm.tasks.ProcessResources
apply {
plugin("java")
}
(tasks.getByName("processResources") as ProcessResources).apply {
filesMatching("application.properties") {
expand(project.properties)
}
}
Which is pretty ugly. So i suggest following utility function for this purpose, until one upstream done:
configure<ProcessResources>("processResources") {
filesMatching("application.properties") {
expand(project.properties)
}
}
inline fun <reified C> Project.configure(name: String, configuration: C.() -> Unit) {
(this.tasks.getByName(name) as C).configuration()
}
With updates to the APIs in newer release of the Kotlin DSL and Gradle, you can do something like:
import org.gradle.language.jvm.tasks.ProcessResources
plugins {
java
}
tasks {
"processResources"(ProcessResources::class) {
filesMatching("application.properties") {
expand(project.properties)
}
}
}
And also:
val processResources by tasks.getting(ProcessResources::class) {
filesMatching("application.properties") {
expand(project.properties)
}
}
I have two gradle files, setup.gradle and tests.gradle; each having two gradle tasks of custom type 'EMTest';
test.gradle applies the 'setup.gradle' as apply from: 'setup.gradle'
I want to configure all tasks of type EMTest; For that I added below code at end of tests.gradle
tasks.withType(EMTest) {
println it.name
}
But this only prints the name of tasks in the tests.gradle;
When I run
tasks.all {
println it.name + " " + it.class
}
It however lists the tasks name defined in setup.gradle and the type as EMTest_Decorated (for all 4 types)
NOTE: I use gradle 1.11 (no control over upgrade); What is the issue here ?
UPDATE On 09/Jun
Here is the main file :
apply plugin: 'java';
apply plugin: 'maven'
apply from: 'emcpsrvs_3n_setup.gradle'
buildscript {
repositories {
maven {
url = "${artifactory_contextUrl}/repo" }
}
dependencies {
classpath group:"com.mycompany.myprod.mymodule", name: "TestInfraPlugin", version: "${testinfraVersion}", transitive: true
classpath group: 'org.codehaus.jackson', name: 'jackson-mapper-asl', version:'1.9.13'
classpath group: 'com.mycompany.myprod', name: 'common',version:'0.1'
classpath group: 'org.codehaus.jackson', name: 'jackson-core-asl', version:'1.9.13'
classpath group: 'com.oracle.weblogic',name: 'jettison-1.1', version: '12.1.2-0-0'
}
}
repositories {
/* To check if the jar is available in local maven repository */
mavenLocal()
maven {
url = "${artifactory_contextUrl}/repo"
}
}
apply plugin: 'TestInfraPlugin'
import com.mycompany.myprod.gradle.testinfra.tasks.EMTest;
repositories {
maven {
url = "${artifactory_contextUrl}/repo"
}
}
dependencies {
testConfig group:'com.mycompany.myprod',name:'ui-integ-tests', version: '1.+'
testConfig group: 'com.mycompany.myprod', name: 'emaas-platform-tenant-sdk', version: '0.1+'
}
task unitTests(type: EMTest){
}
// Three tests are disabled due to JIRA-900
task tenantMgmtUITests(type: EMTest,dependsOn: [cleanSmall_deploy_3n_block,small_deploy_3n_block]) {
useWebdriver = true
small_deploy_3n_block.mustRunAfter ([cleanSmall_deploy_3n_block])
options.suiteXmlBuilder().suite('parallel': 'none','name': 'TenantManagementUI') {
test('name': 'TenantManagementUI') {
classes([:]) {
'class'('name': 'com.mycompany.package.MyTest')
}
}
}
}
small_deploy_3n_cleanup.mustRunAfter ([tenantMgmtUITests])
task emcpsrvs_tenant_mgmt_ui_3n(dependsOn: [tenantMgmtUITests,small_deploy_3n_cleanup])
Here is the 'emcpsrvs_3n_setup.gradle' which is being applied above
buildscript {
repositories {
maven {
url = "${artifactory_contextUrl}/repo"
}
}
dependencies {
classpath group: 'com.mycompany.myprod.emdi', name: 'TestInfraPlugin', version: "${testinfraVersion}", transitive: true
}
}
apply plugin: 'TestInfraPlugin'
repositories {
maven {
url = "${artifactory_contextUrl}/repo"
}
}
import com.mycompany.myprod.gradle.testinfra.tasks.EMTest;
ext.integDeployVersion='1.1+'
dependencies {
testConfig group: 'com.mycompany.myprod.test', name: 'smalldeployment', version: "${integDeployVersion}"
}
/* Setup EMaaS Small Deployment */
task small_deploy_3n_block(type: EMTest) {
outputs.upToDateWhen { false }
onlyIf {!System.env.SMALLDEPLOY_IGNORESETUP}
options.suiteXmlBuilder().suite('name': 'setup_3n_env') {
test('name': 'emaas_setup_small_deploy') {
classes([:]) {
'class'('name': 'mycompany.sysman.test.emaas.integ.EmaasSmallDeploy3n') {
methods([:]) {
'include' ('name': 'setupEmaasSmallDeploy')
}
}
}
}
}
useWebdriver = true
useRestAssured = true
}
/* Cleanup EMaaS Small Deployment */
task small_deploy_3n_cleanup(type: EMTest) {
onlyIf {!System.env.SMALLDEPLOY_IGNORESETUP}
options.suiteXmlBuilder().suite('name': 'setup_3n_env') {
test('name': 'emaas_setup_small_deploy') {
classes([:]) {
'class'('name': 'mycompany.sysman.test.emaas.integ.EmaasSmallDeploy3n') {
methods([:]) {
'include' ('name': 'logCollectionAndPostCleanup')
}
}
}
}
}
mustRunAfter ([small_deploy_3n_block])
}
And finally here is the snippet from TestInfraPlugin.groovy (The gradle plugin):
logger.debug "Configuring the EMTest task with default values."
project.afterEvaluate {
project.ext.testClassesDir = new File(project.properties['emdi.T_WORK'] + '/testClasses')
def testTasks = project.tasks.withType(EMTest)
if (testTasks != null && testTasks.size() == 0) {
logger.info "There are no tasks of type EMTest."
return
}
def extractTask = project.tasks.findByPath('extractTestClasses') ?:
project.task('extractTestClasses', type: ExtractConfiguration) {
configuration = project.configurations.testConfig
to = project.testClassesDir
}
/*
* 1. Adding the 'extractTask' to all EMTest, to ensure that 'extractTask' is run before any 'EMTest'.
* 2. For lazy evaluation of lrgConfig, we are NOT running the task here, but just adding as dependent task.
*/
testTasks.each { task ->
logger.debug "Adding dependsOn extractTask for task: ${task.name}"
task.dependsOn extractTask
}
} // end afterEvaluate
}
What is afterEvaluate{} block doing:
It checks if there are any tasks of type EMTest and if there are creates a task to extract a configuration (named testConfig). This extract task is added as dependency on all the tasks of type EMTest such that the extract task run as first task before running any other task.
What is happening
The extractTestClasses task is added as dependency to ONLY the two tasks unitTests and tenantMgmtUITests and thus the small_deploy_3n_block is getting executed before extractTestClasses resulting setup to fail, which in turn results test to fail.
When you reference tasks with no object it is implicitly using project.tasks.
tasks.withType(EMTest) {
println it.name
}
That is why you'll only get tasks from your current project.
To include all projects you can use:
allprojects {
tasks.withType(EMTest) { println it.name }
}
In this closure you are referencing it.tasks, where it loops over every individual project.
This will not work as expected since Gradle will probably not have loaded all of your sub-projects at this point, and not have reached every task define found during the full configuration phase. Therefore you must define the closure to run after all the projects are fully evaluated:
allprojects {
afterEvaluate {
tasks.withType(EMTest) { println it.name }
}
}