I have a problem in organizing my build in manageable way. In my gradle build there are large number of tasks since for each specific work element, gradle require a task of that type. For ex: to copy a file gradle needs Copy task type, and to zip a file gradle needs Zip task type, to delete few files gradle needs Delete task type. Javadoc , Jar , Ear etc also require their task types. Eventually build has large number of tasks and to run all these in specific order I have to use
taskA.mustRunAfter taskB
Applying these rules to for all the tasks make build more unmanageable. Is there a way to program all those work elements ( i.e, copy files, zip files, create a jar and javadoc, etc ) in one task ( like provided in ant )
Is there a reason you aren't using dependsOn? You'll need to specify a lot more mustRunAfter statements to achieve the same ordering that dependsOn can get you. At least, that's my experience. Only use mustRunAfter when you have an ordering issue that explicitly is not a dependency.
For example, we use gradle to install and uninstall testing environments. If both are run than install.mustRunAfter uninstall (to do a refresh) but clearly install doesn't always actually depend on uninstall so we use mustRunAfter.
That said, one way you can run many elements in one task is to use a series of ant goals.
task lotsOfStuffAtOnce {
ext.paulaProp = 'brillant'
} << {
ant.copy ...
ant.zip ...
ant.jar ...
}
I still think that overall the preferred philosophy is to use tasks with a proper mix of dependsOn and mustRunAfter, but different situations call for different results so who am I to judge? If you're looking to accomplish several steps all as one unit of work (one task) the above code is the cleanest way I can think of doing it.
If you don't have experience calling ant from gradle, the user docs will show you that it's a pretty mechanical transformation.
Related
I am writing a plugin for Gradle. I spent a lot of time and do not know how to solve the problem correctly.
I need automatically run build task before running my own task. DependsOn is not suitable as it tightly binds tasks, and I want to take advantage of Gradle's input, output principle. What commonly can be triggered in project as input paramether for my task for such behaviours?.
gradle customTask
result:
:build
+----:customTask
My castomTask uses project.configuration() and I try mark it as #InputFile but it dosent trigger build task.
I am specifically looking to see if you can create a task that depends on the internal "assembleRelease" gradle task in a multi-project environment. There are many, many independent pieces that need building here, so changing in all of the build files is not optimal.
My basic situation is looking for a task, for example "release" that can call :a:assembleRelease, :b:assembleRelease and :c:assembleRelease without having to type in the names of all of the projects to begin with.
I have attempted the following task definition in the root project build.gradle.kts:
tasks.register("release") {
finalizedBy(tasks.getByPath("assembleRelease"))
}
as well as tasks.getByName, and trying to go through the gradle taskGraph. The task graph doesn't appear to be the right path at all, but I'm not sure if this is a possible task or is against the conventions of gradle to do things this way.
You would just define a task that depends on multiple other tasks.
Assuming the root project:
tasks {
register("release") {
dependsOn(
named(":a:assembleRelease"),
named(":b:assembleRelease:),
named(":c:assembleRelease")
)
}
}
You can add essentially an infinite amount of task dependencies. See Authoring Tasks for more details, specifically Adding dependencies to a task.
We have a directory structure like so
java
build/build.gradle (This does NOT exist yet, but we want this)
servers
server1/build.gradle
server2/build.gradle
libraries
lib1/build.gradle
lib2/build.gradle
We have 11 servers and 14 libraries with varying uses of dependencies. EACH server is a composite build ONLY depending on libraries (we don’t allow servers to depend on each other). In this way, as our mono-repo grows, opening up server1 does NOT get slower and slower as more and more gradle code is added(ie. gradle only loads server1 and all it’s libraries and none of the other libraries OR servers are loaded keeping things FAST).
Ok, so one problem we are running into is duplication now which is why we need build/build.gradle file AND we want EVERY module in our mono repo to include that somehow for a few goals(each goal may need a different solution)
GOAL 1: To have an ext { … } section containing a Map of Strings to gradle dependencies much like so
deps = [
'web-webserver': "org.webpieces:http-webserver:${webpiecesVersion}",
'web-webserver-test': "org.webpieces:http-webserver-test:${webpiecesVersion}",
'web-devrouter': "org.webpieces:http-router-dev:${webpiecesVersion}"
]
In this way, we want ALL our projects to them import dependencies like so
compile deps['web-webserver']
GOAL 2: We want to 'include' a standard list of plugins so we are versioning all gradle plugins the same across the repo. While the above configures all jars to avoid jar hell in a mono-repo, we would like to do the same with just this section
plugins {
id 'com.github.sherter.google-java-format' version '0.9'
}
Of course, it each project may also want to add a few more plugins OR even not depend on this section(in case of an emergency and trying to just get the job done).
GOAL 3: We want checkstyle configuration (or any plugin config) to be defined the SAME for all projects (eventually!!!). We would like the checkstyle gradle to live in a common area but have all libraries somehow pull it in. Again, it would be nice for it to be optional in that, I can pull the gradle section into my build.gradle OR can create a new one in case of emergencies so I don't have to fix all projects in the monorepo right away.
IDEALLY, perhaps I kind of want configuration injection where when I run server1/build.gradle, it actually runs java/build/build.grade as it’s parent somehow but with overrides (IF I declare 'extends xxx.gradle' maybe) then all libraries it uses also use java/build/build.gradle as their parent. I am not sure this is possible or feasible. I am pretty sure 'extends xxx' doesn't exist in gradle.
Are any of these GOALS possible?
thanks,
Dean
I have been working on a monorepo with the exact same requirement as you, using gradle composite builds as well. The way we have solved this problem is by using pre compiled plugins
You need to do a new gradle project with only the code you want to share. This will create a plugin, that you can just add as a composite build and apply to the other projects.
I'm a bit confused by why you don't just use a "standard" gradle top level build file and compose the others as subprojects.
This solves all 3 of your goals
If you are concerned by build speed, you can target each server individually simply by running
./gradlew :server1:build
But if you are not able to do this for some reason you can use the apply from: syntax as described here
I have a pretty large git project that I have a large single build.gradle that defines the sub projects.. For the first few years... no problem. After the first few years it gets kind of slow on the Gradle Command line configuration step because of the number of sub projects.
For example, when I run a gradle command I see a pause on:
> Configuring > 0/77 projects > ...
The pause gets bigger with each project added it seems like.
Another big problem is when i go to import the project or refresh the gradle deps with intelliJ, it takes a long time on the "analyzing dependencies" really delaying development as we need to have a pretty large wait everytime we sync dependencies.
What do people do in the field typically to get passed this? I have had to just eat the delay which is painful.
Do people usually split up the build.gradle file into unrelated chunks so that you get a network of multi-project builds? Or should there be a way to make that not be so damn slow?
I have analyzed the dependencies, all pretty simple actually.
Ondemand, Daemon and, parallel deploys happen in some limited way.
But the same problem stands...
What other tricks are out there? My plan is to separate the main parts of my code in sections and load them independently so that the gradle commands and intelliJ refreshes are smaller UP-TO-DATE check and configuration setup. It would be nice if we didn't have to split up our build.gradle file just because of sub-project limit.
Following our discussion in the comments of your question, it is fairly clear that you don't want to go down the route of publishing some of the interface jars to a repository to remove some of the subprojects.
Given that, here are a few suggestions. I know you have already tried some of them, but there are a few here that you haven't mentioned above which might help you slightly:
Profiling and dry run
First of all, try using the following command:
./gradlew {task} --dry-run --profile
This will perform a dry run of the task with name {task} and create a profile html file under $buildDir/reports/profile. You can use this to see which tasks are taking up the majority of the time across congifuration, dependency resolution and task execution. If you can see that one task is taking a particularly long time, have a look at it and see if you can cut down what is happening in that task.
Configuration on demand
To reduce configuration time, try enabling configuration on demand in your gradle.properties file like so:
org.gradle.configureondemand=true
This will mean that instead of configuring everything, gradle will attempt to only configure the necessary projects required to run a task.
Use the gradle daemon
The gradle daemon is a background process which doesn't exit when your gradle task finishes, and can be reused. This limits the amount of overhead of starting a new gradle jvm process every build. To activate the gradle daemon, add the following line to your gradle.properties file:
org.gradle.daemon=true
Use the most recent possible versions of Gradle and Java
Something is simple as upgrading your gradle version can improve speed. If you haven't upgraded for a while, or are using a fairly old version, consider upgrading. There may be some initial pain in upgrading, but it will be worth it in the long run. The same applies for upgrading java versions. Obviously, this is not always possible due to other constraints, but try and use the most recent java version available to you.
Avoid using dynamic dependencies
Gradle allows your to specify ranges for dependencies that your project is able to use like so:
dependencies {
compile "commons-lang:commons-lang:2.+"
}
This means that gradle will always try and find the latest version of the dependency that matches the constraint. This flexibility comes at a performance cost as gradle will have to go online to check what version to use. For this reason, and a few others that I won't go into here, it is a good idea to specify the version of a dependency explicitly.
Parallelize the build
This allows unrelated subprojects to be built in parallel, which speeds up the build process. To enable this, set the following in gradle.properties:
org.gradle.parallel=true
Ignore non-relevant tasks
Say you want to do a build, but don't necessarily want to run all of the tests again. You can run the following task, and the test task will not be run. This can significantly increase build time:
./gradlew build -x test
EDIT -- Following some further research, and after finding this answer, I had further suggestions.
Disable parts of the build using properties
As shown in the linked question, if you have a project and you only want to run the functional tests under certain circumstances, you can add the following to the root build.gradle file:
project('functional-tests') {
test {
onlyIf {
project.hasProperty("functionalTests")
}
}
}
This means that functional tests will only be run if you run a task like so:
./gradlew test -PfunctionalTests
Disable certain subprojects in the settings.gradle
You could also disable certain subprojects in the settings.gradle like so:
if (someBoolean) {
include 'functional-tests'
}
First off, I have no code to show since I'm completely stumped with this one. This is bad form for an SO question so I really do apologize - I worked entire yesterday on the related build script but couldn't get anything useful to show for this.
I am currently working on a build script that as part of the jar task (or rather its doLast {} closure) would verify the current JAR against previous published JAR from my own Artifactory using the SemVer API. Everything else that I have works except for actually downloading the previous version of the project; I can't seem to be able to devise a working script.
My approach so far was based on the reasoning that as Gradle uses Ivy as its dependency management system I should be able to just call some of Ivy's Ant tasks with the right parameters - the same as current project so I actually have access to group, name and current version easily - and then get the path to downloaded artefact file and use it as input for the aforementioned SemVer API. Being a bit of Gradle newbie and not actually have used Ivy for a few years even my struggle revealed to me that at this point I have no idea how to really do this in a clean way. One of my major hurdles with this has also so far been that Gradle's documentation is too extensive making it difficult to find things when I don't even remember the right terms for certain bits I want to have.
As I'm not providing any code/build script samples of what I have so far, I don't mind if your answers are just nudges to right direction.
Gradle has not used Ivy internally for some time, so I doubt very much if your approach of using Ivy ant tasks will work.
I would probably do this in a separate task, rather than add actions to the jar task. There is probably a more elegant way, but you could try something like:
configurations { lastPublishedVersion }
configurations.lastPublishedVersion.transitive=false
dependencies {
lastPublishedVersion group: group, name: name, version: "+"
}
task checkSemVer {
dependsOn jar
dependsOn configurations.lastPublishedVersion
doLast {
println configurations.lastPublishedVersion.getSingleFile()
}
}
PS If you get this working it would make for a very interesting gradle plugin if you wanted to release it :)