Setting up a multi-project Gradle build - gradle

I'm setting up a multi-module Gradle build for a legacy system at work (replacing the current Ant build). However, I'm new to Gradle, and I'm not sure what's the best way to do it. And I want to do it right, because this build script will be around for a long time. I have found a way to do things that works, but when I google around and read answers on StackOverflow, I see people using a different way, which --in my case-- doesn't work. But maybe I'm doing something wrong. I've also been reading in the Gradle in Action book, but haven't found this particular issue there.
I have a build.gradle file in the root of my project, with a bunch of subdirectories that each contain a sub-project. Most of these are regular Java projects, but there are some Wars and Ears in there, too, which require some special packaging love. Each sub-project has its own build.gradle file which, at this point, only contains a list of dependencies, nothing more.
The build.gradle file in the root of my projects looks something like this (I left out the War stuff for brevity):
configure(javaProjects()) {
jar.doFirst {
manifest {
...
}
}
}
configure(earProjects()) {
apply plugin: 'ear'
ear.doFirst {
manifest {
...
}
}
}
Set<String> javaProjects() {
subprojects - earProjects()
}
Set<String> earProjects() {
subprojects.findAll { it.name.endsWith(".ear") }
}
The only reason why I'm doing things this way, is because it was the first solution I tried that I could get to work in my situation. Now that the script is growing, though, it starts to feel a little clunky. Also, the doFirst thing seems a little awkward.
But when I look on StackOverflow, I see recommendations of using constructs like this:
allprojects {
tasks.withType(Jar) {
manifest {
...
}
}
tasks.withType(Ear) {
manifest {
...
}
}
}
This seems much nicer, but I don't seem to be able to rewrite my script in that way. I get errors like this one:
Cannot change configuration ':some.subproject:compile' after it has been resolved.
I don't know what to do about this error, and I can't seem to google it either, for some reason.
So, my question is: have I indeed been doing things the wrong way? Or rather: in a way that is not idiomatic Gradle? For the sake of maintainability, I'd like to do things as idiomatically as possible. And if so: what can I do about the error message?

In general you should do things like described in your second snippet:
allprojects {
tasks.withType(Jar) {
manifest {
...
}
}
}
But there are some limitations where this isn't sufficient. The error message you get means that you modify the compile configuration AFTER the configuration is already resolved. That happens for example when you do something like
configurations.compile.files.each...
during the configuration phase (e.g. in your manifest block like seen above) and in another place (e.g. in one of your subprojects build.gradle files):
dependencies{
compile "org.acme:somelib:1.2.3"
}
The other problem with this is, that you resolve the compile dependencies every time you invoke your build script, even when no jar task is triggered.
The suggested workaround is to replace
tasks.withType(Jar) {
manifest {
...
}
}
with
tasks.withType(Jar) {
doFirst{
manifest {
...
}
}
}
That means that resolving the configuration is postponed to the execution phase of gradle and really just triggered when needed.
When you configure a project in a multiproject build you can think of that each snippet that is part of the whole configuration. you're not configuring the project 'twice' but you configure different aspects of the project at different places.
This is a known limitation of the current gradle configuration model.
You can still use
configure(earProjects()) {
}
that doesn't matter here. IMO it is just a matter of personal preference. The gradle project itself uses 'configure'.
Personally I prefer to apply the plugins like Ear or war on the projects build.gradle file to mark a project as a ear/war project.
To share common configurations among all ear projects, you could have something like this in your root build.gradle file:
allprojects{
plugins.withType(EarPlugin){
// only applied if it is a ear project
// put ear specific logic here
}
}

Related

How can I access the dependencies of an application from within the build file of a dependency embedded in the application?

I have a Gradle-based library that is imported as a dependency into consuming applications. In other words, an application that consumes my library will have a build.gradle file with a list of dependencies that includes both my library as well as any other dependencies they wish to import.
From within my library's build.gradle file, I need to write a Gradle task that can access the full set of dependencies declared by the consuming application. In theory, this should be pretty straightforward, but hours of searching has not yielded a working solution yet.
The closest I've come is to follow this example and define an additional task in the library's build.gradle file that runs after the library is built:
build {
doLast {
project.getConfigurations().getByName('runtime')
.resolvedConfiguration
.firstLevelModuleDependencies
.each { println(it.name) }
}
}
I keep getting an error message that the 'runtime' configuration (passed into getByName and referenced in the Gradle forum post I linked) cannot be found. I have tried other common Gradle configurations that I can think of, but I never get any dependencies back from this code.
So: what is the best way to access the full set of dependencies declared by a consuming application from within the build file of one of those dependencies?
Okay, I mostly figured it out. The code snippet is essentially correct, but the configuration I should have been accessing was 'compileClasspath' or 'runtimeClasspath', not 'runtime'. This page helped me understand the configuration I was looking for.
The final build task in the library looks roughly like this:
build {
doLast {
// ...
def deps = project.getConfigurations().getByName('compileClasspath')
.resolvedConfiguration
.firstLevelModuleDependencies
.each {
// it.name will give you the dependency in the standard Gradle format (e.g."org.springframework.boot:spring-boot:1.5.22.RELEASE")
}
}
}

Gradle Kotlin DSL: get sourceSet of another project

Currently, we're trying to migrate our existing build.gradle scripts to the new Kotlin DSL. Right now, we are struggling with the jar task configuration.
Our project is a simple multi-project. Let's say we've Core and Plugin and Plugin uses classes from Core. Now, when building Plugin, the target jar should include any used classes from Core.
This is how it looked like before:
jar {
from sourceSets.main.output
from project(':Core').sourceSets.main.output
}
And this is the current solution we've with Kotlin DSL:
val jar: Jar by tasks
jar.apply {
from(java.sourceSets["main"].allSource)
from(project(":Core").the<SourceSetContainer>()["main"].allSource)
}
However, the above example just gives me an Extension of type 'SourceSetContainer' does not exist. Currently registered extension types: [ExtraPropertiesExtension] error. I've also tried other code snippets I've found, but none of them have been working so far.
I have also tried this (like suggested in the first answer):
val jar: Jar by tasks
jar.apply {
from(java.sourceSets["main"].allSource)
from(project(":Core").sourceSets.getByName("main").allSource)
}
But then the IDE (and also the jar task) argues that sourceSets is not available: Unresolved reference. None of the following candidates is applicable because of receiver type mismatch: public val KotlinJvmProjectExtension.sourceSets: NamedDomainObjectContainer<DefaultKotlinSourceSet> defined in org.gradle.kotlin.dsl.
I hope that someone can help us, because it is very frustrating to spend hours in configuration instead of writing any usefull code.
Thank you very much in advance.
You can access the SourceSetContainer by
project(":Core").extensions.getByType(SourceSetContainer::class)
it seems <T : Any> Project.the(extensionType: KClass<T>): T looks in the convention of the project, while val Project.sourceSets: SourceSetContainer get() looks in the extensions ExtensionContaier instead. This is somewhat odd, as the documentation for the says "Returns the plugin convention or extension of the specified type."
Note that you may need to do your sourceSet manipulation in gradle.projectsEvaluated, because otherwise the sourceSet in question may not be configured yet if the corresponding project is not yet evaluated.
If you get access to the project, then everything should looks like your actual groovy gradle script:
project(":Core").sourceSets.getByName("main").allSource
So regarding your actual code:
val jar: Jar by tasks
jar.apply {
from(java.sourceSets["main"].allSource)
from(project(":Core").sourceSets.getByName("main").allSource)
}

Gradle - common part of DSL in separate (in other) git repository

We use our cusrom plugin and define the script in this way (This is an approximate pseudocode):
//It is common part for every script (1)
environments {
"env1" {
server mySettings("host1", "port1", "etc")
}
"env2" {
server mySettings("host2", "port2", "etc")
}
... //another common scopes
}
and
def defaultSettings(def envHost, def envPort = "15555" ...) {
return {
// Specific settings for the current script (package names, versions etc)
}
}
So in all my scripts (which are separate projects and are in separate git repositories) the common part (1) is repeated.
Is there any correct way to define the common part as a specific project (this can not be part of the plugin - the common part also changes periodically)?
I want to refer to this part when creating a new project and describe only the project-specific settings.
It looks like gradle multi-project builds, but common part should be in other git repository/Nexus.
Important clarification - the common part can also be in the Nexus, have a version ( to have POM descriptor).
It's quite common to have an "opinionated" plugin and a "base" plugin. Gradle uses this concept quite often.
One example is the java plugin automatically applies the java-base plugin. So the java-base plugin contains all of the tasks (logic) but doesn't actually do anything. The java plugin adds the tasks and configures them (eg it adds the src/main/java and src/test/java conventions). So the java-base plugin is not opinionated, the java plugin is opinionated.
So, you could do the same, have a base plugin and a opinionated plugin which
Applies the base plugin
Configures the environments specific for your use case
Note also that you can move logic from build.gradle to a plugin if you put the logic within a project.with { ... } closure. Eg:
class MyPlugin implements Plugin<Project> {
void apply(Project project) {
project.with {
subprojects { ... }
configurations { ... }
dependencies { ... }
task foo(type: Bar) { ... }
}
}
}
There is another solution to your problem. The approach may be less clean than using an opinionated plugin, but it allows you to manage simple Gradle scripts independently from your projects:
The apply from: term to include Gradle scripts is not limited to file paths, but can also handle URLs. This way, you can simply manage your scripts in a standalone repository and provide the newest version via a web server.
To test this way of script distribution and access, you can even use the raw file view feature provided by various repository platforms like GitHub or Bitbucket:
apply from: 'https://raw.githubusercontent.com/<user>/<repo>/<branch>/<file>'
The biggest disadvantage of this approach is the fact, that you need to have access to the local or even global web server for each build, if you need to ensure company-external or offline builds, you should stick to #LanceJavas solution and use a custom plugin.

How to enforce gradle plugins applying order?

In our company we wrote custom gradle plugin which is doing some stuff when application plugin is present. But when application plugin is included in build.gradle after our plugin, our plugin doesn't discover application plugin and actions are not executed.
Is there any way in gradle to enforce plugin applying order? Or any other solution?
Small excerpt from our plugin:
void apply(Project project) {
if (project.plugins.hasPlugin(ApplicationPlugin) {
//some stuff, doesn't work if "application" appears after this plugin
}
}
According to this thread on the Gradle forums, withType(Class) and withId(String) are lazily evaluated and only applied when the referenced plugin is applied.
void apply(Project project) {
project.plugins.withType(ApplicationPlugin) {
//some stuff, doesn't work if "application" appears after this plugin
}
}
As far I know after some investigating, only solution is preserving order of applied plugins.
Why don't you use plugin dependencies. Your custom gradle plugin should be dependent on ApplicationPlugin.
See following code from GroovyPlugin which is dependent on JavaPlugin.
public void apply(Project project) {
project.getPlugins().apply(GroovyBasePlugin.class);
project.getPlugins().apply(JavaPlugin.class);
project.getConfigurations().getByName(COMPILE_CONFIGURATION_NAME).extendsFrom(
project.getConfigurations().getByName(GroovyBasePlugin.GROOVY_CONFIGURATION_NAME)
);
configureGroovydoc(project);
}

How to run gradle task of liquibase „dropAll“ twice?

We’re using liquibase, gradle and the com.augusttechgroup:gradle-liquibase-plugin. We’re currently facing the bug
https://liquibase.jira.com/browse/CORE-1803
which kills our continuoes integration server. Until this problem is resolved, we would like to use a workaround so that we can run the „dropAll“ task in gradle twice.
gradle dropAll dropAll
doesn’t work and
gradle dropAll && gradle dropAll
is no option because of our continuos integration servers which can’t manage this.
Is there a way to make a task run twice or do I have to work around the gradle plugin and write my own dropAll task like #judoole proposed here Liquibase 3.0.1 Gradle integration?
The simplest solution I found was just to declare two databases.
liquibase {
activities {
main {
url 'jdbc:postgresql://localhost:5432/db'
username 'user'
password 'passwd'
...
}
secondRun {
url 'jdbc:postgresql://localhost:5432/db'
username 'user'
password 'passwd'
...
}
}
runList = 'main'
}
dropAll.doFirst { liquibase.runList = 'main,secondRun' }
dropAll.doLast { liquibase.runList = 'main' }
Per default the 'runList' only contains the main db. But for dropAll he sets both databases active, so that dropAll will run on both ... which are actually the same.
I would consider using JavaExec, like the linked StackOverflow question, especially if you're only using xml and not Groovy to write your changelog. The beauty of the liquibase-gradle-plugin is just these strange cases you describe. From the Github repo:
Let's suppose that for each deployment, you need to update the data
model for your application's database, and wou also need to run some
SQL statements in a separate database used for security.
Now I haven't used the plugin, because I didn't need those features. But I did run into bugs in liquibase and needed to use a spesific version, namely 3.0.4, as generateChangelog has a strange bug since 3.0.5. Using the JavaExec version I was able to track down a version that suited my needs.
Now, the liquibase-gradle-plugin doesn't need to be your only weapon of choice. Gradle has plenty of room for writing your own little tasks. Also those who do some sql. Maybe try something along these lines and see if that works:
configurations {
driver
}
dependencies {
driver '<your-sql-driver>'
}
//"Bug" in Gradle. Groovy classes are loaded first. They need to know about sql driver
//Or I think it's still so
URLClassLoader loader = GroovyObject.class.classLoader
configurations.driver.each { File file ->
loader.addURL(file.toURL())
}
task deleteFromTables(description: 'Deletes everything.') <<{
def props = [user: "<username>", password: "<password>", allowMultiQueries: 'true'] as Properties
def sql = Sql.newInstance("<url>", props, "<driver-classname>)
try {
//Here you can do your magic. Delete something, or simple drop the database.
//After dropping it, you'd probably want another task for creating
//it back up again
sql.execute("DELETE ...")
} finally {
sql.close()
}
}
Gradle makes sure that each task gets executed at most once per Gradle invocation. If you need to do some work twice, you need to declare two tasks. Chances are that the plugin provides a suitable task type for you to declare a second dropAll task.

Resources