So we have a huge multi-project codebase with structure like below:
C:\Eclipse\Workspace->
AR
DC
CI
..
..
Each project has a build.gradle file which has 80% of the code same with only dependencies section changing for all the projects.
What I want to achieve:
I want to create a parent project named "BuildAllProjects" which would be the ONLY project having build.gradle, settings.gradle and gradle.properties and propose to have a properties file for mentioning the dependencies of each project, something like:
AR=j2ee,commons-lang,FW,DA,Common
DC=commons-codec,FW,DA,Common,spring-core
and then use the gradle expand[] properties to dynamically fill the dependencies for the project which I am building, so for instance, if I am building AR, I may want to run:
gradle -PAR build
which will read dependencies for "AR" from the properties and expand in the form :
dependencies {
compile name 'j2ee'
compile name 'commons-lang'
}
Do you guys think this is possible or is this the WORST way of achieving it? I am new to GRADLE overall and information provided above is based on knowledge that I have acquired in a weeks time. Please provide your suggestions to implement this at the BEST of gradle.
Thanks,
Yogendra
Layering a properties file based build language on top of Gradle's build language doesn't strike me as desirable. Instead, I recommend to focus on writing clean and DRY Gradle build scripts. You can learn more about Gradle's powerful abstraction capabilities (configuration injection, script plugins, buildSrc, custom plugins and extensions, etc.) in the Gradle User Guide.
In a typical multi-project build, subproject build scripts mostly contain dependency declarations, whereas most other configuration happens in the root build script and/or script plugins.
Related
We have a directory structure like so
java
build/build.gradle (This does NOT exist yet, but we want this)
servers
server1/build.gradle
server2/build.gradle
libraries
lib1/build.gradle
lib2/build.gradle
We have 11 servers and 14 libraries with varying uses of dependencies. EACH server is a composite build ONLY depending on libraries (we don’t allow servers to depend on each other). In this way, as our mono-repo grows, opening up server1 does NOT get slower and slower as more and more gradle code is added(ie. gradle only loads server1 and all it’s libraries and none of the other libraries OR servers are loaded keeping things FAST).
Ok, so one problem we are running into is duplication now which is why we need build/build.gradle file AND we want EVERY module in our mono repo to include that somehow for a few goals(each goal may need a different solution)
GOAL 1: To have an ext { … } section containing a Map of Strings to gradle dependencies much like so
deps = [
'web-webserver': "org.webpieces:http-webserver:${webpiecesVersion}",
'web-webserver-test': "org.webpieces:http-webserver-test:${webpiecesVersion}",
'web-devrouter': "org.webpieces:http-router-dev:${webpiecesVersion}"
]
In this way, we want ALL our projects to them import dependencies like so
compile deps['web-webserver']
GOAL 2: We want to 'include' a standard list of plugins so we are versioning all gradle plugins the same across the repo. While the above configures all jars to avoid jar hell in a mono-repo, we would like to do the same with just this section
plugins {
id 'com.github.sherter.google-java-format' version '0.9'
}
Of course, it each project may also want to add a few more plugins OR even not depend on this section(in case of an emergency and trying to just get the job done).
GOAL 3: We want checkstyle configuration (or any plugin config) to be defined the SAME for all projects (eventually!!!). We would like the checkstyle gradle to live in a common area but have all libraries somehow pull it in. Again, it would be nice for it to be optional in that, I can pull the gradle section into my build.gradle OR can create a new one in case of emergencies so I don't have to fix all projects in the monorepo right away.
IDEALLY, perhaps I kind of want configuration injection where when I run server1/build.gradle, it actually runs java/build/build.grade as it’s parent somehow but with overrides (IF I declare 'extends xxx.gradle' maybe) then all libraries it uses also use java/build/build.gradle as their parent. I am not sure this is possible or feasible. I am pretty sure 'extends xxx' doesn't exist in gradle.
Are any of these GOALS possible?
thanks,
Dean
I have been working on a monorepo with the exact same requirement as you, using gradle composite builds as well. The way we have solved this problem is by using pre compiled plugins
You need to do a new gradle project with only the code you want to share. This will create a plugin, that you can just add as a composite build and apply to the other projects.
I'm a bit confused by why you don't just use a "standard" gradle top level build file and compose the others as subprojects.
This solves all 3 of your goals
If you are concerned by build speed, you can target each server individually simply by running
./gradlew :server1:build
But if you are not able to do this for some reason you can use the apply from: syntax as described here
What is the key difference between includeBuild(...) and implementation(project(...)) in the Gradle build system? I cannot really see the usecase difference after reading the documentation:
https://docs.gradle.org/current/userguide/declaring_dependencies.html#sub:project_dependencies
https://docs.gradle.org/current/userguide/composite_builds.html#separate_composite
What I am trying to do: Share same codebase for two separate projects: data classes (kotlix.serialization), external database dao, interfaces. It is not a full library but just some code snippets.
How can I connect the two projects in Intellij so that type hinting works?
Composite Build (by using includeBuild) is a way to create a dependency between autonomous Gradle Projects.
Project import, is a way to create a dependency between two modules within a same Gradle Project.
Composite Build is far more powerful and is also meant to be the new way of factorizing gradle configuration between multiple projects that you traditionally do with the buildSrc technique.
I found this "Structuring Large Projects" article to be easier to read than the "Composite Builds" doc.
An excellent sample project that demonstrates the power of Composite Builds can be found in Gradle sample_structuring_software_projects.
Project dependency case
The tree would look like this:
settings.gradle.kts
module1/build.gradle.kts
module2/build.gradle.kts
And you are declaring a dependency in module1/build.gradle.kts like this:
dependencies {
implementation(project("com.domain:module2"))
}
The dependency will be resolved only if both projects are declared as sub-modules of a common root project.
It means you have a root settings.gradle.kts like this:
rootProject.name = "rootProject"
include(":module1")
include(":module2")
Composite build case
The projects do not need to have common "umbrella" root project.
Each project is a totally independent project.
One project can simply declare a dependency on the other project (without even the target project knowing about it).
Tree:
project1/build.gradle.kts
project1/settings.gradle.kts
project2/build.gradle.kts
project2/settings.gradle.kts
In project1/settings.gradle.kts:
rootProject.name = "project1"
includeBuild("../project2") //No more ':' as it is not a module
In project2/settings.gradle.kts:
rootProject.name = "project2"
In project1/build.gradle.kts like this:
dependencies {
implementation("com.domain:project2")
}
I have the same problem. Reading on the first link, the next para says:
Local forks of module dependencies
A module dependency can be substituted by a dependency to a local fork
of the sources of that module, if the module itself is built with
Gradle. This can be done by utilising composite builds. This allows
you, for example, to fix an issue in a library you use in an
application by using, and building, a locally patched version instead
of the published binary version. The details of this are described in
the section on composite builds.
So, it should be implementation project according to me.
P.S. The code completion is working on one of my sub project but not on the other. I am still trying to figure that out
In Maven one can use the base pom file concept(https://www.atlassian.com/blog/archives/maven_in_our_development_proce_3) - the file everyone would then reuse
Is it a similar concept in Gradle? We use Gradle to build Java projects
You should take a look at this answer
Basically, the parent project can hold the role of "parent POM" and thanks to Gradle's allprojects, subprojects and extra properties mechanisms, you can reproduce Maven's main behavior. Also you can split your configuration in several .gradle files and include them with apply from. Finally, regular and custom plugins can be defined in the parent project, for target or all sub-projects.
I am using gradle and its local repository is at \.gradle\caches\modules-2\files-2.1 which has all the downloaded jar but not my modules.Is there any specific place I should be searching it for ?
I need it as is in settings.gradle I am having a dependency path specified like :
include ':model'
project (':model').projectDir = new File(settingsDir, './model')
in a new project. Also I don't want to give path in that way because if I have a dependency from multiple projects on this project then mentioning path will be difficult and weird.
How can I make gradle search it from local maven or gradle repositories.
I'm still not sure what is being asked here, and I suspect there is some confusion over how multi-project builds work. So I'm going to attempt to provide a general-purpose answer.
The first question you need to answer is whether you're interested in dependencies between projects that are part of the same build — as in part of a multi-project build — or in separate builds.
Project dependencies (multi-project builds)
Project dependencies are covered in the user manual and only apply to multi-project builds. They use a logical path, using colons as 'path' separators, to specify the location of the target module, like so:
dependencies {
implementation project(":model")
}
At this point, Gradle needs to know where ":model" exists on the file system. There's no getting around that. You have a few options:
Follow the convention of directory structure matching the logical path structure, i.e. have a MyBigProject/model directory containing the ":model" child project
Specify the file path of ":model" in settings.gradle, e.g. with project(":model").projectDir = new File(rootDir, "unusual/path/to/model")
Automate the discovery of projects
The most common approach is the first one. The second is not unusual, particularly if you want to put child projects into a separate directory, like subprojects — something the build of Gradle itself does. I haven't seen the last option done, and I don't know whether it runs into problems.
For the sake of completeness, and at your own risk if you use something like it, here's an example of automatic discovery of projects in the settings.gradle file:
rootDir.eachDir { File dir ->
if ("build.gradle" in dir.listFiles()*.name) {
include dir.name
}
}
This fragment basically looks for directories within the root project folder that have a build.gradle file in them and adds them as child projects. The child projects' directory names become the projects' names.
It's not particularly clever, and you should really use different names for the build files, but it may give you some ideas to work with.
Non-project dependencies
As with project dependencies, Gradle needs to know where to get the corresponding JAR or other form of artifact for a specified module. You normally specify Maven Central or something similar for this, but there are other useful, but less common, options:
Copy a project's artifacts into the local Maven repository — both the Maven Plugin and Maven Publish Plugin support this
Publish to a Maven-compatible repository using a file:// URL rather than an HTTP/HTTPS one, which protects your projects from corruption of Maven Local
Worth noting is that Gradle supports composite builds that allow you to substitute a normal dependency with (effectively) a project dependency from another build. So if model were part of a separate build but you had the source code and build locally, you could make changes and immediately test them in another build's project without going through the whole "install" intermediate step that's common in the Maven world (and Gradle pre-composite-builds).
Hope all this makes sense.
I have a need to convert the maven project to gradle project. All is going fine, but there is one problem that I faced, in the current project we are using dependency management throw parent pom of maven and in gradle as long as I researched there is no possibility to do so. I was thinking to convert somehow parent pom to maven bom and use in build.gradle because I know that gradle can use maven boms.
Do anyone have better ideas how to accoplish that ?
Or may be someone also had this kind of problem, can suggest best ways to do it ?
Regards
You could use my gradle-maven-transform plugin to transform your pom.xmls into gradle scripts.
You can use the DependencyAggregator to find versions which are common across all projects and generate a root build.gradle containing the common versions.
Each project script can then reference a version variable from the root project instead of hard coding
You might choose to use nebula dependency recomnender plugin to manage common versions. The maven-transform plugin can generate scripts to support this style of declaration too
Whilst you are evaluating the gradle build you "apply" the generated gradle scripts in your gradle build. During this time both maven and gradle builds will work in parallel. Once your happy with the gradle build you copy paste the generated scripts into your build.gradle files and ditch maven for ever! Yay!