I have a settings plugin that is multi-project aware. It currently applies itself to every project identified in settings.gradle. However, this is a problem because each subproject doesn't have the same configuration, and I control some plugin behavior through project properties set on the command-line via -P.
The problem is that this fails on projects that don't have the configuration necessary to use that property, and I know of know way to apply a property to a specific subproject via the command-line.
Instead of iterating over settings.gradle.allprojects, is there a way to know what projects have actually been included as part of the build? For example when I do:
gradle :subproject-name:build :other-subproject-name:build -PsomeProperty
I would like to know that only subproject-name and other-subproject-name were called so that I can apply the settings plugin to only those projects.
Or is there a way to "scope" project properties somehow, to only a particular project?
... is there a way to know what projects have actually been included as part of the build?
This is a misconception. All projects are part of the build when settings.gradle includes them, and they'll all get configured so they have an associated Project.
What you're ultimately looking for is, given the tasks that will execute, what are the subprojects who own those tasks? To do that, you can grab Gradle's task graph, and for all of the tasks that will execute, find the project that owns each task.
gradle.taskGraph.whenReady { graph ->
def projects = graph.allTasks.collect { it.project }.toSet()
projects.each {
println "Project is being used in this build: " + it
}
}
Related
I have the following modules with the top one depending on the next depending on the next ( These links have VERY SIMPLE build.gradle and settings.gradle files)
https://github.com/deanhiller/webpieces/tree/master/core/core-ssl
https://github.com/deanhiller/webpieces/tree/master/core/core-datawrapper
https://github.com/deanhiller/webpieces/tree/master/core/core-util
https://github.com/deanhiller/webpieces/tree/master/core/core-logging
I temporarily added an throw new RuntimeException to a test in core-datawrapper and core-util and built the project core-ssl (in the repo, ../../gradlew build)
The settings of core-ssl (found in above link and pasted here) is
includeBuild '../core-datawrapper'
includeBuild '../core-mock'
The settings of core-datawrapper (again in above links)
includeBuild '../core-util'
I clear out core-util/build and I see these targets run
> Task :core-util:compileJava
> Task :core-util:classes
> Task :core-util:jar
That is it. Why is the tests not running? I thought build depended on assemble and test separately?
The same for ../../gradlew clean and ../../gradlew publish
Ideally, I want my target to affect all transitive projects as well. As developers add projects, I don't want to have to add code to each gradle project in the transitive deps list either.
Yes, that’s correct.
This is most probably due to the original use-case of composite builds.
You have some binary dependency on some library, and then you want to change something in the library and test it in your project before even committing or publishing the library.
So the composite build result replaces the binary dependeny and only as much work as really necessary is done, meaning the jar is built.
If you use composite builds as normal structuring element, you either need to wire the lifecycle tasks you want to have wired yourself or search for a plugin that maybe does it.
As developers add projects, I don't want to have to add code to each gradle project in the transitive deps list either.
You can programmatically iterate through the included builds using gradle.includedBuilds, so you can do the wiring in a generic way.
Background
(Please keep in mind I've simplified the problem for purposes of discussion here)
I've got a set of applications and dependent libraries, something like this (each with a src/ directory and build.gradle):
appa/
appb/
libx/
liby/
libz/
In build.gradle, the dependencies are currently declared like this:
appa/build.gradle:
compile "com.asdf:libx:1.0"
compile "com.asdf:liby:1.0"
appb/build.gradle:
compile "com.asdf:liby:1.0"
liby/build.gradle:
compile "com.asdf:libz:1.0"
What problem am I trying to solve
Say I'm working on appa, and I need to make changes to libx. I need to do multiple steps:
Pull libx from source control and make changes locally
Rebuild and push changes to some repo (not prod!)
Rebuild appa (pulling the recently updated libx from repo)
If my testing reveals a bug in libx, I've got to repeat that over again.
This is super-annoying when working in IDEs like Eclipse, where even though my projects are logically using other projects, I've got to still use the artifacts as dependencies.
Wouldn't it be great if I can just pull the project locally, and projects that logically depend on it will automatically use the source project instead of artifact for building?
What I've done so far
I've written a small gradle plugin (referenced in each project's build.gradle) that identifies com.asdf dependencies, and uses dependency substitution to replace the artifact dependency with a project dependency if that project exists locally.
configurations.all {
resolutionStrategy.dependencySubstitution {
all { DependencySubstitution dependency ->
if (dependency.requested instanceof ModuleComponentSelector && dependency.requested.group == 'com.asdf') {
def targetProject = findProject(":${dependency.requested.module}")
if (targetProject != null) {
dependency.useTarget targetProject
}
}
}
}
}
Yay! With a few modifications to settings.gradle (see below), I've accomplished my goal... Except...
Where I'm stuck
I need to modify settings.gradle to include lines like this for every dependency (otherwise findProject doesn't resolve the dependent project during build):
include ':libx'
project(':libx').projectDir = new File(settingsDir, '../libx')
While it's possible to go through all the settings.gradle files and do this (I've done it for a handful as my proof-of-concept), it's ugly, repetitive, and is logically the same information that is being passed to compile arguments for the build.gradle dependencies.
It's also error-prone when someone adds a new dependency but doesn't update settings.gradle, or introduces a typo between them.
I've also tried making settings.gradle just define projects for all directories it finds at that level, but then building any project turns into a mega-build of all projects.
(I've tried several other things, but my question is getting long in the tooth already)
My question
What's a better way to do this, without duplicating information between settings.gradle and build.gradle? I want to make it so adding new dependencies is still just as easy as adding the compile reference in build.gradle, without touching settings.gradle...
I'm still rather new to groovy/gradle, so maybe I'm missing something that's obvious to the more experienced gradle master?
I believe your use-case is the motivation for Composite Builds.
I have a demo here, which writes to a jars folder as a mock publishing of artifacts. Be sure to check-out the README.md as the demo is a mini-laboratory for trying out the use-case before and after composite builds.
In the demo mainBuild is appa; utils is libx. The key syntax in mainBuild/settings.gradle (here) is:
includeBuild '../utils'
This tells Gradle to use the local codebase instead of the published artifact. Of course, one would not commit this line to source-control.
I am using gradle and its local repository is at \.gradle\caches\modules-2\files-2.1 which has all the downloaded jar but not my modules.Is there any specific place I should be searching it for ?
I need it as is in settings.gradle I am having a dependency path specified like :
include ':model'
project (':model').projectDir = new File(settingsDir, './model')
in a new project. Also I don't want to give path in that way because if I have a dependency from multiple projects on this project then mentioning path will be difficult and weird.
How can I make gradle search it from local maven or gradle repositories.
I'm still not sure what is being asked here, and I suspect there is some confusion over how multi-project builds work. So I'm going to attempt to provide a general-purpose answer.
The first question you need to answer is whether you're interested in dependencies between projects that are part of the same build — as in part of a multi-project build — or in separate builds.
Project dependencies (multi-project builds)
Project dependencies are covered in the user manual and only apply to multi-project builds. They use a logical path, using colons as 'path' separators, to specify the location of the target module, like so:
dependencies {
implementation project(":model")
}
At this point, Gradle needs to know where ":model" exists on the file system. There's no getting around that. You have a few options:
Follow the convention of directory structure matching the logical path structure, i.e. have a MyBigProject/model directory containing the ":model" child project
Specify the file path of ":model" in settings.gradle, e.g. with project(":model").projectDir = new File(rootDir, "unusual/path/to/model")
Automate the discovery of projects
The most common approach is the first one. The second is not unusual, particularly if you want to put child projects into a separate directory, like subprojects — something the build of Gradle itself does. I haven't seen the last option done, and I don't know whether it runs into problems.
For the sake of completeness, and at your own risk if you use something like it, here's an example of automatic discovery of projects in the settings.gradle file:
rootDir.eachDir { File dir ->
if ("build.gradle" in dir.listFiles()*.name) {
include dir.name
}
}
This fragment basically looks for directories within the root project folder that have a build.gradle file in them and adds them as child projects. The child projects' directory names become the projects' names.
It's not particularly clever, and you should really use different names for the build files, but it may give you some ideas to work with.
Non-project dependencies
As with project dependencies, Gradle needs to know where to get the corresponding JAR or other form of artifact for a specified module. You normally specify Maven Central or something similar for this, but there are other useful, but less common, options:
Copy a project's artifacts into the local Maven repository — both the Maven Plugin and Maven Publish Plugin support this
Publish to a Maven-compatible repository using a file:// URL rather than an HTTP/HTTPS one, which protects your projects from corruption of Maven Local
Worth noting is that Gradle supports composite builds that allow you to substitute a normal dependency with (effectively) a project dependency from another build. So if model were part of a separate build but you had the source code and build locally, you could make changes and immediately test them in another build's project without going through the whole "install" intermediate step that's common in the Maven world (and Gradle pre-composite-builds).
Hope all this makes sense.
So we have a huge multi-project codebase with structure like below:
C:\Eclipse\Workspace->
AR
DC
CI
..
..
Each project has a build.gradle file which has 80% of the code same with only dependencies section changing for all the projects.
What I want to achieve:
I want to create a parent project named "BuildAllProjects" which would be the ONLY project having build.gradle, settings.gradle and gradle.properties and propose to have a properties file for mentioning the dependencies of each project, something like:
AR=j2ee,commons-lang,FW,DA,Common
DC=commons-codec,FW,DA,Common,spring-core
and then use the gradle expand[] properties to dynamically fill the dependencies for the project which I am building, so for instance, if I am building AR, I may want to run:
gradle -PAR build
which will read dependencies for "AR" from the properties and expand in the form :
dependencies {
compile name 'j2ee'
compile name 'commons-lang'
}
Do you guys think this is possible or is this the WORST way of achieving it? I am new to GRADLE overall and information provided above is based on knowledge that I have acquired in a weeks time. Please provide your suggestions to implement this at the BEST of gradle.
Thanks,
Yogendra
Layering a properties file based build language on top of Gradle's build language doesn't strike me as desirable. Instead, I recommend to focus on writing clean and DRY Gradle build scripts. You can learn more about Gradle's powerful abstraction capabilities (configuration injection, script plugins, buildSrc, custom plugins and extensions, etc.) in the Gradle User Guide.
In a typical multi-project build, subproject build scripts mostly contain dependency declarations, whereas most other configuration happens in the root build script and/or script plugins.
I have a maven multi-module project (please see the attached image for the structure). I am in the process of migrating to Gradle.
We have multiple profiles inside AppBuild/pom.xml like shown below; and we run our maven build from AppBuild with -P option for the profiles.
Since I am new to Gradle, I am not able to decide what is the best way to go about it. Now, I am thinking to put some conditional include in settings.gradle based on some -P argument, but I am not too sure. Could someone help me how to go about this? What is the best way with some examples.
Thanks in advance.
~ Niranjan
I recently came across the same question, since I have a multiproject build where I want to include a special subproject only on some occasions.
This special subproject only depends on one other subproject and takes approx 1-2min to configure. So I don't want to have it included if I don't want to build it. That saves some build time.
So, what I did was to modify the settings.gradle
if(file('MySpecialProject').exists()){
include ':MySpecialProject'
}
When I do not want to build MySpecialProject I can simply rename/delete its folder. In my case, the folder is a symlink, since this 'subproject' resides in it's own git repo.
Even if #spy writes that if/then/else is not possible in settings, it did work for me.
Conditional include isn't straightforward in Gradle (it's not a first-class feature), and it's unclear why you would need it for so few subprojects. I recommend to start without and add all four subprojects to settings.gradle.