Adding subproject resolved dependencies to earlib in gradle 5.5.1 - gradle

I have a multi-project build with 100+ EAR files. The subprojects more or less use a convention like this:
subproject1
-| subproject1JAR
-| subproject1EAR
-| subproject1WAR
subproject2
-| subproject2JAR
-| subproject2EAR
-| subproject2WAR
When I refer to JAR, below, I'm referring to the business logic/SDK contained in, e.g., subproject1.jar.
I have some logic in an afterEvaluate block in subprojects{} that parses the .classpath files of the subprojects in order to build the EAR files. This is working but one thing I'm not getting as compared to the legacy (manual) builds is that the EAR/lib folder needs to contain all the dependencies of the JAR file.
Apologies for the wall of code here, I'm just trying to provide the full context for this question.
// this is from subprojects
if(proj.projectDir.name.endsWith('EAR')){ <-- configures EAR proj based on folder name
proj.apply plugin: 'ear'
proj.ear {
baseName = proj.projectDir.name
// snip other ear config stuff
}
if(f_ui.exists()){ // <-- f_ui is a File pointed at what ought to be the WAR subproject
proj.dependencies{
deploy project(path:":${proj_war}WAR", configuration:'archives')
}
}
if(f_jar.exists()){ // <-- f_jar is a File pointed at what ought to be the JAR subproject
proj.dependencies{
earlib project(path:":$proj_jar", configuration:'archives')
}
def jar_proj = project(":$proj_jar")
proj.evaluationDependsOn(":$proj_jar")
jar_proj.configurations.runtime.allDependencies.forEach { <-- trying to add deps here but
println "##> DEPENDENCY $it" <-- these are empty here
proj.dependencies.add('earlib', it)
}
}
// ## END OF EAR PROJECT CONFIGURATION
}
You can see above where I'm trying to add the dependencies OF THE JAR to the earlib configuration but at this point it seems the dependencies have not been resolved so no dependencies are added.
I believe this unanswered question seems to be asking the same thing.
Any suggestions on how I should be doing this in order to get the dependencies of the JAR into the ear/lib folder?

I certainly don't think this is the best solution but it seems to be working so I'll describe what I did.
Create a custom configuration into which all the dependencies I needed to show up in the earlib were added:
subprojects { Project proj ->
def projname = proj.name
afterEvaluate {
logger.info "## {} afterevaluation #0 started",projname
["Proj1","proj2", "proj3"].each{ depson ->
if (classpathFileContains(proj,depson)) {
proj.configurations.maybeCreate('extra')
proj.dependencies.add('extra', project(":${depson}"))
}
}
...
Add those as implementation dependencies:
try {
// if the project has 'extra', add them to the implementation
proj.configurations.named('extra')
proj.configurations.extra.dependencies.each{
proj.dependencies.add('implementation', it)
}
} catch(e) {
logger.info "## Project {} did not have an extra", proj.name
}
Add the jar file and then add the extra dependencies from the custom configuration:
proj.dependencies{
earlib project(":$proj_jar")
}
try {
jar_proj.configurations.named('extra')
jar_proj.configurations.extra.dependencies.each{
proj.dependencies.add('earlib', it.copy())
}
...
Incidentally, I ran across several other instances of people having
this challenge with earlib (see
ex1,
ex2,
ex3,
plus the reference in the question).
I'm going to try to simplify things and will update this answer if I have an aha moment but as of now this approach is working.

Related

How to create multi project fat jar as bundle of some libraries with buildSrc

First of all, sorry for my poor english.
Goal
I want create multi project containing some custom libraries as subproject with gradle.
For centralized dependency version control, using buildSrc and setting versions (spring-boot, detekt, ktlint etc.)
my-core-project(root)
+buildSrc
+-src/main/kotlin
+--int-test-convention.gradle.kts
+--library-convention.gradle.kts
+library-A
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+library-B
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+build.gradle.kts
+setting.gradle.kts
buildSrc contains common tasks for libraries(integration test, detekt, etc.)
library-A and library-B are custom libraries based on spring boot.
There is no application module or any main method.
my goal is using method of library-A and/or library-B with another separated project with adding my-core-project to dependency.
Problem
./gradlew build created 3 jar files
my-core-project
+build/libs
+-my-core-project.jar
+library-A
+-build/libs
+--library-A.jar
+library-B
+-build/libs
+--library-B.jar
copied 3 jar files to libs directory under project which actually using these library,
tried adding dependency created jar
with implementation(files("libs/library-A.jar")), class and methods are resolved well.
but with implementation(files("libs/my-core-project.jar")),
class and methods are not unresolved.
when check my-core-project.jar, recognized that any information of sub projects contained.
Here is my setting.gradle.kts and build.gradle.kts of root directory.
# setting.gradle.kts
pluginManagement {
repositories {
mavenCentral()
gradlePluginPortal()
}
}
rootProject.name = "my-core-project"
include(
"library-A",
"library-B"
)
# build.gradle.kts
plugins {
id("java-library")
id("io.spring.dependency-management")
}
group = "com.demo"
version = "0.0.1-SNAPSHOT"
dependencies {
api(project(":library-A"))
api(project(":library-B"))
}
repositories {
mavenCentral()
}
Tried things
In my opinion, my-core-project.jar should be fatJar(uberJar),
so i added FatJar task like this
val fatJar = task("fatJar", type = Jar::class) {
archiveBaseName.set("${project.name}-fat")
from(configurations.runtimeClasspath.get().map { if (it.isDirectory) it else zipTree(it) })
with(tasks.jar.get() as CopySpec)
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
tasks {
"build" {
dependsOn(fatJar)
}
}
but cannot resolve class and method,
additionally occurs version conflict with other dependency of projects using this package, due to library-A created as fatJar too.
Question
Is there a simple way packaging/bundling sub-modules into one jar file?
if there are tasks like this already in gradle, prefer to use that.
Modifying fatJar task like "add jar of submodules, not contained dependencies" can solve this problem?(even couldn't try completely newbie to gradle and kts.)
if so, can you show me how to modify task?
tried shadowJar already. that solved version-conflict problem with relocate option. but still couldn't resolve package in library-A
If structure has problem, is there a good practice/example for "bundle of library"?
thanks for reading.
TL;DR
If someone faced this problem, try set archive name shorter than current one.
For someone who faced same problem, sharing some informations.
as result, resolved this problem.(maybe even not problem)
current shadowJar configure is as following
tasks.named<ShadowJar>("shadowJar").configure {
archiveBaseName.set("shorten-name")
archiveClassifier.set("")
exclude("**/*.kotlin_metadata")
exclude("**/*.kotlin_builtins")
}
exclude kotlin_metadata, kotlin_builtins
set shorten name(original project name was 30 long characters)
I have no idea but shorten jar file name has no problem.
Interesting one is, upload in artifactory package with original(long) name worked well.
I don't know Gradle declaring dependency with files has length constraints.
implementation(files("path/to/package"))
And now it works well with original name with local jar package file.

Execute Gradle task after subprojects are configured

I have a multi-project Gradle build where subprojects are assigned version numbers independent of the root project. I'd like to inject this version number into a few resource files in each subproject. Normally, I'd do this by configuring the processResources task for each subproject in the root build. However, the problem is that Gradle appears to be executing the processResources task before loading the subprojects' build files and is injecting "unspecified" as the version.
Currently, my project looks like this:
/settings.gradle
include 'childA' // ... and many others
/build.gradle
subprojects {
apply plugin: 'java'
apply plugin: 'com.example.exampleplugin'
}
subprojects {
// This has to be configured before processResources
customPlugin {
baseDir = "../common"
}
processResources {
// PROBLEM: version is "unspecified" here
inputs.property "version", project.version
// Inject the version:
from(sourceSets.main.resources.srcDirs) {
include 'res1.txt', 'res2.txt', 'res3.txt'
expand 'version':project.version
}
// ...
}
}
/childA/build.gradle
version = "0.5.424"
I looked into adding evaluationDependsOnChildren() at the beginning of root's build.gradle, but that causes an error because childA/build.gradle runs before customPlugin { ... }. I've tried using dependsOn, mustRunAfter, and other techniques, but none seem have the desired effect. (Perhaps I don't fully understand the lifecycle, but it seems like the root project is configured and executed before the subprojects. Shouldn't it configure root, then configure subprojects, and then execute?)
How can I get inject the version of each subproject into the appropriate resource files without a lot of copy/paste or boilerplate?
You could try using this method, with a hook:
gradle.projectsEvaluated({
// your code
})
I got this figured out for myself. I'm using a init.gradle file to apply something to the rootProject, but I need data from a subproject.
First option was to evaluate each subproject before I modified it:
rootProject {
project.subprojects { sub ->
sub.evaluate()
//Put your code here
But I wasn't sure what side effects forcing the sub project to evaluate would have so I did the following:
allprojects {
afterEvaluate { project ->
//Put your code here
Try doing it like this:
subprojects { project ->
// your code
}
Otherwise project will refer to your root project where no version has been specified.

Generate CLASSPATH from all multiproject gradle build dependencies

We're using an external testing tool (Squish) which runs after our main gradle build. This requires visibility of classes under test. Currently the CLASSPATH environment variable is built from various bash scripts and other string manipulation. I'm aiming to get gradle to do this automagically.
Since the build is done on a very slow source control system (clearcase) it is preferable for the tests to be run against the class files/JARs as left by the build rather than extra copies / compression into a single JAR etc.
The classpath needs to contain
Generated JAR files, typically in build/libs/project-x.jar
Test classes typically build/classes/test
Test resources typically build/resources/test
Any 3rd party jars any of the child projects depend upon, log4j, spring etc.
It's a complex multiproject build, but I've simplified to following example with parent and two children.
Parent settings.gradle
include ':child1'
include ':child2'
Parent build.gradle
allprojects {
apply plugin: 'java'
repositories {
mavenCentral()
}
}
Child 1 build.gradle
dependencies {
compile 'org.springframework:spring-context:4.1.2.RELEASE'
compile 'org.springframework:spring-beans:4.1.2.RELEASE'
}
Child 2 build.gradle
dependencies {
project (":child1")
}
What I've got so far. Is this the right approach? Can it be simplified or completely rewritten in a better way?
task createSquishClasspath << {
def paths = new LinkedHashSet()
paths.addAll(subprojects.configurations.compile.resolvedConfiguration.resolvedArtifacts.file.flatten())
paths.addAll(subprojects.jar.outputs.files.asPath)
paths.addAll(subprojects.sourceSets.test.output.resourcesDir)
paths.addAll(subprojects.sourceSets.test.output.classesDir)
paths.each {
println "${it}"
}
println paths.join(File.pathSeparator)
}
Output
C:\so-question\Parent\local-repo\spring\spring-context-4.1.2.RELEASE.jar
C:\so-question\Parent\local-repo\spring\spring-beans-4.1.2.RELEASE.jar
C:\so-question\Parent\child1\build\libs\child1.jar
C:\so-question\Parent\child2\build\libs\child2.jar
C:\so-question\Parent\child1\build\resources\test
C:\so-question\Parent\child2\build\resources\test
C:\so-question\Parent\child1\build\classes\test
C:\so-question\Parent\child2\build\classes\test
In summary the best answer so far is to join all the paths from the projects dependencies and all the test.output.resourcesDir and test.output.classesDir
task createSquishClasspath << {
def paths = new LinkedHashSet()
paths.addAll(subprojects.configurations.compile.resolvedConfiguration.resolvedArtifacts.file.flatten())
paths.addAll(subprojects.jar.outputs.files.asPath)
paths.addAll(subprojects.sourceSets.test.output.resourcesDir)
paths.addAll(subprojects.sourceSets.test.output.classesDir)
paths.each {
println "${it}"
}
println paths.join(File.pathSeparator)
}

Gradle - can I include task's output in project dependencies

I have a task that generates java sources and a set of jars from these sources (say, project a). I would like to export these jars to dependent projects (say, project b). So here's roughly what I have right now:
//a.gradle
configurations{
generatedJars
}
task generateJars(type: JavaExec) {
//generate jars ...
outputs.files += //append generated jars here
}
dependencies{
generatedJars generateJars.outputs.files
}
//b.gradle
dependencies{
project(path: ':a', configuration: 'generatedJars')
}
It works OK, except that adding generateJars.outputs.files as a dependency does not tell gradle that it has to run generateJars task when there are no jars generated yet. I have tried adding the task itself as a dependency hoping that it would work in the same way as it does when you add a jar/zip task to an artifact configuration (e.g. artifacts{ myJarTask }), but it throws an error telling me that I cannot do that. Of course I can inject the generateJars task somewhere in the build process before :b starts evaluating, but that's clumsy and brittle, so I would like to avoid it.
I feel like I should be adding the generated jars to artifacts{ ... } of the project, but I am not sure how to make them then visible to dependent projects. Is there a better way of achieving this?
Dependent projects (project b) will need to do setup IntelliJ IDEA module classpath to point to project a's generated jars. Something rather like this (pseudo-code):
//b.gradle
idea{
module{
scopes.COMPILE.plus += project(path: ':a', configuration: 'generatedJars').files
}
}
So far I have tried simply adding a project dependecy on :a's generatedJars in :b, but Idea plugin simply adds module :a as a module-dependency and assumes that it exports its generated jars (which is probably a correct assumption), therefore not adding the generated jars to :b's classpath.
Any help would be greatly appreciated!
First, do you need a separate configuration? That is, do you have clients of a that should not see the generated Jars? If not, you can add the generated Jars to the archives configuration, which will simplify things.
Second, the correct way to add the generated Jars to the configuration is (instead of the dependencies block):
artifacts {
generatedJars generateJars
}
This should make sure that the generateJars task gets run automatically when needed.
Third, I'd omit the += after outputs.files, although it might not make a difference. You should also add the necessary inputs.
Fourth, why do you need a JavaExec task to generate the Jars? Can you instead add the generated sources to some source set and let Gradle build them?
Fifth, IDEA doesn't have a concept corresponding to Gradle's project configuration dependencies. Either an IDEA module fully depends on another module, or not at all. You have two options: either use a module dependency and make the generated sources a source folder of the depended-on module (preferably both in the Gradle and the IDEA build), or pass the generated Jars as external dependencies to IDEA. In either case, you should probably add a task dependency from ideaModule to the appropriate generation task. If this still doesn't lead to a satisfactory IDEA setup, you could think about moving the generation of the Jars into a separate subproject.
For my use case, I had a C++ project which generated some native libraries which my java project needed to load in order to run.
In the project ':native' build.gradle:
task compile(type: Exec, group: 'build') {
dependsOn ...
outputs.files(fileTree('/some/build/directory') {
include 'mylib/libmy.so'
})
...
}
In project java application build.gradle:
configurations {
nativeDep
}
// Add dependency on the task that produces the library
dependencies {
nativeDep files(project(':native').tasks.findByPath('compile'))
}
// Unfortunately, we also have to do this because gradle will only
// run the ':native:compile' task if we needed the tasks inputs for another
// task
tasks.withType(JavaCompile) {
dependsOn ':native:compile'
}
run {
doFirst {
// Use the configuration to add our library to java.library.path
def libDirs = files(configurations.nativeDep.files.collect {it.parentFile})
systemProperty "java.library.path", libDirs.asPath
}
}

In gradle, how can I generate the ivy.xml and pom.xml files?

Currently, our gradle setup is geared to publish to certain maven repositories when a build is done. For a particular customer, I need to zip up the jars, license files, pom.xml, and ivy.xml files, and send it all in a zip. To do this, I just need to define an alternate location to publish it to. All the documentation on the gradle site seems aimed at writing one set of publishing rules, not an alternative set.
I was hoping to simply write a different task which would focus on building this customer-specific zip file. So far, I have it collecting all the jars (which includes source and runnable code) along with the license and notice file. But I haven't cracked the nut on defining a local ivy repository and a local maven repository that is only part of this alt task.
task alt {
dependsOn subprojects*.tasks*.matching { task -> task.name == 'assemble' }
subprojects.each{project ->
if (project.hasProperty('sourceJar')) {
evaluationDependsOn(project.name)
}
}
File altDir = mkdir("$buildDir/alt")
subprojects.each { project ->
if (project.hasProperty('sourceJar')) {
// Extra the module name from the path of the sub-project
String submodule = project.projectDir.absolutePath.split(File.separator).last()
File subfolder = mkdir(altDir.absolutePath + "/${project.group}/${group}.${submodule}/$version")
project.tasks.withType(Jar).each {archiveTask ->
copy {
from archiveTask.archivePath
from("$rootDir") {
include 'license.txt'
include 'notice.txt'
}
into subfolder
}
}
}
}
}
Here's the gradle docs that tells you how to generate the pom. Also if you are looking to install that file to your local repository you could use the mechanism described in this blog entry . Essentially all you have to do
configure(install.repositories.mavenInstaller) {
pom.project {
version '1.0'
artifactId 'your.artifact.id'
groupId 'your.group.id'
}
}
It doesn't look like it possible at the moment to generate the ivy.xml through gradle, but once you have your pom file you could use ivy itself to generate the ivy file described here.
Section 64.5 of http://www.gradle.org/docs/current/userguide/publishing_ivy.html covers "Generating the Ivy module descriptor file without publishing".
The documentation is a little broken (eg inconsistent naming conventions). The following works for me:
apply plugin: 'ivy-publish'
publishing {
publications {
aoeu(IvyPublication)
}
}
This will generate a target generateDescriptorFileForAoeuPublication.

Resources