I am trying to get the dependencies list for each modules/library in a project.
Here is the structure of the project:
Project
+ --- library1
+ --- library2
+ --- library3
+ --- library4
+ --- library3
The build.gradle of project is
dependencies {
compile project(':library1')
}
library2 and library4 all depend on library3. and the way they compile dependencies are the same with project.
I would want to get the dependencies for each modules/libraries so that i can decide the sequence of bumping version for these libraries and uploading to maven in a script file.
So I would first want to save these dependencies for each module in a variable and the script file can read it and do the automation work.
I have tried using Conifguration.getAllDependencies() in custom gradle build file. But it doesnt work.
Anyone know how would I do that?
Any suggestions would be much appreciated!
====
UPDATE:
I have tried adding a task under project's gradle file
List<ProjectDependency> dependencies =
project.configurations.collectMany {
it.allDependencies
}.findAll {
it instanceof ProjectDependency
}
But the dependencies returns 0 size of list
For a simple project, one can get its dependencies by invoking
project.configurations*.
dependencies*.
findAll { it instanceof ExternalDependency }.
flatten()
Note that ExternalDependency, not ProjectDependency is used.
For dealing with dependencies on a real project, gradle-versions-plugin by Ben Manes offers some nice reports
gradle-dependency-lock plugin by Netflix provides dependency update automation
Related
First of all, sorry for my poor english.
Goal
I want create multi project containing some custom libraries as subproject with gradle.
For centralized dependency version control, using buildSrc and setting versions (spring-boot, detekt, ktlint etc.)
my-core-project(root)
+buildSrc
+-src/main/kotlin
+--int-test-convention.gradle.kts
+--library-convention.gradle.kts
+library-A
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+library-B
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+build.gradle.kts
+setting.gradle.kts
buildSrc contains common tasks for libraries(integration test, detekt, etc.)
library-A and library-B are custom libraries based on spring boot.
There is no application module or any main method.
my goal is using method of library-A and/or library-B with another separated project with adding my-core-project to dependency.
Problem
./gradlew build created 3 jar files
my-core-project
+build/libs
+-my-core-project.jar
+library-A
+-build/libs
+--library-A.jar
+library-B
+-build/libs
+--library-B.jar
copied 3 jar files to libs directory under project which actually using these library,
tried adding dependency created jar
with implementation(files("libs/library-A.jar")), class and methods are resolved well.
but with implementation(files("libs/my-core-project.jar")),
class and methods are not unresolved.
when check my-core-project.jar, recognized that any information of sub projects contained.
Here is my setting.gradle.kts and build.gradle.kts of root directory.
# setting.gradle.kts
pluginManagement {
repositories {
mavenCentral()
gradlePluginPortal()
}
}
rootProject.name = "my-core-project"
include(
"library-A",
"library-B"
)
# build.gradle.kts
plugins {
id("java-library")
id("io.spring.dependency-management")
}
group = "com.demo"
version = "0.0.1-SNAPSHOT"
dependencies {
api(project(":library-A"))
api(project(":library-B"))
}
repositories {
mavenCentral()
}
Tried things
In my opinion, my-core-project.jar should be fatJar(uberJar),
so i added FatJar task like this
val fatJar = task("fatJar", type = Jar::class) {
archiveBaseName.set("${project.name}-fat")
from(configurations.runtimeClasspath.get().map { if (it.isDirectory) it else zipTree(it) })
with(tasks.jar.get() as CopySpec)
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
tasks {
"build" {
dependsOn(fatJar)
}
}
but cannot resolve class and method,
additionally occurs version conflict with other dependency of projects using this package, due to library-A created as fatJar too.
Question
Is there a simple way packaging/bundling sub-modules into one jar file?
if there are tasks like this already in gradle, prefer to use that.
Modifying fatJar task like "add jar of submodules, not contained dependencies" can solve this problem?(even couldn't try completely newbie to gradle and kts.)
if so, can you show me how to modify task?
tried shadowJar already. that solved version-conflict problem with relocate option. but still couldn't resolve package in library-A
If structure has problem, is there a good practice/example for "bundle of library"?
thanks for reading.
TL;DR
If someone faced this problem, try set archive name shorter than current one.
For someone who faced same problem, sharing some informations.
as result, resolved this problem.(maybe even not problem)
current shadowJar configure is as following
tasks.named<ShadowJar>("shadowJar").configure {
archiveBaseName.set("shorten-name")
archiveClassifier.set("")
exclude("**/*.kotlin_metadata")
exclude("**/*.kotlin_builtins")
}
exclude kotlin_metadata, kotlin_builtins
set shorten name(original project name was 30 long characters)
I have no idea but shorten jar file name has no problem.
Interesting one is, upload in artifactory package with original(long) name worked well.
I don't know Gradle declaring dependency with files has length constraints.
implementation(files("path/to/package"))
And now it works well with original name with local jar package file.
i'm currently writing a small plugin but i'm stuck when i want to get a list of all dependencies that are used.
what i'm doing
inside the plugin i create a new configuration
def config = project.configurations.create(sourceSet.getTaskName('foo', 'bar'))
in the build.gradle that uses the plugin i add some dependencies to this configuration
dependencies {
fooTestBar(project(':module'))
}
and in module/build.gradle i have
plugins {
id 'java-library'
}
dependencies {
implementation('org.apache.commons:commons-collections4:4.4')
api('org.springframework:spring-context:5.2.11.RELEASE')
}
when i now do the following in the plugin
List<ResolvedArtifact> = config.resolvedConfiguration.firstLevelModuleDependencies.allModuleArtifacts.flatten()
i get the artifacts from both declarations in :module, but what i'm interested in is only the api dependency, means the one that is also used when compiling the project
it looks like the entire configurations is treated as a runtime configuration, so i have all artifacts including the transitive ones from both declarations, instead of only the api one including the transitive ones from api
until now i was not able to find any way to see if a resolved dependency / artifact is of type api which i do not want to have in my result list
i had to add the attribute for the usage
config.attributes {
attribute( Usage.USAGE_ATTRIBUTE, objects.named( Usage, Usage.JAVA_API ) )
}
https://docs.gradle.org/current/userguide/variant_model.html
https://docs.gradle.org/current/userguide/variant_attributes.html
thanks Chris Doré on https://discuss.gradle.org/t/custom-configuration-and-resolving-only-compile-dependencies/38891
This article describes an interesting feature of Gradle 4.10+ called a source dependency:
https://blog.gradle.org/introducing-source-dependencies
It allows to use a Git (for example a GitHub) source code repository to build a dependency from it. However it seems like it supports only Gradle projects as source dependencies. Is it possible to use a Maven project as well and if it's possible, please show an example.
When I try to use this feature with Maven project Gradle tries to find the build.gradle file there anyway (I see it when run Gradle with the --info option) and fails with an error message like:
Git repository at https://github.com/something/something.git did not contain a project publishing the specified dependency.
The short answer
... is: "no".
Under the hood, source dependencies are composite builds. These needs to be Gradle projects as the external projects are sort of merged with the main project.
The long answer
... is: "yes but it is hard".
It is actually mentioned in the same blog post you linked to (emphasis mine):
Source dependencies make these use cases simpler to implement. Gradle takes care of automatically checking out the correct versions of dependencies, making sure the binaries are built when required. It does this everywhere that the build is run. The checked out project doesn’t even need to have an existing Gradle build. This example shows a Gradle build consuming two source dependencies that have no build system by injecting a Gradle build via plugins. The injected configuration could do anything a regular Gradle plugin can do, such as wrapping an existing CMake or Maven build.
Because it sounded like it wasn't the biggest thing in the world to create bridge between a Maven and a Gradle project in source dependencies, I gave it a shot. And I have it working except for transitive dependencies. You will basically need to do what is shown in the examples linked to above, but instead of building native libraries, you make a call-out to Maven (e.g. using a Maven plugin for Gradle).
However, the scripts I ended up with are complex enough that I would suggest you instead build the Maven project yourself, deploy it to a local Maven repository and then add that repository to the Gradle project.
<edit>
The loooooooong answer
Alright, so here is how to actually do it. The feature is poorly documented, and appears to be mostly targeted towards native projects (like C++ or Swift).
Root project setup
Take a normal Gradle project with the Java plugin applied. I did a "gradle init" in an empty folder. Assume that in this project, you are depending on a library called `` that you later want to include as a source dependency:
// [root]/build.gradle
dependencies {
implementation 'org.example:my-maven-project:1.1'
}
Note that the version number defined here must match a Git tag in the repository. This is the code revision that will be checkout out.
Then in the settings file, we define a source dependency mapping for it:
// [root]/settings.gradle
rootProject.name = 'my-project'
includeBuild('plugins') // [1]
sourceControl {
gitRepository("https://github.com/jitpack/maven-simple") { // [2]
producesModule("org.example:my-maven-project") // [3]
plugins {
id "external-maven-build" // [4]
}
}
}
[1]: This includes a Gradle project called plugins that will be explained later.
[2]: This is just an arbitrary Maven project that I found, which was relatively simple. Substitute with the actual repository you have.
[3]: This is the name of the Maven module (the same as in the dependency block) that we are defining a source build for
[4]: This defines a custom settings plugin called external-maven-build that is defined in the plugins project, which will be explained later.
Plugins project structure
Inside the root project, we define a new Gradle project. Again, you can use gradle init to initialize it as a Groovy (or whatever you like) project. Delete all generated sources and tests.
// [root]/plugins/settings.gradle
// Empty, but used to mark this as a stand-alone project (and not part of a multi-build)
// [root]/plugins/build.gradle
plugins {
id 'groovy'
id 'java-gradle-plugin' // [1]
}
repositories {
gradlePluginPortal() // [2]
}
dependencies {
implementation "gradle.plugin.com.github.dkorotych.gradle.maven.exec:gradle-maven-exec-plugin:2.2.1" // [3]
}
gradlePlugin {
plugins {
"external-maven-build" { // [4]
id = "external-maven-build"
implementationClass = "org.example.ExternalMavenBuilder"
}
}
}
[1]: In this project, we are defining a new Gradle plugin. This is a standard way to do that.
[2]: To invoke Maven, I am using another 3rd party plugin, so we need to add the Gradle plugin portal as a repository.
[3]: This is the plugin used to invoke Maven. I am not too familiar with it, and I don't know how production ready it is. One thing I noticed is that it does not model inputs and outputs, so there are no built-in support for up-to-date checking. But this can be added retrospectively.
[4]: This defines the custom plugin. Notice that it has the same ID as used in the settings file in the root project.
Plugin implementation class
Now comes the fun stuff. I chose to do it in Groovy, but it can be done in any supported JVM languages of cause.
The plugin structure is just like any other Gradle plugin. One thing to note is that it is a Settings plugin, whereas you normally do Project plugins. This is needed as it we are basically defining a Gradle project at run-time, which needs to be done as part of the initialization phase.
// [root]/plugins/src/main/groovy/org/example/ExternalMavenBuilder.groovy
package org.example
import com.github.dkorotych.gradle.maven.exec.MavenExec
import org.gradle.api.Plugin
import org.gradle.api.artifacts.ConfigurablePublishArtifact
import org.gradle.api.initialization.Settings
class ExternalMavenBuilder implements Plugin<Settings> {
void apply(Settings settings) {
settings.with {
rootProject.name = 'my-maven-project' // [1]
gradle.rootProject {
group = "org.example" //[2]
pluginManager.apply("base") // [3]
pluginManager.apply("com.github.dkorotych.gradle-maven-exec") // [4]
def mavenBuild = tasks.register("mavenBuild", MavenExec) {
goals('clean', 'package') // [5]
}
artifacts.add("default", file("$projectDir/target/maven-simple-0.2-SNAPSHOT.jar")) { ConfigurablePublishArtifact a ->
a.builtBy(mavenBuild) // [6]
}
}
}
}
}
[1]: Must match the Maven module name
[2]: Must match the Maven module group
[3]: Defines tasks like "build" and "clean"
[4]: The 3rd party plugin that makes it more easy to invoke Maven
[5]: For options, see https://github.com/dkorotych/gradle-maven-exec-plugin
[6]: Adds the Maven output as an artifact in the "default" configuration
Be aware that it does not model transitive dependencies, and it is never up-to-date due to missing inputs and outputs.
This is as far as I got with a few hours of playing around with it. I think it can be generalized into a generic plugin published to the Gradle portal. But I think I have too much on my plate as it is already. If anyone would like to continue on from here, you have my blessing :)
I'm using intellij (2019.1.1) for a java gradle (5.4.1) project and use lombok (1.18.6) for autogenerating code. Intellij puts generated sources under out/production/classes/generated/... and gradle puts them under build/generated/sources/...
This is fine, and I like that intellij keeps it's own build artifacts separate from gradles, however, intellij seems to look in both directories when running projects and it complains about the duplicate generated classes.
What is the best practice for using intellij with gradle and autogenerated sources? Do you:
tell intellij to output to the same directory as gradle (this
could lead to odd behaviour if a process outside of intellij updates
a file under build/)
tell intellij to perform all tasks with
gradle (i hear this is slower than intellij's make)
tell intellij
to simply ignore the 'build' directory (how do you even do this? and
why does intellij even care about 'build/' when it knows it outputs
to 'out/')
UPDATE: to clarify the situation, the issue is NOT with lombok autogenerated code, it is with hibernate-jpamodelgen. The problem remains the same (duplicate generated sources) but I want to clarify the it is the sources generated by jpamodelgen and not lombok.
UPDATE 2: I have tried the following configuration in an attempt to tell intellij where the generated sources live and also to tell intellij to ignore the build directory. Sadly, this did not work (still get duplicate class error on the generated source files).
apply plugin: 'idea'
idea {
module {
sourceDirs += file('out/production/classes/generated')
generatedSourceDirs += file('out/production/classes/generated')
excludeDirs += file('build')
}
}
UPDATE 3:
Tried the advice from M.Riccuiti and deleted build/, out/, .idea/, .gradle/ and reimported the gradle project but intellij is still seeing the generated sources in the build/ directory.
Here is an approach that finally worked for me. The trick is to notice that when gradle generates the classes, it puts them in:
build\generated\sources\annotationProcessor\java\main\com...
but intellij has the production sources directory set to "generated" in this case, the sources go to:
build\generated\sources\annotationProcessor\java\main\generated\com...
if you compile with gradle first and then use idea, you get both of them, which causes the problem!
To solve this, replace "generated" and "generated_test" in the intellij annotation processors "production sources directory " and "test sources directory " configuration with just a "/" this makes both gradle and intellij generate the sources in the SAME directory, overwriting each other as needed. Also make sure that the "store generated sources relative to" is set to "module content root" and REBUILD the application to clear out any other sources.
The solution I proposed in previous comment was working fine in IDEA 2018.3.x but after upgrading to IDEA 2019.1 I again got this duplicate class exception...
Below is a working solution to make this work with Gradle 5.x (tested with 5.4) and IDEA 2019.1 , to implement your solution #3 which I think is the best option (do not mix gradle & idea generated output directories, and do not delegate IDEA action do Gradle )
The key point is to use excludeDirs property from idea.module extension to make IDEA ignore generated sources managed by Gradle under build/generated/sources/...
ext {
// path to Gradle generated main sources directory
gradleGeneratedMainSourcesDir = "$buildDir/generated/sources/annotationProcessor/java/main/"
// path to Gradle generated test sources directory
gradleGeneratedTestSourcesDir = "$buildDir/generated/sources/annotationProcessor/java/test/"
// path to IDEA generated sources directory
ideaGeneratedSourcesDir = "$projectDir/out/production/classes/generated"
}
idea {
module {
// exclude main & test sources generated by Gradle from project source directories
excludeDirs += file(gradleGeneratedMainSourcesDir)
excludeDirs += file(gradleGeneratedTestSourcesDir)
// include generated sources directory managed by IDEA
sourceDirs += file(ideaGeneratedSourcesDir)
generatedSourceDirs += file(ideaGeneratedSourcesDir)
}
}
See complete sample project based on this configuration here : https://github.com/mricciuti/sample-springboot-gradle-idea
You can enter to IntelliJ Settings (Preferences):
Preferences | Build, Execution, Deployment | Build Tools | Gradle | Runner
Then you tick the checkbox Delegate IDE build/run action to Gradle
Finally, you clean and build again. The issues will be resolved.
I am building an Android Library project using the Android Gradle plugin (version 0.9.2) and it appears to have a bug (reported) in that while a "provided" dependency is correctly handled (not included) in the generated aar artifact, that dependency is incorrectly included in the generated debug test apk file.
It strikes me that a reasonable workaround is to remove the dependency jar file that is added by the :preDexDebugTest task as the last step for that task. But this task is dynamically generated so getting a handle to it is eluding me at the moment, hence the question.
In your app's build.gradle file, add
afterEvaluate {
def preDexDebugTest = tasks['preDexDebugTest']
// Do something with preDexDebugTest ...
}
That way it should be possible to operate on the preDexDebugTest task.