i'm currently writing a small plugin but i'm stuck when i want to get a list of all dependencies that are used.
what i'm doing
inside the plugin i create a new configuration
def config = project.configurations.create(sourceSet.getTaskName('foo', 'bar'))
in the build.gradle that uses the plugin i add some dependencies to this configuration
dependencies {
fooTestBar(project(':module'))
}
and in module/build.gradle i have
plugins {
id 'java-library'
}
dependencies {
implementation('org.apache.commons:commons-collections4:4.4')
api('org.springframework:spring-context:5.2.11.RELEASE')
}
when i now do the following in the plugin
List<ResolvedArtifact> = config.resolvedConfiguration.firstLevelModuleDependencies.allModuleArtifacts.flatten()
i get the artifacts from both declarations in :module, but what i'm interested in is only the api dependency, means the one that is also used when compiling the project
it looks like the entire configurations is treated as a runtime configuration, so i have all artifacts including the transitive ones from both declarations, instead of only the api one including the transitive ones from api
until now i was not able to find any way to see if a resolved dependency / artifact is of type api which i do not want to have in my result list
i had to add the attribute for the usage
config.attributes {
attribute( Usage.USAGE_ATTRIBUTE, objects.named( Usage, Usage.JAVA_API ) )
}
https://docs.gradle.org/current/userguide/variant_model.html
https://docs.gradle.org/current/userguide/variant_attributes.html
thanks Chris Doré on https://discuss.gradle.org/t/custom-configuration-and-resolving-only-compile-dependencies/38891
Related
First of all, sorry for my poor english.
Goal
I want create multi project containing some custom libraries as subproject with gradle.
For centralized dependency version control, using buildSrc and setting versions (spring-boot, detekt, ktlint etc.)
my-core-project(root)
+buildSrc
+-src/main/kotlin
+--int-test-convention.gradle.kts
+--library-convention.gradle.kts
+library-A
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+library-B
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+build.gradle.kts
+setting.gradle.kts
buildSrc contains common tasks for libraries(integration test, detekt, etc.)
library-A and library-B are custom libraries based on spring boot.
There is no application module or any main method.
my goal is using method of library-A and/or library-B with another separated project with adding my-core-project to dependency.
Problem
./gradlew build created 3 jar files
my-core-project
+build/libs
+-my-core-project.jar
+library-A
+-build/libs
+--library-A.jar
+library-B
+-build/libs
+--library-B.jar
copied 3 jar files to libs directory under project which actually using these library,
tried adding dependency created jar
with implementation(files("libs/library-A.jar")), class and methods are resolved well.
but with implementation(files("libs/my-core-project.jar")),
class and methods are not unresolved.
when check my-core-project.jar, recognized that any information of sub projects contained.
Here is my setting.gradle.kts and build.gradle.kts of root directory.
# setting.gradle.kts
pluginManagement {
repositories {
mavenCentral()
gradlePluginPortal()
}
}
rootProject.name = "my-core-project"
include(
"library-A",
"library-B"
)
# build.gradle.kts
plugins {
id("java-library")
id("io.spring.dependency-management")
}
group = "com.demo"
version = "0.0.1-SNAPSHOT"
dependencies {
api(project(":library-A"))
api(project(":library-B"))
}
repositories {
mavenCentral()
}
Tried things
In my opinion, my-core-project.jar should be fatJar(uberJar),
so i added FatJar task like this
val fatJar = task("fatJar", type = Jar::class) {
archiveBaseName.set("${project.name}-fat")
from(configurations.runtimeClasspath.get().map { if (it.isDirectory) it else zipTree(it) })
with(tasks.jar.get() as CopySpec)
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
tasks {
"build" {
dependsOn(fatJar)
}
}
but cannot resolve class and method,
additionally occurs version conflict with other dependency of projects using this package, due to library-A created as fatJar too.
Question
Is there a simple way packaging/bundling sub-modules into one jar file?
if there are tasks like this already in gradle, prefer to use that.
Modifying fatJar task like "add jar of submodules, not contained dependencies" can solve this problem?(even couldn't try completely newbie to gradle and kts.)
if so, can you show me how to modify task?
tried shadowJar already. that solved version-conflict problem with relocate option. but still couldn't resolve package in library-A
If structure has problem, is there a good practice/example for "bundle of library"?
thanks for reading.
TL;DR
If someone faced this problem, try set archive name shorter than current one.
For someone who faced same problem, sharing some informations.
as result, resolved this problem.(maybe even not problem)
current shadowJar configure is as following
tasks.named<ShadowJar>("shadowJar").configure {
archiveBaseName.set("shorten-name")
archiveClassifier.set("")
exclude("**/*.kotlin_metadata")
exclude("**/*.kotlin_builtins")
}
exclude kotlin_metadata, kotlin_builtins
set shorten name(original project name was 30 long characters)
I have no idea but shorten jar file name has no problem.
Interesting one is, upload in artifactory package with original(long) name worked well.
I don't know Gradle declaring dependency with files has length constraints.
implementation(files("path/to/package"))
And now it works well with original name with local jar package file.
I think I am missing some point in how dependencies management works in gradle.
Let's say I have the following project structure:
project
---api
------api-commons
------api-v1
------api-v2
------api-v3
where all api* directories are modules. All api-v* need a particular dependency (let's say common-dependency).
My aim would be to import it in api-commons build.gradle file:
dependencies {
implementation 'common-dependency'
}
while in the build.gradle files of the other modules api-v* put:
dependencies{
implementation project(':api:api-commons')
}
I would expect this to work but it doesn't. The code in the api-v* modules simply acts like the dependency is not declared. Indeed, if I import the dependency in the single modules, the code works as expected.
Am I doing a wrong assumption? Doesn't the dependency inheritance work like that?
Declaring a dependency in the implementation configuration conceptually means it is internal to the module (it is used in the implementation but not part of the public API). Such a dependency will not be exposed to the compile classpath of consumers, though it will still be on the runtime classpath.
An advantage of this way of modelling dependencies is that you do not need to recompile consuming projects if an implementation dependency changes. Another is that by encapsulating them, it is less likely that a consumer will start depending on them directly and then breaking if you change them.
If you want to expose the dependency to consumers, you have to declare it as part of the API for the module. You do that by applying the java-library plugin and using the api configuration.
Example:
// api-commons/build.gradle
plugins {
id 'java-library'
}
dependencies {
api 'common-dependency'
}
Read more about it in the Gradle user guide
Let say, I have to moved below common code(*.java files) from below 2 service/modules to sharedprocessing-data which is present inside shared-libraries service/modules :
abc-service
xyz-servcie
Address.java
Employee.java
Department.java
Vaccation.java
Name.java
City.java
Order.java
Steps0:
In shared- service/modules folder create additional module inside existing shared-libraries module
Name it as sharedprocessing-data
Steps1:
Move(refactor) common code inside this module
Steps2:
In parent folder (root) update settings.gradle file
rootProject.name = "root"
include 'abc-service'
include 'xyz-service'
include 'shared-libraries:sharedprocessing-data'
Step3:
In each of abc-service and xyz-flow service modules, update build.gradle file
dependencies
{
implementation project(':shared-libraries:sharedprocessing-data')
}
I am very excited about the incubating Gradle's version catalogs and have been experimenting with it. I’ve found that the information in my gradle/libs.versions.toml is accessible in the build.gradle.kts scripts for my app and utility-lib projects.
However, I am unable to use the content of the toml file for buildSrc/build.gradle.kts or the convention files.
The only way that I could build was to hard-code the dependencies into those files, as I did before the version catalog feature.
In the buildSrc folder, I created a settings.gradle.kts file and inserted the dependencyResolutionManagement code for versionCatalogs, which is pointing to the same file as for my app and utility-lib projects.
Based on the Gradle7 docs, it seems that sharing a version catalog with buildSrc and modules is possible… I’d appreciate a nudge into getting it to work with buildSrc, if possible.
Here is a simple sample project, which I created via gradle init: my-version-catalog
Thank you for your time and help,
Mike
With Gradle 7.3.3, it is possible. Note version catalogs are GA since Gradle 7.4
The code snippet assumes Gradle is at least 7.4, but if you need them prior that version, insert enableFeaturePreview("VERSION_CATALOGS") at the beginning of each settings.gradle.kts.
Using buildSrc
buildSrc/settings.gradle.kts
dependencyResolutionManagement {
versionCatalogs {
create("libs") {
from(files("../gradle/libs.versions.toml"))
}
}
}
buildSrc/build.gradle.kts
dependencies {
implementation(libs.gradleplugin.intellij) // <- the lib reference
}
You can even use the version catalog for plugins
gradle/libs.versions.toml
...
[plugins]
kotlin-jvm = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" }
jetbrains-changelog = { id = "org.jetbrains.changelog", version.ref = "changelog-plugin" }
jetbrains-intellij = { id = "org.jetbrains.intellij", version.ref = "intellij-plugin" }
hierynomus-license = { id = "com.github.hierynomus.license", version.ref = "license-plugin" }
nebula-integtest = { id = "nebula.integtest", version.ref = "nebula-integtest-plugin" }
build.gradle.kts
plugins {
id("java")
alias(libs.plugins.kotlin.jvm)
alias(libs.plugins.nebula.integtest)
alias(libs.plugins.jetbrains.intellij)
alias(libs.plugins.jetbrains.changelog)
alias(libs.plugins.hierynomus.license)
}
Note for accessing the catalog within scripts, please refer to the below section, the trick is the same.
Using convention plugins and included build
In the main project include a the Gradle project that holds the convention plugins.
build.gradle.kts
includeBuild("convention-plugins") // here it's a subfolder
convention-plugins/settings.gradle.kts
dependencyResolutionManagement {
repositories {
gradlePluginPortal()
}
versionCatalogs {
create("libs") {
from(files("../gradle/libs.versions.toml"))
}
}
}
rootProject.name = "convention-plugins"
The trick to enable convention plugins to access the version catalog is split in two part, add an ugly implementation dependency that locate where the version catalog generated classes are located.
libs.javaClass.superclass.protectionDomain.codeSource.location
Then in the convention plugin refer to the libs extension via Project::the.
val libs = the<LibrariesForLibs>()
This is tracked by gradle/gradle#15383.
convention-plugins/build.gradle.kts
plugins {
`kotlin-dsl`
}
dependencies {
implementation(libs.gradleplugin.kotlin.jvm)
// https://github.com/gradle/gradle/issues/15383
implementation(files(libs.javaClass.superclass.protectionDomain.codeSource.location))
}
And in the actual convention plugin
import org.gradle.accessors.dm.LibrariesForLibs
plugins {
id("org.jetbrains.kotlin.jvm")
}
// https://github.com/gradle/gradle/issues/15383
val libs = the<LibrariesForLibs>()
dependencies {
detektPlugins(libs.bundles.kotlinStuff) // access catalog entries
}
The org.gradle.accessors.dm.LibrariesForLibs class is generated by gradle is somewhere in local gradle folder ./gradle/<version>/dependency-accessors/<hash>/classes
Quick note that older IntelliJ IDEA currently (2022.3) reports alias(libs.gradleplugin.thePlugin) as an error in the editor,
although the dependencies are correctly resolved.
This tracked by KTIJ-19369, the ticket indicates this is actually a bug in Gradle Kotlin DSL gradle/gradle#22797, and someone made a simple IntelliJ IDEA plugin to hide this error until resolved.
Brice, it looks like a can of worms to go down that path, particularly for my situation, where I'm trying to use a libs.version.toml file from an android project, but the custom plugin is of course from a java/kotlin project. I tried creating the libs file by hardwiring the path to the toml file in the custom plugin. It might work if both were java projects, but I never tried that since that's not what I'm after. The ideal solution would be for the plugin to use the libs file from the project it is applied to, but it looks like the version catalog needs to be created in the settings file, before you even have access to "Project", so that's why you would have to hardwire the path.
Short answer. No, but there are other techniques for a custom plugin to get project version data from the project it is applied to.
This article describes an interesting feature of Gradle 4.10+ called a source dependency:
https://blog.gradle.org/introducing-source-dependencies
It allows to use a Git (for example a GitHub) source code repository to build a dependency from it. However it seems like it supports only Gradle projects as source dependencies. Is it possible to use a Maven project as well and if it's possible, please show an example.
When I try to use this feature with Maven project Gradle tries to find the build.gradle file there anyway (I see it when run Gradle with the --info option) and fails with an error message like:
Git repository at https://github.com/something/something.git did not contain a project publishing the specified dependency.
The short answer
... is: "no".
Under the hood, source dependencies are composite builds. These needs to be Gradle projects as the external projects are sort of merged with the main project.
The long answer
... is: "yes but it is hard".
It is actually mentioned in the same blog post you linked to (emphasis mine):
Source dependencies make these use cases simpler to implement. Gradle takes care of automatically checking out the correct versions of dependencies, making sure the binaries are built when required. It does this everywhere that the build is run. The checked out project doesn’t even need to have an existing Gradle build. This example shows a Gradle build consuming two source dependencies that have no build system by injecting a Gradle build via plugins. The injected configuration could do anything a regular Gradle plugin can do, such as wrapping an existing CMake or Maven build.
Because it sounded like it wasn't the biggest thing in the world to create bridge between a Maven and a Gradle project in source dependencies, I gave it a shot. And I have it working except for transitive dependencies. You will basically need to do what is shown in the examples linked to above, but instead of building native libraries, you make a call-out to Maven (e.g. using a Maven plugin for Gradle).
However, the scripts I ended up with are complex enough that I would suggest you instead build the Maven project yourself, deploy it to a local Maven repository and then add that repository to the Gradle project.
<edit>
The loooooooong answer
Alright, so here is how to actually do it. The feature is poorly documented, and appears to be mostly targeted towards native projects (like C++ or Swift).
Root project setup
Take a normal Gradle project with the Java plugin applied. I did a "gradle init" in an empty folder. Assume that in this project, you are depending on a library called `` that you later want to include as a source dependency:
// [root]/build.gradle
dependencies {
implementation 'org.example:my-maven-project:1.1'
}
Note that the version number defined here must match a Git tag in the repository. This is the code revision that will be checkout out.
Then in the settings file, we define a source dependency mapping for it:
// [root]/settings.gradle
rootProject.name = 'my-project'
includeBuild('plugins') // [1]
sourceControl {
gitRepository("https://github.com/jitpack/maven-simple") { // [2]
producesModule("org.example:my-maven-project") // [3]
plugins {
id "external-maven-build" // [4]
}
}
}
[1]: This includes a Gradle project called plugins that will be explained later.
[2]: This is just an arbitrary Maven project that I found, which was relatively simple. Substitute with the actual repository you have.
[3]: This is the name of the Maven module (the same as in the dependency block) that we are defining a source build for
[4]: This defines a custom settings plugin called external-maven-build that is defined in the plugins project, which will be explained later.
Plugins project structure
Inside the root project, we define a new Gradle project. Again, you can use gradle init to initialize it as a Groovy (or whatever you like) project. Delete all generated sources and tests.
// [root]/plugins/settings.gradle
// Empty, but used to mark this as a stand-alone project (and not part of a multi-build)
// [root]/plugins/build.gradle
plugins {
id 'groovy'
id 'java-gradle-plugin' // [1]
}
repositories {
gradlePluginPortal() // [2]
}
dependencies {
implementation "gradle.plugin.com.github.dkorotych.gradle.maven.exec:gradle-maven-exec-plugin:2.2.1" // [3]
}
gradlePlugin {
plugins {
"external-maven-build" { // [4]
id = "external-maven-build"
implementationClass = "org.example.ExternalMavenBuilder"
}
}
}
[1]: In this project, we are defining a new Gradle plugin. This is a standard way to do that.
[2]: To invoke Maven, I am using another 3rd party plugin, so we need to add the Gradle plugin portal as a repository.
[3]: This is the plugin used to invoke Maven. I am not too familiar with it, and I don't know how production ready it is. One thing I noticed is that it does not model inputs and outputs, so there are no built-in support for up-to-date checking. But this can be added retrospectively.
[4]: This defines the custom plugin. Notice that it has the same ID as used in the settings file in the root project.
Plugin implementation class
Now comes the fun stuff. I chose to do it in Groovy, but it can be done in any supported JVM languages of cause.
The plugin structure is just like any other Gradle plugin. One thing to note is that it is a Settings plugin, whereas you normally do Project plugins. This is needed as it we are basically defining a Gradle project at run-time, which needs to be done as part of the initialization phase.
// [root]/plugins/src/main/groovy/org/example/ExternalMavenBuilder.groovy
package org.example
import com.github.dkorotych.gradle.maven.exec.MavenExec
import org.gradle.api.Plugin
import org.gradle.api.artifacts.ConfigurablePublishArtifact
import org.gradle.api.initialization.Settings
class ExternalMavenBuilder implements Plugin<Settings> {
void apply(Settings settings) {
settings.with {
rootProject.name = 'my-maven-project' // [1]
gradle.rootProject {
group = "org.example" //[2]
pluginManager.apply("base") // [3]
pluginManager.apply("com.github.dkorotych.gradle-maven-exec") // [4]
def mavenBuild = tasks.register("mavenBuild", MavenExec) {
goals('clean', 'package') // [5]
}
artifacts.add("default", file("$projectDir/target/maven-simple-0.2-SNAPSHOT.jar")) { ConfigurablePublishArtifact a ->
a.builtBy(mavenBuild) // [6]
}
}
}
}
}
[1]: Must match the Maven module name
[2]: Must match the Maven module group
[3]: Defines tasks like "build" and "clean"
[4]: The 3rd party plugin that makes it more easy to invoke Maven
[5]: For options, see https://github.com/dkorotych/gradle-maven-exec-plugin
[6]: Adds the Maven output as an artifact in the "default" configuration
Be aware that it does not model transitive dependencies, and it is never up-to-date due to missing inputs and outputs.
This is as far as I got with a few hours of playing around with it. I think it can be generalized into a generic plugin published to the Gradle portal. But I think I have too much on my plate as it is already. If anyone would like to continue on from here, you have my blessing :)
I have a task that generates java sources and a set of jars from these sources (say, project a). I would like to export these jars to dependent projects (say, project b). So here's roughly what I have right now:
//a.gradle
configurations{
generatedJars
}
task generateJars(type: JavaExec) {
//generate jars ...
outputs.files += //append generated jars here
}
dependencies{
generatedJars generateJars.outputs.files
}
//b.gradle
dependencies{
project(path: ':a', configuration: 'generatedJars')
}
It works OK, except that adding generateJars.outputs.files as a dependency does not tell gradle that it has to run generateJars task when there are no jars generated yet. I have tried adding the task itself as a dependency hoping that it would work in the same way as it does when you add a jar/zip task to an artifact configuration (e.g. artifacts{ myJarTask }), but it throws an error telling me that I cannot do that. Of course I can inject the generateJars task somewhere in the build process before :b starts evaluating, but that's clumsy and brittle, so I would like to avoid it.
I feel like I should be adding the generated jars to artifacts{ ... } of the project, but I am not sure how to make them then visible to dependent projects. Is there a better way of achieving this?
Dependent projects (project b) will need to do setup IntelliJ IDEA module classpath to point to project a's generated jars. Something rather like this (pseudo-code):
//b.gradle
idea{
module{
scopes.COMPILE.plus += project(path: ':a', configuration: 'generatedJars').files
}
}
So far I have tried simply adding a project dependecy on :a's generatedJars in :b, but Idea plugin simply adds module :a as a module-dependency and assumes that it exports its generated jars (which is probably a correct assumption), therefore not adding the generated jars to :b's classpath.
Any help would be greatly appreciated!
First, do you need a separate configuration? That is, do you have clients of a that should not see the generated Jars? If not, you can add the generated Jars to the archives configuration, which will simplify things.
Second, the correct way to add the generated Jars to the configuration is (instead of the dependencies block):
artifacts {
generatedJars generateJars
}
This should make sure that the generateJars task gets run automatically when needed.
Third, I'd omit the += after outputs.files, although it might not make a difference. You should also add the necessary inputs.
Fourth, why do you need a JavaExec task to generate the Jars? Can you instead add the generated sources to some source set and let Gradle build them?
Fifth, IDEA doesn't have a concept corresponding to Gradle's project configuration dependencies. Either an IDEA module fully depends on another module, or not at all. You have two options: either use a module dependency and make the generated sources a source folder of the depended-on module (preferably both in the Gradle and the IDEA build), or pass the generated Jars as external dependencies to IDEA. In either case, you should probably add a task dependency from ideaModule to the appropriate generation task. If this still doesn't lead to a satisfactory IDEA setup, you could think about moving the generation of the Jars into a separate subproject.
For my use case, I had a C++ project which generated some native libraries which my java project needed to load in order to run.
In the project ':native' build.gradle:
task compile(type: Exec, group: 'build') {
dependsOn ...
outputs.files(fileTree('/some/build/directory') {
include 'mylib/libmy.so'
})
...
}
In project java application build.gradle:
configurations {
nativeDep
}
// Add dependency on the task that produces the library
dependencies {
nativeDep files(project(':native').tasks.findByPath('compile'))
}
// Unfortunately, we also have to do this because gradle will only
// run the ':native:compile' task if we needed the tasks inputs for another
// task
tasks.withType(JavaCompile) {
dependsOn ':native:compile'
}
run {
doFirst {
// Use the configuration to add our library to java.library.path
def libDirs = files(configurations.nativeDep.files.collect {it.parentFile})
systemProperty "java.library.path", libDirs.asPath
}
}