How does gradle choose actually libraries ending with "-jvm"? - maven

I'm migrating from Gradle to Bazel.
I had in my gradle build a testImplementation dependency to io.kotest:kotest-runner-junit5:5.4.2. It works perfectly.
I add the same dependency to my Bazel config files (WORKSPACE and BUILD), but I get compilation errors, as if the library doesn't exist.
I go and check if Bazel doesn't bring transitive dependencies, but it does.
I check the POM of the library and it turns out it has no dependencies.
I see in maven there's another called io.kotest:kotest-runner-junit5-jvm:5.4.2.
I use that one instead. Voilá, it works!
But why? how is gradle picking the -jvm artifact instead?

There are 3 parts to your question
How does Gradle what variants are available?
How does Gradle find the variants?
How does Gradle figure out how to pick the right variant?
The short answer is that
Gradle publishes an additional Module Metadata file that contains info on what variants are available, and where to find them.
Gradle doesn't use just the Maven group:artifact:version coordinates to find dependencies - it uses variant attributes to 'tag' available modules and the requests for modules, so it can match the requested dependency with those available in a very fine grained way.
Kotlin Multiplatform Variants
Kotlin Multiplatform libraries publish several artifacts: a 'common' published artifact, and a variant for each platform that it targets.
For io.kotest:kotest-runner-junit5 that means there's
A 'common' library
https://repo1.maven.org/maven2/io/kotest/kotest-runner-junit5/5.4.2/
and a -jvm variant
https://repo1.maven.org/maven2/io/kotest/kotest-runner-junit5-jvm/5.4.2/
Typically dependencies are determined by Maven POM and maven-metadata.xml files. That will be what Bazel is doing. The same is true for a project that uses Maven. So how does Gradle figure out what variants are available, and which ones to use?
Variant availability: Gradle Module Metadata
Gradle publishes an additional metadata file. https://docs.gradle.org/current/userguide/publishing_gradle_module_metadata.htm. It's like a pom.xml, but with a lot more info. The extension is .module, but the content is JSON.
Looking in https://repo1.maven.org/maven2/io/kotest/kotest-runner-junit5/5.4.2/, we can see the file.
Because it's JSON, we can look inside. There's some metadata
{
"formatVersion": "1.1",
"component": {
"group": "io.kotest",
"module": "kotest-runner-junit5",
"version": "5.4.2",
"attributes": {
"org.gradle.status": "release"
}
},
...
There's also a variants array. One of the variants is the -jvm variant, as well as a available-at.url, which is a relative path linking to the available variants within the Maven repository.
...
"variants": [
...
{
"name": "jvmRuntimeElements-published",
"attributes": {
"org.gradle.category": "library",
"org.gradle.libraryelements": "jar",
"org.gradle.usage": "java-runtime",
"org.jetbrains.kotlin.platform.type": "jvm"
},
"available-at": {
"url": "../../kotest-runner-junit5-jvm/5.4.2/kotest-runner-junit5-jvm-5.4.2.module",
"group": "io.kotest",
"module": "kotest-runner-junit5-jvm",
"version": "5.4.2"
}
}
...
]
}
That's how Gradle discovers the available variants.
Variant selection: Attribute matching
There's actually several variants in the module, and there would be even more if more Kotlin Multiplatform targets were enabled, so the final question "how does Gradle figure out what variant is needed?"
The answer comes from the "attributes" that are associated with the variant. They're just key-value strings that Gradle uses to match what's required, to what's available.
https://docs.gradle.org/current/userguide/variant_attributes.html#attribute_matching
The attributes might say
I want a Java 8 JAR for org.company:some-artifact:1.0.0
or
I want a Kotlin Native 1.7.0 source files for io.kotest:something:2.0.0
They're just key-value strings, so they can really be anything. I've created attributes for sharing TypeScript files, or JaCoCo XML report files.
Why do we never see these attributes when we write Gradle files?
When you add a dependency in Gradle
// build.gradle.kts
plugins {
kotlin("jvm") version "1.7.20"
}
dependencies {
testImplementation("io.kotest:kotest-runner-junit5:5.4.2")
}
There's no attributes. So how does Gradle know to select the -jvm variant?
Gradle sees you've added a dependency using testImplementation, which is a Configuration.
(Aside: I think the name 'Configuration' is confusing. It's not configuration for how the project behaves. I like to think of it more like how a group of naval warships might have a 'configuration' for battle, or a 'configuration' for loading supplies. It's more about the 'shape', and it's not about controlling Gradle properties or actions.)
When Configurations are defined, they're also tagged with attributes, which Gradle will use to play matchmaker between the request for "kotest-runner-junit5" and what it discovers in the registered repositories
In the case of testImplementation("io.kotest:kotest-runner-junit5:5.4.2"), Gradle can see that testImplementation has Attributes that say "I need a JVM variant", and it can use that to find a matching dependency using the module metadata in Maven Central.

Related

Gradle - don't specify version in included dependencies names

My application gets packaged as ear and I have used earlib and deploy configuration. However for all those dependencies version gets mentioned in the jar names.
For example, if I mention dependencies as below,
earlib 'com.xyz:abc:1.0.1'
In generated ear I can see jar name as abc-1.0.1.jar but I want to get it included simply as abc.jar.
Declare a dependency without a version
Gradle lets you declare a dependency without a version but you have to define a dependency constraint, which basically is the definition of your dependency version. This is commonly used in large projects:
dependencies {
implementation 'org.springframework:spring-web'
}
dependencies {
constraints {
implementation 'org.springframework:spring-web:5.0.2.RELEASE'
}
}
Declare a dynamic version
Another option is to declare a dynamic version by using the plus operator. This allows you to use the latest relase of a dependency while you pack your application. Doing so is potentially dangerous since its bears the risk of breaking the application:
apply plugin: 'java-library'
repositories {
mavenCentral()
}
dependencies {
implementation 'org.springframework:spring-web:5.+'
}
Declaring a file dependency
If you don't want to rely on a binary repository at all but provide dependencies yourself, you can declare a file dependency, from the directories ant, libs and tools. This allows you to name and version dependencies as you like but you have to maintain them yourself:
configurations {
antContrib
externalLibs
deploymentTools
}
dependencies {
antContrib files('ant/antcontrib.jar')
externalLibs files('libs/commons-lang.jar', 'libs/log4j.jar')
deploymentTools fileTree(dir: 'tools', include: '*.exe')
}
Note I recommend against removing the versions as they are important diagnostic information when the application doesn't work.
The ear task is an instance of the Ear task type, which in turn is basically a specialised form of the standard Zip task type. All archiving tasks allow you to rename files as they are packed.
For example, the following might work:
ear {
rename '(.+)-[^-].+(\\.jar)', '$1$2'
lib {
rename '(.+)-[^-].+(\\.jar)', '$1$2'
}
}
I strongly recommend that you check out the new user manual chapter on Working with files for more information about copying and archiving files. Hopefully I'll remember to update this answer with the non-release-candidate link once Gradle 4.7 is out.
Also, if you have any feedback on that chapter let me know.
EDIT Based on OP's feedback, I discovered that the Ear task uses a child copy specification for the JARs in the earlib configuration. Child specifications are independent of both the main one and other child specs, so the main rename() doesn't apply to the earlib files. That's why we add a rename() via the lib {} block.

What is supposed to happen to dependencies after gradle build?

I am trying out Gradle, and am wondering, what is supposed to happen to a project's dependencies after you run gradle build? For example, my sample projects don't run on the command line after they are built, because they are missing dependencies. They seem to compile fine, as gradle doesn't give me errors or warnings about finding the dependencies.
Gradle projects I've made in IntelliJ Idea have the same problem. They compile and run inside the IDE, but are missing dependencies and can't run on the command line.
So what is supposed to happen to the dependencies I declare in the build.gradle file? Shouldn't they be output somewhere together with my .class files? Otherwise, what is the point of gradle when I could manage this by editing my classpath?
Edit: Here is my build.gradle file:
apply plugin: 'java'
jar {
manifest {
attributes('Main-Class': 'Animals')
}
}
repositories {
flatDir{
dirs "D:\\libs\\gradleRepo"
}
}
dependencies {
compile name: "AnimalTypes-1.0-SNAPSHOT"
}
sourceSets{
main{
java {
srcDirs=['src']
}
}
}
Your Gradle build only takes care of the compile time and allows you to use the specified dependencies in your code (it adds them to the compile classpath). But it does not take care of the runtime. Once the JAR is build, you need to specify the runtime classpath and provide all required dependencies.
You may think, that this is bad or a disadvantage, but actually it is totally fine and intended, because if you build a Java library, you won't need to execute it, you just want to specify it as a dependency for another project. If you would distribute your library to a Maven repository, all dependencies from Maven repositories (module dependencies) would end up in a POM descriptor as transitive dependencies.
Now, if you want to build a runnable Java application, simply use the Gradle Application Plugin (apply plugin: 'application'), which will create a ZIP file containing the dependencies and start scripts providing your runtime classpath for execution.
Third-party plugins can also produce so-called fat JARs, which are JAR files with all dependencies included. It depends on your use case if you should use them, because often dependency management via repositories is the better way to go.

Gradle, OSGI and dependency management

I'm new to Gradle, please, help me to understand the following. I'm trying to build an OSGI web app via Intellij Idea + Gradle. I've found that Gradle has OSGI plugin, which is described here:
https://docs.gradle.org/current/userguide/osgi_plugin.html
But I have no idea on how to add dependency on, for example, org.apache.felix.dependencymanager which is OSGI bundle. So, I need this jar while compilation, and I don't need it in my resulting jar. I think, that I need something similar to maven 'provided' scope, or something like that.
P.S. Does anyone understand, what 'TBD' means in Gradle documentation? Does this means it has to be implemented in future, or is some mechanism is implemented, but is not yet described in docs?
Please check out the plugin I wrote, osgi-run, which was designed to make it extremely easy to play with OSGi without using any external tools like Eclipse (though osgi-run can generate a Manifest file for you, which you can point at from your IDE to get IDE OSGi support - this is what I do using IntelliJ), just Gradle.
With osgi-run, you just add a dependency to whatever you want as with any Java project... whether it should be provided by the environment or not does not matter at compile time, this is a deployment-time concern.
For example, add to your build.gradle file:
apply plugin: 'osgi' // or other OSGi plugin if you prefer
repositories {
mavenCentral() // add repos to get your dependencies from
}
dependencies {
compile "org.apache.felix:org.apache.felix.dependencymanager:4.3.0"
}
Note: the osgi plugin is just required to turn your jar into a bundle. osgi-run does not do that.
If you have any runtime dependencies that should be present in the OSGi environment but not in the compile classpath, do something like this:
dependencies {
...
osgiRuntime 'org.apache.felix:org.apache.felix.configadmin:1.8.8'
}
Now write some code, and once you're ready to run a OSGi container with your stuff in it, add these lines to the build.gradle file:
// this should be the first line
plugins {
id "com.athaydes.osgi-run" version "1.4.3"
}
...
// deployment to OSGi container config
runOsgi {
// which bundles do you want to add?
// transitive deps will be automatically added
bundles += project
// do not deploy jars matching these regexes (not needed, this is the default)
excludedBundles = ['org\\.osgi\\..*']
// make the manifest visible to the IDE for OSGi support
copyManifestTo file( 'auto-generated/MANIFEST.MF' )
}
Run:
gradle createOsgiRuntime
And find your full OSGi environment, ready to run, in the build/osgi directory.
Run it with:
build/osgi/run.sh # or run.bat in Windows
You can even run it during the build already:
gradle runOsgi
So you probably want to make your own provided configuration.
configurations {
// define new scope
provided
}
sourceSets {
// add the configurations to the compile classpath but not runtime
main.compileClasspath += configurations.provided
// be sure to add the provided configs to your tests too if needed
test.compileClasspath += configurations.provided
}
dependencies {
// declare your provided dependencies
provided 'org.apache.felix:org.apache.felix.dependencymanager:4.3.0'
}
Also the suggestion above about using the bndtool directly instead of the gradle provided osgi plugin is a good one. The gradle plugin has many deficiencies and is really just a wrapper to the bndtool anyways. Also the gradle team has declared they do not have the bandwidth or expertise to fix the osgi plugin [1].
[1] https://discuss.gradle.org/t/the-osgi-plugin-has-several-flaws/2546/5

Why does Gradle jar of jars result in duplicate libraries?

Objective: Create Jar of Jars like Maven does.
I have this relevant snippet:
jar {
into('lib') {
from configurations.compile, configurations.runtime
}
}
There must be something wrong with my conceptualization of what compile / runtime are. If I do this, I get duplicate copies of every library in the resulting jar. I'm really only trying to include everything that might go in and cover all the scopes. I want something I can easily rubber stamp for most jar situations as this is a very large build. I can get more specific for war files.
So... Why does this happen?
The gradle documentation says this about the runtime configuration:
"runtime: The dependencies required by the production classes at runtime. By default, also includes the compile time dependencies."
Just by copying from the runtime configuration, you would get all of the compile dependencies aswell. You are basically telling gradle to copy the same dependencies twice.

Gradle: What Is The Default Configuration and How Do I Change It

When I run the "dependencies" task there are several sections: compile, runtime, testCompile ...
One of those sections is "default - Configuration for default artifacts." What is this section and what is it used for?
How do I change what is in the "default configuration"?
Details: Gradle 1.7
Unless your build is publishing Ivy modules, the default configuration is mainly relevant when dealing with project dependencies in a multi-project build. Given a multi-project build with projects A and B, if A declares a project dependency on B without explicitly naming a configuration (e.g. dependencies { compile project(":B") }, A (more precisely A's compile configuration) will depend on project B's default configuration. In other words, dependencies { compile project(":B") } is a shortcut for dependencies { compile project(path: ":B", configuration: "default") }.
The default configuration extends from the runtime configuration, which means that it contains all the dependencies and artifacts of the runtime configuration, and potentially more. You can add dependencies and artifacts in the usual way (using a dependencies/artifacts block in B's build script). Alternatively, B could declare a custom configuration, and A could depend on that by explicitly naming that configuration (e.g. dependencies { compile project(path: ":B", configuration: "myCustomConfig") }.
When using the gradle java plugin the 'default' configuration extendsFrom 'runtime', 'runtimeOnly', 'implementation'
If you do not use the java plugin then you can define it yourself like this
configurations {
"default"
}
The java plugin sets up the default configuration here: https://github.com/gradle/gradle/blob/85d30969f4672bb2739550b4de784910a6810b7a/subprojects/plugins/src/main/java/org/gradle/api/plugins/JavaPlugin.java#L437
The documentation is not that good in this area.
An example of "serving" a default artifact from a composite build.
The example creates a subproject that refers to a dependency in another project. This can be nessesary when working with composite builds, as only the "default" group can be depended upon.
We use this to take many jars from a single project and serve it as different dependencies when referencing the project as a composite build.
apply plugin: 'base'
configurations {
depend
}
dependencies {
depend project(path: ':', configuration: 'ConfWithArtifact')
}
artifacts {
"default" (file: configurations.depend.singleFile) {
builtBy(configurations.depend)
}
}
The default configuration is actually created by the base plugin, and so you don't need to define it yourself.
I've also had the problem with composite builds only compositing from the default configuration, but I solved it slightly differently:
plugins {
id 'base'
}
configurations {
bootstrap
it.'default'.extendsFrom bootstrap
}
dependencies {
bootstrap project(path: ':other', configuration: 'otherConfiguration')
}
This approach allows the artifact from the :other project to keep its transitive dependencies, assuming you're interested in keeping them.

Resources