Gradle cannot find prefix version transitive dependency which does not exist - gradle

I have a multi-module gradle project where in a module I add dependencies in compile configuration on runtime.
Those dependencies fetch a transitive dependency with a version prefix that does not exist.
So case is like this
compile 'group:moduleA:version.+'
This moduleA downloads moduleB with same version.+ prefix notation and that downloads another moduleC with same prefix notation, moduleC is present in artifactory with version 10 and above and 8 and below, so there are no versions which are number 9, and gradle insist on finding moduleC with version 9, it doesn't fetch versions above or below it.
How can I make gradle fetch another version if the version its trying to find is not there?
Please comment for any clarification and thanks for helping.
EDIT: Want to clarify that + in version part is not resolving to a number which is correct and present on artifactory, like 9.1 or 9.12.
gradle determines this version, which is incorrect like 9.1 is present but it resolves to 9.2 or some other number which is not there.
EDIT2: Task which is used to fetch dependencies and then add them in compile configuration.
task addAdditionalDependencies {
doLast {
Object slurper = new JsonSlurper().parseText(api.jsonResponse())
Set<String> dependencyNames = configurations.compile.dependencies.collect { it.name }
List<Map<String, String>> artifactPaths = slurper.results.collect {
String[] pathSegments = it.path.split('/')
if (!dependencyNames.contains(pathSegments[1]) && project.name != pathSegments[1]) {
[group: pathSegments[0],
name: pathSegments[1],
version: "version.+",
configuration: 'compile']
} else [:]
}
artifactPaths.each {
if (!it.isEmpty()) {
project.dependencies.add('compile', it)
}
}
// we have to call this because app does not have any source files and so compileJava does not download
// dependencies
configurations.compile.files
}
}

When using a version like 9.+, Gradle will looks for all version that matches the prefix, that is the part before the +.
There is however no way to make Gradle ignore that part in case no such version exist.
If the version of moduleC can be anything, then you could simply use + without any prefix.
Note that doing something like that could expose you to breakage in a build even though nothing changed, aside from a new version of moduleC being published.
You could also combine this dynamic version resolution with dependency locking to have a finer grained control on when to upgrade moduleC.

Related

How to create multi project fat jar as bundle of some libraries with buildSrc

First of all, sorry for my poor english.
Goal
I want create multi project containing some custom libraries as subproject with gradle.
For centralized dependency version control, using buildSrc and setting versions (spring-boot, detekt, ktlint etc.)
my-core-project(root)
+buildSrc
+-src/main/kotlin
+--int-test-convention.gradle.kts
+--library-convention.gradle.kts
+library-A
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+library-B
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+build.gradle.kts
+setting.gradle.kts
buildSrc contains common tasks for libraries(integration test, detekt, etc.)
library-A and library-B are custom libraries based on spring boot.
There is no application module or any main method.
my goal is using method of library-A and/or library-B with another separated project with adding my-core-project to dependency.
Problem
./gradlew build created 3 jar files
my-core-project
+build/libs
+-my-core-project.jar
+library-A
+-build/libs
+--library-A.jar
+library-B
+-build/libs
+--library-B.jar
copied 3 jar files to libs directory under project which actually using these library,
tried adding dependency created jar
with implementation(files("libs/library-A.jar")), class and methods are resolved well.
but with implementation(files("libs/my-core-project.jar")),
class and methods are not unresolved.
when check my-core-project.jar, recognized that any information of sub projects contained.
Here is my setting.gradle.kts and build.gradle.kts of root directory.
# setting.gradle.kts
pluginManagement {
repositories {
mavenCentral()
gradlePluginPortal()
}
}
rootProject.name = "my-core-project"
include(
"library-A",
"library-B"
)
# build.gradle.kts
plugins {
id("java-library")
id("io.spring.dependency-management")
}
group = "com.demo"
version = "0.0.1-SNAPSHOT"
dependencies {
api(project(":library-A"))
api(project(":library-B"))
}
repositories {
mavenCentral()
}
Tried things
In my opinion, my-core-project.jar should be fatJar(uberJar),
so i added FatJar task like this
val fatJar = task("fatJar", type = Jar::class) {
archiveBaseName.set("${project.name}-fat")
from(configurations.runtimeClasspath.get().map { if (it.isDirectory) it else zipTree(it) })
with(tasks.jar.get() as CopySpec)
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
tasks {
"build" {
dependsOn(fatJar)
}
}
but cannot resolve class and method,
additionally occurs version conflict with other dependency of projects using this package, due to library-A created as fatJar too.
Question
Is there a simple way packaging/bundling sub-modules into one jar file?
if there are tasks like this already in gradle, prefer to use that.
Modifying fatJar task like "add jar of submodules, not contained dependencies" can solve this problem?(even couldn't try completely newbie to gradle and kts.)
if so, can you show me how to modify task?
tried shadowJar already. that solved version-conflict problem with relocate option. but still couldn't resolve package in library-A
If structure has problem, is there a good practice/example for "bundle of library"?
thanks for reading.
TL;DR
If someone faced this problem, try set archive name shorter than current one.
For someone who faced same problem, sharing some informations.
as result, resolved this problem.(maybe even not problem)
current shadowJar configure is as following
tasks.named<ShadowJar>("shadowJar").configure {
archiveBaseName.set("shorten-name")
archiveClassifier.set("")
exclude("**/*.kotlin_metadata")
exclude("**/*.kotlin_builtins")
}
exclude kotlin_metadata, kotlin_builtins
set shorten name(original project name was 30 long characters)
I have no idea but shorten jar file name has no problem.
Interesting one is, upload in artifactory package with original(long) name worked well.
I don't know Gradle declaring dependency with files has length constraints.
implementation(files("path/to/package"))
And now it works well with original name with local jar package file.

Gradle7 Version Catalog: How to use it with buildSrc?

I am very excited about the incubating Gradle's version catalogs and have been experimenting with it. I’ve found that the information in my gradle/libs.versions.toml is accessible in the build.gradle.kts scripts for my app and utility-lib projects.
However, I am unable to use the content of the toml file for buildSrc/build.gradle.kts or the convention files.
The only way that I could build was to hard-code the dependencies into those files, as I did before the version catalog feature.
In the buildSrc folder, I created a settings.gradle.kts file and inserted the dependencyResolutionManagement code for versionCatalogs, which is pointing to the same file as for my app and utility-lib projects.
Based on the Gradle7 docs, it seems that sharing a version catalog with buildSrc and modules is possible… I’d appreciate a nudge into getting it to work with buildSrc, if possible.
Here is a simple sample project, which I created via gradle init: my-version-catalog
Thank you for your time and help,
Mike
With Gradle 7.3.3, it is possible. Note version catalogs are GA since Gradle 7.4
The code snippet assumes Gradle is at least 7.4, but if you need them prior that version, insert enableFeaturePreview("VERSION_CATALOGS") at the beginning of each settings.gradle.kts.
Using buildSrc
buildSrc/settings.gradle.kts
dependencyResolutionManagement {
versionCatalogs {
create("libs") {
from(files("../gradle/libs.versions.toml"))
}
}
}
buildSrc/build.gradle.kts
dependencies {
implementation(libs.gradleplugin.intellij) // <- the lib reference
}
You can even use the version catalog for plugins
gradle/libs.versions.toml
...
[plugins]
kotlin-jvm = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" }
jetbrains-changelog = { id = "org.jetbrains.changelog", version.ref = "changelog-plugin" }
jetbrains-intellij = { id = "org.jetbrains.intellij", version.ref = "intellij-plugin" }
hierynomus-license = { id = "com.github.hierynomus.license", version.ref = "license-plugin" }
nebula-integtest = { id = "nebula.integtest", version.ref = "nebula-integtest-plugin" }
build.gradle.kts
plugins {
id("java")
alias(libs.plugins.kotlin.jvm)
alias(libs.plugins.nebula.integtest)
alias(libs.plugins.jetbrains.intellij)
alias(libs.plugins.jetbrains.changelog)
alias(libs.plugins.hierynomus.license)
}
Note for accessing the catalog within scripts, please refer to the below section, the trick is the same.
Using convention plugins and included build
In the main project include a the Gradle project that holds the convention plugins.
build.gradle.kts
includeBuild("convention-plugins") // here it's a subfolder
convention-plugins/settings.gradle.kts
dependencyResolutionManagement {
repositories {
gradlePluginPortal()
}
versionCatalogs {
create("libs") {
from(files("../gradle/libs.versions.toml"))
}
}
}
rootProject.name = "convention-plugins"
The trick to enable convention plugins to access the version catalog is split in two part, add an ugly implementation dependency that locate where the version catalog generated classes are located.
libs.javaClass.superclass.protectionDomain.codeSource.location
Then in the convention plugin refer to the libs extension via Project::the.
val libs = the<LibrariesForLibs>()
This is tracked by gradle/gradle#15383.
convention-plugins/build.gradle.kts
plugins {
`kotlin-dsl`
}
dependencies {
implementation(libs.gradleplugin.kotlin.jvm)
// https://github.com/gradle/gradle/issues/15383
implementation(files(libs.javaClass.superclass.protectionDomain.codeSource.location))
}
And in the actual convention plugin
import org.gradle.accessors.dm.LibrariesForLibs
plugins {
id("org.jetbrains.kotlin.jvm")
}
// https://github.com/gradle/gradle/issues/15383
val libs = the<LibrariesForLibs>()
dependencies {
detektPlugins(libs.bundles.kotlinStuff) // access catalog entries
}
The org.gradle.accessors.dm.LibrariesForLibs class is generated by gradle is somewhere in local gradle folder ./gradle/<version>/dependency-accessors/<hash>/classes
Quick note that older IntelliJ IDEA currently (2022.3) reports alias(libs.gradleplugin.thePlugin) as an error in the editor,
although the dependencies are correctly resolved.
This tracked by KTIJ-19369, the ticket indicates this is actually a bug in Gradle Kotlin DSL gradle/gradle#22797, and someone made a simple IntelliJ IDEA plugin to hide this error until resolved.
Brice, it looks like a can of worms to go down that path, particularly for my situation, where I'm trying to use a libs.version.toml file from an android project, but the custom plugin is of course from a java/kotlin project. I tried creating the libs file by hardwiring the path to the toml file in the custom plugin. It might work if both were java projects, but I never tried that since that's not what I'm after. The ideal solution would be for the plugin to use the libs file from the project it is applied to, but it looks like the version catalog needs to be created in the settings file, before you even have access to "Project", so that's why you would have to hardwire the path.
Short answer. No, but there are other techniques for a custom plugin to get project version data from the project it is applied to.

Custom configuration and resolving only compile dependencies

i'm currently writing a small plugin but i'm stuck when i want to get a list of all dependencies that are used.
what i'm doing
inside the plugin i create a new configuration
def config = project.configurations.create(sourceSet.getTaskName('foo', 'bar'))
in the build.gradle that uses the plugin i add some dependencies to this configuration
dependencies {
fooTestBar(project(':module'))
}
and in module/build.gradle i have
plugins {
id 'java-library'
}
dependencies {
implementation('org.apache.commons:commons-collections4:4.4')
api('org.springframework:spring-context:5.2.11.RELEASE')
}
when i now do the following in the plugin
List<ResolvedArtifact> = config.resolvedConfiguration.firstLevelModuleDependencies.allModuleArtifacts.flatten()
i get the artifacts from both declarations in :module, but what i'm interested in is only the api dependency, means the one that is also used when compiling the project
it looks like the entire configurations is treated as a runtime configuration, so i have all artifacts including the transitive ones from both declarations, instead of only the api one including the transitive ones from api
until now i was not able to find any way to see if a resolved dependency / artifact is of type api which i do not want to have in my result list
i had to add the attribute for the usage
config.attributes {
attribute( Usage.USAGE_ATTRIBUTE, objects.named( Usage, Usage.JAVA_API ) )
}
https://docs.gradle.org/current/userguide/variant_model.html
https://docs.gradle.org/current/userguide/variant_attributes.html
thanks Chris Doré on https://discuss.gradle.org/t/custom-configuration-and-resolving-only-compile-dependencies/38891

Error when updating XText versions for Gradle 6

I am attempting to update an EMF/XCore project to newer versions to get around a versioning roadblock. Currently the repository that houses this project must target Gradle 4 and Bndtools 4.3 because of problems when targeting newer versions. I readily admit that a problem here is my lack of understanding of XText, XCore, and otherwise. Now I find myself on a machine without access to Java 8 which has forced an attempt to update the project settings if at all possible.
The last safe targets for this project were XText 2.16.0 and org.xtext.builder version 1.0.21. The examples I have been able to locate match the settings of this project for the most part.
So now, attempting to run in a Gradle 6.3 environment with OpenJDK 13 (if the Java version is the issue that can be changed)...
Notes
Some changes are based on researching solutions to my build problem and some comments are added
${mavenURL} is currently pointing to Maven Central
I added the compile platform() line based on XText's Release notes. It does not seem to help this issue, though
I changed version numbers to match those found in the Maven BOM
I have attempted various combinations of changing the org.xtext.builder version as well as targeting both EMF 2.21 and 2.20
I have tried a lower XText version. 2.17.0 fails with a different issue
build.gradle:
plugins {
id 'org.xtext.builder' version '2.0.8'
id 'java' // Helpful? Doesn't appear so
}
repositories {
maven() {
url "${mavenURL}"
}
}
ext.xtextVersion = "2.21.0"
ext.emfVersion = "2.20.0"
dependencies {
compile platform("org.eclipse.xtext:xtext-dev-bom:${xtextVersion}")
xtextLanguages 'org.eclipse.emf:org.eclipse.emf.codegen.ecore.xtext:1.4.0'
xtextLanguages('org.eclipse.emf:org.eclipse.emf.ecore.xcore:1.12.0') {
exclude group: 'org.antlr', module: 'antlr-runtime'
}
xtextLanguages "org.eclipse.xtext:org.eclipse.xtext.ecore:${xtextVersion}"
xtextLanguages "org.eclipse.xtext:org.eclipse.xtext.xbase.lib:${xtextVersion}"
xtextLanguages 'org.eclipse.emf:org.eclipse.emf.codegen:2.19.0'
xtextLanguages 'org.eclipse.emf:org.eclipse.emf.codegen.ecore:${emfVersion}'
xtextLanguages 'org.eclipse.emf:org.eclipse.emf.mwe2.runtime:2.11.2'
compile "org.eclipse.xtext:org.eclipse.xtext:${xtextVersion}"
compile "org.eclipse.xtext:org.eclipse.xtext.xbase:${xtextVersion}"
compile 'org.eclipse.emf:org.eclipse.emf.ecore.xcore.lib:1.4.0'
}
xtext {
version = "${xtextVersion}"
languages {
ecore {
setup = 'org.eclipse.xtext.ecore.EcoreSupport'
}
codegen {
setup = 'org.eclipse.emf.codegen.ecore.xtext.GenModelSupport'
}
xcore {
setup = 'org.eclipse.emf.ecore.xcore.XcoreStandaloneSetup'
generator.outlet.producesJava = true
// Have also tried generator.javaSourceLevel = '1.8'
}
}
sourceSets {
main {
srcDir 'model'
// Move the generated Xcore output to the src directory
output {
dir(xtext.languages.xcore.generator.outlet, 'src-gen')
}
}
}
}
Multiple tries have ended up resulting in the following error:
> ./gradlew jar
...
> Task :com.example.project:generateXtext FAILED
ERROR:A generic type in this context must refer to a classifier or a type parameter (file:...file.xcore)
...
> Xtext validation failed, see build log for details.
for a multitude of lines. The XCore file looks something like this
#GenModel(
complianceLevel="8.0",
loadInitialization="false",
modelDirectory="com.example.project/src-gen",
bundleManifest="false",
updateClasspath="false",
oSGiCompatible="true",
forceOverwrite="true")
#Ecore(
nsPrefix="query",
nsURI="something"
)
// These lines had to be added to resolve the above annotations no longer working
annotation "http://www.eclipse.org/emf/2002/GenModel" as GenModel
annotation "http://www.eclipse.org/emf/2002/Ecore" as Ecore
interface Node wraps an.external.Node {}
abstract class ENode extends Node
{
op Object getParent() // This line fails due to not referring to object or classifier
{
eContainer
}
}
I have seen this error pop up in bug reports on XText sites (such as https://github.com/eclipse/xtext-maven/issues/71), and they were resolved. But generally the projects reference Github examples such as https://github.com/itemis/itemis-blog/tree/xcore_gradle or https://github.com/ghillairet/xcore-gradle-example which both point to versions of XText and the corresponding builder which were indeed working prior to this upgrade attempt.
The project is an XText project. Relevant portion of the .project file is here:
<buildSpec>
<buildCommand>
<name>bndtools.core.bndbuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>org.eclipse.xtext.ui.shared.xtextBuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>org.eclipse.jdt.core.javabuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>bndtools.core.bndnature</nature>
<nature>org.eclipse.jdt.core.javanature</nature>
<nature>org.eclipse.xtext.ui.shared.xtextNature</nature>
<nature>org.eclipse.pde.PluginNature</nature>
</natures>
Any help would be appreciated.
I received assistance from the folks working on the XText Gradle plug-in via https://github.com/xtext/xtext-gradle-plugin/issues/171.
The biggest issue is that src-gen cannot exist on the classpath prior to execution of the generateXText task. To assist with this updating the clean task to remove the src-gen folder is recommended. Additionally, the second 3 compile dependencies should be xtextLanguages dependencies.

In gradle, how to use a variable for a plugin version?

One of my build scripts imports that nebula plugin:
plugins {
id 'nebula.ospackage' version '3.5.0'
}
I've been moving all of my version info into a separate file that all projects have access to and am wondering what is the correct syntax to convert to something like:
plugins {
id 'nebula.ospackage' version "$versions.nebula_gradle_ospackage_plugin"
}
When I try running the above with "gradle clean build", I get the following error:
build file 'build.gradle': 2: argument list must be exactly 1 literal
non empty string
See
https://docs.gradle.org/2.7/userguide/plugins.html#sec:plugins_block
for information on the plugins {} block
# line 2, column 33.
id 'nebula.ospackage' version "$versions.nebula_gradle_ospackage_plugin"
The linked article shows how I could use the "buildscript" block, which works, but it seems like there must be a way to make this work in a single line?
As of Gradle 5.6, you can declare your plugin versions in the gradle.properties file, and reference these properties in plugins block.
For example, the gradle.properties file:
springBootVersion=2.2.0.RELEASE
the plugins block in build.gradle:
plugins {
id "org.springframework.boot" version "${springBootVersion}"
}
See: https://github.com/gradle/gradle/issues/1697#issuecomment-506910915.
See also:
https://docs.gradle.org/5.6/release-notes.html#central-management-of-plugin-versions-with-settings-script
https://docs.gradle.org/5.6/userguide/plugins.html#sec:plugin_management
You cannot use variable here:
Where «plugin version» and «plugin id» must be constant, literal,
strings. No other statements are allowed; their presence will cause a
compilation error.
This is an old post, but the bug is still open, and I found this post looking for workarounds.
It seems you were aware of this already, and actually have BoygeniusDexter's answer, but I think this may help others finding this post like I did. The following workaround is based on the Gradle docs and solved the problem for me:
buildscript {
ext {
springBootVersion = '2.0.4.RELEASE'
}
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
}
}
plugins {
id 'java'
// and other plugins
id 'io.spring.dependency-management' version '1.0.6.RELEASE'
}
// but the one with the variable version is applied the old way:
apply plugin: 'org.springframework.boot'
// We can use the variable in dependencies, too:
dependencies {
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web', version: springBootVersion
// ...
}
In addition to #zhouji comment, you can not only specify the versions in gradle.properties, but you can also define it programmatically like:
buildscript {
ext {
kotlinVersion = "${JavaVersion.current().isCompatibleWith(JavaVersion.VERSION_16) ? '1.5.32' : '1.4.32'}"
}
}
plugins {
id 'org.jetbrains.kotlin.jvm' version "${kotlinVersion}"
}
Gradle 7
Gradle 7 introduced version catalogs for declaring dependencies and sharing them between projects/modules.
Plugins can be declared in the catalog:
// settings.gradle
dependencyResolutionManagement {
versionCatalogs {
libs {
version('android-plugins', '7.3.0')
plugin('android-application', 'com.android.application').versionRef('android-plugins')
// A library declaration using the same version.
library('some-library', 'com.group', 'artifact').versionRef('android-plugins')
}
}
}
The declaration can be referred to in the plugins block:
// build.gradle
plugins {
alias(libs.plugins.android.application) apply false
}
The version can also be referred to in the plugins block:
// build.gradle
plugins {
id 'com.android.application' version libs.versions.android.plugins apply false
}
See the version catalogs documentation for more information.
Note: In the declaration, - is used as separator in the dependency name. On the consuming side . is used instead, as - is the sign for the minus operator of the groovy language and can't be used in property names.
So the separators are different on the declaring and consuming side - which threw me off at first.
Summary
Omit the version from the plugin block
Move/specify the version in the buildscript.dependencies block, instead
Use a variable in that block that is a const variable
In short, plugin blocks require string literals or properties whereas dependency blocks permit varables. The original question asks for doing this in a "single line" and with this approach, your plugin block is shorter and all your dependencies live in one place for all modules. Depending on your goals, that might be better than doing everything in "one line."
Full Example
For Android, I was able to do this by omitting the version from the plugin block and then specifying it as a const in the buildscript dependencies, instead. That block allows variables, whereas the plugin block only allows string literals. From there, I use an object in buildSrc because that provides the most flexibility, while keeping all dependency info in one file for all modules. So my setup looks like this:
├── project/
│ ├── build.gradle
| └── app/
| └── build.gradle
| └── buildSrc/
| └── build.gradle.kts
| └── src/main/java/com/example/package/Deps.kt
From there, to specify any plugin version as a variable in one place, the app/build.gradle file omits the version (using kotlin as an example but the same approach works for any plugin):
app/build.gradle
plugins {
id 'kotlin-android' // omit version property here but provide it in buildscript dependencies
...
}
...
dependencies {
...
// Dagger
implementation Deps.Dagger.ANDROID_SUPPORT
kapt Deps.Dagger.ANDROID_PROCESSOR
kapt Deps.Dagger.COMPILER
}
Then the buildscript dependency section (usually in the parent build.gradle file but can also be in the same file) provides the version USING A VARIABLE!
project/build.gradle
import com.example.package.Deps // object in the buildSrc folder with all version info
buildscript {
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:${Deps.kotlinVersion}"
}
}
Lastly, all the version information is managed in buildSrc, with an object in the Deps.kt file (there are many blog posts out there on using buildSrc with Android projects):
Deps.kt
object Deps {
// For use in the top-level buildscript dependency section which only works with a constant rather than `Deps.Kotlin.version`
const val kotlinVersion = "1.3.72"
object Kotlin : Version(kotlinVersion) {
val STDLIB = "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$version"
}
object Dagger : Version("2.25.2") {
val ANDROID_SUPPORT = "com.google.dagger:dagger-android-support:$version"
val ANDROID_PROCESSOR = "com.google.dagger:dagger-android-processor:$version"
val COMPILER = "com.google.dagger:dagger-compiler:$version"
}
}
open class Version(#JvmField val version: String)
Overall, I really like this setup. It's flexible, all dependency info is in one place, even across multiple modules and, best of all, it allows for code completion in the gradle files, when typing dependencies!
// gradle.properties
springBootVersion=2.6.7
// settings.gradle.kts
pluginManagement {
val springBootVersion: String by settings
plugins {
id("org.springframework.boot") version springBootVersion
}
}
// build.gradle.kts
plugins {
id("org.springframework.boot")
}
I hit this issue.
when I ran
gradle clean test build
instead of
./gradlew clean test build
and my "gradle" was at version 4.x
but my ./gradlew resolved to 6.9 or higher.
aka, "oops, I forgot my build.gradle files were coded to use later versions of gradle(w), but I snapfu-ed and executed the older 4.x version of regular "gradle"

Resources