Gradle 4.8+ breaks ivy publish with custom configurations - gradle

I've got a gradle file which is working in some ancient version of gradle but I want to upgrade to gradle 5.0. Unfortunately its using ivy rather than maven to publish its jars. I've cut it down to a simple test case.
I'm not sure if I'm missing something or its a bug or what. I've attached the gradle below. I'm running it
./gradlew wrapper && ./gradlew publish --info && cat build/publications/ivy/ivy.xml
It works as expected with 4.7. It publishes the main jar and the source jar and adds the dependencies.
If I switch to 4.8 it breaks, it only publishes the source jar, main jar and dependencies are missing.
If I switch to 4.8 and comment out the configurations bit it publishes the main jar and dependencies again.
Perhaps there's a new way of doing things but if so I've failed to find where its documented. Here's the source build.gradle.
plugins {
id 'java'
id 'ivy-publish'
}
sourceSets {
testSupport {
java {
compileClasspath += main.output
runtimeClasspath += main.output
}
}
test {
java {
compileClasspath += testSupport.output
runtimeClasspath += testSupport.output
}
}
}
dependencies {
compile group: 'com.ibm.icu', name: 'icu4j', version: '58.2'
compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.3'
compile group: 'io.swagger', name: 'swagger-parser', version: '1.0.32'
}
task sourceJar(type: Jar) {
from sourceSets.main.allJava
}
task testSupportJar(type: Jar) {
from sourceSets.testSupport.output
appendix "test-support"
}
task testSupportSourceJar(type: Jar) {
from sourceSets.testSupport.java.srcDirs
appendix "test-support-sources"
}
artifacts {
archives sourceJar
archives testSupportJar
archives testSupportSourceJar
}
publishing {
repositories {
ivy {
name = 'myRepo'
url = "file://${buildDir}/repo"
layout "pattern", {
artifact "[organisation]/[module]/[revision]/jars/[artifact].[ext]"
ivy "[organisation]/[module]/[revision]/ivys/ivy-[revision].xml"
}
}
}
publications {
ivy(IvyPublication) {
organisation = 'com.example.com'
// If you comment out the configurations below it will generate sensible ivy.xml
configurations {
"compile" {}
"runtime" {}
}
from components.java
artifact(sourceJar) {
type "source"
extension "src.jar"
conf "runtime"
}
}
}
}
wrapper {
// 4.7 works but 4.8+ doesn't.
gradleVersion = '4.7'
}

Oh man I just figured it out. Its the relative ordering of from components.java and the configurations element bits. If configurations is first it seems to take precedence over from components.java and the latter is seemingly ignored. If you put from components.java before configurations it works and you don't have to manually declare the configs it generates by default any more.
FFS gradle.

Related

Why can't Gradle resolve org.connectbot.jbcrypt:jbcrypt:1.0.0 from the Maven Central Repository?

I'm using Gradle 6.9 and here is my build.gradle file:
plugins {
id "groovy"
id "java"
}
group "com.matthiasdenu"
version "1.0-SNAPSHOT"
repositories {
mavenCentral()
maven {
url 'https://repo.jenkins-ci.org/releases/'
}
}
ext {
jobDslVersion = "1.77"
jenkinsVersion = "2.252"
}
sourceSets {
jobs {
groovy {
srcDirs "jobs"
compileClasspath += main.compileClasspath
}
compileClasspath += sourceSets.main.output
runtimeClasspath += sourceSets.main.output
}
}
dependencies {
compile("org.jenkins-ci.main:jenkins-war:${jenkinsVersion}"){
// https://github.com/sheehan/job-dsl-gradle-example/issues/87
exclude group: "org.jenkins-ci.ui", module: "bootstrap"
}
}
test {
useJUnitPlatform()
}
This is the error message I'm getting:
Execution failed for task ':compileTestGroovy'.
> Could not resolve all files for configuration ':testCompileClasspath'.
> Could not find org.connectbot.jbcrypt:jbcrypt:1.0.0.
Searched in the following locations:
- https://repo.maven.apache.org/maven2/org/connectbot/jbcrypt/jbcrypt/1.0.0/jbcrypt-1.0.0.pom
- https://repo.jenkins-ci.org/releases/org/connectbot/jbcrypt/jbcrypt/1.0.0/jbcrypt-1.0.0.pom
Required by:
project : > org.jenkins-ci.main:jenkins-war:2.252 > org.jenkins-ci.main:jenkins-core:2.252
Possible solution:
- Declare repository providing the artifact, see the documentation at https://docs.gradle.org/current/userguide/declaring_repositories.html
What's odd is that the 1.0.0 artifact doesn't show up at https://repo.maven.apache.org/maven2/org/connectbot/jbcrypt/. I also noticed that the urls don't quite match either. Like if I try to get v1.0.1 it doesn't resolve either because it expects an extra "jbcrypt" for the group name.
I have this problem even when using the latest jenkins-war release (2.304).
What's going on?
You have to add the Jenkins public repository to your configuration. It contains all libraries available in the releases repository and all required dependencies.
The file exists: https://repo.jenkins-ci.org/public/org/connectbot/jbcrypt/jbcrypt/1.0.0/jbcrypt-1.0.0.jar
repositories {
mavenCentral()
maven {
url 'https://repo.jenkins-ci.org/public/'
}
}

Adding external source files to a kotlin project

I have Kotlin sources located at, say, repo/project_a/src/. I created a Kotlin Gradle project in IntelliJ IDEA, located at repo/project_b/.... And I can't for the life of me figure out how to add the sources. If I add them through project structure menu it works fine, but as soon as it wants to re-read the gradle file id deletes the structure (It warns as much in the UI).
This is my gradle file:
plugins {
id 'org.jetbrains.kotlin.jvm' version '1.2.70'
}
group 'cli'
version '1.0'
repositories {
mavenCentral()
}
dependencies {
compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8"
}
compileKotlin {
kotlinOptions.jvmTarget = "1.8"
}
I've tried adding all variations of
sourceSets {
main {
kotlin {
srcDirs += "repo/project_a/"
}
}
}
But it does absolutely nothing.
Any ideas?
The path you are giving to Gradle will compile to the current project path plus "repo/project_a/". Try with:
sourceSets {
main {
kotlin {
srcDirs += "../project_a/"
}
}
}

How to build Google protocol buffers and Kotlin using Gradle?

I'm trying to build a project that uses both Google protocol buffers and Kotlin using Gradle. I want the proto files to compile into Java source, which is then called from my Kotlin code.
My source files are arranged like this:
src/main/proto/*.proto
src/main/kotlin/*.kt
src/test/kotlin/*.kt
Here's my build.gradle file:
version '1.0-SNAPSHOT'
apply plugin: 'kotlin'
apply plugin: 'java'
apply plugin: 'com.google.protobuf'
repositories {
mavenCentral()
maven { url "http://dl.bintray.com/kotlin/kotlin-eap-1.1" }
}
buildscript {
ext.kotlin_version = '1.1-M02'
repositories {
mavenCentral()
maven { url "http://dl.bintray.com/kotlin/kotlin-eap-1.1" }
}
dependencies {
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.0'
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
}
}
protobuf {
protoc {
artifact = 'com.google.protobuf:protoc:3.0.0'
}
}
dependencies {
compile 'com.google.protobuf:protobuf-java:3.0.0'
compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
testCompile 'junit:junit:4.12'
}
When I run ./gradlew assemble I get a bunch of "Unresolved reference" errors during :compileKotlin. Afterwards I can see that there are no Java source files generated, so it appears that the proto compiler is not being invoked at all.
If I remove the apply plugin: 'kotlin' line, then ./gradlew assemble successfully generates the Java source, but of course my Kotlin source is never compiled.
How do I fix my build.gradle so that I can call my protobuf code from Kotlin?
To get protobuf-gradle-plugin and kotlin-gradle-plugin to cooperate, you need to ensure that the Java code is (re)generated before invoking the Kotlin compiler.
For Gradle's default source sets, main and test, you can do that like this:
compileKotlin.dependsOn ':generateProto'
compileTestKotlin.dependsOn ':generateTestProto'
If you are using other source sets, you'll need to make adjustments.
Older versions of protobuf-gradle-plugin also required updating sourceSets, but newer versions do not seem to require this.
// Don't do this with protobuf-gradle-plugin 0.9.0 or higher
sourceSets.main.java.srcDirs += "${protobuf.generatedFilesBaseDir}/main/java"
sourceSets.test.java.srcDirs += "${protobuf.generatedFilesBaseDir}/test/java"
For Kotlin and Android:
android {
sourceSets {
debug.java.srcDirs += 'build/generated/source/proto/debug/java'
release.java.srcDirs += 'build/generated/source/proto/release/java'
}
}
An additional source directory has to be added for every build type. In this sample there are two build types: debug and release.
If you're using grpc, another line has to be added per build type:
android {
sourceSets {
debug.java.srcDirs += 'build/generated/source/proto/debug/java'
debug.java.srcDirs += 'build/generated/source/proto/debug/grpc'
release.java.srcDirs += 'build/generated/source/proto/release/java'
release.java.srcDirs += 'build/generated/source/proto/release/grpc'
}
}
At least with Kotlin 1.0.6, protobuf-gradle-plugin 0.8.0, protobuf 3.2.x and grpc 1.x it's not required to fiddle with the task order.
if you are working with multiple build types and flavors in android and with protobuf-lite use below with kotlin.
for example I have debug and release builds with demo and prod flavors it will create demoDebug, demoRelease and prodDebug and prodRelease variants.
then use
`
android{
sourceSets {
debug.java.srcDirs += 'build/generated/source/proto/demoDebug/javalite'
debug.java.srcDirs += 'build/generated/source/proto/prodDebug/javalite'
release.java.srcDirs += 'build/generated/source/proto/demoRelease/javalite'
release.java.srcDirs += 'build/generated/source/proto/prodRelease/javalite'
}
}
`
tie the different compileKotlin with generateProto
tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).all {
if (getName() == 'compileDemoDebugKotlin')
dependsOn(':app:generateDemoDebugProto')
if (getName() == 'compileDemoReleaseKotlin')
dependsOn(':app:generateDemoReleaseProto')
if (getName() == 'compileProdDebugKotlin')
dependsOn(':app:generateProdDebugProto')
if (getName() == 'compileProdReleaseKotlin')
dependsOn(':app:generateProdReleaseProto')
}
For the gradle setup :
plugins {
id 'com.android.application'
id 'kotlin-android'
id 'com.google.protobuf' version "0.8.17"
}
Then at the bottom of the build.gradle
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.10.0"
}
// Generates the java Protobuf-lite code for the Protobufs in this project. See
// https://github.com/google/protobuf-gradle-plugin#customizing-protobuf-compilation
// for more information.
generateProtoTasks {
all().each { task ->
task.builtins {
java {
option 'lite'
}
}
}
}
}

Building a fully executable Spring Boot 1.3 war from a Gradle multi project build

I'm trying to build a fully executable WAR using Spring Boot 1.3 as per https://docs.spring.io/spring-boot/docs/current/reference/html/deployment-install.html. If I build a single Gradle project, it all works fine, but I havea multi project build, where I have a "root" project and then several projects underneath it, and I cannot get it to build anything but a standard, "fat" WAR file, without the providedRuntime of Jetty and without the scripts to make it run.
Does anyone know how to do this?
In my root project, I have the following (abridged):
buildscript {
repositories {
mavenCentral()
}
ext {
springBootVersion = '1.3.0.RELEASE'
}
dependencies {
classpath "org.springframework.boot:spring-boot-gradle-plugin:$springBootVersion"
}
}
allprojects {
//Put instructions for all projects
repositories {
mavenCentral() // jcenter is missing spring-orm.4.1.6.RELEASE jar file so try mavenCentral first
jcenter {
url "http://jcenter.bintray.com/"
}
maven { url 'http://repo.opensourceagility.com/release' }
}
}
subprojects {
apply plugin: 'java'
apply plugin: 'spring-boot'
}
and then in the subproject which is a web project, and which I'm trying to build, I have:
apply plugin: 'war'
dependencies {
// Include related projects
compile project(':project-model')
compile project(':project-dynamoDB')
// Core Spring Boot - note version is set in main build.gradle file
compile 'org.springframework.boot:spring-boot-starter-web'
// Remove Tomcat (included in -web) and include Jetty instead
providedRuntime 'org.springframework.boot:spring-boot-starter-jetty'
// Other Spring modules
compile 'org.springframework.boot:spring-boot-starter-social-facebook'
compile 'org.springframework.boot:spring-boot-starter-social-linkedin'
compile 'org.springframework.social:spring-social-google:1.0.0.RELEASE'
compile 'org.springframework.boot:spring-boot-starter-thymeleaf'
compile 'org.springframework.boot:spring-boot-starter-security'
compile 'org.springframework.boot:spring-boot-starter-actuator'
compile 'org.springframework.boot:spring-boot-devtools'
compile 'org.springframework:spring-context'
compile 'org.springframework:spring-context-support'
}
configurations {
providedRuntime.exclude group: 'org.springframework.boot', module:'spring-boot-starter-tomcat'
all*.exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging' // exclude when using log4j
}
springBoot {
mainClass = 'rs.web.Weblication'
executable = true
}
bootRun {
addResources = true
}
processResources {
// exclude resources if they look like they're profile dependent but don't match the current env/profile
eachFile { d ->
if(d.name.endsWith('.xml') || d.name.endsWith('.yaml') || d.name.endsWith('.properties')) {
//def fname = d.name.replaceFirst(~/\.[^\.]+$/, '')
//if(fname.indexOf("-") > -1 && ! fname.endsWith("-" + environment)) {
// d.exclude()
//} else {
// replace #variables# listed below in properties/config files
filter(org.apache.tools.ant.filters.ReplaceTokens, tokens: [
activeProfiles: environment
])
//}
}
}
}
war {
baseName = 'project-web'
version = '1.0.0'
manifest {
attributes 'Implementation-Title': baseName,
'Implementation-Version': version
}
webXml = file('src/main/resources/web.xml')
// rename the war task which has profiles appended from warName-profile,profile2.war
// to warName-profile.profile2.war
classifier = environment.replaceAll(',','-')
}
but when I build it (./gradlew build, or ./gradlew subprojectname:build), all is well and a working WAR is created, but not an executable one.
With a single project, I have it working fine.
Ah ha, right well I build a test multi-project build and it worked OK, so it was clearly the configuration above.
I worked through a process of elimination and it turns out that the problematic area was the line
classifier = environment.replaceAll(',','-')
which is intended to rename files with environment variables as part of the name. This process seems to get in the way of the script addition; perhaps it could be applied afterwards if it's really necessary.

How to exclude dependencies in the POM file generated by the Gradle

I'm using the "maven" plugin to upload the artifacts created by Gradle build to Maven central repository. I'm using a task similar to the following one:
uploadArchives {
repositories {
mavenDeployer {
beforeDeployment { MavenDeployment deployment -> signing.signPom(deployment) }
pom.project {
name 'Example Application'
packaging 'jar'
url 'http://www.example.com/example-application'
scm {
connection 'scm:svn:http://foo.googlecode.com/svn/trunk/'
url 'http://foo.googlecode.com/svn/trunk/'
}
licenses {
license {
name 'The Apache License, Version 2.0'
url 'http://www.apache.org/licenses/LICENSE-2.0.txt'
}
}
}
}
}
}
However the POM file created by this task does not report correctly the dependencies that have been excluded in my Gradle build file. For example:
dependencies {
compile('org.eclipse.jgit:org.eclipse.jgit.java7:3.5.2.201411120430-r') { exclude module: 'commons-logging' }
compile('com.upplication:s3fs:0.2.8') { exclude module: 'commons-logging' }
}
How to have excluded dependencies managed correctly in the resulting POM file?
You can simply override the dependencies of the pom by filtering out the unwanted dependencies, e.g. to exclude junit you can add the following lines to the mavenDeployer configuration:
pom.whenConfigured {
p -> p.dependencies = p.dependencies.findAll {
dep -> dep.artifactId != "junit"
}
}
The problem was that in the exclude definition was not specified the group but only the module.
Adding the both of them the exclusions are added correctly in the POM file. For example:
compile('org.eclipse.jgit:org.eclipse.jgit.java7:3.5.2.201411120430-r') {
exclude group: 'commons-logging', module: 'commons-logging'
}
compile('com.upplication:s3fs:0.2.8') {
exclude group: 'commons-logging', module: 'commons-logging'
}
Using 'exclude' on a Gradle dependency is normally the correct answer, but I still needed to remove some of my "runtimeOnly" dependencies from the POM that led me to this StackOverflow page. My testing using Gradle 4.7 seems to show that using "compileOnly" leaves the dependency out of the pom entirely, but "runtimeOnly" adds a "runtime" dependency in the pom, which in my case, is not what I wanted. I couldn't figure out a "standard" Gradle way of leaving runtime dependencies out of the POM.
The pom.whenConfigured method shown in another answer works for legacy "maven" plugin publishing, but doesn't work for the newer "maven-publish" plugin. My experimentation led to this for "maven-publish":
publishing {
publications {
mavenJava(MavenPublication) {
from components.java
pom.withXml {
asNode().dependencies.dependency.each { dep ->
if(dep.artifactId.last().value().last() in ["log4j", "slf4j-log4j12"]) {
assert dep.parent().remove(dep)
}
}
}
}
}
}

Resources