I would like to create fat jar using gradle. I am using the shadowJar plugin(com.github.johnrengelman.shadow).
The problem that I am facing is that I cannot find a way to instruct the plugin to package a certain libraries with their transitive dependencies.
For example: I am using hive-jdbc which depends on org.apache.httpcomponents:httpclient:4.5.2, however the final jar packaged by shadowJar does not include httpclient
Another option is to drop the shadowJar section from the gradle.build file, but then the plugin will include everything that is part of the compile classpath and huge jar file will be created.
I am using this build file in order to package a jar for spark-submmit that includes many libraries as part of the runtime environment, so I do not need all the compile dependencies to be packaged (just the ones I am specifying explicitly and their transitive dependencies)
build.gradle:
import com.github.jengelman.gradle.plugins.shadow.transformers.AppendingTransformer
plugins {
id "com.github.johnrengelman.shadow" version "2.0.3"
}
apply from: "../Parent_2.2/build.gradle"
group 'com.ebates.bigdata'
version '1.0-SNAPSHOT'
dependencies{
compile("com.databricks:spark-avro_2.11:4.0.0"){force = true}
compile("org.apache.hadoop:hadoop-client:2.6.5"){force = true}
compile("org.apache.hadoop:hadoop-aws:2.6.5"){force = true}
compile("org.postgresql:postgresql:9.4.1207"){force = true}
compile("org.apache.hive:hive-jdbc:2.2.0"){force = true}
}
configurations.all {
resolutionStrategy.cacheChangingModulesFor 0, 'seconds'
}
shadowJar {
mergeServiceFiles()
transform(AppendingTransformer) {
resource = 'reference.conf'
shadowJar.classifier = ""
zip64 true
dependencies {
include(dependency("com.databricks:spark-avro_2.11:4.0.0"))
include(dependency("org.postgresql:postgresql:9.4.1207"))
include(dependency("org.apache.hive:hive-jdbc:2.2.0"))
}
}
}
Related
I made a decision to migrate from Dependency Management Plugin to Gradle built-in BOM import support. Since Gradle built-in BOM import support has better performance But
I run into the issue:
I cannot find alternatives for dependency and dependencySet in native Gradle:
dependencyManagement {
dependencies {
dependency("org.springframework:spring-core:4.0.3.RELEASE")
}
}
//or
dependencyManagement {
dependencies {
dependencySet(group:'org.slf4j', version: '1.7.7') {
entry 'slf4j-api'
entry 'slf4j-simple'
}
}
}
and then I could use dependency without version
dependencies {
compile 'org.springframework:spring-core'
}
How can I get the same behavior in naive Gradle? I mean: I'd like to define a version once as I did it when using Dependency Management Plugin
Solution below helps to avoid versions copy-paste. However it isn't the same with Dependency Management plugin.
For Gradle Kotlin Dsl:
You can create buildSrc with you own code, when you can place any constants.
Algorithm:
Create folder buildSrc/src/main/kotlin
Create file buildSrc/src/main/kotlin/Versions.kt with content:
object Versions {
const val junitVersion = "5.5.5" // just example
}
Create file buildSrc/build.gradle.kts with content:
plugins {
`kotlin-dsl`
}
Use the following syntax in your gradle.kts files:
dependencies {
testImplementation("org.junit.jupiter:junit-jupiter:${Versions.junitVersion}")
}
For Gradle Groovy:
Create file gradle.properties
Put versions there with syntax like okhttp_version=4.2.0
Use the following syntax in your gradle files:
dependencies {
compile group: 'com.squareup.okhttp3', name: 'okhttp', version: okhttp_version
}
You can do so on the gradle.properties file. I.e.:
# APPLICATION PROPERTIES
name=projectName
group=com.domain
version=1.0.0
description=A brief description
gradleScripts=https://raw.githubusercontent.com/hexagonkt/hexagon/1.2.0/gradle
# DEPENDENCIES VERSIONS
kotlinVersion=1.3.61
kotlinCoroutinesVersion=1.3.2
Or in settings.gradle if you don't want to create another file:
rootProject.name = "hexagon-contact-application"
gradle.rootProject {
allprojects {
version = "1.0.0"
group = "org.hexagonkt"
description = "Contact application backend api"
}
extensions.gradleScripts = "https://raw.githubusercontent.com/hexagonkt/hexagon/1.0.18/gradle"
extensions.kotlinVersion = "1.3.50"
extensions.kotlinCoroutinesVersion = "1.3.2"
extensions.hexagonVersion = "1.0.21"
extensions.logbackVersion = "1.2.3"
extensions.bcryptVersion="0.8.0"
extensions.javaJwtVersion="3.8.2"
}
And if you want to avoid adding the version variable to all related dependencies, you can create a method in the build file:
plugins {
id 'org.jetbrains.kotlin.jvm' version '1.3.50'
}
apply from: "$gradleScripts/kotlin.gradle"
apply from: "$gradleScripts/service.gradle"
apply from: "$gradleScripts/junit.gradle"
defaultTasks("test")
mainClassName = 'com.hexagonkt.contact.ApplicationKt'
applicationDefaultJvmArgs = ["-Xms64M", "-Xmx2G", "-XX:+UseNUMA", "-XX:+UseParallelGC"]
dependencies {
httpkt(it, "http_server_jetty")
httpkt(it, "store_mongodb")
httpkt(it, "hexagon_web")
implementation("at.favre.lib:bcrypt:$bcryptVersion")
implementation("com.auth0:java-jwt:$javaJwtVersion")
testImplementation("com.hexagonkt:port_http_client:$hexagonVersion")
}
private void httpkt(final def dependencies, final String artifact) {
dependencies.implementation("com.hexagonkt:$artifact:$hexagonVersion")
}
I'm fairly new to Groovy and I'm trying to wrap my head around Gradle. If I import the org.jvnet.hudson.plugins through Grapes it works perfectly and the dependency is resolved. But if I try to retrieve the dependency using Gradle the dependency is not resolved.
The package org.eclipse.hudson:hudson-core:3.2.1 works with both Gradle and Grape.
A dependency that is not resolved using Gradle
compile 'org.jvnet.hudson.plugins:checkstyle:3.42'
A dependency which is resolved using Grape
#Grab('org.jvnet.hudson.plugins:checkstyle:3.42')
A dependency which is resolved using Gradle
compile 'org.eclipse.hudson:hudson-core:3.2.1'
Error during Gradle build
line 3, column 1.
import hudson.plugins.checkstyle.CheckStyleResultAction;
^
The build.gradle
apply plugin: 'groovy'
repositories {
mavenCentral()
maven {
url "http://repo.jenkins-ci.org/releases/"
}
}
configurations {
ivy
}
sourceSets {
main {
groovy {
srcDirs = ['src/']
}
}
test {
groovy {
srcDirs = ['test/']
}
}
}
dependencies {
compile 'org.codehaus.groovy:groovy-all:2.4.11'
compile "org.apache.ivy:ivy:2.4.0"
ivy "org.apache.ivy:ivy:2.3.0"
// Works
compile 'org.eclipse.hudson:hudson-core:3.2.1'
// Does not work
compile 'org.jvnet.hudson.plugins:checkstyle:3.42'
}
tasks.withType(GroovyCompile) {
groovyClasspath += configurations.ivy
}
You're probably not actually downloading the jar you think you are. Looks like the default artifact that comes back from the org.jvnet.hudson.plugins:checkstyle:3.42 dependency is actually a file named checkstyle-3.42.hpi.
To get the jar which contains the classes instead, use:
compile group: 'org.jvnet.hudson.plugins', name: 'checkstyle', version:'3.42', ext: 'jar'
Then that class will be found on your classpath (and you'll be on to locating the next missing dependency).
I have a Kotlin Gradle project, and I would like to include Kotlin's runtime and stdlib in the jar file. I'm currently using this, but it's not including the runtime or stdlib when I build the project using the build.gradle configuration.
compileKotlin {
kotlinOptions {
includeRuntime = true
noStdlib = false
}
}
This is the Gradle code I'm using to include the runtime/stdlib in the jar, but it isn't working like I expect it to. Here's the full build.gradle file for some context:
https://github.com/BenWoodworth/CrossPlatformGreeter/blob/bd1da79f36e70e3d88ed871bc35502ecc3a852fb/build.gradle#L35-L43
Kotlin's Gradle documentation seems to indicate that setting kotlinOptions.includeRuntime to true should include the Kotlin runtime in the resulting .jar.
https://kotlinlang.org/docs/reference/using-gradle.html#attributes-specific-for-kotlin
Edit:
This might be related. When I run compileKotlin, I'm getting a couple of warnings related to the runtime:
:compileKotlin
w: Classpath entry points to a non-existent location: <no_path>\lib\kotlin-runtime.jar
BUILD SUCCESSFUL
Here's an alternative I came up with. It'll add the Kotlin runtime and stdlib to the jar using the jar task.
jar {
from {
String[] include = [
"kotlin-runtime-${version_kotlin}.jar",
"kotlin-stdlib-${version_kotlin}.jar"
]
configurations.compile
.findAll { include.contains(it.name) }
.collect { it.isDirectory() ? it : zipTree(it) }
}
}
Gradle Kotlin DSL:
tasks.withType<Jar> {
val include = setOf("kotlin-stdlib-1.4.0.jar")
configurations.runtimeClasspath.get()
.filter { it.name in include }
.map { zipTree(it) }
.also { from(it) }
}
Try this:
Build script
Unpack jar
Add kotlin runtime and rapack it
type gradle packJar to create jar with kotlin runtime in it
or
type gradle runJar to create and run the jar file
build Script
I'm trying to build a fully executable WAR using Spring Boot 1.3 as per https://docs.spring.io/spring-boot/docs/current/reference/html/deployment-install.html. If I build a single Gradle project, it all works fine, but I havea multi project build, where I have a "root" project and then several projects underneath it, and I cannot get it to build anything but a standard, "fat" WAR file, without the providedRuntime of Jetty and without the scripts to make it run.
Does anyone know how to do this?
In my root project, I have the following (abridged):
buildscript {
repositories {
mavenCentral()
}
ext {
springBootVersion = '1.3.0.RELEASE'
}
dependencies {
classpath "org.springframework.boot:spring-boot-gradle-plugin:$springBootVersion"
}
}
allprojects {
//Put instructions for all projects
repositories {
mavenCentral() // jcenter is missing spring-orm.4.1.6.RELEASE jar file so try mavenCentral first
jcenter {
url "http://jcenter.bintray.com/"
}
maven { url 'http://repo.opensourceagility.com/release' }
}
}
subprojects {
apply plugin: 'java'
apply plugin: 'spring-boot'
}
and then in the subproject which is a web project, and which I'm trying to build, I have:
apply plugin: 'war'
dependencies {
// Include related projects
compile project(':project-model')
compile project(':project-dynamoDB')
// Core Spring Boot - note version is set in main build.gradle file
compile 'org.springframework.boot:spring-boot-starter-web'
// Remove Tomcat (included in -web) and include Jetty instead
providedRuntime 'org.springframework.boot:spring-boot-starter-jetty'
// Other Spring modules
compile 'org.springframework.boot:spring-boot-starter-social-facebook'
compile 'org.springframework.boot:spring-boot-starter-social-linkedin'
compile 'org.springframework.social:spring-social-google:1.0.0.RELEASE'
compile 'org.springframework.boot:spring-boot-starter-thymeleaf'
compile 'org.springframework.boot:spring-boot-starter-security'
compile 'org.springframework.boot:spring-boot-starter-actuator'
compile 'org.springframework.boot:spring-boot-devtools'
compile 'org.springframework:spring-context'
compile 'org.springframework:spring-context-support'
}
configurations {
providedRuntime.exclude group: 'org.springframework.boot', module:'spring-boot-starter-tomcat'
all*.exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging' // exclude when using log4j
}
springBoot {
mainClass = 'rs.web.Weblication'
executable = true
}
bootRun {
addResources = true
}
processResources {
// exclude resources if they look like they're profile dependent but don't match the current env/profile
eachFile { d ->
if(d.name.endsWith('.xml') || d.name.endsWith('.yaml') || d.name.endsWith('.properties')) {
//def fname = d.name.replaceFirst(~/\.[^\.]+$/, '')
//if(fname.indexOf("-") > -1 && ! fname.endsWith("-" + environment)) {
// d.exclude()
//} else {
// replace #variables# listed below in properties/config files
filter(org.apache.tools.ant.filters.ReplaceTokens, tokens: [
activeProfiles: environment
])
//}
}
}
}
war {
baseName = 'project-web'
version = '1.0.0'
manifest {
attributes 'Implementation-Title': baseName,
'Implementation-Version': version
}
webXml = file('src/main/resources/web.xml')
// rename the war task which has profiles appended from warName-profile,profile2.war
// to warName-profile.profile2.war
classifier = environment.replaceAll(',','-')
}
but when I build it (./gradlew build, or ./gradlew subprojectname:build), all is well and a working WAR is created, but not an executable one.
With a single project, I have it working fine.
Ah ha, right well I build a test multi-project build and it worked OK, so it was clearly the configuration above.
I worked through a process of elimination and it turns out that the problematic area was the line
classifier = environment.replaceAll(',','-')
which is intended to rename files with environment variables as part of the name. This process seems to get in the way of the script addition; perhaps it could be applied afterwards if it's really necessary.
I'm trying to build a JAR from a specific package of classes so I want to exclude all the other packages from this JAR. This is what I have...
task receiverJar(type: Jar) {
baseName = "receivers"
from sourceSets.main.output
include 'com/foo/receivers/**'
exclude 'com/foo/cli/**'
exclude 'com/foo/tdl/**'
with jar
}
When I execute gradle receiverJar I still get all the other packages and classes in my JAR file.
task receiverJar(type: Jar) {
enabled = true
baseName this.name + '-receivers'
from sourceSets.main.output
manifest {
attributes 'Implementation-Version': project.version
}
include 'com/foo/tdl/**'
exclude 'com/foo/cdl/**'
}