How can I build a runnable jar with SBT?
Perhaps Spring-Boot isn't suitable for SBT built applications? Any suggestions on this?
I tried using sbt-assembly but it fails when I try to run it.
Note that sbt run works
mainClass in assembly := Some("com.xagongroup.xagon.app.XagonETL")
assemblyMergeStrategy in assembly := {
case PathList("META-INF", _ # _*) => MergeStrategy.discard
case _ => MergeStrategy.first
}
Stack Trace
org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [com.x.app.XETL]; nested exception is java.lang.Ille
galArgumentException: No auto configuration classes found in META-INF/spring.factories. If you are using a custom packaging, make sure that file is correct.
at org.springframework.context.annotation.ConfigurationClassParser.processDeferredImportSelectors(ConfigurationClassParser.java:556)
at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:185)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:308)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:228)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:270)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:93)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:687)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:525)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107)
at com.x.app.XETL$.main(XETL.scala:21)
at com.x.app.XETL.main(XETL.scala)
Caused by: java.lang.IllegalArgumentException: No auto configuration classes found in META-INF/spring.factories. If you are using a custom packaging, make sure that file is correct.
at org.springframework.util.Assert.notEmpty(Assert.java:277)
at org.springframework.boot.autoconfigure.AutoConfigurationImportSelector.getCandidateConfigurations(AutoConfigurationImportSelector.java:153)
at org.springframework.boot.autoconfigure.AutoConfigurationImportSelector.selectImports(AutoConfigurationImportSelector.java:95)
at org.springframework.context.annotation.ConfigurationClassParser.processDeferredImportSelectors(ConfigurationClassParser.java:547)
... 15 common frames omitted
I solved the issue by moving to sbt-native-packager
plugins.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.2.0")
build.sbt
scriptClasspath := Seq("*")
mainClass in Compile := Some("com.x.app.XETL")
enablePlugins(JavaAppPackaging)
Running:
packaging
sbt universal:stage
starting the app:
target\universal\stage\bin\x.bat
I've solved this without any plugins. It seems the only trick here is to not discard META-INF/spring.factories files in your build.sbt
assemblyMergeStrategy in assembly := {
case PathList("META-INF", "spring.factories") => MergeStrategy.filterDistinctLines
case PathList("META-INF", _*) => MergeStrategy.discard
case _ => MergeStrategy.first
}
In order to save someone else's time:
Brief
Spring Boot uses special jar format which has different inner structure, changes to manifest and can contain jars inside, which should be just stored in resulting archives as-is (it is called ZipEntry.STORED in contrast with ZipEntry.DEFLATED).
Solution
Neither of plugins I knew (sbt-assembly or sbt-native-packager) could do it from the box or with configuration, so I just wrote a custom task for this which can package my relatively simple project (no modules and subprojects).
File layout
...usual build folders...
project/
SpringBootJar.scala
build.sbt
build.sbt
Note the addition of spring-boot-loader in the dependencies and springBootJar task.
import SpringBootJar.Keys.springBootJar
import SpringBootJar.springBootJarTask
val springBootVersion = "2.7.6"
lazy val myProject = (project in file("."))
.settings(
name := "my-project",
scalaVersion := "3.2.1",
...
libraryDependencies ++= Seq(
"org.springframework.boot" % "spring-boot-starter-parent" % springBootVersion pomOnly(),
"org.springframework.boot" % "spring-boot-starter-jdbc" % springBootVersion,
.... other depenencies ....
"org.springframework.boot" % "spring-boot-loader" % springBootVersion % "compile"
),
springBootJar := springBootJarTask.value
)
SpringBootJar.scala
import sbt.{Keys, _}
import Keys._
import sbt.io.Using.fileOutputStream
import java.io.BufferedOutputStream
import java.nio.file.Files
import java.util.jar.{Attributes, JarEntry, JarFile, JarOutputStream, Manifest}
import java.util.zip.{CRC32, ZipEntry, ZipOutputStream}
import scala.collection.immutable.TreeSet
object SpringBootJar {
object Keys {
val springBootJar = taskKey[Unit]("Create spring boot fat jar")
}
val springBootJarTask = Def.task {
val log = streams.value.log
// check if spring-boot-loader is already a dependency
libraryDependencies.value.find(_.name == "spring-boot-loader") match {
case Some(_) => log.info("spring-boot-loader: present")
case None => sys.error(s"Consider added spring-boot-loader to libraryDependencies")
}
// name of resulting artifact
val destArtifact = target.value / s"${name.value}-${version.value}-sb.jar"
val classpath = (Runtime / fullClasspath).value
val externalClasspath = (Runtime / externalDependencyClasspath).value
val classDir = (Compile / classDirectory).value
val resourceDir = (Compile / resourceDirectory).value
// create directory structure
val rootDir = IO.createUniqueDirectory(target.value)
val springBootLoaderDir = rootDir / "org" / "springframework" / "boot" / "loader"
val appClassesDir = rootDir / "BOOT-INF" / "classes"
val libsDir = rootDir / "BOOT-INF" / "lib"
// get the location of the spring-boot-loader library on the file system
log.info("Seeking for spring-boot-loader jar")
val springBootLoaderFile = classpath.find(_.data.getName.startsWith("spring-boot-loader")) match {
case Some(af) => af.data
case None => sys.error("Couldn't find spring-boot-loader, interrupting")
}
// create dir structure
log.info("Creating directory structure")
IO.createDirectories(Seq(springBootLoaderDir, appClassesDir, libsDir))
// creating manifest
log.info("Creating MANIFEST")
val manifest = new Manifest()
val mainAttributes = manifest.getMainAttributes
mainAttributes.put(Attributes.Name.MANIFEST_VERSION, "1.0")
mainAttributes.put(Attributes.Name.SPECIFICATION_TITLE, name.value)
mainAttributes.put(Attributes.Name.SPECIFICATION_VERSION, version.value)
mainAttributes.put(Attributes.Name.SPECIFICATION_VENDOR, organization.value)
mainAttributes.put(Attributes.Name.IMPLEMENTATION_TITLE, name.value)
mainAttributes.put(Attributes.Name.IMPLEMENTATION_VERSION, version.value)
mainAttributes.put(Attributes.Name.IMPLEMENTATION_VENDOR, organization.value)
mainAttributes.put(new Attributes.Name("Start-Class"), (Compile / mainClass).value.getOrElse(""))
mainAttributes.put(Attributes.Name.MAIN_CLASS, "org.springframework.boot.loader.JarLauncher")
// copy all from org/springframework/boot/loader/ to the same path
locally {
log.info("Copying org/springframework/boot/loader/ classes")
val jar = new JarFile(springBootLoaderFile)
val entries = jar.entries()
while (entries.hasMoreElements()) {
val entry = entries.nextElement()
if (!entry.isDirectory && entry.getName.startsWith("org/springframework/boot/loader/")) {
val targetPath = entry.getName.stripPrefix("org/springframework/boot/loader/")
val targetFile = new File(springBootLoaderDir, targetPath)
targetFile.getParentFile.mkdirs()
Files.copy(jar.getInputStream(entry), targetFile.toPath)
}
}
jar.close()
}
// copying all own app files
log.info("Copying app classes")
IO.copyDirectory(classDir, appClassesDir)
// copying app resources
log.info("Copying app resources")
IO.copyDirectory(resourceDir, appClassesDir)
// copying all external libs
log.info("Copying libs")
IO.copy(externalClasspath.filterNot(_.data.base.startsWith("spring-boot-loader")).map {
f => f.data -> libsDir / f.data.getName
})
// creating jar
log.info("Creating jar")
jar(
(rootDir ** "*").get.map { file => (file, file.relativeTo(rootDir).get.getPath) },
destArtifact,
manifest,
Some(System.currentTimeMillis())
)
log.info("Cleaning up")
IO.delete(rootDir)
log.info(s"Created: ${destArtifact.getAbsolutePath}")
}
private def jar(sources: Traversable[(File, String)], outputJar: File, manifest: Manifest, time: Option[Long]): Unit = {
val localTime = time.map(t => t - java.util.TimeZone.getDefault.getOffset(t))
if (outputJar.isDirectory)
sys.error("Specified output file " + outputJar + " is a directory")
else {
val outputDir = outputJar.getParentFile match {
case null => new File(".")
case parentFile => parentFile
}
IO.createDirectory(outputDir)
val emptyCRC = new CRC32().getValue // The CRC32 for an empty value, needed to store directories in zip files
withJarOutput(outputJar, manifest, localTime) { output =>
writeZip(sources.toSeq, output, localTime) { (file, name) =>
val entry = new JarEntry(name)
if (file == null || file.isDirectory) {
entry.setSize(0)
entry.setMethod(ZipEntry.STORED)
entry.setCrc(emptyCRC)
} else if (file.ext == "jar") {
val jarBytes = Files.readAllBytes(file.toPath)
entry.setMethod(ZipEntry.STORED)
entry.setSize(jarBytes.length)
entry.setCompressedSize(jarBytes.length)
entry.setCrc({
val crc = new CRC32()
crc.update(jarBytes)
crc.getValue
})
}
entry
}
}
}
}
private def withJarOutput(file: File, manifest: Manifest, time: Option[Long])(f: ZipOutputStream => Unit) = {
fileOutputStream(false)(file) { fileOut =>
val zipOut = {
val os = new JarOutputStream(fileOut)
val e = new ZipEntry(JarFile.MANIFEST_NAME)
e.setTime(time.getOrElse(System.currentTimeMillis))
os.putNextEntry(e)
manifest.write(new BufferedOutputStream(os))
os.closeEntry()
os
}
try
f(zipOut)
finally
zipOut.close()
}
}
private def writeZip(sources: Seq[(File, String)], output: ZipOutputStream, time: Option[Long])(
createEntry: (File, String) => ZipEntry
): Unit = {
val files = sources.flatMap {
case (file, name) => if (file.isFile) (file, normalizeToSlash(name)) :: Nil else Nil
}.sortBy {
case (_, name) => name
}
val now = System.currentTimeMillis
def addDirectoryEntry(file: File, name: String): Unit = {
output.putNextEntry {
val e = createEntry(file, name)
e.setTime(time.getOrElse(now))
e
}
output.closeEntry()
}
def addFileEntry(file: File, name: String): Unit = {
output.putNextEntry {
val e = createEntry(file, name)
e.setTime(time.getOrElse(IO.getModifiedTimeOrZero(file)))
e
}
IO.transfer(file, output)
output.closeEntry()
}
// Calculate directories and add them to the generated Zip
allDirectoryPaths(files).foreach(addDirectoryEntry(null, _))
// Add all files to the generated Zip
files foreach { case (file, name) => addFileEntry(file, name) }
}
private def normalizeToSlash(name: String): String = {
val sep = java.io.File.separatorChar
if (sep == '/') name else name.replace(sep, '/')
}
private def relativeComponents(path: String): List[String] =
path.split("/").toList.dropRight(1)
private def directories(path: List[String]): List[String] =
path.foldLeft(List(""))((e, l) => (e.head + l + "/") :: e)
private def directoryPaths(path: String): List[String] =
directories(relativeComponents(path)).filter(_.length > 1)
private def allDirectoryPaths(files: Iterable[(File, String)]) =
TreeSet[String]() ++ (files flatMap { case (_, name) => directoryPaths(name) })
}
Usage
sbt springBootJar or springBootJar in sbt console.
Conclusion
This was tested with scala 3.2.1 in project and sbt 1.7.1. I don't have time to make it into a plugin, and I don't mind if anyone reuse my findings to make more general and ready to use solution.
Related
Below is my code in build.gradle
`tasks.register("javaExe",JavaExec::class.java) {
main = "com.company.main"
val subproject = project(":utils").extensions.getByName("sourceSets") as SourceSetContainer
classpath = subproject["main"].runtimeClasspath
val baseDirectory = project.projectDir.absolutePath
val URL = environment["URL"]
val DBURL = environment["DBURL"]
val JREURL = environment["JREURL"]
args(baseDirectory, URL, DBURL, JREURL)
}
tasks.register("copydep",Copy::class) {
shouldRunAfter("javaExe")
//some copy task
}
tasks.register("runBuild") {
dependsOn("javaExe")
dependsOn("copydep")
doLast{
//some code
}
}`
when I run the runBuild task it should execute in order like javaExe,copydep and runBuild but the copydep task is not working always.
I have verified the source and destination path location both are preset.
when we remove the javaExe depandencies copy task working perfectly.
I have to download a jar file (https://repo1.maven.org/maven2/org/mock-server/mockserver-netty/5.13.2/mockserver-netty-5.13.2-jar-with-dependencies.jar).
Maven central does not contain this with-dependencies and I cannot just specify it in a dependency list (implementation("group:artifact:version-with-dependencies)).
I can download it (by creating a custom task) to the build directory, but I also would like to cache it like other dependencies (to avoid downloading it each time after cleaning the build folder).
So, it would be nice either to specify this long url and Gradle would download/cache it automatically, or to use some API to cache it manually.
Based on this comment solution in Kotlin:
plugins {
id("de.undercouch.download") version "5.1.0"
}
val url = "https://repo1.maven.org/maven2/org/mock-server/mockserver-netty/5.13.2/mockserver-netty-5.13.2-jar-with-dependencies.jar"
task("downloadMockServerToCache", Download::class) {
val destPath = urlToCachePath(url)
src(url)
dest(destPath)
overwrite(false)
}
task("copyMockServerFromCache", Copy::class) {
dependsOn("downloadMockServerToCache")
val destPath = urlToCachePath(url)
from(destPath) {
include(urlToCachedFile(url))
}
into("build/bin")
rename { "$mockServerArtifact.jar" }
}
fun urlToCachePath(url: String): String {
val urlWithoutProtocol = url.substringAfter("//")
val reversedDomain = reversedDomain(urlWithoutProtocol)
val fileName = urlToCachedFile(url)
val gradleHome = System.getProperty("user.home")
return "$gradleHome/.gradle/caches/modules-2/files-2.1/de.undercouch/cache/$reversedDomain/$fileName"
}
fun urlToCachedFile(url: String): String {
return url.substringAfterLast("/").substringBefore("?")
}
fun reversedDomain(urlWithoutProtocol: String) =
urlWithoutProtocol.substringBefore("/").split(".").reversed().joinToString("/")
In my code there is const val VERSION = $version.
I want to replace $version with real version string which is in my build.gradle.kts.
How can I do this?
Working example here.
One way is to use a template file (stored outside of src tree). Let's call it TemplateVersion.kt:
class Version() {
val version = "__VERSION";
}
and then in build.gradle.kts, as initial part of the compileKotlin task, we generate Version.kt from TemplateVersion.kt:
val sourceFile = File(rootDir.absolutePath + "/resources/TemplateVersion.kt")
val propFile = File(rootDir.absolutePath + "/gradle.properties")
val destFile = File(rootDir.absolutePath + "/src/main/kotlin/${targetPackage}/Version.kt")
tasks.register("generateVersion") {
inputs.file(sourceFile)
inputs.file(propFile)
outputs.file(destFile)
doFirst {
generateVersion()
}
}
tasks.named("compileKotlin") {
dependsOn("generateVersion")
}
fun generateVersion() {
val version: String by project
val rootDir: File by project
val inputStream: InputStream = sourceFile.inputStream()
destFile.printWriter().use { out ->
inputStream.bufferedReader().forEachLine { inputLine ->
val newLine = inputLine.replace("__VERSION", version)
out.println(newLine)
}
}
inputStream.close()
}
where gradle.properties is:
version=5.1.50
It is trivially easy to add more fields to Version.kt, such as a build-timestamp.
(Edit: This has been updated with a proper generateVersion task that will detect changes to gradle.properties. The compileKotlin task will invoke this task).
I am trying to convert the gradle files for a project with both iOS and Android targets from groovy
I have these two lines in the groovy file
val srcFile = kotlin.targets."$target".compilations.main.getBinary("FRAMEWORK", buildType)}
val targetDir = getProperty("configuration.build.dir")
The first challenge is the "$targets" as an attribute. Target is property which is a string, so this is like using a string as an attribute name. But I would expect targets is a map, and so is compilations?
val srcFile = kotlin.targets[target].compilations[main].getBinary("FRAMEWORK", buildType)}
val targetDir = getProperty("configuration.build.dir")
seems to be valid kotlin, so I am assuming the groovy is like js and objects and maps can both be accessed by [] and . notations. Assuming this is correct, the problem becomes that both getBinary and getProperty are unkown.
Any help with this appreciated.
For reference, these lines are part of a task:
task("copyFramework") {
val buildType:String = project.findProperty("kotlin.build.type") as String??: "DEBUG"
val target:String = project.findProperty("kotlin.target")as String? ?: "ios"
dependsOn("link${buildType.toLowerCase().capitalize()}Framework${target.capitalize()}")
doLast {
val srcFile = kotlin.targets[target].compilations["main"].getBinary("FRAMEWORK", buildType)
val targetDir = getProperty("configuration.build.dir")
copy {
from(srcFile.parent)
into(targetDir)
include("app.framework/**")
include("app.framework.dSYM")
}
}
converted (so far) from groovy:
task copyFramework {
def buildType = project.findProperty("kotlin.build.type") ?: "DEBUG"
def target = project.findProperty("kotlin.target") ?: "ios"
dependsOn "link${buildType.toLowerCase().capitalize()}Framework${target.capitalize()}"
doLast {
def srcFile = kotlin.targets."$target".compilations.main.getBinary("FRAMEWORK", buildType)
def targetDir = getProperty("configuration.build.dir")
copy {
from srcFile.parent
into targetDir
include 'app.framework/**'
include 'app.framework.dSYM'
}
}
}
Looks like it requires type casting:
val srcFile = (kotlin.targets[target] as KotlinNativeTarget).compilations["main"].getBinary("FRAMEWORK", buildType)
How could I download a file within a gradle task with kotlin-dsl?
I want to download a file from an external source (URL) and store + rename it inside my projectfolder.
I tried to search it but I only found groovy solutions.
Thanks in advance!
I found a solution using https://github.com/michel-kraemer/gradle-download-task:
import de.undercouch.gradle.tasks.download.Download
...
plugins {
....
id("de.undercouch.download").version("3.4.3")
}
...
task<DefaultTask>("my-download-task") {
...
val url = " ... "
val dest = File("...")
task<Download>("download-task") {
src(url)
dest(dest)
}
dependsOn("download-task")
}
Simple with no external dependency.
buildscript {
repositories {
jcenter()
google()
}
dependencies {
classpath("com.android.tools.build:gradle:4.0.1")
}
}
tasks.register("downloadPdf"){
val path = "myfile.pdf"
val sourceUrl = "https://file-examples-com.github.io/uploads/2017/10/file-sample_150kB.pdf"
download(sourceUrl,path)
}
fun download(url : String, path : String){
val destFile = File(path)
ant.invokeMethod("get", mapOf("src" to url, "dest" to destFile))
}