How could I download a file within a gradle task with kotlin-dsl?
I want to download a file from an external source (URL) and store + rename it inside my projectfolder.
I tried to search it but I only found groovy solutions.
Thanks in advance!
I found a solution using https://github.com/michel-kraemer/gradle-download-task:
import de.undercouch.gradle.tasks.download.Download
...
plugins {
....
id("de.undercouch.download").version("3.4.3")
}
...
task<DefaultTask>("my-download-task") {
...
val url = " ... "
val dest = File("...")
task<Download>("download-task") {
src(url)
dest(dest)
}
dependsOn("download-task")
}
Simple with no external dependency.
buildscript {
repositories {
jcenter()
google()
}
dependencies {
classpath("com.android.tools.build:gradle:4.0.1")
}
}
tasks.register("downloadPdf"){
val path = "myfile.pdf"
val sourceUrl = "https://file-examples-com.github.io/uploads/2017/10/file-sample_150kB.pdf"
download(sourceUrl,path)
}
fun download(url : String, path : String){
val destFile = File(path)
ant.invokeMethod("get", mapOf("src" to url, "dest" to destFile))
}
Related
I have to download a jar file (https://repo1.maven.org/maven2/org/mock-server/mockserver-netty/5.13.2/mockserver-netty-5.13.2-jar-with-dependencies.jar).
Maven central does not contain this with-dependencies and I cannot just specify it in a dependency list (implementation("group:artifact:version-with-dependencies)).
I can download it (by creating a custom task) to the build directory, but I also would like to cache it like other dependencies (to avoid downloading it each time after cleaning the build folder).
So, it would be nice either to specify this long url and Gradle would download/cache it automatically, or to use some API to cache it manually.
Based on this comment solution in Kotlin:
plugins {
id("de.undercouch.download") version "5.1.0"
}
val url = "https://repo1.maven.org/maven2/org/mock-server/mockserver-netty/5.13.2/mockserver-netty-5.13.2-jar-with-dependencies.jar"
task("downloadMockServerToCache", Download::class) {
val destPath = urlToCachePath(url)
src(url)
dest(destPath)
overwrite(false)
}
task("copyMockServerFromCache", Copy::class) {
dependsOn("downloadMockServerToCache")
val destPath = urlToCachePath(url)
from(destPath) {
include(urlToCachedFile(url))
}
into("build/bin")
rename { "$mockServerArtifact.jar" }
}
fun urlToCachePath(url: String): String {
val urlWithoutProtocol = url.substringAfter("//")
val reversedDomain = reversedDomain(urlWithoutProtocol)
val fileName = urlToCachedFile(url)
val gradleHome = System.getProperty("user.home")
return "$gradleHome/.gradle/caches/modules-2/files-2.1/de.undercouch/cache/$reversedDomain/$fileName"
}
fun urlToCachedFile(url: String): String {
return url.substringAfterLast("/").substringBefore("?")
}
fun reversedDomain(urlWithoutProtocol: String) =
urlWithoutProtocol.substringBefore("/").split(".").reversed().joinToString("/")
I'm using gradle 7-4-1 and I'm trying to use a catalog to share dependencies between subprojects as documentation
https://docs.gradle.org/current/userguide/platforms.html#sec:sharing-catalogs
This is my settings.gradle.kts
dependencyResolutionManagement {
versionCatalogs {
create("libs") {
version("log4j", "2.17.1")
library("log4j-api", "org.apache.logging.log4j", "log4j-api").versionRef("log4j")
library("log4j-core", "org.apache.logging.log4j", "log4j-core").versionRef("log4j")
library("log4j-slf4j-impl", "org.apache.logging.log4j", "log4j-slf4j-impl").versionRef("log4j")
bundle("log4j", listOf("log4j-api", "log4j-core", "log4j-slf4j-impl"))
}
}
}
rootProject.name = "gawds-db"
include("db-server", "db-client", "db-common")
enableFeaturePreview("VERSION_CATALOGS")
And I'm trying to import into my build.gradle.kts (located in my subproject)
plugins {
`maven-publish`
application
`java-library`
}
java {
toolchain {
languageVersion.set(JavaLanguageVersion.of(11))
}
}
repositories {
mavenCentral()
}
configure<JavaApplication> {
mainClass.set("com.gawds.db.MainApp")
}
tasks.compileJava {
options.isIncremental = true
options.isFork = true
options.isFailOnError = false
}
tasks.named<Test>("test") {
useJUnitPlatform()
}
dependencies {
implementation(libs.log4j.api)
implementation(libs.log4j.core)
implementation(libs.log4j.log4j.slf4j.impl)
}
Note: also tried
implementation(libs.bundles.log4j)
But neither referencing libs.alias nor libs.bundles.alias-bundle work.
My project structure is:
/
| settings.gradle.kts
| db-server /
| build.gradle.kts
Just in case helps someone else. First, not sure why intellij marks as red, however from cmd line looks like it works.
From settings.gradle.kts get rid of
enableFeaturePreview("VERSION_CATALOGS")
This was required in previous versions but not in gradle 7.4.x
Also u need to add the repos in the catalog section, see below:
dependencyResolutionManagement {
repositories {
mavenCentral()
}
versionCatalogs {
create("libs") {
version("log4jVersion", "2.17.1")
library("log4j-api", "org.apache.logging.log4j", "log4j-api").versionRef("log4jVersion")
library("log4j-core", "org.apache.logging.log4j", "log4j-core").versionRef("log4jVersion")
library("log4j-slf4j-impl", "org.apache.logging.log4j", "log4j-slf4j-impl").versionRef("log4jVersion")
bundle("log4j", listOf("log4j-api", "log4j-core", "log4j-slf4j-impl"))
}
}
}
rootProject.name = "gawds-db"
include("db-server", "db-client", "db-common")
Another important point that I did wrong in my tests are:
Assuming this alias:
log4j-slf4j-impl
the typesafe equivalent is
log4j.slf4j.impl
so we need to call in the build.gradle.kts:
implementation(libs.log4j.slf4j.impl)
Also u can print the alias to check the name using:
val versionCatalog = extensions.getByType<VersionCatalogsExtension>().named("libs")
println("${versionCatalog.libraryAliases}")
To improve the previous code, we can create a task to print the catalog alias:
abstract class PrintCatalogAlias #Inject constructor(private val r : Project) : DefaultTask() {
#org.gradle.api.tasks.TaskAction
fun catalogAlias() {
val versionCatalog = r.rootProject.project.extensions.getByType<VersionCatalogsExtension>().named("libs")
println("${versionCatalog.libraryAliases}")
}
}
tasks.register<PrintCatalogAlias>("catalog")
In my code there is const val VERSION = $version.
I want to replace $version with real version string which is in my build.gradle.kts.
How can I do this?
Working example here.
One way is to use a template file (stored outside of src tree). Let's call it TemplateVersion.kt:
class Version() {
val version = "__VERSION";
}
and then in build.gradle.kts, as initial part of the compileKotlin task, we generate Version.kt from TemplateVersion.kt:
val sourceFile = File(rootDir.absolutePath + "/resources/TemplateVersion.kt")
val propFile = File(rootDir.absolutePath + "/gradle.properties")
val destFile = File(rootDir.absolutePath + "/src/main/kotlin/${targetPackage}/Version.kt")
tasks.register("generateVersion") {
inputs.file(sourceFile)
inputs.file(propFile)
outputs.file(destFile)
doFirst {
generateVersion()
}
}
tasks.named("compileKotlin") {
dependsOn("generateVersion")
}
fun generateVersion() {
val version: String by project
val rootDir: File by project
val inputStream: InputStream = sourceFile.inputStream()
destFile.printWriter().use { out ->
inputStream.bufferedReader().forEachLine { inputLine ->
val newLine = inputLine.replace("__VERSION", version)
out.println(newLine)
}
}
inputStream.close()
}
where gradle.properties is:
version=5.1.50
It is trivially easy to add more fields to Version.kt, such as a build-timestamp.
(Edit: This has been updated with a proper generateVersion task that will detect changes to gradle.properties. The compileKotlin task will invoke this task).
I have created a Kotlin MPP to share Json utilities between JVM and JS. All the code lies in the common source set and I have configured the necessary targets with their respective dependencies. Without further configuration I'm able to use the utilities from both JVM and JS but not from the common source set of another MPP, which has to do with the way Gradle handles metadata.
I already found the solution (taken from https://medium.com/xorum-io/crafting-and-publishing-kotlin-multiplatform-library-to-bintray-cbc00a4f770)
afterEvaluate {
project.publishing.publications.all {
groupId = group
if (it.name.contains('metadata')) {
artifactId = "$libraryName"
} else {
artifactId = "$libraryName-$name"
}
}
}
and I also got it to work with the Gradle Kotlin DSL:
afterEvaluate {
publishing.publications.all {
this as MavenPublication
artifactId = project.name + "-$name".takeUnless { "metadata" in name }.orEmpty()
}
}
However, this doesn't feel quite right yet.
There is no such code snippet in the official documentation.
The documentation advertises that a single dependency from the common source set should suffice to automatically resolve target specific dependencies: https://kotlinlang.org/docs/reference/building-mpp-with-gradle.html#metadata-publishing. I had to add the dependency for each target, respectively, for it to work.
this as MavenPublication is necessary because Publication has no field artifactId.
I use project.name instead of libraryName.
Is this even remotely the right way to do things or am I missing some other option which would make the whole process trivial?
Right now I'm using Kotlin 1.3.72 and Gradle 5.2.1 with enableFeaturePreview("GRADLE_METADATA") in settings.gradle.kts. I also tried it with Gradle 6.5.1 (latest) but it behaves exactly the same.
For now I'm glad that it's working at all but I suspect there is a cleaner way to do this. I'd really appreciate if someone with a bit more Gradle expertise could clear things up for me or point me into the right direction.
Edit:
gradle.build.kts for completeness. Although there isn't much going on here.
group = "org.example"
version = "1.0-SNAPSHOT"
plugins {
kotlin("multiplatform") version "1.3.72"
`maven-publish`
}
repositories {
mavenCentral()
}
kotlin {
jvm()
sourceSets {
val commonMain by getting {
dependencies {
implementation(kotlin("stdlib-common"))
}
}
val jvmMain by getting {
dependencies {
implementation(kotlin("stdlib"))
}
}
}
}
There wasn't really a problem after all. The solution is to simply add enableFeaturePreview("GRADLE_METADATA") to the consuming project too.
According to https://kotlinlang.org/docs/reference/building-mpp-with-gradle.html#metadata-publishing this shouldn't be necessary:
In earlier Gradle versions starting from 5.3, the module metadata is
used during dependency resolution, but publications don't include any
module metadata by default. To enable module metadata publishing, add
enableFeaturePreview("GRADLE_METADATA") to the root project's
settings.gradle file.
Weirdly it only works when both publishing project and consuming project have metadata enabled, even when both use the latest Gradle version.
enableFeaturePreview("GRADLE_METADATA") is enabled by default in latest gradle.
According to this you need to substitute "kotlinMultiplatform" by "" an empty string.
I finally managed to accomplish publishing to bintray with maven-publish plugin only, without outdated bintray library.
Here is my full maven.publish.gradle.kts:
import java.io.FileInputStream
import java.util.*
import org.gradle.api.publish.PublishingExtension
apply(plugin = "maven-publish")
val fis = FileInputStream("local.properties")
val properties = Properties().apply {
load(fis)
}
val bintrayUser = properties.getProperty("bintray.user")
val bintrayApiKey = properties.getProperty("bintray.apikey")
val bintrayPassword = properties.getProperty("bintray.gpg.password")
val libraryVersion: String by project
val publishedGroupId: String by project
val artifact: String by project
val bintrayRepo: String by project
val libraryName: String by project
val bintrayName: String by project
val libraryDescription: String by project
val siteUrl: String by project
val gitUrl: String by project
val licenseName: String by project
val licenseUrl: String by project
val developerOrg: String by project
val developerName: String by project
val developerEmail: String by project
val developerId: String by project
project.group = publishedGroupId
project.version = libraryVersion
afterEvaluate {
configure<PublishingExtension> {
publications.all {
val mavenPublication = this as? MavenPublication
mavenPublication?.artifactId =
"${project.name}${"-$name".takeUnless { "kotlinMultiplatform" in name }.orEmpty()}"
}
}
}
configure<PublishingExtension> {
publications {
withType<MavenPublication> {
groupId = publishedGroupId
artifactId = artifact
version = libraryVersion
pom {
name.set(libraryName)
description.set(libraryDescription)
url.set(siteUrl)
licenses {
license {
name.set(licenseName)
url.set(licenseUrl)
}
}
developers {
developer {
id.set(developerId)
name.set(developerName)
email.set(developerEmail)
}
}
organization {
name.set(developerOrg)
}
scm {
connection.set(gitUrl)
developerConnection.set(gitUrl)
url.set(siteUrl)
}
}
}
}
repositories {
maven("https://api.bintray.com/maven/${developerOrg}/${bintrayRepo}/${artifact}/;publish=1") {
credentials {
username = bintrayUser
password = bintrayApiKey
}
}
}
}
And build.gradle.kts:
plugins {
id("kotlin-multiplatform")
}
kotlin {
sourceSets {
jvm()
js() {
browser()
nodejs()
}
linuxX64()
linuxArm64()
mingwX64()
macosX64()
iosArm64()
iosX64()
val commonMain by getting {
dependencies {
}
}
val commonTest by getting {
dependencies {
implementation(kotlin("test-common"))
implementation(kotlin("test-annotations-common"))
}
}
val jsMain by getting {
dependencies {
}
}
val jsTest by getting {
dependencies {
implementation(kotlin("test-js"))
}
}
val jvmMain by getting {
dependencies {
}
}
val jvmTest by getting {
dependencies {
implementation(kotlin("test"))
implementation(kotlin("test-junit"))
}
}
val nativeMain by creating {
dependsOn(commonMain)
dependencies {
}
}
val linuxX64Main by getting {
dependsOn(nativeMain)
}
val linuxArm64Main by getting {
dependsOn(nativeMain)
}
val mingwX64Main by getting {
dependsOn(nativeMain)
}
val macosX64Main by getting {
dependsOn(nativeMain)
}
val iosArm64Main by getting {
dependsOn(nativeMain)
}
val iosX64Main by getting {
dependsOn(nativeMain)
}
}
}
apply(from = "maven.publish.gradle.kts")
Please note, there are bintray.user and bintray.apikey properties in local.properties file.
Also gradle.properties contains rest listed properties above:
libraryVersion = 0.5.22
libraryName = MultiplatformCommon
libraryDescription = Kotlin multiplatform extensions
publishedGroupId = com.olekdia
artifact = multiplatform-common
bintrayRepo = olekdia
bintrayName = multiplatform-common
siteUrl = https://gitlab.com/olekdia/common/libraries/multiplatform-common
gitUrl = https://gitlab.com/olekdia/common/libraries/multiplatform-common.git
.........
kotlin.mpp.enableGranularSourceSetsMetadata = true
systemProp.org.gradle.internal.publish.checksums.insecure = true
If you haven't created organisation in bintray you need to change in this url:
https://api.bintray.com/maven/${developerOrg}/${bintrayRepo}/${artifact}/;publish=1
developerOrg by bintrayUser, where the last one is your user name at bintray.com
I want to download the dependency artifacts manually in the future after Gradle has all the dependency artifacts available, hence I would like to get the URLs which Gradle used to download those artifacts.
Is there a way to get the URL of dependencies which artifacts have been downloaded by Gradle?
use gson for a example:
dependencies {
// https://mvnrepository.com/artifact/com.google.code.gson/gson
compile 'com.google.code.gson:gson:2.8.6'
}
create a task to print url:
task getURLofDependencyArtifact() {
doFirst {
project.configurations.compile.dependencies.each { dependency ->
for (ArtifactRepository repository : project.repositories.asList()) {
def url = repository.properties.get('url')
//https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.8.6/gson-2.8.6.jar
def jarUrl = String.format("%s%s/%s/%s/%s-%s.jar", url.toString(),
dependency.group.replace('.', '/'), dependency.name, dependency.version,
dependency.name, dependency.version)
try {
def jarfile = new URL(jarUrl)
def inStream = jarfile.openStream();
if (inStream != null) {
println(String.format("%s:%s:%s", dependency.group, dependency.name, dependency.version)
+ " -> " + jarUrl)
return
}
} catch (Exception ignored) {
}
}
}
}
}
run ./gradlew getURLofDependencyArtifact
Task :getURLofDependencyArtifact
com.google.code.gson:gson:2.8.6 -> https://jcenter.bintray.com/com/google/code/gson/gson/2.8.6/gson-2.8.6.jar
PS:the result dependency your project's
repositories {
jcenter()
mavenCentral()
}
so, the result maybe:
Task :getURLofDependencyArtifact
com.google.code.gson:gson:2.8.6 -> https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.8.6/gson-2.8.6.jar
using Gradle version 6.0 or above, another way of outputting the URLs is to mix --refresh-dependencies with --info
// bash/terminal
./gradlew --info --refresh-dependencies
// cmd
gradlew --info --refresh-dependencies
or output to file
// bash/terminal
./gradlew --info --refresh-dependencies > urls.txt
// cmd
gradlew --info --refresh-dependencies > urls.txt
note on --refresh-dependencies
It’s a common misconception to think that using --refresh-dependencies
will force download of dependencies. This is not the case: Gradle will
only perform what is strictly required to refresh the dynamic
dependencies. This may involve downloading new listing or metadata
files, or even artifacts, but if nothing changed, the impact is
minimal.
source: https://docs.gradle.org/current/userguide/dependency_management.html
see also: How can I force gradle to redownload dependencies?
Wanted something similar but on Android and Kotlin DSL so based on #andforce's answer developed this which hopefully will be useful for others also,
import org.jetbrains.kotlin.utils.addToStdlib.firstNotNullResult
import java.net.URL
val dependenciesURLs: Sequence<Pair<String, URL?>>
get() = project.configurations.getByName(
"implementation"
).dependencies.asSequence().mapNotNull {
it.run { "$group:$name:$version" } to project.repositories.mapNotNull { repo ->
(repo as? UrlArtifactRepository)?.url
}.flatMap { repoUrl ->
"%s/%s/%s/%s/%s-%s".format(
repoUrl.toString().trimEnd('/'),
it.group?.replace('.', '/') ?: "", it.name, it.version,
it.name, it.version
).let { x -> listOf("$x.jar", "$x.aar") }
}.firstNotNullResult { url ->
runCatching {
val connection = URL(url).openConnection()
connection.getInputStream() ?: throw Exception()
connection.url
}.getOrNull()
}
}
tasks.register("printDependenciesURLs") {
doLast {
dependenciesURLs.forEach { (dependency: String, url: URL?) -> println("$dependency => $url") }
}
}
Update: It might not able to find indirect dependencies however.
We need to take care about aar also.
project.configurations.getByName(
"implementation"
).dependencies.each { dependency ->
for (ArtifactRepository repository : rootProject.repositories.asList()) {
def url = repository.properties.get('url')
def urlString = url.toString()
if (url.toString().endsWith("/")) {
urlString = url.toString()
} else {
urlString = url.toString() + "/"
}
def jarUrl = String.format("%s%s/%s/%s/%s-%s.jar", urlString,
dependency.group.replace('.', '/'), dependency.name, dependency.version,
dependency.name, dependency.version)
def aarUrl = String.format("%s%s/%s/%s/%s-%s.aar", urlString,
dependency.group.replace('.', '/'), dependency.name, dependency.version,
dependency.name, dependency.version)
try {
def jarfile = new URL(jarUrl)
def inStreamJar = jarfile.openStream();
if (inStreamJar != null) {
println(String.format("%s:%s:%s", dependency.group, dependency.name, dependency.version)
+ " -> " + jarUrl)
return
}
} catch (Exception ignored) {
}
try {
def aarfile = new URL(aarUrl).setURLStreamHandlerFactory()
def inStreamAar = aarfile.openStream();
if (inStreamAar != null) {
println(String.format("%s:%s:%s", dependency.group, dependency.name, dependency.version)
+ " -> " + aarUrl)
return
}
} catch (Exception ignored) {
}
}
}