I'm having some real problems understanding how all the pieces in gradle to create and upload an artifact fit together.
My intention in this script is simple: I want to download a source tarball and possibly a bunch of dependencies, run a "build.sh" shellscript which will end up creating a binary tarball and have the gradle script publish it to an artifact repo.
The main idea is that I can use gradle's dependency management, maven artifact knowledge and build parallelization and avoidance to control the execution of the build scripts themselves, mainly to manage my set of third party binary dependencies...
The following script fails with a 400 error, I suspect it's because I'm not linking the artifact with the actual output file.
What's the right and proper way to go?
apply plugin: 'maven'
version 'testarch-4.2'
repositories {
maven {
url "http://nexus/..."
}
}
configurations {
sourceArchive
binaryArchive
}
dependencies {
sourceArchive "org.gnu:bash:4.2:src#tgz"
}
task buildFromSource(type: Exec) {
inputs.files configurations.sourceArchive.files
outputs.file file("${project.name}-${project.version}.tgz")
executable './build.sh'
def myArgs = configurations.sourceArchive.files.path
myArgs.add(0, outputs.files.asPath)
args myArgs
}
artifacts {
// Is this really the only way to transform a singleton collection
// into the singleton?
// def outputFile
// buildFromSource.outputs.files.each { outputFile = it }
// Nope: this is better magic:
def outputFile = buildFromSource.outputs.files.singleFile
println outputFile.path
binaryArchive file: outputFile, name: 'bash'
// binaryArchive file: file(buildFromSource.outputs.files.asPath), name: 'bash'
}
uploadArchives {
configuration = configurations.binaryArchive
repositories.mavenDeployer {
repository(url: "http://nexus/..") {
authentication(userName: "me", password: "secret!")
}
pom.groupId = 'org.gnu'
}
}
uploadArchives.dependsOn buildFromSource
The error I get is:
* What went wrong:
Execution failed for task ':uploadArchives'.
> Could not publish configuration 'binaryArchive'
> Error deploying artifact 'org.gnu:bash:tgz': Error deploying artifact: Failed to transfer file: http://nexus/.../org/gnu/bash/testarch-4.2/bash-testarch-4.2.tgz. Return code is: 400
Updated from comments, same error - trying to get access to the nexus logs for further debugging.
Error from Nexus is "missing entity", see: Missing Request Entity response to a PUT to Nexus
The root cause of my problem was I was testing with an empty file. Nexus doesn't like empty files. As soon as I put content in it, Nexus was happy and the code was working.
Related
My gradle build has a subproject with a task that produces a file
$ ./gradlew :strings:tokenizeStrings # creates strings/string_tokens.csv
then in my root project I have a task which consumes that file
tasks.generateLocalizationFiles {
inputTokensCsvFile.set(layout.projectDirectory.file("strings/string_tokens.csv"))
}
this works, but since gradle doesn't know about the dependency, it only works if I run the two tasks manually in the right order
$ ./gradlew :strings:tokenizeStrings
$ ./gradlew :generateLocalizationFiles
I want to add the proper dependency to gradle so that I can run just :generateLocalizationFiles and it will go into the subproject and do whatever it needs to. But I can't figure out the right way to do it.
What I've tried:
Following Simple sharing of artifacts between projects, I tried adding a consumable configuration to the suproject build script
val localizationData by configurations.creating {
isCanBeConsumed = true
isCanBeResolved = false
}
tasks.tokenizeStrings {
artifacts {
add("localizationData", outputTokensCsvFile) {
builtBy(this)
}
}
}
and then a resolvable configuration plus the dependency to the root project build script
val localizedStringData by configurations.creating {
isCanBeConsumed = false
isCanBeResolved = true
}
// hook up our resolvable configuration to the strings' consumable configuration
dependencies {
localizedStringData(project(mapOf(
"path" to ":strings",
"configuration" to "localizationData")
))
}
tasks.generateLocalizationFiles {
dependsOn(localizedStringData)
inputTokensCsvFile.set(localizedStringData.singleFile)
}
but that fails, seemingly because the consumable configuration is not populated?
Caused by: java.lang.IllegalStateException: Expected configuration ':localizedStringData' to contain exactly one file, however, it contains no files.
You need to add the outgoing artifact directly in the subproject build script, not inside the task configuration (which is only run lazily). You also don't need builtBy if you're using a RegularFileProperty for the artifact.
val localizationData by configurations.creating {
isCanBeConsumed = true
isCanBeResolved = false
}
artifacts {
add("localizationData", tasks.tokenizeStrings.flatMap { it.outputTokensCsvFile })
}
The trick is to use flatMap to lazily access the task. You should similarly use map when passing it to the task resolving the data. That allows for lazy task creation and implicitly tells gradle about the dependency between the two:
tasks.generateLocalizationFiles {
inputTokensCsvFile.set(localizedStringData.elements.map { it.first().asFile })
}
This still feels somewhat hacky, since it would be very clumsy if you wanted to repeat this for many artifacts, but it does seem to be the idiomatic way of doing it in gradle since it doesn't require any explicit dependency creation via builtBy/dependsOn.
I'm trying to publish a library to an AWS S3 Maven repository using this guide. After finally getting it to upload the artifacts to the S3 bucket without error, I made it a dependency of a new project per the guide.
When I tried to build the new project, an error occurred stating that it couldn't find the first of my library's dependencies. Sure enough, there was no pom.xml file generated that would have included that dependency (and others).
Not knowing a lot about how program Gradle tasks, I think the problem is within the pom.withXml {} portion of the script. Or the problem may occur before that since there's not even an empty pom.xml file.
Here is the entire script:
apply plugin: 'com.android.library'
apply plugin: 'maven-publish'
// update these next lines to fit your submodule
group = 'com.myproject'
version = '1.0'
// Add sources as an artifact
task sourceJar(type: Jar) {
from android.sourceSets.main.java.srcDirs
classifier "source"
}
// Loop over all variants
android.libraryVariants.all { variant ->
variant.outputs.all { output ->
// This creates a publication for each variant
publishing.publications.create(variant.name, MavenPublication) {
// The sources artifact from earlier
artifact sourceJar
// Variant dependent artifact, e.g. release, debug
artifact source: output.outputFile, classifier: output.name
// Go through all the dependencies for each variant and add them to the POM
// file as dependencies
pom.withXml {
def dependencies = asNode().appendNode('dependencies')
// Filter out anything that's not an external dependency. You shouldn't
// be publishing artifacts that depend on local (e.g. project) dependencies,
// but who knows...
configurations.getByName(variant.name + "CompileClasspath").allDependencies
.findAll { it instanceof ExternalDependency }
.each {
def dependency = dependencies.appendNode('dependency')
dependency.appendNode('groupId', it.group)
dependency.appendNode('artifactId', it.name)
dependency.appendNode('version', it.version)
}
}
}
}
}
// Ensure that the publish task depends on assembly
tasks.all { task ->
if (task instanceof AbstractPublishToMaven) {
task.dependsOn assemble
}
}
// Configure the destination repository with
// S3 URL and access credentials
publishing {
repositories {
maven {
url "s3://sdk.myproject.com.s3.amazonaws.com"
credentials(AwsCredentials) {
accessKey AWS_ACCESS_KEY
secretKey AWS_SECRET_KEY
}
}
}
}
Any ideas about what's going wrong? Thank you!
I'm trying to create a test task rule using the example provided in the grails gradle doc but I keep getting "a task with that name already exists" error.
My build script is as follows:
import org.grails.gradle.plugin.tasks.* //Added import here else fails with "Could not find property GrailsTestTask"
buildscript {
repositories {
jcenter()
}
dependencies {
classpath "org.grails:grails-gradle-plugin:2.0.0"
}
}
version "0.1"
group "example"
apply plugin: "grails"
repositories {
grails.central() //creates a maven repo for the Grails Central repository (Core libraries and plugins)
}
grails {
grailsVersion = '2.3.5'
groovyVersion = '2.1.9'
springLoadedVersion '1.1.3'
}
dependencies {
bootstrap "org.grails.plugins:tomcat:7.0.50" // No container is deployed by default, so add this
compile 'org.grails.plugins:resources:1.2' // Just an example of adding a Grails plugin
}
project.tasks.addRule('Pattern: grails-test-app-<phase>') { String taskName ->
println tasks //shows grails-test-app-xxxxx task. Why?
//if (taskName.startsWith('grails-test-app') && taskName != 'grails-test-app') {
// task(taskName, type: GrailsTestTask) {
// String testPhase = (taskName - 'grails-test-app').toLowerCase()
// phases = [testPhase]
// }
//}
}
Running $gradle grails-test-integration
or in fact anything of the form $gradle grails-test-app-xxxxxxxx yields the error "Cannot add task 'gradle grails-test-app-xxxxxxxx as a task with that name already exists".
Can someone please advise how I can resolve this error? Thanks.
If you don't mind overriding the task created by the plugin, you might want to try
task(taskName, type: GrailsTestTask, overwrite: true)
In general, when using task rules that can be called multiple times (for instance if you have multiple tasks depending on a task eventually added by your rules), I use the following test before actually creating the task:
if (tasks.findByPath(taskName) == null) {tasks.create(taskName)}
This will call the task() constructor only if this task name does not exists.
I have a gradle project which requires some data files available somewhere on the internet using http. The goal is that this immutable remote file is pulled once upon first build. Subsequent build should not download again.
How can I instruct gradle to fetch the given file to a local directory?
I've tried
task fetch(type:Copy) {
from 'http://<myurl>'
into 'data'
}
but it seems that copy task type cannot deal with http.
Bonus question: is there a way to resume a previously aborted/interrupted download just like wget -c does?
How about just:
def f = new File('the file path')
if (!f.exists()) {
new URL('the url').withInputStream{ i -> f.withOutputStream{ it << i }}
}
You could probably use the Ant task Get for this. I believe this Ant task does not support resuming a download.
In order to do so, you can create a custom task with name MyDownload. That can be any class name basically. This custom task defines inputs and outputs that determine whether the task need to be executed. For example if the file was already downloaded to the specified directory then the task is marked UP-TO-DATE. Internally, this custom task uses the Ant task Get via the built-in AntBuilder.
With this custom task in place, you can create a new enhanced task of type MyDownload (your custom task class). This task set the input and output properties. If you want this task to be executed, hook it up to the task you usually run via task dependencies (dependsOn method). The following code snippet should give you the idea:
task downloadSomething(type: MyDownload) {
sourceUrl = 'http://www.someurl.com/my.zip'
target = new File('data')
}
someOtherTask.dependsOn downloadSomething
class MyDownload extends DefaultTask {
#Input
String sourceUrl
#OutputFile
File target
#TaskAction
void download() {
ant.get(src: sourceUrl, dest: target)
}
}
Try like that:
plugins {
id "de.undercouch.download" version "1.2"
}
apply plugin: 'java'
apply plugin: 'de.undercouch.download'
import de.undercouch.gradle.tasks.download.Download
task downloadFile(type: Download) {
src 'http://localhost:8081/example/test-jar-test_1.jar'
dest 'localDir'
}
You can check more here: https://github.com/michel-kraemer/gradle-download-task
For me works fine..
The suggestion in Ben Manes's comment has the advantage that it can take advantage of maven coordinates and maven dependency resolution. For example, for downloading a Derby jar:
Define a new configuration:
configurations {
derby
}
In the dependencies section, add a line for the custom configuration
dependencies {
derby "org.apache.derby:derby:10.12.1.1"
}
Then you can add a task which will pull down the right files when needed (while taking advantage of the maven cache):
task deployDependencies() << {
String derbyDir = "${some.dir}/derby"
new File(derbyDir).mkdirs();
configurations.derby.resolve().each { file ->
//Copy the file to the desired location
copy {
from file
into derbyDir
// Strip off version numbers
rename '(.+)-[\\.0-9]+\\.(.+)', '$1.$2'
}
}
}
(I learned this from https://jiraaya.wordpress.com/2014/06/05/download-non-jar-dependency-in-gradle/).
Using following plugin:
plugins {
id "de.undercouch.download" version "3.4.3"
}
For a task which has the purpose of only downloading
task downloadFile(type: Download) {
src DownloadURL
dest destDir
}
For including download option into your task:
download {
src DownloadURL
dest destDir
}
For including download option with multiple downloads into your task:
task downloadFromURLs(){
download {
src ([
DownloadURL1,
DownloadURL2,
DownloadURL3
])
dest destDir
}
}
Hope it helped :)
just now ran into post on upcoming download task on gradle forum.
Looks like the perfect solution to me.. Not (yet) available in an official gradle release though
Kotlin Version of #Benjamin's Answer
buildscript {
repositories {
jcenter()
google()
}
dependencies {
classpath("com.android.tools.build:gradle:4.0.1")
}
}
tasks.register("downloadPdf"){
val path = "myfile.pdf"
val sourceUrl = "https://file-examples-com.github.io/uploads/2017/10/file-sample_150kB.pdf"
download(sourceUrl,path)
}
fun download(url : String, path : String){
val destFile = File(path)
ant.invokeMethod("get", mapOf("src" to url, "dest" to destFile))
}
I build a jar file without using Gradle Jar task (I need to be using Ant task for that inside my task). How do I configure uploadArchives to be able to install JAR in specified repository.
I have tried to override default artifact with
uploadArchives {
repositories {
mavenDeployer {
// some Maven configuration
}
}
}
artifacts {
archives file: file('bin/result.jar')
}
but I'm getting an error that there may not be 2 artifacts with the same type and classifier, which means this configuration adds rather that overrides configuration.
You are right, artifacts closure can only add artifacts to the given configuration (see ArtifactHandler API).
You have two options:
1) Add an artifact filter as described here (see ch. 45.6.4.1. "Multiple artifacts per project"). If you use this, try declaring your archives configuration like:
artifacts {
archives file: file('bin/result.jar'), name: 'result', type: 'jar'
}
This way, you something like this in your artifact filter:
addFilter('result') {artifact, file ->
artifact.name == 'result'
}
2) Upload it as a separate maven module. If result.jar is the only jar you are uploading this may be a good solution.
configurations {
resultArchives
}
uploadResultArchives {
repositories {
mavenDeployer {
repository(url: "same/url/here")
}
}
}
artifacts{
resultArchives file: file('bin/result.jar')
}
Hope this helps.