TL;DR
I'm trying to configure two Gradle projects in a way that one project uses files built by the other one.
The first project is added to the second one by includeBuild and the file is defined in the second project as a dependency.
Project testA
settings.gradle:
rootProject.name = 'testA'
build.gradle:
group = 'org.test'
version = '0.0.0.1_test'
task someZip (type: Zip) {
from './settings.gradle'
archiveName = 'xxx.zip'
destinationDir = file("${buildDir}/test")
}
artifacts {
//TODO add something here?
}
Project testB
settings.gradle:
rootProject.name = 'testB'
if (System.getenv('LOCAL_COMPILATION') == 'true') {
includeBuild '../testA'
}
build.gradle:
if (System.getenv('LOCAL_COMPILATION') != 'true') {
repositories {
maven { url '192.168.1.100' }
}
}
configurations {
magic
}
dependencies {
magic 'org.test:xxx:0.0.0.+#zip'
}
task ultimateZip (type: Zip) {
from configurations.magic
archiveName = 'ultimate.zip'
destinationDir = file("${buildDir}/ultimate-test")
}
Description
You may noticed that the example has an option use a maven repository. I wanted to highlight that eventually there will be a possibility to do that.
Using Maven repository is not the point of this question, though, other than the solution should not interfere with that.
(In other words you can assume that System.getenv('LOCAL_COMPILATION') == 'true'.)
The question is how to define the artifact in a way that the other project is able to recognize it.
The preferred solution should be similar to what the Java plugin does because I'm using jar dependencies in my projects and they are working both through includeBuild and through a repository.
The following setup should work (tested with Gradle 5.5.1). It mostly corresponds to your original setup with the exception of the changes indicated by XXX.
Project testA
settings.gradle:
rootProject.name = 'testA'
build.gradle:
group = 'org.test'
version = '0.0.0.1_test'
task someZip (type: Zip) {
from './settings.gradle'
archiveName = 'xxx.zip'
destinationDir = file("${buildDir}/test")
}
// XXX (replaced your empty "artifacts" block)
configurations.create('default')
def myArtifact = artifacts.add('default', someZip) {
name = 'xxx'
}
// XXX (only added to show that publishing works)
apply plugin: 'maven-publish'
publishing {
repositories {
maven { url = 'file:///tmp/my-repo' }
}
publications {
myPub(MavenPublication) {
artifactId myArtifact.name
artifact myArtifact
}
}
}
Project testB
settings.gradle:
rootProject.name = 'testB'
if (System.getenv('LOCAL_COMPILATION') == 'true') {
// XXX (added a dependency substitution to let Gradle know that
// "org.test:xxx" corresponds to the testA project)
includeBuild('../testA') {
dependencySubstitution {
substitute module('org.test:xxx') with project(':')
}
}
}
build.gradle:
if (System.getenv('LOCAL_COMPILATION') != 'true') {
repositories {
// XXX (only changed to show that resolution still works after
// publishing)
maven { url = 'file:///tmp/my-repo' }
}
}
configurations {
magic
}
dependencies {
magic 'org.test:xxx:0.0.0.+#zip'
}
task ultimateZip (type: Zip) {
from configurations.magic
archiveName = 'ultimate.zip'
destinationDir = file("${buildDir}/ultimate-test")
}
As requested in the comments, here’s some more explanation on the created default configuration and the added artifact in project testA.
Composite builds in Gradle currently have the limitation that substituted project dependencies “will always point to the default configuration of the target project”. In your example this means that testA needs to publish in the default configuration. We thus first create the default configuration. Note that some plugins (like java) already create this configuration; you don’t need to create it yourself when using such plugins.
It doesn’t seem to be mentioned explicitly anywhere but as you seem to have found out yourself already, the PublishedArtifacts of a project (as declared with project.artifacts) are important for Gradle to figure out the dependency wiring in composite builds. Hence, we make sure to declare such a PublishedArtifact in testA using this API. The artifact (e.g., its extension) is configured based on the properties of the someZip task. The name seems to not be taken from the someZip task in your example because you manually set archiveName; hence we need to explicitly declare it. If you use archiveBaseName = 'xxx' in someZip instead, then you don’t need the closure when adding the artifact.
Related
I have a library containing usual classes and those especially for unit tests. So I want to publish two separate artifacts from the same project. My working solution with the maven plugin looks like this:
task jarTest (type: Jar, dependsOn: testClasses) {
from sourceSets.test.output
archiveBaseName.set('foo-test')
description = 'test utilities'
}
artifacts {
archives jarTest
}
uploadArchives {
repositories {
mavenDeployer {
//...
addFilter('foo') {artifact, file ->
artifact.name == 'foo'
}
addFilter('foo-test') {artifact, file ->
artifact.name == 'foo-test'
}
}
}
}
Unfortunately the maven plugin is deprecated and will be removed in Gradle 7. maven-publish is the suggested replacement and I'm looking for a replacement solution.
My current attempt looks like
publishing {
publications {
mavenJava(MavenPublication) {
artifact jar
artifact jarTest
}
}
}
There is the obvious problem that there are two artifacts with the same name.
Setting the name like this does not work:
artifactId = jar.archiveBaseName
this neither:
afterEvaluate {
artifactId = jar.archiveBaseName
}
It's possible to configure the artifact like this
artifact(jar) {
classifier "src"
extension "zip"
}
But there is no property for the name regarding the documentation (https://docs.gradle.org/current/dsl/org.gradle.api.publish.maven.MavenArtifact.html).
So I'm looking for sth. like the addFilter from the maven plugin.
I'm trying to publish a library to an AWS S3 Maven repository using this guide. After finally getting it to upload the artifacts to the S3 bucket without error, I made it a dependency of a new project per the guide.
When I tried to build the new project, an error occurred stating that it couldn't find the first of my library's dependencies. Sure enough, there was no pom.xml file generated that would have included that dependency (and others).
Not knowing a lot about how program Gradle tasks, I think the problem is within the pom.withXml {} portion of the script. Or the problem may occur before that since there's not even an empty pom.xml file.
Here is the entire script:
apply plugin: 'com.android.library'
apply plugin: 'maven-publish'
// update these next lines to fit your submodule
group = 'com.myproject'
version = '1.0'
// Add sources as an artifact
task sourceJar(type: Jar) {
from android.sourceSets.main.java.srcDirs
classifier "source"
}
// Loop over all variants
android.libraryVariants.all { variant ->
variant.outputs.all { output ->
// This creates a publication for each variant
publishing.publications.create(variant.name, MavenPublication) {
// The sources artifact from earlier
artifact sourceJar
// Variant dependent artifact, e.g. release, debug
artifact source: output.outputFile, classifier: output.name
// Go through all the dependencies for each variant and add them to the POM
// file as dependencies
pom.withXml {
def dependencies = asNode().appendNode('dependencies')
// Filter out anything that's not an external dependency. You shouldn't
// be publishing artifacts that depend on local (e.g. project) dependencies,
// but who knows...
configurations.getByName(variant.name + "CompileClasspath").allDependencies
.findAll { it instanceof ExternalDependency }
.each {
def dependency = dependencies.appendNode('dependency')
dependency.appendNode('groupId', it.group)
dependency.appendNode('artifactId', it.name)
dependency.appendNode('version', it.version)
}
}
}
}
}
// Ensure that the publish task depends on assembly
tasks.all { task ->
if (task instanceof AbstractPublishToMaven) {
task.dependsOn assemble
}
}
// Configure the destination repository with
// S3 URL and access credentials
publishing {
repositories {
maven {
url "s3://sdk.myproject.com.s3.amazonaws.com"
credentials(AwsCredentials) {
accessKey AWS_ACCESS_KEY
secretKey AWS_SECRET_KEY
}
}
}
}
Any ideas about what's going wrong? Thank you!
I wrote a gradle script where I am creating the zip and war file and then I need to upload/publish it to the artifactory but the issue is I specified the war file in my artifact task even after that it is publishing everything to the artifactory zip,tar and war instead of only war file.
apply plugin: 'war'
apply plugin: 'java'
apply plugin: 'distribution'
//-- set the group for publishing
group = 'com.xxx.discovery'
/**
* Initializing GAVC settings
*/
def buildProperties = new Properties()
file("version.properties").withInputStream {
stream -> buildProperties.load(stream)
}
//add the jenkins build version to the version
def env = System.getenv()
if (env["BUILD_NUMBER"]) buildProperties.coveryadBuildVersion += "_${env["BUILD_NUMBER"]}"
version = buildProperties.coveryadBuildVersion
println "${version}"
//name is set in the settings.gradle file
group = "com.aaa.covery"
version = buildProperties.discoveryadBuildVersion
println "Building ${project.group}:${project.name}:${project.version}"
repositories {
maven {
url "http://cxxxxt.tshaaaaa.tho.com:9000/artifactory/libselease"
}
maven {
url "http://cxxxxt.tshaaa.tho.com:9000/artifactory/cache"
}
}
dependencies {
compile ([
"com.uters.omni:HermesSessionAPI:1.2",
"com.uters.proxy:ProxyResources:1.1",
"com.uters.omni:SeshataDirectory:1.0.1" ,
"com.euters.omni:CybeleInfrastructure:1.1.2",
"com.euters:JSONBus:1.4.1",
"javaee:javaee-api:5"
])
}
distributions {
main {
contents {
from {
war.outputs
}
}
}
}
// for publishing to artifactory
artifacts {
archives war
}
According to gradle distribution plugin documentation:
All of the files in the “src/$distribution.name/dist” directory will automatically be included in the distribution.
And also,
The distribution plugin adds the distribution archives as candidate for default publishing artifacts.
In other words, by default all the files will be published so this explains the behavior you're experiencing.
What you can probably do in order to workaround this behavior is to define the contents copySpec more accurately by explicitly exclude the unwanted files, i.e.:
distributions {
main {
contents {
exclude('**/.zip')
exclude('**/.tar')
from {
war.outputs
}
}
}
}
Note that I didn't try the above by myself though so some fine tuning might be needed. However I believe that you can find the data you need in the CopySpec Interface documentation
I've hacked together combination of build.gradle and settings.gradle below for creating an ad-hoc multi-module project out of several single-module projects (e.g., an application and all of its dependencies, or a shared library and everything that uses that library).
settings.gradle:
// find all subprojects and include them
rootDir.eachFileRecurse {
if (it.name == "build.gradle") {
def projDir = it.parentFile
if (projDir != rootDir) {
include projDir.name
project(":${projDir.name}").projectDir = projDir
}
}
}
build.gradle::
// Make sure we've parsed subproject dependencies
evaluationDependsOnChildren()
// Map of all projects by artifact group and name
def declarationToProject = subprojects.collectEntries { p -> [toDeclaration(p), p] }
// Replace artifact dependencies with subproject dependencies, if possible
subprojects.each { p ->
def changes = [] // defer so we don't get ConcurrentModificationExceptions
p.configurations.each { c ->
c.dependencies.each { d ->
def sub = declarationToProject[[group:d.group, name:d.name]]
if (sub != null) {
changes.add({
c.dependencies.remove(d)
p.dependencies.add(c.name, sub)
})
}
}
}
for (change in changes) {
change()
}
}
This works, but it's hard to share -- if somebody else wants to do something similar they have to copy my *.gradle files or cut and paste.
What I'd like to do is take this functionality and encapsulate it in a plugin. The build.gradle part looks easy enough to do in the plugin apply() method, but it seems like the list of subprojects is already set in stone before the plugin gets a chance at it. Is there any way to get in earlier in the build process, e.g. by applying to something other than Project? Or should I resign myself to giving my plugin a task for overwriting settings.gradle?
Solution: Per Peter Niederweiser's answer, I moved the code above into two plugins, one to be called from settings.gradle and the other to be called from build.gradle. In settings.gradle:
buildscript {
repositories { /* etc... */ }
dependencies { classpath 'my-group:my-plugin-project:1.0-SNAPSHOT' }
}
apply plugin: 'find-subprojects'
And in build.gradle:
buildscript {
repositories { /* etc... */ }
dependencies { classpath 'my-group:my-plugin-project:1.0-SNAPSHOT' }
}
evaluationDependsOnChildren()
apply plugin: 'local-dependencies'
Note that calling the plugin from settings.gradle doesn't work in Gradle 1.11 or 1.12 but does work in Gradle 2.0.
You'd need to apply a plugin in settings.gradle, which I believe is supported in recent versions.
I am trying to use the avro-gradle-plugin on github, but have not gotten any luck getting it to work. Does anyone have any sample code on how they get it to work?
I figured out how to do it myself. The following is a snippet that I would like to share for people who might run into the same issues as I did:
apply plugin: 'java'
apply plugin: 'avro-gradle-plugin'
sourceCompatibility = "1.6"
targetCompatibility = "1.6"
buildscript {
repositories {
maven {
// your maven repo information here
}
}
dependencies {
classpath 'org.apache.maven:maven-artifact:2.2.1'
classpath 'org.apache.avro:avro-compiler:1.7.1'
classpath 'org.apache.avro.gradle:avro-gradle-plugin:1.7.1'
}
}
compileAvro.source = 'src/main/avro'
compileAvro.destinationDir = file("$buildDir/generated-sources/avro")
sourceSets {
main {
java {
srcDir compileAvro.destinationDir
}
}
}
dependencies {
compileAvro
}
I found "com.commercehub.gradle.plugin.avro" gradle plugin to work better.
use the folllowing:
// Gradle 2.1 and later
plugins {
id "com.commercehub.gradle.plugin.avro" version "VERSION"
}
// Earlier versions of Gradle
buildscript {
repositories {
jcenter()
}
dependencies {
classpath "com.commercehub.gradle.plugin:gradle-avro-plugin:VERSION"
}
}
apply plugin: "com.commercehub.gradle.plugin.avro"
more details at https://github.com/commercehub-oss/gradle-avro-plugin
When evaluating a plugin the following questions needs to be asked:
Are generated files included into source jar?
Is plugin fast? Good plugin use avro tools api instead of forking VM for every file. For large amount of files creating VM for every file can take 10min to compile.
Do you need intermediate avsc files?
Is build incremental (i.e. do not regenerate all files unless one of the sources changed)?
Is plugin flexible enough to give access to generated schema files, so further actions, such as registration schema in schema repository can be made?
It is easy enough to implement without any plugin if you are not happy with plugin or need more flexibility.
//
// define source and destination
//
def avdlFiles = fileTree('src/Schemas').include('**/*.avdl')
// Do NOT generate into $buildDir, because IntelliJ will ignore files in
// this location and will show errors in source code
def generatedJavaDir = "generated/avro/java"
sourceSets.main.java.srcDir generatedJavaDir
//
// Make avro-tools available to the build script
//
buildscript {
dependencies {
classpath group:'org.apache.avro', name:'avro-tools' ,version: avro_version
}
}
//
// Define task's input and output, compile idl to schema and schema to java
//
task buildAvroDtos(){
group = "build"
inputs.files avdlFiles
outputs.dir generatedJavaDir
doLast{
avdlFiles.each { avdlFile ->
def parser = new org.apache.avro.compiler.idl.Idl(avdlFile)
parser.CompilationUnit().getTypes().each { schema ->
def compiler = new org.apache.avro.compiler.specific.SpecificCompiler(schema)
compiler.compileToDestination(avdlFile, new File(generatedJavaDir))
}
}
}
}
//
// Publish source jar, including generated files
//
task sourceJar(type: Jar, dependsOn: buildAvroDtos) {
from sourceSets.main.allSource
// Package schemas into source jar
into("Schemas") { from avdlFiles }
}
// Clean "generated" folder upon "clean" task
clean {
delete('generated')
}
Configuration for avro with gradle as build tool need to add along with applying java plugin.
below changes in settings.gradle
pluginManagement {
repositories {
gradlePluginPortal()
mavenCentral()
}
}
below changes in build.gradle
plugins {
id "com.github.davidmc24.gradle.plugin.avro" version "1.3.0"
}
repositories {
mavenCentral()
}
dependencies {
implementation "org.apache.avro:avro:1.11.0"
}
generateAvroJava {
source("${projectDir}/src/main/resources/avro")//sourcepath avrofile
}
if you want to generate setter methods too add this task too in build.gradle
avro {
createSetters = true
}
link for reference