Specify version when deploying to nexus from maven - maven

I've forked Confluent's Kafka Connect HDFS writer and now I'd like to deploy a version of this jar to my local Nexus.
mvn clean deploy Works like a charm and deploys the jar.
https://[nexus]/repository/releases/io/confluent/kafka-connect-hdfs/5.0.0/kafka-connect-hdfs-5.0.0.jar
So far so good, but to make a distinction between the confluent versions and my own deployment I'd like to change the version of the build to something like 5.0.0-1 or so (preferably the tag name when pushed, but that's step 2)
The pom.xml is basically the same as the 5.0.0-post release, but here the most important parts:
<parent>
<groupId>io.confluent</groupId>
<artifactId>kafka-connect-storage-common-parent</artifactId>
<version>5.0.0</version>
</parent>
<artifactId>kafka-connect-hdfs</artifactId>
<packaging>jar</packaging>
<name>kafka-connect-hdfs</name>
<organization>
<name>Confluent, Inc.</name>
<url>http://confluent.io</url>
</organization>
<url>http://confluent.io</url>
<description>
A Kafka Connect HDFS connector for copying data between Kafka and Hadoop HDFS.
</description>
...
<dependencies>
...
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-connect-storage-common</artifactId>
<version>${confluent.version}</version>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-connect-storage-core</artifactId>
<version>${confluent.version}</version>
</dependency>
...
</dependencies>
<build>
<plugins>
<plugin>
<groupId>io.confluent</groupId>
<version>0.10.0</version>
<artifactId>kafka-connect-maven-plugin</artifactId>
...
So first I added <version> tags to the pom.xml, but it started using that as the default for all confluent.version and complained that it couldn't find for example: https://[nexus]/repository/releases/io/confluent/kafka-connect-storage-hive/5.0.0-1/kafka-connect-storage-hive-5.0.0-1.pom
Next I tried the versions plugin from maven
mvn versions:set -DnewVersion=5.0.0-1 clean deploy
But that complained about the parent:
[ERROR] Failed to execute goal org.codehaus.mojo:versions-maven-plugin:2.7:set (default-cli) on project kafka-connect-hdfs: Project version is inherited from parent. -> [Help 1]
I don't even care if the version is 5.0.0 in the code, I just wat to deploy to a different version in our artifactory.
I'm not a maven expert, so maybe I'm missing some very basic clue, but all help is welcome.

So there were some good suggestions, but in the end the one thing that worked best for me in our setup was to use the deploy:deploy-file command for maven.
mvn deploy:deploy-file \
-Dfile=target/kafka-connect-hdfs-5.0.0.jar \
-DrepositoryId=[nexus id] \
-Durl=[nexus url] \
-Dversion=$TAG \
-DgroupId=io.confluent \
-DartifactId=kafka-connect-hdfs
The major downside was that I had to respecify parameters that were already present in the pom.xml (artifactId, groupId, ect), but it works, and that's what counts :-)

You can specify the version with ${revision} parameter.
To do this, you need to add <version> tag with this variable in pom.xml:
<artifactId>kafka-connect-hdfs</artifactId>
<version>5.0.0-${revision}</version>
<packaging>jar</packaging>
And then provide it to the maven command. E.g.,
mvn clean package -Drevision=01 will generate kafka-connect-hdfs-5.0.0-01.jar file.

Related

Maven: automatically update only some libraries to their latest version

I have created a parent pom project and some libraries that I manage.
So, for example:
<parent>
<groupId>my.group</groupId>
<artifactId>parent</artifactId>
<version>1.2.3-SNAPSHOT</version>
</parent>
<dependencies>
...a loooot of them
<dependency>
<groupId>my.group</groupId>
<artifactId>lib</artifactId>
<version>1.2.3-SNAPSHOT</version>
</dependency>
..otheeeeers
</dependencies>
Now, I'd like to always use the latest release or snapshot depending I'm on the develop or release branch.
Problem is that the "new" maven 3 approach only has:
mvn versions:use-latest-releases //release
mvn versions:use-latest-versions //snapshot
But this would update every other dependency and I don't want this! I just want to update the ones having groupId my.group.
Is this possible?
edit:
Following khmarbaise advice I'm using:
mvn versions:use-latest-versions "-Dincludes=com.project.my::::"
But it seems that it only works for non-snapshot versions.
ie. If I define 0.0.2 version of my lib it works, while 0.0.2-SNAPSHOT is not seen..is this normal?
Another thing I noticed is that it only works for inline properties, for example this will not work:
<my-lib.version>0.0.1-SNAPSHOT</my-lib.version>
<groupId>com.project.my</groupId>
<artifactId>mylib</artifactId>
<version>${my-lib.version}</version> <--- undetected, I need to put 0.0.1-SNAPSHOT explicitely
Following the advice of khmarbaise, just call something like
mvn versions:use-latest-releases -Dincludes=mygroup:*:*:*:*

Issue in adding third party dependency in Maven during Jenkins build - ODM

I am trying to build ODM projects outside of eclipse using the Jenkins pipeline and Maven plugin. I am following the link : https://www.ibm.com/support/knowledgecenter/en/SSQP76_8.9.0/com.ibm.odm.dserver.rules.designer.run/build_topics/con_buildcmd_intro.html
Though this link works well without the Jenkins pipeline in my local(Windows), but when I try to run the same setup in Jenkins(Linux machine), I am getting the following error :
[WARNING] The POM for com.ibm.rules.buildcommand:rules-compiler-maven-plugin:jar:8.10.0.0 is missing, no dependency information available
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-compiler-plugin is missing. # helloWorld:hello-xom:[unknown-version], /var/lib/jenkins/workspace/odm-devops-build/Hello XOM/pom.xml, line 19, column 21
[ERROR] Unresolveable build extension: Plugin com.ibm.rules.buildcommand:rules-compiler-maven-plugin:8.10.0.0 or one of its dependencies could not be resolved: Failure to find com.ibm.rules.buildcommand:rules-compiler-maven-plugin:jar:8.10.0.0 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced #
[ERROR] Unknown packaging: decisionservice # helloWorld:hello-main:[unknown-version], /var/lib/jenkins/workspace/odm-devops-build/Hello Main Service/pom.xml, line 14, column 16
The pom file which is referred in the above error is as below:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>helloWorld</groupId>
<artifactId>parent</artifactId>
<version>1.0.0</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hello-main</artifactId>
<packaging>decisionservice</packaging>
<build>
<plugins>
<plugin>
<groupId>com.ibm.rules.buildcommand</groupId>
<artifactId>rules-compiler-maven-plugin</artifactId>
<configuration>
<deployments>
<deployment>
<name>simple dep</name>
</deployment>
</deployments>
<resolvers>
<resolver>
<!-- The values of the kind and url of the project correspond to the 'kind' and 'url' attribute values of an 'entries' element in the .ruleproject file. -->
<kind>JAVA_PROJECT</kind>
<url>platform:/Hello XOM</url>
<!-- The artifactKey references the groupId and artifactId of a Maven dependency. -->
<artifactKey>${project.groupId}:hello-xom</artifactKey>
</resolver>
</resolvers>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>hello-xom</artifactId>
<type>jar</type>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>
Please let me know if anyone has faced a similar issue or has a possible resolution to the above issue.
Thanks in advance.
Possible resolution: "install the Build Command Maven plugin and its related Operational Decision Manager artifacts in your Maven repository" ... "on each machine that contains build agents"
To deploy the Build Command Maven plugin to a remote Maven repository, you must configure Maven in your environment. You then use the following command: mvn deploy:deploy-file -Dfile=rules-compiler-maven-plugin.jar -DpomFile=rules-compiler-maven-plugin.pom
This requires access to the Jenkins / Maven server, and at many companies would be done by DevOps.
You install the Build Command Maven plugin and its related Operational Decision Manager artifacts in your Maven repository.
Deploying the Build Command plugin
You deploy the plugin only once on each machine that contains build agents.
To deploy the Build Command Maven plugin to a remote Maven repository, you must first configure Maven in your environment.
Then, in <InstallDir>/buildcommand/rules-compiler, open a command prompt.
Use the following command:
mvn deploy:deploy-file -Dfile=rules-compiler.jar -DpomFile=rules-compiler-maven-plugin.pom
If you do not have a remote repository, you can test the plugin locally. You run the following command:
mvn install:install-file -Dfile=rules-compiler.jar -DpomFile=rules-compiler-maven-plugin.pom
This command adds the following plugin in your local Maven repository:
com/ibm/rules/buildcommand/rules-compiler
 Deploying the annotations archive
If you want to build COBOL projects or projects that use XOM annotations, you must also deploy the annotations archive to your environment before you can build the projects.
In <InstallDir>/buildcommand/rules-compiler, open a command prompt.
Use the following command:
mvn deploy:deploy-file -Dfile=rules-compiler.jar -DpomFile=xom-annotations.pom
If you do not have a remote repository, you can test the plugin locally. You run the following command:
mvn install:install-file -Dfile=rules-compiler.jar -DpomFile=xom-annotations.pom
This command adds the following plugin in your local Maven repository:
com/ibm/rules/buildcommand/xom-annotation
Source

Jenkins fails to build multi-module Maven project

I have a multi-module Maven project where I have multiple micro services as modules so I have modules listed in my parent pom.xml like below:
<modules>
<module>core</module>
<module>model-base</module>
<module>module1</module>
<module>module2</module>
...
<module>module5</module>
<module>module7</module>
<module>module6</module>
</modules>
Here the module7 is dependent on module5, 6 so I have dependencies listed like below in my module7 pom.xml:
<parent>
<artifactId>pojectA</artifactId>
<groupId>com.domain</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>module7</artifactId>
<dependencies>
<dependency>
<groupId>com.domain</groupId>
<artifactId>core</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.domain</groupId>
<artifactId>module5</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.domain</groupId>
<artifactId>module6</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>
When I run mvn clean package in my local the module5, 6 called before the module7 as expected but in Jenkins it is trying to build module 5 then module7 making build fail saying:
[ERROR] Failed to execute goal on project module7: Could not resolve dependencies for project module7:jar:1.0-SNAPSHOT: Could not find artifact module6:jar:1.0-SNAPSHOT -> [Help 1]
Do I need to run any other jobs or re-order the modules in my pom.xml, how is it differ from local to Jenkins? Appreciate any help on this.
The order of modules is not relevant. Maven recognizes which project depends on which other project(s) and sets the build order in the reactor accordingly. See POM Reference, Aggregation (or Multi-Module):
You do not need to consider the inter-module dependencies yourself when listing the modules, i.e. the ordering of the modules given by the POM is not important. Maven will topologically sort the modules such that dependencies are always build before dependent modules.
Add Pre-Step as per below attached screenshot. This will compile all your top modules.
Then we can execute which ever module we want.
As is probably quite well understood, the issue is that the dependencies between the child modules fail because they aren't installed in the local repository yet (because they are yet to be built). The goal that causes this (for me anyway) is mvn test, which is invoked by mvn package. Your local build probably works because at some point you've done a mvn install and this has bootstrapped your system.
In Jenkins the only way I've found to make these builds work is to use the Pre-build step invoking a Maven target of install, and then build the main step as usual.

Maven copy parents declared in the POMs

My final goal is to create a Maven repository in a certain directory containing only a certain set of artifacts and all their dependencies.
For this I use the following command:
mvn.bat dependency:copy-dependencies -f dependencies.pom
-DoutputDirectory=localRepoDir -Dmdep.useRepositoryLayout=true
-Dmdep.copyPom=true -Dmdep.addParentPoms=true
dependencies.pom being:
<project>
<modelVersion>4.0.0</modelVersion>
<description>Dependencies</description>
<groupId>com.dummy</groupId>
<artifactId>dummy</artifactId>
<version>1.0.0</version>
<dependencies>
<dependency>
<groupId>com.dependency1</groupId>
<artifactId>dep1</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>com.dependency2</groupId>
<artifactId>dep2</artifactId>
<version>1.0.0</version>
</dependency>
</dependencies>
</project>
When doing this, I notice that parents declared in the dependencies' poms are not copied from the .m2 Local maven repository to the destination directory.
Perhaps I'm missing something and there's a better way to do this, since it's kind of a hack to use a pom file to declare the artifacts I want to copy (together with their dependencies).
Turns out that maven was using version 2.8 as default for the dependency plugin. When explicitly indicating it to use the latest version (2.10), it worked just fine.
The addParentPoms parameter was already introduced on 2.8 for copy-dependencies, so I guess it must be a bug in the 2.8 release.
mvn org.apache.maven.plugins:maven-dependency-plugin:2.10:copy-dependencies

Maven: Including external JAR in POM

I know questions like this have been asked and answered before, and have taken the time to read those threads, but somehow they won't help me. I have locally added own Java code to the DSpace software, and my code depends on another library, so I'll have to include this library (JAR) into the <dependencies> section of the POM, right? This is what my entry looks like:
<dependency>
<groupId>de.mannheim.ids</groupId>
<artifactId>pid</artifactId>
<version>0.0.1-SNAPSHOT</version>
<systemPath>/absolute/path/to/jar/pid-client-0.0.1-SNAPSHOT.jar</systemPath>
<scope>system</scope>
</dependency>
This does not help but results in a Compile Error - the relevant classes cannot be found. I have also tried mvn install:install-file -Dfile..., which tells me everything is fine with the jar (BUILD SUCCESS), but the subsequent mvn package fails with the usual error.
What could I be doing wrong?
You should be able to use the artifact as usual after using mvn install:install.
If that command returned Build successful you should be able to use this in your pom:
<dependency>
<groupId>de.mannheim.ids</groupId>
<artifactId>pid</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
Maven will find it like it would find any jar in your local repository.

Resources