Maven update versions goal builds old version in Jenkins - maven

I've got a Jenkins/maven build where we need to be able to produce release artifacts with arbitrary version numbers. The version number is passed in as a parameter to the build job. I'm using versions:set to update the version from what is in the pom.xml to the $VERSION parameter. When the build is run, it appears that maven does in fact update the version in the working copy of the pom.xml, however it builds the artifact with the old version number.
pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<name>someservice</name>
<groupId>com.domain.group</groupId>
<artifactId>someservice</artifactId>
<version>1.0.0-SNAPSHOT</version>
Invoke top-level maven targets in Jenkins job:
-V
-B
-e
-U
clean
versions:set
-DnewVersion=${VERSION}
package
deploy
-Dmaven.test.skip=true
My build output looks something like this (with a $VERSION of 0.1.2)
[INFO] ------------------------------------------------------------------------
[INFO] Building someservice 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- versions-maven-plugin:2.2:set (default-cli) # someservice ---
[INFO] Searching for local aggregator root...
[INFO] Local aggregation root: /svn/group/someservice/trunk
[INFO] Processing change of com.domain.group:someservice:1.0.0-SNAPSHOT -> 0.1.2
[INFO] Processing com.domain.group:someservice
[INFO] Updating project com.domain.group:someservice
[INFO] from version 1.0.0-SNAPSHOT to 0.1.2
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.147s
[INFO] Finished at: Wed May 20 11:40:46 PDT 2015
[INFO] Final Memory: 10M/180M
[INFO] ------------------------------------------------------------------------
I've noticed that while the initial run will build with the old version, if the working dir is not reverted, subsequent runs will correctly build with the new version in the updated pom.xml. How can I get the pom file updated with the new version before the artifacts are built?
Constraints:
While it may be possible for us to have changes made to the pom.xml, it's not directly under our control and is a less practical solution. In an ideal world the pom would already be updated with the correct version and we wouldn't be doing this at all.
I know I can just add a shell step prior to the maven build and do a sed replace on pom.xml or something similar. That does work, but it's ugly and won't scale well with more complex poms. A native maven solution is greatly preferred.
I've tested this independently of jenkins, (i.e. mvn versions:set -DnewVersion=0.1.2) and the behavior is the same

What you want to achieve is not possible with current maven versions, as stated in the Usage section of the versions-maven-plugin:
Maven 2.0, 2.1, 2.2 and 3.0 do not currently support re-reading modifications of the pom.xml within one invocation of Maven.
How about adding another build step here?

Related

Tycho build fails on Jenkins only for SCM Trigger

I have a little bit of strange Problem with Jenkins, Maven and Tycho and it is hard to find out who is the culprit.
All SCM Triggered Builds fail but all manually triggered builds succeed.
Jenkins Version : Jenkins ver. 1.527
Maven Version : 3.05
I have a modularized tycho build:
<modules>
<module>../main.plugin.test</module>
<module>../main.plugin.internationalization.at</module>
<module>../crud.plugin</module>
<module>../rest.plugin</module>
<module>../main.plugin</module>
<module>../main.feature</module>
<module>../product</module>
<module>../target-definition</module>
<module>../rest.plugin.test</module>
</modules>
Jenkins is configured as a simple maven build with modules.
It just executes:
-X clean deploy
When an SCM-Build is triggered some modules are not build.
When I build it manually everything is fine:
This behaviour is consistent. I already tried
switching to a different Maven version (3.05 / 3.04)
deleting the whole workspace prior to building
clean checkout of all sources
running of -X deploy (without clean)
Any amount of manual invocations succeeds. And any amount of scm triggers fails.
Both Maven logs look exactly the same until (working):
[INFO] Reactor Build Order:
[INFO]
[INFO] client-master
[INFO] crud-plugin
[INFO] main-plugin
[INFO] ------------.main.plugin.test
[INFO] ------------.main.plugin.internationalization.at
[INFO] rest-plugin
[INFO] main-plugin
[INFO] ------------.product
[INFO] target-definition
[INFO] ------------.rest.plugin.test
and (failing):
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] main-plugin
[INFO] ------------.plugin.test
[INFO] ------------.main.plugin.internationalization.at
[INFO] rest-plugin
[INFO] main-plugin
[INFO] ------------.product
[INFO] ------------.rest.plugin.test
The final error message therefore is:
Caused by: java.lang.IllegalStateException: ------------..client:------------..crud.plugin:eclipse-plugin:1.0.0-SNAPSHOT does not provide an artifact with classifier 'null'
Customer specific module names are replaced with ------------. in this question.
I have heard repeatedly of vague problems with Tycho builds on Jenkins. The reason for these problems seems to be that some Jenkins plugin triggering these builds hooks into the Maven lifecycle and this somehow collides with what Tycho does in the Maven internals.
For the problem that you are describing, it seems that the Jenkins plugin that you are using is changing the module build order. This may be okay for a normal Maven build (where all dependencies are declared in the POMs), but may fail for a Maven/Tycho build, where dependencies are computed by Tycho during the build.
To avoid this problem, you should trigger the Maven build in a way that is closer to a normal command line build. I found that the Invoke top-level Maven targets build step from the Maven Integration plugin works without problems.

Different behavior when running the Maven versions:use-latest-versions goal

The command I'm running is:
mvn -U -e versions:use-latest-versions -Dincludes=com.test.engines:engines -DgenerateBackupPoms=false -DallowMajorUpdates=false -DallowMinorUpdates=false -DallowSnapshots=false
I have, in my pom, a version such as 1.2.3-5-SNAPSHOT and I want to update it to the latest released version. This could be, for example, 1.2.3-7, or 1.2.3-5-SNAPSHOT may not have been released yet meaning the latest version is 1.2.3-4.
When running a maven command on my local Windows machine the versions plugin is working as I require it, however when running it on a remote Linux box (where I really need it to) it doesn't.
My local box shows:
When running a maven command on my local Windows machine the versions plugin is working as I require it, however when running it on a remote Linux box (where I really need it to) it doesn't.
[INFO] [versions:use-latest-versions {execution: default-cli}]
[INFO] Incremental version changes allowed
[INFO] Incremental version changes allowed
[INFO] artifact com.test.engines:engines: checking for updates from central
Props: {project.version=12.4.0-10-SNAPSHOT, project.parent.version=1.0.0, project.parent.groupId=com.test.pom, project.artifactId=edge-web, project.groupId=com.test.edge-web, project.parent.artifactId=pom}
[INFO] Updated com.test.engines:engines:jar:12.4.0-41-SNAPSHOT to version 12.4.0-40
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
Whereas the remote box shows:
[INFO] --- versions-maven-plugin:2.0:use-latest-versions (default-cli) # edge-web ---
[INFO] Incremental version changes allowed
[INFO] Incremental version changes allowed
[INFO] artifact com.test.engines:engines: checking for updates from nexus
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
When running:
mvn -Dplugin=versions help:describe
The output on both boxes shows the same version number:
Name: Versions Maven Plugin
Description: Versions Plugin for Maven 2. The Versions Plugin updates the
versions of components in the POM.
Group Id: org.codehaus.mojo
Artifact Id: versions-maven-plugin
Version: 2.0
Goal Prefix: versions
Does anyone know how I can get the same behavior as I do on my local machine? Or have any suggestions as to how I can investigate further?
I was using different versions of Maven. It seems even though the plugin version was the same I guess the method to resolve later versions in the core of maven must be different.
For reference the Mavan version can be checked using:
mvn --version
I also had to do similar to what is described here to allow me to run both Maven 2 and 3 from a single script.

maven deploy-file failed with 503: cannot find maven-metadata.xml

I am using Maven to integrate our project with others, maybe in a unpopular way and encountered issue.
We have a project that used to compile with Ant. Ant script is big and awesome, so when we are using Maven for integration, it is decided to keep compiling with Ant.
Now let's say Ant compile output is res-1.0-SNAPSHOT.tar.gz. (any filename can be possible but it IS tar.gz) And I am deploying the file to a nexus-hosted snapshot repository called "snapshots".
I tried to deploy with this command:
mvn deploy:deploy-file \
-DgroupId="com.my-company" \
-DartifactId="res" \
-Dversion="1.0-SNAPSHOT" \
-Dpackaging="tar.gz" \
-Dfile="res-1.0-SNAPSHOT.tar.gz" \
-Durl="http://our-nexus-ip/nexus/content/repositories/snapshots" \
-DrepositoryId="snapshots"
I have a simple settings.xml in ~/.m2 with proxy and server settings. However server settings is not being used in current progress yet, wrong passwords don't get errors.
The output is like this:
[[root#cnbi maven]# ./run.sh
+ mvn deploy:deploy-file -DgroupId=com.my-company -DartifactId=res -Dversion=1.0-SNAPSHOT -Dpackaging=tar.gz -Dfile=res-1.0-SNAPSHOT.tar.gz -Durl=http://135.252.234.142:8081/nexus/content/repositories/snapshots -DrepositoryId=snapshots
Warning: JAVA_HOME environment variable is not set.
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'deploy'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Default Project
[INFO] task-segment: [deploy:deploy-file] (aggregator-style)
[INFO] ------------------------------------------------------------------------
[INFO] [deploy:deploy-file]
[INFO] Retrieving previous build number from snapshots
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error retrieving previous build number for artifact 'com.my-company:res:tar.gz': repository metadata for: 'snapshot com.my-company:res:1.0-SNAPSHOT' could not be retrieved from repository: snapshots due to an error: Error transferring file
Server returned HTTP response code: 503 for URL: http://135.252.234.142:8081/nexus/content/repositories/snapshots/com/my-company/res/1.0-SNAPSHOT/maven-metadata.xml
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51 seconds
[INFO] Finished at: Tue Jun 12 08:44:13 CST 2012
[INFO] Final Memory: 7M/209M
[INFO] ------------------------------------------------------------------------
You see, it fails to find maven-metadata.xml. It is for certain, the file and its folder "com/my-company" do not exist at all in the repository.
Besides any misuse of Maven you may find, my questions are:
1) am I using Maven in the right way? (deploying tar.gz, using deploy:deploy-file...)
2) are there incorrect parameters?
3) what is maven-metadata.xml for? It is not there but Maven insists to find it -- I guess it is generated, am I missing some steps?
The solution might be stupid, I am really not familiar with Maven. Unfortunately it has to be done... Please, help me out of this.
Maven version is 2.0.11. Let me know if you want to know more.
I had the same problem and there was a bug in our nexus.
Using maven3 (with same settings.xml, pom.xml and .m2 repo) solved our problem and its easier than upgrading nexus.
You can check this bug also.
If you're still stuck with using Ant, I would recommend you at least consider adding Ivy to the picture, as your dependency manager. If I recall correctly, it was able to update maven-metadata.xml files in the repository.
Have a look at this example.

Transitively download a Maven artifact to the local repository

I am trying to download a specific artifact (and all of its dependencies) to a machine's local repository.
It would seem that using the dependency:get goal would be the best option for this, but despite the documentation it does not seem to actually get the transitive dependencies.
Here is an example where I have tried to use dependency:get to download the spring-core jar and all of its many dependencies. You'll notice that the spring-core jar is the only thing downloaded despite the fact that this was done after cleaning the local repository.
$ mvn org.apache.maven.plugins:maven-dependency-plugin:2.2:get -DrepoUrl=http://repo1.maven.org/maven2/ -Dartifact=org.springframework:spring-core:3.0.5.RELEASE -Dtransitive=true
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.2:get (default-cli) # standalone-pom ---
Downloading: http://repo1.maven.org/maven2/org/springframework/spring-core/3.0.5.RELEASE/spring-core-3.0.5.RELEASE.jar
Downloaded: http://repo1.maven.org/maven2/org/springframework/spring-core/3.0.5.RELEASE/spring-core-3.0.5.RELEASE.jar (374 KB at 548.4 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.401s
[INFO] Finished at: Wed May 25 00:29:47 CDT 2011
[INFO] Final Memory: 7M/107M
[INFO] ------------------------------------------------------------------------
My questions are:
Is this a bug with the dependency:get goal?
If not, what am I doing wrong?
Are there any alternatives methods I could use to accomplish my initially stated goal?
If this is a one time or irregular occurrence for you, The simplest thing to do would be to define the dependency in a POM and run mvn package or similar to retrieve the dependency artifacts. You could also try mvn dependency:sources if you'd like to have the source jars too.
If this is something you want to do more regularly or as part of a process, you could look at using Aether directly to retrieve the dependencies for you.
Another approach if this is something you need to do regularly to manage groups of artifacts into your internal development ecosystem is to use Nexus' procurement suite to retrieve the dependencies and manage them into your repository.
You might can go with this solution
1) Download the artifact as you described (I tested with version 2.5.2)
c:\test>mvn -DrepoUrl=http://repo1.maven.org/maven2/ -Dartifact=org.springframework:spring-core:2.5.2 -Dtransitive=true
2) Download the pom (-Dpackaging=pom) of this artifact
c:\test>mvn -DrepoUrl=http://repo1.maven.org/maven2/ -Dartifact=org.springframework:spring-core:2.5.2 -Dtransitive=true -Dpackaging=pom org.apache.maven.plugins:maven-dependency-plugin:2.2:get
3) Use the downloaded pom to copy all dependencies via the dependency:copy-dependency gaol
c:\test>mvn -DoutputDirectory=C:/test/dependency -f C:/<path-to-repository>/org/springframework/spring-core/2.5.2/spring-core-2.5.2.pom dependency:copy-dependencies
You will find the dependencies (including test and optional scope!) in the created c:\test\dependency folder. To exclude test and optional scope use -DincludeScope=runtime.
You need to dynamically build some path information (e.g. path to the pom in your repository) to set up this solution and also need to bring the artifact itself together with its dependencies but it should work in a script without generating a special pom (which might be easier).
It would appear the answer to question #1 (Is this a bug with the dependency:get goal?) is yes. As of 5/25/2011 issue MDEP-308 is still unresolved.

Maven check for updated dependencies in repository

Is there a Maven plugin that allows you to check if there are newer versions of dependencies available in the repository?
Say, you are using dependency X with version 1.2. Now a new version of X is released with version 1.3. I'd like to know, based on the dependencies used in my project, which dependencies have newer versions available.
The Maven Versions plugin and it's display-dependency-updates mojo are what you're looking for:
mvn versions:display-dependency-updates
Here is what the output looks like:
[INFO] ------------------------------------------------------------------------
[INFO] Building Build Helper Maven Plugin
[INFO] task-segment: [versions:display-dependency-updates]
[INFO] ------------------------------------------------------------------------
[INFO] [versions:display-dependency-updates]
[INFO]
[INFO] The following dependency updates are available:
[INFO] org.apache.maven:maven-artifact ........................ 2.0 -> 2.0.9
[INFO] org.apache.maven:maven-plugin-api ...................... 2.0 -> 2.0.9
[INFO] org.apache.maven:maven-project ....................... 2.0.2 -> 2.0.9
[INFO] org.codehaus.plexus:plexus-utils ....................... 1.1 -> 1.5.6
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17 seconds
[INFO] Finished at: Fri Aug 15 10:46:03 IST 2008
[INFO] Final Memory: 10M/167M
[INFO] ------------------------------------------------------------------------
If you want to receive email notifications when newer artifacts versions are available on Maven Central you can create an account on artifact-listener and choose which artifact you want to follow.
You can either search manually for artifacts or directly upload your pom.xml.
You will periodically received notifications like this one (available in english and french for now) :
In projects with a large number of dependancies, you sometimes keep your versions in a properties section.
<properties>
<assertj.version>3.15.0</assertj.version>
<aws-sdk.version>1.11.763</aws-sdk.version>
<cxf.version>3.3.6</cxf.version>
In the case where you are only interested in updates to those versions, you can use the following command
mvn versions:display-property-updates
This gives a more condensed view and only returns the versions you need to update in the properties section.
The VersionEye Maven Plugin is doing the same: versioneye_maven_plugin.
VersionEye can notify you about new versions on Maven Repositories, too. It is a language agnostic tool and beside Java it supports 7 other languages. Beside the simple follow/notify feature it can also directly monitor GitHub and BitBucket repositories and notify your about out-dated dependencies in your projects.
There is also a REST JSON API, for tool integrations.
By the way, I'm the dude who started this project. Let me know if you have questions.
The ideal way to do it is to set dependency versions as properties in pom.xml and then running the below command to get the updated versions for your specific/custom dependencies.
<properties>
<java.version>1.8</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<skip.tests>true</skip.tests>
<spring-cloud-gcp.version>1.2.3.RELEASE</spring-cloud-gcp.version>
<spring-cloud.version>Hoxton.SR6</spring-cloud.version>
<spring-cloud-stream-schema.version>2.2.1.RELEASE</spring-cloud-stream-schema.version>
<confluent.version>5.5.1</confluent.version>
<avro.version>1.10.0</avro.version>
<janino.version>3.1.2</janino.version>
<swagger.version>2.9.2</swagger.version>
<google-cloud-logging-logback.version>0.118.1-alpha</google-cloud-logging-logback.version>
<spring-cloud-stream-binder-kafka.version>3.0.6.RELEASE</spring-cloud-stream-binder-kafka.version>
</properties>
mvn versions:display-property-updates
[INFO] The following version properties are referencing the newest available version:
[INFO] ${avro.version} .............................................. 1.10.0
[INFO] ${spring-cloud-stream-schema.version} ................. 2.2.1.RELEASE
[INFO] ${janino.version} ............................................. 3.1.2
[INFO] The following version property updates are available:
[INFO] ${spring-cloud-gcp.version} .......... 1.2.3.RELEASE -> 1.2.5.RELEASE
[INFO] ${google-cloud-logging-logback.version} 0.118.1-alpha -> 0.118.2-alpha
[INFO] ${spring-cloud-stream-binder-kafka.version} 3.0.6.RELEASE -> 3.0.8.RELEASE
[INFO] ${confluent.version} ................................. 5.5.1 -> 6.0.0
[INFO] ${swagger.version} ................................... 2.9.2 -> 3.0.0
[INFO] ${spring-cloud.version} .................... Hoxton.SR6 -> Hoxton.SR8
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.572 s
[INFO] Finished at: 2020-10-06T09:35:08-07:00
[INFO] ------------------------------------------------------------------------
Another way to achieve this is by executing the command mvn versions:display-dependency-updates but the problem I face with this approach is that it also shows me updates for the nested dependencies which are not too useful for me.
You can use the Versions Maven Plugin[1] to generate reports in your Maven site to get a list of possible updates. With regard to Spring's irregularity, it appears to use the Mercury versioning system[2]. When configuring the Versions plugin, you can add a special rule for Spring stuff:
http://mojo.codehaus.org/versions-maven-plugin/
http://docs.codehaus.org/display/MAVEN/Mercury+Version+Ranges
I might be a bit late to join the party but a more clear way to get more readable html file or a xml file as report which can be taken for further automation using:
mvn versions:dependency-updates-report
This report plugin not just shows more comprehensive details on updates but also has options to update to latest versions. You can find the documentation for it to use various parameters.
You can use Maven Check, a command line tool, which is standalone unlike the Versions Maven Plugin. It also works with Gradle projects.
Output example:
2 build file(s) found, checking for artifact updates
my-gradle-project\build.gradle
[COMPILE ONLY] com.google.guava:guava 31.0-android -> 31.1-android
1 artifact update(s) available
my-maven-project\pom.xml
[DEPENDENCY] org.apache.commons:commons-lang3 3.10 -> 3.12.0
[BUILD PLUGIN] org.apache.maven.plugins:maven-compiler-plugin 3.10.0 -> 3.10.1
2 artifact update(s) available
2/2 build file(s) checked, 3 artifact update(s) available
Disclaimer: I am the author of Maven Check.

Resources