Our maven projects all build on a jenkins and then deploy the artifacts to a maven repository.
For some reason however there seems to be some kind of cache or update lag when building a project that has dependencies to other projects that just got newly build.
Say there's a bug in artifact A and B depends on A. Now I fix the bug locally, update my dependencies in B, run B and everything's fine. I check in obviously. Now I build A on Jenkins and afterwards B. I then get the newly built B and run it. The bug is still there. Hours later if I build B again it will kindly get the new A and the bug is gone..
So there must be a way to force jenkins to use the new A on its server. using mvn -u or mvn --update-snapshots will not help. It will check for updates and find none..
I believe there is no such caching behavior possible in Maven/Jenkins, as all your snapshot is built locally in the same instance of Jenkins. There is no need for the "update snapshot" option because you are not getting it from remote repo. However, I would recommend you to take a look in the repository setting in Jenkins. In Jenkins, we can change which maven repository to use (by default, it is shared by whole machine, but it can be changed to per-workspace or per-executor etc. (Under your project configuration, in Build -> Advanced, you can check for "Use private Maven repository", and in Manage Jenkins -> Configure System -> Maven Project Configuration, you can select the way to manage Local Maven Repository. I am suspecting you have changed it to "Local to executor"
If you have any setting that cause the build of A installed in a different repository that B use, you may hit problem you faced.
Related
We have a Jenkins build which in it's initial stage tries to determine the latest deployed version of some dependencies via version ranges, which are also built by that same Jenkins instance. Each build is for several artifacts, all with a shared version, so we want to select the latest version which has been completed. We do this by using a special pom which has dependencies on specific poms - these are deployed last by the other builds.
The issue is that if one of the other builds is running, but has not yet deployed the pom, maven's version range resolution winds up selecting the version for the build in progress and then fails as some of the dependencies haven't been built yet.
This isn't a downstream build kicked off by the dependency. In any case, we've tried the "Enable isolated resolution for downstream builds" and "Resolve artifacts from Artifactory" but neither have helped.
If the project is a normal UI "maven project" then there is a Use private Maven repository checkbox that uses the workspace specific maven repository cache
Normally, Jenkins uses the local Maven repository as determined by Maven — the exact process seems to be undocumented, but it's ~/.m2/repository and can be overridden by in ~/.m2/settings.xml (see the reference for more details.)
This normally means that all the jobs that are executed on the same node shares a single Maven repository. The upside of this is that you can save the disk space, but the downside of this is that sometimes those builds could interfere with each other. For example, you might end up having builds incorrectly succeed, just because your have all the dependencies in your local repository, despite that fact that none of the repositories in POM might have them.
There are also some reported problems regarding having concurrent
Maven processes trying to use the same local repository.
When this option is checked, Jenkins will tell Maven to use
$WORKSPACE/.repository as the local Maven repository. This means each
job will get its own isolated Maven repository just for itself. It
fixes the above problems, at the expense of additional disk space
consumption.
When using this option, consider setting up a Maven artifact manager
so that you don't have to hit remote Maven repositories too often.
If you'd prefer to activate this mode in all the Maven jobs executed
on Jenkins, refer to the technique described here.
If it's a pipeline job they a mavenLocalRepo setting see: https://plugins.jenkins.io/pipeline-maven/
If it was a freestyle job, your only choice to do what that option does by hand
By using this option, Jenkins will tell Maven to use a custom path for the build as the local Maven repository by using -Dmaven.repo.local
If specified as a relative path then this value will be resolved against the workspace root and not the current working directory.
ie. $WORKSPACE/.repository if .repository value is specified.
The issue is due to the Jenkins maven local repository - it looks like in the current configuration this is shared between the builds. Thus, the in-flight build manages to pick up installed poms that haven't yet been deployed.
We solved this by adding -Dmaven.local.repo=${WORKSPACE}/.m2/repository for when we resolve the version ranges - this way the build is isolated from the other concurrent builds for this stage.
I have created pipelines which will clean, compile, test, & deploy the code every time we update it. But in case of a technical issue/or error I want to add a step which allows me to revert/rollback the "build" or deployment.
Is there a maven command that can be used to revert/rollback builds/deployments?
No.
If you just run clean package, then another clean will erase the results of package, but if you have run install or deploy the artifact is send to the local repository or remote repository. This cannot be "rolled back" by Maven.
The Maven Release Plugin has a rollback goal that rolls back the changes that it made to version control:
version changes in POMs are reverted
created tag is deleted
created branch is deleted
However, this rollback does not delete files that were uploaded to the remote repository! If the remote repository does not allow artifacts with final versions to get deleted or modified, then you would end up with a problem, because your next attempt would fail to re-upload.
The Maven Release Plugin internally uses the Maven Deploy Plugin to upload files, and the latter has an experimental deployAtEnd feature. Unfortunately, I did not manage to get it to work for my build (using the suggestion from this SO comment) - I still had a build in which the Maven Release Plugin uploaded files before exiting with a failure.
I got a maven project (myApp) depending on another maven project in snapshot version.
like:
<dependency>
<groupId>org.group.dep</groupId>
<artifactId>arty</artifactId>
<version>12.1.4-SNAPSHOT</version>
</dependency>
But I got a problem with this after the "arty" got an update without changing the version (I know that would be the cleanest solution).
I build the myApp local and got still the old version of the "arty" dependency.
I verified tow option working for me (and a college):
1) Manual cleaning of the local repository: navigating to my .m2/repo/org/group/dep/arty and deleted all folders inside. After rebuilding the myApp local it was working fine - arty was downloaded form the artifactory.company.com again with the updated content.
2) Local building of the arty package so it got updated in the local repository. After rebuilding the myApp local it was working fine.
But I got similar problem on the Jenkins:
I got a Jenkins job just building org.group.myApp without building before org.group.dep.arty. It failed for the missing changes form "arty".
What can I do now to solve my problem there?
I can not rely on first building org.group.dep.arty as I can not be sure for Jenkins to run both jobs on the same host using same local repository (I don't want to change that).
Somehow the myApp-Jobs was failing after I manually cleared on that Jenkins node the org.group.dep.arty in the repository and running than the myApp-job (was somehow not downloading the package).
I finally found the mvn -u but as I tried this I was as well disappointed.
I tried different maven versions on that jenkins and got the same result.
Is there no way to force the update of the snapshot versions?
Is this "another project" is a part of the same multi-module project?
If so you can build your project with --also-make options so that maven will effectively rebuild your module and all of its dependencies
If its an entirely different project, use mvn -U to forcefully download all the snapshot dependencies of your project.
If there is a particular issue with one concrete dependency consider using mvn dependency:get. This get goal of maven-dependency-plugin downloads one specific artifact from the remote repository
Here is a link to the plugin documentation
The simplest solution to redownload -SNAPSHOT is by using the command line option: -U or as long option --update-snapshots
Furthermore your project sounds like the need for a multi module build which prevents such issues. Or you might need to define those Jobs depending on each other (There is an option to build if a SNAPSHOT has been updated in Jenkins).
I have a maven multi module project with several modules. I want to deploy them (mvn deploy) only if they all pass a full mvn install (which includes the tests).
Currently, I run a mvn install on the project. If all modules pass, I run mvn deploy to do the deployment. The problem I see is the waste of time calling mvn twice (even if I skip tests on the second run).
Does anyone have an idea on this?
EDIT: I have learned that using Artifactory as a repository manager and the maven-artifactory-plugin with your maven setup will add the atomic deploy behaviour to the mvn deploy command. See the Build Integration section in the Artifactory documentation.
[DISCLOSURE - I'm associated with JFrog. Artifactory creator.]
Take a look at the deployAtEnd parameter of Maven Deployment plugin: http://maven.apache.org/plugins/maven-deploy-plugin/deploy-mojo.html
This is a bit tricky. Maven is not atomic when it executes the build life-cycle. So a broken set of artifacts may end up in a repository.
One solution I know is Nexus Pro: http://www.sonatype.com/Products/Nexus-Professional/Features - it allows you to promote builds or define certain repos as staging. So only verified versions get promoted to be used. Maybe artifactory has something similar - I just don't know.
If that solution is too expensive you probably need to create a cleanup build or profile to remove artifacts that where already uploaded. My first guess would be to write a Maven plugin to use the the proxy remote API or maybe the maven features are already sufficient. But since deploy means update the meta-data xml files too I dont think there is a delete - not sure on this either.
We have three artifacts:
common.jar : with common classes.
public.war : depending on the common.jar, contains only public site resources.
internal.war : depends on both common.jar and public.war, adding authentication
information and security context resource files. Also contains
few administration site classes.
Currently I have structured these in such way, that internal.war overlays itself with public.war.
Building the project locally, installing the artifacts to local repo, works perfectly.
Problems start when trying to get the Hudson builds working with following sequence:
Build all projects in dependency order.
Modify common.jar (say, add a new class method)
Modify internal.war classes in such way that they are compile-time dependent on changes done in 2. step.
Commit both changes, triggering the Hudson builds.
Internal.war build fails because it can not find the symbols added in step 2.
Somehow the build in step 5. is using an old version of the common.jar, and failing because of it.
The common.jar version number does not change, let's say it's 1.0.0-SNAPSHOT for the purposes of this example.
If I DO change the common.jar version number, the build works. (Supposedly because there is only one release by a release version number).
Now, what could cause this using of old artifacts in Hudson builds?
We are running maven builds on Hudson with command "clean package -e -X -U"
"Deploy artifacts to maven repository" has been checked.
It's hard to definitively answer this without access to the real poms, but here is what I would do:
1) Make sure Hudson is using the exact same version of Maven as you are on your local machine
2) Examine the effective pom.xml of internal.war on the Hudson machine in a terminal via mvn help:effective-pom making sure you are running the same mvn executable as your Hudson job does. You need to verify the version of the common.jar in the effective pom.xml of internal.war. It could be different than what you expect due to profiles or settings.xml differences.
3) Check the settings.xml file for your Hudson install of Maven. In particular you need to verify all is well in your distributionManagement, servers, and repositories stanzas. Another good way to check this is to go to your internal.war project and run mvn help:effective-settings and see if what is there matches what is on your local machine.
Something is awry and it won't take long to find with the right analysis.