Maven noob here. We have tens of repos with POMs that build rpms and recently moved our CI system to CentOS 6. As a result rpm builds are no longer backwards compatible with RHEL 5. Apparently I need to state within the spec or the maven POM:
<defineStatement>_source_filedigest_algorithm md5</defineStatement>
<defineStatement>_binary_filedigest_algorithm md5</defineStatement>
Can I add this to our parent POM somehow? Otherwise I need to script the addition of these lines to a slew of repositories and that gets ugly and error-prone quickly.
Is that a property? Is that a configuration item for a plugin? If it's a property, you could define a active profile on each machine building it. This would be appropriate if it changes based on the machine running the build. The profile would be defined in the conf/settings.xml of the maven installation, and defined as active if it should be applied to all builds. However, not knowing where that value is meant to be sent, I can only guess right now.
Related
I have a set of applications, all use Maven and the local repository. The applications form a dependency tree using <dependency> in their pom.xml. All of these projects have -SNAPSHOT in their version.
Is it possible for Maven (or some compatible dependency manager) to build an application together with all of its local dependencies whose source changed?
I do not want to create a multi-module project, because:
the projects are exactly libraries, not modules;
I do not want an additional complexity just to have a form of build which is already precisely defined;
I want the process to be dynamic: if a library is mature enough to be put into a remote repository, it would be no more rebuilt with the main project and that's ok.
For now, there is a lot of refactoring, moving code from one library to another etc. and it happens often that substantial parts of the dependency tree need to be rebuilt. I thus need to manually write mvn install in several projects in order to assure that there is no stale code.
No, it doesn't work. Even with a multi-module project, maven does not detect which modules have changed sources in it and which do not.
There was a (flaky) implementation in Maven 2, but it was not continued in 3.x, see How to get maven 3.0 to only build modules with local scm changes
I hoped they would include it again in maven 4, but I didn't see it yet: https://maarten.mulders.it/2020/11/whats-new-in-maven-4/
I once did a similar setup, but had to use shell scripts with some git magic to get it working.
You can also decide to put your libraries in separate repo's from the start, and use the repo tool that google uses for android development: https://github.com/GerritCodeReview/git-repo/blob/main/README.md
Once you run mvn install on the particular Maven project, it will be accessible for all other Maven projects, which are on the same workstation, during dependency collection (before the compile phase).
Official Maven Build Lifecycle description:
install - install the package into the local repository, for use as a dependency in other projects locally
It's not necessary to keep libraries as part of the same project(or have it as a multi-module project). But once you want to share those libraries with your teammates, you would need either to force them installing libraries locally (as you did), or store those libraries at some external repo, like Artifactory or Nexus
Is there a way to determine that a new SNAPSHOT version of a dependency is available in a Nexus? Ideally it would be a command line that would take as input a group/artifact/version and would return true if it is the case.
Say I have mygroup.b1:1.0.0-SNAPSHOT which has as dependency mygroup.a1:1.0.0-SNAPSHOT, both version controlled and managed by my continuous deployment environment.
I would like for a build of mygroup.b1:1.0.0-SNAPSHOT to be triggered when a new SNAPSHOT of mygroup.a1 has been deployed in Nexus.
In my case, the continuous environment is GOCD Thoughtworks and I would like to trigger a pipeline based on the dependencies defined in a pom.xml file (that of mygroup.b1), some of which are SNAPSHOTs (such as mygroup.a1) that other pipelines could deploy.
I have looked at https://github.com/jenkinsci/maven-dependency-update-trigger-plugin which seems to do this by checking a dependency is downloaded when it is resolved. Since this is a Jenkins plugin it is not exactly what I am looking for.
I have also looked at https://github.com/aresok/go-maven-poller but the process seems tedious: is it required to define all dependent packages so that GOCD can poll Nexus?
Another thing is the usage of the versions:display-dependency-updates goal of the versions-maven-plugin http://www.mojohaus.org/versions-maven-plugin/display-dependency-updates-mojo.html but it does not seem to allow filtering on gav.
EDIT: added an example
EDIT: added information on the usage of the maven-versions-plugin
I've spent years trying to deploy libraries that use native code to Maven Central. I've run into the following problems:
There weren't any good plugins for building native code using Maven. native-maven-plugin was a very rigid build system that, among other things, made it difficult to debug the resulting binaries. You'd have to manually synchronize the native-maven-plugin build system with the native IDE you use for debugging.
Maven did not replace variables in deployed pom.xml files: MNG-2971, MNG-4223. This meant that libraries had to declare platform-specific dependencies once per Maven profile (as opposed to declaring the dependency once and setting a different classifier per profile); otherwise, anyone who depended on your library had to re-define those same properties in their project file in order to resolve transitive dependencies. See Maven: Using inherited property in dependency classifier causes build failure.
Jenkins had abysmal support for running similar logic across different platforms (e.g. "shell" vs "batch" tasks, and coordinating a build across multiple machines)
Running Windows, Linux and Mac in virtual machines was way too slow and fragile. Even if you got it working, attempting to configure the VMs as Jenkins slaves was a lesson in frustration (you'd get frequent intermittent build errors).
Maven Central requires a main jar for artifacts that are platform-specific: OSSRH-975
Sonatype OSS Repository Hosting and maven-release-plugin assumed that it would be possible to release a project in an atomic manner from a single machine but I need to build the OS-specific bits on separate machines.
I'm going to use this Stackoverflow question to document how I've managed to overcome these limitations.
Here is how I overcame the aforementioned problems:
I used CMake for building native code. The beauty of this system is that it generates project files for your favorite (native) IDE. You use the same project files to compile and debug the code. You no longer need to synchronize the two systems manually.
Maven didn't support CMake, so I built my own plugin: https://github.com/cmake-maven-project/cmake-maven-project
I manually hard-coded platform-specific dependencies into each Maven profile, instead of defining the dependency once with a different classifier per profile. This was more work, but it doesn't look like they will be fixing this bug in Maven anytime soon.
I plan to investigate http://www.mojohaus.org/flatten-maven-plugin/ and https://github.com/mjiderhamn/promote-maven-plugin as alternatives in the near future.
Jenkins pipeline does a good job of orchestrating a build across multiple machines.
Running Jenkins slaves on virtual machines is still very error-prone but I've managed to workaround most of the problems. I've uploaded my VMWare configuration steps and Jenkins job configuration to help others get started.
I now create an empty JAR file for platform-specific artifacts in order to suppress the Sonatype error. This was actually recommended by Sonatype's support staff.
It turns out that maven-release-plugin delegates to other plugins under the hood. Instead of invoking it, I do the following:
Use mvn versions:set to change the version number from SNAPSHOT to a release and back.
Tag and commit the release myself.
Use nexus-staging:rc-open, nexus-staging:deploy -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}, and nexus-staging:rc-close to upload artifacts from different platforms into the same repository. This is called a Staging Workflow (referenced below).
Upon review, release the repository to Maven Central.
Important: do not enable <autoReleaseAfterClose> in the nexus-staging plugin because it closes the staging repository after each deploy instead of waiting for all deploys to complete.
Per https://issues.sonatype.org/browse/NEXUS-18753 it isn't possible to release SNAPSHOT artifacts atomically (there is no workaround). When releasing SNAPSHOTs, you need to skip rc-open, rc-close and invoke nexus-staging:deploy without -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}. Each artifact will be uploaded into a separate repository.
See my Requirements API for a real-life example that works.
Other quirks to watch out for:
skipNexusStagingDeployMojo must be false in last reactor module (otherwise no artifacts will be deployed): https://issues.sonatype.org/browse/NEXUS-12365. The best workaround is to use Maven profiles to omit whatever modules you want when deploying (don't use skipNexusStagingDeployMojo at all)
skipLocalStaging prevents deploying multiple artifacts into the same repository: https://issues.sonatype.org/browse/NEXUS-12351
I'm trying to set up Jenkins so that when it builds a Maven Job, it should set the build name of that build to include, somehow, the Maven version of that project. If I can't get it to work for all builds, I'd like it to at least work on a maven release.
I've looked into both the Build Name Setter plugin (which is fairly underpowered IMO) and the M2 Release Plugin (which we already use).
I see that the Build Name Setter plugin has access to Environment variables, and I was considering checking whether or not the M2 Release Plugin set any.
I also had a look at this question on SO:
Getting Maven Version in Jenkins
This is some interesting information...Maybe I could modify the Build Name Setter plugin to use some of the code listed there?
Do you guys have any ideas?
I ended up adding code to the M2 Release Plugin (Open Source FTW) to offer the option of appending the version number to the build name on successful builds. I made a Pull request on GitHub a week ago and I haven't heard back (which is fine, it's totally up to them), but I figured I'd mention my solution for anybody that might stumble across this page in the future.
I have an artifact that should be built for several target platforms:
Linux x86
Windows x86
ARM11
Unfortunately due to the lack of crosscompilers, it is not possible to create all versions of the artifact in one go.
Using other words, the goal is to have in the repository something like this
artifact-1.0.0-linux.zip
artifact-1.0.0-windows.zip
artifact-1.0.0-arm11.zip
artifact-1.0.1-linux.zip
artifact-1.0.1-windows.zip
artifact-1.0.1-arm11.zip
...
Note that the versions are in sync. How to accomplish this?
The thing is that the release process upgrades version of the pom.xml after every build. So by building consecutively on various platforms I can achieve having
artifact-1.0.0-linux.zip
artifact-1.0.1-windows.zip
artifact-1.0.2-arm11.zip
artifact-1.0.3-linux.zip
artifact-1.0.4-windows.zip
artifact-1.0.5-arm11.zip
...
but this is not what I am looking for.
I could
run on Linux
mvn release:prepare release:perform -DpushChanges=false
(with pushChanges set to false release won't increase version number in SCM)
and then run on Windows
mvn release:prepare release:perform
(this will increase the version number)
But then the responsibility to trigger the release processes on various platforms in the proper order lies with me. Is there a way maven can help me with this?
Do you have any suggestions?
Thanks
PS. Note that this is not a question about how to organize into modules. It is about how to synchronize release processes of a single artifact on multiple platforms.
Have you found a solution for this yet?
It's good to know I'm not the only one fighting with Maven :-)
Anyway,
Are you allowed to deploy a released version to nexus?
I'm thinking, you could do this:
1 - Do a "mvn release:prepare release:perform" from a windows machine - that should get artifact-1.0.1-windows.zip into nexus.
2 - Checkout the artifact-1.0.1 tag from source control
3 - Do a "mvn deploy" from linux and arm11 (whatever that is :P) - that should get -linux.zip and -arm11.zip into nexus as well.
Though, I believe that depending on how nexus is configured it won't let you redeploy anything with the same GAV (even if the classifier is different)
As I see, you use classifiers (artifact's file name suffix) like linux, windows or arm11 to distinguish various artifact's releases, intended for specific platforms. So, if you create multi-module project managed by Maven, where modules would be artifacts with same groupId, same artifactId, same version (probably inherited from their common parent), but different classifier, you'll get exactly what you want. In such case, you always release your multi-module POM (usually it is common parent for its modules as well) to have all modules released at once. Assuming same-version-for-all-modules-policy (which seems to fit pretty well here), you can basically execute:
mvn release:prepare release:perform -DautoVersionSubmodules
and that's it. You will get artifact-1.0.0-linux.zip, artifact-1.0.0-windows.zip, artifact-1.0.0-arm11.zip artifacts released. Next development version will be set to 1.0.1-SNAPSHOT for all modules (via inheritance from parent).
You helped me out with your question...I didn't know about the pushChanges=false option.
Using Jenkins you can set up a job on each platform that performs the maven release in series using the "Trigger builds remotely (e.g., from scripts)" feature. If you desire, you can use a Parameterized Build to choose which version to build (If you use that, then you need the Parameterized Trigger Plugin to trigger the build on the other platforms).