Is it possible to deploy arbitrary zip archive artifacts to Nexus through Maven as snapshots?
We have a build step that is not supported through any application-specific Maven plugin. Instead, our full build and deployment process is as follows:
1) Maven POM compiles the Java component of the build, using Jenkins.
2) Shell script calls create a deployable artifact shell scripts were wrapped around calling a code generation application, which are then zipped up into an archive by the application itself. I need these artifacts deployed to Nexus as both snapshots, and as releases as appropriate.
I tried using the maven-assembly-plugin however this assumes that the plugin itself is creating the zip archive, not simply deploying an archive that was produced by some other method.
I would prefer to do this within Maven since our Nexus settings and credentials are already within the environment and do not need to be passed manually on the command line. Using the Nexus UI for this is not a viable option since this needs to be part of a standard build-deploy-test process, which may happen many times per day, for a couple dozen applications.
For completeness, I'm answering my own question (oh bother...)
I resolved this issue by using the maven-assembly-plugin, which allows you to define arbitrary artifacts, and deploy them (snapshots or releases) to Nexus. The assembly plugin uses a bill of materials (src.xml) that defines the exact contents of the artifact (either including or excluding files, directories, changing file permissions, etc). This can also be used for creating Java uber jars, but it appears that using the Maven Shade Plugin is the preferred method for creating uber jars.
Maven Assembly Plugin main webpage
Related
I'm a bit confused between maven assembly plugin and build-helper-maven plugin.
I've also read in the maven documentation that The assemblies/archive created by the Assembly Plugin gets deployed during the deployed phase.Hence, they can be deployed to the remote nexus repositories.
The purpose of maven assembly plugin is to archive many things into one(say in tar.gz format).
However, the attach-artifact goal present within build-helper-maven plugin has the same role i.e.archiving,installing and deploying the artifact.
With that being said,why would anyone use both of them together? I've seen people using both of them together. Isn't one of the plugins an alternative choice of the other?
Kindly advise.
If you use maven-assembly-plugin you can create as you already mentioned any kind of archives (range from very simple to very complex structures) they will be by default be attached to your project which means they will be deployed into remote repositories in one go if you do mvn deploy with no supplemental configuration.
The build-helper-maven-plugin is intended to add an artifact (one goal of this plugin) which is usually not generated by Maven itself which most of the cases is a smell. If people using them together (in the use case to create an archive and attach it) this makes no sense.
Apart from that the build-helper-maven-plugin can also be used to add other source directories for example for scala, kotlin projects (other goals for example add-source etc.)
So those plugins are not alternatives they have different intentions/use cases.
I've spent years trying to deploy libraries that use native code to Maven Central. I've run into the following problems:
There weren't any good plugins for building native code using Maven. native-maven-plugin was a very rigid build system that, among other things, made it difficult to debug the resulting binaries. You'd have to manually synchronize the native-maven-plugin build system with the native IDE you use for debugging.
Maven did not replace variables in deployed pom.xml files: MNG-2971, MNG-4223. This meant that libraries had to declare platform-specific dependencies once per Maven profile (as opposed to declaring the dependency once and setting a different classifier per profile); otherwise, anyone who depended on your library had to re-define those same properties in their project file in order to resolve transitive dependencies. See Maven: Using inherited property in dependency classifier causes build failure.
Jenkins had abysmal support for running similar logic across different platforms (e.g. "shell" vs "batch" tasks, and coordinating a build across multiple machines)
Running Windows, Linux and Mac in virtual machines was way too slow and fragile. Even if you got it working, attempting to configure the VMs as Jenkins slaves was a lesson in frustration (you'd get frequent intermittent build errors).
Maven Central requires a main jar for artifacts that are platform-specific: OSSRH-975
Sonatype OSS Repository Hosting and maven-release-plugin assumed that it would be possible to release a project in an atomic manner from a single machine but I need to build the OS-specific bits on separate machines.
I'm going to use this Stackoverflow question to document how I've managed to overcome these limitations.
Here is how I overcame the aforementioned problems:
I used CMake for building native code. The beauty of this system is that it generates project files for your favorite (native) IDE. You use the same project files to compile and debug the code. You no longer need to synchronize the two systems manually.
Maven didn't support CMake, so I built my own plugin: https://github.com/cmake-maven-project/cmake-maven-project
I manually hard-coded platform-specific dependencies into each Maven profile, instead of defining the dependency once with a different classifier per profile. This was more work, but it doesn't look like they will be fixing this bug in Maven anytime soon.
I plan to investigate http://www.mojohaus.org/flatten-maven-plugin/ and https://github.com/mjiderhamn/promote-maven-plugin as alternatives in the near future.
Jenkins pipeline does a good job of orchestrating a build across multiple machines.
Running Jenkins slaves on virtual machines is still very error-prone but I've managed to workaround most of the problems. I've uploaded my VMWare configuration steps and Jenkins job configuration to help others get started.
I now create an empty JAR file for platform-specific artifacts in order to suppress the Sonatype error. This was actually recommended by Sonatype's support staff.
It turns out that maven-release-plugin delegates to other plugins under the hood. Instead of invoking it, I do the following:
Use mvn versions:set to change the version number from SNAPSHOT to a release and back.
Tag and commit the release myself.
Use nexus-staging:rc-open, nexus-staging:deploy -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}, and nexus-staging:rc-close to upload artifacts from different platforms into the same repository. This is called a Staging Workflow (referenced below).
Upon review, release the repository to Maven Central.
Important: do not enable <autoReleaseAfterClose> in the nexus-staging plugin because it closes the staging repository after each deploy instead of waiting for all deploys to complete.
Per https://issues.sonatype.org/browse/NEXUS-18753 it isn't possible to release SNAPSHOT artifacts atomically (there is no workaround). When releasing SNAPSHOTs, you need to skip rc-open, rc-close and invoke nexus-staging:deploy without -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}. Each artifact will be uploaded into a separate repository.
See my Requirements API for a real-life example that works.
Other quirks to watch out for:
skipNexusStagingDeployMojo must be false in last reactor module (otherwise no artifacts will be deployed): https://issues.sonatype.org/browse/NEXUS-12365. The best workaround is to use Maven profiles to omit whatever modules you want when deploying (don't use skipNexusStagingDeployMojo at all)
skipLocalStaging prevents deploying multiple artifacts into the same repository: https://issues.sonatype.org/browse/NEXUS-12351
I have a java application.
I can run the Maven Release task which will do some nice things for me:
Change Version number from 1.0.0-SNAPSHOT to 1.0.0
Increment the version number in my pom to 1.0.1-SNAPSHOT
Tag the release in source control
Upload the resulting package to my maven repository
I'd like to take things a step further. I have some post-build steps that I'm currently doing manually.
Update the launch4j configuration xml file with the appropriate version
Wrap the resulting jar in an executable using launch4j
Copy the resulting EXE into a package directory
Copy several supporting files into the package directory
Zip the package directory up
Email the package to my testers.
Eventually I'm going to have the additional task of building an installer leveraging the package directory.
I don't know that maven or ant are the right tools for automating my remaining 6 tasks, but it looks like either one or a combination of both could potentially accomplish what I need.
I could probably write a batch file or a simple perl script to do these things quicker than figuring out how to do them, but I would prefer to keep things as standard as possible so that I'm not taking on the additional responsibility of supporting a hack of a release process perpetually.
It seems to me that these are tasks that might not be standard part of build/release, but are commonly seen enough that there should be a best/most common practice for accomplishing them.
I would suggest to use the maven-assembly-plugin as well as the maven-launch4j-plugin during your build.
Update the launch4j configuration xml file with the appropriate version
put a placeholder into the configuration xml and let maven replace it during the
the build.
Wrap the resulting jar in an executable using launch4j
use the launch4j-maven-plugin to create the executable.
Copy the resulting EXE into a package directory
I would suggest to put the resulting artifact into a repository manager instead
of a separate folder, cause in Maven all artifacts are stored within
a repository. It might be necessary to setup your own repository manager
(Artifactory, Nexus, Archiva).
Copy several supporting files into the package directory
Using them as resources (src/main/resources) they will be copied
automatically.
Zip the package directory up
Use the maven-assembly-plugin to create a resulting zip file.
Email the package to my testers.
You can use a CI like Jenkins etc. to send the final mail or
you can take a look into maven-changes-plugin which might be solution.
This means all your mentioned steps can be handled by Maven during a usual build. This means in the end you can use the maven-release-plugin to produce a full release which contains all the above steps and produces all the wished artifacts.
If I were you, I would try a combination of the following:
Maven release plugin, unless it is not flexible enough for SCM related processes. If using SVN as SCM, I would use directly SVNKit(inside a custom Maven plugin), if flexibility is a concern.
Maven launch4j plugin.
Maven assembly plugin
Maven Ant Run plugin and/or one or more in-house Maven plugins for the remaining tasks.
I have an Ant script for automating a few tasks that are not build related by mainly as updates to the EAR file. These include calling SoapUI exe for some web services, unpacking the EAR file modifying a few classes and repacking it.
Now, for some reason I would like to do the same using maven scripts. From going through Stack Overflow and maven tutorial, I find that this can be replicated using Ant plugins inside maven. Also, I could not find direct substitutes for unpacking the EARs or calling executables in maven. This scripts will not have anything to do with the standard build process that maven is meant for and only caters to some cleanup or update of already deployed EAR.
So how do I go about this? Use Ant plugins inside maven or is there a better maven way to do this?
Thanks,
This may be an incomplete answer but -
Maven is more of a framework and it's going to want to build your application. What it will not do is modify source. It will process source files (.java, .ear, whatever) and put them in a target/ directory. Of course anything is possible (re: hackable) but this is off the rails as far as Maven is concerned if you want in place modification of source - besides, isn't the point of source that it's source, and if you're automating a task that should be part of your build, deploy, startup, etc.?
Maven resource filtering is how to process a source file and stamp dynamic information into your resources.
If you can provide more information regarding how you are processing your .ear files exactly you can possibly put together the pieces in the Maven process resources phase using existing plugins. Worst case you can write your own plugin.
We have a web application in which we are using Ant as a build tool. There is a urgent requirement to create web services (API) and this will be a separate project. For now, to make it available to our customers we have decided to use our web application and remove all unnecessary files (like velocity files, properties, xml etc) and make a jar of it. This jar will be used in our web service project.
In Maven, I want to checkout my web application svn branch, compile it, make a single jar of it and add as a dependency in my project. Is this possible? If yes, then please show me the way.
I'm new to Maven please explain your answers with more detail.
Thanks.
Short Answer
Get your .war deployed into a Maven Repository (local or remote) from your ant build
Child Projects will embed your .war as a dependency, creating a war with your custom services + your original .war file
It is advised that those Child Projects turn into an Archetype, so creating custom services gets easier
Long Answer
From your SCM, you could modify your build.xml file and use Maven Tasks for Ant.
In particular, the install and deploy examples are helpful in order to guide you on deployment your .war into a Maven repository (local or remote)
Then, a .war artifact (when accessible from a Repository) is able to be consumed from other .war applications.
Look into the maven-war-plugin Overlays Feature. In particular, this answer offers you more advice:
combine different maven web-projects into a single project
Other than that, I suggest you could also combine with Maven Archetypes (they're now easier than ever), so you could create skeleton projects for your webservices, already depending on this .war dependency.