I recently encountered a difficulty to find back a transitively dependent Maven artifact when building our product, which triggered me to think what is the correct strategy to deal with the situation.
In my product building CI, I need to use a plugin called Maven-Unix-Plugin, which is visible to us. This plugin transitively depends on function-java, which is not visible to us.
We have a Nexus repo, which proxy to the Maven repo. So, in the ideal situation, we don't need to store either maven-unix-plugin or functional-java.
However, I found the expected version of maven-unix-plugin still exists in Maven repository, but its dependent version of functional-java disappeared! More ironically, for all versions of maven-unix-plugin, whose dependent version of functional-java disappeared! In short, the maven-unix-plugin is not usable anymore!
In simple, this sounds an integrity problem of Maven repository. However, my major thinking is how we can avoid this on our side? Now, I managed to find back the old versioned functional-java and put it in our Nexus. However, first of first, it is impossible for us to figure out all of the transitive dependents and put all of them in our Nexus. The ideal picture of Maven is we should NOT be required to do that!
Of course, if we continuously build and maintain our system (and keep updating the dependency version ideally), we should have not faced this problem. However, we are a very small team to maintain a long list of huge legacy systems. The last build for this product is actually 2 years ago!
Any brilliant insights what is good strategies for this kind of headaches to happen in the future?
First of all, MavenCentral very rarely removes artifacts, and only if absolutely necessary. So this problem is likely not to hit you again.
If you want to make sure that this problem will not come again, make sure that you backup the storage of your Nexus repo. All artifacts that are drawn from MavenCentral through your Nexus will be stored in your Nexus storage. So even if MavenCentral is completely wiped out tomorrow, they will still exist in your Nexus.
Related
Let's say I have an aggregate Maven project. com.example:foo:1.0.0 is the aggregate parent POM, and it contains the child project com.example:bar:2.0.0. I publish them both to Maven Central using the nexus-staging-maven-plugin. Note that I do not use dependency management in the parent POM for the version of com.example:bar:2.0.0, even though it is a child project. That is, the two versions are not tied together.
I also use com.example:foo:1.0.0 as the parent POM in other projects, so I would prefer that its coordinates don't change unnecessarily (e.g. be published multiple times with different coordinates, even if the contents have not changed).
It turns out that com.example:bar:2.0.0 changes quite frequently, so I want to release com.example:bar:2.1.0. But I will be building from the aggregate parent POM in that repository, com.example:foo:1.0.0. Must I increment the version of the parent POM as well, even though the parent POM has not changed, and even though the versions are not tied to each other in any way? I would prefer to have the nexus-staging-maven-plugin simply realize that com.example:foo:1.0.0 has already been published, skip it, and continue and publish com.example:bar:2.1.0.
This question also extends to a sibling module that has already been published. Is there some way to configure nexus-staging-maven-plugin to skip a module that has already been published, yet continue building and publishing the other sibling projects that have not been published yet?
From this simplified example I'm sure someone will be tempted to answer, "but you shouldn't do it like that". In my more complex example, there are reasons it might be advantageous to do it like I'm explaining; and besides, that response would still leave the actual question unanswered.
I also realize that I could do some tests to see what happens, and perhaps I will, but I'm posting the question here to 1) get an authoritative answer, 2) promote discussion of the options, and 3) leave some definitive answer here for other who might have the same question. Thank you.
AFAIK, the nexus-staging-maven-plugin does not have a user property to detach or ignore certain artifacts.
You may be able to write your own plugin that takes the Maven project and eliminates a certain artifact from it.
Consider a project that is used in other projects, and this project has version 1.0 and is present in the Nexus.
After that, some changes are done in this project.
Now there are two solutions:
Increment the version of the project to 2.0, and delete the version 1.0 from Nexus. When the developers try to get the dependencies from the Nexus with version 1.0 they will get an error that this version does not exist and need to change the version to 2.0.
Change the functionalities of this project and inform the crew that some changes are done, but this is not the practice at all.
Is there any functionality in Maven and Nexus to simplify this task and make this all happen in the backend so the developers can't do anything, or is this not possible?
If you have an old version of an artifact that must not be used anymore because it has some dangerous bug, or it does not work with the new database structure or something like this, it may be advisable to move it to some non-public Nexus repository (and also delete it from the local repository of the build server), so that nobody can use it for release builds (people can use it for local builds, but this is usually not dangerous).
If you want to manage standard versions throughout your company, it is a good idea to have a parent pom or some boms which collect versions in a <dependenyManagment> section and can be included by the developers. This way, you only need to inform them to change one version number (namely the one of the parent pom or bom) instead of many.
Still, you are left with the problem that people do not read company newsletters. I know the problem that many developers of jars compile and test their source code against very old versions of their dependencies while the war/ear (that includes the jar) uses new versions.
I have some snapshots in a maven snapshot repo that I wish to completely delete. I want to delete these artifacts so that team members will be unable to resolve them if they attempt to continue to use them (they were moved to a new group). Is there a way I can undeploy them from mvn or is this a task that only an admin of the mvn snapshot repo can take care of?
While you can delete it from your server, you can't really stop them from using it; if they have a local copy they will continue to resolve it. You don't really want to block them either - the whole point of maven is a declarative representation of what to use. If they want to build a legacy version of their code for whatever reason (testing, reproducibility, roll-back,...), this would likely block that.
You could push a new snapshot and flag all the code as deprecated so their complier complains (assuming they update).
Now for new projects or updating existing ones we use a common parent POM that uses a dependency management section so that keeps things in pretty good shape. But you have to make sure they stick with your parent pom.
Others have done some work with the enforcer plugin.
It seems people have been talking about this for years here:
https://jira.codehaus.org/browse/MNG-3952
and here:
http://maven.40175.n5.nabble.com/Deprecating-and-banning-artifacts-with-repository-metadata-td124274.html
We check all of our source code's dependent third-party JARs into source control along with our source code. When needed, we manually download updates to third party JARs and replace those JARs that are in source control with the newer versions. We haven't felt the need to use Maven yet as this process seems simple enough for us. But are we missing something of great value by not using Maven? Or does our scenario not warrant using Maven?
"JARs dont change much", I hear this all the time.....
Storing jars in the SCM is simple in the beginning of the project. Over time the number of jars gets larger and larger.... Wait 2 or 3 years and nobody remembers where the jars came from, what their licensing terms were and most commonly what versions are being used (important to know when analysing security vulnerabilities).....
The best article I've read recently making the case for a repository manager is:
http://www.sonatype.com/people/2012/07/wait-you-dont-have-a-repository-manager/
A little irreverant, but does make a valid point about the kind of technical inertia one encounters all the time.
Switching a project team from ANT to Maven can be scary.... Maven works quite differently, so I find it is best deployed with greenfield or adventurous project teams. For the old-school ANT users, I recommend using the Apache ivy plugin. Ivy allows such teams to outsource the management of their dependencies but keep the build technology they're comfortable with.
Ultimately the biggest benefit of using Maven are not dependency management. It's the standized build process. I've seen several failed attempts to create a "standard" ANT build process. Problem every build engineer has his opinion on what the standard should be.... Maven's approach of forcing users to write build plugins may appear restrictive in the beginning, but just like the iPhone eventually developers discover "there's a Maven plugin for that" :-)
When it comes to dependency management Maven really can be quite valuable. As Mark O'Connor suggests, running a local repository manager would likely be better than checking the artifacts into source control.
There are many tools (like m2e in eclipse) that can help with dependency management and provide valuable feedback on which modules or dependencies require which other dependencies. Maven will also make sure to get the appropriate version of a dependency even if different modules depend on different versions of a given library. That will help prevent duplicate versions of the same jar showing up in your deployed project as long as they have the same group and artifact id.
Even for a very simple project I don't think I would resort to checking dependencies into the source control system.
It's not only about 3rd Party Libraries. Mostly if you have multiple repositories. In our case, we had four repositories with lots of inter- and intra-dependencies.
Actually I started this answer and then I had to go for 15 minutes to talk to some colleague about a problem happened after someone forgot to update the .jar of one project in the other's lib directory.
And it looks more professional :)
For a project with a large number of Maven artifacts (both internally generated as well as external ones), how does one go about automating the releasing of the internally controlled artifacts as part of an overall product release.
Things to be aware of about this question, we use Jenkins and the Maven release plugin. So the operation of releasing a single artifact is automated (albeit the operation to kick-start the process is manual). However the process of releasing all the changed artefacts over the course of a release is not automated (i.e. one has to manually kick-start the release of each artifact). Part of the problem is that almost nothing is released until the end of the release, prior to that everything remains in SHAPSHOT. We have a huge number of components as well as numerous applications/services (over 30) which rely on the plethora of components. So it is not just the case of picking a component and releasing, there are release dependency hierarchies that must be followed (i.e. start at the bottom releasing components that do not use other components and then work your way up until all the applications/services are released).
It is also worth noting that we use two common parent poms which, for the most part, control the versions of the external artifact dependencies and the internal component dependencies. Some pom files for components and applications may override this, but this is (or should be) an exception and should be for a good, but temporary, reason. So when an internal artifact is released, the version in the corresponding parent dependency pom should also be updated.
The product has a release number (of course), however the various pom files technically do not share this version number. While this is not strictly true, the idea as that when parts of the software are set to end-of-life, they will not be updated in the future, thus while a limited number of artifact versions match the product's version at present, this will eventually not be the case.
Any thoughts on ways to get this process automated would be greatly appreciated. Also if you feel what I have described seems to be a crazy way to manage the software, then please provide a comment. Thank you.
You might be able to make use of the Maven Versions plugin which can help formalise versions for projects.
For example, the use-next-releases goal may allow you to release the lowest level of project and then more rapidly bring those released versions into their dependencies.
There may also be scope to use the use-next-versions goal if you fancy releasing components as necessary and simply bring your projects to the "latest" version thats been formally released.