I have some snapshots in a maven snapshot repo that I wish to completely delete. I want to delete these artifacts so that team members will be unable to resolve them if they attempt to continue to use them (they were moved to a new group). Is there a way I can undeploy them from mvn or is this a task that only an admin of the mvn snapshot repo can take care of?
While you can delete it from your server, you can't really stop them from using it; if they have a local copy they will continue to resolve it. You don't really want to block them either - the whole point of maven is a declarative representation of what to use. If they want to build a legacy version of their code for whatever reason (testing, reproducibility, roll-back,...), this would likely block that.
You could push a new snapshot and flag all the code as deprecated so their complier complains (assuming they update).
Now for new projects or updating existing ones we use a common parent POM that uses a dependency management section so that keeps things in pretty good shape. But you have to make sure they stick with your parent pom.
Others have done some work with the enforcer plugin.
It seems people have been talking about this for years here:
https://jira.codehaus.org/browse/MNG-3952
and here:
http://maven.40175.n5.nabble.com/Deprecating-and-banning-artifacts-with-repository-metadata-td124274.html
Related
So I'm working for a customer that uses mvn deploy statements in his build scripts and I'm trying to figure out a way to prevent maven to accidentally overwrite artifacts in the release repo of Artifactory, for instance if a developer forgets to mark his POM version with -SNAPSHOT on his feature branch.
I'm no maven expert, but I've seen some suggestions, like using certain maven plugins, but these plugins' usage must be configured in the POM and then I'm back where I started, what if this is forgotten on a feature branch? There must be an established method to ensure that no artifacts from feature branches are deployed into the release repo and that no artifacts from release branches are deployed into the snapshot repo by accident.
One way I can think of and that also has been suggested is, to simply disallow redeployment on the release repo in Artifactory, but what if I have a validation build that fires after a PR is created and then another CI build fires and tries to redeploy?
Is there another good way to achieve this?
One solution is to ensure specific users/groups do not have the Delete permission.
See more here: https://www.jfrog.com/confluence/display/JFROG/Permissions#Permissions-RepositoryPermissions.
NOTE: I haven't used Artifactory in a while, but this makes sense according to the docs.
Let's say I have an aggregate Maven project. com.example:foo:1.0.0 is the aggregate parent POM, and it contains the child project com.example:bar:2.0.0. I publish them both to Maven Central using the nexus-staging-maven-plugin. Note that I do not use dependency management in the parent POM for the version of com.example:bar:2.0.0, even though it is a child project. That is, the two versions are not tied together.
I also use com.example:foo:1.0.0 as the parent POM in other projects, so I would prefer that its coordinates don't change unnecessarily (e.g. be published multiple times with different coordinates, even if the contents have not changed).
It turns out that com.example:bar:2.0.0 changes quite frequently, so I want to release com.example:bar:2.1.0. But I will be building from the aggregate parent POM in that repository, com.example:foo:1.0.0. Must I increment the version of the parent POM as well, even though the parent POM has not changed, and even though the versions are not tied to each other in any way? I would prefer to have the nexus-staging-maven-plugin simply realize that com.example:foo:1.0.0 has already been published, skip it, and continue and publish com.example:bar:2.1.0.
This question also extends to a sibling module that has already been published. Is there some way to configure nexus-staging-maven-plugin to skip a module that has already been published, yet continue building and publishing the other sibling projects that have not been published yet?
From this simplified example I'm sure someone will be tempted to answer, "but you shouldn't do it like that". In my more complex example, there are reasons it might be advantageous to do it like I'm explaining; and besides, that response would still leave the actual question unanswered.
I also realize that I could do some tests to see what happens, and perhaps I will, but I'm posting the question here to 1) get an authoritative answer, 2) promote discussion of the options, and 3) leave some definitive answer here for other who might have the same question. Thank you.
AFAIK, the nexus-staging-maven-plugin does not have a user property to detach or ignore certain artifacts.
You may be able to write your own plugin that takes the Maven project and eliminates a certain artifact from it.
I recently encountered a difficulty to find back a transitively dependent Maven artifact when building our product, which triggered me to think what is the correct strategy to deal with the situation.
In my product building CI, I need to use a plugin called Maven-Unix-Plugin, which is visible to us. This plugin transitively depends on function-java, which is not visible to us.
We have a Nexus repo, which proxy to the Maven repo. So, in the ideal situation, we don't need to store either maven-unix-plugin or functional-java.
However, I found the expected version of maven-unix-plugin still exists in Maven repository, but its dependent version of functional-java disappeared! More ironically, for all versions of maven-unix-plugin, whose dependent version of functional-java disappeared! In short, the maven-unix-plugin is not usable anymore!
In simple, this sounds an integrity problem of Maven repository. However, my major thinking is how we can avoid this on our side? Now, I managed to find back the old versioned functional-java and put it in our Nexus. However, first of first, it is impossible for us to figure out all of the transitive dependents and put all of them in our Nexus. The ideal picture of Maven is we should NOT be required to do that!
Of course, if we continuously build and maintain our system (and keep updating the dependency version ideally), we should have not faced this problem. However, we are a very small team to maintain a long list of huge legacy systems. The last build for this product is actually 2 years ago!
Any brilliant insights what is good strategies for this kind of headaches to happen in the future?
First of all, MavenCentral very rarely removes artifacts, and only if absolutely necessary. So this problem is likely not to hit you again.
If you want to make sure that this problem will not come again, make sure that you backup the storage of your Nexus repo. All artifacts that are drawn from MavenCentral through your Nexus will be stored in your Nexus storage. So even if MavenCentral is completely wiped out tomorrow, they will still exist in your Nexus.
I am using Artifactory to support an enterprise multi-module project. Often, we change the names of modules and the associated dependencies in POM files are not updated to use the new module name. Because SNAPSHOT dependencies are not automatically cleaned up on a regular interval, these old module references can stay there for months. I discovered a few when I migrated Artifactory to another server and the old module dependencies resulted in build errors. I am building these SNAPSHOT artifacts nightly using Jenkins so I would like some way to automate cleaning up the SNAPSHOT artifacts.
Does Artifactory (or another artifact server such as Nexus) support a concept where if a SNAPSHOT artifact is older than X days, the artifact is deleted? Is there another way to automate artifact server cleanup to accomplish what I want to do? The only thing I can think of is to create a cron job to clear out libs-snapshot-local on a regular interval before the nightly build starts. Has someone already built this capability?
As far as I know, Artifactory doesn't have an automated way to delete modules that are older than a certain value. At my shop we've written a Groovy client that uses Artifactory's REST API to do exactly this.
Note that, if your artifacts are shared libraries, you need to be careful that nothing depends on them before you delete them. Our script takes this into account, too.
If you're interested in following up, post a comment and I'll see if it's OK to share our script with you.
Another solution might be a user plugin. You can write a simple Groovy script that will run in Artifactory itself (as opposite to remote invocation by REST Gareth proposed) on a scheduled basis, searching for artifacts not downloaded for a long time and deleting them.
I've made a Ruby script to delete artifacts which aren't download for X days. The way it works just like what JBaruch mentioned in his answer.
It isn't a plugin. It works with Artifactory Open Source. Plugin is only supported by Artifactory Pro.
The source code: https://gist.github.com/aleung/5203736
I am attempting to create a way in which hermetic builds can be achieved while still relying on SNAPSHOT dependencies in your project.
For the purposes of example, say I have a project which has a dependency structure like this:
┌ other-1.2-SNAPSHOT
mine-1.2.3 ──┤
└ thing-3.1-SNAPSHOT ── gizmo-6.1.3-SNAPSHOT
What I would like to do is resolve all the SNAPSHOT dependencies locally to something which is related to my current version and then deploy those as releases to my Nexus' release repository. Not all of these dependencies are internal so I cannot simply just make a release on each.
So, in this example, other-1.2-SNAPSHOT would become something like other-1.2-mine-1.2.3 and thing-3.1-SNAPSHOT would become thing-3.1-mine-1.2.3. This is relatively trivial in about 60 lines of python.
The problem, however, is in resolving transitive SNAPSHOTs to concrete versions. So I also need to convert gizmo-6.1.3-SNAPSHOT to gizmo-6.1.3-mine.1.2.3 and have thing-3.1-mine-1.2.3 depend on it.
This is only an example of one way in which to achieve what I want. The goal is that in a year or two down the road I can checkout my release branch for version 1.2.3 and be able to run mvn clean package or the like without having to worry about resolving long-since-gone SNAPSHOT dependencies.
It's important that this branch be compilable and not just retain all dependencies using something like the jar-and-dependencies functionality of the assembly plugin. I'd like to potentially be able to modify the source files and make another release build (e.g., applying a hotfix).
So,
Is there anything like this available that will be able to convert SNAPSHOT dependencies in a recursive fashion to be concrete?
Are there any plugins which manage this kind of thing for you? The release plugin had promise with some configuration options on its branch goal but it doesn't resolve external deps to the degree that I want.
Are other techniques available for creating hermetic Maven builds?
This is not a widely used technique, but you can always check your specific SNAPSHOT dependencies into your project as a "project" repository, as described in this blog post: Maven is to Ant as a Nail Gun is to a Hammer
In short, use the Dependencies Plugin to create repository located in your project directory. The below is copied from the linked blog post (which you should read):
1) Run mvn -Dmdep.useRepositoryLayout=true -Dmdep.copyPom=true dependency:copy-dependencies
"This creates /target/dependencies with a repo-like layout of all your projects dependencies"
2) Copy target/dependencies/ to something like libs/
3) Add a repository declaration like the following to your POM:
<repositories>
<repository>
<releases />
<id>snapshots-I-need-forever</id>
<name>snapshots-I-need-forever</name>
<url>file:///${basedir}/libs</url>
</repository>
</repositories>
You make this an automated part of your build/release process: step 1 by configuring the Dependencies plugin to a lifecycle phasephase, and step 2 using AntRun Plugin to move the downloaded dependencies to the right place..
Hope this works for you. I have to go take a shower now...
The maven versions plugin will do most of what you want.
http://mojo.codehaus.org/versions-maven-plugin/
However you will almost certianly need to run it in a pre-build step in which you resolve all the dependencies and update the pom file accordingly. Then re-run maven (which re-reads the pom) to run the real build. You might be able to configure everything within the pom itself triggered with a separate goal thus avoiding a separate script.
This works better if you use particular versions instead of SNAPSHOT dependencies and let the pre-build step upgrade them if necessary. The only real difference for dependency resolution is that maven will always re-download -SNAPSHOT dependencies whereas it will only download normal dependencies if there is a new version available. However many plugins (including the versions plugin) treat -SNAPSHOT dependencies differently causing problems. Since every CI build has a new version number I never use -SNAPSHOT, prefering a different tag like -DEV with more predictable behaviour for things like developer local builds etc.
I've spent a lot of time getting maven to do things similar to this. Most maven projects I know have some kind of pre-build step in order to set version numbers or get around other limitations such as this. Trying to do all this in one step usually fails because maven only reads the pom once, string substitution doesn't work in a few places and the deployed/installed pom doesn't generally doesn't contain the results of string substituion or changes made during the build.