I was wondering if there is a Hudson/Jenkins plugin that ties repository artifacts to the build that created them? I was looking at the question "remove artifacts from nexus repository" and thinking that deleting a build in Jenkins should also offer the option to remove the artifacts the build created.
We are currently running Jenkins 1.447 and the Nexus Open Source 1.9.2.3. Our Jenkins builds create artifacts in our Nexus repository using the maven deploy goal. I appears that once those artifacts have been deployed, there is no similarly automated mechanism to remove them. We would like to tie the Jenkins build to its Nexus artifacts. I figure if we have decided to remove the build from Jenkins, we have no use for the build and therefore, have no need to store the artifacts for that build either. We would like deleting the build to trigger deleting the Nexus artifacts.
If there's nothing available I figure I could start writing something, but I wanted to check and see how others handle this.
you can use the REST API from Nexus to build your own Jenkins Plugin that does that Job for you. You could store the Jenkins Build Job number using the Nexus custom-metadata plugin. Once the build is deleted you can have your custom Jenkins Plugin delete all artifacts in Nexus that have that build number in their metadate. I had a similar Problem and wrote a custom Jenkins Plugin. Have a look at the tutorial and the source code on github. It should be pretty straight forward.
Tutorial:
http://blog.codecentric.de/en/2012/08/tutorial-create-a-jenkins-plugin-to-integrate-jenkins-and-nexus-repository/
Sourcecode:
https://github.com/marcelbirkner/nexus-metadata-plugin/
You can purge artifacts from a local repository via the maven-dependency-plugin.
If you have a release it does not make sense to number it based on the numbers of the build server. The usual use case is to use SNAPSHOT's for exactly this purpose. Furthermore the usual use case is to delete SNAPSHOT's based on a scheduled task after a time from Nexus but releases will never be deleted from Nexus.
Since you know the name of the release, you could build a custom job or trigger to use a wget command to delete the artifact from the nexus repository.
As the proper user in Nexus you do have the ability to delete release artifacts, not just SNAPSHOTS.
Related
So I'm working for a customer that uses mvn deploy statements in his build scripts and I'm trying to figure out a way to prevent maven to accidentally overwrite artifacts in the release repo of Artifactory, for instance if a developer forgets to mark his POM version with -SNAPSHOT on his feature branch.
I'm no maven expert, but I've seen some suggestions, like using certain maven plugins, but these plugins' usage must be configured in the POM and then I'm back where I started, what if this is forgotten on a feature branch? There must be an established method to ensure that no artifacts from feature branches are deployed into the release repo and that no artifacts from release branches are deployed into the snapshot repo by accident.
One way I can think of and that also has been suggested is, to simply disallow redeployment on the release repo in Artifactory, but what if I have a validation build that fires after a PR is created and then another CI build fires and tries to redeploy?
Is there another good way to achieve this?
One solution is to ensure specific users/groups do not have the Delete permission.
See more here: https://www.jfrog.com/confluence/display/JFROG/Permissions#Permissions-RepositoryPermissions.
NOTE: I haven't used Artifactory in a while, but this makes sense according to the docs.
is it possible in Maven to configure different local repositories for SNAPSHOT and RELEASE artifacts?
The reason I am asking, we are using Jenkins for continuous build for our project. To ensure the consistency (if same artifact is built from different Jenkins jobs because of race condition, we can experience chaotic behavior) before build start, we create a fresh local repository for Jenkins.
Now the problem is, our project is huge, so for every build we have to download lots of dependencies from our Nexus but when you think about it, there is no reason to download every time new the RELEASE artifacts. The RELEASE artifacts don't change from build to build, for ex, Spring 4.5, httpclient 4.0, aspectj 1.8.1 is same for one build to another.
So actually to ensure the consistency, we only should not have the SNAPSHOT dependencies in the repository. If we could have two local repositories one for RELEASE artifacts and the other for SNAPSHOT's, then before every build start, we could delete the SNAPSHOT repository but re-use the local RELEASE repository, which would save me gigabytes of download from Nexus.
I know we can do RELEASE, SNAPSHOT configurations for remote repositories, is it possible to do same sort of configuration for local repositories?
If this is not possible, how would you solve this problem.
There is currently no way to achieve this, and yes, I agree with the sentiment.
A reasonably recent versions of Jenkins' Maven plugin allow you to specify a custom local repository without having to edit a settings.xml file — the option is right there at the job definition screen (in the Advanced section, select Use private Maven repository).
So, what I would do is use this option, and precede the Maven build step with a script that deletes all directories, in the local private repository, which end in -SNAPSHOT.
It's repulsive, but I can't think of any other way.
I'm running Nexus and I have a snapshots Maven repository. When I publish artifacts to that repository I also run a "Rebuild Maven Metadata Files" type task that runs for about 5 minutes and updates "LATEST" to actually point to the newest artifacts. Can I update that LATEST reference some other way? I publish with Gradle. Can Gradle just update that specific groups metadata?
Thanks
The maven metadata file is NOT managed by Nexus but by the client side tool. By default that is Maven and it downloads the metadata file as part of the deployment, updates it with the new snapshot data and uploads it again.
Potentially Gradle does not do that correctly and that is the reason you have to use the workaround of using the Nexus scheduled task. I don't think Gradle has any further tools to fix it up better. The only thing I can think of you could do is to kick off the scheduled task from Gradle automatically. But you would still have the timing problem.
Best would be to file a bug against Gradle and get it fixed.
I am using Artifactory to support an enterprise multi-module project. Often, we change the names of modules and the associated dependencies in POM files are not updated to use the new module name. Because SNAPSHOT dependencies are not automatically cleaned up on a regular interval, these old module references can stay there for months. I discovered a few when I migrated Artifactory to another server and the old module dependencies resulted in build errors. I am building these SNAPSHOT artifacts nightly using Jenkins so I would like some way to automate cleaning up the SNAPSHOT artifacts.
Does Artifactory (or another artifact server such as Nexus) support a concept where if a SNAPSHOT artifact is older than X days, the artifact is deleted? Is there another way to automate artifact server cleanup to accomplish what I want to do? The only thing I can think of is to create a cron job to clear out libs-snapshot-local on a regular interval before the nightly build starts. Has someone already built this capability?
As far as I know, Artifactory doesn't have an automated way to delete modules that are older than a certain value. At my shop we've written a Groovy client that uses Artifactory's REST API to do exactly this.
Note that, if your artifacts are shared libraries, you need to be careful that nothing depends on them before you delete them. Our script takes this into account, too.
If you're interested in following up, post a comment and I'll see if it's OK to share our script with you.
Another solution might be a user plugin. You can write a simple Groovy script that will run in Artifactory itself (as opposite to remote invocation by REST Gareth proposed) on a scheduled basis, searching for artifacts not downloaded for a long time and deleting them.
I've made a Ruby script to delete artifacts which aren't download for X days. The way it works just like what JBaruch mentioned in his answer.
It isn't a plugin. It works with Artifactory Open Source. Plugin is only supported by Artifactory Pro.
The source code: https://gist.github.com/aleung/5203736
I'm building a project, which is made up from several (sometimes unrelated) modules and some more non standard java modules (built with ANT).
Each maven module is deployed to the releases repository on completion.
If the build fails in the middle, I might have some modules already deployed, so if I try to rebuild, the new attempt to deploy will fail since the artifacts are already deployed.
Is it possible to force a deploy or instead, remove the deployed artifact before I deploy again?
It sounds like the middleware admins have configured your remote repo instance (Nexus or Artifactory or whatever) to not allow artifact redeployment, and as #khmarbaise says there are good reasons for that. Nexus can be configured to allow artifact deletion by users in a particular role or with artifact deletion privileges. If your admins have it set up that way perhaps you can request the delete privilege and remove the offending artifacts. Or, perhaps the Nexus admin will agree to do it for you.
If neither of these is possible, here are some things to try which might keep this from happening in the future:
If you are using the release plugin, do a dry run (-DdryRun=true on the release:prepare command line) first. Maven should report any errors without committing to SCM.
Try running mvn install on your group of projects first. This will install the artifacts to your local repo, not the remote. If there's a problem you can whack the artifacts out of your local repo and start from scratch, repeating until you get a complete build.
If you are running a multi-module build, there are command line options that allow resuming a Maven build from a particular project forward.
Define -Dmaven.deploy.skip=true on the Maven command line. This is similar to suggestion #2, except Maven will actually load & configure the deploy plugin, it just won't do the actual deploy to the remote repo. Once everything works, remove the skip property.
I know it might be late, but in Nexus there's an option where allows the redeployment of artifacts.
Just select the repositories in the left, choose the repository you want to change the policy and then set it to Allow Redeploy.
The possible options have been increased ;)
Use the parameter deployAtEnd (More information: here). With this parameter the artifacts are deployed only, if all artifacts were built successfully.