I am using Artifactory to support an enterprise multi-module project. Often, we change the names of modules and the associated dependencies in POM files are not updated to use the new module name. Because SNAPSHOT dependencies are not automatically cleaned up on a regular interval, these old module references can stay there for months. I discovered a few when I migrated Artifactory to another server and the old module dependencies resulted in build errors. I am building these SNAPSHOT artifacts nightly using Jenkins so I would like some way to automate cleaning up the SNAPSHOT artifacts.
Does Artifactory (or another artifact server such as Nexus) support a concept where if a SNAPSHOT artifact is older than X days, the artifact is deleted? Is there another way to automate artifact server cleanup to accomplish what I want to do? The only thing I can think of is to create a cron job to clear out libs-snapshot-local on a regular interval before the nightly build starts. Has someone already built this capability?
As far as I know, Artifactory doesn't have an automated way to delete modules that are older than a certain value. At my shop we've written a Groovy client that uses Artifactory's REST API to do exactly this.
Note that, if your artifacts are shared libraries, you need to be careful that nothing depends on them before you delete them. Our script takes this into account, too.
If you're interested in following up, post a comment and I'll see if it's OK to share our script with you.
Another solution might be a user plugin. You can write a simple Groovy script that will run in Artifactory itself (as opposite to remote invocation by REST Gareth proposed) on a scheduled basis, searching for artifacts not downloaded for a long time and deleting them.
I've made a Ruby script to delete artifacts which aren't download for X days. The way it works just like what JBaruch mentioned in his answer.
It isn't a plugin. It works with Artifactory Open Source. Plugin is only supported by Artifactory Pro.
The source code: https://gist.github.com/aleung/5203736
Related
So I'm working for a customer that uses mvn deploy statements in his build scripts and I'm trying to figure out a way to prevent maven to accidentally overwrite artifacts in the release repo of Artifactory, for instance if a developer forgets to mark his POM version with -SNAPSHOT on his feature branch.
I'm no maven expert, but I've seen some suggestions, like using certain maven plugins, but these plugins' usage must be configured in the POM and then I'm back where I started, what if this is forgotten on a feature branch? There must be an established method to ensure that no artifacts from feature branches are deployed into the release repo and that no artifacts from release branches are deployed into the snapshot repo by accident.
One way I can think of and that also has been suggested is, to simply disallow redeployment on the release repo in Artifactory, but what if I have a validation build that fires after a PR is created and then another CI build fires and tries to redeploy?
Is there another good way to achieve this?
One solution is to ensure specific users/groups do not have the Delete permission.
See more here: https://www.jfrog.com/confluence/display/JFROG/Permissions#Permissions-RepositoryPermissions.
NOTE: I haven't used Artifactory in a while, but this makes sense according to the docs.
I'm in the latter stages of setting up a CI environment for my project. I'm using Maven, Jenkins and Artifactory Pro and can successfully build my project and deploy it's artifacts to Artifactory. I have also written a bash script to retrieve the resulting artifacts of a specific build from Artifactory and copy them somewhere.
The main part I'm missing right now is automated versioning. I've looked at enabling Artifactory release management, which is really cool, but involves the rebuilding of the project. I'm really trying to follow the mantra of 'Build Once, Deploy Anywhere', so any rebuilding is a no-no.
My question boils down to: Is there an automated way (either with one of the aforementioned tools, or a plugin) to handle versioning, without rebuilding an artifact?
Artifactory Pro allows you to easily extend Artifactory's behavior with your own plugins written in groovy. (https://www.jfrog.com/confluence/display/RTF/User+Plugins)
You can find here, an example of Promote extension, that will change your artifacts versions without the needs of new build.
You can find more usefully examples in the GitHub "artifactory-user-plugins" repository.
I have some snapshots in a maven snapshot repo that I wish to completely delete. I want to delete these artifacts so that team members will be unable to resolve them if they attempt to continue to use them (they were moved to a new group). Is there a way I can undeploy them from mvn or is this a task that only an admin of the mvn snapshot repo can take care of?
While you can delete it from your server, you can't really stop them from using it; if they have a local copy they will continue to resolve it. You don't really want to block them either - the whole point of maven is a declarative representation of what to use. If they want to build a legacy version of their code for whatever reason (testing, reproducibility, roll-back,...), this would likely block that.
You could push a new snapshot and flag all the code as deprecated so their complier complains (assuming they update).
Now for new projects or updating existing ones we use a common parent POM that uses a dependency management section so that keeps things in pretty good shape. But you have to make sure they stick with your parent pom.
Others have done some work with the enforcer plugin.
It seems people have been talking about this for years here:
https://jira.codehaus.org/browse/MNG-3952
and here:
http://maven.40175.n5.nabble.com/Deprecating-and-banning-artifacts-with-repository-metadata-td124274.html
Wherever you read about continuous delivery or continuous integration it's recommended to use an artifact repository to store the artifacts even though Jenkins already stores them for each build.
So why is it recommended to use an artifact repository? Is there a smooth solution to work with the artifacts of the Jenkins builds, ex. to use these artifacts for deployment?
An artifact repository and continuous integration tools serve two different purposes and one cannot be substituted with the other. Check this video from Artifactory, one of the providers of artifact repositories, about why one should use an artifact repository.
Jenkins stores the artifacts as plain files without versioning while artifacts in an artifact repository can be version controlled. So you have a lot more flexibility in retrieving artifacts and governing them. Read this very good article on why we need them. Surely not all of those things are supported by continuous integration tools like Jenkins.
Moreover, you can also look at the Artifactory plugin for Jenkins which integrates the two.
An artifact repository is needed but the artifact repo is a conceptual piece an not always a distinct tool. With Jenkins you should have MD5 signatures and (I think) a way of downloading the files you want (web service call, right?) from your remote server. Certainly, if you're doing something simple like using the Jenkins build pipeline plugin, it should be able to access the right versions of the files smoothly.
Alternatively, if you are using a separate deployment tool, the better ones bundle an artifact repository.
Regardless, you want what the ITIL folks call a Definitive Media/Software Library. Definitive in that the bits are secure, trusted, and official. And a library in that they can be easily looked up and accessed. When working with an artifact repository, you need to make sure its adequately secure. It is backed up. It is accessible for your deployments (including to production). If you look at Jenkins and it meets your criteria in those categories, consider yourself done. If it's lacking, and I wouldn't be surprised if it was, then you need either a dedicated tool like the Maven repos, or something bundled with the deploy tooling.
For more of my rambling on the subject, there's a recorded webcast. The slides for that are up on Slideshare.
I haven't kept up to date with Jenkins, we still use a version of the CI when it was orginally called Hudson.
In your projects your poms you should normally point to your own artifact repository were you can fetch and deploy your own (company) projects.
Using an artifact repository with your CI server, it can then deploy successfully built snapshot and releases which can be available to other developers.
I was wondering if there is a Hudson/Jenkins plugin that ties repository artifacts to the build that created them? I was looking at the question "remove artifacts from nexus repository" and thinking that deleting a build in Jenkins should also offer the option to remove the artifacts the build created.
We are currently running Jenkins 1.447 and the Nexus Open Source 1.9.2.3. Our Jenkins builds create artifacts in our Nexus repository using the maven deploy goal. I appears that once those artifacts have been deployed, there is no similarly automated mechanism to remove them. We would like to tie the Jenkins build to its Nexus artifacts. I figure if we have decided to remove the build from Jenkins, we have no use for the build and therefore, have no need to store the artifacts for that build either. We would like deleting the build to trigger deleting the Nexus artifacts.
If there's nothing available I figure I could start writing something, but I wanted to check and see how others handle this.
you can use the REST API from Nexus to build your own Jenkins Plugin that does that Job for you. You could store the Jenkins Build Job number using the Nexus custom-metadata plugin. Once the build is deleted you can have your custom Jenkins Plugin delete all artifacts in Nexus that have that build number in their metadate. I had a similar Problem and wrote a custom Jenkins Plugin. Have a look at the tutorial and the source code on github. It should be pretty straight forward.
Tutorial:
http://blog.codecentric.de/en/2012/08/tutorial-create-a-jenkins-plugin-to-integrate-jenkins-and-nexus-repository/
Sourcecode:
https://github.com/marcelbirkner/nexus-metadata-plugin/
You can purge artifacts from a local repository via the maven-dependency-plugin.
If you have a release it does not make sense to number it based on the numbers of the build server. The usual use case is to use SNAPSHOT's for exactly this purpose. Furthermore the usual use case is to delete SNAPSHOT's based on a scheduled task after a time from Nexus but releases will never be deleted from Nexus.
Since you know the name of the release, you could build a custom job or trigger to use a wget command to delete the artifact from the nexus repository.
As the proper user in Nexus you do have the ability to delete release artifacts, not just SNAPSHOTS.