Actualize repository group - maven

I created a repository group and linked a few other hosted repositories under that : one release repo, and another snaphost repo.
I noticed that group repository is out of sync with the linked repositories: contains such snapshot artifacts (old, not used) which does not exist under my snapshot repository - since I created a scheduled task to clean up snapshots older then 14 days in.
In this case I always delete the problematic folder on the filesystem and then, and when I run a maven build next time using that public group repository url, artifacts get fetched and up to date versions can be seen then.
To make it more clear
Hi,I am trying to explain the situation. I have a Jenkins job, which had published an artifact with version eg. 3.28-SNAPSHOT to the snapshot repo before. Since I have snapshot and release hosted repos, I created a repo group and add these to it. Then some changes were done on the mentioned Jenkins job and version number (build number) started from 0 agagin (3.0-SNAPSHOT)...
From this point of view the lower versions point to the newer artifacts and the higher versions point to the older ones. As I mentioned I have also a houskeeping script: shell script, not nexus scheduled task due to that is a littlebit slow, which deletes the snapshots older than 14 days + then I run update index nexus scheduled task to make local storage and nexus in sync .
After this clean up those versions with 3.28-... were deleted from snapshot repo, but these outdated artifacts remained under the repository group. So my question comes from this point: Why artifacts are duplicated (I mean on the local storage consuming disk space) when I create a repository group pointing to other repositories (release, snapshot)?
How can I force not to copy each artifact from the linked repos, just to maintain a metadata where requested artifacts can be downloaded from? Or if it is not possible since the implementation does not support it, how can I resync my repository group to follow snapshot and release repo changes (clean up shell script + update index outcome)?
How can I resolve this situation?
Thanks in advance!

Related

Strategies for building and storing Feature branches using Maven and Artifactory

I am currently trying to strategize the best way to store feature branches in Artifactory and use the feature branch SNAPSHOT builds in a dev environment.
The feature branches are of the form: feature/X.X-Feature-DTBXXXXX-SNAPSHOT
So far I have created a separate repo in Artifactory and pointed the target release repo in Jenkins to deploy artifacts to that repo on building.
This works fine except that because Maven looks for the qualifier SNAPSHOT in the branch name, it gets uploaded to the snapshot repo on Artifactory instead of the Feature repo.
The problem with this approach is that I am risking pollution of branches that are in the SNAPSHOT repo by uploading feature branches there.
I need to upload a feature branch to a Feature repo with names of the type feature/X-X-Feature-DTBXXXXX-SNAPSHOT and not to the SNAPSHOT repo.
Any help or pointers would be great.

How to automatically delete old versions of artifacts from hosted maven repository on OSS 3.0.0?

We are building and deploying multiple releases for various services in a single day. Due to this we are wasting a lot of storage for storing older versions of artifacts which will never be used again.
Is there a way to automatically delete older versions and just keep few versions such as last 10 in OSS 3.0.0?
I searched there documentation but couldn't find anything that works automatically. Currently I have to manually select and delete them which is very error prone and time consuming.
Few details about my setup:
"File" type "blob" is used for storage.
Repository is self "hosted" with format "maven2"
There are a few options you can use in Nexus Repository 3.x for Snapshots, from https://books.sonatype.com/nexus-book/reference3/admin.html#admin-system-tasks:
Purge unused Maven snapshot versions
Remove snapshots from Maven repository
As for Releases, removing Releases can be an anti-pattern, you generally should keep your releases around if others rely on them, etc...
There is a JIRA ticket for Removing Releases which you can follow at: https://issues.sonatype.org/browse/NEXUS-10821
This is also answered here: Purge old release from Nexus 3

Maven fails to download artifacts from local nexus

I have a problem with downloading artifacts from our local nexus, sorry if this is a bit long.
Our sources tree is divided into several projects, let's call them A and B. B is dependent on a release version of A that is deployed to our local Nexus server.
Whenever I release a new A the next several builds (In TeamCity) fail to download the new artifacts and I see the error:
Could not resolve dependencies for project B-groupId:B-artifactId:jar:B-version:
Could not find artifact A-groupId:A-artifactId:jar:A-newVersion
Here are some relevant facts:
We are building with the -T 1C maven option
The artifact DOES exist in nexus - if I go the the download URL it works
When I build it locally it works
Eventually things work out, meaning it fails to download a certain artifact the first time, the next time it succeeds but fails on another and so on until all artifacts are downloaded
Another project also released to the same local repository works fine when its version is updated
I see these multiple downloading lines in the log:
Downloading: http://nexus.company.com:8081/nexus/content/groups/public/com/company/group/artifact/1.0.10/artifact-1.0.10.pom
this line repeats several time for each of the just released artifacts
It doesn't seem to be an issue of the nexus index (Like I mentioned - building locally works fine, and also on some of the TeamCity agents it works)
Also - doesn't seem to be a network problem because both the TeamCity agent and the nexus server are in the same datacenter
Sorry if this was a long read, but I would really appreciate any help. This thing is bugging us crazy.
Thanks

How to get artifactory to update the maven-metadata.xml for a virtual repo?

long time reader, first time asker...
I have a standalone network (no internet access). It has an artifactory server which has virtual libs-snapshot and libs-release repos. Under libs-snapshot, there are 4 local snapshot repos. The reason for this is that we get a dump of all the artifactory repos from somewhere else (non-connected), and import it to this network. But we have to modify a subset of the snapshot artifacts there. So we created another local snapshot repo, call it mine-snapshot-local (maven 2 repo, set as unique, max artifacts=1?), and added it to the top of the libs-snapshot virtual. In theory, this would allow us to modify the handful of artifacts we needed to, deploy to our own repo, and local developers would pick those up. But we would still have access to the 99% of other artifacts from the periodic dump from the other non-connected system. In addition, we can import the drops from the other network, which are concurrently being modified, on a wholesale basis without touching our standalone network repo (mine-snapshot-local). I guess we're "branching" artifactory repos...
I realize we could probably just deploy straight into one of the imported repos, but the next time we get a dump from the other network, all those custom modified artifacts would go away... so I'd really like to get this method to work if possible.
from my local eclipse, the maven plugin deploys artifacts explicitly, and without error, to the mine-snapshot-local repo. The issue I'm seeing is that the maven-metadata.xml for the virtual libs-snapshot is not being updated. The timestamp of that file is updated, and if I browse with a web browser to libs-snapshot/whatever_package, I can see my newly deployed artifacts, with newer timestamps than the existing snapshots. But the maven-metadata.xml file still contains pointers to the "older" snapshot.
maven-metadata.xml is successfully updated in the mine-snapshot-local repo, but it is as if artifactory is not merging all the metadata files together correctly for the virtual repo. Or, more likely, I have misconfigured something to cause it to ignore our top-layer local repo somehow (but why would the snapshot jar/pom still show up there?).
We are using artifactory 2.6.1 (and do not have the option to upgrade).
I've tried a number of things: setting the snapshot repos to unique, nonunique, deployer, limiting the number of snapshots, etc. None of it seems to make much of a difference.
The one thing I do see as possibly being an issue is the build number assigned to a snapshot. For example, in the imported repo, the artifact might have a timestamp that is a week old but a build number of 4355. In my new repo, when i deploy, i have a much newer timestamp, but the build number is 1 (or something much, much smaller than 4355).
Am I barking up the wrong tree by trying to have multiple local snapshot repos like this? It seems like this should be ok, but maybe not.
You are using a very (but very) old version of Artifactory and it could be that you are suffering from an issue that was long gone. The normal behavior should be that if you have 4 maven repositories and you updated/deployed new artifacts into one of those repositories, the Virtual repository should aggregate the metadata from all of the listed repositories.
Just to verify, you mentioned that you are deploying from Eclipse, are you referring to P2? If so just a side note, Artifactory will not calculate metadata for P2 artifacts.

Moving Nexus Repository, Backup Only Certain Artifacts?

I have a Sonatype Nexus repository on an older machine, and I have purchased a newer server which will become my new repository host. In the installation of Nexus on the older machine I have an extensive collection of artifacts, the vast majority of which are now obsolete and can be safely removed from Nexus.
I know it is possible for me to move all of the artifacts from the old installation into the new installation by simply copying the sonatype-work directory to the new box. My question is this: If I want to prune the artifacts in that directory down to only what I need right now (probably about 20% of the repository contents) what steps would I have to take other than deleting the unwanted artifacts? For example, would I need to force Nexus to rebuild indexes? Thanks for the help!
You could just install the new Nexus and proxy off the old one via one proxy repo in addition to Central and other repos. Then you run this for a while and only things not found in other public repositories you configure will be proxied from the old Nexus instance.
At a later stage you could run scheduled task on the old repo that removed old items.
When you are satisfied you got everything you need, you do one last backup and then take the old Nexus instance offline.
Of course the other option is to just not worry and migrate it all. In the end you really only have to migrate what you actually deployed (so probably releases and 3rd party repos).
The easiest option btw. is to just copy the whole sonatype-work folder over to the new machine and fire it up with a new Nexus install there and flick the switch.

Resources