Artifactory repository pull replication using exclude patterns - caching

I have created a remote repository in a server by setting up pull replication of a repository in another server; since that repository is large I have added exclude patterns conditional to the setup to exclude all these artifacts that I don't need in my remote repository; everything seems to work fine, however my system logfile in reports a lot of warning messages for those exclude artifacts:
2018-05-30 19:45:38,267 [replication-consumer-1527709500001-0] [WARN ]
(o.a.r.s.RepositoryServiceImpl:1061) - Cannot set properties on
'xxxx-cache:yyyyyy/zzzzzzz/195383/aaaaaa/bbbbbb': Item not found
(I intentionally removed the actual names of my dirs and artifacts)
as you can see the message is about the cache repository created
Do you know how can I eliminate those warnings and get a cleaner system logfile?
Thanks

It seem that you have stumbled upon a known issue: https://www.jfrog.com/jira/browse/RTFACT-16597
You are welcome to vote and watch.

Related

Artifactory - Problems with overit-central and This item is not cached

We've been experiencing problems with our overit-central repository on Artifactory. Some elements appear as "not-cached" and while trying to retrieve them from a build, we're encountering an "element not found" error. That same element does exist in the overit-central repo, but somehow our Artifactory instance doesn't seem to find it or cache it. (The problem doesn't happen with all the elements of the repo, and we don't know why. Some of them are actually cached)
We've already run a re-index of said repository but still the "This item is not cached" continues to appear.
We tried creating a new remote repository hooked to the same url and running some maven re-indexes from Artifactory, but it didn't help.
How do we force the caching of the elements from Artifactory? The documentation in the JFrog support page mentions it, but doesn't explain how to do it.
Thank you in advance
Regards
A remote repository in Artifactory is a mirror of the endpoint you have configured. It doesn't download the artifacts by default and this is the reason it says "Artifact is not cached". It is the intended behavior and it lets you know that you have never downloaded this artifact.
In order to cache an artifact, you have to simply download it from the remote endpoint. This can be done in two ways:
From the UI - Right click --> download
Using the API "curl -uUSERNAME:PASSWORD http://ARTIFACTORY_URL:PORT/artifactory/REMOTE_REPOSITORY/PATH_TO_ARTIFACT
Once the artifact is downloaded, it means that it is cached and you will be able to see it in the "name-of-the-remote-repository-cache" repository which shows the downloaded cached artifacts.

Suddenly spring-boot-maven-plugin:2.7.1:build-image could not download spring-cloud-bindings

In the AWS CodeBuild pipeline I got this error during the build-image task:
could not download https://repo.spring.io/release/org/springframework/cloud/spring-cloud-bindings/1.10.0/spring-cloud-bindings-1.10.0.jar
The same build fail on my PC and the artifact spring-cloud-bindings-1.10.0.jar doesn't exists anymore on repo.spring.io.
Today the build-image task is working and the spring-cloud-bindings-1.10.0.jar is available from url https://repo.spring.io/release/org/springframework/cloud/spring-cloud-bindings/1.10.0/spring-cloud-bindings-1.10.0.jar.
The problem cause was a spring repo temporary unavailability.
This is an evidence this task doesn't use maven repository cache mechanism.
For future readers having the same or similar issues, you might want to read the relevant article from Spring team here.
Basically the sum up of the article is, that you better stop retrieving dependencies from repo.spring.io and switch to maven central instead.
Reason is that the instance of repo.spring.io was sponsored in the past from JFrog, Inc but as the situation has now changed they are basically moving to other instances.

2 Teamcity Web applications have problems accessing git repository

I have 2 separate Teamcity web application instances running on separate servers accessing the same git repository.
This seems to be causing issues with the VCS root configuration.
The specific error message is:
Error collecting changes for VCS repository ... 'git fetch' command failed... 301 Moved permanently
I get this message for every build configuration with every teamcity project in my installation.
I must maintain 2 separate teamcity application instances on different servers - one that builds Microsoft Windows software artifacts and the other that build Linux based artifacts.
The only workaround that I currently have for this problem is to delete the VCS root and set it up again - only to discover that the next day the problem is back.
I can't keep doing this forever - there must be a better way to address this.
Does anyone know how?
After working with Jetbrains support, they root caused the issue to the use of http:// in the github URL instead of https://.
So, IOW, when you specify a github repository URL, you must use https:// in a temmcity VCS root configuration.

Nexus 3.1.0-04 OSS - no artifacts/assets seen in repository/components

I am playing around with Nexus oss 3.1.0-04 OSS. I created a new maven style repository called test and it is proxying from http://repo1.maven.org/maven2/org/apache/maven; After setting this up, I tried to view the contents of test repository but there is nothing seen. I get a "no component found in repository". Why is this? What is that am missing? If I type the URL http://repo1.maven.org/maven2/org/apache/maven on a browser am able to see all its contents.
By default, the local proxy is empty. The best way to get components in is to build a maven project. Of course, make sure your maven settings are configured to point to Nexus - https://books.sonatype.com/nexus-book/reference3/maven.html#maven-sect-single-group).
I should also add, Nexus 3 provides a task for this: Publish Maven indexes
- Maven indexes can be used to download an index of available components to your repo, allowing users connecting to it to use the index to discover components. The task publishes the index for all or a specific Maven repository, hosted, group or proxy. This task will not populate the Browse UI, we intentionally did this so you only see what components and assets are available locally. More on the task here: https://books.sonatype.com/nexus-book/reference3/admin.html

Automatic routing filter rejected remote request - Nexus

I'm trying to get the netty-codec-hhtp going in my maven project. I have a completely standard Sonatype Nexus set up to proxy requests to Maven Central.
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-codec-http</artifactId>
<version>4.0.9.Final</version>
</dependency>
This fails when building using maven. If I search for it manually in Nexus I find it, but if I go to download the jar it tells me:
404 - Not Found
Automatic routing filter rejected remote request for path /io/netty/netty-codec-http/4.0.9.Final/netty-codec-http-4.0.9.Final.jar from M2Repository(id=central)
What does this even mean, why am I getting it, and maybe more importantly, how do I fix it?
I am using Nexus 2.5.0-04 with Maven 3.0.4
Downloading other artifacts seems to work just fine.
UPDATE: This turned out to be an issue with CDN configuration that should now be resolved. The steps below for forcing and/or disabling remote discovery are left for reference.
It means that Automatic Routing for Central is active, and that the discovered rules does not contain io.jetty as allowed prefix.
This should not happen, as the default configuration should update the rules on daily basis (as seen on screenshot below, showing the default Automatic Routing configuration for Central).
The remedy is to either force the update of rules (I did check, prefix /io is among rules Central publishes), or disable remote discovery completely.
Try steps as shown below on screenshot:
From Repository Management with Nexus - 6.4. Managing Routing:
Routing can be considered the internal activities Nexus perform in order to determine, where to look for a specific component in a Maven repository. The routing information has an impact on the performance of component retrieval as well as determining the availability of components.
(...) Automatic Routing is handled by Nexus on a per repository basis.(...)The Routing information consists of the top two levels of the directory structure of the repository and is stored in a prefixes.txt file. It allows Nexus to automatically route only component requests with the corresponding groupId values to a repository to avoid unnecessary index or even remote repository access.
Since Maven central repo contains that artifact I assume that the automatic routing rules forbids remote download for that artifact. The error message you posted suggests it also.
You can read how to add a routing rule under 6.4.2. Manual Routing Configuration. If my assumption is correct, this situation shoud be resolved by adding an inclusive rule type with ^/io/netty/.* route for central repo.
I faced the same issue when configuring a bridge between 2 Nexus.
In the first:
I added 2 proxy repo, targeting 2 public repo groups in the second,
then aggregate those 2 proxy repos in a new group,
that I finally added to the repo list of the first Nexus public repo group
Using the first Nexus, each time I requested a dependency only present in the 2nd, I encountered the Automatic routing filter rejected remote request for path... exception.
As mentionned by Tamas, this issue comes from the Routing >> Discovery feature that was enabled in my 2 proxy repos, that were targeting remote Nexus repository groups.
Once disabled, the issue was solved.
The exact explanation of this need is in some way explained in the official Sonatype documentation for Repository Management with Nexus, in chapter 6.4.1 on Automatic Routing:
The Routing tab for a proxy repository displayed in Figure 6.18,
“Automatic Routing for a Proxy Repository” contains the Discovery
section. It displays the Status and a more detailed Message about the
prefix file access. The Last run field displays the date and time of
the last execution of the prefix file discovery. Such an execution can
be triggered by pressing the Update now button. Otherwise, the Update
Interval allows you to trigger a new discovery every one, two, three,
six, nine or twelve hours or as a daily or weekly execution.
[...]
For a proxy repository, the prefix file is either downloaded from the
remote repository or a generation is attempted by scraping the remote
repository. This generation is not attempted for remote Nexus
repository groups, since they are too dynamic in nature and should not
be proxied directly. Scraping of hosted or proxy repositories as well
as Subversion-based repositories is supported.
Regards,
Thomas

Resources