maven local repository proxy external - maven

I am supporting an Archiva internal repository in an organization where some build environments don't have access to the open internet to fetch dependencies. I thought I could configure Archiva as a mirror for everything external (mirror with mirrorOf set to "external.*,!snapshots") and set it up with proxies so that, if one of my builds asks for something that is not in Archiva, it will use the proxies to go get the dependencies from the external sources I set up with proxies. After that, it would be cached in Archiva and would not have to be fetched from outside.
So I have proxies in my Archiva config for codehaus, sonatype (several), the basic Central, Cloudera for Hadoop stuff etc.; but I can't find a configuration that keeps my builds from trying to go directly to the external sources (which they can't reach) and yet satisfies the things that are not already in my Archiva by using the proxies.
Is there some trick configuration I'm missing here? Or do I need to switch to Artifactory or Nexus to get this kind of functionality?

You need to set the maven settings on each of the build environments to point to your archiva instance.
e.g in $user.home/.m2/settings.xml
<mirror>
<id>InternalMirror</id>
<mirrorOf>*</mirrorOf>
<name>Internal Mirror.</name>
<url>http://archivaserver/archivaPath/</url>
</mirror>
I haven't used Archiva before, but I have done this successfully with Nexus

Switching to a nexus will be moer preferable.
I have attached a settings.xml which can be used for nexus....
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>F:\m2\repository</localRepository>
<pluginGroups>
<pluginGroup>org.codehaus.sonar</pluginGroup>
</pluginGroups>
<proxies>
<proxy>
<active>true</active>
<protocol>http</protocol>
<host>your-host-id</host>
<port>8080</port>
</proxy>
</proxies>
<mirrors>
<mirror>
<id>nexus</id>
<mirrorOf>*,!sonar</mirrorOf>
<url>http://nexus-repo-url/content/groups/public</url>
<name>Nexus</name>
</mirror>
</mirrors>
<profiles>
<profile>
<id>NexusProfile</id>
<activation>
<activeByDefault>true</activeByDefault>
<jdk>1.5</jdk>
</activation>
<repositories>
<repository>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
<id>nexus</id>
<name>default-repos</name>
<url>http://nexus-repo-url/content/groups/default-repos</url>
</repository>
<repository>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
<id>nexus</id>
<name>default-repos</name>
<url>http://nexus-repo-url/content/groups/default-repos</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
<id>nexus</id>
<name>default-repos</name>
<url>http://nexus-repo-url/content/groups/default-repos</url>
</pluginRepository>
<pluginRepository>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
<id>nexus</id>
<name>default-repos</name>
<url>http://nexus-repo-url/content/groups/default-repos</url>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>NexusProfile</activeProfile>
</activeProfiles>
<servers>
<server>
<id>nexus</id>
<username>anonymous</username>
<password>password</password>
</server>
</servers>
</settings>

I recommend you read http://maven.apache.org/guides/mini/guide-mirror-settings.html for configuring your settings.
And I recommend you to use Archiva 1.4 version series which is more performant.
We are currently rewriting the ui see a real sample https://archiva-repository.apache.org/archiva/index.html?request_lang=en .
If you want your own version you must download a snapshot from here: https://builds.apache.org/view/A-F/view/Archiva/job/archiva-all-maven-3.x-jdk-1.6/

Related

maven - multiple repository lookup configuration

I'm using maven v3.0.1 for my projects. I've projects which depends on artifacts from a corporate remote repository. Also I had a local archiva repository in company which hold local artifacts, which are not in to corporate remote repository.
I would like to make the settings.xml in such a way that for all projects, it will lookup for the specified artifact first in the corporate remote repository, if not found there, look up for the artifact in the local archiva repository.
I added local repository in <repository> tag and enabled the profile for that <activeProfile>. But the looking up is not happening as expected. On analysis found that mirrorOf setting plays a role there. Following are my settings.xml.
<?xml version="1.0" encoding="UTF-8"?>
<settings>
<mirrors>
<mirror>
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://corporate-repo:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>daily</updatePolicy>
</snapshots>
</repository>
<repository>
<id>spring-snapshot</id>
<name>Spring Maven SNAPSHOT Repository</name>
<url>http://repo.springsource.org/libs-snapshot</url>
</repository>
<repository>
<id>internal</id>
<name>Local Release Repository</name>
<url>http://local-repo:8081/repository/internal</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>daily</updatePolicy>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>
How can I modify the settings.xml to do the lookup as I required ? Whether any way to provide two urls in the mirrorOf settings. I tried
<mirror>
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://corporate-repo:8081/nexus/content/groups/public,http://local-repo:8081/repository/internal</url>
</mirror>
No errors in xml parsing, but lookup is not working. Can any one shed light on how to resolve this issue
I experimented with lot of options and finally resolved by tweaking the mirrorOf settings to <mirrorOf>*,!internal</mirrorOf>
So mirror settings are,
<mirrors>
<mirror>
<id>nexus</id>
<mirrorOf>*,!internal</mirrorOf>
<url>http://corporate-repo:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
* ensures that for all request except internal repo !internal do lookup in corporate repo. For internal repo, do look up in local-repo configured with repository id as internal

Using Nexus repository instead of public Maven in Idea Intellij

How can I force Idea Intellij to download from Nexus repository instead of Maven Public repo?
Where can I configure that in Idea properties?
It's not defined in pom.xml files and I'd prefer not to.
Not sure if it will work also on Idea Intellij ( I am using spring source studio - eclipse clon) , but should because maven configuration is in settings.xml in .M2 directory which is located in your home directory. Just use something like this ( my server is running on nexus-server:8081):
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>releases</id>
<username>user-name</username>
<password>your-password</password>
</server>
<server>
<id>snapshots</id>
<username>user-name</username>
<password>your-password</password>
</server>
</servers>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://nexus-server:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Disable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>

Prefetch maven-metadata.xml with nexus

With the maven3 configuration file settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<settings>
<!--<host>webproxy</host>-->
<mirrors>
<mirror>
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://richter-local.de:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<servers>
<server>
<id>nexus</id>
</server>
</servers>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
<updatePolicy>interval:10080</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>interval:10080</updatePolicy>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
<updatePolicy>interval:10080</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>interval:10080</updatePolicy>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
<interactiveMode>true</interactiveMode>
<usePluginRegistry>false</usePluginRegistry>
<offline>false</offline>
</settings>
and a nexus maven proxy (version 2.7.2-03) I have to wait for a bunch of very slow downloads of maven-metadata.xml file every day after 00:00 a.m. (take about 5 mintues together for some KB which ought to be transfered within seconds). How can I prefetch those file in order to download them my local nexus instance?
It sounds like you have one or more proxy repositories that have remotes that are responding very slowly. Once a day the cache timeouts for these expires, resulting in very slow retrieval.
See here for information on how to debug this, and configure Nexus to prevent the issue from occurring again:
https://support.sonatype.com/entries/25884097-Troubleshooting-slow-maven-metadata-xml-download-speeds

Maven Nexus private repository access

I have just set up a nexus repository in my local network. This repository exposes access to the Maven central repository, plus three different private hosted repositories: snapshot, release and ext3rdparty. All these repositories are part of the public group in nexus, which I use as a central access point for all repositories:
http://nexus.server/nexus/content/groups/public/
My problem is: Using NetBeans as development envirnment, I can access to all repositories except the ext3rdparty. i.e. i got a compilation error when I try to build projects which are based on libraries stored in the ext3rdparty repo, and NetBeans can't find / navigate any of the artifacts available in this repository.
I suspect this is a settings.xml / pom.xml problem and not a NetBeans one because as I said, I can access to artifacts that I've released in other repositories, but I can't figure-out what's the problem.
Here is the FULL content of my settings.xml to redirect maven to my own repository:
<mirrors>
<mirror>
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://nexus.server/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>nexus</activeProfile>
</activeProfiles>
<servers>
<server>
<id>nexus</id>
<username>user</username>
<password>***</password>
</server>
<server>
<id>releases</id>
<username>user</username>
<password>***</password>
</server>
<server>
<id>snapshots</id>
<username>user</username>
<password>***</password>
</server>
</servers>
And here is the repositories-related part of my parent pom to allow deployment:
<distributionManagement>
<repository>
<id>releases</id>
<name>Releases</name>
<url>http://nexus.server/nexus/content/repositories/releases/</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>Snapshots</name>
<url>http://nexus.server/nexus/content/repositories/snapshots/</url>
</snapshotRepository>
</distributionManagement>
Any idea of what's wrong ?
Thanks in advance!
EDIT:
Half of a day lost: I finally ended up with removing all dependencies from the ext3rdparty repository, uploading them again to the repository, and updating all indexes and now everything works fine...
Don't really know what was the problem here, but at least the pom and the settings were OK. Thanks for your help.
If you have that sort of problem you can right click on the affected repository in the Repositories list and activate a Repair Index or Update Index and then check the status in the System Feeds for System updates.
Did you add your ext3rdparty repo to your public group?

Maven archetype catalog : specify custom location(s)

I'm deploying a Nexus repository for Maven, and custom archetypes on it.
I would like to execute mvn archetype:generate and be prompted a list of internal + custom archetypes.
The only way I found to prompt custom archetypes (in an ergonomic way, meaning no URL) is to define the archetype-catalog path as a property in the settings. This is not a valid solution because I want several catalogs (and this property cannot be overriden in CLI).
Does anybody have a clue on how to do that ?
Thanks in advance,
[EDIT]
I found an issue report related : http://jira.codehaus.org/browse/ARCHETYPE-273
And I noticed that during archetype:generate, maven tries to reach the central repository :
[DEBUG] Searching for remote catalog: http://repo1.maven.org/maven2/archetype-catalog.xml
[DEBUG] Searching for remote catalog: http://repo1.maven.org/maven2
It ends by a "Connection Timed out" because I did not (and don't want to) specify a proxy...
I don't understand why maven doesn't check nexus catalog...
I also have a Nexus configured to mirror the Maven repositories and thus the remote catalog too.
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://afbwt03:8081/nexus/content/groups/JavaRepo/</url>
</mirror>
and:
<profile>
<id>nexus</id>
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
I am able to access the remote catalog only when I use the following Maven command line:
mvn archetype:generate -DarchetypeCatalog=http://afbwt03:8081/nexus/content/groups/JavaRepo
If I don't define the archetypeCatalog variable, I get the same behavior as you do: trying to access the repo1. ... although some mirrors are configured.
You need to have
the property archetypeRepository defined in the active profile in your .m2/settings.xml
the repositories and pluginRepositories redirected to your mirror, on the same id "central".
and of course, the mirror defined
Apache maven documentation on archetype plugin specifies that archetypeRepository is definable user property:
http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html
Your .m2/settings.xml should have these minimal elements
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<mirrors>
<mirror>
<id>central</id>
<name>Mirror for maven central</name>
<url>http://mymvnhost:8081/nexus/content/groups/public/</url>
<mirrorOf>*</mirrorOf>
</mirror>
</mirrors>
<profiles>
<profile>
<id>dev</id>
<properties>
<archetypeRepository>http://mymvnhost:8081/nexus/content/groups/public/</archetypeRepository>
</properties>
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>dev</activeProfile>
</activeProfiles>
</settings>
With maven-archetype-plugin:3.1.1 you have to
add/edit archetype-catalog.xml on your repo to list your custom archetypes
edit your settings.xml to add this repo with id archetype.
invoke mvn archetype:generate -DarchetypeCatalog=remote
From https://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html :
If you want the catalogs to come from a different repository, please add the following to your settings.xml
<repository>
<id>archetype</id>
<url>https://repository.domain.com/path/to/repo/</url>
</repository>

Resources