maven repository proxy confusion - maven

I am using maven 3.0 ( with nexus setup ) for building my projects and am getting build failures :
Caused by: org.sonatype.aether.transfer.ArtifactNotFoundException: Could not find artifact directory:apacheds-core:jar:${apacheds_version} in central (http://localhost:8081/nexus/content/repositories/central)
at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:945)
at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:940)
My basic confusion is : When maven sees a dependency in the pom.xml , how does it go about looking for artifacts in remote repositories ?
My current understanding is :
It will first look in the local repo ( .m2/repository ).
If it does not find there , then it will try to search the repository specified in settings.xml under repository tag.
Question : Does it try all the repositories mentioned . or Just the first one ? Below I have mentioned 5 repos : does maven search all these one by one or just the first one ?
<repositories>
<repository>
<id>central</id>
<url>http://localhost:8081/nexus/content/repositories/central</url>
</repository>
<repository>
<id>remote</id>
<url>http://localhost:8081/nexus/content/repositories/remote-proxy-nexus-central</url>
</repository>
<repository>
<id>thirdParty</id>
<url>http://localhost:8081/nexus/content/repositories/thirdparty</url>
</repository>
<repository>
<id>codehaus</id>
<url>http://localhost:8081/nexus/content/repositories/codehaus-snapshots</url>
</repository>
<repository>
<id>public</id>
<url>http://localhost:8081/nexus/content/groups/public</url>
</repository>
</repositories>
My last confusion is about proxies section in the settings.xml. What are these locations :
<proxy>
<id>remote-proxy-nexus-central</id>
<active>true</active>
<protocol>http</protocol>
<host>repo1.maven.org/maven2</host>
<port>8080</port>
</proxy>

I can tell you we use a local Nexus and have all our users have the following in their settings.xml:
<mirror>
<id>our-mirror</id>
<name>Org Public Mirror</name>
<url>http://host/nexus/content/groups/public</url>
<mirrorOf>*</mirrorOf>
</mirror>
This causes any call by maven to go to Nexus to get a dependency. You are right about maven first looking in local .m2.
Nexus proxies many repositories and has a union of them all (for the maven processes calling it).
This means that a developer's local maven knows only of a single repository: Nexus. Nexus will serve all the needed dependencies id they are in one of its proxied/hosted repositories.
As for Proxy, we have an organization proxy, but the Nexus is in the org (it has the proxy configured to allow access to the outer world), so maven does not need this specific configuration.
I hope this gives you some information to get started.
I strongly urge you to look into Nexus/Maven related configurations at: http://www.sonatype.org/

The correct setup for using Maven with Nexus is documented in the book Repository Management with Nexus. The sample settings.xml is here. Read the description and note that you need to add the overrides for the central repository to enable snapshots.
Once you have done that you do NOT configured a proxy in your settings.xml since it will be available in your local network without a proxy (typically). Instead you configure the proxy settings in Nexus so that it in turn can get out to the repositories like Central that you are proxying. The global proxy configuration is documented here and if required you can also configure specifics per proxy repository e.g. if you need a username/password for a repository you are proxying because it is private..

Related

Why are settings.xml and pom.xml are not directing Maven to deploy to Artifactory?

I am trying to point Maven to a local Artifactory (OSS) repo (both on my Mac) so I can build code provided by a vendor. I have created an empty Maven local repo and virtual repo in Artifactory, and can use Generate Settings to create the snippets. My ID and encrypted password seem to be working OK in ~/.m2/settings.xml
I have updated the code's pom.xml file and that seems to be causing Maven to resolve from Artifactory.
No matter how I configure in ~/.m2/settings.xml or in the master settings.xml or with -Dmaven.repo.local, Maven wants to deploy to a disk path to the local repo.
If I specify a partial path, Maven says "Using local repository at " a path consisting of the code's home + the path I said.
If I specify a path starting with a / (root), Maven says "Using local repository at /whatever"
I want to specify "https://localhost/artifactory/reponame" so Maven will deploy there. Artifactory generates snippets that contain that kind of URL. But since a URL doesn't start with a /, Maven insists on prepending the code home location to the URL, which of course is nonsense.
There must be some combo of and or <distributionManagement> <repository> <id>central</id> <name> <url>https://localhost/artifactory/... that makes this work. What am I missing?
pom.xml snippet:
<distributionManagement>
<repository>
<id>central</id>
<name>LocalMac-releases</name>
<url>https://localhost/artifactory/TBX</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>LocalMac-snapshots</name>
<url>https://localhost/artifactory/TBX</url>
</snapshotRepository>
</distributionManagement>
Master settings.xml snippet: localRepository is commented out so it takes the default. When commented in whatever I enter does take effect... combined with ${user.home}.
<!-- localRepository
| The path to the local repository maven will use to store artifacts.
|
| Default: ${user.home}/.m2/repository
<localRepository>/path/to/local/repo</localRepository>
-->

How can I get jenkins/Jfrog to pull from my own artifactory AND another remote one?

I currently have artifactory set up on my jenkins pipeline correctly. I can see that it does indeed fetches artifacts but I noticed there is a few artifacts that arent being pulled. These artifacts dont seem to be in my jfrog server but i do specify in the POM file where they can be found and pulled. Here they are:
<repository>
<id>osgeo</id>
<name>OSGeo Release Repository</name>
<url>https://repo.osgeo.org/repository/release/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
<repository>
<id>osgeo-snapshot</id>
<name>OSGeo Snapshot Repository</name>
<url>https://repo.osgeo.org/repository/snapshot/</url>
<snapshots><enabled>true</enabled></snapshots>
<releases><enabled>false</enabled></releases>
</repository>
I guess maven/jenkins only checks my artifactory server and doesnt reach out to these repos at all. Is there a way where I can tell maven to not just look in my artifactory repo and also look in these repos as well? I have this snippet of code in my main pom file so im clueless as to why it doesnt pull from them.
Artifactory does reach out these remote endpoints if they are configured. Check if these "https://repo.osgeo.org/repository/snapshot" and "https://repo.osgeo.org/repository/release" are configured in the remote repository and if not, create the maven remote repositories in Artifactory and point the URL textbox to these remote endpoints. Add these 2 remote repositories in the Maven virtual repository and try to pull from Artifactory.

Setup AEM Adobe CQ5 6.1 project to build/install offline

I'm new to CQ5 and looking for steps/settings I may need to do to setup an AEM adobe CQ5 6.1 project to build/install offline (not connected to internet).
I've to use our internal network Nexus (which has lot of general dependencies available except AEM related).
I've to use Maven & Java7.
Looking for possible issues/resolutions, steps & any helpful info.
Thanks all for your inputs.
After a detailed investigation, this is now resolved. Just want to share the findings so that it may help others.
The content-package-maven-plugin which was uploaded to our local nexus got corrupted.
Interestingly maven was not throwing any errors when i was trying mvn install command on my AEM project. Rather it was giving NoClassDefFound error for a further dependency of content-package-maven-plugin.
From the output of, mvn dependency:resolve-plugins, it was confirmed that the dependencies are not fully resolved for particular this plugin JAR, I tried to manually download the artifact from nexus & try to open/unjar it.
The artifact pom was corrupted and hence I uploaded the fresh artifact in nexus, and this time all went smooth.
My AEM project is now building fine with local nexus dependencies.
a typical AEM project does have lots of dependencies. There are some dependencies for the AEM platform(including granite, sling, osgi etc). These dependencies are downloaded from the adobe public repositories, unless you have a nexus repository in your company where all these dependencies are available.
If your Organization specific Nexus repository can connect to internet and download dependencies (AEM and non AEM related) atleast for the first time , then you do not need to connect to internet from your local sand box.
Theoretically, your organization specific nexus repo also might not be
connected to the internet and all the JARs can be uploaded to Nexus
manually. But, that would be the last thing I will do in my life !
Your project can download all the dependencies from the Nexus repo(only) in multiple ways.
Configure the repository configuration in your pom.xml file to point
to your internal Nexus repo.
<repositories>
<repository>
<id>internal</id>
<name>Internal Public Repository</name>
<url>URL to the public repository of your internal nexus</url>
<layout>default</layout>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>internal</id>
<name>Internal Public Repository</name>
<url>URL to the public repository of your internal nexus</url>
<layout>default</layout>
</pluginRepository>
</pluginRepositories>
Change your Maven settings (settings.xml) to use your internal Nexus
every time it downloads dependencies (Recommened).
settings.xml can be found in your <user>/.m2 directory or <maven-installation>/conf
<mirrors>
<mirror>
<id>public</id>
<mirrorOf>*</mirrorOf>
<url>URL to the public repository of your internal nexus</url>
</mirror>
</mirrors>

Local Nexus Repository Acting Like A Proxy For Some Maven Artifacts

I am using my own version of the Nexus web app repository installed on my local machine. I have Nexus configured with only one repository, the one where I store my snapshots:
http://localhost:8081/nexus/content/repositories/MySnapshots/
Note that after the Nexus installation I removed all the default repositories and added just my own. (Perhaps this was a bad idea?)
When I do a mvn clean install I noticed that some of the 3rd party artifacts are downloading straight from the remote repository. For example, here is one of the output lines from the build:
Downloading: http://repo.maven.apache.org/maven2/com/sun/org/apache/xml/internal/resolver/...
The strange thing is that I see other artifacts are going through my local Nexus to ultimately get to the artifact:
Downloading: http://localhost:8081/nexus/content/repositories/MySnapshots/org/apache/maven/wagon/wagon-provider-api...
Notice how the first part of the download url is my local repository but everything after MySnapshots is from apache.org.
It's almost like my Nexus repository is acting like a proxy to maven.apache.org for some artifact downloads but for others it goes straight to the source.
Can anyone tell me why this is happening?
I would't be bothered so much by this if all my builds succeeded all the time but sometimes, when I am compile large projects, I get build failures due to not being able to find an artifact.
For example, when I try to build another project that depends on eclipse jdt stuff I get the following error:
Downloading: http://localhost:8081/nexus/content/repositories/MySnapshots/eclipse/jdt/core/eclipse.jdt.core
Could not find artifact eclipse.jdt.core:eclipse.jdt.core
I am not sure if this means that my Nexus is not configured properly or if there really is no artifact eclipse.jdt.com. If the downloads were not going through my local Nexus repository I would then investigate the pom/settings.xml files. Instead this makes me wonder if it's due to my Nexus configuration.
If you would like to see my settings.xml for Maven and my pom file for the project I am building when I see this you can view them here:
settings.xml: http://pastebin.com/NvLr5bEA
pom.xml: http://pastebin.com/PJ0P3RaK
If you like to use the local nexus as a proxy as usual than you have to configure the settings.xml like this:
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
The tricky thing is the mirror thing which reroutes every call to the configured nexus instance.
Things you mentioned like eclipse parts can be problematic, cause only a few artifacts are available via maven central. Furthermore you should leave the defaults like maven central, release repository and the snapshots repository unchanged, cause these are the repository you need.
I don't think its a proxy issue , upto my understanding for the first case when it is downloading from Maven Central Repo , it might be possible that same artifact is not available in your nexus repository , that's why it is going to Maven Central Repo.
In the second case it is available in your nexus so reactor didn't try to download it from Maven Central Repo.

Set maven to use archiva repositories WITHOUT using activeByDefault?

I am very close to finally having a working setup with archiva and maven.
The last thing that's really boggling me, is how to set up my internal and snapshot repositories - without using a profile which contains activeByDefault set to true.
I am using a SUPER super pom - a company-wide pom which contains distributionManagement information for releases. I was thinking that I could specify the repositories in this pom, and configure the authentication settings in settings.xml? Can I use repositories tag without a profile? There should be no "profile" for my internal and snapshot repositories, as they will never change...
What I'm trying to steer clear from, is using a "default" profile, which is active all the time. I hear activeByDefault is NOT a best practice and I don't intend to use it.
With that said, how should I go about doing this? My internal repo is a mirror of the maven central repo, so I would like to lock down my developers to ONLY use our internal artifact server. Remember - I do NOT want a profile with activeByDefault set to true. I cannot stress this enough! Should I use Maven mirrors? Should I "add" additional repositories?
If I take the repositories tag instead of the mirrors tag, will maven force builds to use ONLY my archiva settings, instead of the default maven central?
Or is what I seek to accomplish able to be done using only the mirrors tag in maven? I know how to configure repo credentials when using repositories tag, but not with mirrors. How is this done? Is providing credentials for anything in mirrors tags the same as for anything in repositories tags?
Am I missing something obvious? I've had it up to here with getting things up and running using maven. I know it will be worthwhile in the end, but it is surely causing me a ton of aggravation and resources seem to be sparse. Either that, or people are content using it however they please without regard to best-practices.
Thank you
To use your internal repo as a mirror of central you need to setup a mirror like this (in settings.xml):
<mirrors>
<mirror>
<id>my-internal-repo</id>
<mirrorOf>central</mirrorOf> // use * for do mirror of all
<name>Clinker Maven Repository</name>
<url>http://my-repo-host/my-repo-path</url>
</mirror>
</mirrors>
If my-internal-repo is protected you can set credentials:
<servers>
<server>
<id>my-internal-repo</id>
<username>youruser</username>
<password>yourpassword</password>
</server>
</servers>
Please, note the server id tag content should match the id of your mirror.
To use your internal-snapshots repository you must set a repository in your project POM, since the use of snapshots artifact should be controlled and clearly defined at the project level, not at the settings level:
<repository>
<id>internal-snapshots</id>
<url>http://your-repo-host/internal-snapshots-path</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
And finally, you must exclude internal-snapshots from the mirror:
<mirrors>
<mirror>
<id>my-internal-repo</id>
<mirrorOf>central, !internal-snapshots</mirrorOf> // use * to do mirror of all
<name>Clinker Maven Repository</name>
<url>http://my-repo-host/my-repo-path</url>
</mirror>
</mirrors>
and add a server (if it's protected):
<servers>
<server>
<id>my-internal-repo</id>
<username>youruser</username>
<password>yourpassword</password>
</server>
<server>
<id>internal-snapshots</id>
<username>youruser</username>
<password>yourpassword</password>
</server>
</servers>

Resources