I currently have artifactory set up on my jenkins pipeline correctly. I can see that it does indeed fetches artifacts but I noticed there is a few artifacts that arent being pulled. These artifacts dont seem to be in my jfrog server but i do specify in the POM file where they can be found and pulled. Here they are:
<repository>
<id>osgeo</id>
<name>OSGeo Release Repository</name>
<url>https://repo.osgeo.org/repository/release/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
<repository>
<id>osgeo-snapshot</id>
<name>OSGeo Snapshot Repository</name>
<url>https://repo.osgeo.org/repository/snapshot/</url>
<snapshots><enabled>true</enabled></snapshots>
<releases><enabled>false</enabled></releases>
</repository>
I guess maven/jenkins only checks my artifactory server and doesnt reach out to these repos at all. Is there a way where I can tell maven to not just look in my artifactory repo and also look in these repos as well? I have this snippet of code in my main pom file so im clueless as to why it doesnt pull from them.
Artifactory does reach out these remote endpoints if they are configured. Check if these "https://repo.osgeo.org/repository/snapshot" and "https://repo.osgeo.org/repository/release" are configured in the remote repository and if not, create the maven remote repositories in Artifactory and point the URL textbox to these remote endpoints. Add these 2 remote repositories in the Maven virtual repository and try to pull from Artifactory.
Related
I have copied all contents from an old Nexus server to a new server. I am able to login to the new server, see all the uploaded artifacts, etc.
On the Jenkins build master, the maven builds are running obviously as the jenkins user. The home directory for that user is /var/lib/jenkins. Under there, I have copied the .m2 folder and changed all the old URL's in the settings.xml to point to the new URL.
However, when I run a maven deploy, while the build downloads some dependencies from the new nexus server, at the end of the build comes the mvn-deploy plugin and that tries to upload to the old server.
Do the artifacts, war, jar files have a record of the nexus server? Is there some setting in Jenkins that I'm missing? I did a grep for the old nexus address in /var/lib/jenkins folder and none of the config xml's have any mention of the old nexus.
Is the URL hardcoded in your pom? Maybe the pom is not using the params you set in the settings.xml?
The repositories configured in the settings.xml are the source of Maven artifacts. The destination for new artifacts is in the distributionManagement block. This is usually somewhere in your POM, or in a parent POM. The block will look something like the below. Chances are, the old URL is there.
<distributionManagement>
<repository>
<id>releases</id>
<name>Release Repository</name>
<url>${repository.url}/nexus/content/repositories/releases</url>
<layout>default</layout>
<uniqueVersion>true</uniqueVersion>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>Snapshot Repository</name>
<url>${repository.url}/nexus/content/repositories/snapshots</url>
<layout>default</layout>
<uniqueVersion>true</uniqueVersion>
</snapshotRepository>
</distributionManagement>
I've discovered today, while investigating a bug regarding my project being no more deployable, that maven tries to load dependencies from all available repositories.
I don't find it wise, specifically regarding our internal artifacts, that are searched on amven central, Sonatype FlexMojos, and other external release repositories (specificall when the last one provides a custom 404 page that maven wrongly interprets as valid content, replacing my valid artifact with an invalid one).
So, is is possible to tell maven which repositories to lookup for SNAPSHOTs artifacts, and which to lookup for RELEASE ones ?
Would it help if you made it explicit to not look for snapshots in your repo?
<repositories>
<repository>
<id>central</id>
<name>Maven Repository Switchboard</name>
<layout>default</layout>
<url>http://repo1.maven.org/maven2</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
My solution is to set up a repository mirror in the intranet. That way, Maven needs to know a single URL to download everything.
I then use the internal mirror to bundle repositories (local and remote ones). If snapshot repos are first in the search path, the mirror server will almost never publish the names of my internal artifacts (only when I forgot to deploy them).
On top of that, it will reduce the download times for the rest of the team plus it will make sure that everyone can still work even when there are network problems outside of your control.
I am using my own version of the Nexus web app repository installed on my local machine. I have Nexus configured with only one repository, the one where I store my snapshots:
http://localhost:8081/nexus/content/repositories/MySnapshots/
Note that after the Nexus installation I removed all the default repositories and added just my own. (Perhaps this was a bad idea?)
When I do a mvn clean install I noticed that some of the 3rd party artifacts are downloading straight from the remote repository. For example, here is one of the output lines from the build:
Downloading: http://repo.maven.apache.org/maven2/com/sun/org/apache/xml/internal/resolver/...
The strange thing is that I see other artifacts are going through my local Nexus to ultimately get to the artifact:
Downloading: http://localhost:8081/nexus/content/repositories/MySnapshots/org/apache/maven/wagon/wagon-provider-api...
Notice how the first part of the download url is my local repository but everything after MySnapshots is from apache.org.
It's almost like my Nexus repository is acting like a proxy to maven.apache.org for some artifact downloads but for others it goes straight to the source.
Can anyone tell me why this is happening?
I would't be bothered so much by this if all my builds succeeded all the time but sometimes, when I am compile large projects, I get build failures due to not being able to find an artifact.
For example, when I try to build another project that depends on eclipse jdt stuff I get the following error:
Downloading: http://localhost:8081/nexus/content/repositories/MySnapshots/eclipse/jdt/core/eclipse.jdt.core
Could not find artifact eclipse.jdt.core:eclipse.jdt.core
I am not sure if this means that my Nexus is not configured properly or if there really is no artifact eclipse.jdt.com. If the downloads were not going through my local Nexus repository I would then investigate the pom/settings.xml files. Instead this makes me wonder if it's due to my Nexus configuration.
If you would like to see my settings.xml for Maven and my pom file for the project I am building when I see this you can view them here:
settings.xml: http://pastebin.com/NvLr5bEA
pom.xml: http://pastebin.com/PJ0P3RaK
If you like to use the local nexus as a proxy as usual than you have to configure the settings.xml like this:
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
The tricky thing is the mirror thing which reroutes every call to the configured nexus instance.
Things you mentioned like eclipse parts can be problematic, cause only a few artifacts are available via maven central. Furthermore you should leave the defaults like maven central, release repository and the snapshots repository unchanged, cause these are the repository you need.
I don't think its a proxy issue , upto my understanding for the first case when it is downloading from Maven Central Repo , it might be possible that same artifact is not available in your nexus repository , that's why it is going to Maven Central Repo.
In the second case it is available in your nexus so reactor didn't try to download it from Maven Central Repo.
I am using maven 3.0 ( with nexus setup ) for building my projects and am getting build failures :
Caused by: org.sonatype.aether.transfer.ArtifactNotFoundException: Could not find artifact directory:apacheds-core:jar:${apacheds_version} in central (http://localhost:8081/nexus/content/repositories/central)
at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:945)
at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:940)
My basic confusion is : When maven sees a dependency in the pom.xml , how does it go about looking for artifacts in remote repositories ?
My current understanding is :
It will first look in the local repo ( .m2/repository ).
If it does not find there , then it will try to search the repository specified in settings.xml under repository tag.
Question : Does it try all the repositories mentioned . or Just the first one ? Below I have mentioned 5 repos : does maven search all these one by one or just the first one ?
<repositories>
<repository>
<id>central</id>
<url>http://localhost:8081/nexus/content/repositories/central</url>
</repository>
<repository>
<id>remote</id>
<url>http://localhost:8081/nexus/content/repositories/remote-proxy-nexus-central</url>
</repository>
<repository>
<id>thirdParty</id>
<url>http://localhost:8081/nexus/content/repositories/thirdparty</url>
</repository>
<repository>
<id>codehaus</id>
<url>http://localhost:8081/nexus/content/repositories/codehaus-snapshots</url>
</repository>
<repository>
<id>public</id>
<url>http://localhost:8081/nexus/content/groups/public</url>
</repository>
</repositories>
My last confusion is about proxies section in the settings.xml. What are these locations :
<proxy>
<id>remote-proxy-nexus-central</id>
<active>true</active>
<protocol>http</protocol>
<host>repo1.maven.org/maven2</host>
<port>8080</port>
</proxy>
I can tell you we use a local Nexus and have all our users have the following in their settings.xml:
<mirror>
<id>our-mirror</id>
<name>Org Public Mirror</name>
<url>http://host/nexus/content/groups/public</url>
<mirrorOf>*</mirrorOf>
</mirror>
This causes any call by maven to go to Nexus to get a dependency. You are right about maven first looking in local .m2.
Nexus proxies many repositories and has a union of them all (for the maven processes calling it).
This means that a developer's local maven knows only of a single repository: Nexus. Nexus will serve all the needed dependencies id they are in one of its proxied/hosted repositories.
As for Proxy, we have an organization proxy, but the Nexus is in the org (it has the proxy configured to allow access to the outer world), so maven does not need this specific configuration.
I hope this gives you some information to get started.
I strongly urge you to look into Nexus/Maven related configurations at: http://www.sonatype.org/
The correct setup for using Maven with Nexus is documented in the book Repository Management with Nexus. The sample settings.xml is here. Read the description and note that you need to add the overrides for the central repository to enable snapshots.
Once you have done that you do NOT configured a proxy in your settings.xml since it will be available in your local network without a proxy (typically). Instead you configure the proxy settings in Nexus so that it in turn can get out to the repositories like Central that you are proxying. The global proxy configuration is documented here and if required you can also configure specifics per proxy repository e.g. if you need a username/password for a repository you are proxying because it is private..
i have a repository setup like this
<repository>
<id>jboss</id>
<name>JBoss Repository for Maven</name>
<url>https://repository.jboss.org/nexus/content/groups/public/</url>
</repository>
<repository>
<id>itextpdf.com</id>
<name>Maven Repository for iText</name>
<url>http://maven.itextpdf.com/</url>
</repository>
how do i set up the dependency to use the itext repository only for itext and jboss for everything else. for some reason the dependencies are being downloaded by both and one of them that is in the itext repository is different than the jboss and is crashing my system.
You shouldn't need to do this if the artifact only exists in one of the repositories (maven will try each repository in order and will stop once it finds it). The itext repo doesn't have any of the jboss jars so listing it first should result in it being checked first (and when it fails to resolve the jboss jars, it'll try the jboss repo).
Alternatively, if you have your own Maven repository server (Nexus, for instance), acting as a proxy for the public maven repositories, you can set up a "Repository Routing" in which you tell it which repositories to search for specific artifacts (based on group ID).