Maven Not Downloading/Recognizing Snapshot - download

We use Maven/Nexus/Hudson in our team. The Maven version on our Hudson server is 3.0.4 and has stopped being able to download snapshot dependencies when building jobs. I'm not sure what has changed (no on the team has done anything, so they say), but something has happened because builds that worked Monday morning stopped working Monday afternoon.
When I try to build a project in Hudson that has a snapshot dependency, I get this error:
[WARNING] The POM for com.company:my-client:jar:1.9-SNAPSHOT is missing, no dependency information available
I believe I have nailed the issue down to the fact that Maven isn't downloading the maven-metadata.xml file and therefore can't resolve the dependency to the timestamp version. For example, in my local build (using Maven 3.0.3), I see this in my Maven output:
Downloading: http://ip:8080/nexus/content/groups/public/com/company/my-client/1.9-SNAPSHOT/maven-metadata.xml
Downloaded: http://ip:8080/nexus/content/groups/public/com/company/my-client/1.9-SNAPSHOT/maven-metadata.xml (1004 B at 20.0 KB/sec)
Downloading: http://ip:8080/nexus/content/groups/public/com/company/my-client/1.9-SNAPSHOT/my-client-1.9-20130625.202822-1.pom
Downloaded: http://ip:8080/nexus/content/groups/public/com/company/my-client/1.9-SNAPSHOT/my-client-1.9-20130625.202822-1.pom (3 KB at 57.5 KB/sec)
...
Downloading: http://ip:8080/nexus/content/groups/public/com/company/my-client/1.9-SNAPSHOT/my-client-1.9-20130625.202822-1.jar
...
Downloaded: http://ip:8080/nexus/content/groups/public/com/company/my-client/1.9-SNAPSHOT/my-client-1.9-20130625.202822-1.jar (10 KB at 153.9 KB/sec)
Both builds are accessing our Nexus repo and the Hudson Maven has no problems accessing non-snapshot dependencies from it, so it's not a connection issue.
Why would Maven not recognize the SNAPSHOT and download the maven-metadata.xml to get the latest timestamp version of the dependency?

I hit this problem as well. We inadvertently solved the issue by adding the source repositories into our project POM. Despite still routing through our mirror, this change allowed the project to resolve the metadata to the latest snapshot version.
Example:
<project>
<repositories>
<repository>
<id>XYZ-SNAPSHOTS</id>
<url>http://nexus.xyz.org/nexus/content/groups/enterprise-snapshots/</url>
<releases><enabled>false</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
</project>
After the fact, I discovered that we had not configured the mirror correctly. The real issue seems to be that snapshots were not enabled on the mirror in the settings.xml, even though they were enabled in Nexus itself.
When I added the repos in the project, I enabled snapshots without thinking about it, which then allowed Maven to resolve the snapshots. This fixed it, but the better solution is to update settings.xml to fix it for all mirrored repos.
For reference, here is the proper settings.xml configuration (taken from Sonatype):
<settings>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/repository/maven-public/</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>

After much hair-pulling, we discovered that our settings.xml file for the hudson user had the nexus profile removed. This appears to have the effect of not being able to query the snapshot repo although it could get artifacts from the release repo.
The last edit of the file before we fixed it was May 29th so it's still a mystery as to why the jobs built in the morning of June 24th but not in the afternoon. Perhaps something was being cached and that was refreshed.

Related

Order of maven repositories in settings.xml

I have a settings.xml which looks like below:-
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>nReleases</id>
<username>test</username>
<password>test123</password>
</server>
<server>
<id>nSnapshots</id>
<username>test</username>
<password>test123</password>
</server>
</servers>
<profiles>
<profile>
<id>space</id>
<repositories>
<repository>
<id>central</id>
<url>http://repo1.maven.org/maven2</url>
</repository>
<repository>
<id>nReleases</id>
<name>Releases</name>
<url>http://someserver/repository/maven-releases/</url>
</repository>
<repository>
<id>nSnapshots</id>
<name>Snapshots</name>
<url>http://someserver/repository/maven-snapshots/</url>
</repository>
</repositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>space</activeProfile>
</activeProfiles>
</settings>
I want maven to check repo1.maven.org first and then only look in someserver. Can someone let me know how this can be achieved?
I am using Maven 3.3.9.
I have gone through this question and tried to change the order in which repos are declared in profile but it did not help.
<profile>
<id>space</id>
<repositories>
<repository>
<id>nReleases</id>
<name>Releases</name>
<url>http://someserver/repository/maven-releases/</url>
</repository>
<repository>
<id>nSnapshots</id>
<name>Snapshots</name>
<url>http://someserver/repository/maven-snapshots/</url>
</repository>
<repository>
<id>central</id>
<url>http://repo1.maven.org/maven2</url>
</repository>
</repositories>
</profile>
Logs while doing mvn clean install
tuk-MacBook-Pro-4:camel tuk$ mvn clean install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building camel 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: http://someserver/repository/maven-releases/org/apache/camel/camel-redis/2.16.3/camel-redis-2.16.3.pom
Downloading: http://someserver/repository/maven-snapshots/org/apache/camel/camel-redis/2.16.3/camel-redis-2.16.3.pom
The order of the repository inside the settings was not clearly specified in Maven 2, but starting with Maven 3 (and the fix of of MNG-4400), the repositories are always tried in their declaration order in the settings.
What can happen, and what is likely the cause of your problem, is that Maven tries a repository, fails in doing so, and stores in your local repository the fact that it tried and failed. This results in the creation of .lastUpdated files in your local repository, storing this information. The consequence is that Maven will not re-try to download the dependency from a repository where it knows the download failed in the past. Thus, when you start a command and the project requires an artifact not present in your local repository, Maven will still try the repositories in their order of declaration, but it will skip the ones it knows already failed.
But you can force it to bypass this mechanism by passing the -U flag on the command line. It forces Maven to update the releases and snapshots dependencies, without looking into .lastUpdated files. This way, it will re-try every active remote repositories.

How to use the internet for maven builds instead, if the corporate maven repository is offline or down?

I have a corporate repository defined in my POM:
<distributionManagement>
<repository>
<id>central</id>
<name>libs-release-local</name>
<url>http://bi-pub.wgresorts.com:8081/artifactory/libs-release-local</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>libs-snapshot-local</name>
<url>http://bi-pub.wgresorts.com:8081/artifactory/libs-snapshot-local</url>
<uniqueVersion>false</uniqueVersion>
</snapshotRepository>
</distributionManagement>
But they keep unplugging and moving or rebooting that box, so sometimes it's not available. Is there a way to tell my maven poms "try and use the corporate repository, but if it's down use the internet?" I tried specifying the repository and plugin repository outside of the distribution management section to no avail.
May anyone offer guidance please?
In settings.xml I have a mirror and a local repository defined:
<mirrors>
<mirror>
<mirrorOf>*</mirrorOf>
<name>repo</name>
<url>http://bi-pub.wgresorts.com:8081/artifactory/repo</url>
<id>repo</id>
</mirror>
</mirrors>
<localRepository>C:\apache-maven-3.1.1\.m2\repository</localRepository>
I don't want to have to keep changing the settings.xml depending on the whims of the IT department, can I set it up so it tries the corporate one goes to the internet if it's down?
EDIT
If you used the id central, you'll have to first create a pom with central that points to the real one on the internet before you can fix this. It can also point to your corporate repository but with a different ID. Then it will fall over to the internet if your corporate repository is down.
I think you may also have to delete the mirror section from the settings.xml
For better or worse, binary repository is a critical piece of the production pipeline.
You don't ask "what options do I have if my site is down", do you? You just keep your site up. Same story here, your IT department should realize that.
You can run Artifactory in DR mode (active/passive) or even in HA mode (active/active). Your repository has all the means to stay up 24/7, so you really don't need those workarounds.
First of all your settings.xml file is not correct.
<settings>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>
You have to use the above definition to say maven to search for artifacts as well plugins into the mirror repository (doesn't matter if it's a nexus or artifactory).
As supplemental you have defined uniqueVersion which has no effect starting with Maven 3.
Apart from that the definition in setting.xml is for reading whereas the definition in pom.xml distributionManagement is used for writing which means deploying artifacts or the site.
So in both cases you need the repository up and working as already mentioned by JBaruch. If you IT department is not able to handle such cases (basic IT operations) i would hardly think about that.
Every build in your company depends on the availability of the Repository Manager either for downloading artifacts or for deploying results of the builds and for following steps of the pipeline as well.

How do you configure maven to ignore repositories specified in POM files?

I have a project on a company intranet (read - no outside internet access). I have a server running Artifactory with all required maven artifacts. I have a settings.xml file pointing maven to the running Artifactory server. Everything is happy and maven can download dependencies until an artifact specifying a repository in the POM file (in my case org/eclipse/jetty/jetty-project/7.5.4.v20111024/jetty-project-7.5.4.v20111024.pom). Then maven attempts to load the remaining dependencies from the repo specified in the POM file instead of from Artifactory. This breaks the build. How do you configure maven to ignore repositories specified in POM files?
Thanks,
Nathan
As a workaround, define another repository in your settings.xml with the same ID (is it oss.sonatype.org, defined in jetty-parent:19, that's the problem?) and point it at your repo. Maven will use that definition in favour of the one in the pom.
There's an open issue filed against Maven (MNG-3056) to allow this to be configured so only your repo would be used; in general, if you have a local repository, that would be the behaviour you would want.
That's a great answer Joe. Thank you. I was looking for it for quite some time.
I just quote an example, in which I had the same problem as Nathan.
I use a Maven enterprise repository (Nexus or Artifactory) and I am behind a proxy, that means that I cannot download directly (and do not want to) from any other repositories than mine.
Jasper reports net.sf.jasperreports:jasperreports:6.2.0 defines in its pom a couple of repositories.
<groupId>net.sf.jasperreports</groupId>
<artifactId>jasperreports</artifactId>
<version>6.2.0</version>
...
<repositories>
<repository>
<id>jasperreports</id>
<url>http://jasperreports.sourceforge.net/maven2</url>
</repository>
<repository>
<id>jaspersoft-third-party</id>
<url>http://jaspersoft.artifactoryonline.com/jaspersoft/third-party-ce-artifacts/</url>
</repository>
</repositories>
This causes the following exception:
C:\my-project>mvn verify
[INFO] Scanning for projects...
[INFO] Building my-project 1.0.0-SNAPSHOT
[INFO]
Downloading: http://mynexus/nexus/content/groups/ch-public/com/lowagie/itext/2.1.7.js4/itext-2.1.7.js4.pom
Downloading: http://jasperreports.sourceforge.net/maven2/com/lowagie/itext/2.1.7.js4/itext-2.1.7.js4.pom
Downloading: http://jaspersoft.artifactoryonline.com/jaspersoft/third-party-ce-artifacts/com/lowagie/itext/2.1.7.js4/itext-2.1.7.js4.pom
[INFO] BUILD FAILURE
[INFO] Could not resolve dependencies for project ... :
Failed to collect dependencies at net.sf.jasperreports:jasperreports:jar:6.2.0 ->
com.lowagie:itext:jar:2.1.7.js4: Failed to read artifact descriptor for com.lowagie:itext:jar:2.1.7.js4:
Could not transfer artifact com.lowagie:itext:pom:2.1.7.js4
from/to jasperreports (http://jasperreports.sourceforge.net/maven2):
Connect to jasperreports.sourceforge.net:80 [jasperreports.sourceforge.net/216.34.181.96]
failed: Connection timed out:
The solution as described by Joe is:
In global settings.xml (C:/maven-installation/conf/settings.xml) or private settings.xml (~/.m2/settings.xml) add the following profile:
<profiles>
<profile>
<id>ignore-repositories</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<repositories>
<repository>
<id>jasperreports</id>
<url>http://mynexus/nexus/content/groups/ch-public/
</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>jaspersoft-third-party</id>
<url>http://mynexus/nexus/content/groups/ch-public/
</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
Important: the repository id in the profiles (jasperreports , jaspersoft-third-party) matches exactly the id of the repository used in pom.xml - in this case the pom.xml of net.sf.jasperreports:jasperreports:6.2.0
Do not forget to add the "external" repositories to the "proxy" list of your Maven Enterprise Repository

What does "http://central" mean in my Maven settings.xml?

I've been copying "sample" settings.xml files for ages now, and almost all of them seem to include a repository with the URL http://central. This bugs me, because of course there could actually be a machine on the local domain called "central", so this is a valid URN, but it also must (might?) have some special meaning to Maven.
Is it just shorthand that's commonly used, but the actual URL is ignored? Could I replace it with something else, or remove it entirely? Is it documented anywhere?
If it matters, I develop on a corporate network that has an internal iBiblio mirror, that acts as "central" for us.
AFAIK, it is a bogus URL which mentions at Configure Maven to Download from Nexus as the following example: -
<settings>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>
The nexus profile is configured to download from the central repository with a bogus URL of http://central.
This URL is overridden by the mirror setting in the same settings.xml file to point to the URL of your single Nexus group. The nexus group is then listed as an active profile in the activeProfiles element.
I hope this may help.
We also used exactly this sample config, even with the same comments for about 15(!) years without any problems. For maven builds it does not matter, as always nexus is requested for any dependencies.
But today I tried to built a project with a combined ant+maven build which failed to get dependencies via artifact:remoteRepository Task from central Repo.
The solution was to fix the "url" tag in the central repo: https://repo.maven.apache.org/maven2
So it looks like it doesn't matter in most cases, but to avoid any corner cases, always use the right url.

Should i still use Nexus Repository Manager even if I am the only developer in my network?

I recently wanted to integrate Nexus to my home-based Java Project, just to learn what it is good for. As i read from the posts here, and from the nexus Website, its main usage aim is:
Maven Central Repo has more than 200k artifacts. Get a local copy of your artifacts. Do not download every time you need them. When you specify an artifact, first nexus will be asked if it has the artifact, if yes it will be read from your local cache. if not nexus will download it from the central repo. Your build will continue to work regardless of the original artifact in Central Repo. Speed up your builds etc.
Now what i don't understand here is: Is it not the same as maven does? First time when i define a new artifact in a pom.xml, the jar will be downloaded from the central repo. It will be placed in ~/.m2/repository. Next time it will be read from here as long as there is a copy of it. Even if i create a new project, the downloaded jars will be used from this repository.
I think, I would need Nexus if i have another developer in my network, who also needs these jars. So we would not need to download these jars separately from central. We would define a network based repo (Nexus) and the jars would be downloaded to this repository. Other developers would reach this repo without the need of reaching central repo.
In my case, i cant see any advantages of nexus. I now have this settings.xml which Nexus suggests to use:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository/>
<interactiveMode/>
<usePluginRegistry/>
<offline/>
<pluginGroups/>
<servers/>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
<proxies/>
</settings>
The jars i have in /.m2/repository/ directory are exactly same as the jars in http://localhost:8081/nexus/content/groups/public/.
What is the advantage of Nexus in my case?
There are several advantages:
When you start a new project, such as building an open source project from scratch, the downloads will be faster.
If Maven Central is ever offline or responding slowly, it won't impact you as severely. The Nexus cache will buffer you from that.
You can set up your own hosted repository and deploy your snapshot releases (complete with timestamp in the version name) "for real" - rather than just with -SNAPSHOT as in your .m2 cache
It's a great way to learn how real Maven repositories work, which is an important skill inside a larger organization (and which you don't want to learn through trial-and-error on a production Nexus repo).
Edit
See: Sonatype: Why use a repository?

Resources