Maven 3 - Look up custom repository only for specific groupid - maven

Is it possible to tell maven to look up user defined repository only for some specific groupids?
It seems to try the user defined repo first for all artifacts, and ends up taking lot longer to build.
[EDIT]
e.g. artifacts belonging to group "com.example" are hosted on the private repository; but other OSS artifacts like "org.apache", "org.codehaus" etc. are not. Maven tries searching the third-party artifacts in the private repo first, and then in central. I would like to filter the private repo lookups to "com.example" only

I was trying to find a definitive source, but the answer is no. Basically Maven cycles through the repositories you have defined until it either finds the dependency it needs, or there are no other repositories to check.
Off hand I think the order is it goes through repositories found in your pom.xml first to last, and then the repositories it finds in your settings.xml first to last.
So the trick, if there is one, when you have a single dependency that will be found in a certain repositories, and that is all that is there, is to make it one of the last repositories to be checked, or as close to last.
So what you could do, is add the something like the following in your pom.xml:
<repositories>
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>Maven Repository Switchboard</name>
<url>http://repo1.maven.org/maven2</url>
</repository>
<!-- Your custom repository here -->
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>custom</id>
<name>Custom Repo</name>
<url>Custom repo URL</url>
</repository>
</repositories>
That way central gets checked first. Its hacky, but should help with speed.

Related

Why maven is not able to download "SNAPSHOT" artifacts from azure artifacts repository? (Snapshot translation to timestamp is malfunctioning)

Maven is not able to download artifact when referred by version such as 'XXX-SNAPSHOT' but it is able to download every other dependency from it. It even download the SNAPSHOT if & only if I replace 'SNAPSHOT' by the real timestamp, for example, replace 155-SNAPSHOT by 155-20220224.175558-1.
We are hosting SNAPSHOTS and releases both under the same azur artifact feed. Any advice on how to approach/fix this missing snapshot translation to timestamp?
I wasn't able find the detailed documentation on how this snapshot translation mechanism works on client or server side, can anyone please point it to me?
<dependency>
<groupId>com.st.services.clients</groupId>
<artifactId>st-ccs-web-service</artifactId>
<!-- <version>155-SNAPSHOT</version> --><!-- not works -->
<version>155-20220224.175558-1</version>
</dependency>
<repositories>
<repository>
<name>My Hosted Maven Repo Group</name>
<id>maven-feed</id>
<url>https://pkgs.dev.azure.com/my-org/my-project/_packaging/maven-feed/maven/v1
</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
Not the outcome you were after, but,
Having also tried this myself, I think Azure DevOps simply does not support this.
There are a few aspects of Azure DevOps's Artifacts feature that doesn't entirely align with Maven/how Maven does things. Unfortunately, this appears to be one of them.
(An example of another is the management of timestamped snapshots of the same version as individual versions, which Sonatype Nexus can do but DevOps Artifacts cannot).

How Maven project downloads dependencies from Maven central?

Have a few Maven projects in our environment, and currently it is downloading the required dependencies mentioned in the pom.xml file from Maven central.
I was trying to figure out where the URL for Maven central is provided in pom file or project level, and unable to find out. I thought it could be in settings.xml, and found the below script:
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
Can someone provide inputs on the details of providing the Maven central URL in project/pom file level or where do we mention it ?
Thanks !
It is defined by maven core, more accurate in maven-model-builder (so called super pom).
If you like to change that which means usually to redirect everything via an internal repository manager the way to go is via the user settings.xml see for details: https://maven.apache.org/settings.html
The user settings is located in users home directory $HOME/.m2/settings.xml.
You should prevent to define repositories in pom.xml files, because it happens that the location of such repositories (repo managers) may change which would result in pom.xml which can not being built anymore after a time.

Pom file showing the exceptions [duplicate]

Recently Maven build jobs running in Jenkins are failing with the below exception saying that they couldn't pull dependencies from Maven Central and should use HTTPS. I'm not sure how to change the requests from HTTP to HTTPS. Could someone guide me on this matter?
[ERROR] Unresolveable build extension:
Plugin org.apache.maven.wagon:wagon-ssh:2.1 or one of its dependencies could not be resolved:
Failed to collect dependencies for org.apache.maven.wagon:wagon-ssh:jar:2.1 ():
Failed to read artifact descriptor for org.apache.maven.wagon:wagon-ssh:jar:2.1:
Could not transfer artifact org.apache.maven.wagon:wagon-ssh:pom:2.1 from/to central (http://repo.maven.apache.org/maven2):
Failed to transfer file: http://repo.maven.apache.org/maven2/org/apache/maven/wagon/wagon-ssh/2.1/wagon-ssh-2.1.pom.
Return code is: 501, ReasonPhrase:HTTPS Required. -> [Help 2]
Waiting for Jenkins to finish collecting data[ERROR]
Plugin org.apache.maven.plugins:maven-clean-plugin:2.4.1 or one of its dependencies could not be resolved:
Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:2.4.1:
Could not transfer artifact org.apache.maven.plugins:maven-clean-plugin:pom:2.4.1 from/to central (http://repo.maven.apache.org/maven2):
Failed to transfer file: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.4.1/maven-clean-plugin-2.4.1.pom.
Return code is: 501 , ReasonPhrase:HTTPS Required. -> [Help 1]
The reason for the observed error is explained in Central 501 HTTPS Required
Effective January 15, 2020, The Central Repository no longer supports
insecure communication over plain HTTP and requires that all requests
to the repository are encrypted over HTTPS.
It looks like latest versions of Maven (tried with 3.6.0, 3.6.1) are already using the HTTPS URL by default.
Here are the dates when the major repositories will switch:
Your Java builds might break starting January 13th (if you haven't yet switched repo access to HTTPS)
Update: Seems like from maven 3.2.3 maven central is accessed via HTTPS
See https://stackoverflow.com/a/25411658/5820670
Maven Change log
(http://maven.apache.org/docs/3.2.3/release-notes.html)
I am facing the same problem. There are two solutions that I tried, and both works fine for me.
Update the Maven version repository (Maven version >= 3.2.3)
Restrict the current Maven version to use HTTPS links.
Update the Maven version repository:
Download the Apache Maven binary that includes the default https addresses (Apache Maven 3.6.3 binary). And open the Options dialog window in tools of NetBeans menu bar (Java Maven Dialog View). And select browse option in Maven Home List Box (Maven Home List Box View). After adding the Apache Maven newly downloaded version (Updated Maven Home List Box View), the project builds and runs successfully.
Restrict the current Maven version to use HTTPS links:
Include the following code in pom.xml of your project.
<project>
...
<pluginRepositories>
<pluginRepository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
<releases>
<updatePolicy>never</updatePolicy>
</releases>
</pluginRepository>
</pluginRepositories>
<repositories>
<repository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
</project>
Effective January 15, 2020, The Central Repository no longer supports
insecure communication over plain HTTP and requires that all requests
to the repository are encrypted over HTTPS.
If you're receiving this error, then you need to replace all URL
references to Maven Central with their canonical HTTPS counterparts.
(source)
We have made the following changes in my project's build.gradle:
Old:
repositories {
maven { url "http://repo.maven.apache.org/maven2" }
}
New:
repositories {
maven { url "https://repo.maven.apache.org/maven2" }
}
Try to hit the below URL in any browser. It will return 501
http://repo.maven.apache.org/maven2/org/apache/maven/wagon/wagon-ssh/2.1/wagon-ssh-2.1.pom
Please try with https. It will download a pom.xml file:
https://repo.maven.apache.org/maven2/org/apache/maven/wagon/wagon-ssh/2.1/wagon-ssh-2.1.pom
Please add it (https://repo.maven.apache.org/maven2) in the setting.xml file:
<repositories>
<repository>
<id>Central Maven repository</id>
<name>Central Maven repository https</name>
<url>https://repo.maven.apache.org/maven2</url>
</repository>
</repositories>
I was using a clean install of Maven/Java on a Docker container.
For me, I had to cd $M2_HOME/conf and edit the settings.xml file there. Add the following block inside <mirrors>...</mirrors>
<mirror>
<id>central-secure</id>
<url>https://repo.maven.apache.org/maven2</url>
<mirrorOf>central</mirrorOf>
</mirror>
Update the central repository of Maven and use https instead of http.
<repositories>
<repository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
Add this in pom.xml file. It works fine for me
<pluginRepositories>
<pluginRepository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
<releases>
<updatePolicy>never</updatePolicy>
</releases>
</pluginRepository>
</pluginRepositories>
<repositories>
<repository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
Maven is moving to HTTPS and disabling HTTP access
Short story, from January 15, 2020, Maven Central repository is not longer supporting HTTP connections (other repositories are doing the same). Therefore, you will indicate your Maven/Gradle settings to use an HTTPS URL.
Solution:
You can choose one of the following three approaches.
Add a repository in your project´s pom.xml file
<project>
...
<repositories>
<repository>
<id>central maven repo</id>
<name>central maven repo https</name>
<url>https://repo.maven.apache.org/maven2</url>
</repository>
</repositories>
</project>
Add the repository into a profile in the settings.xml file.
<profile>
<id>my profile</id>
<repositories>
<repository>
<id>central maven repo</id>
<name>central maven repo https</name>
<url>https://repo.maven.apache.org/maven2</url>
</repository>
</repositories>
</profile>
Update you maven version to a new one that uses https values as default. The lastest one at this moment 3.6.3 Download here
For Gradle:
Only replace the URL for the HTTPS version.
repositories {
maven { url "https://repo.maven.apache.org/maven2" }
}
I was added following code segment to setting.xml and it was resolved the issue,
<mirrors>
<mirror>
<id>maven-mirror</id>
<name>Maven Mirror</name>
<url>https://repo.maven.apache.org/maven2</url>
<mirrorOf>central</mirrorOf>
</mirror>
</mirrors>
I was using an outdated version of Maven (3.0.3 and 3.1). These older versions no longer supports http repositories (as mentioned above). Upgrading to Maven 3.6 was the fix for me.
As stated in other answers, https is now required to make requests to Maven Central, while older versions of Maven use http.
If you don't want to/cannot upgrade to Maven 3.2.3+, you can do a workaround by adding the following code into your MAVEN_HOME\conf\settings.xml into the <profiles> section:
<profile>
<id>maven-https</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<repositories>
<repository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
This will be always an active setting unless you disable/override it in your POM when needed.
I have the same issue, but I use GitLab instead of Jenkins. The steps I had to do to get over the issue:
My project is in GitLab so it uses the .yml file which points to a Docker image I have to do continuous integration, and the image it uses has the http://maven URLs. So I changed that to https://maven.
That same Dockerfile image had an older version of Maven 3.0.1 that gave me issues just overnight. I updated the Dockerfile to get the latest version 3.6.3
I then deployed that image to my online repository, and updated my Maven project ymlfile to use that new image.
And lastly, I updated my main projects POM file to reference https://maven... instead of http://maven
I realize that is more specific to my setup. But without doing all of the steps above I would still continue to get this error message
Return code is: 501 , ReasonPhrase:HTTPS Required
For me (corporate coder) also adding a mirror repository in the settings.xml fixed the issue. I am also using Maven inside a docker container.
<mirrors>
<mirror>
<id>https-mirror</id>
<name>Https Mirror Repository</name>
<url>https://repo1.maven.org/maven2</url>
<mirrorOf>central</mirrorOf>
</mirror>
</mirrors>
For all the corporate coders, ideally,
if you get this error, it means that your code base is still being built from open-source community. You need to over ride the "central" repository with your in house company Maven repository manager.
You can go to your settings.xml and override your central repository URL from http:// to https://
<M2_HOME>/conf/settings.xml
Find the mirrors sections and add the following entry:
<mirror>
<id>other-mirror</id>
<name>Other Mirror Repository</name>
<url>https://other-mirror.repo.other-company.com/maven2</url>
<mirrorOf>central</mirrorOf>
</mirror>
In the URL section, if you were using either http://repo1.maven.org/maven2/ or http://repo.maven.apache.org/maven2/ then
Replace http://repo1.maven.org/maven2/ with https://repo1.maven.org/maven2/
Replace http://repo.maven.apache.org/maven2/ with https://repo.maven.apache.org/maven2/
You need to ideally use your company source control management/repository URL over here. As this will block any contact with open source Maven repository community.
As mentioned in other answers, effective from 15 January 2020, the central Maven repository doesn't support insecure communication over plain HTTP.
If you are using Netbeans older version, you have to make changes in maven to use https over http
Open C:\Program Files\NetBeans8.0.2\java\maven\conf\settings.xml
and paste below code in between mirrors tag
<mirror>
<id>maven-mirror</id>
<name>Maven Mirror</name>
<url>https://repo.maven.apache.org/maven2</url>
<mirrorOf>central</mirrorOf>
</mirror>
It will force maven to use https://repo.maven.apache.org/maven2 url.
Using Ubuntu 16.04, java 1.8.0_201.
I un-installed old maven and installed Maven 3.6.3,
still got this error that Maven dependencies are failing with a 501 error.
Realized it could be a truststore/keystore issue associated with requiring https.
Found that you can now configure -Djavax options using a jvm.config file, see: https://maven.apache.org/configure.html.
As I am also using Tomcat I copied the keystore & truststore config from Tomcat (setenv.sh) to my jvm.config and then it worked!
There is also an option to pass the this config in 'export MAVEN_OPTS' (when using mvn generate) but although this stopped the 501 error it created another: it expected a pom file.
Creating a separate jvm.config file works perfectly, just put it in the root of your project.
Hopefully this helps someone, took me all day to figure it out!
Same issue is also occuring for jcenter.
From 13 Jan 2020 onwards, Jcenter is only available at HTTPS.
Projects getting their dependencies using the same will start facing issues. For quick fixes do the following in your build.gradle
instead of
repositories {
jcenter ()
//others
}
use this:
repositories {
jcenter { url "http://jcenter.bintray.com/"}
//others
}
The error:
Failed to transfer file: http://repo.maven.apache.org/maven2/org/apache/maven/wagon/wagon-ssh/2.1/wagon-ssh-2.1.pom.
Return code is: 501 , ReasonPhrase:HTTPS Required.
Root cause analysis:
Maven central is expecting that the clients use https, but the client is making plain HTTP request only.
Therefore, the request for downloading the package named 'wagon-ssh-2.1.pom' had failed.
How to fix the problem?
Replace the URL "http://repo.maven.apache.org/maven2"
with "https://repo.maven.apache.org/maven2"
in pom.xml file or build.gradle file of the project.
My current environment does not support HTTPS, so adding the insecure version of the repo solved my problem: http://insecure.repo1.maven.org as per Sonatype
<repositories>
<repository>
<id>Central Maven repository</id>
<name>Central Maven repository insecure</name>
<url>http://insecure.repo1.maven.org</url>
</repository>
</repositories>
The following link got me out of the trouble,
https://support.sonatype.com/hc/en-us/articles/360041287334-Central-501-HTTPS-Required
You could make the changes either in your maven, apache-maven/conf/settings.xml.
Or, if you are specifying in your pom.xml, make the change there.
Before,
<repository>
<id>maven_central_repo</id>
<url>http://repo.maven.apache.org/maven2</url>
</repository>
Now,
<repository>
<id>maven_central_repo</id>
<url>https://repo.maven.apache.org/maven2</url>
</repository>
Note the change from http to https
Sharing this in case anyone needs it:
Old Gradle config( without Gitlab , Docker deployments , for simple projects)
repositories {
google()
jcenter()
maven { url "http://dl.bintray.com/davideas/maven" }
maven { url 'https://plugins.gradle.org/m2/' }
maven { url 'http://repo1.maven.org/maven2' }
maven { url 'http://jcenter.bintray.com' }
}
New config :
repositories {
google()
jcenter()
maven { url "https://dl.bintray.com/davideas/maven" }
maven { url 'https://plugins.gradle.org/m2/' }
maven { url 'https://repo1.maven.org/maven2' }
maven { url 'https://jcenter.bintray.com' }
}
Notice the https. Happy coding :)
Originally from https://stackoverflow.com/a/59796324/32453 though this might be useful:
Beware that your parent pom can (re) define repositories as well, and if it has overridden central and specified http for whatever reason, you'll need to fix that (so places to fix: ~/.m2/settings.xml
AND also parent poms).
If you can't fix it in parent pom, you can override parent pom's repo's, like this, in your child pom (extracted from the 3.6.3 default super pom, seems they changed the name from repo1 as well):
<repositories>
<repository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url> <!-- the https you've been looking for -->
<layout>default</layout>
<snapshots>
<enabled>false</enabled> <!-- or set to true if desired, default is false -->
</snapshots>
</repository>
</repositories>
This error occured to me too. I did what Muhammad umer said above. But, it only solved error for spring-boot-dependencies and spring-boot-dependencies has child dependencies. Now, there were 21 errors. Previously, it was 2 errors. Like this:
Non-resolvable import POM: Could not transfer artifact org.springframework.cloud:spring-cloud-dependencies:pom:Hoxton.SR3 from/to central
and also https required in the error message.
I updated the maven version from 3.2.2 to 3.6.3 and java version from 8 to 11. Now, all errors of https required are gone.
To update maven version
Download latest maven from here: download maven
Unzip and move it to /opt/maven/
Set the path export PATH=$PATH:/opt/maven/bin
And, also remove old maven from PATH
On an old grails environment the only thing that works without upgrading is:
settings.xml
<settings>
<mirrors>
<mirror>
<id>centralhttps</id>
<mirrorOf>central</mirrorOf>
<name>Maven central https</name>
<url>http://insecure.repo1.maven.org/maven2/</url>
</mirror>
</mirrors>
</settings>
I downloaded latest eclipse and tarted to use from here https://www.eclipse.org/downloads/packages/release/ which resolved my problem.
I hit this problem with the latest version (August 2020) (after not using Maven on this machine for ages) and was scratching my head as to why it could still be an issue after reading these answers.
Turns out I had an old settings.xml sitting in the .m2/ folder in my home directory with some customisations from years ago.
However, even deleting that file didn't fix it for me. I ended up deleting the entire .m2 folder.
I don't think there was anything else in it except for downloaded resources. Maybe just deleting folders like repository/org/apache/maven/archetype would have been sufficient.
I downloaded the last netbeans version 12.2, and the problem was resolved.
Add the following repository in pom.xml.
<project>
...
<repositories>
<repository>
<id>central</id>
<name>Maven Plugin Repository</name>
<url>https://repo1.maven.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
...
</project>

Why is maven hitting repo.maven.apache.org for local jars?

I have a project built and installed in the local maven repository ~/.m2 named com.example.project. From time to time, not always, when I build another project that depends on com.example.project, maven says in debug messages:
Downloading: https://repo.maven.apache.org/maven2/com/example/project/maven-metadata.xml
Why is maven querying that URL and why does this happen occasionally? First, there is nothing on that URL, second, the jar already exists on local repository, third, I didn't change the version number of the local jar dependency. So let us say I have 1000 local projects, will it query URL1, URL2, ..., URL1000. What is the logic of this query to a non-existing URL?
Why is maven querying that URL and why does this happen occasionally?
occasionally: is specifically daily, because it is the default updatepolicy for the maven builtin Central Repository.
If you take a look at the maven super POM, which all pom files inherited from, you will find the repository is configured as follows
<repositories>
<repository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<release> element is not appearing here, so that means the repository will take the default values for it.
enabled (default is true)
updatePolicy (default is daily) --> possible values: "always", "daily" (default), "interval:XXX" (in minutes) or "never" (only if it doesn't exist locally).
hope this helps

Multiple repositories in Maven with the same package

Assuming I have this in my settings.xml file:
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>one</id>
<name>release</name>
<url>http://localhost:8081/artifactory/A</url>
</repository>
<repository>
<snapshots />
<id>two</id>
<name>snapshot</name>
<url>http://localhost:8081/artifactory/B</url>
</repository>
Both URLs contain a package called org.apache.something but different versions. How do I force maven to choose from A and notB.
Since repository A is handling release versions and B is handling snapshots I'm expecting no conflict between them.
Simply declaring your dependency version should guide the choice between them.
Remember however that repository are searched in the order you declare them, so the first one owning the version you declare will win.
But again I expect no differences on same artifact+stable_version among different repositories.

Resources