Is there such a thing as a local remote Maven repository? - maven

I have a build box on which is installed:
Maven
Bamboo
Archiva
I have configured Bamboo to grab my Maven project from a remote Git source and then build it with the goals 'clean install'.
I have configured Archiva with two repos:
mirror - a mirror of central
dev - repo for my artifacts
I have made the following changes to Maven settings.xml:
# Define local repo - this is the same location as i have set up for the Archiva 'dev' repo.
<localRepository>/opt/maven-repo/dev</localRepository>
# Define the Archiva mirror i set up
<mirror>
<id>mirror</id>
<url>http://localhost:8080/repository/mirror/</url>
<mirrorOf>external:*</mirrorOf>
</mirror>
When I execute the build Maven grabs everything external via the mirror and then adds the built artifact to dev, along with the other jars it grabbed from mirror. So i now have some duplicate jars...
\repo\mirror\junit\junit
\repo\mirror\classworlds\classworlds
\repo\dev\junit\junit
\repo\dev\classworlds\classworlds
\repo\dev\me\myartifact
My question is, is the correct approach? Really I want to keep 'dev' with just my artifacts and mirror with everything from central - i don't want duplicates.
Should I be using the LocalRepository config in settings.xml or should I be using 'mvn deploy' to put the artifact in my Archiva repository by a different method?
Could someone clarify the different use cases for a local and remote repository?
Finally, how should I be defining my POM? Currently, I have just defined
<distributionManagement>
<repository>
<id>dev</id>
<url>file:///repo/dev</url>
</repository>
</distributionManagement>
Should i be adding in my mirror?

To put artifacts in your repository manager you should use the default which is maven-deploy-plugin which can be controlled by distributionManagement. The target where to put those things is controlled by defining the distributionManagement.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
...
<distributionManagement>
<repository>
<id>releases</id>
<name>Release</name>
<url>http://urlArchiva/releases/</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>Snapshots</name>
<url>http://urlArchiva/snapshots/</url>
</snapshotRepository>
...
</distributionManagement>
...
</project>
The repositories which are used to consume artifacts from is defined in the settings.xml
<settings>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://serverUrlArchiva/public/</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>
On Bamboo you should be able to control which settings.xml is used to have an local repository per build which makes build independent from each other.

Related

Efficient Resolution for 501 HTTPS Required Error in Maven Build for Moving (http://repo1.maven.org/maven2 & http://repo.spring.io) to HTTPS?

Need Efficient Solution for moving to HTTPS for mvn clean install build success
Have project with numerous internal dependencies which use http://repo1.maven.org/maven2 & http://repo.spring.io for dependency download
Tried following solutions at my end
Added following settings.xml in ~/.m2/ folder. Tried Profiling options also.
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
https://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>${user.home}/.m2/repository</localRepository>
<interactiveMode/>
<offline/>
<pluginGroups/>
<servers/>
<mirrors>
<mirror>
<id>central-repository</id>
<name>Maven Repository HTTPS</name>
<url>https://repo1.maven.org/maven2</url>
<mirrorOf>central</mirrorOf>
</mirror>
<mirror>
<id>central-spring-repository</id>
<name>Spring Repository HTTPS</name>
<url>https://repo.spring.io</url>
<mirrorOf>springcentral</mirrorOf>
</mirror>
<mirror>
<id>organisation-repo1</id>
<name>org-repo1</name>
<url>http://org.maven.com/repo1</url>
<mirrorOf>repo1</mirrorOf>
</mirror>
<mirror>
<id>organisation-repo2</id>
<name>org-repo2</name>
<url>http://org.maven.com/repo2</url>
<mirrorOf>repo2</mirrorOf>
</mirror>
</mirrors>
<proxies/>
<profiles/>
<activeProfiles/>
</settings>
Still faced some issues for internal jars dependencies in the project.
Failed to read artifact descriptor for
com.netflix.hystrix:hystrix-core:jar:1.5.13: Could not transfer
artifact com.netflix.hystrix:hystrix-core:pom:1.5.13 from/to cxf-repo
(http://repo1.maven.org/maven2/): Transfer failed for
http://repo1.maven.org/maven2/com/netflix/hystrix/hystrix-core/1.5.13/hystrix-core-1.5.13.pom 501 HTTPS Required
Added following in pom.xml
<repositories>
<repository>
<id>central-maven</id>
<name>central https Releases</name>
<url>https://repo.maven.apache.org/maven2</url>
</repository>
<repository>
<id>central-spring</id>
<name>central spring https Releases</name>
<url>https://repo.spring.io</url>
</repository>
<repositories>
Have read this doc for maven central repo movement to HTTPS
https://blog.sonatype.com/central-repository-moving-to-https but couldn't get any proper resolution for internal dependency issue.
Some dependencies in my case are still calling http url's of central and spring
maven
Either upgrade your Maven version
or
Restrict the current Maven version to use HTTPS links by modifying your POM:
Include the following code in pom.xml of your project.
<project>
...
<repositories>
<repository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
taking a backup of .m2 folder and doing clean and install worked for me.

Not retrieving jars from maven nexus setup specific to project

Got 2 Nexus maven repo - one serving or holding common jars - mostly from maven central & some others. And other - project specific maven nexus, where it holds 2 thirdparty jars which are needed for compilation of current project of interest.
Below is updates which is added to refer to local nexus maven setup & corresponding dependencies..
pom.xml snippet :
<project
...
<!-- download plugins from this *proj specific* repo -->
<repositories>
<repository>
<id>zzz-maven</id>
<name>zzz-maven</name>
<url>http://blah.blah.com/nexus/content/repositories/zzz-maven</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>zzz.zzz-report<groupId>
<artifactId>zzz-report<artifactId>
<version>1.2<version>
</dependency>
...
<!-- And other dependency to fetch jars from common nexus (which is working fine) -->
Added below to settings.xml (highlighted in bold text) - one covering Url to retrieve proj specific jars & other part of authenticating to proj specific Nexus
<settings>
<mirrors>
<mirror>
<id>nexus</id>
<name>Common nexus across org - Anonymous access </name>
<url>http://common-nexusxyz.com/nexus/content/repositories/maven</url>
<mirrorOf>*</mirrorOf>
</mirror>
</mirrors>
<profiles>
</profile>
<profile>
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
<!--**Added this one** -->
</profile>
<profile>
<id>zzz-maven</id>
<repositories>
<repository>
<id>zzz-maven</id>
<name>zzz-maven</name>
<url>http://blah.blah.com/nexus/content/repositories/zzz-maven</url>
</repository>
</repositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>nexus</activeProfile>
<activeProfile>zzz-maven</activeProfile> <!--**activated** additional one here -->
</activeProfiles>
<servers>
<server>
<id>zzz-maven</id>
<username>userNameForZZZ</username> <!--**Specified** explicit password needed for proj specific maven nexus here -->
<password>passwordForZZZ</password>
</server>
</servers>
</settings>
But still throws out Warning & then the error for mvn install or mvn compile like :
[WARNING] The POM for zzz.zzz-report:zzz-report:jar:1.2 is missing, no dependency information available
Would like to know what is missing - so that it will download proj specific jars from proj specific maven nexus?
Tried with both <repositories> & <pluginRepositories> option in pom to consider for download
Expect it to connect to proj specific maven nexus & download dependency jar defined in pom
You have defined
<mirror>
<id>nexus</id>
<name>Common nexus across org - Anonymous access </name>
<url>http://common-nexusxyz.com/nexus/content/repositories/maven</url>
<mirrorOf>*</mirrorOf>
</mirror>
which means that all requests (due to mirrorOf having the value *) are redirected to that particular repository, no matter what other repositories are defined in the POM or the settings.xml.

How to fetch artifacts from own repo using maven?

How do I create a simple project that fetches artifacts from my repo using maven? And where will these artifacts get saved to - my default maven repo?
I am using Apache Archiva.
Just simply start using a repository manager like Nexus and that's it. Please configure your settings.xml file according to the following:
<settings>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>URL OF your ARCHIVA SERVER</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>
The artifacts which are being downloaded are first stored into the Archiva and of course on your hard drive $HOME/.m2/repository.
If you like to deploy artifact into archiva you need to configure the distributionManagement in your pom file similar like this:
<distributionManagement>
<repository>
<id>releases</id>
<name>Archiva RElease repo</name>
<url>http://URL OF YOUR ARCHIVA/releases/</url>
</repository>
<snapshotRepository>
<id>snapshots<id>
<name>Archiva Snapshots repo</name>
<url>http://URL OF YOUR ARCHIVA/snapshots/</url>
</snapshotRepository>
...
</distributionManagement>
...
To test this you can use mvn deploy to see if the artiacts are being deployed to the snapshot repository. You can change the version of your test project into 1.0 and redo a mvn deploy which will try to deploy the artifacts into the release repository.

Build/Deploy third-party project in maven but upload to my snapshots repository

I build weekly a large third-party project (apache sling) in Jenkins. I've got jenkin's settings.xml set up to use my own archiva server to download all dependencies (using the mirror section).
However, I haven't figured out how to get the build to upload snapshots my own snapshots repository when doing a 'deploy'. It instead tries to upload the snapshots to an apache.org snapshots server and fails.
Is there a way to configure settings.xml to override the snapshots server in a similar way it's possible to override the repository? This has to be done without editing the project's pom.xml.
The reason I need to do this is because I need access to some of the snapshot versions as dependencies in another project, and I don't want to have to manually upload them all into archiva.
Here's my settings.xml:
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>/Users/Shared/Jenkins/.m2/repository</localRepository>
<servers>
<server>
<id>astra.internal</id>
<username>-deleted-</username>
<password>-deleted-</password>
</server>
<server>
<id>astra.snapshots</id>
<username>-deleted-</username>
<password>-deleted-</password>
</server>
</servers>
<mirrors>
<mirror>
<id>central-proxy</id>
<name>Local proxy of central repo</name>
<url>http://-deleted-.com/repository/internal</url>
<mirrorOf>central</mirrorOf>
</mirror>
<mirror>
<id>snapshots</id>
<name>Local proxy of snapshots</name>
<url>http://-deleted-.com/repository/internal</url>
<mirrorOf>snapshots</mirrorOf>
</mirror>
</mirrors>
<profiles>
<profile>
<id>Repository Proxy</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<!-- ******************************************************* -->
<!-- repositories for jar artifacts -->
<profile>
<!-- ******************************************************* -->
<repositories>
<repository>
<id>astra.internal</id>
<url>http://-deleted-.com/repository/internal/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>astra.snapshots</id>
<url>http://-deleted-.com/repository/snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
</settings>
You can overwrite the Distribution management section with command line parameters.
mvn clean deploy -DaltDeploymentRepository=ui-snapshots::default::http://localhost:8081/artifactory/libs-snapshot-local/
Have a look at the maven deploy plugin.
Alternatively, you should be able to use the Jenkins post build step. There you need to specifically provide the Repo url anyway.
The location of the repository is part of the pom.xml, the distributionManagement often in one of the upper-most parents. So it is not something you can set from the settings.xml.
There's a good chance that you can change it with the distMgmtSnapshotsUrl property, see org.apache:apache:13:pom, assuming the sling-project uses this as its (indirect) parent.
As far as I know you have to edit the pom file for the project and have this section (with your server settings of course):
<distributionManagement>
<repository>
<id>archiva.internal</id>
<name>Internal Release Repository</name>
<url>dav:http://localhost:8080/repository/internal</url>
</repository>
<snapshotRepository>
<id>archiva.snapshots</id>
<name>Internal Snapshot Repository</name>
<url>http://localhost:8080/repository/snapshots</url>
</snapshotRepository>
</distributionManagement>
Would it help you to put it in a company wide pom you inherit from?

Nexus is not caching maven central plugins

Im using Maven 3.0.4 and Nexus 2.0.6. I have set up my settings.xml as the Nexus instruction show for using a single repository.
I get the error below when maven tries to run maven -U clean.
[ERROR] Plugin org.apache.maven.plugins:maven-clean-plugin:2.4.1 or one of its d
ependencies could not be resolved: Failed to read artifact descriptor for org.ap
ache.maven.plugins:maven-clean-plugin:jar:2.4.1: Could not find artifact org.apa
che.maven.plugins:maven-clean-plugin:pom:2.4.1 in nexus (http://localhost:8081/n
exus/content/groups/public) -> [Help 1]
If I remove the nexus mirror from the settings and go directly to maven central the command works. The settings for the maven repo in nexus show that it is in service and it is in the public group (its listed last).
I am not behind a proxy to access the internet.
Here is my settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<offline>false</offline>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
<profile>
<id>maven-central</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://repo1.maven.org/maven2/</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://repo1.maven.org/maven2/</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>
Make sure the Central proxy repository is properly configured, and the proxied URL is http://repo1.maven.org/maven2/. Check that you can see cached artifacts at the repository's URL, should be http://localhost:8081/nexus/content/repositories/central/org/apache/maven/plugins/maven-clean-plugin/2.4.0/maven-clean-plugin-2.4.1.pom.
Make sure you have a Central proxy at all, is there anything listed at http://localhost:8081/nexus/content/repositories/central/.
If you're behind a proxy, you can configure the proxy under the Default HTTP Proxy Settings (optional) section in the Administration->Nexus pane.
Then, make sure the Public Repositories group repository is configured to include the Central repository in its list of included repositories.
If everything looks fine so far, check the logs, maybe there's a helpful message in there.
Try downloading this directly through a web browser:
http://localhost:8081/nexus/content/groups/public/org/apache/maven/plugins/maven-clean-plugin/2.4.1/maven-clean-plugin-2.4.1.pom
If this doesn't work check the sonatype-work/nexus/logs/nexus.log file for more information about the failure.
I had the same symptom as the OP (Nexus was not mirroring an artifact) and found that it was caused by a route definition.
For example, you have an artifact org.blabla:blabla-api:1.0 which is in Maven Central. However you have set up a route matching .*/org/blabla/.* that forces any matching requests to look only in the proxied repository blabla-public ... but unfortunately blabla-public doesn't contain that particular artifact.
Solution: either update the route to add Central to the list of repos used by the route, or delete the route.
(This probably wasn't the cause for the OP, but I'm posting it in case it helps any other visitors.)

Resources