Maven private remote repository setup - maven

I'm trying to set up a private remote repository for our team. Now the repository serves the following with so far I have tried -
When a 'mvn compile' is issued, then it searches dependencies in "~/.m2". If it is not found there then it goes to the remote private repository and when the dependencies still absent here then it goes to the central repository.
Before compilation, I have to put all dependencies in our private (and remote) repository along with checksum and metadata.
When we need an artifact(eg - spring-mvc), we need to put it in the remote private repository along with all dependencies of that artifact (spring-mvc). In this case "spring-mvc" is downloaded from our remote private repository. But the dependencies of "spring-mvc" are not downloaded from our remote private repository. They are downloaded from the central repository.
Now what I am trying to do are -
Configuring a private remote repository 'R' in such a way that I do not have to put all the dependencies to it by hand. When a "mvn compile" issued then first of all; dependencies are searched in "~/.m2" if not found then it goes to the private remote repository 'R'. If the dependencies are found in 'R' then "~/.m2" will get it from 'R'. If the dependencies are not found in 'R' then these dependencies are downloaded from the central repositories and keep the dependencies in 'R' for further uses. After that '~/.m2' will get them from 'R'. Here I do not need to put the dependencies in our remote private repository 'R' by hand.
'~/.m2' will get all dependencies of "spring-mvc" (which is the main dependency mentioned in my project's pom.xml) from the remote private repository 'R' and 'R' will get them from the respective central repository.
Can anyone suggest some way/or tutorial for reference to meet these two above targets? Thanks in advance.
EDIT : I have tried with Nathaniel Waisbrot's answer and nexus-2.7.0-06-bundle. After hours of endeavors, I am able to set it up while I am using jre-7. With jre-6, nexus cannot be started. But our project is deployed with JDK-6. We do not want to the change the current jre version. With jre-6 apache-archiva is OK. But I am not sure whether I can achieve all the goals in previous section. Is there any archiva user/expert who can tell me whether I can meet these goals in the previous section?

I have setup nexus for mirroring my maven local repository (at ~/.m2/repository.) Since this post is still visited by a lot of people, I think it would be helpful for others if I share how I configured nexus as a repository manager. This procedure works perfectly for me in Ubuntu 12.04. Here it is -
1. Download nexus-2.11.1-01-bundle.tar.gz or latest version of nexus oss.
2. Extract the tar file in you home directory-
$ tar -xvf nexus-2.11.1-01-bundle.tar.gz
Now you will get two directories - nexus-2.11.1-01 and sonatype-work in your home directory.
3. Copy these two directories to /usr/local/ directory (they can be copied to other place) -
$ cp -r nexus-2.11.1-01 /usr/local/
$ cp -r sonatype-work /usr/local/
The executable/configuration files related to nexus are stored in nexus-2.11.1-01 directory and the jar file mentioned in pom.xml are stored in sonatype-work directory.
These jar files are mirror of your ~/.m2/repository. First time you issue a mvn package command then all the jars are stored here. After then when you issue mvn package again then all jars are downloaded from the nexus repository instead of downloading from the central repository.
4. Go to the /usr/local/ directory -
$ cd /usr/local/
5. Create a link to nexus-2.11.1-01 -
$ sudo ln -s nexus-2.7.0-06 nexus
6. Now to run nexus type the following in terminal -
$ bash nexus/bin/nexus console
Here nexus is attached with your console. If you close your console then the nexus server will be terminated. When you are trying to run nexus for a Ubuntu server machine then you may use screen.
Note: While trying to run nexus by using the command above there may occur 2 problems. If you do not found any problem then skip next 2 steps (step - 7 and 8)
7. First problem may occur due to insufficient permission. Read the error message and take necessary steps. But as a quick solution you may do this -
$ sudo chmod -R 777 nexus-2.11.1-01/
$ sudo chmod -R 777 sonatype-work/
8. If you are using any jdk version lower than java 7 than the following error message may be shown -
wrapper | Launching a JVM... wrapper | JVM exited while loading the
application. jvm 1 | Exception in thread "main"
java.lang.UnsupportedClassVersionError:
org/sonatype/nexus/bootstrap/jsw/JswLauncher : Unsupported major.minor
version 51.0
In this case use jdk7 to run the command mentioned in step 6. In ubuntu its pretty easy. Assuming you have two jdk - jdk6 and jdk7. Your project runs on jdk6. Then only for running nexus you may do this from your terminal (assuming your jdk7 in /usr/lib/jvm/jdk1.7.0_45 directory) -
$ export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_45
$ export PATH=$JAVA_HOME/bin:$PATH
9. Now in browser type the address - http://localhost:8081/nexus/. If step 1 to 6 (if errors occurred then step 1 to 8) are done perfectly you may successfully find the login screen. The default login user name is - admin and password is - admin123
10. Stop nexus. Just close the terminal or press Ctrl+C at step 6's terminal. In you ~/.m2 directory create an empty file named - settings.xml. Copy the following content into this settings.xml file -
<settings>
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>
11. And add these following lines in your project's pom.xml file -
<distributionManagement>
<snapshotRepository>
<id>my-snapshots</id>
<name>My internal repository</name>
<url>http://localhost:8081/nexus/content/repositories/snapshots</url>
</snapshotRepository>
<repository>
<id>my-releases</id>
<name>My internal repository</name>
<url>http://localhost:8081/nexus/content/repositories/releases</url>
</repository>
</distributionManagement>

Sonatype Nexus does everything you want, needs very little configuration for what you're asking, and has a nice interface. There's some details on the basic setup under the question "Maven Internal Repository, Is it Really This Hard?"
Basically, you can set up Nexus as a caching repository: your Maven client will only visit the internal repository. When the internal repository can't find an artifact, it goes to Central (or any other external repositories you define), fetches it for you, and stores it for later.
This is cool because if you want to transition from a grab-whatever-dependencies-you-want free-for-all to a locked-down environment, Nexus makes it easy. You set it up to pull from Central, run your build from a clean machine (forcing Nexus to ingest all the current Central dependencies) and then turn off the proxy-repo feature (so any new/changed dependencies will be managed by devops).

What you've described is pretty much the default configuration of most repository managers, including Archiva. Does the following help in getting it set up to use from Maven?
http://archiva.apache.org/docs/1.3.6/userguide/using-repository.html

Related

Why are settings.xml and pom.xml are not directing Maven to deploy to Artifactory?

I am trying to point Maven to a local Artifactory (OSS) repo (both on my Mac) so I can build code provided by a vendor. I have created an empty Maven local repo and virtual repo in Artifactory, and can use Generate Settings to create the snippets. My ID and encrypted password seem to be working OK in ~/.m2/settings.xml
I have updated the code's pom.xml file and that seems to be causing Maven to resolve from Artifactory.
No matter how I configure in ~/.m2/settings.xml or in the master settings.xml or with -Dmaven.repo.local, Maven wants to deploy to a disk path to the local repo.
If I specify a partial path, Maven says "Using local repository at " a path consisting of the code's home + the path I said.
If I specify a path starting with a / (root), Maven says "Using local repository at /whatever"
I want to specify "https://localhost/artifactory/reponame" so Maven will deploy there. Artifactory generates snippets that contain that kind of URL. But since a URL doesn't start with a /, Maven insists on prepending the code home location to the URL, which of course is nonsense.
There must be some combo of and or <distributionManagement> <repository> <id>central</id> <name> <url>https://localhost/artifactory/... that makes this work. What am I missing?
pom.xml snippet:
<distributionManagement>
<repository>
<id>central</id>
<name>LocalMac-releases</name>
<url>https://localhost/artifactory/TBX</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>LocalMac-snapshots</name>
<url>https://localhost/artifactory/TBX</url>
</snapshotRepository>
</distributionManagement>
Master settings.xml snippet: localRepository is commented out so it takes the default. When commented in whatever I enter does take effect... combined with ${user.home}.
<!-- localRepository
| The path to the local repository maven will use to store artifacts.
|
| Default: ${user.home}/.m2/repository
<localRepository>/path/to/local/repo</localRepository>
-->

Setup AEM Adobe CQ5 6.1 project to build/install offline

I'm new to CQ5 and looking for steps/settings I may need to do to setup an AEM adobe CQ5 6.1 project to build/install offline (not connected to internet).
I've to use our internal network Nexus (which has lot of general dependencies available except AEM related).
I've to use Maven & Java7.
Looking for possible issues/resolutions, steps & any helpful info.
Thanks all for your inputs.
After a detailed investigation, this is now resolved. Just want to share the findings so that it may help others.
The content-package-maven-plugin which was uploaded to our local nexus got corrupted.
Interestingly maven was not throwing any errors when i was trying mvn install command on my AEM project. Rather it was giving NoClassDefFound error for a further dependency of content-package-maven-plugin.
From the output of, mvn dependency:resolve-plugins, it was confirmed that the dependencies are not fully resolved for particular this plugin JAR, I tried to manually download the artifact from nexus & try to open/unjar it.
The artifact pom was corrupted and hence I uploaded the fresh artifact in nexus, and this time all went smooth.
My AEM project is now building fine with local nexus dependencies.
a typical AEM project does have lots of dependencies. There are some dependencies for the AEM platform(including granite, sling, osgi etc). These dependencies are downloaded from the adobe public repositories, unless you have a nexus repository in your company where all these dependencies are available.
If your Organization specific Nexus repository can connect to internet and download dependencies (AEM and non AEM related) atleast for the first time , then you do not need to connect to internet from your local sand box.
Theoretically, your organization specific nexus repo also might not be
connected to the internet and all the JARs can be uploaded to Nexus
manually. But, that would be the last thing I will do in my life !
Your project can download all the dependencies from the Nexus repo(only) in multiple ways.
Configure the repository configuration in your pom.xml file to point
to your internal Nexus repo.
<repositories>
<repository>
<id>internal</id>
<name>Internal Public Repository</name>
<url>URL to the public repository of your internal nexus</url>
<layout>default</layout>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>internal</id>
<name>Internal Public Repository</name>
<url>URL to the public repository of your internal nexus</url>
<layout>default</layout>
</pluginRepository>
</pluginRepositories>
Change your Maven settings (settings.xml) to use your internal Nexus
every time it downloads dependencies (Recommened).
settings.xml can be found in your <user>/.m2 directory or <maven-installation>/conf
<mirrors>
<mirror>
<id>public</id>
<mirrorOf>*</mirrorOf>
<url>URL to the public repository of your internal nexus</url>
</mirror>
</mirrors>

Local Nexus Repository Acting Like A Proxy For Some Maven Artifacts

I am using my own version of the Nexus web app repository installed on my local machine. I have Nexus configured with only one repository, the one where I store my snapshots:
http://localhost:8081/nexus/content/repositories/MySnapshots/
Note that after the Nexus installation I removed all the default repositories and added just my own. (Perhaps this was a bad idea?)
When I do a mvn clean install I noticed that some of the 3rd party artifacts are downloading straight from the remote repository. For example, here is one of the output lines from the build:
Downloading: http://repo.maven.apache.org/maven2/com/sun/org/apache/xml/internal/resolver/...
The strange thing is that I see other artifacts are going through my local Nexus to ultimately get to the artifact:
Downloading: http://localhost:8081/nexus/content/repositories/MySnapshots/org/apache/maven/wagon/wagon-provider-api...
Notice how the first part of the download url is my local repository but everything after MySnapshots is from apache.org.
It's almost like my Nexus repository is acting like a proxy to maven.apache.org for some artifact downloads but for others it goes straight to the source.
Can anyone tell me why this is happening?
I would't be bothered so much by this if all my builds succeeded all the time but sometimes, when I am compile large projects, I get build failures due to not being able to find an artifact.
For example, when I try to build another project that depends on eclipse jdt stuff I get the following error:
Downloading: http://localhost:8081/nexus/content/repositories/MySnapshots/eclipse/jdt/core/eclipse.jdt.core
Could not find artifact eclipse.jdt.core:eclipse.jdt.core
I am not sure if this means that my Nexus is not configured properly or if there really is no artifact eclipse.jdt.com. If the downloads were not going through my local Nexus repository I would then investigate the pom/settings.xml files. Instead this makes me wonder if it's due to my Nexus configuration.
If you would like to see my settings.xml for Maven and my pom file for the project I am building when I see this you can view them here:
settings.xml: http://pastebin.com/NvLr5bEA
pom.xml: http://pastebin.com/PJ0P3RaK
If you like to use the local nexus as a proxy as usual than you have to configure the settings.xml like this:
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/nexus/content/groups/public</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>nexus</id>
<!--Enable snapshots for the built in central repo to direct -->
<!--all requests to nexus via the mirror -->
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<!--make the profile active all the time -->
<activeProfile>nexus</activeProfile>
</activeProfiles>
The tricky thing is the mirror thing which reroutes every call to the configured nexus instance.
Things you mentioned like eclipse parts can be problematic, cause only a few artifacts are available via maven central. Furthermore you should leave the defaults like maven central, release repository and the snapshots repository unchanged, cause these are the repository you need.
I don't think its a proxy issue , upto my understanding for the first case when it is downloading from Maven Central Repo , it might be possible that same artifact is not available in your nexus repository , that's why it is going to Maven Central Repo.
In the second case it is available in your nexus so reactor didn't try to download it from Maven Central Repo.

Maven: unresolvable build extension

I've been looking at google and nothing really points to this problem. When I run "mvn clean install" it returns the following error.
[ERROR] Unresolveable build extension: Plugin
org.sonatype.flexmojos:flexmojos-maven-plugin:3.8 or one of its
dependencies could not be resolved: Failed to collect dependencies for
org.sonatype.flexmojos:flexmojos-maven-plugin:jar:3.8 ()
I'm trying to figure out how to import the maven plugin flexmojos but there are no clear directions on how to do this.
How would I import this plugin into my project?
The dependency you are looking for does exist.
To troubleshoot this problem further we'd need to see your POM and the rest of your build output.
Taking a stab in the dark:
Is this the first time you're running this build on this machine? If so, a very common "gotcha" is a corporate firewall preventing access to Maven Central. The solution in this case is setup a Maven repository manager like Nexus, or configure Maven to use a HTTP proxy.
I had a similar problem. I set up the proxy in Eclipse then I discovered I also set up the proxy in my settings.xml. I deleted the proxy from settings.xml and all worked out.
I hope my situation will help!
I solved this by adding another profile parameter from my pom.xml to my mvn command, e.g. "-Pprofile-name" pointing to a non-Maven repo definition embedded inside that profile, since Maven might be looking for repo definitions to be standing alone in a settings.xml, which isn't always the case.
I resolved this by adding a settings.xml file in the ~\.m2 directory, with the appropriate configuration pointing to our internal libraries.
If this Problem is displayed in the Eclipse Environment, this is because the m2e Connector tries to download the Plugins in to your ~\.m2\ repository
to solve this, open your Eclipse Settings: Window->Preferences and go to the ->Maven->User Settings Section.
Check if either Global Settings or User Settings is connected to the settings.xml File that your Maven uses.
Generally: Maven or your m2e connector tries to download these plugins via the plugin-repositories configured in your settings.xml it's can't find them because the repository is unknown or not reachable because you are behind a proxy or so:
08.11.18, 15:54:47 MEZ: [WARN] Failed to build parent project for com.xxx.xxx.xxx:eclipse-plugin:1.0.0-SNAPSHOT
08.11.18, 15:54:47 MEZ: [WARN] Failure to transfer org.apache.maven.plugins:maven-site-plugin/maven-metadata.xml from http://repository.sonatype.org/content/groups/sonatype-public-grid was cached in the local repository, resolution will not be reattempted until the update interval of tycho has elapsed or updates are forced. Original error: Could not transfer metadata org.apache.maven.plugins:maven-site-plugin/maven-metadata.xml from/to tycho (http://repository.sonatype.org/content/groups/sonatype-public-grid): repository.sonatype.org
go to your settings.xml file and add:
Mirrors (in this example: central repository. Do so for any other repos accordingly e.g. tycho):
<mirror>
<id>central</id>
<name>Our mirror for central repo</name>
<url>http://<your host to>/nexus/content/repositories/central/</url>
<mirrorOf>central</mirrorOf>
</mirror>
Repository:
<repository>
<id>central</id>
<url>http://<your host to>/nexus/content/groups/public/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>daily</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</releases>
</repository>
this should fix your problems:
Eclipse Specific: if Errors prevails (e.g. unknown packaging) add lifecycle mappings via Window->Preferences->Maven->Discovery->Open Catalog and add Tycho Connector
Further you can add the lifecycle-mapping plugin that handles these lifecycle mappings in the eclipse environment: eclipse m2e lifecycle mappings
Maybe you end up here because you are running nexus for a long time.
I finally found the error Summary tab in the Repositories page.
At some point Maven Central decided to require https: and the URL listed in my configuration still was using http:.
Update the URL to use https:, Save and everything worked smoothly again!

Maven: Manually install a Jar from local folder and then go online and grab & install all its dependencies

In my project's pom file I have a dependency that I can see in Maven central repository, javax.enterprise:cdi-api:jar:1.0-SP4 but for some reason I am getting on some of my computers the following error while building the project (hence the build fails):
[INFO] Unable to find resource 'javax.enterprise:cdi-api:jar:1.0-SP4' in repository central (http://repo1.maven.org/maven2)
Any idea why I get this error only on some of my computers (I can say all the computers are using the same network)?
Why do I get this error at all? As far as I can see the jar IS in the maven central repository.
Even though maven could not locate/download the jar on some of the computers, I was able to download it manually through the browser on these computers. How could it be?
Okay, so I've downloaded the jar manually through the browser at one of the problematic computers. I want to install it manually to the local repository on this computer. So from the command line I do:
mvn install:install-file -DgroupId=javax.enterprise -DartifactId=cdi-api -Dversion=1.0-SP4 -Dpackaging=jar -Dfile=path/to/file
...but none of its dependencies has been downloaded as well. Is there some maven plugin to install the file to the local repository AND download & install all its dependencies to the local repository?
If so, please show me how to use it.
Please check in the computers where the jar are not downloaded if the settings.xml file present in .m2 folder is having the url of the jar needed. If not add them.
You are not able to download because maven searches in the urls defined in the settings.xml file only. So if the url is not present there, even if the jar is in the central repository it wont be downloaded.
In your browser you can navigate to that link and download. Maven wont be able to do it for the reason given in the second point
Ok, looks like you need to mention your repository path in your pom file, What is the url of the repository? is it local to your network or some open url? for example: http://repo2.mvn.org/???
Sample example would be like this::
<repositories>
<repository>
<id>codehausSnapshots</id>
<name>Codehaus Snapshots</name>
<releases>
<enabled>false</enabled>
<updatePolicy>always</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
<checksumPolicy>fail</checksumPolicy>
</snapshots>
<url>http://snapshots.maven.codehaus.org/maven2</url>
<layout>default</layout>
</repository>
</repositories>
If you could paste the pom file, it would be easier to help you.
To answer your 3 point: Check the settings.xml file in .m2\settings.xml, make sure that same content is present. Sometimes settings.xml are modified directly in maven installation directory. so you better check in that location also.
To Answer your 4th point: mvn install:install installs (in simple terms copies) an artifact (jar/ear/zip/any artifact that meant for distribution) into maven local repository or into remote repository. So in your command, you just tried to install only cdi-api jar.
Whenever you run mvn compile, mvn package, mvn install, maven will scan your pom file and downloads all dependency mentioned into your local repository. I would suggest you to look your local repository and see how is it creating directory structure.

Resources