Scenario:
New enterprise policy forbids from using public repositories. For every dependency used (which are a lot) we have to provide to the devOps team the artifact group, id, version and from which repository it came.
We have used more than one external repository on our code base. Since all the artifacts are now on the local repository, we don't know where they came from.
What I was able to do:
I can use
mvn dependency:list -DoutputAbsoluteArtifactFilename=true -DprependGroupId
on each project to get artifact group, id, version.
And mvn dependency:list-repositories to get the repositories used in that project
What's missing:
Knowing which artifact is from which repository.
Thanks in advance
I am afraid this will not be possible in maven. Prior to maven 3.0.x, maven did not even kept track of remote repositories. After Maven 3.0.x, it moved to eclipse aether. Eclipse aether create _remote.repositories file for every repository which has repository information. It contains repository id which is defined in your settings.xml file. You could,
Delete your local repo contents.
Run following command,
find . -name _remote.repositories -exec grep "^[^#;]" {} \; > repo_info.txt
This will atleast give you the mapping.
Make a list of all external repositories that you use. Furthermore, take the list of all groupId, artifactId, version (Either by dependency:list each project or by scanning the local repository). Now for each dependency, ask the external repositories via REST if they contain this dependency.
If you start with MavenCentral, you have a high chance that you find your dependency in the first try.
Related
We have a dependency third-party library that is available online in jar form, but it is not in Maven Repository, or known to be in any other repository.
How can we use pom.xml to auto-retrieve this dependency, based on a URL?
We don't want to store it in our Git repo, because that's A Bad Thing.
The idea here is that when people check out the project, they can use their IDE Maven integration (or just mvn command line tools) to download all the dependencies. So we would want to be able to also download this other third party dependency just like all the ones in Maven repo.
I have not been able to come up with an answer to this based on searches -- all solutions seem to be "download it first and create a local repo." Obviously Maven can download from the Internet, since that's how it connects to Maven Central and other repos. So I don't see why it cannot download arbitrary URLs that present packages in recognizable formats.
Long term, the best solution is to use your own artifact repository like Nexus, Artifactory or Archiva.
All of these have a manual upload function that you can use to set the groupId, artifactId and version, so you can then refer to the artifact as usual.
If you want to go really low tech, I think you can just put some machine's local repository behind an Apache, provided you grant read/write access.
Then you need to add your new repository in the Maven settings.xml file, as described here.
Maven uses the coordinates to navigate the repository (which has a specific layout) and verify artifact checksums for corruption/tampering using metadata files in specific locations of the repo.
AFAIK this is similar to other package management systems like APT and RubyGems that use repo manifests and don't allow arbitrary URL downloads.
Skipping the repository manager
If you really don't want or can't use a repository manager, you can always download the artifact and manually install it using the Maven Install Plugin:
mvn install:install-file -Dfile=your-artifact-1.0.jar -DgroupId=org.some.group -DartifactId=your-artifact -Dversion=1.0
However, you'll have to do this on every machine that runs the build, every time that artifact needs to change.
I am using maven 3.0 with nexus as remote repository.
I am wondering, is there a way to download all the artifacts with the same groupId using maven without knowing the name of all the artifacts? In other words, is it possible to download all the artifacts from remote repository automatically by only mentioning the group Id?
For example, I want to download all the artifacts in group com.example.here to my local repository:
<groupId>com.example.here</groupId>
artifacts:
<artifactId>a1</artifactId>
...
<artifactId>an</artifactId>
and the version that I want is the latest.
You can use lftp (should be included in many UNIX distributions) to mirror a part of a Maven repository. Since the repository uses sub-directories based on groupIDs, you can use that do fetch all artifacts belonging to a certain group. E.g.
$ lftp https://repository.jboss.org/nexus/content/groups/public/apache-xalan/ -e 'mirror .'
Will grab everything under the apache-xalan groupId.
This may not work with every Maven repository. Some may only allow access to the directory structure at a certain depth or not at all.
Its not possible to do this since a dependency in maven needs to be referred by groupId:artifactId:version:classifier (optional).
One way I can think of is to create an assembly project that has dependencies on all the projects you want to download and the version optionally set to LATEST. You can use maven assembly plugin to create an archive from the dependency sets.
I can't seem to run mvn -o package because it complains with
The repository system is off line but the artifact
com.liferay.portal:util-bridges:jar:6.1.20 is not available in the
local repository.
But I checked my local repository and that artifact does exist there. I also tried the solution of setting updatePolicy to never in the settings.xml file but that failed to work.
Prior to Maven 3.0.x, Maven did not track the origin of files in the local repository.
This could result in build issues, especially if you were building something that listed the (now dead) very borked java.net2 repository... Not only did that repository change released artifacts (extremely bad and evil practice) but it also published artifacts at the same coordinates as artifacts on central but with different content (unbelievably evil)
So you could have the build work (because you had commons-io:commons-io:2.0 from central) wipe your local repo and the build fails (because you now get commons-io:commons-io:2.0 from java.net2 which was a completely different artifact with different dependencies in the pom) or vice versa.
The above situation is one of the drivers for using a maven repository manager, because that allows you to control the subset of a repository that you expose downstream and the order in which artifacts are resolved from multiple repositories (usually referred to as routing rules)
In any case, when maven switched to Aether as the repository access layer, the decision was made to start tracking where artifacts come from.
So with Maven 3.0.x, when an artifact is downloaded from a repository, maven leaves a _maven.repositories file to record where the file was resolved from. If you are building a project and the effective list of repositories does not include the location that the artifact was resolved from, then Maven decides that it is as if the artifact was not in the cache, and will seek to re-resolve the artifact...
There are a number of bugs in 3.0.x though... The most critical being how offline is handled... Namely: when offline, maven 3.0.x thinks there are no repositories, so will always find a mismatch against the _maven.repositories file!!!
The workaround for Maven 3.0.x is to delete these files from your local cache, eg
$ find ~/.m2/repository -name _maven.repositories -exec rm -v {} \;
The side effect is that you loose the protections that Maven 3.0.x is trying to provide.
The good news is that Maven 3.1 will have the required fix (if we can ever get our act together and get a release out the door)
With Maven 3.1 when in offline mode the _maven.repositories file is (semi-)ignored, and there is also an option to ignore that file for online builds (referred to as legacy mode)
At this point in time (June 1st 2013) the 4th attempt to cut a release that meets the legal and testing requirements is in progress... So, assuming that the 4th time is lucky, I would hope to see 3.1.0-alpha-1 released in 3-4 days time... But it could be longer given that we want to give the changes in 3.1 enough time to soak to ensure uses builds don't break (there was a change in an API exposed (by accident-ish - the API is needed by the site and dependency plugin) that plugin authors have depended on (even though they shouldn't have) so there is potential, though we think we have all the bases covered)
Hope that answers your question (and maybe a few more you didn't know you had ;-) )
I also had to remove _remote.repositories in the same way as the _maven.repositories described above. I'm using Maven 3.1.1
find ~/.m2/repository -name _remote.repositories -exec rm -v {} \;
I had this issue when i was using apache-maven-3.0.4, the issue is gone right after i move to apache-maven-3.3.1.
I had this issue in Ubuntu Linux when I've installed local artifacts via a shell script. The solution was to delete the local artifacts and install them again "manually" - calling mvn install:install-file via terminal.
I've refactored all repository configuration out of my various projects into a parent pom that I've created for the specific purpose of having a single point of configuration for stuff like repo's and distribution management. This is supposed to be best practice as I understand it. In deployed my parent pom to the nexus server, and then tried to run my child projects. They can't find my parent pom . . . this kind of makes sense to me since, wihtout the parent pom they don't know about the nexus repo . . . seems like a chicken and egg kind of thing? Am I missing something obvious?
It's true that your project needs to know where to find the repositories to download its dependencies. But before going to external repositories, Maven will check your local repository to see if the artifacts it needs are in there. Does your local repository contain the parent pom? If not, you can add it by running
mvn install
on the parent pom. The problem may have been caused by deploying the parent pom directly to Nexus, bypassing your local one. You can avoid this in future by deploying using
mvn deploy
This will first install the artifact locally, and then deploy it to the external repository (Nexus, in your case). More details here: http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
If you're in a situation whereby your parent pom is built and deployed by others, this won't help. You can either specify the repositories in your project's pom, or in your local settings.xml. A common approach is to expect all developers to include a repository definition in their local settings.xml which points to your Nexus repository, using it as a mirror for all other repositories. You can then configure each external repository you need in Nexus, and let it retrieve any dependencies you need for you. I'm not familiar with Nexus, but more details on mirroring can be found here: http://maven.apache.org/guides/mini/guide-mirror-settings.html
You must maintain the repositories definitions for each child (maven module), this is main best practice in a build process: "Make the build portable".
Any developer/system should be able to build any module without source dependencies to the parent or other modules, only with the repository reference.
I have a few dependencies like this. For example : jdic
In my pom.xml, I defined a dependency
<dependency>
<artifactId>jdic</artifactId>
<groupId>jdic</groupId>
<version>0.8.6</version>
</dependency>
And I have a remote repository (internal server): eg http://repo/thirdparty/
And jdic can be found in http://repo/thirdparty/jdic/jar/jdic.jar
As you notice, the naming is not conventional groupId:artifactId:version, instead its just jdic.jar
so when I run maven compile
mvn clean compile
maven tells me that it cannot resolve dependencies.
I'm aware that we can just download those jars to the local repository .m2/repository and run
mvn compile (ie without clean)
and it the jar will not be a problem. But is there any other way that I can make it retrieve from the remote despite its unconventional name and lack of metadata/pom info for those dependencies?
I already have a mirror to this internal repository that overrides the central
You could install the jdic in your internal/mirror repository with the version (jdic-0.8.6.jar).
If you are using a repository manager and you uploaded the jar to it, it would automagically create a pom for it as well as ensure the dependency was created with version, as per maven convention.
A remote Maven repository is NOT any web server putting files in any way you want. There are lots of convention to follow. One of them is the way to represent the version (which is one essential element of an artifact). With your "remote repository" it is clear that there is no way Maven can find the artifact.
Setup a real Maven remote repository to host such files. You can have a look in Nexus and Artifactory.