I am trying to understand maven a little more. How is maven able to download the dependencies of the main dependency of the application? For example assuming my application has main dependency like this:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.7.0</version>
<scope>provided</scope>
</dependency>
Now, when maven downloads this jar , it downloads the dependencies for this jar as well. For example, see the screen shot below:
As can be seen, maven has not only downloaded the hadoop-hdfs-2.7.0.jar but also all it dependencies.
Now, my questions is how maven knows what are the dependencies for the "top-level" dependency, that is in this case the "top-level" dependency is hadoop-hdfs, so what all jars it has to download for this?
I see this as well in the .m2/respository for hadoop-hdfs:
I opened the .pom file, the contents are (partly):
<project>
....
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.7.0</version>
<description>Apache Hadoop HDFS</description>
<name>Apache Hadoop HDFS</name>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-annotations</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-auth</artifactId>
<scope>provided</scope>
</dependency>
<dependencies>
...
</project>
What is this hadoop-hdfs-2.7.0.pom ? Does this file give information to maven what are the dependencies to be downloaded for hadoop-hdfs-2.7.0.jar?
Can anyone help me clear these things?
First of all you are right, the hadoop-hdfs-2.7.0.pom tells Maven
about the libraries that hadoop depends upon. But, when using hadoop
as a dependency in your project, maven uses the below strategies to
finalize the list of dependencies in addition to using the
hadoop-hdfs-2.7.0.pom.
If a dependency is specified with groupid, artifactid and version in the current project under the dependencies tag, it takes the first
precedence. This is how hadoop-hdfs got added in your project.
Dependency Management takes the next precedence. When a dependency is specified only with group and artifact id's under dependencies tag
but at the same time, the dependency is defined under
dependencyManagement tag with version and transitively inside hadoops pom.xml also,
the one under the dependencyManagement tag will be given preference.
Dependency Mediation takes the last precedence. Dependencies are resolved using dependency mediation. Meaning, in your case the
dependencies mentioned inside hadoop-hdfs-2.7.0.pom are the transitive
dependencies (indirectly depends on these dependencies since your
dependency "hadoop-hdfs" requires it) of your project and this process continues
recursively until all child dependencies are resolved.
Note: There are other features such as excluding dependencies, marking
one optional and importing a list of dependencies. But they are used
sparsely. More information with examples can be found in the below URL
[https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management][1]
Related
My project is a fairly large project consisting of many maven modules (but not microservices). I was trying to do Moving from spring to spring-bom on WAS but seems lot of clashes in versions. So for example one of my modules is using commons-collectionsversion 2.6.0 and my current project is using 3.2.2. I want the same jar to be used across. Since its more of a migration project I cannot do changes in container or repository changes at this time. I should only make sure that all the version are compatible with each other. My plan :
I want to include a dependency which is with in some other dependency
into the current pom as a dependency.
Also I want other jars in this pom (which exists as a dependency) to included the dependency
Is there anyway to do it?
I didn't completely understand your question, but the can help you to define a cross-module dependency version, as long as you place it in the parent-pom file.
<dependencyManagement>
<dependency>
<groupId>com.group</groupId>
<artifactId>project-1</artifactId>
<version>1.0.0</version>
</dependency>
</dependencyManagement>
and then define the dependency in the relevant module without providing it a version (it will be inherited from the parent-pom's <dependencyManagment> tag:
<dependencies>
<dependency>
<groupId>com.group</groupId>
<artifactId>project-1</artifactId>
</dependency>
</dependencies>
I thought that the order of Maven dependencies doesn't matter before and regard this as a pro of it. And this is my old pom.xml's dependencies:
<dependencies>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet</artifactId>
<version>2.19</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
<version>4.1.7.RELEASE</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.ext</groupId>
<artifactId>jersey-spring3</artifactId>
<version>2.19</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-moxy</artifactId>
<version>2.19</version>
</dependency>
</dependencies>
It works well, and today I wanna move spring dependency to the bottom so that those jersey related can be together. However then I can no longer make it working, my Jetty complains:
[ERROR] Failed to execute goal org.eclipse.jetty:jetty-maven-plugin:9.3.0.M1:run (default-cli) on project mtest: Execution default-cli of goal org.eclipse.jetty:jetty-maven-plugin:9.3.0.M1:run failed: A required class was missing while executing org.eclipse.jetty:jetty-maven-plugin:9.3.0.M1:run: org/apache/commons/logging/LogFactory
That is really confusing, so do I have to concern about dependencies order? How do I know the correct order?
The order of dependencies does matter because of how Maven resolves transitive dependencies, starting with version 2.0.9. Excerpt from the documentation:
(...) this determines what version of a dependency will be used when multiple versions of an artifact are encountered. (...) You can always guarantee a version by declaring it explicitly in your project's POM. (...) since Maven 2.0.9 it's the order in the declaration that counts: the first declaration wins.
To expand upon the other answer (which states that the declaration order affects Maven's dependency mediation for transitive dependencies), there are a few tools you can use:
mvn dependency:tree [-Dscope=[runtime|test]] will show you what dependencies will be available for the selected scope. See here for details
mvn dependency:build-classpath gives you order in which dependencies are available on your classpath (if two or more classpath entries have the same class, the earlier one wins). See here for details
I don't know much about your situation, but it's often the case that you wind up with the wrong version of 1 or more jars at compile/runtime. Declaring your own version of the library in question or locking down the version with <dependencyManagement> are options here.
Now to answer your other question - how do you know what the right order is when declaring dependencies?
My suggestion - the right declaration order is the one that gets you the versions of the dependencies you want, in the order you want them in. Use the tools above to check your dependencies, and tweak the declared order if necessary.
Note that most jars contain disjointedly-named classes, so the exact order in which jars appear on your classpath is usually not that important. The only exception I've noticed is some jars in SLF4J which intentionally shadow classes from the other logger libraries it's intended to replace.
I have the following dependency (only so far) pom.xml
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-jpa</artifactId>
<version>1.6.2.RELEASE</version>
</dependency>
This dependency obviously depends on other "dependencies" via it's pom.xml... when maven finished downloading these dependencies I noticed that it didn't grab the latest version of the Spring stuff (4.0.6.RELEASE)... it grabbed a 3.2.x version.
How can I force maven to grab the latest version of the Spring stuff? Do I need to explicitly modify my pom.xml to include all the dependencies or is there some "magic" I can use for this?
Thanks.
Spring "Bill Of Materials"
Salvation may come from special "bill of materials" POMs supported by Maven and published by Spring. Quoting from Maven "Bill Of Materials" Dependency in their manual:
It is possible to accidentally mix different versions of Spring JARs when using Maven. For example, you may find that a third-party library, or another Spring project, pulls in a transitive dependency to an older release. If you forget to explicitly declare a direct dependency yourself, all sorts of unexpected issues can arise.
To overcome such problems Maven supports the concept of a "bill of materials" (BOM) dependency. You can import the spring-framework-bom in your dependencyManagement section to ensure that all spring dependencies (both direct and transitive) are at the same version.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-framework-bom</artifactId>
<version>4.0.6.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Would this work for you?
Looking at the spring-data-jpa artifact pom file, we can see that it has a parent called spring-data-parent with current version 1.4.2.RELEASE. It's pom describes dependencies and their versions. Currently spring version is at 3.2.10.RELEASE
One way you can possibly accomplish what you want is to add explicit dependency on spring artifacts. But you would still have to define their versions.
I'm new to Maven, I try to use Maven with Spring, Hibernate in my project. After go though the Spring and Hibernate reference, I found that "there is no need to explicitly specify the dependent liberaries in POM.xml file for such Apache commons liberaries".
My questions is that : If my other parts of project refer to Apache commons liberary, such as commons-io, SHOULD I explicit specify this dependency in POM.xml file?
You should define those dependencies in Maven which your project is using. For example, even though some library depends on commons-io but if your code needs this then you should directly define commons-io in your pom.xml
You should not worry about the dependencies of the libraries you have defined in your pom.xml. Maven will do that for you.
Maven is used to avoid the issue of having to run down JAR files that are dependent on other JAR files. Of course you do not HAVE to use maven to do this, but you should. Maven will automatically download the dependent JAR files of the JAR file you require. THe hibernate-entity manager JAR file, for example, has over 100 dependencies and maven does the work for you.
Anyway,even if you do add the commons-io file to the build path/classpath of the maven project,and then update the project configuration, maven will kick it out.
You can provide a lib name on a site like mvnrepository.com to see what it depends on (e.g. take a look at a section called "This artifact depends on ..." in case of spring-webmvc library). Those dependencies (which your artifact depends on) are called transitive dependencies. You don't have to specify these in your pom.xml as maven will resolve them for you.
For the sake of readability you should only state those dependencies in your module that you rely on directly. You want JUnit to test your software, only declare JUnit; you need hibernate to use ORM, declare hibernate, and so on. Leave the rest to Maven.
And most of the time you should state what you intend to use in the very module you want to use it in. So if you want to use a dependency in more than one module, consider moving it into a dependencyManagement block in a parent pom and referencing it from there in the module you want it in.
parent pom.xml
<dependencyManagement>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
<scope>test</scope>
</dependency>
</dependencies>
</dependencyManagement>
child pom.xml
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
</dependencies>
This guarantees you version-stability and still allows you to find out what a module uses by only looking in it's pom (and not all over the place).
I have added around 100 jars in my local Apache Archiva. Now i will want to add all these dependency jar to my project Pom.xml file.
Can it possible to add all these dependency by single Copy-paste? Right now i have to copy each individual dependency from Apache Archiva and paste into my project pom.xml file.I have to copy-paste these lines in my Pom.xml file for each jar which is very tough task.
<dependency>
<groupId>org.csdc</groupId>
<artifactId>dom4j</artifactId>
<version>1.6.1</version>
</dependency>
It's very unlikely that you need all 100 jars as direct dependencies. In maven, you have to list your direct dependencies - one by one, yes. However, you don't need to list your transitive dependencies because maven will manage that for you. This is one of the most fundamental improvements over older manual classpath management java building.
No All dependency of all jar,
because of in that jars some of the dependency have same group Id ,
so that have fetch all the jars that included.
some of the dependency is writing in pom.xml file
for example code is
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-annotations</artifactId>
<version>3.4.0.GA</version>
</dependency>
the above dependency fetch all jars of related to hibernate-annotation
- hinernate-annotation
- hibernate-common-annotation
- hibernate-core jar files to be fetched.....