I have added around 100 jars in my local Apache Archiva. Now i will want to add all these dependency jar to my project Pom.xml file.
Can it possible to add all these dependency by single Copy-paste? Right now i have to copy each individual dependency from Apache Archiva and paste into my project pom.xml file.I have to copy-paste these lines in my Pom.xml file for each jar which is very tough task.
<dependency>
<groupId>org.csdc</groupId>
<artifactId>dom4j</artifactId>
<version>1.6.1</version>
</dependency>
It's very unlikely that you need all 100 jars as direct dependencies. In maven, you have to list your direct dependencies - one by one, yes. However, you don't need to list your transitive dependencies because maven will manage that for you. This is one of the most fundamental improvements over older manual classpath management java building.
No All dependency of all jar,
because of in that jars some of the dependency have same group Id ,
so that have fetch all the jars that included.
some of the dependency is writing in pom.xml file
for example code is
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-annotations</artifactId>
<version>3.4.0.GA</version>
</dependency>
the above dependency fetch all jars of related to hibernate-annotation
- hinernate-annotation
- hibernate-common-annotation
- hibernate-core jar files to be fetched.....
Related
I am running storm jar in a cluster ,where I configured hadoop,kafka,storm cluster
when I run the jar in local mode it works fine ,when I run it on storm cluster, I am finding respective error in Storm UI:
java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosTicket(Ljavax/security/auth/Subject;)Z at
org.apache.hadoop.security.UserGroupInformation.<init>(UserGroupInformation.java:666) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:861) at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:820)
pom.xml
Click here to view POM file
After some google I found I found we have add hadoop auth jar.even after i finding same error
I think you're packaging an old Hadoop jar.
Take a look at the storm-hdfs POM https://github.com/apache/storm/blob/v1.0.6/external/storm-hdfs/pom.xml. When you use the Shade plugin, the jar you end up with will contain all your dependencies, including transitive ones brought in through direct dependencies. Storm-hdfs declares a dependency on a list of Hadoop jars. You need to make sure that you're declaring the same list of Hadoop jars in your POM if you want to use a different version of Hadoop from the default.
Specifically what's happening is that you haven't declared hadoop-auth in your POM, so your POM gets packaged with the default version of that jar (2.6.1). Since that version of hadoop-auth is incompatible with the other Hadoop jars (which are 2.9.1), you get an exception at runtime.
You should either exclude all Hadoop jars from your import of storm-hdfs and then put the jars you want to use in Storm's lib directory, or add the right versions of the Hadoop jars to your dependency list in your POM.
Edit:
I think I found your issue. You haven't set the scope of storm-core to provided. Since storm-core depends on hadoop-auth, and you haven't declared it explicitly, Maven will try to guess which version of hadoop-auth you need based on where the dependency appears in the tree. Since hadoop-auth appears as 2.9.1 through some of your Hadoop dependencies, but 2.6.1 through storm-core, you happen to get 2.6.1 put in your jar.
If you want to avoid this kind of thing in the future, you should use Maven's dependencyManagement block https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management.
i.e. you should add something like the following to your pom, and then remove the exclusions of hadoop jars.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>${hadoop.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-auth</artifactId>
<version>${hadoop.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>${hadoop.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
I have the below project structure under lib for project 2.
Project 2 (under lib)
- Maven dependencies (JARs)
- project 1 JAR (it has JARs in lib)
- Maven dependencies of project 1 (JARs)
I want to exclude all JARs under project 1 while preparing project 2.
I'm currently using the below in my POMs (both project 1 and project 2)
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
When building project 1, I want it to build as an executable JAR with lib.
However when building project 2, I want project 1 as only a compiled JAR (meaning with only class files and not having lib (JARs)).
Project 1 is included as a normal dependency in Project 2. Can anyone help me out?
If you want to exclude all dependencies of a given dependencies, use exclusions
https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html
and do it like
<dependency>
<groupId>sample.ProjectA</groupId>
<artifactId>Project-A</artifactId>
<version>1.0</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
Of course, you have to be sure that the missing transitive dependencies are not needed at runtime.
If I understand correctly project 1 builds a jar which has inside a lib directory with the dependencies.
You could in project 1 have 2 assembly executions, one the basic jar and another with dependencies and execute manifest, normally you use a different "classifier" for multiple artifacts of the same project.
Otherwise you would need to "repackage" the jars to remove the libs you don't want using some plugin like Shade.
I think the first option is much better, also by default if you publish it to a repository both jars will be published so other people can get one or the other.
I am trying to understand maven a little more. How is maven able to download the dependencies of the main dependency of the application? For example assuming my application has main dependency like this:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.7.0</version>
<scope>provided</scope>
</dependency>
Now, when maven downloads this jar , it downloads the dependencies for this jar as well. For example, see the screen shot below:
As can be seen, maven has not only downloaded the hadoop-hdfs-2.7.0.jar but also all it dependencies.
Now, my questions is how maven knows what are the dependencies for the "top-level" dependency, that is in this case the "top-level" dependency is hadoop-hdfs, so what all jars it has to download for this?
I see this as well in the .m2/respository for hadoop-hdfs:
I opened the .pom file, the contents are (partly):
<project>
....
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.7.0</version>
<description>Apache Hadoop HDFS</description>
<name>Apache Hadoop HDFS</name>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-annotations</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-auth</artifactId>
<scope>provided</scope>
</dependency>
<dependencies>
...
</project>
What is this hadoop-hdfs-2.7.0.pom ? Does this file give information to maven what are the dependencies to be downloaded for hadoop-hdfs-2.7.0.jar?
Can anyone help me clear these things?
First of all you are right, the hadoop-hdfs-2.7.0.pom tells Maven
about the libraries that hadoop depends upon. But, when using hadoop
as a dependency in your project, maven uses the below strategies to
finalize the list of dependencies in addition to using the
hadoop-hdfs-2.7.0.pom.
If a dependency is specified with groupid, artifactid and version in the current project under the dependencies tag, it takes the first
precedence. This is how hadoop-hdfs got added in your project.
Dependency Management takes the next precedence. When a dependency is specified only with group and artifact id's under dependencies tag
but at the same time, the dependency is defined under
dependencyManagement tag with version and transitively inside hadoops pom.xml also,
the one under the dependencyManagement tag will be given preference.
Dependency Mediation takes the last precedence. Dependencies are resolved using dependency mediation. Meaning, in your case the
dependencies mentioned inside hadoop-hdfs-2.7.0.pom are the transitive
dependencies (indirectly depends on these dependencies since your
dependency "hadoop-hdfs" requires it) of your project and this process continues
recursively until all child dependencies are resolved.
Note: There are other features such as excluding dependencies, marking
one optional and importing a list of dependencies. But they are used
sparsely. More information with examples can be found in the below URL
[https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management][1]
I'm new to Maven, I try to use Maven with Spring, Hibernate in my project. After go though the Spring and Hibernate reference, I found that "there is no need to explicitly specify the dependent liberaries in POM.xml file for such Apache commons liberaries".
My questions is that : If my other parts of project refer to Apache commons liberary, such as commons-io, SHOULD I explicit specify this dependency in POM.xml file?
You should define those dependencies in Maven which your project is using. For example, even though some library depends on commons-io but if your code needs this then you should directly define commons-io in your pom.xml
You should not worry about the dependencies of the libraries you have defined in your pom.xml. Maven will do that for you.
Maven is used to avoid the issue of having to run down JAR files that are dependent on other JAR files. Of course you do not HAVE to use maven to do this, but you should. Maven will automatically download the dependent JAR files of the JAR file you require. THe hibernate-entity manager JAR file, for example, has over 100 dependencies and maven does the work for you.
Anyway,even if you do add the commons-io file to the build path/classpath of the maven project,and then update the project configuration, maven will kick it out.
You can provide a lib name on a site like mvnrepository.com to see what it depends on (e.g. take a look at a section called "This artifact depends on ..." in case of spring-webmvc library). Those dependencies (which your artifact depends on) are called transitive dependencies. You don't have to specify these in your pom.xml as maven will resolve them for you.
For the sake of readability you should only state those dependencies in your module that you rely on directly. You want JUnit to test your software, only declare JUnit; you need hibernate to use ORM, declare hibernate, and so on. Leave the rest to Maven.
And most of the time you should state what you intend to use in the very module you want to use it in. So if you want to use a dependency in more than one module, consider moving it into a dependencyManagement block in a parent pom and referencing it from there in the module you want it in.
parent pom.xml
<dependencyManagement>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
<scope>test</scope>
</dependency>
</dependencies>
</dependencyManagement>
child pom.xml
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
</dependencies>
This guarantees you version-stability and still allows you to find out what a module uses by only looking in it's pom (and not all over the place).
I'm working on a multi-project, and right now I have a structure that resembles this (actually there are a couple of jar projects and a couple of war projects)
/myProj
|_______projA (jar)
| |____pom.xml
| |____target/jar files
|_______projB (war)
| |___pom.xml
| |___web-inf/lib/jarfiles
|_______projEar
| |___pom.xml
|___pom.xml
What I want to achieve, is to make projA and projB to read their dependences from a common shared folder, instead of keeping their own copy.
Actually, I don't really care where they read them from at compile time, but when I package my EAR file, I want each jar/war to appear just once, hence reducing the EAR size.
I've tried declaring the dependencies on the parent pom, declaring the dependencies as and some other things, but so far I haven't achieved this.
Is there an easy way to achieve this? Any simple maven plugin?
Thanks in advance.
You should be able to do this by adding the JAR as a dependency to your EAR's pom.xml:
<dependencies>
<dependency>
<groupId>com.mycompany</groupId>
<artifactId>myapp-web</artifactId>
<version>1.0-SNAPSHOT</version>
<type>war</type>
</dependency>
<dependency>
<groupId>com.mycompany</groupId>
<artifactId>myapp-utils</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>jar</type>
</dependency>
</dependencies>
...and specifying the dependency as provided in your WARs' pom.xml:
<dependency>
<groupId>com.mycompany</groupId>
<artifactId>myapp-utils</artifactId>
<version>0.0.1-SNAPSHOT</version>
<scope>provided</scope>
</dependency>
If Maven/other tooling has already copied the JAR to your WEB-INF/lib directory, you may need to delete the file manually prior to rebuilding.
This should result in an EAR of the form:
META-INF/MANIFEST.MF
lib/myapp-utils-0.0.1-SNAPSHOT.jar
META-INF/application.xml
myapp-web.war
When you are moving to Maven, you should not store the dependency JAR's in your code base. I would suggest you to create a central Maven repository which will contain all the dependencies.
Refer mvn install to first install these artifacts into the local repository. Also, you can refer to the maven central repository to get artifacts while building.
What you need to do is: remove all the dependency jar's from the source code, and all their dependency in the pom.xml. These would be downloaded and packaged from the maven central repository as and when required. Set the Dependency Scope of the artifacts accordingly.