Maven Dependencies : non existing library 'hadoop-common-2.6.0-cdh5.9.0.jar' - maven

I am trying to implement an Spark app WordCount, but i get an error when I add hadoop-common dependency:
'Maven Dependencies' references non existing library '/root/.m2/repository/org/apache/hadoop/hadoop-common/2.6.0-cdh5.9.0/hadoop-common-2.6.0-cdh5.9.0.jar'
and this my pom.xml :
<dependencies>
<!-- https://mvnrepository.com/artifact/org.scala-lang/scala-library -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.6</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0-cdh5.9.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
my haddop version is 2.6.0-cdh5.9.0.
Any help please!!


replace
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0-cdh5.9.0</version>
</dependency>
with
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
</dependency>
or use external jar and add http://repo.spring.io/libs-milestone/org/apache/hadoop/hadoop-common/2.6.0-cdh5.9.0/hadoop-common-2.6.0-cdh5.9.0.jar
hope this helps

Related

How to add maven dependencies into packaged jar? packaging type is as eclipse-plugin

I am building an eclipse rcp application using Maven/tycho. I have added few dependencies into pom.xml and now i want these dependencies to be part of packaged jar which will be created after building my application. In manifest file its giving me some error as "Package 'com.sun.xml.stream.dtd' does not exist in this plug-in". enter image description here
Attached is the manifest.mf and here is my pom.xml
4.0.0
com.test.jars
com.test.jars
2.22.0.qualifier
eclipse-plugin
com.test
com.test.sampleapp.build
1.0.0-SNAPSHOT
../com.test.sampleapp.build/pom.xml
<dependencies>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.6</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>poi</groupId>
<artifactId>poi-2.5.1-final</artifactId>
<version>20040804</version>
</dependency>
<dependency>
<groupId>poi</groupId>
<artifactId>poi</artifactId>
<version>1.8.0-dev-20020919</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jsr173_api</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.sun.xml.stream</groupId>
<artifactId>sjsxp</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
</dependency>
<dependency>
<groupId>net.jimmc</groupId>
<artifactId>jshortcut</artifactId>
<version>0.4-oberzalek</version>
</dependency>
<dependency>
<groupId>javax.activation</groupId>
<artifactId>activation</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
<version>1.4</version>
</dependency>
</dependencies>

Resolving same classes defined in different dependencies

I'm running into a strange issue where I have two dependencies in my Maven file that both define BasicLineFormatter.class. The issue that I am running into is that one of my dependencies obviously uses a different version that the code is expecting and it throws and error.
How can I tell Maven that I want to use my standard http-core dependency for this class rather than the one that is causing an issue.
RESOURCE LOCATION: jar:file:/var/lib/tomcat8/webapps/ROOT/WEB-INF/lib/tika-app-1.10.jar!/org/apache/http/message/BasicLineFormatter.class
My pom.xml looks like this:
<groupId>xxx</groupId>
<artifactId>xxx</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<name>xxx</name>
<dependencies>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.3.1</version>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-bundle</artifactId>
<version>1.18.3</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.8.3</version>
</dependency>
<dependency>
<groupId>com.mashape.unirest</groupId>
<artifactId>unirest-java</artifactId>
<version>1.4.6</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.3</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.6</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpasyncclient</artifactId>
<version>4.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.3.6</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20140107</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.pdfbox</groupId>
<artifactId>pdfbox</artifactId>
<version>1.8.10</version>
</dependency>
<dependency>
<groupId>org.apache.pdfbox</groupId>
<artifactId>fontbox</artifactId>
<version>1.8.10</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.2</version>
</dependency>
<dependency>-->
<groupId>org.apache.tika</groupId>
<artifactId>tika-app</artifactId>
<version>1.10</version>
</dependency>
</dependencies>
My code is getting the BasicLineFormatter class from the Tika dependency which is causing the issue. I want it to be coming from my defined http-core which is the correct version. It also seems to be ignoring the ordering the dependencies in the file.

Use - Shade Plugin - https://maven.apache.org/plugins/maven-shade-plugin/examples/includes-excludes.html
Reference - Maven dependency: exclude one class

how to fix java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration

I am using hadoop and hbase. the hbase version is 1.1.1.2.3. I try to use maven to compile, run the project. But it has the org.apache.hadoop.hbase.HBaseConfiguration not find problem.
Here is my pom.xml:
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.2</version>
</dependency>
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.4.6</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.4.0a</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>11.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>0.23.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-protocol</artifactId>
<version>1.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.94.27</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-server</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-testing-util</artifactId>
<version>1.1.1</version>
</dependency>
</dependencies>
dependencies are as above, I really do not know what to add more. Could anyone help? with many thanks. I know there are many problems similar but none of them is using the newest version base and I tried there solution, does not work. thanks
Furthermore, I tried to add the dependency: http://mvnrepository.com/artifact/org.apache.hbase/hbase/1.1.1
but get error: Failed to execute goal on project hbaseTable: Could not resolve dependencies for project apache.hbase:hbaseTable:jar:0.0.1-SNAPSHOT: Failure to find org.apache.hbase:hbase:jar:1.1.1 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

What are the minimal pom dependencies of Xpand?

I use Xpand for code generation inside a maven plugin project. So, I need to add the required libraries as dependencies in pom.xml in order to define and use Xpand templates. What is the minimum list of dependencies that should be added to the pom?

you may take the following as starting point
<dependencies>
<dependency>
<groupId>org.eclipse.xpand</groupId>
<artifactId>org.eclipse.xtend</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>org.eclipse.xpand</groupId>
<artifactId>org.eclipse.xpand</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>org.eclipse.xpand</groupId>
<artifactId>org.eclipse.xtend.typesystem.emf</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>org.eclipse.emf</groupId>
<artifactId>org.eclipse.emf.mwe.utils</artifactId>
<version>1.3.5</version>
</dependency>
<dependency>
<groupId>org.eclipse.emf</groupId>
<artifactId>org.eclipse.emf.mwe.core</artifactId>
<version>1.3.5</version>
</dependency>
<dependency>
<groupId>org.eclipse.emf</groupId>
<artifactId>ecore</artifactId>
<version>2.3.0-v200706262000</version>
</dependency>
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
<version>20040117.000000</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>org.eclipse.emf</groupId>
<artifactId>org.eclipse.emf.mwe2.runtime</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr-runtime</artifactId>
<version>3.5.2</version>
</dependency>
<dependency>
<groupId>com.ibm.icu</groupId>
<artifactId>icu4j</artifactId>
<version>55.1</version>
</dependency>
</dependencies>

Unable to configure maven dependencies correctly

I have a web-app with following dependencies in pom:
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-web-api</artifactId>
<version>6.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.derby</groupId>
<artifactId>derby</artifactId>
<version>10.9.1.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>4.1.9.Final</version>
<scope>test</scope>
</dependency>
With this configuration, I get following error in junit test:
java.lang.ClassFormatError: Absent Code attribute in method that is
not native or abstract in class file javax/persistence/Persistence
I found some info on this error here.
So, I removed 'javaee-web-api' dependency.
After that I get compilation error 'package javax.persistence does not exist', and many more.
So, I changed scope of 'hibernate-entitymanager' to default (compile).
The working pom (dependencies) look like this:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.derby</groupId>
<artifactId>derby</artifactId>
<version>10.9.1.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>4.1.9.Final</version>
</dependency>
Now everything is OK, except generated war contains hibernate jar, which I do not want.
Any suggestion how to fix this, correct maven pom?

If you are working with an in application server, you can replace the hibernate dependency with a (still vendor specific) JPA API with provided scope like this ( example is for JBoss with hibernate):
<dependency>
<groupId>org.hibernate.javax.persistence</groupId>
<artifactId>hibernate-jpa-2.0-api</artifactId>
<scope>provided</scope>
</dependency>

Finally, I got this working, with same set of dependencies, using profiles.
Here is relevant section of pom:
<profiles>
<profile>
<id>test</id>
<dependencies>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>4.1.9.Final</version>
</dependency>
</dependencies>
</profile>
<profile>
<id>package</id>
<dependencies>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-web-api</artifactId>
<version>6.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>4.1.9.Final</version>
<scope>test</scope>
</dependency>
</dependencies>
</profile>
</profiles>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.derby</groupId>
<artifactId>derby</artifactId>
<version>10.9.1.0</version>
<scope>test</scope>
</dependency>
</dependencies>
Now, I use following commands to package and test,
mvn -Ptest test
mvn -Ppackage package -DskipTests
-DskipTests is required, as, otherwise, package goal triggers 'test' causing the same error I started with, as it uses same set of dependencies configured for 'package'.
java.lang.ClassFormatError: Absent Code attribute in method that is
not native or abstract in class file javax/persistence/Persistence

I was having the same problem and was able to solve it by adding a couple of dependencies to the Maven pom.xml file. See this answer for more details.

Resources