I am migrating my application from hadoop 1.0.3 to hadoop 2.2.0 and maven build had hadoop-core marked as dependency. Since hadoop-core is not present for hadoop 2.2.0. I tried replacing it with hadoop-client and hadoop-common but I am still getting this error for ant.filter. Can anybody please suggest which artifact to use?
previous config :
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.0.3</version>
</dependency>
New Config:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.2.0</version>
</dependency>
Error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project event: Compilation failure: Compilation failure:
[ERROR] /opt/teamcity/buildAgent/work/c670ebea1992ec2f/event/src/main/java/com/intel/event/EventContext.java:[27,36] package org.apache.tools.ant.filters does not exist
[ERROR] /opt/teamcity/buildAgent/work/c670ebea1992ec2f/event/src/main/java/com/intel/event/EventContext.java:[27,36] package org.apache.tools.ant.filters does not exist
[ERROR] /opt/teamcity/buildAgent/work/c670ebea1992ec2f/event/src/main/java/com/intel/event/EventContext.java:[180,59] cannot find symbol
[ERROR] symbol: class StringInputStream
[ERROR] location: class com.intel.event.EventContext
We mainly depend on hdfs api for our application. When we migrated to hadoop 2.X, we were surprised to see the changes in dependencies. We started adding dependencies one at a time. Today we depend on the following core libraries.
hadoop-annotations-2.2.0
hadoop-auth-2.2.0
hadoop-common-2.2.0
hadoop-hdfs-2.2.0
hadoop-mapreduce-client-core-2.2.0
In addition to these we depend on test libraries too. Based on your needs, you may want to include hadoop-hdfs and hadoop-mapreduce-client to the dependencies along with hadoop-common.
Try with these artifacts, word fine on my sample project wordcount
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.2.1</version>
</dependency>
Maven dependencies can be got from this link.
As far as hadoop-core dependies goes, hadoop-core was the name for hadoop 1.X and just renaming the version to 2.X wont help. Also in a hadoop 2.X project using the hadoop 1.X dependency gives an error like
Caused by: org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
Thus it is suggested not to use it. I have been using the following dependencies in my hadoop
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-common</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
You can try these.
Related
I'm using Maven 3.3. Do I need to add any special repositories to access the servlet-api 3.0.1 jar? I've added this to my pom.xml file
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>3.0.1</version>
<scope>provided</scope>
</dependency>
but I get this error when I try and do a build ...
[ERROR] Failed to execute goal on project core: Could not resolve dependencies for project org.collegeboard.springboard:core:jar:99.0.0-SNAPSHOT: Could not find artifact javax.servlet:servlet-api:jar:3.0.1 in thirdparty (https://nexus.getmyco.com/nexus/content/repositories/thirdparty/) -> [Help 1]
I need this dependency because I'm told taht anoterh dependency (spring-test-4.0.6.RELEASE) requires this.
Look at the pom of spring-test-4.0.6.RELEASE: http://search.maven.org/#artifactdetails|org.springframework|spring-test|4.0.6.RELEASE|jar
It actually depends on the following:
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.0.1</version>
<scope>compile</scope>
<optional>true</optional>
</dependency>
I have a project that executes a JMeter tests suite though maven.
I use com.lazerycode.jmeter s jmeter-maven-plugin.
With version 2.0.3 my project runs well. but when I update teh plugins version to 2.1.0 the following error is thrown.
[ERROR] Failed to execute goal com.lazerycode.jmeter:jmeter-maven-plugin:2.1.0:configure (configure) on project my-regression: Could not find artifact commons-math3:commons-math3:jar:3.4.1 in central (https://repo.maven.apache.org/maven2) -> [Help 1]
I know by going through articles like this, that this can be resolved by excluding the dependencies.
but its just I don't know how to configure that dependency.
I tried the below.
<dependencies>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-math3</artifactId>
<version>3.4.1</version>
</dependency>
<dependency>
<groupId>commons-math3</groupId>
<artifactId>commons-math3</artifactId>
<version>3.4.1</version>
<exclusions>
<exclusion>
<groupId>commons-math3</groupId>
<artifactId>commons-math3</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
Which didn't work.
Will be glad to provide more info.
Use version 2.6.0 of the plugin and add under configuration node:
<excludedArtifacts>
<exclusion>commons-pool2:commons-pool2</exclusion>
<exclusion>commons-math3:commons-math3</exclusion>
</excludedArtifacts>
I have hadoop and Hbase installed, both working fine as far as I can tell. When trying to the built jar with hadoop, I get a
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
error, using Hbase version 0.90.2 in my maven dependency.
I think this is quite an old version of Hbase and I am unsure if this old version is compatible with hadoop 2.7.2 or even Java 8. Thus I tried using Hbase version 0.99.2 in my maven dependency, but then I get a
Failed to execute goal on project exercise_2: Could not resolve dependencies for project com.company.exercise_2:exercise_2:jar:1.0-SNAPSHOT: Failure to find org.apache.hbase:hbase:jar:0.99.2 in http://repo.maven.apache.org/maven2 was cached in the local repository
error from the maven plugin. What am I doing wrong?
Here is my pom.xml:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.7.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.7.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.7.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.99.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.1.2</version>
<scope>provided</scope>
</dependency>
Seems like this is jar caching issue, I think HbaseConfiguration is common class regardless of which version of hbase used.
Can you manually delete local repository file of hbase and try mvn XXXX command once again.
Maven will then try to download and fix the class path.
for cross checking, use mvn ... -X option to see which version of jar its trying to download.
Since scope of this jar is
provided
Cross check the hbase version of this jar in your cluster. by using "hbase classpath" and check whether this jar version is closely matching with your jar file version of maven repository of your pom.xml.
That should fix.
Wanted to make use of openejb on top of tomcat v7 using maven instead of installing tomee. Referring to Apache documentation, 3 dependencies have to be added to the maven project.
<dependency>
<groupId>org.apache.openejb</groupId>
<artifactId>javaee-api</artifactId>
<version>6.0-6</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.openejb</groupId>
<artifactId>openejb-core</artifactId>
<version>4.7.4</version>
</dependency>
<dependency>
<groupId>org.apache.openejb</groupId>
<artifactId>tomee</artifactId>
<version>1.7.4</version>
</dependency>
but the last depency generates following error: Missing artifact org.apache.openejb:tomee:jar:1.7.4
The correct artifactId being visible in:
http://mvnrepository.com/artifact/org.apache.openejb/apache-tomee/1.7.4
is apache-tomee and not tomee
so, replace the last depency by the following one and the problem will be solved:
<dependency>
<groupId>org.apache.openejb</groupId>
<artifactId>apache-tomee</artifactId>
<version>1.7.4</version>
</dependency>
maven noob, be patient...
I'm upgrading from cdh3u1 to apache hadoop 0.20.203.0 and pig 0.9.0. I used to have:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.2-cdh3u1</version>
</dependency>
<dependency>
<groupId>org.apache.pig</groupId>
<artifactId>pig</artifactId>
<version>0.8.1-cdh3u1</version>
</dependency>
and running them from inside eclipse, with junit run configuration worked great.
Now I have:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.203.0</version>
</dependency>
<dependency>
<groupId>org.apache.pig</groupId>
<artifactId>pig</artifactId>
<version>0.9.0</version>
</dependency>
and I got NoClassDefFoundError: jline/ConsoleReaderInputStream on runtime.
I ended with adding all these dependencies manually until it worked:
<dependency>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
<version>0.9.94</version>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr-runtime</artifactId>
<version> 3.2 </version> <- this is 3.0.1 in cdh3u1, but probably changed in pig 0.9.0
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>r06</version>
</dependency>
What gives? why isn't maven automatically pulling my dependencies and putting them in the classpath?
Maven has a feature called Transitive dependencies, so you don´t have to specify the libraries that your own dependencies require.
ConsoleReaderInputStream is in the Jline JAR. When you were using Pig.0.8.1-cdh3u1, you didn´t have to add the Jline dependency because it is declared in Pig.0.8.1-cdh3u1.pom. Pig 0.9.0.pom does not have Jline dependency declared anymore, that´s the reason you had to add it by yourself. As for the reason JLine was removed from Pig, you have to ask the developers of that project.