Running Cassandra 2.2.8, Win7, JDK8, Spark2, HAve thse in the CP: Cassandra core 3.12, spark-cassandra-2.11, Spark-cassandra-java-2.11, Spark2.11, spark-network-common_2.11, Guava-16.0.jar, sacala2.11.jar, etc
Trying to run a basic example- compiles fine, but when when I try to run- at the first line itself get error:
SparkConf conf = new SparkConf();
java.lang.NoClassDefFoundError: org/spark_project/guava/cache/CacheLoader
Missing spark-network-common is supposed to cause this error - but I do have it. Any conflicting jars?
Thanks
So the answer is: don't exactly know the answer but the problem was solved. Used the the pom and created a maven project in eclipse. it brought in several (dozen) jars and it finally worked. So likely some conflicting/missing jar - tried to look into it- hard to figure out.
Maybe you should check the repository. To check the whether jar with the lastupdated .If it has lastupdated, and then del those files. And download again.
Related
I'm trying to implement adding maven dependencies to my ant build using this article. I beleive I have everything set up as indicated by the article but since it is not working I must be missing something. I am getting this error:
Could not load definitions from resource org/apache/maven/artifact/ant/antlib.xml. It could not be found.
My searches for this error seem to imply I an missing maven-ant-tasks-2.1.3.jar. However, it is in the path indicated by the classpath. Can someone help be debug this? TIA.
I found at least part of the problem. I had the wrong file name in the classpath. It was maven-ant-tasks.jar and it should have been maven-ant-tasks-2.1.3.jar.
Unfortunately I now have a different problem which I think implies a corrupt jar file.
Unable to obtain resource from /usr/build/lib/maven-ant-tasks-2.1.3.jar: java.util.zip.ZipException: error in opening zip file
Downloading the jar file again did not help.
Never mind. Apparently the Apache maven web site has a corrupt version. I downloaded it from the mvnrepository and that worked.
I am having some difficulties running a simple pig script to import data into HBase using HBaseStorage
The error I have encountered is given by:
Caused by: <file demo.pig, line 14, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[rdf:predicate rdf:object]'
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.initScan(HBaseStorage.java:427)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:368)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:239) 13_21.51.28.tar.gz
... 29 more
According to other questions and threads, the main response/answer to this issue would be to register the appropriate jars required for the HBaseStorage references. What I am stumped by is how am I supposed to identify the required JAR given the appropriate Pig function.
I even tried to open the various jar files under the hbase and pig folders to ensure the appropriate classes are registered in the pig script.
For example, since java.lang.NoSuchMethodError was caused by org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
I imported specifically the jar that contains org.apache.hadoop.hbase.client.Scan, to no avail.
Pig's documentation does not provide any obvious links and help that I can refer to.
I am using Hadoop 2.7.0, HBase 1.0.1.1., Pig 0.15.0.
If you need any other clarification, feel free to ask me again. Would really appreciate it if someone could help me out with this issue.
Also, is it better to install Hadoop and the relevant softwares from scratch, or is it better to directly get one of the Hadoop bundles available?
There is something wrong with the released jar: hbase-client-1.0.1.1.jar
you can test it with this code, the error will show up:
Scan scan = new Scan();
scan.setCacheBlocks(true);
I've tried other set functions, like setCaching, it throws the same error. While I checked the source code, those functions exist. Maybe just compile hbase-client-1.0.1.1.jar manually, I'm still looking for better solution...
============
Update for above, found the root cause is hbase-client-1.0.1.1.jar incompatibility with older versions.
https://issues.apache.org/jira/browse/HBASE-10841
https://issues.apache.org/jira/browse/HBASE-10460
There is a change of return value for set functions, jars compiled with old version won't work with current.
For your question, you can modify the pig script $PIG_HOME/bin/pig, set debug=true, then it will just print running info.
Did you register required jars.
Most important jars habse,zookeeper and guava
I solved the similar kind of issue by registering zookeeper jar in my pigscript
I started working with Hive just recently, so I may be a little new to this, I compiled a jar using Maven Build and for some reason when I am trying to add it in the hive, it won't work. I get the following error:
Query returned non-zero code: 1, cause: ex-0.0.0.1-SNAPSHOT.jar does not exist.
I uploaded the file using hue, and I can find it if I do dfs -ls in hive.
What am I missing? (I was able to load a jar I got online)
Thanks!
If you can find your jar by -lsing to it and it was properly built, usually this error is cause by incorrectly putting quotes around the path to the jar.
Incorrect:
add jar '/root/complete/path/to/jar.jar';
Correct:
add jar /root/complete/path/to/jar.jar;
I installed Hadoop version 1.0.3 and its related eclipse plugin successfully. All the Hadoop functionalities and examples are working pretty well, but when I want to use its plugin on eclipse, it could not connect to hdfs and I get the error:
An internal error occurred during: "Connecting to DFS localhost".
org/apache/commons/configuration/Configuration.
could anybody help me how to solve this problem!
Thanks
You are facing this problem because the plugin is missing some necessary jars. In order to solve the problem you need to rebuild the plugin after including the necessary jars. I have seen this kind of questions a lot on SO, and they all point out to the same thing. Please see these links :
Eclipse Hadoop plugin issue(Call to localhost/127.0.0.1:50070 )Can any body give me the solution for this?
Hadoop eclipse mapreduce is not working?
Installing Hadoop's Eclipse Plugin
I did follow the following blog instructions to make Hadoop eclipse plugin 1.0.4 :
http://iredlof.com/part-4-compile-hadoop-v1-0-4-eclipse-plugin-on-ubuntu-12-10/
but it seems it has some missing parts like:
in MANIFEST.MF you should add:
/lib/commons-cli-1.2.jar
and in build-contrib.xml you should also add:
<property name="commons-cli.version" value="1.2"/>
I hope these are useful!
you must run hadoop with command line first!!
./[hadoop-path]/bin/start-all.sh
i am trying to installing selenium-server-standalone-2.0b1.jar actually for that i have to set all jar files into the classpath of a java project. But i don't know that where all the jars under this folder "selenium-server-standalone-2.0b1.jar" ? and how to install selenium-server-standalone-2.0b1.jar? Please help me out
I didn't get your question correctly. If you are looking for the jars,you can download from the following sites.Try to used the new jars,as these come with lot of fixes.
Just add to classpath & ready to go.
http://seleniumhq.org/download/
Archive Releases:- http://release.seleniumhq.org/