I started working with Hive just recently, so I may be a little new to this, I compiled a jar using Maven Build and for some reason when I am trying to add it in the hive, it won't work. I get the following error:
Query returned non-zero code: 1, cause: ex-0.0.0.1-SNAPSHOT.jar does not exist.
I uploaded the file using hue, and I can find it if I do dfs -ls in hive.
What am I missing? (I was able to load a jar I got online)
Thanks!
If you can find your jar by -lsing to it and it was properly built, usually this error is cause by incorrectly putting quotes around the path to the jar.
Incorrect:
add jar '/root/complete/path/to/jar.jar';
Correct:
add jar /root/complete/path/to/jar.jar;
Related
hey hi I have written code and i tried running the jar but i got an exception saying that jar file cannot be opened and i even changed the file permissions and path is also set right but it is still showing me error saying that jar file cannot be opened.
Please suggest some solutions.
that is the error i got and even i have gone through the wordcount example also but could not get any answer.
Is your jar file in local file system or hdfs? The jar file should be in local file system.
I have Oozie installed and running. I am trying to run the example workflows, and got an error that I need to create the shared library. So I put the shared folder in HDFS under /user/{username}/share/lib.
When I run: bin/oozie-setup.sh sharelib create -fs hdfs://{host}:8020, I get an error: java.lang.ClassNotFoundException: org.apache.htrace.SamplerBuilder
I have tried adding the htrace core jar into my libext folder of oozie and that gives me a Java Servlet error. Did I miss a step? I cannot seem to find anything related to this.
It ended up being an issue with my /libext directory. I went back and added the ext js zip and then found all ".jar" files in my hadoop directory. After filtering out the overlap of tomcat, etc with the Oozie UI, i was able to get an example workflow running!
I wanted to execute a hadoop example and it says jar not found. The code was run using hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+
The error produced was:
Not a valid JAR : /home/rahul/hadoop/hadoop-examples-*.jar
Try to check the jar specified path ,i think it is the cause of the issue
I'm using HDP 2.4 in CentOS 6.7.
I have created the cluster with Ambari, so Oozie was installed and configured by Ambari.
I got two errors while running Oozie/Sqoop related to jar file location. The first concerned postgresql-jdbc.jar, since the Sqoop job is incrementally importing from Postgres. I added the postgresql-jdbc.jar file to HDFS and pointed to it in workflow.xml:
<file>/user/hdfs/sqoop/postgresql-jdbc.jar</file>
It solved the problem. But the second error seems to concern kite-data-mapreduce.jar. However, doing the same for this file:
<file>/user/hdfs/sqoop/kite-data-mapreduce.jar</file>
does not seem to solve the problem:
Failing Oozie Launcher, Main class
[org.apache.oozie.action.hadoop.SqoopMain], main() threw exception,
org/kitesdk/data/DatasetNotFoundException
java.lang.NoClassDefFoundError:
org/kitesdk/data/DatasetNotFoundException
It seems strange that this is not automatically configured by Ambari and that we have to copy jar files into HDFS as we start getting errors.
Is this the correct methodology or did I miss some configuration step?
This is happening due to the missing jars in the classpath. I would suggest you to use the property oozie.use.system.libpath=true in the job.properties file. All the sqoop related jars will be added automatically in the classpath. Then add only custom jar you need to the lib directory of the workflow application path., all the sqoop related jars will be added from the /user/oozie/share/lib/lib_<timestamp>/sqoop/*.jar.
I am having some difficulties running a simple pig script to import data into HBase using HBaseStorage
The error I have encountered is given by:
Caused by: <file demo.pig, line 14, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[rdf:predicate rdf:object]'
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.initScan(HBaseStorage.java:427)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:368)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:239) 13_21.51.28.tar.gz
... 29 more
According to other questions and threads, the main response/answer to this issue would be to register the appropriate jars required for the HBaseStorage references. What I am stumped by is how am I supposed to identify the required JAR given the appropriate Pig function.
I even tried to open the various jar files under the hbase and pig folders to ensure the appropriate classes are registered in the pig script.
For example, since java.lang.NoSuchMethodError was caused by org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
I imported specifically the jar that contains org.apache.hadoop.hbase.client.Scan, to no avail.
Pig's documentation does not provide any obvious links and help that I can refer to.
I am using Hadoop 2.7.0, HBase 1.0.1.1., Pig 0.15.0.
If you need any other clarification, feel free to ask me again. Would really appreciate it if someone could help me out with this issue.
Also, is it better to install Hadoop and the relevant softwares from scratch, or is it better to directly get one of the Hadoop bundles available?
There is something wrong with the released jar: hbase-client-1.0.1.1.jar
you can test it with this code, the error will show up:
Scan scan = new Scan();
scan.setCacheBlocks(true);
I've tried other set functions, like setCaching, it throws the same error. While I checked the source code, those functions exist. Maybe just compile hbase-client-1.0.1.1.jar manually, I'm still looking for better solution...
============
Update for above, found the root cause is hbase-client-1.0.1.1.jar incompatibility with older versions.
https://issues.apache.org/jira/browse/HBASE-10841
https://issues.apache.org/jira/browse/HBASE-10460
There is a change of return value for set functions, jars compiled with old version won't work with current.
For your question, you can modify the pig script $PIG_HOME/bin/pig, set debug=true, then it will just print running info.
Did you register required jars.
Most important jars habse,zookeeper and guava
I solved the similar kind of issue by registering zookeeper jar in my pigscript