Running Hive through Oozie - hadoop

Question

I'm facing some issues while running hive through oozie. In the oozie console, I'm getting the following error:
2013-05-03 04:48:24,248 WARN HiveActionExecutor:542 - USER[ambari_qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000013-130502155316029-oozie-oozi-W]
ACTION[0000013-130502155316029- oozie-oozi-W#hive-node] Launcher exception: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException:
org.apache.hcatalog.security.HdfsAuthorizationProvider
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: org.apache.hcatalog.security.HdfsAuthorizationProvider
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:293)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:669)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:303)
at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:280)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:467)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1178)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: org.apache.hcatalog.security.HdfsAuthorizationProvider
at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:342)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:287)
... 19 more
Caused by: java.lang.ClassNotFoundException: org.apache.hcatalog.security.HdfsAuthorizationProvider
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:335)
... 20 more
I have mentioned the system classpath in my job.properties file. Here is my job properties file:
nameNode=hdfs://ip-10-0-0-92:8020
jobTracker=ip-10-0-0-93:50300
queueName=default
wfeRoot=wfe
oozie.use.system.libpath=true
oozie.libpath=/user/oozie/share/lib/hive
oozie.wf.application.path=${nameNode}/user/${user.name}/${wfeRoot}/hive-oozie
And here is my workflow.xml file:
<action name="hive-node">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="${nameNode}/user/${wf:user()}/${wfeRoot}/output-data/hive"/>
<mkdir path="${nameNode}/user/${wf:user()}/${wfeRoot}/output-data"/>
</prepare>
<job-xml>hive-site.xml</job-xml>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
<property>
<name>oozie.log.hive.level</name>
<value>DEBUG</value>
</property>
<property>
<name>oozie.hive.defaults</name>
<value>hive-default.xml</value>
</property>
</configuration>
<script>script.q</script>
</hive>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Hive failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
I have copied hive-site.xml & hive-default.xml files into hdfs.
Any idea whats going on here?

Answer 1

I got the same error when I did not have the jars for HCatalog in my Oozie ShareLib folder. This line in your error output is what gives it away.
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: org.apache.hcatalog.security.HdfsAuthorizationProvider
To verfiy, run
hadoop fs -ls /user/oozie/share/lib/hive | grep hcatalog
You should expect to see both hcatalog-core-0.5.0 and hcatalog-server-extensions-0.5.0 (the version might be different depending on how your distribution packages it).
The solution is to run the following. You will need to run it as hdfs (or whichever user is your cluster's root user) by running sudo su hdfs if dfs.permissions = true on your cluster.
cd /usr/lib/hcatalog/share/hcatalog
ls
hadoop fs -put hcatalog-core-0.5.0.jar /user/oozie/share/lib/hive
hadoop fs -put hcatalog-server-extensions-0.5.0.jar /user/oozie/share/lib/hive
hadoop fs -chmod 777 /user/oozie/share/lib/hive/hcatalog-core-0.5.0.jar
hadoop fs -chmod 777 /user/oozie/share/lib/hive/hcatalog-server-extensions-0.5.0.jar
Again, the exact file names and file paths will vary by Hadoop distribution. I'm suggesting 777 permissions (read, write, and execute for all owner, group, and all other users) because it is the most permissive and will guarantee to get you through this bug. If you have security concerns, different permissions might be required. You can look at the other files in the folder to get a reasonable default.
This problem can occur when setting up or upgrading a new Hadoop cluster if your distributions upgrade tools/documentation miss this step.