I am setting up oozie for first time. After all the setup is done, when I ran example workflows, they failed. And -errorlog shows following message
org.apache.oozie.action.ActionExecutorException:
File /user/userName/share/lib does not exist
I did run the below command to create sharelib while doing setup and I can see the sharelib with all the jars in hdfs webconsole-
./oozie-setup.sh sharelib create -fs hdfs://localhost:9000
but when I verify the available sharelib using
oozie admin -shareliblist
it shows only [Availble Sharelib] message with no libs actually listed.
Any idea, what can be the issue?
Related
I am getting below error while executing sqoop export command(in shell script) with oozie.
"java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver"
sqoop export from cli(edge node) works fine.
I have added the ojdbc6.jar to below locations.
/opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/sqoop/lib/
(HDFS locations)
/user/oozie/share/lib/sqoop/ and
/user/oozie/share/lib/lib_20161215195933/sqoop
i have also set oozie.use.system.libpath=true in my oozie job.properties file
Please guide me if i am missing any setting.
log content
Thanks & Regards,
Sonali
Make sure that you upload a file to a directory /user/oozie/share/lib/sqoop (it could looks like /user/oozie/share/lib/lib_${timestamp}/sqoop for Cloudera and HDP).
Check if ojdbc6.jar file is correct - check if it contains OracleDriver.class and make sure size of the file is ok. It could be error while downloading.
Check permissions to ojdbc6.jar file (eventually, you can try to give 755 permissions to this file). Check who is the owner of the file - it should be oozie by default.
Update Oozie sharelib by execute below command (run this command on the host where Oozie Server is located):
sudo -u oozie oozie admin -oozie http://<Oozie_Server_Host>:11000/oozie -sharelibupdate
Verify sharelib for sqoop:
sudo -u oozie oozie admin -oozie http://<Oozie_Server_Host>:11000/oozie -shareliblist sqoop*
You can always restart Oozie service. It should update sharelib.
Create a directory named lib next to your workflow.xml in HDFS and put jars in there. Oozie will automatically make those jars available to all actions in that workflow.
Cloudera users should check this article. Especially paragraph 'One Last Thing'.
I tried running my first oozie job by following a blog post.
I used oozie-examples.tar.gz, after extracting, placed examples in hdfs.
I tried running map-reduce job in it but unfortunately got an error.
Ran below command:
oozie job -oozie http://localhost:11000/oozie -config /examples/apps/map-reduce/job.properties -run
Got the error:
java.io.IOException: configuration is not specified at
org.apache.oozie.cli.OozieCLI.getConfiguration(OozieCLI.java:787) at
org.apache.oozie.cli.OozieCLI.jobCommand(OozieCLI.java:1026) at
org.apache.oozie.cli.OozieCLI.processCommand(OozieCLI.java:662) at
org.apache.oozie.cli.OozieCLI.run(OozieCLI.java:615) at
org.apache.oozie.cli.OozieCLI.main(OozieCLI.java:218) configuration is
not specified
I don't know which configuration it is asking for as I am using Cloudera VM and it has by default got all the configurations set in it.
oozie job -oozie http://localhost:11000/oozie -config /examples/apps/map-reduce/job.properties -run
The -config parameter takes an local path not an HDFS path. The workflow.xml needs to be present in the HDFS and path is defined in the job.properties file with the property:
oozie.wf.application.path=<path to the workflow.xml>
i am getting the following error while running the following oozie command
hadoop#master1:~/work/oozie-4.1.0/bin$ oozie-setup.sh -hadoop 0.20.200 $HADOOP_HOME -extjs /home/hadoop/work/ext-2.2.zip
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
Usage : oozie-setup.sh
prepare-war [-d directory] [-secure] (-d identifies an alternative directory
for processing jars
-secure will configure the war file to use HTTPS (SSL))
sharelib create -fs FS_URI [-locallib SHARED_LIBRARY] (create sharelib for
oozie,
FS_URI is the fs.default.name
for hdfs uri; SHARED_LIBRARY, path to the
Oozie sharelib to install, it can be a tarball
or an expanded version of it. If ommited,
the Oozie sharelib tarball from the Oozie
installation directory will be used)
(action failes if sharelib is already installed
in HDFS)
sharelib upgrade -fs FS_URI [-locallib SHARED_LIBRARY] (upgrade existing
sharelib, fails if there
is no existing sharelib installed in HDFS)
db create|upgrade|postupgrade -run [-sqlfile ] (create, upgrade or postupgrade oozie db with an
optional sql File)
(without options prints this usage information)
EXTJS can be downloaded from http://www.extjs.com/learn/Ext_Version_Archives
Any idea to solve this?
EXT.js iw for WEBUI right,
It shows the we need to Prepare war that means need to add the EXT.js by passing the command
oozie-setup.sh prepare-war
or by
addtowar.sh -extjs EXTJS_PATH
Make sure that the ext.js file was present in the LIBEXT in the oozie home while using preparewar.
Reference:
https://oozie.apache.org/docs/4.2.0/AG_Install.html#Oozie_Server_Setup
The details of my oozie job are given below:
I am getting the following error:
57-oozie-oozi-W#MR] Error starting action [MR]. ErrorType [FAILED], ErrorCode [EJ001], Message [Could not locate Oozie sharelib]
org.apache.oozie.action.ActionExecutorException: Could not locate Oozie sharelib
at org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAction(JavaActionExecutor.java:603)
at org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActionExecutor.java:698)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:689)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1135)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
I have put lib in /user/root ,user/oozie .But still it is not able to find it.
Quoting the Oozie documentation a MapReduce action should not care about Shared Libs:
The Oozie sharelib TAR.GZ file bundled with the distribution contains
the necessary files to run Oozie map-reduce streaming, pig, hive,
sqooop, and distcp actions ... other actions (mapreduce, shell, ssh,
and java) do not require the sharelib to be installed.
Anyway, the root dir for the Shared Libs should be something like
/user/oozie/share/lib/
Your setup is probably missing the "share" part (see Quick Start)
And if you want to try a non-default location then look at the documentation for "sharelib create" command, and also for specifics about Oozie server config.
http://oozie.apache.org/docs/4.1.0/AG_Install.html#Oozie_Share_Lib
plus #Oozie_Server_Setup
I am running a simple Oozie workflow on Cloudera VM. The sub-workflow calls a shell script which sends a test email. However, I am getting the JA009 error:
(JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.).
I have already changed mapreduce.framework.name from yarn to classic in the following places:
/etc/oozie/conf/hadoop-conf/core-site.xml
/etc/oozie/conf/hadoop-config.xml
/etc/hadoop/conf/mapred-site.xml
Also, in /etc/hadoop/conf/hadoop-env.sh I changed:
"export HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce"
to:
"export HADOOP_MAPRED_HOME=/usr/lib/hadoop-0.20-mapreduce"
Is there anything I am missing? Yarn is not showing up in hadoop fs -ls /user (hive, pig, spark etc are). So I am assuming Yarn is not pre-installed here.