While installing pig version 0.17.0 on my ubuntu system i am facing an error after i run a command pig -help , it throws an error : Cannot locate pig-core-h2.jar. do 'ant jar', and try again
input command : pig -help
output : Cannot locate pig-core-h2.jar. do 'ant jar', and try again
pig version - 0.17.0
i have already configured the .bashrc file with all the given variables.
export PIG_HOME=/usr/local/pig-0.17.0
export PATH=$PIG_HOME/bin:$HADOOP_HOME/bin:$PATH
export PIG_CLASSPATH=$HADOOP_HOME/conf
Does $PIG_HOME/pig-0.17.0-core-h2.jar (/usr/local/pig-0.17.0/pig-0.17.0-core-h2.jar) exist in your environment?
If not, I would reinstall from the tarball.
Related
I have installed Hadoop 3.1.1 and it is working. However, when I try to compile the WordCount example, I am receiving this error:
/usr/local/hadoop/libexec/hadoop-functions.sh: line 2358: HADOOP_COM.SUN.TOOLS.JAVAC.MAIN_USER: bad substitution
/usr/local/hadoop/libexec/hadoop-functions.sh: line 2453: HADOOP_COM.SUN.TOOLS.JAVAC.MAIN_OPTS: bad substitution
To compile, I used the next line:
hadoop com.sun.tools.javac.Main WordCount.java
I have the next variables in the .bashrc:
#Hadoop variables
export HADOOP_HOME=/usr/local/hadoop
export CONF=$HADOOP_HOME/etc/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
#Java home
export JAVA_HOME=${JAVA_HOME}/java-8-oracle
#Path Java Tools
export HADOOP_CLASSPATH=$JAVA_HOME/lib/tools.jar
This time I am using Java 8 of Oracle because the apt-get of Ubuntu 18.08 LTS does not give me the option of installing the OpenJDK8. I have updated and upgraded Ubuntu.
I have read a lot of different post and possible solutions, but I cannot solve it.
This is a viable solution I found in https://janzhou.org/2014/how-to-compile-hadoop.html
Set the HADOOP_CLASSPATH:
export HADOOP_CLASSPATH=$(bin/hadoop classpath)
Compile:
javac -classpath ${HADOOP_CLASSPATH} -d WordCount/ WordCount.java
I run the simulator with following command:
bin/slsrun.sh --input-rumen = <sample-data/2jobs2min-rumen-jh.json>
and it gave the following error:
-su: syntax error near unexpected token `newline'
Note: PWD is $HADOOP_ROOT/share/hadoop/tools/sls
It comes with your Hadoop distribution. The script is located in the tools directory: $HADOOP_HOME/share/hadoop/tools/sls/bin/slsrun.sh. A description of its usage is available at https://hadoop.apache.org/docs/r2.4.1/hadoop-sls/SchedulerLoadSimulator.html. I successfully followed the steps on my Hadoop 2.8.0 installation.
I am trying to install pig 0.13 in ubuntu
I downloaded the pig-0.13.0.tar.gz and installed in $HOME path
After that I edit .bashrc file with below export commands
Please look at my .bashrc file
export HADOOP_HOME=$HOME/hadoop-1.2.1
export HADOOP_CONF_DIR=$HOME/hadoop-1.2.1/conf
export HADOOP_MAPRED_HOME=$HOME/hadoop-1.2.1/conf
export HADOOP_COMMON_HOME=$HOME/hadoop-1.2.1
export HADOOP_HDFS_HOME=$HOME/hadoop-1.2.1
###################JAVA############################
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk-i36
export PATH=$PATH:$HOME/hadoop-1.2.1/bin
########################PIG#######################
export PIG_HOME=/home/user/pig-0.13.0
export PATH=$PATH:$PIG_HOME/bin
But when i try to test whether pig is launced correctly or not.. it is not activated..
I reboot my machine and applied source .bashrc as well but nothing helped
user#ubuntu:~$ ls
Desktop examples.desktop Music pig-0.13.0.tar.gz Templates
Documents hadoop-1.2.1 Pictures Public Videos
Downloads hadoop-1.2.1.tar.gz pig-0.13.0 surender
user#ubuntu:~$ hadoop version
Warning: $HADOOP_HOME is deprecated.
Hadoop 1.2.1
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches /branch-1.2 -r 1503152
Compiled by mattf on Mon Jul 22 15:23:09 PDT 2013
From source with checksum 6923c86528809c4e7e6f493b6b413a9a
This command was run using /home/user/hadoop-1.2.1/hadoop-core-1.2.1.jar
user#ubuntu:~$ pig -help
pig: invalid option -- 'h'
usage: pig
user#ubuntu:~$
my $HOME is /home/user
Could somebody help me on this?
I downloaded pig 0.14.0 and I am running Hadoop 2.6.0 on MAC OSX. I followed all the installation steps for PIG at https://github.com/ucbtwitter/getting-started/wiki/Installing-Pig .I had set JAVA_HOME correctly as mentioned.
Even after running the ant "-Dhadoopversion=23 jar" command I am getting the same error "Cannot locate pig-core-h2.jar. do 'ant -Dhadoopversion=23 jar', and try again".
This error constantly arising
Cannot locate pig-core-h2.jar. do 'ant -Dhadoopversion=23 jar', and try again.
I studied the shell script by opening pig-0.14.0/bin/pig file and found that this error is related to the setting of CLASSPATH and PIG_HOME and JAVA_HOME variables.
Then I found that I mispelled the PIG_HOME then I corrected it.
Next I ran that specified command('ant -Dhadoopversion=23 jar') in the pig installation directory.
Then I got this error
Not a valid JAR: /Users/../../../pig-0.14.0/pig-0.14.0-SNAPSHOT-core-h2.jar /Users/../../../pig-0.14.0/pig-0.14.0-core-h2.jar
To resolve it remove that jar file in that location.
Then I got it working.
Find the path to the file pig-*-core-h2.jar.
I installed pig using brew install pig and found the jar in the path /usr/local/Cellar/pig/0.17.0/libexec
Run export PIG_HOME=/usr/local/Cellar/pig/0.17.0/libexec
This will fix your error.
i did this to fix the pig
mv /data/mapr/pig/pig-0.14/pig-0.14.0-mapr-1603-core-h2.jar /data/mapr/pig/pig-0.14/pig-0.14.0-mapr-1603-core-h2.jar.orig
The following solutions works:
Please make sure in your .bash_profile or .bashrc you have following environment variables:
export PIG_HOME="/Library/apache-pig-0.15.0"
export PATH="/Library/apache-pig-0.15.0/bin:${PATH}"
restart the machine or restart the unix terminal;
I replaced /Library/apache-pig-0.15.0/ with "home/cwu/Downloads/pig-0.15.0-src
"
I'm familiar with the Cloudera architecture but not MapR. I have a RHEL vm and previously installed the MapR client software using this documentation. I am able to submit mapreduce jobs and query HDFS as expected.
I followed this documentation (after I installed the MapR yum repo) and installed sqoop like so:
yum install mapr-sqoop
If I try to sqoop in some data, or even just issue the command sqoop, I receive the following error:
/opt/mapr/sqoop/sqoop-1.4.4/bin/configure-sqoop: line 47: /opt/mapr/bin/versions.sh: No such file or directory
Error: /opt/mapr/hadoop/hadoop- does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.
I have a /opt/mapr/hadoop/hadoop-0.20.2 directory. I've tried setting HADOOP_COMMON_HOME and HADOOP_HOME to both /opt/mapr/hadoop and /opt/mapr/hadoop/hadoop-0.20.2 yet still receive the same error.
-- Update:
I issued a find / -name hadoop and noted the last line which was /usr/bin/hadoop.
I then set HADOOP_COMMON_HOME to /usr, and was then asked to set HADOOP_MAPRED_HOME, HBASE_HOME, and HCAT_HOME, which I all set to /usr.
This error however is still present:
/opt/mapr/sqoop/sqoop-1.4.4/bin/configure-sqoop: line 47: /opt/mapr/bin/versions.sh: No such file or directory
I opened up this file and commented out line 47. This allowed me to use the sqoop command, but the import job failed and I received a lot of Error: Unsupported major.minor version.
There should be a patch for this if not fixed already,
Here is temp solution:
mapr-client does not give versions.sh , only mapr-core does. Simple fix is to
manually copy that file from a node with mapr-core installed and customize the
versions therein. sqoop then works fine.