I downloaded pig 0.14.0 and I am running Hadoop 2.6.0 on MAC OSX. I followed all the installation steps for PIG at https://github.com/ucbtwitter/getting-started/wiki/Installing-Pig .I had set JAVA_HOME correctly as mentioned.
Even after running the ant "-Dhadoopversion=23 jar" command I am getting the same error "Cannot locate pig-core-h2.jar. do 'ant -Dhadoopversion=23 jar', and try again".
This error constantly arising
Cannot locate pig-core-h2.jar. do 'ant -Dhadoopversion=23 jar', and try again.
I studied the shell script by opening pig-0.14.0/bin/pig file and found that this error is related to the setting of CLASSPATH and PIG_HOME and JAVA_HOME variables.
Then I found that I mispelled the PIG_HOME then I corrected it.
Next I ran that specified command('ant -Dhadoopversion=23 jar') in the pig installation directory.
Then I got this error
Not a valid JAR: /Users/../../../pig-0.14.0/pig-0.14.0-SNAPSHOT-core-h2.jar /Users/../../../pig-0.14.0/pig-0.14.0-core-h2.jar
To resolve it remove that jar file in that location.
Then I got it working.
Find the path to the file pig-*-core-h2.jar.
I installed pig using brew install pig and found the jar in the path /usr/local/Cellar/pig/0.17.0/libexec
Run export PIG_HOME=/usr/local/Cellar/pig/0.17.0/libexec
This will fix your error.
i did this to fix the pig
mv /data/mapr/pig/pig-0.14/pig-0.14.0-mapr-1603-core-h2.jar /data/mapr/pig/pig-0.14/pig-0.14.0-mapr-1603-core-h2.jar.orig
The following solutions works:
Please make sure in your .bash_profile or .bashrc you have following environment variables:
export PIG_HOME="/Library/apache-pig-0.15.0"
export PATH="/Library/apache-pig-0.15.0/bin:${PATH}"
restart the machine or restart the unix terminal;
I replaced /Library/apache-pig-0.15.0/ with "home/cwu/Downloads/pig-0.15.0-src
"
Related
I installed Kotlinc through the zip file kotlin-compiler-1.8.0.zip and extracted it and moved the kotlinc\bin files to my C:\Program files. I later then verified that I had installed it correctly by running the kotlinc -version command in the terminal and got back info: kotlinc-jvm 1.5.21 (JRE 16.0.2+7-67). So I believe I have that working just fine. So I believe now the error is arising from Java somehow.
When I go to run my simple hello world program with this:
kotlinc main.kt -include-runtime -d main.jar
I am met with the error:
/mnt/c/Program Files/kotlinc/bin/kotlinc: line 98: java: command not found
Previously I was just getting: kotlinc command not found. I later read on StackOverflow someone else was having the same problem and the answer to that solution was just add it to the environment Variable path, in which I did but I very soon then ran into this issue.
I have since tried everything I have came across on this issue, I've reinstalled the kotlinc compiler and java and put the Java\jdk16.0.2\bin in my environment variable path as well. When I try to run the simple command kotlinc help in Ubuntu I am also met with the same line 98: java command not found error.
I tried to start hadoop, but it failed with nothing started. Following the console log.
Mac:sbin lqs2$ sh start-all.sh
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-functions.sh: line 398:
syntax error near unexpected token `<'
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-functions.sh: line 398:
`done < <(for text in "${input[#]}"; do'
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-config.sh: line 70:
hadoop_deprecate_envvar: command not found
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-config.sh: line 87:
hadoop_bootstrap: command not found
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-config.sh: line 104:
hadoop_parse_args: command not found
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-config.sh: line 105:
shift: : numeric argument required
WARNING: Attempting to start all Apache Hadoop daemons as lqs2 in 10
seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
I have tried any ways to solve it but nothing woked. Even I reinstalled the latest version. But the error is the same. It almost drives me mad.
Any answer is helpful. Thanks.
Hadoop scripts require bash, not sh
$ chmod +x start-all.sh
$ ./start-all.sh
Though I would suggest starting HDFS and YARN separately so that you can isolate other issues
You also need to downgrade Hadoop to at least the latest 2.7 release for Spark to work
In my case, I was seeing this error in OSX after installing Hadoop using HomeBrew. The solution was to do a fresh install after downloading the Hadoop (3.2.1) binary directly from the official website. While installing, I had set HADOOP_HOME and JAVA_HOME environment variables.
A word of caution: I found that the issue can occur if the following environment variables are defined in hadoop-env.sh :
export HDFS_NAMENODE_USER="root"
export HDFS_DATANODE_USER="root"
export HDFS_SECONDARYNAMENODE_USER="root"
export YARN_RESOURCEMANAGER_USER="root"
export YARN_NODEMANAGER_USER="root"
I had initially added these variables while trying to fix the issue. Ultimately I removed them and the error disappeared.
Note, I executed all the Hadoop commands and scripts as non-root user, and also upgraded bash to version 5.0.17.
While installing pig version 0.17.0 on my ubuntu system i am facing an error after i run a command pig -help , it throws an error : Cannot locate pig-core-h2.jar. do 'ant jar', and try again
input command : pig -help
output : Cannot locate pig-core-h2.jar. do 'ant jar', and try again
pig version - 0.17.0
i have already configured the .bashrc file with all the given variables.
export PIG_HOME=/usr/local/pig-0.17.0
export PATH=$PIG_HOME/bin:$HADOOP_HOME/bin:$PATH
export PIG_CLASSPATH=$HADOOP_HOME/conf
Does $PIG_HOME/pig-0.17.0-core-h2.jar (/usr/local/pig-0.17.0/pig-0.17.0-core-h2.jar) exist in your environment?
If not, I would reinstall from the tarball.
I'm familiar with the Cloudera architecture but not MapR. I have a RHEL vm and previously installed the MapR client software using this documentation. I am able to submit mapreduce jobs and query HDFS as expected.
I followed this documentation (after I installed the MapR yum repo) and installed sqoop like so:
yum install mapr-sqoop
If I try to sqoop in some data, or even just issue the command sqoop, I receive the following error:
/opt/mapr/sqoop/sqoop-1.4.4/bin/configure-sqoop: line 47: /opt/mapr/bin/versions.sh: No such file or directory
Error: /opt/mapr/hadoop/hadoop- does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.
I have a /opt/mapr/hadoop/hadoop-0.20.2 directory. I've tried setting HADOOP_COMMON_HOME and HADOOP_HOME to both /opt/mapr/hadoop and /opt/mapr/hadoop/hadoop-0.20.2 yet still receive the same error.
-- Update:
I issued a find / -name hadoop and noted the last line which was /usr/bin/hadoop.
I then set HADOOP_COMMON_HOME to /usr, and was then asked to set HADOOP_MAPRED_HOME, HBASE_HOME, and HCAT_HOME, which I all set to /usr.
This error however is still present:
/opt/mapr/sqoop/sqoop-1.4.4/bin/configure-sqoop: line 47: /opt/mapr/bin/versions.sh: No such file or directory
I opened up this file and commented out line 47. This allowed me to use the sqoop command, but the import job failed and I received a lot of Error: Unsupported major.minor version.
There should be a patch for this if not fixed already,
Here is temp solution:
mapr-client does not give versions.sh , only mapr-core does. Simple fix is to
manually copy that file from a node with mapr-core installed and customize the
versions therein. sqoop then works fine.
I am editing conf/hadoop-env.sh to define JAVA_HOME variable. But i am getting these errors.
bin/hadoop: line 350: C:\Program: command not found
/bin/java: No such file or directoryes\Java\jdk1.6.0_26\bin
/bin/java: cannot execute: No such file or directoryk1.6.0_26\bin
I am using cygwin for hadoop 1.2.1 on my windows.
My java is installed in C:\Program Files\Java\jdk1.6.0_26\bin
I have set the environment variable and edited the env.sh file in this way.
export JAVA_HOME=C:\ProgramFiles\Java\jdk1.6.0_26\bin
When i am trying to echo JAVA_HOME,it is giving C:\Program only
But still it is not working for me. Can anyone please suggest me what to do. I have tried everything possible i read on internet.