I tried to start hadoop, but it failed with nothing started. Following the console log.
Mac:sbin lqs2$ sh start-all.sh
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-functions.sh: line 398:
syntax error near unexpected token `<'
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-functions.sh: line 398:
`done < <(for text in "${input[#]}"; do'
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-config.sh: line 70:
hadoop_deprecate_envvar: command not found
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-config.sh: line 87:
hadoop_bootstrap: command not found
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-config.sh: line 104:
hadoop_parse_args: command not found
/Users/lqs2/Library/hadoop-3.1.1/libexec/hadoop-config.sh: line 105:
shift: : numeric argument required
WARNING: Attempting to start all Apache Hadoop daemons as lqs2 in 10
seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
I have tried any ways to solve it but nothing woked. Even I reinstalled the latest version. But the error is the same. It almost drives me mad.
Any answer is helpful. Thanks.
Hadoop scripts require bash, not sh
$ chmod +x start-all.sh
$ ./start-all.sh
Though I would suggest starting HDFS and YARN separately so that you can isolate other issues
You also need to downgrade Hadoop to at least the latest 2.7 release for Spark to work
In my case, I was seeing this error in OSX after installing Hadoop using HomeBrew. The solution was to do a fresh install after downloading the Hadoop (3.2.1) binary directly from the official website. While installing, I had set HADOOP_HOME and JAVA_HOME environment variables.
A word of caution: I found that the issue can occur if the following environment variables are defined in hadoop-env.sh :
export HDFS_NAMENODE_USER="root"
export HDFS_DATANODE_USER="root"
export HDFS_SECONDARYNAMENODE_USER="root"
export YARN_RESOURCEMANAGER_USER="root"
export YARN_NODEMANAGER_USER="root"
I had initially added these variables while trying to fix the issue. Ultimately I removed them and the error disappeared.
Note, I executed all the Hadoop commands and scripts as non-root user, and also upgraded bash to version 5.0.17.
Related
I installed Kotlinc through the zip file kotlin-compiler-1.8.0.zip and extracted it and moved the kotlinc\bin files to my C:\Program files. I later then verified that I had installed it correctly by running the kotlinc -version command in the terminal and got back info: kotlinc-jvm 1.5.21 (JRE 16.0.2+7-67). So I believe I have that working just fine. So I believe now the error is arising from Java somehow.
When I go to run my simple hello world program with this:
kotlinc main.kt -include-runtime -d main.jar
I am met with the error:
/mnt/c/Program Files/kotlinc/bin/kotlinc: line 98: java: command not found
Previously I was just getting: kotlinc command not found. I later read on StackOverflow someone else was having the same problem and the answer to that solution was just add it to the environment Variable path, in which I did but I very soon then ran into this issue.
I have since tried everything I have came across on this issue, I've reinstalled the kotlinc compiler and java and put the Java\jdk16.0.2\bin in my environment variable path as well. When I try to run the simple command kotlinc help in Ubuntu I am also met with the same line 98: java command not found error.
I run the simulator with following command:
bin/slsrun.sh --input-rumen = <sample-data/2jobs2min-rumen-jh.json>
and it gave the following error:
-su: syntax error near unexpected token `newline'
Note: PWD is $HADOOP_ROOT/share/hadoop/tools/sls
It comes with your Hadoop distribution. The script is located in the tools directory: $HADOOP_HOME/share/hadoop/tools/sls/bin/slsrun.sh. A description of its usage is available at https://hadoop.apache.org/docs/r2.4.1/hadoop-sls/SchedulerLoadSimulator.html. I successfully followed the steps on my Hadoop 2.8.0 installation.
Trying to get get fireplace set up in vim but having trouble with some error messages.
When I type cpr in a clojure buffer the following error shows up:
Error detected while processing function <SNR>53_RunTests..fireplace#capture_te
st_run..<SNR>53_eval..fireplace#client..fireplace#platform..fireplace#register_
port_file..fireplace#nrepl_connection#open..fireplace#nrepl#for..<SNR>75_nrepl_
process..<SNR>75_nrepl_message..<SNR>75_nrepl_call..<SNR>74_nrepl_transport_cal
l..<SNR>74_nrepl_transport_dispatch:
line 6:
E605: Exception not caught: nREPL: zsh:1: command not found: python^#
Error detected while processing function <SNR>53_RunTests..fireplace#capture_te
st_run..<SNR>53_eval..fireplace#client..fireplace#platform:
line 9:
E171: Missing :endif
The thing I understand is that it cant find python. In my shell I can start python with the python command which will fire up 3.4.2.
Im using nixos
:echo has('python') returns 0
echo $PATH and :! echo $PATH return the same path.
What can I try in order to find what is actually causing this error?
You have to install/compile vim with python support for vim-fireplace to work, and this may not be the default on your system. I believe typing :help python will do this for you. If you don't get anything, you may have to recompile vim with python support, for which I'll direct you towards Google for now.
Another option if you have pip installed is to consider switching to neovim. Installing neovim with pip install neovim will get you python support. I've been using neovim lately, and it's worked pretty well for me.
I downloaded pig 0.14.0 and I am running Hadoop 2.6.0 on MAC OSX. I followed all the installation steps for PIG at https://github.com/ucbtwitter/getting-started/wiki/Installing-Pig .I had set JAVA_HOME correctly as mentioned.
Even after running the ant "-Dhadoopversion=23 jar" command I am getting the same error "Cannot locate pig-core-h2.jar. do 'ant -Dhadoopversion=23 jar', and try again".
This error constantly arising
Cannot locate pig-core-h2.jar. do 'ant -Dhadoopversion=23 jar', and try again.
I studied the shell script by opening pig-0.14.0/bin/pig file and found that this error is related to the setting of CLASSPATH and PIG_HOME and JAVA_HOME variables.
Then I found that I mispelled the PIG_HOME then I corrected it.
Next I ran that specified command('ant -Dhadoopversion=23 jar') in the pig installation directory.
Then I got this error
Not a valid JAR: /Users/../../../pig-0.14.0/pig-0.14.0-SNAPSHOT-core-h2.jar /Users/../../../pig-0.14.0/pig-0.14.0-core-h2.jar
To resolve it remove that jar file in that location.
Then I got it working.
Find the path to the file pig-*-core-h2.jar.
I installed pig using brew install pig and found the jar in the path /usr/local/Cellar/pig/0.17.0/libexec
Run export PIG_HOME=/usr/local/Cellar/pig/0.17.0/libexec
This will fix your error.
i did this to fix the pig
mv /data/mapr/pig/pig-0.14/pig-0.14.0-mapr-1603-core-h2.jar /data/mapr/pig/pig-0.14/pig-0.14.0-mapr-1603-core-h2.jar.orig
The following solutions works:
Please make sure in your .bash_profile or .bashrc you have following environment variables:
export PIG_HOME="/Library/apache-pig-0.15.0"
export PATH="/Library/apache-pig-0.15.0/bin:${PATH}"
restart the machine or restart the unix terminal;
I replaced /Library/apache-pig-0.15.0/ with "home/cwu/Downloads/pig-0.15.0-src
"
I'm familiar with the Cloudera architecture but not MapR. I have a RHEL vm and previously installed the MapR client software using this documentation. I am able to submit mapreduce jobs and query HDFS as expected.
I followed this documentation (after I installed the MapR yum repo) and installed sqoop like so:
yum install mapr-sqoop
If I try to sqoop in some data, or even just issue the command sqoop, I receive the following error:
/opt/mapr/sqoop/sqoop-1.4.4/bin/configure-sqoop: line 47: /opt/mapr/bin/versions.sh: No such file or directory
Error: /opt/mapr/hadoop/hadoop- does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.
I have a /opt/mapr/hadoop/hadoop-0.20.2 directory. I've tried setting HADOOP_COMMON_HOME and HADOOP_HOME to both /opt/mapr/hadoop and /opt/mapr/hadoop/hadoop-0.20.2 yet still receive the same error.
-- Update:
I issued a find / -name hadoop and noted the last line which was /usr/bin/hadoop.
I then set HADOOP_COMMON_HOME to /usr, and was then asked to set HADOOP_MAPRED_HOME, HBASE_HOME, and HCAT_HOME, which I all set to /usr.
This error however is still present:
/opt/mapr/sqoop/sqoop-1.4.4/bin/configure-sqoop: line 47: /opt/mapr/bin/versions.sh: No such file or directory
I opened up this file and commented out line 47. This allowed me to use the sqoop command, but the import job failed and I received a lot of Error: Unsupported major.minor version.
There should be a patch for this if not fixed already,
Here is temp solution:
mapr-client does not give versions.sh , only mapr-core does. Simple fix is to
manually copy that file from a node with mapr-core installed and customize the
versions therein. sqoop then works fine.