I am trying to install Hadoop on Fedora 29 and while doing so I encountered with an error as shown below:
[hdadmin#single-node-pseudo-cluster ~]$ vi .bashrc
export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.191.b12-11.fc29.x86_64/
export HADOOP_PREFIX= /usr/local/hadoop-3.2.0/
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_PREFIX/bin
[hdadmin#single-node-pseudo-cluster ~]$ source ~/.bashrc
bash: export: `/usr/local/hadoop-3.2.0/': not a valid identifier
-[the image shows the path of jdk folder][1]
-[the image shows the path of hadoop folder][2]
-Please help me to solve the error, thank you.
[1]: https://i.stack.imgur.com/6p949.png
[2]: https://i.stack.imgur.com/5iefV.png
You need to delete space before path at HADOOP_PREFIX, because the correct syntax to assign a variable is NAME=value.
So, right option: export HADOOP_PREFIX=/usr/local/hadoop-3.2.0/
I just started learning hadoop (CentOS7), here is a question about environment variable:
In my VM, rxie is the logged on user:
HADOOP_CONF_DIR = /opt/hadoop/hadoop-2.7.2/etc/rxie
There is no path of /opt/hadoop/hadoop-2.7.2/etc/rxie
it should be /opt/hadoop/hadoop-2.7.2/etc/hadoop
I am trying to find out where the variable is defined so that I can correct it.
here is what I come up:
bash-4.2# grep -r HADOOP_CONF_DIR ~/.*
/root/./.bashrc:export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
Please note HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop is correct.
I don't see where the erroneous HADOOP_CONF_DIR is defined?
It would be appreciated if anyone can enlighten me and offer a fix, thank you very much.
Run the following:
PS4='+ $BASH_SOURCE:$LINENO:' BASH_XTRACEFD=7 bash -xlic "" 7>trace.out
When that completes, run:
grep HADOOP_CONF_DIR trace.out
The output will show you every time that HADOOP_CONF_DIR was set, the file that set it, and the line number in that file.
Example
Let's take as an example something that I have in my .bashrc file:
$ grep HISTFILESIZE trace.out
++ /home/john1024/.bashrc:193:export HISTFILESIZE=20000
This shows that is was line 193 of ~/.bashrc that set HISTFILESIZE.
How it works
This command starts a bash login shell with diagnostics turned on. The PS4 variable sets the prefix for the diagnostic output so that it contains the file name and line number. For more details, see here.
Other method
Although it is less reliable, one can also try:
grep HADOOP_CONF_DIR ~/.bashrc ~/.bash_profile ~/.profile /etc/profile.d/* /etc/profile /etc/bashrc
Whenever I open a terminal, a message appear on top of it saying what I wrote in the title.
I think I messed up with the /etc/profile file when trying to get maven work.
I read some infos on exporting variables, but couldn't find my error, since I followed a guide that worked fine for other people.
The profile file I mentioned has these lines at bottom:
JAVA_HOME=/usr/local/java/jdk1.8.0
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
JRE_HOME=/usr/local/java/jre1.8.0
PATH=$PATH:$HOME/bin:$JRE_HOME/bin
export JAVA_HOME
export JRE_HOME
export PATH
Setting $JAVA_HOME let me use properly Maven, but creates this annoying message. Can you help me?
As others mentioned in the comments, you are trying to export PATH. instead of PATH somewhere. You might want to check /etc/profile, ~/.bashrc and ~/.bash_profilefiles and try to find this invalid export.
The source you provided is technically ok and you don't need to add any $ signs. Your PATH will contain $HOME/bin value twice though. You might want to change it to look like this:
export JAVA_HOME=/usr/local/java/jdk1.8.0
export JRE_HOME=/usr/local/java/jre1.8.0
export PATH="$PATH:$HOME/bin:$JAVA_HOME/bin:$JRE_HOME/bin"
I am trying to set the Pythonpath on a Ubuntu server without luck.
export PYTHONPATH=$PYTHONPATH:/home/ubuntu/canonicaliser_api
This executes without any errors, however when I want to double check the variable.
export $PYTHONPATH
I get this:
-bash: export: `:/home/ubuntu/canonicaliser_api': not a valid identifier
I can't find anything on google. What is causing this?
The error shown is because you are exporting the value of the shell variable PYTHONPATH which is the path that you specified. Do this instead:
export PYTHONPATH
which tells the shell to export the variable named PYTHONPATH, not its value.
If you simply want to see the value of the variable use
echo $PYTHONPATH
instead of export.
I'm working with Ubuntu 12.04 LTS.
I'm going through the hadoop quickstart manual to make a pseudo-distributed operation. It seems simple and straightforward (easy!).
However, when I try to run start-all.sh I get:
localhost: Error: JAVA_HOME is not set.
I've read all the other advice on stackoverflow for this issue and have done the following to ensure JAVA_HOME is set:
In /etc/hadoop/conf/hadoop-env.sh I have set
JAVA_HOME=/usr/lib/jvm/java-6-oracle
export JAVA_HOME
In /etc/bash.bashrc I have set
JAVA_HOME=/usr/lib/jvm/java-6-oracle
export JAVA_HOME
PATH=$PATH:$JAVA_HOME/bin
export PATH
which java returns:
/usr/bin/java
java –version works
echo $JAVA_HOME returns:
/usr/lib/jvm/java-6-oracle
I've even tried becoming root and explicitly writing the in the terminal:
$ JAVA_HOME=/usr/lib/jvm/java-6-oracle
$ export JAVA_HOME
$ start-all.sh
If you could show me how to resolve this error it would be greatly appreciated.
I'm thinking that my JAVA_HOME is being overridden somehow. If that is the case, could you explain to me how to make my exports global?
I am using hadoop 1.1, and faced the same problem.
I got it solved through changing JAVA_HOME variable in /etc/hadoop/hadoop-env.sh as:
export JAVA_HOME=/usr/lib/jvm/<jdk folder>
The way to solve this problem is to export the JAVA_HOME variable inside the conf/hadoop-env.sh file.
It doesn't matter if you already exported that variable in ~/.bashrc, it'll still show the error.
So edit conf/hadoop-env.sh and uncomment the line "export JAVA_HOME" and add a proper filesystem path to it, i.e. the path to your Java JDK.
# The Java implementation to use. Required.
export JAVA_HOME="/path/to/java/JDK/"
Ran into the same issue on ubuntu LTS 16.04. Running bash -vx ./bin/hadoop showed it tested whether java was a directory. So I changed JAVA_HOME to a folder and it worked.
++ [[ ! -d /usr/bin/java ]]
++ hadoop_error 'ERROR: JAVA_HOME /usr/bin/java does not exist.'
++ echo 'ERROR: JAVA_HOME /usr/bin/java does not exist.'
ERROR: JAVA_HOME /usr/bin/java does not exist.
So I changed JAVA_HOME in ./etc/hadoop/hadoop-env.sh to
export JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre/
and hadoop starts fine.
The way to debug this is to put an "echo $JAVA_HOME" in start-all.sh. Are you running your hadoop environment under a different username, or as yourself? If the former, it's very likely that the JAVA_HOME environment variable is not set for that user.
The other potential problem is that you have specified JAVA_HOME incorrectly, and the value that you have provided doesn't point to a JDK/JRE. Note that "which java" and "java -version" will both work, even if JAVA_HOME is set incorrectly.
extract from etc/hadoop/hadoop-env.sh
The only required environment variable is JAVA_HOME. All others are
optional. When running a distributed configuration it is best to
set JAVA_HOME in this file, so that it is correctly defined on
remote nodes.
This means its better and advised to set JAVA_HOME here.. even though the existing definition reads the JAVA_HOME variable. Perhaps its not getting the value of JAVA_HOME from previously set value... standard apache manual does not tell this :( :(
This error is coming from Line 180
if [[ -z $JAVA_HOME ]]; then
echo "Error: JAVA_HOME is not set and could not be found." 1>&2
exit 1
fi
in libexec/hadoop-config.sh.
Try echo $JAVA_HOME in that script. If it doesn't recognize,
Find your JAVA_HOME using this:
$(readlink -f /usr/bin/javac | sed "s:/bin/javac::")
and replace the line
export JAVA_HOME=${JAVA_HOME}
in /etc/hadoop/hadoop-env.sh with JAVA_HOME you got from above command.
I also had faced the similar problem in hadoop 1.1
I had not noticed that the JAVA_HOME was commented in: hadoop/conf/hadoop-env.sh
It was
/#JAVA_HOME=/usr/lib/jvm/java-6-oracle
Had to change it to
JAVA_HOME=/usr/lib/jvm/java-6-oracle
regardless of debian or any linux flavor, just know that ~/.bash_profile belongs to specific user and is not system wide.
in pseudo-distributed environment hadoop works on localhost so the $JAVA_HOME in .bash_profile is no use anymore.
just export the JAVA_HOME in ~/.bashrc and use it system wide.
Check if your alternatives is pointing to the right one, you might actually be pointing to a different version and trying to alter the hadoop-env.sh on another installed version.
-alternatives --install /etc/hadoop/conf [generic_name] [your correct path] priority {for further check man page of alternatives}
to set alternatives manually,
alternatives --set [generic name] [your current path].
Change the JAVA_HOME variable in conf/hadoop-env.sh
export JAVA_HOME=/etc/local/java/<jdk folder>
echo "export JAVA_HOME=/usr/lib/java" >> $HADOOP_HOME/etc/hadoop/hadoop-env.sh
Notice: Do not use export JAVA_HOME=${JAVA_HOME} !
I put it on the first line of file ~/.bashrc, then it works well!
export JAVA_HOME=/usr/lib/jvm/default-java