Where is the classpath set for hadoop - hadoop

Question

Where is the classpath for hadoop set?
When I run the below command it gives me the classpath. Where is the classpath set?
bin/hadoop classpath
I'm using hadoop 2.6.0

Answer 1

Open your bash profile (~/.profile or ~/.bash_profile) for editing and add the following:
export HADOOP_HOME="/usr/local/Cellar/hadoop" then Replace with your own path
export HADOOP_CLASSPATH=$(find $HADOOP_HOME -name '*.jar' | xargs echo | tr ' ' ':')
Save the changes and reload.
source ~/.profile

Answer 2

As said by almas shaikh it's set in hadoop-config.sh, but you could add more jars to it in hadoop-env.sh
Here is a relevant code from hadoop-env.sh which adds additional jars like capacity-scheduler and aws jar's.
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}
# Extra Java CLASSPATH elements. Automatically insert capacity-scheduler.
for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do
if [ "$HADOOP_CLASSPATH" ]; then
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f
else
export HADOOP_CLASSPATH=$f
fi
done
# ... some other lines omitted
# Add Aws jar
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:share/hadoop/tools/lib/*

Answer 3

When you run hadoop command, it sources a file hadoop-config.sh that resides in $HADOOP_HDFS_HOME/libexec which sets your classpath (CLASSPATH) by picking jars residing in various directories viz.
$HADOOP_HDFS_HOME/share/hadoop/mapreduce
$HADOOP_HDFS_HOME/share/hadoop/common
$HADOOP_HDFS_HOME/share/hadoop/hdfs etc.

Answer 4

As per this blog post, it is in an environment variable named HADOOP_CLASSPATH. You can set it as you would any other environment variable, the specifics of which depend on which shell you use. If you use bash, then you can call like export HADOOP_CLASSPATH=/path/to/wherever:/path/to/wherever/else.

Answer 5

I also encountered the problem and have solved it, but my hadoop version is 2.10.1.
I hope it have some help for people who use a newer hadoop version. So far, the following methods should have worked as well in the latest hadoop version 3.3.0.
You just need to edit your .bashrc or .profile, I will give an example of .bashrc.
# edit .bashrc
$ vim ~/.bashrc
Add HADOOP_HOME, PATH of hadoop bin direcotry and HADOOP_CLASSPATH in .bashrc.
# export HADOOP_HOME=${your hadoop install directory}, an example as follows:
export HADOOP_HOME=/usr/local/hadoop-2.10.1
export PATH=${HADOOP_HOME}/bin:${PATH}
export HADOOP_CLASSPATH=`hadoop classpath`
Then,
$ source ~/.bashrc