I install hadoop and spark in my windows 11
I put the path in my env variable
"C:\BigData\spark-3.1.2-bin-hadoop3.2"
"C:\BigData\spark-3.1.2-bin-hadoop3.2\sbin"
"C:\BigData\spark-3.1.2-bin-hadoop3.2\bin"
"C:\BigData\hadoop-3.2.2\bin"
and i install jdk 1.8 and i put the path java_home
and when i was to excute the spark whith this cmd
"spark-shell"
i have this problem
"he system cannot find the path specified."
what is the solution ?
Related
I want to installed the hadoop(3.2) in my linux system which has installed the hadoop(2.7).When I execute hadoop , I can only get the information of hadoop 2.7 ,even if I change the environment variable. And the most confused thing is when I run echo $HADOOP_HOME , sometimes I can get the path of hadoop 2.7,sometimes hadoop 3.2. I hope someone can help me.
enter image description here
enter image description here
I am trying to install Hadoop 2.6 in Windows 10, while doing that i'm getting below error
C:\hadoop-2.6.2\bin>hadoop -version
The system cannot find the path specified.
Error: JAVA_HOME is incorrectly set.
Please update C:\hadoop-2.6.2\conf\hadoop-env.cmd
'-Xmx512m' is not recognized as an internal or external command, operable program or batch file.
I had the same issue, and it can be fixed by 2 ways.
check your environment variable, if JAVA_HOME setup for the user or not and path setup correctly or not.
Please remove system JAVA_HOME and setup for user.
Go to command line and setup Java Home and Path.
set JAVA_HOME = your jdk home directory
set PATH = %JAVA_HOME%/bin
I had downloaded spark-2.0.1-bin-hadoop2.7 and installed it. I installed JAVA and set JAVA_HOME in System Variables.
But in running I have this Error:
How to it can be fixed ?
I think the problem is with whitespaces in your path.
Try to place downloaded spark in for example. F:\Msc\BigData\BigDataSeminar\Spark\
Also check whether SPARK_HOME, JAVA_HOME and HADOOP_HOME are placed in the path without whitespaces.
I'm new at Mahout and Hadoop.
I've successfully installed Hadoop Cluster with 3 machines, and the cluster is running fine, and I just installed Mahout on the Main namenode for "testing purposes", and I followed the instructions of installation and set the JAVA_HOME, but when I try to run classify-20newsgroups.sh it goes and download the dataset but after that I get the following error:
Error: JAVA_HOME is not set
Then I've revised the .bashrc and confirmed that the JAVA_HOME is set correctly, but it doesn't help.
Also how do I verify that Mahout is configured to run on Hadoop correctly and do you know of any example that can verify this configuration or environment?
The .bashrc is only read by a shell that is non-login, otherwise is read .bash_profile.
So you could set to read .bashrc from .bash_profile (see here What's the difference between .bashrc, .bash_profile, and .environment?) or just a set JAVA_HOME in .bash_profile.
There are another several possibilities to set the JAVA_HOME:
1) set .bashrc from terminal
~$ source .bashrc
2) set JAVA_HOME in open terminal before running classify-20newsgroups.sh
~$ JAVA_HOME=/path
~$ classify-20newsgroups.sh
3) run classify-20newsgroups.sh with JAVA_HOME, i.e.
~$ JAVA_HOME=/path classify-20newsgroups.sh
As for question about Mahout configuration for run on Hadoop. Standart example with classify-20newsgroups should work on hadoop if HADOOP_HOME is set.
You might need to explicitly set JAVA_HOME in hadoop-env.sh
In hadoop-env.sh, look for the comment "#The java implementation to use", and modify the JAVA_HOME path under it.
It should look something like this:
# The java implementation to use.
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
Of course fix the path of JAVA_HOME.
So I'm trying to run a MapReduce, word count example but I need to have Hadoop running. I tried following instructions from here but it doesn't seem to be working. The problem is the environment variable is not being set. I added the line setenv HADOOP_HOME /opt/hadoop-0.20.2 in /etc/launchd.conf but when I run echo $HADOOP_HOME it doesn't print the path.
Set the HADOOP_HOME variable directly in Accumulo's conf/accumulo-env.sh script.