How to Setup Spark on Windows 10, Step by Step - windows

I was trying setup spark on windows 10, found lot of good solutions on the stack overflow. So, i am trying to combine all the solutions and create standardized steps of installation

For installation first you need to download following:
JAVA JDK - http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
SBT and Scala - https://www.scala-lang.org/download/
Winutils.exe - https://github.com/steveloughran/winutils/tree/master/hadoop-2.7.1
Spark - https://spark.apache.org/downloads.html
After downloading is completed
Installing and setting up java
When java installation completed, then:
Create folder BigData under C:\
Copy “Java” folder from "C:\Program Files\" --> "C:\"
Then create Environment Variables with name “JAVA_HOME”.
Advance System Settings --> Environment Variables --> Click on New button
Variable Name: JAVA_HOME
Variable Value: C:\Java\jdk1.8.0_181
Add bin to "Path", go to Advance System Settings-->Environment Variables-->Click on Path --> Click on new --> Write
%JAVA_HOME%\bin
Installing and setting up sbt and scala
Install sbt and scala, under the folder C:\Bigdata, after installation is done with sbt and scala
Create Environment Variable with name “SCALA_HOME”.
Advance System Settings --> Environment Variables --> Click on New button
Variable Name: SCALA_HOME
Variable Value: C:\BigData\scala
Add bin to "Path", go to Advance System Settings-->Environment Variables-->Click on Path --> Click on new --> Write,
%SCALA_HOME%\bin
Setting up Hadoop libraries for windows
Download zip from the mentioned git link above, then unzip the downloaded file from git and then, copy the winutils.exe from the “winutils-master\hadoop-2.7.1\bin” folder to C:\Bigdata\hadoop\bin
Create Environment Variable with name "HADOOP_HOME", Advance Settings --> Environment Variables --> Click on New
Variable Name: HADOOP_HOME
Variable Value: C:\BigData\hadoop
Add bin to "Path", go to Advance Settings --> Environment Variables --> Click on Path--> Click on New , and write
%HADOOP_HOME%\bin
Installing and setting up spark
Extract the downloaded package of spark and then copy the folder to C:\Bigdata\, and rename the copied folder to "spark".
Create Environment Variable with name "SPARK_HOME",
Advance Settings --> Environment Variables --> Click on New -->
Variable Name : SPARK_HOME
Variable Value: C:\BigData\spark
Add bin to Path, Advance Settings --> Environment Variables --> Click on Path --> Click on New --> Write
%SPARK_HOME%\bin
Now create /tmp/hive directory under C:\, and set the permissions by following commands:
open cmd prompt:
mkdir c:\tmp
mkdir c:\tmp\hive
winutils chmod 777 /tmp/hive
Now, setup is completed.
goto cmd prompt and type "spark-shell", to run the spark.
Some things that I faced, and it was giving issue:
Your Computer Name should not contain underscore, that was giving me error.
Java JDK need to be installed and version should be Java 1.8.0_181
Multiple Java version configured, and that was giving me issue, there should be only one java version need to be configure.

Related

Unable to locate jenkins.xml file within executable directory or any parents

I've installed jenkins on windows, and I wanted to use jenkins command to start and stop the program from anywhere in the system.
It turns out that after configuring jenkins and creating the environment variable I get the error below everytime I execute "jenkins start" from outside C:\Program Files (x86)\Jenkins.
Is there any configuration missing?
ERROR
System.IO.FileNotFoundException: Unable to locate jenkins.xml file within executable directory or any parents
at winsw.ServiceDescriptor..ctor()
at winsw.WrapperService.Run(String[] _args, ServiceDescriptor descriptor)
at winsw.WrapperService.Main(String[] args)
SYSTEM ENVIRONMENT VARIABLES
JENKINS_HOME:
C:\Program Files (x86)\jenkins
Path:
%JENKINS_HOME%
You need to run your jenkins install jenkins.xml at the place where is your jenkins.xml
1 rename winsw.exe to jenkins.exe
2 cd c:/...where is your jenkins.exe
3 run jenkins.exe install jenkins.xml (both files must be in the same directory)
In Windows 10, the path of the configuration file (config.xml) is:
C:\ProgramData\Jenkins.jenkins
The Folder ProgramData is hidden, so go to the address bar and type or use the CLI.
Solution: Save your jenkins-agent file as xml format. not just jenkins-agent.xml
we can use notepad++ to save the xml type. similarly remove .exe at end of jenkins-agent.exe
Hope its useful.... cheers !!

Error installing WebLogic server using Console mode in windows 8.1

Hi i have been trying to install the server weblogic of oracle on windows 8.1 but I obtain the follow when I run the configure.cmd file:
ERROR: You must set MW_HOME and it must point to a directory where an
installation of WebLogic exists. Ensure you point this variable to the
extract location of the zip distribution.
How do I correct this error?
There's a readme file linked from the product download page http://www.oracle.com/technetwork/middleware/weblogic/downloads/wls-main-097127.html although your experience would suggest that defining MW_HOME isn't optional! ...
1. Extract the contents of the zip to a directory (eg: /home/myhome/mywls)
This will create a base directory named wls12130 under /home/myhome/mywls
MW_HOME will be the entire directory including the base directory.
(eg: MW_HOME will be /home/myhome/mywls/wls12130).
2. Setup JAVA_HOME and optionally, MW_HOME variables in the current shell as required
for the target platform.
Windows
> set JAVA_HOME=C:\home\myhome\myjavahome
> set MW_HOME=C:\home\myhome\mywls\wls12130
3. Run the installation configuration script in the MW_HOME directory.
This step is required to be run only once. If you move the installation to
another location/machine, you need to rerun this step.
Windows
> configure.cmd
Environment variables are not set properly.
1.- Create product directory
mkdir E:\weblogic\wls << I'm sure you did it and Weblogic binaries are already installed.
2.- set environment variables properly
set JAVA_HOME=_path_to_\jdk1.7.0
set MW_HOME=E:\weblogic\wls
(change above settings according to your installation)
3.- Run configure.cmd

How to set up Spark on Windows?

I am trying to setup Apache Spark on Windows.
After searching a bit, I understand that the standalone mode is what I want.
Which binaries do I download in order to run Apache spark in windows? I see distributions with hadoop and cdh at the spark download page.
I don't have references in web to this. A step by step guide to this is highly appreciated.
Steps to install Spark in local mode:
Install Java 7 or later.
To test java installation is complete, open command prompt type java and hit enter.
If you receive a message 'Java' is not recognized as an internal or external command. You need to configure your environment variables, JAVA_HOME and PATH to point to the path of jdk.
Download and install Scala.
Set SCALA_HOME in Control Panel\System and Security\System goto "Adv System settings" and add %SCALA_HOME%\bin in PATH variable in environment variables.
Install Python 2.6 or later from Python Download link.
Download SBT. Install it and set SBT_HOME as an environment variable with value as <<SBT PATH>>.
Download winutils.exe from HortonWorks repo or git repo. Since we don't have a local Hadoop installation on Windows we have to download winutils.exe and place it in a bin directory under a created Hadoop home directory.
Set HADOOP_HOME = <<Hadoop home directory>> in environment variable.
We will be using a pre-built Spark package, so choose a Spark pre-built package for Hadoop Spark download. Download and extract it.
Set SPARK_HOME and add %SPARK_HOME%\bin in PATH variable in environment variables.
Run command: spark-shell
Open http://localhost:4040/ in a browser to see the SparkContext web UI.
I found the easiest solution on Windows is to build from source.
You can pretty much follow this guide: http://spark.apache.org/docs/latest/building-spark.html
Download and install Maven, and set MAVEN_OPTS to the value specified in the guide.
But if you're just playing around with Spark, and don't actually need it to run on Windows for any other reason that your own machine is running Windows, I'd strongly suggest you install Spark on a linux virtual machine. The simplest way to get started probably is to download the ready-made images made by Cloudera or Hortonworks, and either use the bundled version of Spark, or install your own from source or the compiled binaries you can get from the spark website.
You can download spark from here:
http://spark.apache.org/downloads.html
I recommend you this version: Hadoop 2 (HDP2, CDH5)
Since version 1.0.0 there are .cmd scripts to run spark in windows.
Unpack it using 7zip or similar.
To start you can execute /bin/spark-shell.cmd --master local[2]
To configure your instance, you can follow this link: http://spark.apache.org/docs/latest/
You can use following ways to setup Spark:
Building from Source
Using prebuilt release
Though there are various ways to build Spark from Source.
First I tried building Spark source with SBT but that requires hadoop. To avoid those issues, I used pre-built release.
Instead of Source,I downloaded Prebuilt release for hadoop 2.x version and ran it.
For this you need to install Scala as prerequisite.
I have collated all steps here :
How to run Apache Spark on Windows7 in standalone mode
Hope it'll help you..!!!
Trying to work with spark-2.x.x, building Spark source code didn't work for me.
So, although I'm not going to use Hadoop, I downloaded the pre-built Spark with hadoop embeded : spark-2.0.0-bin-hadoop2.7.tar.gz
Point SPARK_HOME on the extracted directory, then add to PATH: ;%SPARK_HOME%\bin;
Download the executable winutils from the Hortonworks repository, or from Amazon AWS platform winutils.
Create a directory where you place the executable winutils.exe. For example, C:\SparkDev\x64. Add the environment variable %HADOOP_HOME% which points to this directory, then add %HADOOP_HOME%\bin to PATH.
Using command line, create the directory:
mkdir C:\tmp\hive
Using the executable that you downloaded, add full permissions to the file directory you created but using the unixian formalism:
%HADOOP_HOME%\bin\winutils.exe chmod 777 /tmp/hive
Type the following command line:
%SPARK_HOME%\bin\spark-shell
Scala command line input should be shown automatically.
Remark : You don't need to configure Scala separately. It's built-in too.
Here's the fixes to get it to run in Windows without rebuilding everything - such as if you do not have a recent version of MS-VS. (You will need a Win32 C++ compiler, but you can install MS VS Community Edition free.)
I've tried this with Spark 1.2.2 and mahout 0.10.2 as well as with the latest versions in November 2015. There are a number of problems including the fact that the Scala code tries to run a bash script (mahout/bin/mahout) which does not work of course, the sbin scripts have not been ported to windows, and the winutils are missing if hadoop is not installed.
(1) Install scala, then unzip spark/hadoop/mahout into the root of C: under their respective product names.
(2) Rename \mahout\bin\mahout to mahout.sh.was (we will not need it)
(3) Compile the following Win32 C++ program and copy the executable to a file named C:\mahout\bin\mahout (that's right - no .exe suffix, like a Linux executable)
#include "stdafx.h"
#define BUFSIZE 4096
#define VARNAME TEXT("MAHOUT_CP")
int _tmain(int argc, _TCHAR* argv[]) {
DWORD dwLength; LPTSTR pszBuffer;
pszBuffer = (LPTSTR)malloc(BUFSIZE*sizeof(TCHAR));
dwLength = GetEnvironmentVariable(VARNAME, pszBuffer, BUFSIZE);
if (dwLength > 0) { _tprintf(TEXT("%s\n"), pszBuffer); return 0; }
return 1;
}
(4) Create the script \mahout\bin\mahout.bat and paste in the content below, although the exact names of the jars in the _CP class paths will depend on the versions of spark and mahout. Update any paths per your installation. Use 8.3 path names without spaces in them. Note that you cannot use wildcards/asterisks in the classpaths here.
set SCALA_HOME=C:\Progra~2\scala
set SPARK_HOME=C:\spark
set HADOOP_HOME=C:\hadoop
set MAHOUT_HOME=C:\mahout
set SPARK_SCALA_VERSION=2.10
set MASTER=local[2]
set MAHOUT_LOCAL=true
set path=%SCALA_HOME%\bin;%SPARK_HOME%\bin;%PATH%
cd /D %SPARK_HOME%
set SPARK_CP=%SPARK_HOME%\conf\;%SPARK_HOME%\lib\xxx.jar;...other jars...
set MAHOUT_CP=%MAHOUT_HOME%\lib\xxx.jar;...other jars...;%MAHOUT_HOME%\xxx.jar;...other jars...;%SPARK_CP%;%MAHOUT_HOME%\lib\spark\xxx.jar;%MAHOUT_HOME%\lib\hadoop\xxx.jar;%MAHOUT_HOME%\src\conf;%JAVA_HOME%\lib\tools.jar
start "master0" "%JAVA_HOME%\bin\java" -cp "%SPARK_CP%" -Xms1g -Xmx1g org.apache.spark.deploy.master.Master --ip localhost --port 7077 --webui-port 8082 >>out-master0.log 2>>out-master0.err
start "worker1" "%JAVA_HOME%\bin\java" -cp "%SPARK_CP%" -Xms1g -Xmx1g org.apache.spark.deploy.worker.Worker spark://localhost:7077 --webui-port 8083 >>out-worker1.log 2>>out-worker1.err
...you may add more workers here...
cd /D %MAHOUT_HOME%
"%JAVA_HOME%\bin\java" -Xmx4g -classpath "%MAHOUT_CP%" "org.apache.mahout.sparkbindings.shell.Main"
The name of the variable MAHOUT_CP should not be changed, as it is referenced in the C++ code.
Of course you can comment-out the code that launches the Spark master and worker because Mahout will run Spark as-needed; I just put it in the batch job to show you how to launch it if you wanted to use Spark without Mahout.
(5) The following tutorial is a good place to begin:
https://mahout.apache.org/users/sparkbindings/play-with-shell.html
You can bring up the Mahout Spark instance at:
"C:\Program Files (x86)\Google\Chrome\Application\chrome" --disable-web-security http://localhost:4040
The guide by Ani Menon (thx!) almost worked for me on windows 10, i just had to get a newer winutils.exe off that git (currently hadoop-2.8.1): https://github.com/steveloughran/winutils
Here are seven steps to install spark on windows 10 and run it from python:
Step 1: download the spark 2.2.0 tar (tape Archive) gz file to any folder F from this link - https://spark.apache.org/downloads.html. Unzip it and copy the unzipped folder to the desired folder A. Rename the spark-2.2.0-bin-hadoop2.7 folder to spark.
Let path to the spark folder be C:\Users\Desktop\A\spark
Step 2: download the hardoop 2.7.3 tar gz file to the same folder F from this link - https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz. Unzip it and copy the unzipped folder to the same folder A. Rename the folder name from Hadoop-2.7.3.tar to hadoop.
Let path to the hadoop folder be C:\Users\Desktop\A\hadoop
Step 3: Create a new notepad text file. Save this empty notepad file as winutils.exe (with Save as type: All files). Copy this O KB winutils.exe file to your bin folder in spark - C:\Users\Desktop\A\spark\bin
Step 4: Now, we have to add these folders to the System environment.
4a: Create a system variable (not user variable as user variable will inherit all the properties of the system variable) Variable name: SPARK_HOME
Variable value: C:\Users\Desktop\A\spark
Find Path system variable and click edit. You will see multiple paths. Do not delete any of the paths. Add this variable value - ;C:\Users\Desktop\A\spark\bin
4b: Create a system variable
Variable name: HADOOP_HOME
Variable value: C:\Users\Desktop\A\hadoop
Find Path system variable and click edit. Add this variable value - ;C:\Users\Desktop\A\hadoop\bin
4c: Create a system variable Variable name: JAVA_HOME
Search Java in windows. Right click and click open file location. You will have to again right click on any one of the java files and click on open file location. You will be using the path of this folder. OR you can search for C:\Program Files\Java. My Java version installed on the system is jre1.8.0_131.
Variable value: C:\Program Files\Java\jre1.8.0_131\bin
Find Path system variable and click edit. Add this variable value - ;C:\Program Files\Java\jre1.8.0_131\bin
Step 5: Open command prompt and go to your spark bin folder (type cd C:\Users\Desktop\A\spark\bin). Type spark-shell.
C:\Users\Desktop\A\spark\bin>spark-shell
It may take time and give some warnings. Finally, it will show
welcome to spark version 2.2.0
Step 6: Type exit() or restart the command prompt and go the spark bin folder again. Type pyspark:
C:\Users\Desktop\A\spark\bin>pyspark
It will show some warnings and errors but ignore. It works.
Step 7: Your download is complete. If you want to directly run spark from python shell then:
go to Scripts in your python folder and type
pip install findspark
in command prompt.
In python shell
import findspark
findspark.init()
import the necessary modules
from pyspark import SparkContext
from pyspark import SparkConf
If you would like to skip the steps for importing findspark and initializing it, then please follow the procedure given in
importing pyspark in python shell
Here is a simple minimum script to run from any python console.
It assumes that you have extracted the Spark libraries that you have downloaded into C:\Apache\spark-1.6.1.
This works in Windows without building anything and solves problems where Spark would complain about recursive pickling.
import sys
import os
spark_home = 'C:\Apache\spark-1.6.1'
sys.path.insert(0, os.path.join(spark_home, 'python'))
sys.path.insert(0, os.path.join(spark_home, 'python\lib\pyspark.zip'))
sys.path.insert(0, os.path.join(spark_home, 'python\lib\py4j-0.9-src.zip'))
# Start a spark context:
sc = pyspark.SparkContext()
#
lines = sc.textFile(os.path.join(spark_home, "README.md")
pythonLines = lines.filter(lambda line: "Python" in line)
pythonLines.first()
Cloudera and Hortonworks are the best tools to start up with the
HDFS in Microsoft Windows. You can also use VMWare or VBox to initiate Virtual Machine to establish build to your HDFS and Spark, Hive, HBase, Pig, Hadoop with Scala, R, Java, Python.

mvn command is not recognized as an internal or external command

I am getting the mvn command not recognized as an internal or external command.
I have setup the M2_HOME, JAVA_HOME and added %M2_HOME%/bin to the path variable. All are system variables. Still getting the same problem.
Echoing the variables showing me the correct paths.
Path to Maven: C:\apache-maven-3.1.0\apache-maven-3.1.0
M2_HOME:C:\apache-maven-3.1.0\apache-maven-3.1.0
PATH: Other things,C:\Program Files (x86)\Java\jdk1.7.0_40\bin,C:\apache-maven-3.1.0\apache-maven-3.1.0\bin
and I have restarted my computer twice.
Right click on My Computer >> Properties >> Advanced system settings >>
System Properties window will get displayed
Under Advanced >> Environment Variables
Click on New to set Environment Variables
Variable name: JAVA_HOME
Variable value: C:\Program Files\Java\jdk1.8.0_121
Variable name: M2
Variable value: %M2_HOME%\bin
Variable name: M2_HOME
Variable value: C:\Program Files\Apache Software Foundation\apache-maven-3.5.0
Variable name: Path
Variable value: %M2_HOME%\bin
Then click on Ok, ok, ok.
Now restart you command prompt and check again with “mvn –version” to verify the mvn is running, you may restart your system also.
It should be working now.
Restart your machine, after setting up your M2_HOME (pointing to you Maven basedir, NOT the bin dir) and PATH (PATH=%M2_HOME%\bin;%PATH%).
Then do:
dir %M2_HOME%\bin\mvn*
If there is a .bat file, it should work under Windows, as it appears to be finding it. If there isn't one, then your paths are not right and you need to make sure your %PATH% variable really points to the correct path to Maven.
Make sure you are using the proper slashes for your OS. Under Windows they're \.
Write the entire maven path into the Environment PATH variable.
Example:
C:\Program Files\apache-maven-3.2.3\bin;
My PATH variable wasn't reading %M2% or %M2_HOME%\bin properly, and therefore I wrote the full path into the PATH variable.
Working.
This is worked for me:
1-download apache-maven-3.3.9-bin.tar.gz file from https://maven.apache.org/download.cgi
2-copy the folder under c:\programfiles
3-set variables as this:
M2_HOME ----- C:\Program Files\apache-maven-3.3.9
M2 ----- C:\Program Files\apache-maven-3.3.9
add Path variable to this: ;C:\Program Files\apache-maven-3.3.9\bin
then run the cmd as system admin
it worked.
I also was facing with the same issue still after adding path in environment variable and running it as a normal user in command prompt.
Then I opened command prompt and tried running as "Run as Administrator" and I was able to download all the packages with respect to the project.
Are you trying to reference a user variable in system variables? Try echo %path% and the M2 should have been fully expanded to show the file path to your Maven directory. If it hasn't, then that's the problem.
To fix it, you should create a user variable called PATH and add your %M2% reference into there.
Open Command prompt As "Run As a administrator" and try.
I have a stupid comment but someone else will have that issue. I was getting that same error when I was trying to put in ./mvnw clean package and I found out that I had to change it a bit to .\mvnw clean packageand I lived happily ever after.
Windows 10 -
Add new variable "M2_HOME" -
Update variable "path" -
Verify on cmd -
Try %M2_HOME%\bin (\ rather than /)
I faced this problem which kept me busy and buggy for quiet sometime. I was facing the problem (mvn not recognized) after setting up all required environment variables absolutely correctly. So by going by one of the response here, I switched to another version of maven and that fixed the problem.
Not being completely convinced why it worked this way, I then unzipped the problematic-version and updated env-vars which made it work.
The problem was when I initially extracted file from the zip, I modified the directory structure a bit. When you extract the zip, say apache-maven-X.x.x-bin.zip, it creates the folder structure as - "apache-maven-3.5.0-bin\apache-maven-3.5.0..."
In my first attempt I had modified this structure by deleting apache-maven-3.5.0-bin folder and bringing apache-maven-3.5.0 structure one folder up. This was causing the problem.
Try with echo %path% , if this option doesn't show your M2_HOME and others variable values as directory path, then create a new environment variable lets say PATH, and assign like below:
PATH=%JAVA_HOME%\bin;%M2_HOME%\bin
Add this in variable path=.....;%PATH%
Now open a new cmd, and try to echo %path%
it will show all thh system path
Now you can check mvn -version it will solve the problem , if not try to restart the system
P.S. as per doc, you should expend your zip distribution in C:\Program Files\Apache Software Foundation. But ideally it doen't matter
I'm using Maven 3+ version. In my case everything was fine. But while adding the M2_HOME along with bin directory, I missed the '\' at the end.
Previously it was like: %M2_HOME%\bin , which was throwing the mvn not recognizable error.
After adding "\" at the end, mvn started working fine. I guess "\" acts as pointer to next folder.
"%M2_HOME%\bin\" Should work, if you missed it.
Try setting the path of maven first through command prompt.
setpath.bat Open the cmd from the base window of the batch file.
The rest maven commands can be used once path is set through cmd.
One most important and often overlooked aspect is the %MAVEN_HOME%\bin or %M2_HOME%\bin should be the first thing in the %PATH% environment variable.
If you've already set the JAVA_HOME and M2_HOME (or MAVEN_HOME in my case) environment variables and added the \bin folder to the Path environment variable and still not working, then this solution could be for you.
Make sure that you have set your variables in the right order, your %JAVA_HOME%\bin folder should be added before the %MAVEN_HOME%\bin just like the shown image
I tried all way finally below step solved the issue .
In the downloaded zip there is file README , in that its mention that
for windows
set PATH="c:\program files\apache-maven-3.x.y\bin";%PATH%
create new PATH variable and assign first maven then %PATH%
it worked out for me .
Try It once
In your Environement variable :
new system variable:
M2_HOME Your mvn directroy "C:\......\bin"
new user variable:
M2 %M2_HOME%
edit the CLASSPATH by adding %M2%
finally open the cmd and write
path=%CLASSPATH%
enjoy
I had this same error but my problem was I had the following:
M2_HOME = C:\Program Files (x86)\Apache Software Foundation\apache-maven-2.2.1;
Which meant my PATH = %M2_HOME%\bin; (etc)
...became C:\Program Files (x86)\Apache Software Foundation\apache-maven-2.2.1;\bin
i.e. a semicolon was where it shouldn't be.
Which I discovered because Michael Ferry suggested using 'ECHO %PATH%' to see what the actual PATH output was.
I had the same problem. But just restarting my computer after setting up the Maven path resolved the issue.
Variable Name: M2_Home
Variable Value:C:\Apache\apache-maven-3.3.9
Variable Name: Path
Variable Value:C:\ProgramData\Oracle\Java\javapath;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;%JAVA_HOME%\bin\;%M2_HOME%\bin\
I faced similar problems. The article that helped me solve similar issues is by MKyong and is here: ****https://www.mkyong.com/maven/how-to-install-maven-in-windows/****
It is very important to include in maven's path the file that contains the 'bin','boot', 'conf', 'lib' etc. file folders. For example, in my case, the correct path is:
C:\Program Files\Apache Software Foundation\maven\apache-maven-3.5.0-bin\apache-maven-3.5.0
For me it was not working since I was editing Path variable in "User variable" and adding it under "System variable" made it work. Hope it helps.
For Windows you need to do the following:
Windows and type env
Open the edit environment panel
Click Environment Variables
In the system variables section, double click Path
In the dialog, create a System Variable under Path
like below ->
MVN_HOME: C:\Users<username>\Documents\Project\Software\apache-maven-3.6.3\bin
Open a new command prompt and hit mvn, you should be able to now.
I also faced the issue.
But the problem I faced was due to the location name where the maven was stored on system.
The path did contain spaces due to which system was not able to reach the path leading to this issue.
The issue got resolved for me when I moved the maven to a location which did not contain any spaces.
The existing answers are too complicated. I just fixed the same issue by
scoop install maven
That requires you install scoop the package manager for Windows in the first place.
However I recommend you install the scoop so that everything becomes easy.
Yeah so, for me, I fixed it by setting up the M2 variable.
MAVEN_HOME, JAVA_HOME and M2_HOME we set up with their paths, but the command, "mvn -verison" was still showing the error.
So I inserted one more variable, named, "M2" and then set the path of the maven folder (not the bin of the same) and ran the command in cmd.
It worked.
Adding environment variables from command prompt or windows powershell worked for me. I was able to verify the environment variables present using "set" command in command prompt.
You have written three paths above. The first path (path to maven) should be pointing to the bin directory.
Path to Maven: C:\apache-maven-3.1.0\apache-maven-3.1.0\bin;
Below are right. Above path should be corrected.
M2_HOME:C:\apache-maven-3.1.0\apache-maven-3.1.0;
PATH: Other things,C:\Program Files (x86)\Java\jdk1.7.0_40\bin,C:\apache-maven-3.1.0\apache-maven-3.1.0\bin;

hadoop 1.1.2 installation on windows

I am trying to install hadoop 1.1.2 on window machine with cygwin.
From on-line videos and tutorials, I have set up all most everything.
Now problem is when i try to create folder with commands
cd /usr/local/hadoop-1.1.2/bin --> this works proper and set proper path then
./hadoop dfs -mkdir input --> when this execute i get error
in error i get issue related to JAVA_HOME path is not set proper also show me text like /Java/jre7/bin/bin/java which looks wrong path.
but i have set JAVA_HOME path properly its here,
I have set same path with /bin in path variable.
I don't know where i have made mistake.
EDIT
full Error
./hadoop: line 320: C:/Java/jre7/bin/bin/java: No such file or directory
./hadoop: line 390: C:/Java/jre7/bin/bin/java: No such file or directory
./hadoop: line 390: exec: C:/Java/jre7/bin/bin/java: cannot execute: No such file or directory
problems with its solution
1. JAVA PATH ISSUE
first is JAVA_PATH issue
Note: Have to use JDK not JRE.
For Hadoop try to use folder name without space
In Environment variable
JAVA_HOME = C:\Java\jdk1.7.0_25
In path variable add below code with othres with ;sepration
%JAVA_HOME%\bin
In hadoop-env.sh file (you can find this file from C:\cygwin\usr\local\hadoop-1.1.2\conf if you are using windows machine).
Note that, remove # from starting of line and USE \\ twice in file
export JAVA_HOME=C:\\Java\\jdk1.7.0_25
if everything OK with JAVA_PATH you can check from CYGWIN consol
try below code to get javaPath which hadoop will use
echo $JAVA_HOME
here you will get java version path
also can set JAVAPATH from runtime,try below code on CYGWIN treminal
export JAVA_HOME=C:/JAVA/Jdk1.7.0_25
2. USER ISSUE
first of all Note that when you start with hadoop installation use same USER for MASTER and SALVE.
If you have different user then have to generate one extra file with name config(without extension)
if your MASTER's machine name is jubin-pc and username is jubinp and SLAVES machine name is trainees11 and username is trainees(have to do vice-verse for both)
config file(for MASTER) Location C:\cygwin\home\jubinp\.ssh\
Host trainees11
User trainees
config file(for SLAVE) Location C:\cygwin\home\trainees\.ssh\
Host jubin-pc
User jubinp
Solution for hadoop-2.6.0 and early:
Be sure that path to JDK does not contain backspaces.
(my variant C:\Java\jdk1.8.0_25)
Add JAVA_HOME to path
My Computer -> Properties -> Advanced -> Environment Variables -> Create
JAVA_HOME
C:\Java\jdk1.8.0_25
Add ;%JAVA_HOME%\bin to system Path
Open hadoop-env.sh (It located in C:\hadoop-2.6.0\etc\hadoop for my hadoop-2.6.0)
and add line export JAVA_HOME=C:/Java/Jdk1.8.0_25
Quit Cygwin.
Your path to the bin folder of JAVA contains is in another folder named bin ? I don't think so.
Install a JDK (not a JRE) properly in a Path with no blanks. For example : C:\jdk1.7.0_21
In Windows :
Add an environnement variable JAVA_HOME to C:\jdk1.7.0_21
Then, add the JAVA_HOME/bin to your PATH.
Edit hadoop/conf/hadoop-env.sh : Uncomment the JAVA_HOME Export. For my example :
export JAVA_HOME=/cygdrive/c/jdk1.7.0_21/

Resources