hadoop 1.1.2 installation on windows - hadoop

I am trying to install hadoop 1.1.2 on window machine with cygwin.
From on-line videos and tutorials, I have set up all most everything.
Now problem is when i try to create folder with commands
cd /usr/local/hadoop-1.1.2/bin --> this works proper and set proper path then
./hadoop dfs -mkdir input --> when this execute i get error
in error i get issue related to JAVA_HOME path is not set proper also show me text like /Java/jre7/bin/bin/java which looks wrong path.
but i have set JAVA_HOME path properly its here,
I have set same path with /bin in path variable.
I don't know where i have made mistake.
EDIT
full Error
./hadoop: line 320: C:/Java/jre7/bin/bin/java: No such file or directory
./hadoop: line 390: C:/Java/jre7/bin/bin/java: No such file or directory
./hadoop: line 390: exec: C:/Java/jre7/bin/bin/java: cannot execute: No such file or directory

problems with its solution
1. JAVA PATH ISSUE
first is JAVA_PATH issue
Note: Have to use JDK not JRE.
For Hadoop try to use folder name without space
In Environment variable
JAVA_HOME = C:\Java\jdk1.7.0_25
In path variable add below code with othres with ;sepration
%JAVA_HOME%\bin
In hadoop-env.sh file (you can find this file from C:\cygwin\usr\local\hadoop-1.1.2\conf if you are using windows machine).
Note that, remove # from starting of line and USE \\ twice in file
export JAVA_HOME=C:\\Java\\jdk1.7.0_25
if everything OK with JAVA_PATH you can check from CYGWIN consol
try below code to get javaPath which hadoop will use
echo $JAVA_HOME
here you will get java version path
also can set JAVAPATH from runtime,try below code on CYGWIN treminal
export JAVA_HOME=C:/JAVA/Jdk1.7.0_25
2. USER ISSUE
first of all Note that when you start with hadoop installation use same USER for MASTER and SALVE.
If you have different user then have to generate one extra file with name config(without extension)
if your MASTER's machine name is jubin-pc and username is jubinp and SLAVES machine name is trainees11 and username is trainees(have to do vice-verse for both)
config file(for MASTER) Location C:\cygwin\home\jubinp\.ssh\
Host trainees11
User trainees
config file(for SLAVE) Location C:\cygwin\home\trainees\.ssh\
Host jubin-pc
User jubinp

Solution for hadoop-2.6.0 and early:
Be sure that path to JDK does not contain backspaces.
(my variant C:\Java\jdk1.8.0_25)
Add JAVA_HOME to path
My Computer -> Properties -> Advanced -> Environment Variables -> Create
JAVA_HOME
C:\Java\jdk1.8.0_25
Add ;%JAVA_HOME%\bin to system Path
Open hadoop-env.sh (It located in C:\hadoop-2.6.0\etc\hadoop for my hadoop-2.6.0)
and add line export JAVA_HOME=C:/Java/Jdk1.8.0_25

Quit Cygwin.
Your path to the bin folder of JAVA contains is in another folder named bin ? I don't think so.
Install a JDK (not a JRE) properly in a Path with no blanks. For example : C:\jdk1.7.0_21
In Windows :
Add an environnement variable JAVA_HOME to C:\jdk1.7.0_21
Then, add the JAVA_HOME/bin to your PATH.
Edit hadoop/conf/hadoop-env.sh : Uncomment the JAVA_HOME Export. For my example :
export JAVA_HOME=/cygdrive/c/jdk1.7.0_21/

Related

Running Kafka on Windows 10 fails: The system cannot find the path specified

I'm trying to run Kafka locally. I've got Zookeeper installed and running. I've downloaded the binaries for Kafka 2.11-1.1.0. I've adjusted the location of the logs in server.config, to a valid path. When I run .\bin\windows\kafka-server-start.bat .\config\server.properties, I get an error that says
The system cannot find the path specified.
I can see that both those files exist on my computer in the proper locations-what should I be troubleshooting?
I faced this issue while running the kafka-server-start.bat command. I double checked to ensure that there was no spaces in the kafka binaries path as well as correct syntax in JAVA_HOME.
Finally realized that the issue was due to a space in the JAVA_HOME path.
C:\Program Files\Java\jdk1.8.0_144
There is a space between Program and Files. I changed the directory of Java and updated the JAVA_HOME variable to
C:\Java\jdk1.7.0_51
This change solved my issue. I used the setx command to change the value in JAVA_HOME.
setx -m JAVA_HOME "C:\Java\jdk1.7.0_51"
I had this problem as well. In my case I have java installed in C:\Java\bin and JAVA_HOME defined as
JAVA_HOME=c:\Java\bin
I needed to change
c:\Tools\kafka_2.12-2.2.0\bin\kafka-run-class.sh
lines 224 to 229 from this
# Which java to use
if [ -z "$JAVA_HOME" ]; then
JAVA="java"
else
JAVA="$JAVA_HOME/bin/java"
fi
to this:
# Which java to use
if [ -z "$JAVA_HOME" ]; then
JAVA="java"
else
JAVA="$JAVA_HOME/java"
fi
because it was assigning java to C:\Java\bin/bin/java
which was then failing on 306 of the same file.
BTW: I'm using a git bash shell in windows. This allows me to run the bin/*.sh scripts instead of the bin/windows/*.bat scripts
Also I changes the value of the dataDir in
C:\Tools\kafka_2.12-2.2.0\config\zookeeper.properties
to
dataDir=C:\\Tools\\kafka_2.12-2.2.0\\zookeeper-data
I had the same problem, but it wasn't because of the malformed path of JAVA_HOME environment variable.
It was because I had extracted my kafka bineries in a path which contains spaces!
First it was here:
E:\Apache Kafka\kafka_2.11-2.1.0
Then I relocate it in this new path (which doesn't have any spaces!):
E:\ApacheKafka\kafka_2.11-2.1.0
And the problem resolved!
Hope this helps.
Turns out that this error is being thrown by kafka-run-class.bat because the path to JAVA_HOME in system variables is malformed (in my case, I had added /bin to it.) Removing /bin from the path, and having it be an existing Java installation folder, makes everything run fine.
As many have mentioned, this can happen if any of the kafka *.bat scripts that you run ( including zookeeper) refer to a bad JAVA_HOME system variable, or one containing a space.
You can solve this by changing JAVA_HOME to the shortened path name.
For example
set JAVA_HOME=C:\Progra~1\Java\jdk-11.0.1
P.S.
For convenience, I created a CMD desktop shortcut that sets the JAVA_HOME to be right for Kafka, and CDs to my kafka installation. My Desktop shortcut is as follows:
%windir%\System32\cmd.exe /k set JAVA_HOME=C:\Progra~1\Java\jdk-11.0.1& f: & cd f:\kafka\kafka_2.12-2.2.0
To add to the existing answer, you can also get the error when running the Kafka provided zookeeper-server-start command (and almost all other scripts in the bin/windows folder) because it also utilizes kafka-run-class, and therefore relies on a valid JAVA_HOME environment variable path.
cd D:\Downloads\kafka
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
Late reply, but I hope it helps someone. I had the same issue, and found someone had updated JAVA to a newer version, i.e. my JAVA_HOME system variable was pointing to a folder that didnt exist. Strange as I hadnt done it, but the answers above inspired me to check it.
I facing same issue and fixed with below steps as following:
Make sure your environment variable as:
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_261
ZOOKEEPER_HOME=C:\apache-zookeeper-3.6.2
KAFKA_HOME=C:\kafka_2.13-2.7.0
Edit the 'Path' in system variable and type:
%JAVA_HOME% / bin, %ZOOPKEEPER_HOME / bin and %KAFKA_HOME% / bin
Now, open the terminal, Run zoopkeeper and kafka
For Zoopkeeper: zkserver
For Kafka: bin\windows\kafka-server-start.bat .\config\server.properties
It runs without an any issues and errors.
I think JAVA_HOME should keep till JDK path not JDK\bin.
I had been stuck with this for quite some time. Following are the steps I followed to resolve the issue:
In enviroment variables set JAVA_HOME to C:\Program Files\Common Files\Oracle\Java\javapath
If the above doesn't resolve the issue, check if in kafka/bin/kafka-run-class.sh JAVA=$JAVA_HOME/bin/java is used. In this case you might need to simply create a folder bin under javapath and copy the java.exe to it. This should fix the isssue.
I tried all the solutions, I changed JAVA_HOME different methods and it didn't work.
Solution:
launch cmd.exe as administrator and execute the command:
set JAVA_HOME=%ProgramFiles%\Java\jdk-17.0.1
(only will choice your jdk version)
In the case of main, in Environment variables set JAVA_HOME in Both part
User variables & system variables And the problem resolved!
Make sure your java_home path is correct with respect to the directory. This issue happens when the java_home path is wrong.
I relocate it into the new path and I provide this zookeeper bin path properly into environment variable setup and it work for me fine.
A simple solution that worked for me -
Open kafka-run-class.sh file, goto line num 306, enclose $JAVA within "".
exec "$JAVA" $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$#"
This problem appears if there a space in your Java path. Either remove the space from your Java installation path or enclose the path within double-quotes.

Location of ~./mavenrc

I was setting up maven on OS X machine when I encountered the error :
JAVA_HOME should point to a JDK not a JRE.
On changing the JAVA_HOME variable to /Library/Java/JavaVirtualMachines/jdk1.7.0_40.jdk/Contents/Home and updating the PATH variable it still didn't work.
Turns out, JAVA_HOME reads from ~.mavenrc and not bash_profile.
So we have to update the JAVA_HOME in ~.mavenrc. But how do I know the location of the same?
The ~ indicates the $HOME directory, so it should be there. The dot behind the file name indicates it's a hidden file, you have to activate View > Show Hidden Files to see it.

Windows7 run elasticsearch.bat Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME

enter image description here
I want to use elasticsearch on Windows64. At first my Java version is not the same as the Java_Home.Then I reinstalled JAVA. And I found a solution said that the system will find JAVA under C:\ProgramData\Oracle\Java\javapath , So I replaced the latest java.exe javaw.exe , javaws.exe there, but the problem still exists.
I found below code in C:\elasticsearch-5.2.1\bin\elasticsearch could I do any changes to this to fix this problem?
enter image description here
Have you tried following?
Check your Environment variables for "JAVA_HOME" and "Path" inside system variables.
The location for JAVA_HOME is your the location of your jdk
For Path variable you should add new "...\Oracle\Java\javapath"
Add C:\ProgramData\Oracle\Java\javapath; to Path variable in Environment variable.
And Remove JAVA_HOME, as you have already Java Path to Path, you don't need to mention it twice.
It will surely help

Error installing WebLogic server using Console mode in windows 8.1

Hi i have been trying to install the server weblogic of oracle on windows 8.1 but I obtain the follow when I run the configure.cmd file:
ERROR: You must set MW_HOME and it must point to a directory where an
installation of WebLogic exists. Ensure you point this variable to the
extract location of the zip distribution.
How do I correct this error?
There's a readme file linked from the product download page http://www.oracle.com/technetwork/middleware/weblogic/downloads/wls-main-097127.html although your experience would suggest that defining MW_HOME isn't optional! ...
1. Extract the contents of the zip to a directory (eg: /home/myhome/mywls)
This will create a base directory named wls12130 under /home/myhome/mywls
MW_HOME will be the entire directory including the base directory.
(eg: MW_HOME will be /home/myhome/mywls/wls12130).
2. Setup JAVA_HOME and optionally, MW_HOME variables in the current shell as required
for the target platform.
Windows
> set JAVA_HOME=C:\home\myhome\myjavahome
> set MW_HOME=C:\home\myhome\mywls\wls12130
3. Run the installation configuration script in the MW_HOME directory.
This step is required to be run only once. If you move the installation to
another location/machine, you need to rerun this step.
Windows
> configure.cmd
Environment variables are not set properly.
1.- Create product directory
mkdir E:\weblogic\wls << I'm sure you did it and Weblogic binaries are already installed.
2.- set environment variables properly
set JAVA_HOME=_path_to_\jdk1.7.0
set MW_HOME=E:\weblogic\wls
(change above settings according to your installation)
3.- Run configure.cmd

mvn command is not recognized as an internal or external command

I am getting the mvn command not recognized as an internal or external command.
I have setup the M2_HOME, JAVA_HOME and added %M2_HOME%/bin to the path variable. All are system variables. Still getting the same problem.
Echoing the variables showing me the correct paths.
Path to Maven: C:\apache-maven-3.1.0\apache-maven-3.1.0
M2_HOME:C:\apache-maven-3.1.0\apache-maven-3.1.0
PATH: Other things,C:\Program Files (x86)\Java\jdk1.7.0_40\bin,C:\apache-maven-3.1.0\apache-maven-3.1.0\bin
and I have restarted my computer twice.
Right click on My Computer >> Properties >> Advanced system settings >>
System Properties window will get displayed
Under Advanced >> Environment Variables
Click on New to set Environment Variables
Variable name: JAVA_HOME
Variable value: C:\Program Files\Java\jdk1.8.0_121
Variable name: M2
Variable value: %M2_HOME%\bin
Variable name: M2_HOME
Variable value: C:\Program Files\Apache Software Foundation\apache-maven-3.5.0
Variable name: Path
Variable value: %M2_HOME%\bin
Then click on Ok, ok, ok.
Now restart you command prompt and check again with “mvn –version” to verify the mvn is running, you may restart your system also.
It should be working now.
Restart your machine, after setting up your M2_HOME (pointing to you Maven basedir, NOT the bin dir) and PATH (PATH=%M2_HOME%\bin;%PATH%).
Then do:
dir %M2_HOME%\bin\mvn*
If there is a .bat file, it should work under Windows, as it appears to be finding it. If there isn't one, then your paths are not right and you need to make sure your %PATH% variable really points to the correct path to Maven.
Make sure you are using the proper slashes for your OS. Under Windows they're \.
Write the entire maven path into the Environment PATH variable.
Example:
C:\Program Files\apache-maven-3.2.3\bin;
My PATH variable wasn't reading %M2% or %M2_HOME%\bin properly, and therefore I wrote the full path into the PATH variable.
Working.
This is worked for me:
1-download apache-maven-3.3.9-bin.tar.gz file from https://maven.apache.org/download.cgi
2-copy the folder under c:\programfiles
3-set variables as this:
M2_HOME ----- C:\Program Files\apache-maven-3.3.9
M2 ----- C:\Program Files\apache-maven-3.3.9
add Path variable to this: ;C:\Program Files\apache-maven-3.3.9\bin
then run the cmd as system admin
it worked.
I also was facing with the same issue still after adding path in environment variable and running it as a normal user in command prompt.
Then I opened command prompt and tried running as "Run as Administrator" and I was able to download all the packages with respect to the project.
Are you trying to reference a user variable in system variables? Try echo %path% and the M2 should have been fully expanded to show the file path to your Maven directory. If it hasn't, then that's the problem.
To fix it, you should create a user variable called PATH and add your %M2% reference into there.
Open Command prompt As "Run As a administrator" and try.
I have a stupid comment but someone else will have that issue. I was getting that same error when I was trying to put in ./mvnw clean package and I found out that I had to change it a bit to .\mvnw clean packageand I lived happily ever after.
Windows 10 -
Add new variable "M2_HOME" -
Update variable "path" -
Verify on cmd -
Try %M2_HOME%\bin (\ rather than /)
I faced this problem which kept me busy and buggy for quiet sometime. I was facing the problem (mvn not recognized) after setting up all required environment variables absolutely correctly. So by going by one of the response here, I switched to another version of maven and that fixed the problem.
Not being completely convinced why it worked this way, I then unzipped the problematic-version and updated env-vars which made it work.
The problem was when I initially extracted file from the zip, I modified the directory structure a bit. When you extract the zip, say apache-maven-X.x.x-bin.zip, it creates the folder structure as - "apache-maven-3.5.0-bin\apache-maven-3.5.0..."
In my first attempt I had modified this structure by deleting apache-maven-3.5.0-bin folder and bringing apache-maven-3.5.0 structure one folder up. This was causing the problem.
Try with echo %path% , if this option doesn't show your M2_HOME and others variable values as directory path, then create a new environment variable lets say PATH, and assign like below:
PATH=%JAVA_HOME%\bin;%M2_HOME%\bin
Add this in variable path=.....;%PATH%
Now open a new cmd, and try to echo %path%
it will show all thh system path
Now you can check mvn -version it will solve the problem , if not try to restart the system
P.S. as per doc, you should expend your zip distribution in C:\Program Files\Apache Software Foundation. But ideally it doen't matter
I'm using Maven 3+ version. In my case everything was fine. But while adding the M2_HOME along with bin directory, I missed the '\' at the end.
Previously it was like: %M2_HOME%\bin , which was throwing the mvn not recognizable error.
After adding "\" at the end, mvn started working fine. I guess "\" acts as pointer to next folder.
"%M2_HOME%\bin\" Should work, if you missed it.
Try setting the path of maven first through command prompt.
setpath.bat Open the cmd from the base window of the batch file.
The rest maven commands can be used once path is set through cmd.
One most important and often overlooked aspect is the %MAVEN_HOME%\bin or %M2_HOME%\bin should be the first thing in the %PATH% environment variable.
If you've already set the JAVA_HOME and M2_HOME (or MAVEN_HOME in my case) environment variables and added the \bin folder to the Path environment variable and still not working, then this solution could be for you.
Make sure that you have set your variables in the right order, your %JAVA_HOME%\bin folder should be added before the %MAVEN_HOME%\bin just like the shown image
I tried all way finally below step solved the issue .
In the downloaded zip there is file README , in that its mention that
for windows
set PATH="c:\program files\apache-maven-3.x.y\bin";%PATH%
create new PATH variable and assign first maven then %PATH%
it worked out for me .
Try It once
In your Environement variable :
new system variable:
M2_HOME Your mvn directroy "C:\......\bin"
new user variable:
M2 %M2_HOME%
edit the CLASSPATH by adding %M2%
finally open the cmd and write
path=%CLASSPATH%
enjoy
I had this same error but my problem was I had the following:
M2_HOME = C:\Program Files (x86)\Apache Software Foundation\apache-maven-2.2.1;
Which meant my PATH = %M2_HOME%\bin; (etc)
...became C:\Program Files (x86)\Apache Software Foundation\apache-maven-2.2.1;\bin
i.e. a semicolon was where it shouldn't be.
Which I discovered because Michael Ferry suggested using 'ECHO %PATH%' to see what the actual PATH output was.
I had the same problem. But just restarting my computer after setting up the Maven path resolved the issue.
Variable Name: M2_Home
Variable Value:C:\Apache\apache-maven-3.3.9
Variable Name: Path
Variable Value:C:\ProgramData\Oracle\Java\javapath;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;%JAVA_HOME%\bin\;%M2_HOME%\bin\
I faced similar problems. The article that helped me solve similar issues is by MKyong and is here: ****https://www.mkyong.com/maven/how-to-install-maven-in-windows/****
It is very important to include in maven's path the file that contains the 'bin','boot', 'conf', 'lib' etc. file folders. For example, in my case, the correct path is:
C:\Program Files\Apache Software Foundation\maven\apache-maven-3.5.0-bin\apache-maven-3.5.0
For me it was not working since I was editing Path variable in "User variable" and adding it under "System variable" made it work. Hope it helps.
For Windows you need to do the following:
Windows and type env
Open the edit environment panel
Click Environment Variables
In the system variables section, double click Path
In the dialog, create a System Variable under Path
like below ->
MVN_HOME: C:\Users<username>\Documents\Project\Software\apache-maven-3.6.3\bin
Open a new command prompt and hit mvn, you should be able to now.
I also faced the issue.
But the problem I faced was due to the location name where the maven was stored on system.
The path did contain spaces due to which system was not able to reach the path leading to this issue.
The issue got resolved for me when I moved the maven to a location which did not contain any spaces.
The existing answers are too complicated. I just fixed the same issue by
scoop install maven
That requires you install scoop the package manager for Windows in the first place.
However I recommend you install the scoop so that everything becomes easy.
Yeah so, for me, I fixed it by setting up the M2 variable.
MAVEN_HOME, JAVA_HOME and M2_HOME we set up with their paths, but the command, "mvn -verison" was still showing the error.
So I inserted one more variable, named, "M2" and then set the path of the maven folder (not the bin of the same) and ran the command in cmd.
It worked.
Adding environment variables from command prompt or windows powershell worked for me. I was able to verify the environment variables present using "set" command in command prompt.
You have written three paths above. The first path (path to maven) should be pointing to the bin directory.
Path to Maven: C:\apache-maven-3.1.0\apache-maven-3.1.0\bin;
Below are right. Above path should be corrected.
M2_HOME:C:\apache-maven-3.1.0\apache-maven-3.1.0;
PATH: Other things,C:\Program Files (x86)\Java\jdk1.7.0_40\bin,C:\apache-maven-3.1.0\apache-maven-3.1.0\bin;

Resources