I am trying to install hadoop in my VM, and I can visit using :
hdfs dfs -ls hdfs://localhost:9000/
But when I tried to visit from another VM using:
hdfs dfs -ls hdfs://hadoop-vm:9000/
I received a 'connection refused error'.
In the browser I can visit:
http://hadoop-vm:50090 etc
Can anyone tell me how to enable a visit from another vm using hdfs?
Another question is I can not install Hadoop httpfs, and I can not find out any info how to download it at all. Can anyone help me?
-sounds like a firewall or something is blocking access to port 9000. Try to telnet to it both in the VM and remotely.
-otherwise, look at the http://wiki.apache.org/hadoop/ConnectionRefused hints
-don't worry about httpfs.
Related
enter image description here
I have installed the hadoop 2.6.5,when i try to put file from local to hdfs ,there comes this Exception ,i don't know how to solve this problem !!need hlep...
This is going to be a networking problem. The client process (where you ran the hdfs dfs -put command) failed to connect to the DataNode hosts. I can tell from the stack trace that at this point, you have already passed the point of interacting with the NameNode, so connectivity from client to NameNode is fine.
I recommend handling this as a basic network connectivity troubleshooting problem between client and all DataNode hosts. Use tools like ping or nc or telnet to test connectivity. If basic connectivity fails, then resolve it by fixing network configuration.
I have installed spark and hadoop in standalone modes on ubuntu virtualbox for my learning. I am able to do normal hadoop mapreduce operations on hdfs without using spark. But when I use below code in spark-shell,
val file=sc.textFile("hdfs://localhost:9000/in/file")
scala>file.count()
I get "input path does not exist." error. The core-site.xml has fs.defaultFS with value hdfs://localhost:9000. If I give localhost without the port number, I get "Connection refused" error as it is listening on default port 8020. Hostname and localhost are set to loopback addresses 127.0.0.1 and 127.0.1.1 in etc/hosts.
Kindly let me know how to resolve this issue.
Thanks in advance!
I am able to read and write into the hdfs using
"hdfs://localhost:9000/user/<user-name>/..."
Thank you for your help..
Probably your configuration is alright, but the file is missing, or in an unexpected
location...
1) try:
sc.textFile("hdfs://in/file")
sc.textFile("hdfs:///user/<USERNAME>/in/file")
with USERNAME=hadoop, or your own username
2) try on the command line (outside spark-shell) to access that directory/file :
hdfs dfs -ls /in/file
I am installing hadoop on my windows PC. I was able to start yarn and dfs. I ran this command hadoop fs -mkdir /in
It displays the following error:
-mkdir: java.net.UnknownHostException: master
I am newbie Please explain me what has to be done ?
The error indicates the host master cannot be resolved to an IP address. You probably need to add a mapping for it in your hosts file so it can be resolved.
i just start to learn hadoop with hortonworks sandbox
i have HDP 2.3 on a virtualBox and in the setting i have a Bridged Adapter network and a NAT,
when i start the machine everything is ok i can do some hadoop command i can connect the Ambari at 127.0.0.1:8080
but when i run the script in /etc/lib/hue/tools/start_scripts/gen_hosts.sh
to generate hosts with a different ip address every thing going wrong and i can't execute a simple hadoop command like hadoop fs -ls /user
i get this error
ls: Call From sandbox.hortonworks.com/10.24.244.85 to sandbox.hortonworks.com:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
as i said i just started to learn hadoop and i am not a network expert so i will preciate any help
thank you.
I found that you have to restart the services ( HDFS,MapReduce,Hbase,..) from
Ambari when you generate a host.
Hope this will help someone.
turn on name NameNode / HDFS it will work
I have an Amazon Windows VM where i did install MapR-Client 2.1.2, and another MapR cluster waiting for the jobs to be executed.
I set up MAPR_HOME in C:\opt\mapr, and when I execute
hadoop fs -ls /
from C:\opt\mapr\hadoop\hadoop-0.20.2\bin I get:
The system cannot find the path specified
I also configured the MapR-Client with
server\configure.bat -c -C <ip_addr>:7222
and in config\mapr-clusters.conf I can see:
my.cluster.com <ip_addr>:7222
Also I made sure I'm able to ping and ssh from the Windows MapR Client to the cluster master ...
Could someone please help me with that?
Please use
hadoop.bat fs -ls /
and let me know if it's working.