I am using VM with Ambari 2.2 and HDP 2.3 and installing services using Ambari user interface. Issue is NameNode not starting and log indicates error saying port is in use 50070. I tried netstat and other tools to find out if anything is running on port 50070, it is not. I also tried changing 50070 to 50071 but error remains the same except it now says port is in use 50071.Below is the error I get in ambari error file:
resource_management.core.exceptions.Fail: Execution of 'ambari-sudo.sh su hdfs -l -s /bin/bash -c 'ulimit -c unlimited ; /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config /usr/hdp/current/hadoop-client/conf start namenode'' returned 1. starting namenode, logging to /var/log/hadoop/hdfs/hadoop-hdfs-namenode-hdp-m.out
2016-02-07 11:52:47,058 ERROR namenode.NameNode (NameNode.java:main(1712)) - Failed to start namenode.
java.net.BindException: Port in use: hdp-m.samitsolutions.com:50070
When using Ambari, I came across the port is in use 50070 problem. I found it's actually caused by the mismatch of NameNode's host, not port. In sometimes, Ambari will start namenode on HostB and HostC, while your configure are HostA and HostC.
Such a situation could by caused by: Update wrong namenode config when moving namenode
Related
I have hadoop installed in pseudo-distributed mode.
When running the command
hadoop fs -ls
I am getting the following error:
ls: Call From kali/127.0.1.1 to localhost:9000 failed on connection exception:
java.net.ConnectException: Connection refused;
For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
Any suggestions?
If you read the link in the error, I see two immediate points that need addressed.
If the error message says the remote service is on "127.0.0.1" or "localhost" that means the configuration file is telling the client that the service is on the local server. If your client is trying to talk to a remote system, then your configuration is broken.
You should treat pseudodistributed mode as a remote system, even if it is only running locally.
For HDFS, you can resolve that by putting your computer hostname (preferably the full FQDN for your domain), as the HDFS address in core-site.xml. For your case, hdfs://kali:9000 should be enough
Check that there isn't an entry for your hostname mapped to 127.0.0.1 or 127.0.1.1 in /etc/hosts (Ubuntu is notorious for this).
I'm not completely sure why it needs removed, but the general answer I can think of is that Hadoop is a distributed system, and as I mentioned, treat the pseudodistributed mode as if it's remote HDFS server. Therefore, no loopback addresses should use your computers hostname
For example, remove the second line of this
127.0.0.1 localhost
127.0.1.1 kali
Or remove the hostname from this
127.0.0.1 localhost kali
Most importantly (emphasis added)
None of these are Hadoop problems, they are hadoop, host, network and firewall configuration issues
I am very new to Docker and Hadoop system. I have installed the Docker in Ubuntu 16.04 and run the Hadoop image from Cloudera inside a new Docker container. But when I try to run any command in hdfs the error message is shown as:
Call From quickstart.cloudera/172.17.0.2 to quickstart.cloudera:8020 failed on connection exception: java.net.ConnectException: Connection refused;
I could not figure out how to solve this. I expect for kind help.
Port 8020 is for the hdfs-namenode service, so my guess is that service not started or has failed.
Can you try to restart it?
command: sudo service hadoop-hdfs-namenode restart
You can also check the status of the namenode service.
Command: sudo service hadoop-hdfs-namenode status
Also, check the hadoop-hdfs-datanode service as it may also need to be restarted.
command: sudo service hadoop-hdfs-datanode restart
If you still get the error then check the NameNode logs in /var/log/hadoop-hdfs and add it to your question for further analysis.
In my case, restarting namenode, datanode, and yarn resource manager worked.
sudo service hadoop-yarn-resourcemanager restart
sudo service hadoop-hdfs-namenode restart
sudo service hadoop-hdfs-datanode restart
I installed Hadoop (HDP 2.5.3) on 4 VMs with Ambari (1 Ambari Server and 3 Ambari Clients; with the DNS entries server, node0, node1, node2) with HDFS, YARN, MapReduce and Zookeeper.
However, YARN doesn't want to start. When starting the Resource Manager on node1 I get the following error:
resource_management.core.exceptions.ExecutionFailed: Execution of 'curl -sS -L -w '%{http_code}' -X GET 'http://node0:50070/webhdfs/v1/ats/done/?op=GETFILESTATUS&user.name=hdfs' 1>/tmp/tmpgsiRLj 2>/tmp/tmpMENUFa' returned 7. curl: (7) Failed to connect to node0 port 50070: connection refused 000
App Timeline Server and History Server on node1 don't want to start either. Zookeeper, NameNode, DataNode and Nodemanager on Node0 is up. The nodes can reach each other (tried with ping) so that shouldn't be the problem.
Hopefully one can help me. I'm really new to this topic and not really familiar with the system.
You should check the host file (/etc/hosts), see the host name and FNDN, check if there any duplicates name, IP address.
Could you also confirm the firewall activity by steps:
sudo ufw status
And also check the port in iptables (or allow port in firewall: udp, tcp).
I have a ubuntu server VM in virtual box(in Mac OSX). And I configured a Hadoop Cluster via docker: 1 master(172.17.0.3), 2 slave nodes(172.17.0.4, 172.17.0.6). After run "./sbin/start-dfs.sh" under Hadoop home folder, I found below error in datanode machine:
Datanode denied communication with namenode because hostname cannot be
resolved (ip=172.17.0.4, hostname=172.17.0.4): DatanodeRegistration(0.0.0.0,
datanodeUuid=4c613e35-35b8-41c1-a027-28589e007e78, infoPort=50075,
ipcPort=50020, storageInfo=lv=-55;cid=CID-9bac5643-1f9f-4bc0-abba-
34dba4ddaff6;nsid=1748115706;c=0)
Because docker does not support bidirectional name linking and further more, my docker version does not allow editing /etc/hosts file, So I use IP address to set name node and slaves. Following is my slaves file:
172.17.0.4
172.17.0.6
After searching on google and stackoverflow, no solution works for my problem. However I guess that Hadoop Namenode regard 172.17.0.4 as a "hostname", so it reports "hostname can not be resolved" where "hostname=172.17.0.4".
Any Suggestions?
Finally I got a solution, which proved my suppose:
1.upgrade my docker to 1.4.1, following instructions from: https://askubuntu.com/questions/472412/how-do-i-upgrade-docker.
2.write IP=>hostname mappings of master and slaves into /etc/hosts
3.use hostname instead of ip address in Hadoop slaves file.
4."run ./sbin/start-dfs.sh"
5.Done!
I am getting below error when I am trying to configure hadoop plugin in eclipse.
Error:call to localhost:54310 failed on connection exception:java.net.connectException:Connection refused:no further informaion
Hadoop version is 1.0.4
I have installed hadoop in Linux and I am running my Eclipse using Windows.
In the hadoop location window, I have tried with host as localhost and linux server.
MR Master: Host: localhost and port 54311
DFS Master: Host: localhost and port 54310
MR Master: Host: <Linux server name> and port 54311
DFS Master: Host: <Linux server name> and port 54310
In my mapred-site.xml I see this entry entry localhost:54311.
ConnectionRefused error is, you are trying to connect a directory which you dont have permission to read/write.
This may be caused by, a directory created another user(e.g. root) and your master machine is trying to read from/write to that directory.
It is more likely that you are trying to read an input from wrong place. Check your input directory if there is no problem with it, check your output directory