Yarn slave nodes are not communicating with master node? - hadoop

Question

I am not able to see my nodes when I do yarn node -list, even though I have configured /etc/hadoop/conf/yarn-site.xml with the correct properties (it seems to me, at least according to this question Slave nodes not in Yarn ResourceManager).
Here's what I've done so far:
installed resourcemanager on the master
installed nodemanager on the slaves
checked yarn-site.xml for this on ALL the nodes:
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master-node</value>
</property>
after modifying the config file, restarted resourcemanager and nodemanager on the master and slaves, respectively.
But yet when I do yarn node -list I only see
Total Nodes: 0
Node-Id Node-state Node-Http-Address Number-of-Running-Containers
At my nodes, I looked at the .out files in /var/log/hadoop-yarn/ and I see this in them:
ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 244592
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 32768
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 65536
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
EDIT:
when I look at the .log files I see the following, but I'm not sure how to fix it:
INFO org.apache.hadoop.service.AbstractService: Service NodeManager failed in state STARTED; cause:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: <master node ip>:8020:8031 (configuration property 'yarn.resourcemanager.resource-tracker.address')
Caused by: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: <master node ip>:8020:8031 (configuration property 'yarn.resourcemanager.resource-tracker.address')
How do I connect my slave nodes to my master node?

Answer 1

The value set for yarn.resourcemanager.hostname acts as the base value for all the ResourceManager properties. The property yarn.resourcemanager.resource-tracker.address defaults to the value of ${yarn.resourcemanager.hostname}:8031. Refer yarn-default.xml for the complete list of default YARN configurations.
And from the nodemanager ERROR log,
Caused by: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: <master node ip>:8020:8031 (configuration property 'yarn.resourcemanager.resource-tracker.address')
It looks the yarn.resourcemanager.hostname property is configured incorrectly as <master node ip>:8020 instead of <master node ip> on the slave nodes.
Edit the yarn-site.xml on all the nodes to have
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master_node</value> <!-- IP address or Hostname of the node where Resource Manager is started, Omit the port number -->
</property>
Finally, restart the YARN services.

Answer 2

please set all this properties and try
<property>
<name>yarn.resourcemanager.address</name>
<value>master_node:8032</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master_node:8033</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master_node:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master_node:8031</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master_node:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.https.address</name>
<value>master_node:8090</value>
</property>

Answer 3

You need to set ip for yarn.resourcemanager.hostname property. if you want to use the hostname, your machine needs to know to which ip that hostname is pointing to. So you need to add host entry in /etc/hosts file.
To do that,
Open terminal
Type vim /etc/hosts and hit enter
Add this line at end of the file (use key i to enable insertion)
<your resourcemanager ip><space><your hostname>
example: `192.168.1.23 master-node`
Save the file by typing <Esc> + :wq
Restart the nodemanager
I recommend using ambari kind of managing tool to do these kind of stuffs. This allows easy modification of configuration at any time for hadoop environment. Because manual work always have more chance for error.