Failed to send join request to master ElasticSearch on AWS EC2 owned cluster - elasticsearch

Question

I am trying to build a cluster of 3 EC2 instances (I do not want to use the ElasticSearch service of amazon) and after installing the software and configuring it in all three instances I encounter the problem that they do not communicate with each other.
I’m working with ES 5.5.1 on instances with Ubuntu 16.04
All nodes are up and running
All nodes has a Security Groupof AWS with permissions for all traffic between nodes (all ports)
Internal firewall on very machine white list for every node
Master
cluster.name: excelle
node.name: ${HOSTNAME}
node.master: true
path.data: /srv/data
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: 172.31.MAS.TER
discovery.zen.ping.unicast.hosts: ["172.31.MAS.TER", "172.31.NODE.TWO", "172.31.NODE.THREE"]
Node two
cluster.name: excelle
node.name: ${HOSTNAME}
node.master: false
path.data: /srv/data
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: 172.31.NODE.TWO
discovery.zen.ping.unicast.hosts: ["172.31.MAS.TER", "172.31.NODE.TWO", "172.31.NODE.THREE"]
Node 3
cluster.name: excelle
node.name: ${HOSTNAME}
node.master: false
path.data: /srv/data
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: 172.31.NODE.THREE
discovery.zen.ping.unicast.hosts: ["172.31.MAS.TER", "172.31.NODE.TWO", "172.31.NODE.THREE"]
But on logs, on node 3 for exmple...
[2017-08-15T11:01:41,241][INFO ][o.e.d.z.ZenDiscovery ] [es03] failed to send join request to master [{esmaster}{scquEEaETDKMKLHzZvEHZQ}{NdLtMUXtT7WXnv1a4uHWqQ}{172.31.44.107}{172.31.44.107:9300}], reason [RemoteTransportException[[esmaster][172.31.44.107:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[es03][172.31.18.76:9300] connect_timeout[30s]]; nested: IOException[connection timed out: 172.31.18.76/172.31.18.76:9300]; ]
I testing connection from node 3 to master not problem (for network question)
telnet 172.31.MAS.TER 9300
Trying 172.31.MAS.TER...
Connected to 172.31.MAS.TER.
Escape character is '^]'.
What it's wrong? Any idea?

Answer 1

I found an answer to this posted on ElasticSearch
The gem was from manst:
"solution for this error (you must deleted contents of data folder(/var/lib/elasticsearch/nodes/0) and restarted both the servers ):"
I deleted the nodes folder from each of my SpotInst instances and rebooted. My 3 ES distributed master-only nodes all came online. My 8 data-only nodes have connected automatically without any issue.