Configure an ELK cluster in docker containers - elasticsearch

Question

I'm trying to configure an ELK cluster using 2 docker containers.
I'm using the following image:
es241_l240_k461: Elasticsearch 2.4.1, Logstash 2.4.0, and Kibana 4.6.1. (reference: https://hub.docker.com/r/sebp/elk/ )
I have created 2 docker containers for that image with docker-compose; each works perfectly in standalone mode.
I want to link the 2 ELK nodes between each other in way to create the cluster, but I haven't found a proper solution.
The Elasticsearch node in container1 doesn't communicate with the Elasticsearch node in container2.
These are the two docker-compose.yml:
CONTAINER1:
version: '2'
services:
elasticsearch01:
image: sebp/elk:es241_l240_k461
ports:
- "5601:5601"
- "9200:9200"
- "9300:9300"
- "5044:5044"
volumes:
- /opt/ELK1/logstash/conf.d:/etc/logstash/conf.d
privileged: true
CONTAINER2:
version: '2'
services:
elasticsearch02:
image: sebp/elk:es241_l240_k461
ports:
- "5602:5601"
- "9201:9200"
- "9301:9300"
- "5045:5044"
volumes:
- /opt/ELK2/logstash/conf.d:/etc/logstash/conf.d
privileged: true
I'have configured the elasticsearch.yml inside the docker containers in this way:
NODE IN CONTAINER1:
cluster.name: elasticsearchcluster
node.name: node1
network.host: 0.0.0.0
network.bind_host: 0.0.0.0
discovery.zen.ping.unicast.hosts: ["127.0.0.1", "172.21.0.2"]
discovery.zen.minimum_master_nodes: 1
NODE IN CONTAINER2:
cluster.name: elasticsearchcluster
node.name: node2
network.host: 0.0.0.0
network.bind_host: 0.0.0.0
discovery.zen.ping.unicast.hosts: ["127.0.0.1", "172.22.0.2"]
discovery.zen.minimum_master_nodes: 1
The key is the discovery.zen.ping.unicast.hosts parameter: I don't have the real IP address, because it's a docker container.
I tried docker inspect elasticsearch01, I have the following "IPAddress" property:
"NetworkSettings": {
...
"Networks": {
"ELK1_default": {
...
"Gateway": "172.22.0.1",
"IPAddress": "172.22.0.2",
...
}
}
}
But it doesn't work if I set that IP address.
How to configure the cluster properly?
EDIT
Trying the host ip-address and the port, the node 1 starts, the node 2 fails with no errors.
discovery.zen.ping.unicast.hosts: ["127.0.0.1", "192.168.0.1:9300"] -> OK
discovery.zen.ping.unicast.hosts: ["127.0.0.1", "192.168.0.2:9300"] -> FAILS with no errors

Answer 1

Instead of using a prepared docker file with ELK stack, you could go for something like this:
version: '3'
services:
elasticsearch:
image: elasticsearch:2.4.1
ports:
- 9200:9200
networks:
- elk
elasticsearch_slave:
image: elasticsearch:2.4.1
networks:
- elk
depends_on:
- elasticsearch
command: elasticsearch --discovery.zen.ping.unicast.hosts=elasticsearch
logstash:
image: logstash:2.3.3
hostname: logstash
networks:
- elk
volumes:
- ./logstash.conf:/config/logstash.conf
depends_on:
- elasticsearch
ports:
- 5044:5044
command: logstash -f /config/logstash.conf
kibana:
image: kibana:4.5.1
hostname: kibana
networks:
- elk
depends_on:
- elasticsearch
- logstash
ports:
- 5601:5601
networks:
elk:
driver: bridge
Once you start your images using docker-compose up -d, then you can scale slaves using following command docker-compose scale elasticsearch_slave=5
Once that's done - you'll have 5 slaves + client node that opens up port 9200 as a gateway to the whole cluster.
For instance, after doing that, http://localhost:9200/_cat/nodes?v displays following:

Answer 2

Thanks to Evaldas Buinauskas, I've found the solution with the stack!
First, we need only one docker-compose.yml.
In that file we need to configure two services (one for each container to create), and a network to be shared between the two services.
This is the new docker-compose.yml:
version: '2'
services:
elk1:
environment:
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2048m -Xmx2048m"
image: sebp/elk:es241_l240_k461
networks:
- elk_net
ports:
- "5601:5601"
- "9200:9200"
- "9300:9300"
- "5044:5044"
volumes:
- /opt/elk/logstash/conf.d:/etc/logstash/conf.d
privileged: true
elk2:
environment:
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2048m -Xmx2048m"
image: sebp/elk:es241_l240_k461
networks:
- elk_net
ports:
- "5602:5601"
- "9201:9200"
- "9301:9300"
- "5045:5044"
depends_on:
- elk1
volumes:
- /opt/elk/logstash/conf.d:/etc/logstash/conf.d
privileged: true
networks:
elk_net:
driver: bridge
The command docker-compose up will create 3 elements:
the container elk1
the container elk2
the network elk_net
With the docker network inspect elk_net command we can view the (docker) ip addresses assigned to the 2 containers.
The elasticsearch.yml files have to be configured as follows:
cluster.name: elasticsearchcluster
node.name: node1
network.host: 0.0.0.0
network.bind_host: 0.0.0.0
network.publish_host: ${IP_ADDRESS_ELK1}
discovery.zen.ping.unicast.hosts: ["${IP_ADDRESS_ELK2}"]
discovery.zen.minimum_master_nodes: 1
cluster.name: elasticsearchcluster
node.name: node2
network.host: 0.0.0.0
network.bind_host: 0.0.0.0
network.publish_host: ${IP_ADDRESS_ELK2}
discovery.zen.ping.unicast.hosts: ["${IP_ADDRESS_ELK1}"]
discovery.zen.minimum_master_nodes: 1
With this configuration, the cluster works perfectly: the two nodes are merged correctly and the Http get to each elasticsearch server returns all the documents saved in the 2 nodes.