I believe it is simple question but I still do not get it from Docker-compose documentations. What is the difference between links and external_links?
I like external_links as I want to have core docker-compose and I want to extend it without overriding the core links.
What exactly I have, I am trying to setup logstash which depends on the elasticsearch. Elasticsearch is in the core docker-compose and the logstash is in the depending one. So I had to define the elastic search in the depended docker-compose as a reference as logstash need it as a link. BUT Elasticsearch has already its own links which I do not want to repeat them in the dependent one.
Can I do that with external_link instead of link?
I know that links will make sure that the link is up first before linking, does the external_link will do the same?
Any help is appreciated. Thanks.
Use links when you want to link together containers within the same docker-compose.yml. All you need to do is set the link to the service name. Like this:
---
elasticsearch:
image: elasticsearch:latest
command: elasticsearch -Des.network.host=0.0.0.0
ports:
- "9200:9200"
logstash:
image: logstash:latest
command: logstash -f logstash.conf
ports:
- "5000:5000"
links:
- elasticsearch
If you want to link a container inside of the docker-compose.yml to another container that was not included in the same docker-compose.yml or started in a different manner then you can use external_links and you would set the link to the container's name. Like this:
---
logstash:
image: logstash:latest
command: logstash -f logstash.conf
ports:
- "5000:5000"
external_links:
- my_elasticsearch_container
I would suggest the first way unless your use case for some reason requires that they cannot be in the same docker-compose.yml
I think external_link will do not do the same as links in docker-compose up command.
links waits for container to boot up and get IP address which is used in etc/hosts file, therefore external_link has already IP:hostname values name described in docker-compose file.
Moreover links will be deprecated
Here is a link to Docker-Compose project that uses Elasticsearch, Logstash, and Kibana. You will see that I'm using links:
https://github.com/bahaaldine/elasticsearch-paris-accidentology-demo
Related
i don't know how to approach my problem because i don`t find similar cases to have an example.
I want to setup influx with 2 buckets to save telegraf data but only setups with init bucket.
These are the two influxdb services in my docker composer file:
influxdb:
image: influxdb:latest
volumes:
- ./influxdbv2:/root/.influxdbv2
environment:
# Use these same configurations parameters in your telegraf configuration, mytelegraf.conf.
- DOCKER_INFLUXDB_INIT_MODE=setup
- DOCKER_INFLUXDB_INIT_USERNAME=User
- DOCKER_INFLUXDB_INIT_PASSWORD=****
- DOCKER_INFLUXDB_INIT_ORG=org
- DOCKER_INFLUXDB_INIT_BUCKET=data
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=****
ports:
- "8086:8086"
influxdb_cli:
image: influxdb:latest
links:
- influxdb
volumes:
# Mount for influxdb data directory and configuration
- ./influxdbv2:/root/.influxdbv2
entrypoint: ["./entrypoint.sh"]
restart: on-failure:10
depends_on:
- influxdb
when inits runs influxdb setup correctly but doesn`t run the script and telegraf returns 404 when trying to write to buckets.
I ran into the same issue today and as far as I am aware you cannot currently initialize two buckets with the DOCKER_INFLUXDB_INIT_BUCKET environment variable.
So I created a shellscript called createSecondBucket.sh that I found in another answer for this question. It uses the influx cli to create a new bucket. The script looks like this:
#!/bin/sh
set -e
influx bucket create -n YOUR_BUCKET_NAME -o YOUR_ORG_NAME -r 0
Note that I had to change the line endings to unix (LF) to run the script without errors.
Inside my Dockerfile I added the following lines:
COPY ./createSecondBucket.sh /docker-entrypoint-initdb.d
RUN chmod +x /docker-entrypoint-initdb.d/createSecondBucket.sh
which have the effect that the script is executed after the container starts for the first time. I found this information on the MongoDB dockerhub page which you can find here under the "Initializing a fresh instance" headline.
Context:
I am currently trying to create a Jenkins job that builds periodically and updates the images that are in my docker-compose file. I managed to get a basic version of this to work by labeling my services in my docker-compose.yml. For example:
gitlab:
image: 'gitlab/gitlab-ce:latest'
container_name: 'gitlab'
labels:
update: 'notify'
...
letsencrypt:
image: 'jrcs/letsencrypt-nginx-proxy-companion'
container_name: 'letsencrypt-companion'
labels:
update: 'auto'
...
Notify meant that it should pull new docker images periodically and notify me that an image is ready to be updated. Auto means that it is allowed to automatically deploy the new image.
Problem:
I want to make it so that when new images are pulled Jenkins will automatically notify my that new images are ready / deployed. The problem however is that I have to interpret the output of docker-compose pull and docker-compose up -d to know which images were actually new and deployed. I need a solution that works for a Jenkins pipeline (declarative or scripted)
try to watch this video https://www.youtube.com/watch?v=ZL3hMP9BdmQ; I think that's you are looking for.
https://github.com/v2tec/watchtower
"If you mount the config file as described below, be sure to also prepend the url for the registry when starting up your watched image (you can omit the https://). Here is a complete docker-compose.yml file that starts up a docker container from a private repo at dockerhub and monitors it with watchtower. Note the command argument changing the interval to 30s rather than the default 5 minutes."
version: "3"
services:
cavo:
image: index.docker.io/<org>/<image>:<tag>
ports:
- "443:3443"
- "80:3080"
watchtower:
image: v2tec/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /root/.docker/config.json:/config.json
command: --interval 30
I wanted to use Kibana docker image.
Below is the docker-compose that I am using.
kibana:
image: kibana
links:
- "elasticsearch"
ports:
- "9201:9201"
networks:
- cloud
This image gets run on 5601 though I am specifying 9201
I learn about changing the Port for Kibana here
If I do so every time I run the container using docker-compose up it will pull the latest image.
As a reason, I want to put this docker Image into VSTS/GIT so that I can modify the content there and use it.
I don't want just a docker image. I want the configuration of it which I can change as per my requirement.
Any help will be appreciable.
Thanks in advance.
The docker image is a base from which the container is created. When you run docker-compose up docker-compose will use the docker engine to start all the services that you have specified in the compose.yml file.
If the containers do not exist, it will create new ones and if they do exist, it will start your running containers.
The YML file (docker-compose.yml) can be uploaded to whichever repository you wish, but if you wish to upload a image, you will have to use a docker registry.
This is probably not really what you wish to do. You should rather store all your configurations in a mounted directory which is actually on the computer running the docker containers instead of in the image or container itself.
You could also create a docker volume which you store the configuration in, it will persist as long as you do not remove it.
Now, when it comes to the port issue, you are binding the containers 9201 to your 9201, if the service in the container is not running on 9201 you will not be able to reach it through that port.
Port mapping is done with host:container, so if you want to bind your computers 9201 to the containers 5601 you write:
ports:
- "9201:5601"
And it should map 5601 to 9201.
I am using docker-compose as in https://github.com/davidefiocco/dockerized-elasticsearch-indexer/blob/master/docker-compose.yml to initialize a containerized elasticsearch index.
Now, I would like to set a larger value for indices.query.bool.max_clause_count than the default setting using a elasticsearch.yml config file (this is to run some heavy queries as in Elasticsearch - set max_clause_count).
So far I tried to add in the docker-compose.yml a volume with:
services:
elasticsearch:
volumes:
- ./elasticsearch/config/elasticsearch.yml
(and variations thereof) trying to point to a elasticsearch.yml file (that I would like to ship with the rest of the files) with the right max_clause_count setting, but to no avail.
Can someone point me to the right direction?
You can mount the host's directory containing the elasticsearch.yml into the container using
services:
elasticsearch:
volumes:
- path_to/custom_elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
One workaround to perform that (trivial) modification to elasticsearch.yml in the container is to modify directly a relevant Dockerfile with the syntax
USER root
RUN echo "indices.query.bool.max_clause_count: 1000000" >> /usr/share/elasticsearch/config/elasticsearch.yml
so to append the desired custom value.
I'm working with docker for the first time.
I successfully installed elasticsearch and kibana on docker, but when I try to connect kibana with elastic I get a red status with the following errors:
ui settings Elasticsearch plugin is red
plugin:elasticsearch#5.1.1 Authentication Exception
I'm not sure but I think the problem is kibana doesn't pass elastic x-pack authentication.
Now, I'm trying to disable this authentication via elastic yml file, according to the instructions here.
But I can't find the yml file anywhere (I searched /usr/share/elasticsearch but I can't find either config directory or elasticsearch.yml file).
How do I config elastic with docker?
P.S.
I'm working with ubuntu 16.04
For Debian/Ubuntu/Mint, you can find the config files under /etc folder.
/etc/elasticsearch/elasticsearch.yml
Take a look at: https://www.elastic.co/guide/en/elasticsearch/reference/2.4/setup-dir-layout.html
I'm wondering why this is even happening. With the following docker-compose.yml it's working fine for me with security enabled:
---
version: '2'
services:
kibana:
image: docker.elastic.co/kibana/kibana:5.1.1
links:
- elasticsearch
ports:
- 5602:5601
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.1.1
cap_add:
- IPC_LOCK
volumes:
- esdata1:/usr/share/elasticsearch/data
ports:
- 9201:9200
volumes:
esdata1:
driver: local
I successfully run elastic and kibana using the official elastic docker. Somehow, the container version in the official elastic documention didn't work for me.
If you prefer to start a container using docker run and not through compose file. (only use this for dev envs, not recommended on prod envs)
docker network create elastic
docker run --network=elastic --name=elasticsearch docker.elastic.co/elasticsearch/elasticsearch:5.2.2
docker run --network=elastic -p 5601:5601 docker.elastic.co/kibana/kibana:5.2.2
A brief description can be found here:
https://discuss.elastic.co/t/kibana-docker-image-doesnt-connect-to-elasticsearch-image/79511/4