Is it possible to use use a hostname for a custom service. Currently, I have the following:
Redis service: docker-compose.redis.yml
version: '3.6'
services:
redis:
container_name: ddev-${DDEV_SITENAME}-redis
image: redis:latest
restart: always
ports:
- 6379
labels:
com.ddev.site-name: ${DDEV_SITENAME}
com.ddev.approot: $DDEV_APPROOT
com.ddev.app-url: $DDEV_URL
environment:
- VIRTUAL_HOST=$DDEV_HOSTNAME
- HTTP_EXPOSE=6379
volumes: []
web:
links:
- redis:$DDEV_HOSTNAME
Redis Commander Service: docker-compose.commander.yml
version: '3.6'
services:
redis:
container_name: ddev-${DDEV_SITENAME}-commander
image: rediscommander/redis-commander:latest
restart: always
ports:
- 8081
labels:
com.ddev.site-name: ${DDEV_SITENAME}
com.ddev.approot: $DDEV_APPROOT
com.ddev.app-url: $DDEV_URL
environment:
- VIRTUAL_HOST=$DDEV_HOSTNAME
- HTTP_EXPOSE=8081
- REDIS_HOSTS=local:redis:6379
volumes: []
web:
links:
- commander:$DDEV_HOSTNAME
At the moment I can access the Redis Commander from the outside with <project-name>.ddev.local:8081/.
What I want to achieve, if possible is to access the Redis Commander from a custom hostname or subdomain like: comander.<project-name>.ddev.local or commander.local.
After a bit of research and a lot of help from Randy Fay, we were able to accomplish it. We had to run the following:
$ sudo ddev hostname commander.local 127.0.0.1
The Redis Commander Service file(docker-compose.commander.yml) had to be updated to:
version: '3.6'
services:
commander:
container_name: ddev-${DDEV_SITENAME}-commander
image: rediscommander/redis-commander:latest
restart: always
ports:
- 8081
labels:
com.ddev.site-name: ${DDEV_SITENAME}
com.ddev.approot: $DDEV_APPROOT
com.ddev.app-url: $DDEV_URL
environment:
- VIRTUAL_HOST=commander.local
- HTTP_EXPOSE=80
- REDIS_HOSTS=local:redis:6379
volumes: []
web:
links:
- commander:$DDEV_HOSTNAME
- commander:commander.local
for it to work.
Related
I'm struggling to create a Docker Compose to create a Redis Cluster. I saw that there is a Redis Cluster image from Bitnami, I tried but it my Spring Boot App cannot connect to it due to the below error:
I tried another approach is to create 2 Redis instances master-slave and I can connect to it. Now I'm trying to create 6 Redis Instances and later create a Redis Cluster with 3 master and 3 slaves with the following command:
redis-cli --cluster create 127.0.0.1:6380 127.0.0.1:6381 \
127.0.0.1:6382 127.0.0.1:6383 127.0.0.1:6384 127.0.0.1:6385 --cluster-replicas 1
But when I executed the command it said that
Could not connect to Redis at 127.0.0.1:6380: Connection refused
Below is my current Docker-compose.yaml:
version: '3.8'
services:
redis-node-0:
image: redis:latest
container_name: redis-0
ports:
- "6380:6379"
command: ["redis-server","--appendonly yes","--cluster-enabled yes","--cluster-node-timeout 5000"]
volumes:
- redis-cluster_data-0:/redis/data
redis-node-1:
image: redis:latest
container_name: redis-1
ports:
- "6381:6379"
command: ["redis-server","--appendonly yes","--cluster-enabled yes","--cluster-node-timeout 5000"]
volumes:
- redis-cluster_data-1:/redis/data
redis-node-2:
image: redis:latest
container_name: redis-2
ports:
- "6382:6379"
command: ["redis-server","--appendonly yes","--cluster-enabled yes","--cluster-node-timeout 5000"]
volumes:
- redis-cluster_data-2:/redis/data
redis-node-3:
image: redis:latest
container_name: redis-3
ports:
- "6383:6379"
command: ["redis-server","--appendonly yes","--cluster-enabled yes","--cluster-node-timeout 5000"]
volumes:
- redis-cluster_data-3:/redis/data
redis-node-4:
image: redis:latest
container_name: redis-4
ports:
- "6384:6379"
command: ["redis-server","--appendonly yes","--cluster-enabled yes","--cluster-node-timeout 5000"]
volumes:
- redis-cluster_data-4:/redis/data
redis-node-5:
image: redis:latest
container_name: redis-5
ports:
- "6385:6379"
command: ["redis-server","--appendonly yes","--cluster-enabled yes","--cluster-node-timeout 5000"]
volumes:
- redis-cluster_data-5:/redis/data
networks:
default:
name: overlay
volumes:
redis-cluster_data-0:
driver: local
redis-cluster_data-1:
driver: local
redis-cluster_data-2:
driver: local
redis-cluster_data-3:
driver: local
redis-cluster_data-4:
driver: local
redis-cluster_data-5:
driver: local
I'm totally new to Both Docker and Redis, I'm learning so any help would be really appreciated. Thanks in advance.
The way to do this is not obvious, because Redis Cluster doesn't easily work with Docker bridge networking. The simplest way to setup a single-node cluster is to cheat and bind it to 127.0.0.1:
version: '3.8'
services:
redis-single-node-cluster:
image: docker.io/bitnami/redis-cluster:7.0
environment:
- 'ALLOW_EMPTY_PASSWORD=yes'
- 'REDIS_CLUSTER_REPLICAS=0'
- 'REDIS_NODES=127.0.0.1 127.0.0.1 127.0.0.1'
- 'REDIS_CLUSTER_CREATOR=yes'
- 'REDIS_CLUSTER_DYNAMIC_IPS=no'
- 'REDIS_CLUSTER_ANNOUNCE_IP=127.0.0.1'
ports:
- '6379:6379'
You can then connect to it with this Spring Boot config:
spring:
redis:
cluster:
nodes: [localhost:6379]
ssl: false
If you want to setup multiple nodes, you'll have to create a custom network and assign static IPs. You'll probably also have to set network_mode: host.
My service is using RabbitMQ. To work with it, I installed this package https://github.com/vyuldashev/laravel-queue-rabbitmq .
To work with this package, need to configure the config file
'hosts' => [
[
'host' => env('RABBITMQ_HOST', '127.0.0.1'),
'port' => env('RABBITMQ_PORT', 5672),
'user' => env('RABBITMQ_USER', 'guest'),
'password' => env('RABBITMQ_PASSWORD', 'guest'),
'vhost' => env('RABBITMQ_VHOST', '/'),
],
],
As you can see, the ip is specified as 127.0.0.1, but my docker container has a dynamic ip, so it is updated every time and I have to manually fix the config
This is my docker-compose file
version: '3'
services:
#PHP Service
app:
build:
context: .
dockerfile: Dockerfile
image: digitalocean.com/php
container_name: app
restart: unless-stopped
tty: true
environment:
SERVICE_NAME: app
SERVICE_TAGS: dev
working_dir: /var/www
volumes:
- ./:/var/www
- ./docker/php/local.ini:/usr/local/etc/php/conf.d/local.ini
networks:
- postgres
#Nginx Service
webserver:
image: nginx:alpine
container_name: webserver
restart: unless-stopped
tty: true
ports:
- "80:80"
- "443:443"
volumes:
- ./:/var/www
- ./docker/nginx/conf.d/:/etc/nginx/conf.d/
networks:
- postgres
#Redis
redis:
image: 'redis:alpine'
ports:
- "6379:6379"
#PostgreSQL
postgres:
container_name: postgres_container
image: postgres
hostname: postgres
environment:
POSTGRES_DB: postgres
POSTGRES_USER: postgres
POSTGRES_PASSWORD: secret
PGDATA: /data/postgres
volumes:
- postgres:/data/postgres
ports:
- "5432:5432"
networks:
- postgres
restart: unless-stopped
pgadmin:
container_name: pgadmin_container
image: dpage/pgadmin4
environment:
PGADMIN_DEFAULT_EMAIL: ${PGADMIN_DEFAULT_EMAIL:-pgadmin4#pgadmin.org}
PGADMIN_DEFAULT_PASSWORD: ${PGADMIN_DEFAULT_PASSWORD:-admin}
volumes:
- pgadmin:/root/.pgadmin
ports:
- "${PGADMIN_PORT:-5050}:80"
networks:
- postgres
restart: unless-stopped
depends_on:
- postgres
#RabbitMQ
rabbit:
image: "rabbitmq:3-management"
hostname: "rabbit"
environment:
RABBITMQ_ERLANG_COOKIE: "SWQOKODSQALRPCLNMEQG"
RABBITMQ_DEFAULT_USER: "rabbitmq"
RABBITMQ_DEFAULT_PASS: "rabbitmq"
RABBITMQ_DEFAULT_VHOST: "/"
ports:
- "15672:15672"
- "5672:5672"
labels:
NAME: "rabbitmq"
networks:
- postgres
#ElasticSearch
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.4.3
ports:
- "9200:9200"
- "9300:9300"
networks:
- postgres
#Docker Networks
networks:
postgres:
driver: bridge
volumes:
postgres:
pgadmin:
How can I tell the rabbitmq container to connect to the external ip of the app container?
Sorry about the confusion, I somehow mixed up Redis and RabbitMQ. So here is a probably better matching answer:
You can connect to a service on your host from the Docker container even with a changing IP - there is script and a nice answer in this SO post.
Some excerpt:
If you are on Mac or Windows you can set
RABBITMQ_HOST=host.docker.internal
In general more easy I would consider a separate container running RabbitMQ and then using rabbitmq as hostname to connect to.
services:
# ... other services
rabbitmq:
image: rabbitmq:alpine
environment:
RABBITMQ_DEFAULT_USER: "defaultuser"
RABBITMQ_DEFAULT_PASS: "defaultpass"
ports:
- 5672:5672
On my local machine (Ubuntu 18.04, 8GB RAM, i5, HDD) I have two docker-compose files with my laravel project
docker-compose.yml
version: '3.7'
networks:
backend-network:
driver: bridge
frontend-network:
driver: bridge
services:
&app-service app: &app-service-template
container_name: k4fntr_app
build:
context: ./docker/php-fpm
args:
UID: ${UID?Use your user ID}
GID: ${GID?Use your group ID}
USER: ${USER?Use your user name}
user: "${UID}:${GID}"
hostname: *app-service
volumes:
- /etc/passwd/:/etc/passwd:ro
- /etc/group/:/etc/group:ro
- ./:/var/www/k4fntr
environment:
APP_ENV: "${APP_ENV}"
CONTAINER_ROLE: app
FPM_PORT: &php-fpm-port 9000
FPM_USER: "${UID:-1000}"
FPM_GROUP: "${GID:-1000}"
networks:
- backend-network
&queue-service queue:
<<: *app-service-template
container_name: k4fntr_queue
restart: always
hostname: *queue-service
depends_on:
- app
environment:
CONTAINER_ROLE: queue
&schedule-service schedule:
<<: *app-service-template
container_name: k4fntr_schedule
restart: always
hostname: *schedule-service
depends_on:
- app
environment:
CONTAINER_ROLE: scheduler
&sportlevel-listener sportlevel_listener:
<<: *app-service-template
container_name: k4fntr_sl_listener
restart: always
hostname: *sportlevel-listener
ports:
- "${SPORTLEVEL_LISTEN_PORT}:${SPORTLEVEL_LISTEN_PORT}"
depends_on:
- app
environment:
CONTAINER_ROLE: sl_listener
&php-fpm-service php-fpm:
<<: *app-service-template
container_name: k4fntr_php-fpm
user: 'root:root'
restart: always
hostname: *php-fpm-service
ports: [*php-fpm-port]
entrypoint: /fpm-entrypoint.sh
command: php-fpm --nodaemonize
networks:
- backend-network
- frontend-network
echo-server:
container_name: k4fntr_echo
image: oanhnn/laravel-echo-server
volumes:
- ./:/app
environment:
GENERATE_CONFIG: "false"
depends_on:
- app
ports:
- "6001:6001"
networks:
- backend-network
- frontend-network
redis:
container_name: k4fntr_redis
image: redis
restart: always
command: redis-server
volumes:
- ./docker/redis/config/redis.conf:/usr/local/etc/redis/redis.conf
- ./docker/redis/redis-data:/data:rw
ports:
- "16379:6379"
networks:
- backend-network
and docker-compose-dev.yml
version: '3.7'
volumes:
redis-data:
pg-data:
k4fntr_sync:
external: true
services:
&app-service app: &app-service-template
container_name: k4fntr_app
build:
context: ./docker/php-fpm
args:
UID: ${UID?Use your user ID}
GID: ${GID?Use your group ID}
USER: ${USER?Use your user name}
user: "${UID}:${GID}"
hostname: *app-service
volumes:
- /etc/passwd/:/etc/passwd:ro
- /etc/group/:/etc/group:ro
- k4fntr_sync:/var/www/k4fntr:nocopy
environment:
APP_ENV: "${APP_ENV}"
CONTAINER_ROLE: app
FPM_PORT: &php-fpm-port 9000
FPM_USER: "${UID:-1000}"
FPM_GROUP: "${GID:-1000}"
networks:
- backend-network
&php-fpm-service php-fpm:
<<: *app-service-template
container_name: k4fntr_php-fpm
user: 'root:root'
restart: always
hostname: *php-fpm-service
ports: [*php-fpm-port]
entrypoint: /fpm-entrypoint.sh
command: php-fpm --nodaemonize -d "opcache.enable=0" -d "display_startup_errors=On" -d "display_errors=On" -d "error_reporting=E_ALL"
networks:
- backend-network
- frontend-network
mail:
container_name: k4fntr_mail
image: mailhog/mailhog
ports:
- "1025:1025"
- "8025:8025"
networks:
- backend-network
nginx:
container_name: k4fntr_nginx
image: nginx
volumes:
- ./docker/nginx/config/default:/etc/nginx/conf.d
- k4fntr_sync:/var/www/k4fntr:nocopy
depends_on:
- *php-fpm-service
ports:
- "${NGINX_LISTEN_PORT}:80"
networks:
- frontend-network
database:
container_name: k4fntr_database
build: ./docker/postgres
restart: always
environment:
ENV: ${APP_ENV}
TESTING_DB: ${DB_DATABASE_TESTING}
POSTGRES_DB: ${DB_DATABASE}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_PASSWORD: ${DB_PASSWORD}
ports:
- "15432:5432"
volumes:
- ./docker/postgres/prod/:/prod
- ./docker/postgres/pg-data:/var/lib/postgresql/data:rw
networks:
- backend-network
The problem is the fact that when I change some files in my project I have to wait a lot of time. From 15 to 40 seconds. It is impossible for local development. How can I solve this problem?
I learned some information with similar problems with other OS such as Mac or Windows, but I can't found the same problems with Linux.
The problem was that I thought that second file (docker-compose-dev.yml) overrided first file. I mean php-fpm section. If you look at docker-compose-dev you can see that there is the command
command: php-fpm --nodaemonize -d "opcache.enable=0" -d "display_startup_errors=On" -d "display_errors=On" -d "error_reporting=E_ALL"
Actually I used first file (what is very strongely, because I used the command
docker-compose -f docker-compose-dev.yml -f docker-compose.yml up
) and my opcache was cached. This was the main reason why I had to wait so long
I've got a docker-compose project in Visual Studio which starts 3 services. One of them use cosmosdb.
I've followed the instructions on https://hub.docker.com/r/microsoft/azure-cosmosdb-emulator/ to start the emulator in a docker container and it worked.
But now I want to get it up and running through docker-compose file. Following is my current configuration.
version: '3.4'
services:
gateway:
environment:
- ASPNETCORE_ENVIRONMENT=Development
image: ${DOCKER_REGISTRY-}gateway
ports:
- "7000:80"
depends_on:
- servicea
- serviceb
build:
context: .\ApiGateways\IAGTO.Fenix.ApiGateway
dockerfile: Dockerfile
servicea:
environment:
- ASPNETCORE_ENVIRONMENT=Development
image: ${DOCKER_REGISTRY-}servicea
depends_on:
- email.db
build:
context: .\Services\ServiceA
dockerfile: Dockerfile
serviceb:
environment:
- ASPNETCORE_ENVIRONMENT=Development
image: ${DOCKER_REGISTRY-}serviceb
build:
context: .\Services\ServiceB
dockerfile: Dockerfile
email.db:
image: microsoft/azure-cosmosdb-emulator
container_name: cosmosdb-emulator
ports:
- "8081:8081"
I can see the container running when I run docker container list
But requests to https://localhost:8081/_explorer/index.html fails.
Any help on this much appreciated
I was in the same situation but the container was started with the following docker-compose.yml and it became accessible.
I can browse https://localhost:8081/_explorer/index.html
version: '3.7'
services:
cosmosdb:
container_name: cosmosdb
image: microsoft/azure-cosmosdb-emulator
tty: true
restart: always
ports:
- "8081:8081"
- "8900:8900"
- "8901:8901"
- "8979:8979"
- "10250:10250"
- "10251:10251"
- "10252:10252"
- "10253:10253"
- "10254:10254"
- "10255:10255"
- "10256:10256"
- "10350:10350"
volumes:
- vol_cosmos:C:\CosmosDB.Emulator\bind-mount
volumes:
vol_cosmos:
Probably I needed to set "tty" or "volumes".
Using the linux cosmos db image, I set it up like this:
version: '3.4'
services:
db:
container_name: cosmosdb
image: "mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator"
tty: true
restart: always
mem_limit: 2G
cpu_count: 2
environment:
- AZURE_COSMOS_EMULATOR_PARTITION_COUNT=10
- AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true
ports:
- "8081:8081"
- "8900:8900"
- "8901:8901"
- "8979:8979"
- "10250:10250"
- "10251:10251"
- "10252:10252"
- "10253:10253"
- "10254:10254"
- "10255:10255"
- "10256:10256"
- "10350:10350"
volumes:
- vol_cosmos:/data/db
volumes:
vol_cosmos:
Part of the problem is that the emulator takes a while to start, and there is a timeout of 2 minutes before it's just stops waiting.
I'm trying to hack my way through it, but I haven't had much success.
For now the image only works stand alone (via docker run) and that's it.
I'm using an application hosted on a docker container.
This application executes bash scripts / instructions to send mails.
I made another container which executes Postfix as a SMTP Relay.
I want to send mails from my application container by using a bash script using my Postfix container as a relay.
I tried to connect with SSH from my application container to my Postfix container. But that doesn't seem to work.
How can i make it so a script executed in my application container can use my Postfix relay while not allowing anything outside of the docker network, or even better, to only allow some containers, to send mails from this relay.
EDIT 1 : Docker-compose files
Application docker compose :
version: "3.4"
volumes:
[...]
services:
application:
restart: always
build: ./application
depends_on:
- mariadb
container_name: application
volumes:
[...]
ports:
- "80:80"
- "443:443"
- "5669:5669"
deploy:
restart_policy:
window: 300s
links:
- mariadb
external_links:
- smtp-server
mariadb:
restart: always
image: mariadb
command: mysqld --sql-mode=ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
container_name: application-mariadb
volumes:
[...]
environment:
MYSQL_RANDOM_ROOT_PASSWORD: "yes"
deploy:
restart_policy:
window: 300s
Here's my docker compose for my SMTP server :
version: "3.4"
services:
postfix:
restart: always
build: ./postfix
container_name: smtp-server
deploy:
restart_policy:
window: 300s
{a quick response, because I "cicle" in my work ... and I'm taking 10 minutes of clear my mint, I hope it serves you}
Are you using "docker-compose" ?, could you give an example of your YML file? (a little more context)
[you can not connect to by ssh to a container unless you have "supervisor" installed,which I do not recommend at all.]
from what I see, you only need to make private networks; You could use this:
https://docs.docker.com/compose/networking/
to hide everything, I also recommend using a load balancer / Inverse Proxy like TRAEFIK (if they have access to port 80 or 443 in some clear way this ...)
so you only expose 1/2 port(s) (80 + 443 for example) and everything else is protected by your reverse proxy
Watch as I separate the networks as you need the different containers.
bash have access to db and smtp
db does not have access smtp neither nginx
nginx have access to bash
nginx have access to proxy network to expose 80 and 443
no other container is exposed to the outside more than nginx
--
version: "3"
services:
bash:
####### use hostname "smtp" as SMTP server
image: bash
depends_on:
- db
networks:
- smtp_internal_network
- internal_network
- data_network
volumes:
- ../html:/var/www/html
restart: always
db:
image: percona:5.7
# ports: # for debug connections and querys
# - 3306:3306
volumes:
- ../db_data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
networks:
- data_network
restart: always
smtp:
image: mwader/postfix-relay
environment:
- POSTFIX_myhostname=smtp.domain.tld
networks:
- smtp_internal_network
restart: always
nginx:
image: nginx
volumes:
- ../html:/var/www/html
networks:
- external_network
- internal_network
labels:
- "traefik.backend=nginx_${COMPOSE_PROJECT_NAME}"
- "traefik.port=80"
- "traefik.frontend.rule=Host:${FRONTEND_RULE}"
- "traefik.frontend.passHostHeader=true"
- "traefik.enable=true"
- "traefik.docker.network=traefik_proxy"
restart: always
depends_on:
- db
- bash
networks:
external_network:
external:
name: traefik_proxy
internal_network:
driver: bridge
smtp_internal_network:
driver: bridge
data_network:
driver: bridge
Edit:
version: "3"
volumes:
[...]
services:
####### use hostname "smtp" as SMTP server in your application
application:
restart: always
build: ./application
depends_on:
- mariadb
volumes:
[...]
ports:
- "80:80"
- "443:443"
- "5669:5669"
deploy:
restart_policy:
window: 300s
networks:
- smtp_external_network
- data_network
mariadb:
restart: always
image: mariadb
command: mysqld --sql-mode=ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
networks:
- data_network
volumes:
[...]
environment:
MYSQL_RANDOM_ROOT_PASSWORD: "yes"
deploy:
restart_policy:
window: 300s
networks:
smtp_external_network:
external:
name: [ReplaceForFolderParentNameOfSmtpYmlWithoutSquareBrackets]_smtp
data_network:
driver: bridge
--- (in your other file)
services:
smtp:
restart: always
build: ./postfix
networks:
- smtp
deploy:
restart_policy:
window: 300s
networks:
smpt:
driver: bridge