I read an instruction here
https://github.com/dahlsailrunner/northwind-core-api
and exec 2 commands everytime as it is written:
docker volume create --name northwind-pg-data -d local
docker-compose -f .\docker-compose-windows.yml up
But is it possible to add the 1st command to the docker-compose-windows.yml file and perform one command only?
There is the docker-compose file:
version: '3'
services:
db:
image: postgres:12
environment:
POSTGRES_DB: northwind
POSTGRES_USER: northwind_user
POSTGRES_PASSWORD: thewindisblowing
volumes:
- northwind-pg-data:/var/lib/postgresql/data
- ./northwind.sql:/docker-entrypoint-initdb.d/northwind.sql
ports:
- "5432:5432"
volumes:
northwind-pg-data:
external: true
Related
I am trying to run three dockerized services:
Spring-boot app
Keycloak for authentication
Postgres as database
I would like to have both the Spring-boot app and the Keycloak app to use the same Postgres container as their database, but I couldn't find a way to make it work. My docker-compose.yml is as follows:
version: '3.7'
services:
db:
image: 'postgres:13.1-alpine'
container_name: db
ports:
- "5432:5432"
volumes:
- ./app_data:/var/lib/postgresql/data_app
- ./keycloak_data:/var/lib/postgresql/data_keycloak
- ../docker-postgresql-multiple-databases:/docker-entrypoint-initdb.d
environment:
POSTGRES_MULTIPLE_DATABASES: keycloak, app_user
POSTGRES_PASSWORD: password
healthcheck:
test: [ "CMD-SHELL", "pg_isready" ]
interval: 10s
timeout: 5s
retries: 5
keycloak:
image: jboss/keycloak:14.0.0
container_name: keycloak
environment:
- KEYCLOAK_USER=admin
- KEYCLOAK_PASSWORD=admin
- DB_VENDOR=postgres
- DB_ADDR=postgres
- DB_USER=keycloak
- DB_PASSWORD=password
- JDBC_PARAMS=useSSL=false
ports:
- "8080:8080"
depends_on:
- db
healthcheck:
test: "curl -f http://localhost:8080/auth || exit 1"
start_period: 20s
app:
image: 'app.postgre:latest'
build:
context: .
container_name: app
depends_on:
- db
- keycloak
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://db:5432/app
- SPRING_DATASOURCE_USERNAME=app_user
- SPRING_DATASOURCE_PASSWORD=password
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
volumes:
app_data:
postgres_data:
(Note: I tried using the following code: https://github.com/mrts/docker-postgresql-multiple-databases to set-up the needed databases by hand, but even so it still fails. I also tried doing without this script, but that also failed.)
I have tried them and managed to make a docker-compose file which runs the spring app and the database together, and another docker-compose file which runs the keycloak app and the database together, but when I try to bring all three together it fails.
I have a very similar setup with Postgres, Keycloak, pgAdmin and a Golang API service. The skeleton of my docker-compose.yml is like this, give it a try (I omitted some parts for simplicity), it is working for me. I think the important parts here are networks and links, and also setting up multiple databases (as you already do). I use db as the hostname of Postgres server, when I connect to it via pgAdmin for example.
services:
db:
build:
context: .
dockerfile: ./Dockerfile.db
volumes:
networks:
- mynetwork
restart: unless-stopped
ports:
- ${POSTGRES_PORT}:5432
environment:
- POSTGRES_MULTIPLE_DATABASES=${POSTGRES_MULTIPLE_DATABASES}
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
healthcheck:
pgadmin:
image: dpage/pgadmin4
restart: unless-stopped
environment:
volumes:
ports:
networks:
- mynetwork
restart: unless-stopped
depends_on:
- db
api:
build:
context: .
dockerfile: ./Dockerfile.api
ports:
environment:
- POSTGRES_HOST=${POSTGRES_HOST}
- POSTGRES_PORT=${POSTGRES_PORT}
volumes:
networks:
- mynetwork
depends_on:
- db
links:
- db
restart: unless-stopped
keycloak:
image: quay.io/keycloak/keycloak:latest
environment:
- DB_VENDOR=${KEYCLOAK_DB_VENDOR}
- DB_ADDR=${KEYCLOAK_DB_ADDR}
- DB_DATABASE=${KEYCLOAK_DB_DATABASE}
- DB_USER=${KEYCLOAK_DB_USER}
- DB_SCHEMA=${KEYCLOAK_DB_SCHEMA}
- DB_PASSWORD=${KEYCLOAK_DB_PASSWORD}
- KEYCLOAK_USER=${KEYCLOAK_USER}
- KEYCLOAK_PASSWORD=${KEYCLOAK_PASSWORD}
ports:
- ${KEYCLOAK_PORT}:8080
depends_on:
- db
networks:
- mynetwork
links:
- db
restart: unless-stopped
volumes:
volumes:
networks:
mynetwork:
And some important a values from my .env:
POSTGRES_MULTIPLE_DATABASES=mydb,keycloak
POSTGRES_USER=
POSTGRES_PASSWORD=
POSTGRES_HOST=db
POSTGRES_PORT=5432
KEYCLOAK_PORT=8084
KEYCLOAK_DB_VENDOR=POSTGRES
KEYCLOAK_DB_ADDR=db
KEYCLOAK_DB_DATABASE=keycloak
KEYCLOAK_DB_USER=
KEYCLOAK_DB_SCHEMA=public
KEYCLOAK_DB_PASSWORD=
KEYCLOAK_USER=
KEYCLOAK_PASSWORD=
My Dockerfile.db is like this, you don't need the localedef part (I need it for Hungarian localization):
FROM postgres:latest
RUN localedef -i hu_HU -c -f UTF-8 -A /usr/share/locale/locale.alias hu_HU.UTF-8
COPY docker-postgresql-multiple-databases.sh /docker-entrypoint-initdb.d/
And docker-postgresql-multiple-databases.sh contains:
#!/bin/bash
set -e
set -u
function create_user_and_database() {
local database=$1
echo " Creating user and database '$database'"
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE USER $database;
CREATE DATABASE $database;
GRANT ALL PRIVILEGES ON DATABASE $database TO $database;
EOSQL
}
if [ -n "$POSTGRES_MULTIPLE_DATABASES" ]; then
echo "Multiple database creation requested: $POSTGRES_MULTIPLE_DATABASES"
for db in $(echo $POSTGRES_MULTIPLE_DATABASES | tr ',' ' '); do
create_user_and_database $db
done
echo "Multiple databases created"
fi
I'm trying to merge the docker-compose.yml file with the docker-compose2.yml file with bash.
docker-compose.yml :
version: "3"
services:
nexus:
image: sonatype/nexus3
volumes:
- "/opt/nexus3/nexus-data:/nexus-data"
ports:
- "8081:8081"
volumes:
nexus-data: {}
docker-compose2.yml :
version: "3"
services:
nexus2:
image: sonatype/nexus3
volumes:
- "/opt/nexus3/nexus-data:/nexus-data"
ports:
- "8082:8082"
volumes:
nexus-data: {}
Output I Want:
version: "3"
services:
nexus:
image: sonatype/nexus3
volumes:
- "/opt/nexus3/nexus-data:/nexus-data"
ports:
- "8081:8081"
nexus2:
image: sonatype/nexus3
volumes:
- "/opt/nexus3/nexus-data:/nexus-data"
ports:
- "8082:8082"
volumes:
nexus-data: {}
How do I get this output with bash?
The Docker Compose config command does exactly what you need, it takes multiple compose file and merges them.
Just pass them using multiple -f flags:
docker-compose -f docker-compose.yml -f docker-compose2.yml config
or using an environment variable:
COMPOSE_FILE=docker-compose.yml:docker-compose2.yml docker-compose config
The same approach is valid for every Docker Compose command, so if your final target is, for example, to set up your project, you can directly run:
docker-compose -f docker-compose.yml -f docker-compose2.yml up
Check the documentation for further details on how to specify multiple compose files.
I don't think you can do this (easily as a one-liner) in native bash without writing a script. I was curious so I did a quick search and found a yaml manipulation tool which supports merging yaml (docker-compose) files and looks like it fits your use-case.
I used brew to install on MacOS but there are instructions for Linux as well - https://mikefarah.github.io/yq/.
brew install yq
Showing existing files:
$ cat file1.yaml
version: "3"
services:
nexus:
image: sonatype/nexus3
volumes:
- "/opt/nexus3/nexus-data:/nexus-data"
ports:
- "8081:8081"
volumes:
nexus-data: {}
$ cat file2.yaml
version: "3"
services:
nexus2:
image: sonatype/nexus3
volumes:
- "/opt/nexus3/nexus-data:/nexus-data"
ports:
- "8082:8082"
volumes:
nexus-data: {}
Merge both files outputting to stdout:
$ yq m file1.yaml file2.yaml
services:
nexus:
image: sonatype/nexus3
ports:
- 8081:8081
volumes:
- /opt/nexus3/nexus-data:/nexus-data
nexus2:
image: sonatype/nexus3
ports:
- 8082:8082
volumes:
- /opt/nexus3/nexus-data:/nexus-data
version: "3"
volumes:
nexus-data: {}
There may be a native way but I just redirected the stdout to a file:
$ yq m file1.yaml file2.yaml > file3.yaml
$ cat file3.yaml
services:
nexus:
image: sonatype/nexus3
ports:
- 8081:8081
volumes:
- /opt/nexus3/nexus-data:/nexus-data
nexus2:
image: sonatype/nexus3
ports:
- 8082:8082
volumes:
- /opt/nexus3/nexus-data:/nexus-data
version: "3"
volumes:
nexus-data: {}
There are a lot of examples in their documentation for you to explore - https://mikefarah.github.io/yq/merge/.
I am running a postgres database generated by the below docker-compose file on Windows. Before running docker-compose up --build, I created a docker volume with docker volume --name postgresdata --driver local. The latter is done to avoid mounting a Windows folder into Postgres.
However, when I run docker-compose down followed by docker-compose up --build, the database is empty which I would not have expected. Any ideas or suggestions?
This is the docker-compose.yml file I am using:
version: '3.0'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
env_file:
- env_file
volumes:
- postgresdata
networks:
- db1
market_data:
build: .
environment:
PYTHONUNBUFFERED: 'true'
stdin_open: true
tty: true
links:
- db:db
container_name: market_data_container
volumes:
- '.:/market_data'
depends_on:
- db
networks:
- db1
adminer:
image: adminer
restart: always
ports:
- 8080:8080
networks:
- db1
depends_on:
- db
volumes:
market_data:
postgresdata:
external: true
networks:
db1:
driver: bridge
Postgres uses already a volume to persist data, but docker-compose down deletes this volume. You are using named volumes in your compose file, but don't mount it correctly.
version: '3.0'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
env_file:
- env_file
volumes:
- postgresdata:/var/lib/postgresql/data
networks:
- db1
Add the default path for postgres data to your volume postgresdata:/var/lib/postgresql/data. This should fix it.
Though I mount data on external volume, upon docker-compose down I am losing my data. I am using docker-compose version 1.14.0, build c7bdf9e and docker version 17.06.1-ce, build 874a737
# docker-compose.yml
version: '3'
services:
db:
image: postgres
volumes:
- data:/var/lib/postgres/data
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
volumes:
data:
external: true
I would like to accomplish 2 things:
1) Start a CockroachDB cluster with docker compose (works)
2) Execute SQL commands on the cluster (I want to create a Database)
My Docker File Looks like this:
version: '3'
services:
roach-ui:
image: cockroachdb/cockroach
command: start --insecure
expose:
- "8080"
- "26257"
ports:
- "26257:26257"
- "8080:8080"
networks:
- roachnet
db-1:
image: cockroachdb/cockroach
command: start --insecure --join=roach-ui
networks:
- roachnet
volumes:
- ./data/db-1:/cockroach/cockroach-data
networks:
roachnet:
When I run docker-compose up, everything works as expected.
While using google, I found that the solution is to run a bash script, I created the following setup.sh:
sql --insecure --execute="CREATE TABLE testDB"
I tried to run the script via command: bash -c "setup.sh", but Docker says that it can not run the command "bash".
Any Suggestions ? Thanks :)
EDIT:
I am running docker-compose up, the error I am getting:
roach-ui_1 | Failed running "bash"
heimdall_roach-ui_1 exited with code 1
So what you need is an extra init service to initialize the DB. This service will run a bash script to execute commands that will init the DB
setup_db.sh
#!/bin/bash
echo Wait for servers to be up
sleep 10
HOSTPARAMS="--host db-1 --insecure"
SQL="/cockroach/cockroach.sh sql $HOSTPARAMS"
$SQL -e "CREATE DATABASE tarun;"
$SQL -d tarun -e "CREATE TABLE articles(name VARCHAR);"
And then you add this file to execute in the docker-compose.yml
docker-compose.yaml
version: '3'
services:
roach-ui:
image: cockroachdb/cockroach
command: start --insecure
expose:
- "8080"
- "26257"
ports:
- "26257:26257"
- "8080:8080"
networks:
- roachnet
db-1:
image: cockroachdb/cockroach
command: start --insecure --join=roach-ui
networks:
- roachnet
volumes:
- ./data/db-1:/cockroach/cockroach-data
db-init:
image: cockroachdb/cockroach
networks:
- roachnet
volumes:
- ./setup_db.sh:/setup_db.sh
entrypoint: "/bin/bash"
command: /setup_db.sh
networks:
roachnet: