I have a docker-compsose.yml file that launch a postgis service with a shared folder of kml files. I also ave a script that can export all of those kml in my postgis database. However I would like to do so automatically after launch. How can the docker-compose read that file and run the shell command after launch ?
Thank you for the help, I am new using Docker.
version: '2'
services:
postgis:
image: mdillon/postgis
volumes:
- ~/test/dataPostgis:/var/lib/postgresql/data/pgdata
- ./postgresql:/docker-entrypoint-initdb.d
- ./KML_Data:/var/lib/postgresql/data/KML_Data
environment:
PGDATA: /var/lib/postgresql/data/pgdata
POSTGRES_PASSWORD: password
POSTGRES_DB: db
ports:
- 5432:5432
pgadmin:
image: chorss/docker-pgadmin4
ports:
- 5050:5050
volumes:
- ~/test/dataPgadminBackUp:/var/lib/postgresql/data/pgdata
- ./scripts/pgadmin:/tmp/scripts
links:
- postgis
depends_on:
- postgis
Related
I have a problem about running docker-compose.yml in my Spring Boot app.
When I run this command (docker-compose up -d), I got an issue in image part.
I tried to handle with solving this issue but I couldn't do that.
How can I fix it?
Here is my issue shown below.
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
Here is my project : My Project
Here is my docker-compose.yml shown below.
version: '3.8'
services:
logstash:
image: docker.elastic.co/logstash/logstash:7.15.2
user: root
command: -f /etc/logstash/conf.d/
volumes:
- ./logstash/:/etc/logstash/conf.d/
ports:
- "5000:5000"
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
depends_on:
- elasticsearch
kibana:
image: docker.elastic.co/kibana/kibana:7.15.2
user: root
volumes:
- ./kibana/:/usr/share/kibana/config/
ports:
- "5601:5601"
depends_on:
- elasticsearch
entrypoint: ["./bin/kibana", "--allow-root"]
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.15.2
user: root
volumes:
- ./elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- "9200:9200"
- "9300:9300"
environment:
ES_JAVA_OPTS: "-Xmx256m -Xms256m"
app:
image: 'springbootelk:latest'
build:
context: .
container_name: SpringBootElk
depends_on:
- db
- logstash
ports:
- '8077:8077'
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://db:3306/springbootexample?useSSL=false&serverTimezone=Turkey
- SPRING_DATASOURCE_USERNAME=springexample
- SPRING_DATASOURCE_PASSWORD=111111
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
db:
container_name: mysql-latest
image: 'mysql:latest'
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
volumes:
- db-data:/var/lib/mysql
# Volumes
volumes:
db-data:
There are several issues, why it doesn't work.
First thing - your tests aren't set up properly. Application tries to connect to database in the test stage. But during the test stage there is no any running containers yet. You can set it up properly, or remove MainApplicationTests class from the test directory, or by switching off your tests execution, just by adding -Dmaven.test.skip in Dockerfile for the mvnw package command. After it your image will build properly.
Second thing, you need to allow public key retrieval for your application. To do that, you can add allowPublicKeyRetrieval=true to your jdbc url. You can read more about it here: Connection Java - MySQL : Public Key Retrieval is not allowed.
These steps will allow your application to start (at least it will resolve database connectivity problems). But, I have found another issue. You set in your application configuration context-path equal to /api. Also you added #RequestMapping("/api") for your PersonController. And to access the list of persons, you will need to use the following url: http://localhost:8077/api/api/persons. Probably, it is not what you wanted. To fix it, you can remove it from any place.
I want to create mysql database on docker-compose startup from database.sql script. My database.sql script is on location: src/main/java/com/project_name/resources/db/database.sql. How should I wrote that in my docker-compose.yml file? Right now neither works.
volumes:
- ./database.sql:/data/application/database.sql
or something like:
volumes:
- ./database.sql:/src/main/java/com/project_name/resources/db/database.sql
Try like this:
volumes:
- ./src/main/java/com/project_name/resources/db/:/docker-entrypoint-initdb.d/database.sql
Or just use a database migration tool like Flyway or Liquibase.
You can mount the schema and data in Volumes as I demonstrated below, make sure the backup file have proper access permissions and verify the path in your machine.
version: '3.8'
services:
db:
image: mysql:8.0
restart: always
environment:
- MYSQL_DATABASE=DB_NAME
- MYSQL_USER: DB_USER
- MYSQL_ROOT_PASSWORD=DB_PASSEOD
ports:
- '3306:3306'
volumes:
- db:/var/lib/mysql
- ./db/init.sql:/src/main/java/com/project_name/resources/db/database.sql
volumes:
db:
driver: local
I am trying to run three dockerized services:
Spring-boot app
Keycloak for authentication
Postgres as database
I would like to have both the Spring-boot app and the Keycloak app to use the same Postgres container as their database, but I couldn't find a way to make it work. My docker-compose.yml is as follows:
version: '3.7'
services:
db:
image: 'postgres:13.1-alpine'
container_name: db
ports:
- "5432:5432"
volumes:
- ./app_data:/var/lib/postgresql/data_app
- ./keycloak_data:/var/lib/postgresql/data_keycloak
- ../docker-postgresql-multiple-databases:/docker-entrypoint-initdb.d
environment:
POSTGRES_MULTIPLE_DATABASES: keycloak, app_user
POSTGRES_PASSWORD: password
healthcheck:
test: [ "CMD-SHELL", "pg_isready" ]
interval: 10s
timeout: 5s
retries: 5
keycloak:
image: jboss/keycloak:14.0.0
container_name: keycloak
environment:
- KEYCLOAK_USER=admin
- KEYCLOAK_PASSWORD=admin
- DB_VENDOR=postgres
- DB_ADDR=postgres
- DB_USER=keycloak
- DB_PASSWORD=password
- JDBC_PARAMS=useSSL=false
ports:
- "8080:8080"
depends_on:
- db
healthcheck:
test: "curl -f http://localhost:8080/auth || exit 1"
start_period: 20s
app:
image: 'app.postgre:latest'
build:
context: .
container_name: app
depends_on:
- db
- keycloak
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://db:5432/app
- SPRING_DATASOURCE_USERNAME=app_user
- SPRING_DATASOURCE_PASSWORD=password
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
volumes:
app_data:
postgres_data:
(Note: I tried using the following code: https://github.com/mrts/docker-postgresql-multiple-databases to set-up the needed databases by hand, but even so it still fails. I also tried doing without this script, but that also failed.)
I have tried them and managed to make a docker-compose file which runs the spring app and the database together, and another docker-compose file which runs the keycloak app and the database together, but when I try to bring all three together it fails.
I have a very similar setup with Postgres, Keycloak, pgAdmin and a Golang API service. The skeleton of my docker-compose.yml is like this, give it a try (I omitted some parts for simplicity), it is working for me. I think the important parts here are networks and links, and also setting up multiple databases (as you already do). I use db as the hostname of Postgres server, when I connect to it via pgAdmin for example.
services:
db:
build:
context: .
dockerfile: ./Dockerfile.db
volumes:
networks:
- mynetwork
restart: unless-stopped
ports:
- ${POSTGRES_PORT}:5432
environment:
- POSTGRES_MULTIPLE_DATABASES=${POSTGRES_MULTIPLE_DATABASES}
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
healthcheck:
pgadmin:
image: dpage/pgadmin4
restart: unless-stopped
environment:
volumes:
ports:
networks:
- mynetwork
restart: unless-stopped
depends_on:
- db
api:
build:
context: .
dockerfile: ./Dockerfile.api
ports:
environment:
- POSTGRES_HOST=${POSTGRES_HOST}
- POSTGRES_PORT=${POSTGRES_PORT}
volumes:
networks:
- mynetwork
depends_on:
- db
links:
- db
restart: unless-stopped
keycloak:
image: quay.io/keycloak/keycloak:latest
environment:
- DB_VENDOR=${KEYCLOAK_DB_VENDOR}
- DB_ADDR=${KEYCLOAK_DB_ADDR}
- DB_DATABASE=${KEYCLOAK_DB_DATABASE}
- DB_USER=${KEYCLOAK_DB_USER}
- DB_SCHEMA=${KEYCLOAK_DB_SCHEMA}
- DB_PASSWORD=${KEYCLOAK_DB_PASSWORD}
- KEYCLOAK_USER=${KEYCLOAK_USER}
- KEYCLOAK_PASSWORD=${KEYCLOAK_PASSWORD}
ports:
- ${KEYCLOAK_PORT}:8080
depends_on:
- db
networks:
- mynetwork
links:
- db
restart: unless-stopped
volumes:
volumes:
networks:
mynetwork:
And some important a values from my .env:
POSTGRES_MULTIPLE_DATABASES=mydb,keycloak
POSTGRES_USER=
POSTGRES_PASSWORD=
POSTGRES_HOST=db
POSTGRES_PORT=5432
KEYCLOAK_PORT=8084
KEYCLOAK_DB_VENDOR=POSTGRES
KEYCLOAK_DB_ADDR=db
KEYCLOAK_DB_DATABASE=keycloak
KEYCLOAK_DB_USER=
KEYCLOAK_DB_SCHEMA=public
KEYCLOAK_DB_PASSWORD=
KEYCLOAK_USER=
KEYCLOAK_PASSWORD=
My Dockerfile.db is like this, you don't need the localedef part (I need it for Hungarian localization):
FROM postgres:latest
RUN localedef -i hu_HU -c -f UTF-8 -A /usr/share/locale/locale.alias hu_HU.UTF-8
COPY docker-postgresql-multiple-databases.sh /docker-entrypoint-initdb.d/
And docker-postgresql-multiple-databases.sh contains:
#!/bin/bash
set -e
set -u
function create_user_and_database() {
local database=$1
echo " Creating user and database '$database'"
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE USER $database;
CREATE DATABASE $database;
GRANT ALL PRIVILEGES ON DATABASE $database TO $database;
EOSQL
}
if [ -n "$POSTGRES_MULTIPLE_DATABASES" ]; then
echo "Multiple database creation requested: $POSTGRES_MULTIPLE_DATABASES"
for db in $(echo $POSTGRES_MULTIPLE_DATABASES | tr ',' ' '); do
create_user_and_database $db
done
echo "Multiple databases created"
fi
There is a service that uses mongodb. But when I restart computer or docker machine, no data is stored in the database.
docker-compose.yml:
version: "3"
Services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/dockerdata/db
volumes:
- ./dockerdata/db:/data/db
ports:
- 27017:27017
command: mongod
I tried to do database storage on the host, but it didn't help either:
docker-compose.yml:
version: "3"
Services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/c/users/frol/mongodata/db
volumes:
- /c/users/frol/mongodata/db:/data/db
ports:
- 27017:27017
command: mongod
If you make a named volume, docker writes an error:
ERROR: for test_mongodb_1 Cannot create container for service mongodb: fa
To mount local volume: mount /c/users/frol/mongodata/db:/mnt/sda1/var/lib/d
ocker/volumes/test_mongodata/_data, flags: 0x1000: no such file or directory
docker-compose.yml:
version: "3"
services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/c/users/frol/mongodata/db
volumes:
- mongodata:/data/db
ports:
- 27017:27017
command: mongod
volumes:
mongodata:
driver: local
driver_opts:
type: none
device: /c/users/frol/mongodata/db
o: bind
Host - win 8.1, docker toolbox 19.03.1 installed.
Help me, please, I'm a novice. How do I make sure that the database data isn't lost?
You first attempt would work if you just fix a simple typo in your compose file:
version: "3"
services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/data/db # changed
volumes:
- ./dockerdata/db:/data/db
ports:
- 27017:27017
command: mongod
But, since /data/db is the default value of MONGO_DATA_DIR, setting it is pretty redundant.
But I'd prefer to use a named volume, that way the data persists but I don't have to see the "ugly" database storage folder:
version: "3"
services:
...
mongodb:
restart: always
image: mongo:latest
volumes:
- mongodata:/data/db
ports:
- 27017:27017
command: mongod
volumes:
mongodata:
Don't set $MONGO_DATA_DIR; leave it at its default of /data/db.
services:
mongodb:
restart: always
image: mongo:latest
# No need to specifically set $MONGO_DATA_DIR
volumes:
- ./dockerdata/db:/data/db
ports:
- 27017:27017
# No need to override command:
Docker containers have a separate filesystem space from the host filesystem. A typical setup for most databases is to have the database storage in a fixed location inside the container; for MongoDB that's the /data/db directory. You can mount a named volume or filesystem path there, but the code inside the container doesn't know the difference.
If you do set environment variables like $MONGO_DATA_DIR, they need to reflect paths inside the container; they can't directly specify host-system paths. (#ruohola's answer works because it changes the container-filesystem path of the bind mount to match the container-filesystem path in the environment variable; the host ./dockerdata and container /dockerdata paths are totally unrelated.)
As you are defining the data dir explicitly, you need to map the same directory in the volume to persist the data
version: "3"
services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/data/db #data directory
volumes:
- ./dockerdata/db:/data/db #same data directory which you defined above
ports:
- 27017:27017
command: mongod
I am running a postgres database generated by the below docker-compose file on Windows. Before running docker-compose up --build, I created a docker volume with docker volume --name postgresdata --driver local. The latter is done to avoid mounting a Windows folder into Postgres.
However, when I run docker-compose down followed by docker-compose up --build, the database is empty which I would not have expected. Any ideas or suggestions?
This is the docker-compose.yml file I am using:
version: '3.0'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
env_file:
- env_file
volumes:
- postgresdata
networks:
- db1
market_data:
build: .
environment:
PYTHONUNBUFFERED: 'true'
stdin_open: true
tty: true
links:
- db:db
container_name: market_data_container
volumes:
- '.:/market_data'
depends_on:
- db
networks:
- db1
adminer:
image: adminer
restart: always
ports:
- 8080:8080
networks:
- db1
depends_on:
- db
volumes:
market_data:
postgresdata:
external: true
networks:
db1:
driver: bridge
Postgres uses already a volume to persist data, but docker-compose down deletes this volume. You are using named volumes in your compose file, but don't mount it correctly.
version: '3.0'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
env_file:
- env_file
volumes:
- postgresdata:/var/lib/postgresql/data
networks:
- db1
Add the default path for postgres data to your volume postgresdata:/var/lib/postgresql/data. This should fix it.