Data isn't persisted in the database when using MongoDB with Docker volumes? - windows

There is a service that uses mongodb. But when I restart computer or docker machine, no data is stored in the database.
docker-compose.yml:
version: "3"
Services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/dockerdata/db
volumes:
- ./dockerdata/db:/data/db
ports:
- 27017:27017
command: mongod
I tried to do database storage on the host, but it didn't help either:
docker-compose.yml:
version: "3"
Services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/c/users/frol/mongodata/db
volumes:
- /c/users/frol/mongodata/db:/data/db
ports:
- 27017:27017
command: mongod
If you make a named volume, docker writes an error:
ERROR: for test_mongodb_1 Cannot create container for service mongodb: fa
To mount local volume: mount /c/users/frol/mongodata/db:/mnt/sda1/var/lib/d
ocker/volumes/test_mongodata/_data, flags: 0x1000: no such file or directory
docker-compose.yml:
version: "3"
services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/c/users/frol/mongodata/db
volumes:
- mongodata:/data/db
ports:
- 27017:27017
command: mongod
volumes:
mongodata:
driver: local
driver_opts:
type: none
device: /c/users/frol/mongodata/db
o: bind
Host - win 8.1, docker toolbox 19.03.1 installed.
Help me, please, I'm a novice. How do I make sure that the database data isn't lost?

You first attempt would work if you just fix a simple typo in your compose file:
version: "3"
services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/data/db # changed
volumes:
- ./dockerdata/db:/data/db
ports:
- 27017:27017
command: mongod
But, since /data/db is the default value of MONGO_DATA_DIR, setting it is pretty redundant.
But I'd prefer to use a named volume, that way the data persists but I don't have to see the "ugly" database storage folder:
version: "3"
services:
...
mongodb:
restart: always
image: mongo:latest
volumes:
- mongodata:/data/db
ports:
- 27017:27017
command: mongod
volumes:
mongodata:

Don't set $MONGO_DATA_DIR; leave it at its default of /data/db.
services:
mongodb:
restart: always
image: mongo:latest
# No need to specifically set $MONGO_DATA_DIR
volumes:
- ./dockerdata/db:/data/db
ports:
- 27017:27017
# No need to override command:
Docker containers have a separate filesystem space from the host filesystem. A typical setup for most databases is to have the database storage in a fixed location inside the container; for MongoDB that's the /data/db directory. You can mount a named volume or filesystem path there, but the code inside the container doesn't know the difference.
If you do set environment variables like $MONGO_DATA_DIR, they need to reflect paths inside the container; they can't directly specify host-system paths. (#ruohola's answer works because it changes the container-filesystem path of the bind mount to match the container-filesystem path in the environment variable; the host ./dockerdata and container /dockerdata paths are totally unrelated.)

As you are defining the data dir explicitly, you need to map the same directory in the volume to persist the data
version: "3"
services:
...
mongodb:
restart: always
image: mongo:latest
environment:
- MONGO_DATA_DIR=/data/db #data directory
volumes:
- ./dockerdata/db:/data/db #same data directory which you defined above
ports:
- 27017:27017
command: mongod

Related

How to get hostname from one container to another using docker compose?

I have two docker containers. One backend, and the other db (postgres). They both are linked. How do I utilise the backend environment HOST variable in the golang docker container?
From my understanding, both containers have their own IP addresses. I cannot use 'localhost' in the golang container because postgres isn't on localhost, but in an isolated container.
version: "3.7"
services:
backend:
image: golang:1.16
build: ./
working_dir: /app
volumes:
- ./backend/:/app
environment:
HOST: db
command: go run main.go
ports:
- 8080:8080
depends_on:
- db
db:
image: postgres
restart: always
environment:
POSTGRES_USER: gorm
POSTGRES_PASSWORD: gorm
POSTGRES_DB: gorm
ports:
- 9920:9920
I've tried researching how to access this variable as well as check Docker tutorials/documentation, but haven't found a solution.
Docker compose does DNS resolution. You should be able to access your database by name.
Remove:
environment:
HOST: db
Correct the postgres port to 5432:
db:
...
ports:
- 5432:5432
You must be able to connect like so:
db := pg.Connect(&pg.Options{
Addr: "db:5432",
User: "gorm",
Database: "gorm",
Password: "gorm",
})
As for environment variables, you can declare and access them like this:
backend:
environment:
POSTGRES_USER: gorm
...
os.Getenv("POSTGRES_USER")
Docker compose will create a network for your containers where they can communicate and reach each other.
You can make a simple change to your docker compose file by adding a name to your services which will make sure they get the same name each time.
version: "3.7"
services:
backend:
image: golang:1.16
build: ./
container_name: backend
working_dir: /app
volumes:
- ./backend/:/app
command: go run main.go
ports:
- "8080:8080"
depends_on:
- db
db:
image: postgres
restart: always
container_name: db
environment:
POSTGRES_USER: gorm
POSTGRES_PASSWORD: gorm
POSTGRES_DB: gorm
ports:
- "9920:5432"
The default port for Postgres is 5432, so I mapped it to your 9920. Then, you can access the db container from the backend container by specifying:
db:9920

Docker shared volume not working in MacOs

I have a docker-compose.yml file. It works fine in Windows 10 but whenever I try to run that in MacOs it doesnt work especially the shared volume.
Here is the content of my docker-compose.yml file and directory structure
version: '3'
services:
database:
image: mongo
container_name: pcore-database
ports:
- '27017:27017'
node-server:
image: node
container_name: pcore-node-server
volumes:
- ./node-services :/usr/app/node-services
working_dir: /usr/app/node-services
command: npm run dev
ports:
- '3000:3000'
links:
- database
- nginx-server
depends_on:
- database
apache-server:
image: webdevops/php-apache
container_name: pcore-apache-server
working_dir: /app
volumes:
- ./php-services :/app
ports:
- '8000:80'
Check the node-server service and nginx-server
Now when i run command docker-compose up it creates additional directories with same name and throws error.
Check the error and additional directories it created.
I dont know whats going on. Its working fine in windows 10 but in MacOs it creates additional directories and does not share the volumes. Can someone guid me?

Running Sonarqube with docker-compose using bind mount volumes

I’m trying to run Sonarqube in a Docker container on a Centos 7 server using docker-compose. Everything works as expected using named volumes as configured in this docker-compose.yml file:
version: "3"
services:
sonarqube:
image: sonarqube
ports:
- "9000:9000"
networks:
- sonarnet
environment:
- sonar.jdbc.url=jdbc:postgresql://db:5432/sonar
volumes:
- sonarqube_conf:/opt/sonarqube/conf
- sonarqube_data:/opt/sonarqube/data
- sonarqube_extensions:/opt/sonarqube/extensions
- sonarqube_bundled_plugins:/opt/sonarqube/lib/bundled-plugins
db:
image: postgres
networks:
- sonarnet
environment:
- POSTGRES_USER=sonar
- POSTGRES_PASSWORD=sonar
volumes:
- postgresql:/var/lib/postgresql
- postgresql_data:/var/lib/postgresql/data
networks:
sonarnet:
driver: bridge
volumes:
sonarqube_conf:
sonarqube_data:
sonarqube_extensions:
sonarqube_bundled_plugins:
postgresql:
postgresql_data:
However, my /var/lib/docker/volumes directory is not large enough to house the named volumes. So, I changed the docker-compose.yml file to use bind mount volumes as shown below.
version: "3"
services:
sonarqube:
image: sonarqube
ports:
- "9000:9000"
networks:
- sonarnet
environment:
- sonar.jdbc.url=jdbc:postgresql://db:5432/sonar
volumes:
- /data/sonarqube/conf:/opt/sonarqube/conf
- /data/sonarqube/data:/opt/sonarqube/data
- /data/sonarqube/extensions:/opt/sonarqube/extensions
- /data/sonarqube/bundled_plugins:/opt/sonarqube/lib/bundled-plugins
db:
image: postgres
networks:
- sonarnet
environment:
- POSTGRES_USER=sonar
- POSTGRES_PASSWORD=sonar
volumes:
- /data/postgresql:/var/lib/postgresql
- /data/postgresql_data:/var/lib/postgresql/data
networks:
sonarnet:
driver: bridge
However, after running docker-compose up -d, the app starts up but none of the bind mount volumes are written to. As a result, the Sonarqube plugins are not loaded and the sonar postgreSQL database is not initialized. I thought it may be a selinux issue, but I temporarily disabled it with no success. I’m unsure what to look at next.
I think my answer from "How to persist configuration & analytics across container invocations in Sonarqube docker image" would help you as well.
For good measure I have also pasted it in here:
.....
Notice this line SONARQUBE_HOME in the Dockerfile for the docker-sonarqube image. We can control this environment variable.
When using docker run. Simply do:
txt
docker run -d \
...
...
-e SONARQUBE_HOME=/sonarqube-data
-v /PERSISTENT_DISK/sonarqubeVolume:/sonarqube-data
This will make Sonarqube create the conf, data and so forth folders and store data therein. As needed.
Or with Kubernetes. In your deployment YAML file. Do:
txt
...
...
env:
- name: SONARQUBE_HOME
value: /sonarqube-data
...
...
volumeMounts:
- name: app-volume
mountPath: /sonarqube-data
And the name in the volumeMounts property points to a volume in the volumes section of the Kubernetes deployment YAML file.
This again will make Sonarqube use the /sonarqube-data mountPath for creating extenions, conf and so forth folders, then save data therein.
And voila your Sonarqube data is thereby persisted.
I hope this will help others.
N.B. Notice that the YAML and Docker run examples are not exhaustive. They focus on the issue of persisting Sonarqube data.
Try it out BobC and let me know.
Have a great day.
The below code will help you in a single command I hope so.
Create a new docker-compose file named as docker-compose.yaml,
version: "3"
services:
sonarqube:
image: sonarqube:8.2-community
depends_on:
- db
ports:
- "9000:9000"
networks:
- sonarqubenet
environment:
SONAR_JDBC_URL: jdbc:postgresql://db:5432/sonarqube
SONAR_JDBC_USERNAME: sonar
SONAR_JDBC_PASSWORD: sonar
volumes:
- sonarqube_data:/opt/sonarqube/data
- sonarqube_extensions:/opt/sonarqube/extensions
- sonarqube_logs:/opt/sonarqube/logs
- sonarqube_temp:/opt/sonarqube/temp
restart: on-failure
container_name: sonarqube
db:
image: postgres
networks:
- sonarqubenet
environment:
POSTGRES_USER: sonar
POSTGRES_PASSWORD: sonar
volumes:
- postgresql:/var/lib/postgresql
- postgresql_data:/var/lib/postgresql/data
restart: on-failure
container_name: postgresql
networks:
sonarqubenet:
driver: bridge
volumes:
sonarqube_data:
sonarqube_extensions:
sonarqube_logs:
sonarqube_temp:
postgresql:
postgresql_data:
Then, execute the command,
$ docker-compose up -d
$ docker container ps
Sounds like the container is running and, as you mentioned, Sonarqube starts-up. When it starts, is it showing that it's using the H2 in memory db? After running docker-compose up -d, use docker logs -f <container_name> to see what's happening on Sonarqube startup.
To simplify viewing your logs with a known name, I suggest you also add a container name to your Sonarqube service. For example, container_name: sonarqube.
Also, while I know the plan is to deprecate the use of environment variables for the username, password and jdbc connection, I've had better luck in docker-compose using environment variables rather than the corresponding property value. For the connection string, try: SONARQUBE_JDBC_URL: jdbc:postgresql://db/sonar without specifying the default port for postgres.

Docker postgres container loses data that should be stored in volume

I am running a postgres database generated by the below docker-compose file on Windows. Before running docker-compose up --build, I created a docker volume with docker volume --name postgresdata --driver local. The latter is done to avoid mounting a Windows folder into Postgres.
However, when I run docker-compose down followed by docker-compose up --build, the database is empty which I would not have expected. Any ideas or suggestions?
This is the docker-compose.yml file I am using:
version: '3.0'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
env_file:
- env_file
volumes:
- postgresdata
networks:
- db1
market_data:
build: .
environment:
PYTHONUNBUFFERED: 'true'
stdin_open: true
tty: true
links:
- db:db
container_name: market_data_container
volumes:
- '.:/market_data'
depends_on:
- db
networks:
- db1
adminer:
image: adminer
restart: always
ports:
- 8080:8080
networks:
- db1
depends_on:
- db
volumes:
market_data:
postgresdata:
external: true
networks:
db1:
driver: bridge
Postgres uses already a volume to persist data, but docker-compose down deletes this volume. You are using named volumes in your compose file, but don't mount it correctly.
version: '3.0'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
env_file:
- env_file
volumes:
- postgresdata:/var/lib/postgresql/data
networks:
- db1
Add the default path for postgres data to your volume postgresdata:/var/lib/postgresql/data. This should fix it.

How docker volume container (mounted a host directory ) used in compose file version 2

In compose file version 1 , my docker-compose.yml is :
mongo:
image: mongo
volumes_from:
- mongodata
mongodata:
image: mongo
volumes:
- /home/dbdata/mongodb:/data/db
In localhost "/home/dbdata/mongodb" ,i saved some data . And it can be read through mongo container .
But i don't know how to update to compose file version 2. I have updated docker and docker-compose to the latest.
Docker version 1.10.2, build c3959b1
docker-compose version 1.6.2, build 4d72027
OS: ubuntu 15.10
I tried three ways ,but not working.
one : the compose file is :
version: '2'
services:
mongo:
image: mongo
volumes_from:
- mongodata
networks:
- wfij
mongodata:
image: mongo
volumes:
- /home/dbdata/mongodb:/data/db
# this is a spring cloud service , it can read mongo user data
useraccount:
image: user-account
networks:
- wfij
networks:
wfij:
driver: bridge
But the spring user-account service can't read the mongo data .
Another method: the compose file
-
version: '2'
services:
mongo:
image: mongo
volumes:
- mongo-data:/data/db
networks:
- wfij
useraccount:
image: user-account
networks:
- wfij
networks:
wfij:
driver: bridge
volumes:
mongo-data:
volumes: home/dbdata/mongodb
It not works again , the error is : "volumes is not supported" , but if i not set the volume container ,how can i mount the host directory "home/dbdata/mongodb" to the volume .
last , try this :
mongo:
image: mongo
volumes:
- mongo-data
voluems:
- mongo-data: /home/keryhu/dbdata/mongodb:/data/db
The error is :
ERROR: In file './docker-compose.yml', volume 'mongo-data' must be a mapping not a string.
Can help me ?
version: '2'
services:
mongo:
image: mongo
volumes_from:
- mongodata
mongodata:
image: mongo
volumes:
- /home/dbdata/mongodb:/data/db
This should work. You don't have to make any changes for host volumes. It's only important for named volumes.

Resources