I'm trying to mount a directory with configuration files in my docker-compose.yml.
In my case it is logstash, which tells me the mounted directory is empty.
Loading a bash and ls -la in the parent directory shows that the pipeline directory is empty and is owned by root.
One weird thing is, that it worked a few days ago.
docker-compose.yml:
version: '3'
services:
logstash:
image: docker.elastic.co/logstash/logstash:5.6.3
ports:
- 5000:5000
- 8989:8989
volumes:
- C:/PROJECT_DIR/config/logstash/pipeline/:/usr/share/logstash/pipeline/
I found it better to try around with docker itself, as it gives more feedback
docker run --rm -it -v C:/PROJECT_DIR/config/logstash/pipeline/:/usr/share/logstash/pipeline/ docker.elastic.co/logstash/logstash:5.6.3
From here and some googling I found out I had to reset my shared drives credentials under "Docker for Windows" -> Settings... -> Shared Drives, because I had changed my windows domain user password.
If you changed your system username or password then you need to re-apply the credentials to get the volume mount working.
Related
I have spring boot application which I am trying to dockerize for the first time. I am using docker version 20.10.1 and my host pc is ubuntu 20.04
for this spring boot application, I have a data directory , which has data created when the application is running. I want to access this data from the host operating system. That is why I am using volume.
When I try to mount my container to named volume or to a host volume, but it always create anonymous volume regardless of the command I type.
Here is my docker file.
FROM openjdk:15
COPY target/lib/* /usr/src/app/lib/
COPY target/core-api-7.3.6.jar /usr/src/app/lib/core-api-7.3.6.jar
COPY config/application.properties /usr/src/app/config/application.properties
COPY data/poscms/config/* /usr/src/app/data/poscms/config/
WORKDIR /usr/src/app
ENTRYPOINT ["java", "-jar", "lib/core-api-7.3.6.jar"]
VOLUME /usr/src/app/data
/usr/src/app/data this is the directory where core-app.jar application will create its runtime data, I need to access these data from my host pc
Following is the command for building the image
docker build -t core-app:5.0 .
then I create image using following command
docker run -it -d -p 7071:7071 core-app:5.0 -v /home/bob/data/:/usr/src/app/data
when I check the volumes by running following command
docker volume ls
I can see anonymous volume being created by this container
and my host path which is /home/kapila/data/ is empty and container data is not written to host path.
I experience the same behaviour with named volume as well.
I created a named volume using following command
docker volume create tmp
docker run -it -d -p 7071:7071 core-app:5.0 -v tmp:/usr/src/app/data
and still docker create anonymous volume and data is not written to tmp volume
my host PC is ubuntu pc. Could someone point out what I am doing wrong here
I do something like this:
In your project root , have these files pertaining to docker as required:
1. DockerFile 2.docker-compose.yml 3. docker-env-preview.env
DockerFile content
FROM openjdk:8-jdk-alpine
ARG jarfilepath
RUN mkdir /src
WORKDIR /src
VOLUME /src/tomcat
ADD $jarfilepath yourprojectname.jar
docker-compose.yml content
version: '3'
services:
project-name:
container_name: project-name-service
build:
context: .
args:
jarfilepath: ./target/project-0.0.1.jar
env_file:
- docker-env-preview.env
ports:
- "8831:8831"
- '5005:5005'
networks:
- projectname_subnet
command: java -jar -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 projectname.jar --spring.profiles.active=preview
networks:
project-name_subnet:
external: true
docker-env-preview.env
This file will contain your environment variables values. The applicaiton.properties can read this file to fetch the values, like buildserver.ip=${BUILD_SERVER_DOMAIN}. Basically you define what you want need . Like the example below.
GARBABE_SERVER_DOMAIN=h-db-preview
GARBABE_SERVER_PORT=5422
GARBABE_DB=projectdb
GARBABE_USER=user
GARBABE_PASSWORD=pwd
JPA_DDL_AUTO=validate
JPA_DIALECT=org.hibernate.dialect.PostgreSQLDialect
JPA_SHOW_SQL=false
JPA_SE_SQL_COMMENTS=true
JPA_FORMAT_SQL=false
JPA_NON_CONTEXTUAL_CREATION=true
APP_NAME=project-name-service
BUILD_SERVER_METHOD=http
BUILD_SERVER_DOMAIN=7.8.9.4
Commands to execute :
mvn clean package (if you use maven )
docker-compose up -d --build ( execute docker ps -> check the details on the running container),
To view the logs : sudo docker logs <project-name-service> -f
To get into the container console, docker exec -it <project-name-service> bash
I was able to fix the issue, and only change I did, to make it work, is that, to change the base image from
FROM openjdk:15
to
FROM adoptopenjdk/openjdk15:ubi
and now named and host volume mounts are working as expected. I am not sure what is wrong with official openjdk:15 image.
I am on my MacBook terminal. I try to have a jenkins container up and running on my local machine.
I firstly created a docker-compose.yml :
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
ports:
- "8080:8080"
volumes:
- $PWD/jenkins_home:/var/jenkins_home
networks:
- net
networks:
net:
As you can see in the volumes section, I have defined the jenkins_home folder under my current directory as the volume for jenkins data.
Then under my current directory of my machine, I created a folder named jenkins_home. Here is my current directory:
-rw-r--r-- 1 john 1349604816 220 Sep 4 00:08 docker-compose.yml
drwxr-xr-x 2 john 1349604816 64 Sep 4 00:06 jenkins_home
As you can see, I need to change the ownership of jenkins_home folder in order to have jenkins container be able to write data in it (because the uid is not 1000). So, I executed command:
sudo chown 1000:1000 jenkins_home/
Then, my current directory looks like this:
-rw-r--r-- 1 john 1349604816 220 Sep 4 00:08 docker-compose.yml
drwxr-xr-x 2 1000 1000 64 Sep 4 00:06 jenkins_home
After that I run my container by command: docker-compose up. But I ended up with error:
Starting jenkins ... done
Attaching to jenkins
jenkins | touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Permission denied
jenkins | Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?
jenkins exited with code 1
Why I still get the permission error after I changed the ownership of the jenkins_home folder under my current directory on my machine?
P.S. I understand there could be other way to purely have a jenkins container running but still I would like to understand what is wrong with my approach and hopefully could also get it work.
Jenkins needs to create or to use existing jenkins_home directory,
When Docker sees that jenkins_home volume in your machine doesn't exists then it will create it with your osx UID & GID.
If you create the jenkins_home folder you must stay with your current directory permissions and not changed them,
Docker running UID isn't the same as your machine, they may have different UID and GID.
Linux namespaces provide isolation for running processes, limiting
their access to system resources without the running process being
aware of the limitations. For more information on Linux namespaces,
see Linux namespaces.
The best way to prevent privilege-escalation attacks from within a
container is to configure your container’s applications to run as
unprivileged users. For containers whose processes must run as the
root user within the container, you can re-map this user to a
less-privileged user on the Docker host. The mapped user is assigned a
range of UIDs which function within the namespace as normal UIDs from
0 to 65536, but have no privileges on the host machine itself.
There a wonderful video explaining how docker works with namespaces
Does the actual jenkins user/group exist on the Mac?
This is what I do on my linux servers where:
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
On my alpine server:
addgroup -g ${gid} ${group}
adduser -u ${uid} -G ${group} -s /bin/bash -D ${user}
to become
addgroup -g 1000 jenkins
adduser -u 1000 -G jenkins -s /bin/bash -D jenkins
On my centos8 server
groupadd -g ${gid} ${group}
useradd -u ${uid} -g ${group} -s /bin/bash -d ${user}
to become
groupadd -g 1000 jenkins
useradd -u 1000 -g jenkins -s /bin/bash -d jenkins
then:
sudo chown jenkins:jenkins jenkins_home/
I do not use Mac, but I presume it is similar
UPDATE
Based on all the above, try the following:
docker-compose.yml
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
ports:
- 8080:8080
- 50000:50000
volumes:
- $PWD/jenkins_home:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
networks:
- net
networks:
net:
I have added the following:
port 50000 (only if you want to attach build slave servers, opposed to just running builds on the master)
volume /var/run/docker.sock (to be able to use the docker daemon with Jenkins, you need to mount the volume)
!!DO THE FOLLOWING!! Delete the original jenkins_home directory that you created before. Now run 'docker-compose up', since the host volume directory does not exist, docker will now create the required directory on the host which is based on the configuration in the docker-compose.yml (in this case '$PWD/jenkins_home'), thus it will now have the correct ownership and permissions for the jenkins container to use it.
If that doesn't work, make the jenkins container run in privileged mode, see below:
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
privileged: true
user: root
ports:
- 8080:8080
- 50000:50000
volumes:
- $PWD/jenkins_home:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
networks:
- net
networks:
net:
How to mount windows folder using docker compose volumes?
I am trying to set up docker container using docker-compose.
My docker-compose.yml file looks as follow:
php-fpm:
build: php-fpm
container_name: php-fpm
volumes:
- ../project:/var/www/dev
When i enter to the container like this:
docker exec -it php-fpm bash
And display content with ls command the /var/www/dev directory is empty.
Does enyone know the solution for this ?
$ docker -v
Docker version 1.12.0, build 8eab29e
$ docker-compose -v
docker-compose version 1.8.0, build d988a55
I have Windows 10 and docker is installed via Docker ToolBox 1.12.0
#edit
The mounted directory is also empty under Linux enviroment
I fixed it by going to: Local Security Policy > Network List Manager Policies and Double-clicked unidentified Networks then change the location type to private and restarted Docker. Source
I just want to test Docker and it seems something is not working as it should. When I have my docker-compose.yml like this:
web:
image: nginx:latest
ports:
- "80:80"
when in browser I run my docker.app domain (sample domain pointed to docker IP) I'm getting default nginx webpage.
But when I try to do something like this:
web:
image: nginx:latest
volumes:
- /d/Dev/docker/nginx-www/nginx/html/:/usr/share/nginx/html/
ports:
- "80:80"
when I run:
docker-compose up -id
when I run same url in browser I'm getting:
403 Forbidden
nginx/1.9.12
I'm using Windows 8.1 as my host.
Do I do something wrong or maybe folders cannot be shared this way?
EDIT
Solution (based on #HemersonVarela answer):
The volume I've tried to pass was in D:\Dev\docker location so I was using /d/Dev/docker at the beginning of my path. But looking at https://docs.docker.com/engine/userguide/containers/dockervolumes/ you can read:
If you are using Docker Machine on Mac or Windows, your Docker daemon has only limited access to your OS X or Windows filesystem. Docker Machine tries to auto-share your /Users (OS X) or C:\Users (Windows) directory.
so what I needed to do, is to create my nginx-ww/nginx/html directory in C:\users\marcin directory, so I ended with:
web:
image: nginx:latest
volumes:
- /c/Users/marcin/docker/nginx-www/nginx/html/:/usr/share/nginx/html/
ports:
- "80:80"
and this is working without a problem. Files are now shared as they should be
If you are using Docker Machine on Windows, docker has limited access to your Windows filesystem. By default Docker Machine tries to auto-share your C:\Users (Windows) directory.
So the folder .../Dev/docker/nginx-www/nginx/html/ must be located somewhere under C:\Users directory in the host.
All other paths come from your virtual machine’s filesystem, so if you want to make some other host folder available for sharing, you need to do additional work. In the case of VirtualBox you need to make the host folder available as a shared folder in VirtualBox.
You have to set a command to copy your nginx.conf into the nginx container:
Dockerfile:
FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf`
Creat a dir name it nginx and put the Dockerfile & nginx.conf there, then you have to set a build:
docker-compose.yml:
web:
image: nginx:latest
build :./nginx/
volumes:
- /d/Dev/docker/nginx-www/nginx/html/:/usr/share/nginx/html/
ports:
- "80:80"
Then build your containers with : sudo docker-compose build
I am working with Docker, and I want to mount a dynamic folder that changes a lot (so I would not have to make a Docker image for each execution, which would be too costly), but I want that folder to be read-only. Changing the folder owner to someone else works. However, chown requires root access, which I would prefer not to expose to an application.
When I use -v flag to mount, it gives whatever the username I give, I created a non-root user inside the docker image, however, all the files in the volume with the owner as the user that ran docker, changes into the user I give from the command line, so I cannot make read-only files and folders. How can I prevent this?
I also added mustafa ALL=(docker) NOPASSWD: /usr/bin/docker, so I could change to another user via terminal, but still, the files have permissions for my user.
You can specify that a volume should be read-only by appending :ro to the -v switch:
docker run -v volume-name:/path/in/container:ro my/image
Note that the folder is then read-only in the container and read-write on the host.
2018 Edit
According to the Use volumes documentation, there is now another way to mount volumes by using the --mount switch. Here is how to utilize that with read-only:
$ docker run --mount source=volume-name,destination=/path/in/container,readonly my/image
docker-compose
Here is an example on how to specify read-only containers in docker-compose:
version: "3"
services:
redis:
image: redis:alpine
read_only: true
docker-compose
Here is a proper way to specify read-only volume in docker-compose:
Long syntax
version: "3.2" # Use version 3.2 or above
services:
my_service:
image: my:image
volumes:
- type: volume
source: volume-name
target: /path/in/container
read_only: true
volumes:
volume-name:
https://docs.docker.com/compose/compose-file/compose-file-v3/#long-syntax-3
Short syntax
Add :ro to the volume mount definition:
version: "3.0" # Use version 3.0 or above
services:
my_service:
image: my:image
volumes:
- /path/on/host:/path/inside/container:ro
https://docs.docker.com/compose/compose-file/compose-file-v3/#short-syntax-3