Configuring user permissions in docker-compose - windows

I have struggled the past few days with getting docker volumes to use the correct permissions with a headless ubuntu server. I am very new to docker and still a novice with ubuntu so it may well be obvious to others!
My setup:
I have a mini-PC with a relatively small SSD running Windows.
I have an ubuntu server that has multiple (old) HDD that combine to form a storage pool for all my media (photos, movies etc)
I run emby server on the Windows machine to manage the media. However, I use the server for the emby cache and transcoding as I want to minimise the read/write activity on the SSD to prolong its life.
Because I use emby away from home too, I have a VPN to protect my home network with port-forwarding. But because I don't want all my traffic on the VPN I recently decided to try to move emby and the VPN to a docker container so that I could use the Windows PC for other purposes too (and because I wanted to learn a bit on Docker).
I had managed to get my original emby server working with ubuntu network folders by using SAMBA and I also ultimately had to set all permissions on the relevant ubuntu folders with the media to 2777 so that I could get windows to communicate with read/write access. That in itself might suggest a problem.
Note: I have created a user:group on ubuntu called emby:emby and the uid:gid is 996:998.
But for the life of me I am unable to get emby server as a docker container running with the correct permissions as whenever emby needs to access the ubuntu folders to write (it can read fine) then a get an access denied error in the logs. This was not the case when emby server ran on Windows outside a docker container (although as mentioned I had to open all the security of the folders up to get it working).
What I have tried (without any success):
removed VPN
set uid:gid to 996:998 in docker-compose (I have also tried other users like 1000:1000 as root)
I have tried to test by creating a new file from the windows PC into an ubuntu network folder and double-checked that the user:group is emby:emby (which it is)
I also mapped the relevant ubuntu folder to a Windows drive and used this Windows drive in docker-compose
tried changing in the emby container to the network-mode: bridge in emby server
removed the FIREWALL
But I have run out of ideas and nothing I find on google seems to provide the detail I need. Below is my docker-compose file and any suggestion are very welcome! Note I may need my hand-held to walk through the steps...
version: '3.7'
volumes:
emby-movies:
driver_opts:
type: cifs
o: "user=u,password=p,file_mode=0777,dir_mode=0777,iocharset=utf8,vers=3.1.1,rw,uid=996,gid=998,sec=ntlmssp"
device: //192.168.1.162/Entertainment/Movies
emby-tvshows:
driver_opts:
type: cifs
o: "user=u,password=p,file_mode=0777,dir_mode=0777,iocharset=utf8,vers=3.1.1,rw,uid=996,gid=998,sec=ntlmssp"
device: //192.168.1.162/Entertainment/TV_Shows
emby-cache:
driver_opts:
type: cifs
o: "user=u,password=p,file_mode=0777,dir_mode=0777,iocharset=utf8,vers=3.1.1,rw,uid=996,gid=998,sec=ntlmssp"
device: //192.168.1.162/EmbyCache/cache
emby-homepictures:
driver_opts:
type: cifs
o: "user=u,password=p,file_mode=0777,dir_mode=0777,iocharset=utf8,vers=3.1.1,rw,uid=996,gid=998,sec=ntlmssp"
device: //192.168.1.162/Multimedia/Pictures
emby-homevideos:
driver_opts:
type: cifs
o: "user=u,password=p,file_mode=0777,dir_mode=0777,iocharset=utf8,vers=3.1.1,rw,uid=996,gid=998,sec=ntlmssp"
device: //192.168.1.162/Multimedia/Home_Videos
services:
gluetun:
image: qmcgaw/gluetun
container_name: provider
cap_add:
- NET_ADMIN
environment:
- VPN_SERVICE_PROVIDER=zzz
- OPENVPN_USER=xxx
- OPENVPN_PASSWORD=yyy
- SERVER_CITIES=Paris,Lisbon,Prague
- FIREWALL_OUTBOUND_SUBNETS=192.168.1.0/24
ports:
- 7170:7170 # emby HTTP port
- 7180:7180 # emby HTTPS port
volumes:
- C:\Docker\gluetun:/gluetun
restart: unless-stopped
emby:
image: emby/embyserver
container_name: embyserver
network_mode: "service:gluetun" # VPN
environment:
- UID=996 # The UID to run emby as (default: 2)
- GID=998 # The GID to run emby as (default 2)
# - GIDLIST=100 # A comma-separated list of additional GIDs to run emby as (default: 2)
volumes:
- C:\Docker\Emby_Data:/config # Configuration directory
- emby-cache:/mnt/cache # cache directory
- emby-tvshows:/mnt/tvshows # Media directory
- emby-movies:/mnt/movies # Media directory
- emby-homepictures:/mnt/homepictures # Media directory
- emby-homevideos:/mnt/homevideos # Media directory
restart: unless-stopped

Related

Docker mount volume error no such file or directory (Windows)

I am trying to set up an EMQx broker in the form of deploying it with Docker. One of my constraints is to do this on Windows. To be able to use TLS/SSL authentication there must be a place to put certs in the container therefore I'd like to mount a volume.
I have tried several ways and read myriad of comments but I cannot make it work consistently. I always bump into "no such file or directory" message.
More interestingly once I got it to work and also saved the .yml file right after, but next time when I used the command docker-compose up with this yaml("YAML that worked once"), I received the usual and same message ("Resulting error message").
Path where the certs reside -> c:\Users\danha\Desktop\certs
Lines in question (please see the entire YAML below):
volumes:
- vol-emqx-conf://C//Users//danha//Desktop//certs
volumes:
vol-emqx-conf:
driver_opts:
type: none
device: /Users/danha/Desktop/certs
o: bind
YAML that worked once:
version: '3.4'
services:
emqx:
image: emqx/emqx:4.3.10-alpine-arm32v7
container_name: "emqx"
hostname: "emqx"
restart: always
environment:
EMQX_NAME: lms_emqx
EMQX_HOST: 127.0.0.1
EMQX_ALLOW_ANONYMOUS: "false"
EMQX_LOADED_PLUGINS: "emqx_auth_mnesia"
EMQX_LOADED_MODULES: "emqx_mod_topic_metrics"
volumes:
- vol-emqx-conf://C//Users//danha//Desktop//certs
labels:
NAME: "emqx"
ports:
- 18083:18083
- 1883:1883
- 8081:8081
volumes:
vol-emqx-conf:
driver_opts:
type: none
device: //C//Users//danha//Desktop//certs
o: bind
Resulting error message
C:\Users\danha\Desktop\dc>docker-compose up
Creating network "dc_default" with the default driver
Creating volume "dc_vol-emqx-conf" with default driver
Creating emqx ... error
ERROR: for emqx Cannot start service emqx: error while mounting volume '/var/lib/docker/volumes/dc_vol-emqx-conf/_data': failed to mount local volume: mount \\c\Users\danha\Desktop\certs:/var/lib/docker/volumes/dc_vol-emqx-conf/_data, flags: 0x1000: no such file or directory
ERROR: for emqx Cannot start service emqx: error while mounting volume '/var/lib/docker/volumes/dc_vol-emqx-conf/_data': failed to mount local volume: mount \\c\Users\danha\Desktop\certs:/var/lib/docker/volumes/dc_vol-emqx-conf/_data, flags: 0x1000: no such file or directory
ERROR: Encountered errors while bringing up the project.
I have also played around with forward and back slashes but these did not bring any success. At the end I entered a path which gave the result mostly resembling on the correct path in the error message:
YAML neglecting C: from the beginning of path:
version: '3.4'
services:
emqx:
image: emqx/emqx:4.3.10-alpine-arm32v7
container_name: "emqx"
hostname: "emqx"
restart: always
environment:
EMQX_NAME: lms_emqx
EMQX_HOST: 127.0.0.1
EMQX_ALLOW_ANONYMOUS: "false"
EMQX_LOADED_PLUGINS: "emqx_auth_mnesia"
EMQX_LOADED_MODULES: "emqx_mod_topic_metrics"
volumes:
- vol-emqx-conf:/Users/danha/Desktop/certs
labels:
NAME: "emqx"
ports:
- 18083:18083
- 1883:1883
- 8081:8081
volumes:
vol-emqx-conf:
driver_opts:
type: none
device: /Users/danha/Desktop/certs
o: bind
Resulting error message
C:\Users\danha\Desktop\dc>docker-compose up
Creating volume "dc_vol-emqx-conf" with default driver
Creating emqx ... error
ERROR: for emqx Cannot start service emqx: error while mounting volume '/var/lib/docker/volumes/dc_vol-emqx-conf/_data': failed to mount local volume: mount C:\Users\danha\Desktop\certs:/var/lib/docker/volumes/dc_vol-emqx-conf/_data, flags: 0x1000: no such file or directory
ERROR: for emqx Cannot start service emqx: error while mounting volume '/var/lib/docker/volumes/dc_vol-emqx-conf/_data': failed to mount local volume: mount C:\Users\danha\Desktop\certs:/var/lib/docker/volumes/dc_vol-emqx-conf/_data, flags: 0x1000: no such file or directory
ERROR: Encountered errors while bringing up the project.
That also got me thinking that this issue might be related to access rights and file sharing between Windows and WLS2 and the CMD was run in admin mode too, however I could find any answer further down the line that would have helped.
Probably this is pretty a newbie question but any help would be greatly appreciated.

How to mount a NFS file share inside a Windows Docker Container

I am running Docker on a Windows Server 2022.
I need to start a Windows Container - must be a windows container because I am porting a Net Framework App that cannot run on Linux.
My Windows Container needs to write uploaded files to a network share.
I have read lots of discussion, but I could only find example on Linux containers.
Currently I am trying using a Docker Compose file. This is the compose file:
version: "3.9"
services:
web:
image: misterpanel
ports:
- "80:80"
volumes:
- misterpanel_images:/inetpub/wwwroot/Img
volumes:
misterpanel_images:
driver: local
driver_opts:
type: cifs
o: username=<myysernane>,password=<mypassword>,rw,domain=<mydomain>
device: "\\\\<server-ip-address>\\<files-share>"
Misterpanel image was created FROM mcr.microsoft.com/dotnet/framework/aspnet
When running docker-compose up, I am getting the following error:
PS C:\Users\Administrator\docker> docker-compose up
Creating volume "docker_misterpanel_images" with local driver
ERROR: create docker_misterpanel_images: options are not supported on this platform
I have also tryed to map the network drive at the host, and then mount the bind when starting the container like this:
docker run -p 80:80 --dns 8.8.8.8 --mount type=bind,source=z:\,target=c:\inetpub\wwwroot\Img misterpanel
But then I get this error ( even with Z drive working at the host ):
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: z:.
I have tryed all the possible sintaxes for the source ( \Z\ and //Z/ for example )
I could only mount binds on the Windows Container to the local C: drive.
Does anyone has ever mounted a bind to a file share inside a Windows Container?
Any help will be appreciated.
Thanks,

Docker (for Windows) does not mount volume

I'm trying to mount a directory with configuration files in my docker-compose.yml.
In my case it is logstash, which tells me the mounted directory is empty.
Loading a bash and ls -la in the parent directory shows that the pipeline directory is empty and is owned by root.
One weird thing is, that it worked a few days ago.
docker-compose.yml:
version: '3'
services:
logstash:
image: docker.elastic.co/logstash/logstash:5.6.3
ports:
- 5000:5000
- 8989:8989
volumes:
- C:/PROJECT_DIR/config/logstash/pipeline/:/usr/share/logstash/pipeline/
I found it better to try around with docker itself, as it gives more feedback
docker run --rm -it -v C:/PROJECT_DIR/config/logstash/pipeline/:/usr/share/logstash/pipeline/ docker.elastic.co/logstash/logstash:5.6.3
From here and some googling I found out I had to reset my shared drives credentials under "Docker for Windows" -> Settings... -> Shared Drives, because I had changed my windows domain user password.
If you changed your system username or password then you need to re-apply the credentials to get the volume mount working.

Docker compose - share volume Nginx

I just want to test Docker and it seems something is not working as it should. When I have my docker-compose.yml like this:
web:
image: nginx:latest
ports:
- "80:80"
when in browser I run my docker.app domain (sample domain pointed to docker IP) I'm getting default nginx webpage.
But when I try to do something like this:
web:
image: nginx:latest
volumes:
- /d/Dev/docker/nginx-www/nginx/html/:/usr/share/nginx/html/
ports:
- "80:80"
when I run:
docker-compose up -id
when I run same url in browser I'm getting:
403 Forbidden
nginx/1.9.12
I'm using Windows 8.1 as my host.
Do I do something wrong or maybe folders cannot be shared this way?
EDIT
Solution (based on #HemersonVarela answer):
The volume I've tried to pass was in D:\Dev\docker location so I was using /d/Dev/docker at the beginning of my path. But looking at https://docs.docker.com/engine/userguide/containers/dockervolumes/ you can read:
If you are using Docker Machine on Mac or Windows, your Docker daemon has only limited access to your OS X or Windows filesystem. Docker Machine tries to auto-share your /Users (OS X) or C:\Users (Windows) directory.
so what I needed to do, is to create my nginx-ww/nginx/html directory in C:\users\marcin directory, so I ended with:
web:
image: nginx:latest
volumes:
- /c/Users/marcin/docker/nginx-www/nginx/html/:/usr/share/nginx/html/
ports:
- "80:80"
and this is working without a problem. Files are now shared as they should be
If you are using Docker Machine on Windows, docker has limited access to your Windows filesystem. By default Docker Machine tries to auto-share your C:\Users (Windows) directory.
So the folder .../Dev/docker/nginx-www/nginx/html/ must be located somewhere under C:\Users directory in the host.
All other paths come from your virtual machine’s filesystem, so if you want to make some other host folder available for sharing, you need to do additional work. In the case of VirtualBox you need to make the host folder available as a shared folder in VirtualBox.
You have to set a command to copy your nginx.conf into the nginx container:
Dockerfile:
FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf`
Creat a dir name it nginx and put the Dockerfile & nginx.conf there, then you have to set a build:
docker-compose.yml:
web:
image: nginx:latest
build :./nginx/
volumes:
- /d/Dev/docker/nginx-www/nginx/html/:/usr/share/nginx/html/
ports:
- "80:80"
Then build your containers with : sudo docker-compose build

Docker, mount volumes as readonly

I am working with Docker, and I want to mount a dynamic folder that changes a lot (so I would not have to make a Docker image for each execution, which would be too costly), but I want that folder to be read-only. Changing the folder owner to someone else works. However, chown requires root access, which I would prefer not to expose to an application.
When I use -v flag to mount, it gives whatever the username I give, I created a non-root user inside the docker image, however, all the files in the volume with the owner as the user that ran docker, changes into the user I give from the command line, so I cannot make read-only files and folders. How can I prevent this?
I also added mustafa ALL=(docker) NOPASSWD: /usr/bin/docker, so I could change to another user via terminal, but still, the files have permissions for my user.
You can specify that a volume should be read-only by appending :ro to the -v switch:
docker run -v volume-name:/path/in/container:ro my/image
Note that the folder is then read-only in the container and read-write on the host.
2018 Edit
According to the Use volumes documentation, there is now another way to mount volumes by using the --mount switch. Here is how to utilize that with read-only:
$ docker run --mount source=volume-name,destination=/path/in/container,readonly my/image
docker-compose
Here is an example on how to specify read-only containers in docker-compose:
version: "3"
services:
redis:
image: redis:alpine
read_only: true
docker-compose
Here is a proper way to specify read-only volume in docker-compose:
Long syntax
version: "3.2" # Use version 3.2 or above
services:
my_service:
image: my:image
volumes:
- type: volume
source: volume-name
target: /path/in/container
read_only: true
volumes:
volume-name:
https://docs.docker.com/compose/compose-file/compose-file-v3/#long-syntax-3
Short syntax
Add :ro to the volume mount definition:
version: "3.0" # Use version 3.0 or above
services:
my_service:
image: my:image
volumes:
- /path/on/host:/path/inside/container:ro
https://docs.docker.com/compose/compose-file/compose-file-v3/#short-syntax-3

Resources