I have a docker Mac host running an Ubuntu 19.10 guest. I am sharing a folder on my Mac filesystem that contains ~56,000 files totaling 230MB. When I ls that directory from within a docker container, it responds with:
Input/output error: <the directory name>
The folder is shared when I run the container via docker run -v /path/on/mac/fs:/path/in/guest -it my-image:latest /bin/bash.
When I ls the directory from a Mac shell, it works as expected.
Trying with a pared-down version of the directory (29k files) works as expected.
I have found a workaround: disabling "Use gRPC FUSE for file sharing" in the docker dashboard. ls works from within the container, but I still don't really understand the root of the problem.
56k files is not trivial, but also I wouldn't have guessed it's so big as to break docker.
So my questions:
Why might this be happening? Is this actually a gRPC issue, or something else?
Is there any documentation on gRPC FUSE limits? I couldn't find any.
Related
I have installed Docker Toolbox for Windows v 18.09 in Windows 10 Version 10.0.19041 Build 19041 and am trying to run a docker container to run Jupiter Notebook with Pyspark.
I am using Windows Powershell to execute docker commands
docker run hello-world works fine so I can assume that Dockers has been installed correctly. Infact, I had go down to Toolbox v 18.09 before I could get hello world to work.
i use the following command to run the pyspark container
docker run -it --rm -p 8888:8888 --volume=//C/Users/prith/pydev://home/jovyan/work jupyter/pyspark-notebook
from the C:/Users/prith/pydev directory that is mapped to the work directory of the container. The // is slash is required because i am working with Windows. the notebook shows up in http://192.168.99.100:8888 as expected and I can login with the token.
Then the problem starts when i try to create a new notebook or even a text file, I get a permission denied error. Evidently the container cannot write to 'some' directory. I have used Windows filesystem properties to give Everyone all privileges on this particular directory and have also run the Powershell in Adminstrtor mode ( to simulate Ubuntu sudo ..) but nothing works.
Interestingly, I can write into the directory located above the work directory in the container but then I cannot access files written into that directory from Windows because I have mapped my local windows directory to /home/jovyan/work
What do what i want? I want to create Jupyter notebooks in the container and save them in Windows
I know all this works like a charm in Linux-Ubuntu, but unfortunately I am stuck with Windows 10. Please help.
It looks you forgot to add the directory you're trying to mount to FILE SHARING.
Please, do right-click on docker icon (in system tray) -> Settings -> Resources -> FILE SHARING
Then, add your local directory.
Finally, if it doesn't work, try to mount volume with --volume="C:\Users\prith\pydev":/home/joyvan/work
this command seems to work
docker run -it --rm -p 8888:8888 --volume='/c/Users/Public/PyDev'://home/jovyan/work jupyter/pyspark-notebook start-notebook.sh --NotebookApp.token=''
I mounted a folder in docker ubuntu container
docker run -it --rm -v ${pwd}:/files ubuntu /bin/bash
but when i try to write to mounted folder get error
root#0974e91f50ec:/files# cat >a.txt
bash: a.txt: Permission denied
Is it possible to write to mounted folder?
I just tried your example. Here's what I ran on my windows machine.
And the file appears fine.
Also they appear fine in the host system (I created b.txt on the host to check if it appears fine in the container).
PS: Something you have to beware of is that on Windows you can only mount very specific directories (e.g. ~/Documents directory) at least when using Docker toolbox (Not sure about Docker Desktop CE). I think to mount folders from other directories there's bit of work to be done. So it might be a good idea to make sure that whatever you are trying to mount is in fact something allowed by your Docker on Windows.
I use 1.12 version of Docker on Windows, since I can't use the Hyper-V feature with the newer "native" version - so I have my quickstart terminal and communicate to docker host via the invisible underlying virtual box.
Now I have the problem, that I need to mount a local folder to a container, which worked successfully from within the docker-machine by adding
--volume="`pwd`:/root/data"
to the docker run command, but it does not when I launch the same command from my Windows quickstart terminal (even though pwd command works correctly in the terminal).
I tried to find the Windows specific settings for the directory and tested several combinations of format, but no luck. Can anyone help me out on how to correctly specify a Windows folder (e.g. C:\Users\alexander.ruehl) for the volume parameter?
You can use relative path for your volume : --volume="./mydata:/root/data"
Also make sure that you have given the permission for read/write to Docker.
Well using Kitematic on Windows and starting the container hello-world-nginx. I'm totally able to edit the file on Windows and see the modifications at the browser.
But when I create some image and try to edit the file (html), nothing happens.
Here's the command:
docker build -t a-example . && docker run --name a-example -i -t -v //c/Users/pablo/htdocs/www/a-example/src:/www -d -p 80:80 a-example
And here's my Dockerfile:
FROM nginx
RUN mkdir /etc/nginx/logs && touch /etc/nginx/logs/a-example.log
ADD ./nginx.conf /etc/nginx/conf.d/default.conf
ADD /src /www
VOLUME /www
Another question:
Is this message have something to do with?
SECURITY WARNING: You are building a Docker image from Windows against
a non-Win dows Docker host. All files and directories added to build
context will have '-r wxr-xr-x' permissions. It is recommended to
double check and reset permissions f or sensitive files and
directories.
I do not use KiteMatic or Docker Machine to control my local docker instance.
However, Docker Machine uses a boot2docker iso (customized version of Tiny Core Linux with Docker) running in a VirtualBox vm. Virtual Box provides guest Additions to mount host machine folders.
Refer to Boot2Docker guidelines at the time of Docker 1.9.1 release
Or the Docker Machine shares:
On Windows:
- C:\Users mounted as /c/Users
On Linux:
- /home mounted as /hosthome
On OSX:
- /Users mounted as /Users
which is done through Guest Additions:
vbm("sharedfolder", "add", d.MachineName, "--name", shareName, "--hostpath", shareDir, "--automount")
If you opt to use samba shares from your windows host instead, you will need the appropriate TLC packages (from the FAQ):
How do I mount windows shares?
If the sharing computer is at least Windows 2000, you can mount the share with only the filesystems-2.6.29.1-tinycore.tcz extension.
In a root console:
mkdir /mnt/windows
mount -t cifs //9.53.216.11/myshare /mnt/windows -o user=myname,pass=mypassword
And after searching for boot2docker and filesystems-2.6.29.1-tinycore.tcz I got to this GitHub issue:
sudo find / -type d -name 'cifs' seems to indicate that CIFS (Samba) client in the right kernel version is already there... allowing us to run
sudo mount -t cifs //MYPC/hostgithub ~/windows -o user=USER
on the boot2docker VM
Using Docker Volumes
It seems Docker Volumes are a new proposal for handling host folders for Developer Environments, see:
Docker-Volume-Netshare: Although this requires nfs-common or nfs-utils (depending on your distro) to be available on the docker machine, this may not work for CoreOS / Boot2Docker without some other tricks.
It should actually work on boot2docker considering how Docker-Volume-Netshare just wraps shell exec /bin/sh -c "mount -t cifs ..." command. If you use the so0k/mount.cifs_copy on CoreOS you may need a way to specify the path to cifs /opt/bin/mount.cifs.
For Windows users, note also my blog post using the netshare volume driver mentioned above to provision mounts from the Windows host across a Swarm Cluster.
Note: Be careful when using git-for-windows and git-bash as I'm having issues with these converting the paths I pass on to docker.exe
I don't think your volume is available so your container is displaying the static content from /src that you ADDed during the image build.
Kitematic runs a VirtualBox linux machine which runs your docker containers.
The path you have used for the volume: //c/Users/pablo/htdocs/www/a-example/src:/www looks suspect as that's the full windows path.
Docker machine (which is what kitematic uses underneath to manage the virutal machine) will map C:\Users to /Users by default
Try --volume /Users/pablo/htdocs/www/a-example/src:/www
I'm trying to attach a volume to a docker container when it starts as a way of providing input to the docker container. When I type:
docker run swanijam/djredcap-harvest -v Users/swanickj/testFolder/Dock/Input:/Input
The Input folder never appears in the container.
I'm aware that there have been issues of boot2docker not having access to /Users but in my VirtualBox /Users is shared, as is default in boot2docker > 1.3.
The Input folder is being accessed in the ENTRYPOINT line of my Dockerfile, when it is built.
If you need me to show you any code or other files, please let me know!
Please pay attention to the path you are using. It should be:
docker run swanijam/djredcap-harvest -v /Users/swanickj/testFolder/Dock/Input:/Input
Boot2docker automatically mounts the /Users folders from your mac, and you should find it inside your boot2docker instance as:
cd /Users
Hope it helps you.