I'm using Docker 1.13.1 for Mac. The Docker client allows you to change the amount of memory provided to Docker using a simple slider interface.
How can I set this value via docker's command line utility?
For added clarity, this is not per container memory, this is the value of "Total Memory" that's returned when you run docker info.
Thank you
With docker (at least version 18.03.1) the settings for the VM are maintained in a special file located at:
/Users/<username>/Library/Group\ Containers/group.com.docker/settings.json
If you close docker you can edit it directly from the command line using sed, for example the command below will replace the 2 GB limit with a 10GB limit, and create a backup file of the original settings at settings.json.bak
sed -i .bak 's/2048/10240/g' /Users/`id -un`/Library/Group\ Containers/group.com.docker/settings.json
When docker restarts, it will now have 10 GB.
On a Mac, Docker actually runs as a Hyperkit virtual machine. The docker command line utility just interfaces with the docker daemon process running inside that virtual machine.
If you run ps auxwww | grep hyperkit on your Mac, you'll see the hyperkit process running with the amount of memory passed as an argument. This is controlled by the Docker Mac client, and I imagine the saved value is stored in a .plist file somewhere.
In order to modify that on the command line, you'd need to find where the Docker Mac client stores the data, modify it, and restart the hyperkit process.
Related
I'm curious if there's a way to see how much disk space a running Windows container is using in addition to the layers that are part of the container's image. Basically, how much the container "grew" since it was created.
In Linux (Or Linux containers running in a HyperV), this would be docker ps -s, however that command isn't implemented on Windows containers. I also tried docker system df -v but also, not implemented. Perhaps there's a hacky way by looking at a certain directly on disk or something?
I checked on Windows 10 1809 running non-HyperV (process isolation) containers, I'm pretty sure its the same for Windows Server containers.
The data seems to be kept in:
C:\ProgramData\Docker\windowsfilter\{ContainerId}
There's a direct reference to the folder in docker inspect {Id} under GraphDriver\Data\dir.
The folder contains file sandbox.vhdx which appears to be the "writable layer" of each container.
I wasn't able to open it and view the filesystem, but if I write some data inside the container I can force the file to grow:
docker exec <Id> powershell get-childitem c:\ -recurse `> c:\windows\temp\test.txt
The layer persists when the container is stopped/restarted, and the folder is removed when the container is rmed.
While researching I saw an open PR in moby to improve cleanup of this folder.
I'm using docker for windows (docker desktop 2.0.0.3) and docker ps -s is actually implemented.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
81acb264aa0f httpd "httpd-foreground" 6 minutes ago Up 6 minutes 80/tcp httpd 2B (virtual 132MB)
Docker for windows runs on a MobyLinuxVM. You can access the VM and the docker directories:
docker run --privileged -it -v /var/run/docker.sock:/var/run/docker.sock jongallant/ubuntu-docker-client
root#8b58d2fbe186:/# docker run --net=host --ipc=host --uts=host --pid=host –it --security-opt=seccomp=unconfined --privileged --rm -v /:/host alpine /bin/sh
root#8b58d2fbe186:/# chroot /host
Now you can access the docker folders in /var/lib/docker as on linux and check the sizes.
I am running the latest Docker Toolbox, using latest Oracle VirtualBox, with Windows 7 as a host OS.
I am trying to enable non-TLS access to Docker remote API, so I could use Postman REST client running on Windows and hit docker API running on docker-machine in the VirtualBox. I found that if Docker configuration included -H tcp://0.0.0.0:2375, that would do the trick exposing the API on port 2375 of the docker machine, but for the life of me I can't find where this configuration is stored and can be changed.
I did docker-machine ssh from the Toolbox CLI, and then went and pocked around the /etc/init.d/docker file, but no changes to the file survive docker-machine restart.
I was able to find answer to this question for Ubuntu and OSX, but not for Windows.
#CarlosRafaelRamirez mentioned the right place, but I will add a few details and provide more detailed, step-by-step instructions, because Windows devs are often not fluent in Linux ecosystem.
Disclaimer: following steps make it possible to hit Docker Remote API from Windows host, but please keep in mind two things:
This should not be done in production as it makes Docker machine very not secure.
Current solution disables most of the docker-machine and all docker CLI functionality. docker-machine ssh remains operational, forcing one to SSH into docker machine to access docker commands.
Solution
Now, here are the steps necessary to switch Docker API to non-TLS port. (Docker machine name is assumed to be "default". If your machine name has a different name, you will need to specify it in the commands below.)
Start "Docker Quickstart Terminal". It starts Bash shell and is the place where all following commands will be run. Run docker-machine ip command and note the IP address of the docker host machine. Then do
docker-machine ssh
cd /var/lib/boot2docker
sudo vi profile This starts "vi" editor in elevated privileges mode required for editing "profile" file, where Docker host settings are. (If as a Windows user you are not familiar with vi, here's is super-basic crash course on it. When file is open in the vi, vi is not in editing mode. Press "i" to start edit mode. Now you can make changes. After you made all the changes, hit Esc and then ZZ to save changes and exit vi. If you need to exit vi without saving changes, after Esc please type :q! and hit Enter. ":" turns on vi's command mode, and "q!" command means exit without saving. Detailed vi command info is here.)
Using vi, change DOCKER_HOST to be DOCKER_HOST='-H tcp://0.0.0.0:2375', and set DOCKER_TLS=no. Save changes as described above.
exit to leave SSH session.
docker-machine restart
After doocker machine has restarted, your sould be able to hit docker API URL, like http://dokerMachineIp:2375/containers/json?all=1, and get valid JSON back.
This is the end of steps required to achieve the main goal.
However, if at this point you try to run docker-machine config or docker images, you will see an error message indicating that docker CLI client is trying to get to the Docker through the old port/TLS settings, which is understandable. What was not expected to me though, is that even after I followed all the Getting Started directions, and ran export DOCKER_HOST=tcp://192.168.99.101:2375 and export DOCKER_TLS_VERIFY=0, resulting in
$ env | grep DOCKER
DOCKER_HOST=tcp://192.168.99.101:2375
DOCKER_MACHINE_NAME=default
DOCKER_TLS_VERIFY=0
DOCKER_TOOLBOX_INSTALL_PATH=C:\Program Files\Docker Toolbox
DOCKER_CERT_PATH=C:\Users\USERNAME\.docker\machine\machines\default
the result was the same:
$ docker-machine env
Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host
"192.168.99.101:2376"
If you see a problem with how I changed environment variables to point Docker CLI to the new Docker host address, please comment.
To work around this problem, use docker-machine ssh command and run your docker commands after that.
I encountered the same problem and thanks to #VladH made it working not changing any internal Docker profile properties. All you have to do is correctly define Windows local env variables (or configure maven plugin properties, if you use io.fabric8 docker-maven-plugin).
Note that 2375 port is used for non-TLS connections, and 2376 only for TLS connections.
DOCKER_HOST=tcp://192.168.99.100:2376
DOCKER_TLS_VERIFY=0
DOCKER_TOOLBOX_INSTALL_PATH=C:\Program Files\Docker Toolbox
DOCKER_CERT_PATH=C:\Users\USERNAME\.docker\machine\machines\default
I am using docker on windows. With the use of kitematic, I have created an ubuntu container. This ubuntu image has postgresql installed on it.
Wondering if there is any possibility to access the postgres configuration files available in the container from the host (windows machine)?
Where exactly does the container store its file system on the host machine?
I hope it would be part of image file with format VMDK.
Please correct me if I'm wrong.
Wondering if there is any possibility to access the postgres configuration files available in the container from the host (windows machine)
That is not how Docker would allow you to modify a file in a container.
For that, you should mount a host (Windows) folder when starting (docker run -v) your container.
See "Mount a host directory as a data volume"
docker run -d -P --name web -v /c/Users/<myACcount>/src/webapp:/opt/webapp training/webapp python app.py
Issue 247 mentions ~/Library/Application Support/Kitematic for App data, and ~/Kitematic "for easy access to volume data".
I'm working on wrapping some scientific software by docker image using boot2docker on Mac OS X. And that software (https://github.com/voutcn/megahit.git) is using named pipes (in python code, but it's not important) to wire different parts (written in C) to each other. I mount temporary folder from host Mac OS X machine to provide scratch area in docker container (because temporary output of software could be huge) with something like this:
docker run -v /external/folder:/tmp/scratch <image> <args>
It gives me this mount line inside container:
none on /tmp/scratch type vboxsf (rw,nodev,relatime)
And inside this mounted folder named pipe creation fails when it runs inside container. It's not even related to python, C or any particular language. I double checked with linux command mkfifo pipe1 in this folder with an error:
mkfifo: cannot create fifo 'pipe1': Operation not permitted
It works well for any internal not mounted folder inside container though. Why does it happen and how could it be fixed?
PS: Here is what I do to easily reproduce the problem.
1) Mac OS X with boot2docker
2) Dockerfile is:
FROM ubuntu:14.04
#WORKDIR /tmp <- this one would work
WORKDIR /tmp/scratch
ENTRYPOINT [ "mkfifo" ]
CMD [ "pipe1" ]
3) Image building:
docker build --rm -t mine/namedpipes:latest .
4) Running (being in external host folder to be mounted):
docker run -v $(pwd):/tmp/scratch mine/namedpipes:latest
Upgrade to a recent version of Docker for Mac, and your problem will likely be solved: https://docs.docker.com/docker-for-mac/release-notes/#beta-2-release-2016-03-08-1102-beta2
The issue is that FIFOs are actually kernel objects you access using the filesystem, and so you would need extra work to support cross-kernel FIFOs (or unix domain sockets) - a fifo is either valid inside the Linux guest running the docker daemon or in the OS X host, not in both, and it makes sense that you can't create an OS X fifo from inside the linux box. It would be sort of like trying to create a fifo on a network drive, it doesn't make sense as a local IPC mechanism.
Current support for special files is detailed in https://docs.docker.com/docker-for-mac/osxfs/#file-types
The issue for cross-hypervisor support is located at https://github.com/docker/for-mac/issues/483
I installed boot2docker as explained on the docker website. Here are some command runs to show that I have things installed correctly:
$$:~ kv$ boot2docker start
Waiting for VM and Docker daemon to start...
...................ooo
Started.
Writing /Users/kvantum/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/kvantum/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/kvantum/.boot2docker/certs/boot2docker-vm/key.pem
Your environment variables are already set correctly.
$$:~ kv$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
ubuntu 14.04 b39b81afc8ca 11 days ago 188.3 MB
hello-world latest e45a5af57b00 3 weeks ago 910 B
After this, I ran the following command:
docker run -t -i ubuntu:14.04 /bin/bash
Inside the container, I installed zeromq, and started a zeromq server on port 5555 using tcp.
My questions are following:
If I exit out of the container, will it save all the work I do inside it?
I have no idea how to connect to the server running on port 5555. I read something about exposing a port, but I am not sure how to go about doing that. I did an ifconfig inside the container, and tried to connect to the server from the host like this:
$$:~ kv$ ./zmq_client tcp://container_ip:5555
This did not work. Can someone please lists the steps I need to take in order to connect to the server running within the container.
For completion sake, I am providing the list of my environment variables:
TERM_PROGRAM=Apple_Terminal
TERM=xterm-256color
SHELL=/bin/bash
TMPDIR=/var/folders/km/5kbpdx4s7cg4rmyc6d5q9l9r0000gq/T/
DOCKER_HOST=tcp://192.168.109.103:2376
Apple_PubSub_Socket_Render=/tmp/launch-1tWMHJ/Render
TERM_PROGRAM_VERSION=326
OLDPWD=/Users
TERM_SESSION_ID=262CBC8B-0A74-4B70-9F28-D9FA51FF713C
USER=kv
SSH_AUTH_SOCK=/tmp/launch-ZTWNGL/Listeners
__CF_USER_TEXT_ENCODING=0x1F7:0:0
DOCKER_TLS_VERIFY=1
__CHECKFIX1436934=1
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin
PWD=/Users/kv
DOCKER_CERT_PATH=/Users/kv/.boot2docker/certs/boot2docker-vm
HOME=/Users/kv
SHLVL=1
LOGNAME=kv
LC_CTYPE=UTF-8
DISPLAY=/tmp/launch-rco9zt/org.macosforge.xquartz:0
_=/usr/bin/env
One last question I have is about code performance. So within my Mac OS X, I have a docker container running (which runs Ubuntu). If I run the application, like a zeromq based server inside the container, will it not be slower as compared to running it on Mac OS X directly. Please explain the benefits of using docker in such a scenario..
You should really do some more reading and research before turning to SO, then ask about anything you can't figure out. But:
No. If the container is "exited" you can restart it and your files will still be there, but once it is removed your files are gone. You can use docker commit to save them to an image, but the best bet is to use a Dockerfile.
docker run -p 5000:8000 image will expose port 8000 in the container as port 5000 on the host.
Yes, it will be slower due to the boot2docker VM. It would not be slower if you were running on a Linux host. The advantage is that zeromq is now running in an isolated container with all its dependencies.