Update max_map_count for ElasticSearch docker container Mac host - elasticsearch

I'm using this container to start elasticsearch in docker. In accordance with the manual I have to update max_map_count to start the container
sudo sysctl -w vm.max_map_count=262144
but.. I can update it in my host (container) AFTER I start it while I'm unable to start it. Am I doing something wrong?
ERROR: bootstrap checks failed max virtual memory areas
vm.max_map_count [65530] likely too low, increase to at least [262144]
If I try to do it on my host machine (which is Mac) I get the following error.
sysctl: unknown oid 'vm.max_map_count'
Docker engine installs the Lunix VM where all containers are running. So the command to increase the limit should be executed for the Linux host, not for the Mac.
How can I access Linux VM via terminal installed by the Docker engine?

On Docker Toolbox
If you are in docker toolbox try the docker client from terminal and then make the configs:
docker-machine ssh
sudo sysctl -w vm.max_map_count=262144
exit
On Docker For Mac:
The vm_max_map_count setting must be set within the xhyve virtual machine:
$ screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
If it asks you for a username and passwordLog in with root and no password.
If it just has a blank screen, press RETURN.
Then configure the sysctl setting as you would for Linux:
sysctl -w vm.max_map_count=262144
Exit by Control-A Control-\.
Se the docs here.
Persistence
In some cases, this change does not persist across restarts of the VM. So, while screen'd into, edit the file /etc/sysctl.d/00-alpine.conf and add the parameter vm.max_map_count=262144 to the end of file.

On Latest Docker For Mac (Version 18.06.0-ce-mac70):
$ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
On the blank screen, press RETURN.
Then configure the sysctl setting as you would for Linux:
sysctl -w vm.max_map_count=262144
Exit by Control-A Control-\

For those using Docker Desktop on windows 10 you have to execute:
wsl -d docker-desktop in the command line before sysctl -w vm.max_map_count=262144

For those who use docker desktop on mac, you can easily increase the memory by the following steps:
click on docker desktop -> preferences...
navigate to 'Resources'
change the memory to whatever you need
click on 'Apply & Restard'

Folder has been moved and this is the new location -
$screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
and then
sysctl -w vm.max_map_count=262144

For mac users you might have a problem connecting to docker VM, therefore you should run this command to enter the shell of the Docker VM:
docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh
GitHub issue description: https://github.com/docker/for-mac/issues/4822

If you have installed Docker from docker's Mac installer then you will have Docker desktop installed(also includes includes Docker Engine, Docker CLI client, Docker Compose, Notary, Kubernetes, and Credential Helper.)
Here is what Docker desktop looks like in 2021 where you can change memory/swap or any other resources.
Step 1 - click on docker's preferences as shown below.
Step 2 - Click on resources tab, here you can tweak the resources and finally click on "Apply and Restart" button.
Please ignore configuration what I made. You can set based on your requirement.

Related

committed image fails to render desktop as a container

i am attempting to run a windows GUI app in a container on Linux. The intent is to protect an ancient windows app that is no longer supported. So i get a Red Hat developer subscription, install RHEL 8.6 with container tools, run the universal base image 'UBI-INIT', and within the container, i install GNOME desktop with Xrdp, and i successfully render the GUI desktop in a RHEL container.
Now that the container is working well, I commit to an image, but when i run that image, the GUI fails to render. the xrdp session times out as if services are not running and/or ports are not accessible.
Within the container that i ran from the committed image:
i verify that all of the services necessary to support XRDP and GNOME are up and running.
journalctl does not seem to show any errors. There are complaints around rtkit but i see similar errors in the working container.
i see no evidence that an xrdp connection was attempted in the xrdp or xrdp-sesman logs. But i am fairly certain that ports are not the issue because i can ssh to the container.
the commands i used to install and configure the working container are:
podman run -d -v /mnt/share:/share -p 53389:3389 -p 50022:22 --rm --privileged --name ubi-ini registry.access.redhat.com/ubi8/ubi-init;
podman exec -it ubi-ini bash
and within the container i run the following:
timedatectl set-timezone America/New_York
# GNOME desktop GUI
dnf install -y selinux-policy-targeted
dnf groupinstall -y --skip-broken "Server with GUI"
# xrdp
dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
dnf install -y xrdp
echo 'if [ "$DISPLAY" !=' "'\"\"' ]; then xhost +; fi;" >> /etc/profile
sed -i '/^port=3389.*/a port=tcp://:3389' /etc/xrdp/xrdp.ini
useradd -g root -p $(echo "jde" | openssl passwd -1 -stdin) jde
usermod -aG wheel jde
systemctl enable xrdp xrdp-sesman gdm
systemctl unmask systemd-logind.service
systemctl restart sshd xrdp xrdp-sesman dbus gdm systemd-logind.service
I commit the image like this:
podman commit ubi-ini ubi-gui
I run the image with this command:
podman run -d -v /mnt/share:/share -p 63389:3389 -p 60022:22 --rm --privileged --name ubi-gui ubi-gui
xrdp communicates with the desktop manager through systemd UBI-INIT is the only linux base container that supports systemd.
i suspect there is something about the processes in the derived container but when i compare the working and non-working container with ps aux, i don't see significant anomolies.
Any ideas?
I have absolutely no idea about 'XRDP' but I see you use a different host port in your second container instance, is that intentional?
Got this to work by disabling firewall and selinux everywhere, meaning the container host abd the base container UBI-INIT as well. Now the image based on the modified container (with Gnome desktop and XRDP and disabled security) results in a container that serves the GUI desktop.
It's working fine except that gdm (gnome desktop manager) does not start even though it is enabled and all the other enabled services are ok. Still working that one out, but the basic question is answered: it was not the software stack but rather it was security configuration. i suspect selinux in the container somehow interfered with inter-process communication, because i am able to ssh on (mapped) port 22 externally.

Cannot mount a volume without using sudo

The following command works and mounts the local volume:
sudo docker run -ti -v "$PWD/codebase/realsmart-saml-copy":/var/www/html realsmart-docker_smartlogin bash
The following command does not work and does not mount the volume
docker run -ti -v "$PWD/codebase/realsmart-saml-copy":/var/www/html realsmart-docker_smartlogin bash
For some reason, docker is only able to mount volumes using the sudo command, rendering our local docker environment useless on a colleagues laptop. The same docker-compose file works on my laptop (also a mac, same OS).
Any idea as to what the issue might be with his laptop configuration? Or indeed the docker setup.
(The code extract is to make clear the problem with mounting volumes, the same issue presents itself using a compose.yml file.)
Non working code:
docker run -ti -v "$PWD/codebase/realsmart-saml-copy":/var/www/html realsmart-docker_smartlogin bash
No error messages are displayed, but the results are not as expected as the volume does not mount without using sudo.
Try to see if the user is part of the docker group.
It would make sense that sudo works, but not for the local user, if that local user is not part of the docker group.
The solution for anyone interested.
After upgrading to Docker Desktop Boot2Docker has been replaced.
Steps to fix the issue:
docker-machine rm machine-name
unset DOCKER_TLS_VERIFY
unset DOCKER_CERT_PATH
unset DOCKER_MACHINE_NAME
unset DOCKER_HOST
restart Docker Desktop
cd path/to/docker-project.
docker-compose build
docker-compose up (or docker run)
project now available on localhost
Further details: https://docs.docker.com/docker-for-mac/docker-toolbox/
Add your user to the docker group.
sudo usermod -aG docker $USER

Disallow egress from Docker containers on Docker for Mac

I want to disable all outgoing connections that are initiated by docker containers to the outside world. I can do this in linux by adding a rule to the FORWARD chain in linux. How do I do this in Docker for Mac?
I found out that Docker for Mac uses an xhyve vm and that’s where docker0 interface lives. What interface in the host does this connect to? I used nettop on Mac and I see that Docker uses my en0 wireless interface. But, I’m not sure if Docker and xhyve are using the same interface.
Edit: Added docker-for-windows tag because they might have similar solutions (Hoping)
Edit 2: Docker for Mac has changed so the accepted solution changed a bit
Docker
$ docker run --net=host --privileged -ti alpine sh
# apk update && apk add iptables
# iptables -vnL
This and the rules could be turned into a Dockerfile and run with a -- restart option. I think on-failure might work to reapply the rules when Docker for Mac starts up.
Virtual Machine
To get to the linux VM:
mac$ brew install screen
mac$ screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
Since the move to linuxkit, this is not your average linux host, everything's a container:
linuxkit:~# ctr -n services.linuxkit tasks ls
TASK PID STATUS
acpid 925 RUNNING
diagnose 967 RUNNING
host-timesync-daemon 1116 RUNNING
ntpd 1248 RUNNING
vpnkit-forwarder 1350 RUNNING
docker-ce 1011 RUNNING
kubelet 1198 RUNNING
trim-after-delete 1303 RUNNING
vsudd 1398 RUNNING
Use runc to move into the docker-ce (or docker) namespace
linuxkit:~# runc --root /run/containerd/runc/default exec -t docker-ce /bin/sh
docker-ce # iptables -vnL
Note that rules will disappear after a restart of Docker for Mac. I haven't found the secret sauce for persisting system changes yet.
Use ctrl-a then d to exit the screen session otherwise you will bork the terminal.
OSX
For the easy but € option, use Little Snitch and block outbound connections on OSX from com.docker.supervisor via vpnkit.
Try Mac's pfctl command, it's kind of equivalent to iptables.
Here's man page: https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man8/pfctl.8.html

Docker not working in Windows10

I am using Docker version 1.12.2, build bb80604 and VirtualBox 5.1.6.
I was able to install Docker Toolbox correctly but I am not able to start the default VM.
I tried to follow the instruction given here but I am still getting The system cannot find the file specified error. I don't have Hyper-V enabled on system.
F:\>docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
default not found Error open C:\Users\Animesh\.docker\machine\machines\default\config.json: The system cannot find the file specified.
F:\>docker-machine env --shell=cmd
open C:\Users\Animesh\.docker\machine\machines\default\config.json: The system cannot find the file specified.
F:\>docker rm default
An error occurred trying to connect: open //./pipe/docker_engine: The system cannot find the file specified.
How can I solve this issue?
It looks like something bad happened to your docker-machine default machine.
To recover, do the following:
docker-machine rm -f default
Next, open the virtualbox GUI and make sure that there isn't a VM that corresponds with that old "default" VM, just in case. If there is, manually delete it from there.
Now, open the quickstart terminal again and it should create a new "default" virtual machine for you.
Alternatively, you can create it like this:
docker-machine create -d virtualbox --virtualbox-memory 2048 default
At that point, your sequence of commands should work as expected again.
You may need another couple of steps additional to what #programmerq mentioned.
Create empty text file: C:\Users\<user>\.docker\machine\machines\default\config.json
Stop VBoxHeadless.exe service from Windows task manager
Try docker-machine rm -f default and docker-machine create -d virtualbox --virtualbox-memory 2048 default commands afterwards to remove/add default
step 1: run: docker-machine rm -f default - (on your mac terminal or control panel)
step 2: run: docker-machine create -d virtualbox --virtualbox-memory 2048 default
It worked like miracle.
If you are a Windows Home user, you will not be able install the Docker for Windows Desktop edition, as it requires Hyper-V virtualization. This is supported only by Windows Professional and Enterprise editions.
As the Docker for Windows Desktop edition requires Hyper-V virtualization, you will not be able to install it if you are a Windows Home user. It is only supported by Windows Professional and Enterprise editions.
If you have installed Docker Toolbox and you are dealing with errors while trying to start the default machine you can create a new one by running:
docker-machine create box
the in order to see how to connect your Docker Client to the Docker Engine running on the virtual machine created run:
docker-machine env box
the command that you need to run in order to configure your shell will then appear:
eval $("C:\Program Files\Docker Toolbox\docker-machine.exe" env box)
Check that the installation is runnung correctly by running:
docker run hello-world
Then you just need to start your machine every time you need to use docker:
docker-machine start box
and follow the second and third steps above for the configuration.

Keyboard/Mouse are unresponsable when running x-org in a docker container

I want to create a docker image for a GUI application (e.g. Chrome) and I hope this GUI app could run at a bare Linux server without X server installed.
I know it is very easy to create and run a docker image just for X Window Client (The GUI application itself). This needs X server be installed and run at host.
sudo docker run -ti -v /tmp/.X11-unix:/tmp/.X11-unix xorg xterm -display :0
But for me, I need both X client and server run in docker container.
Here's my dockerfile:
FROM ubuntu:14.04
RUN apt-get update && apt-get install -y xorg
And I run the image by command:
sudo docker run -i -t --rm -e DISPLAY=:0 --privileged xorg xinit
The X server could be started and my screen turns black, after a few seconds, the xterm window displays. BUT, I can't use keyboard and mouse. The screen seems like freezen
I have searched and tried many solutions but no one could fix this problem. (the virtual x-server is not I needed)
I have resolved this problem.
At first, I thought maybe x server in docker container cannot access host devices, and I spent much time on LXC/cgroup. For example, I changed the docker exec engine to LXC, and I added option '--lxc-conf='lxc.cgroup.devices.allow = c 13:* rwm', and I also created /dev/input/* in container.
All of these operations are unnecessary.
If we run docker container in privileged mode, all host devices will be added automatically. Or we can use options like '--device=/dev/input/mice' to share host device.
The real problem is that x server could not discovery and add device automatically. I don't know why. But we could modify x server's configuration and customize the device.
add file /etc/X11/xorg.conf.d/10-input.conf
Section "ServerFlags"
Option "AutoAddDevices" "False"
EndSection
Section "ServerLayout"
Identifier "Desktop"
InputDevice "Mouse0" "CorePointer"
InputDevice "Keyboard0" "CoreKeyboard"
EndSection
Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
Option "Device" "/dev/input/event2"
EndSection
Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mice"
Option "ZAxisMapping" "4 5 6 7"
EndSection
and run docker container:
docker run -i -t -v /tmp/.X11-unix:/tmp/.X11-unix --rm --privileged ubuntu startx
At first make sure that proper input modules are installed:
RUN DEBIAN_FRONTEND='noninteractive' apt-get install -y --no-install-recommends xserver-xorg-input-evdev xserver-xorg-input-all
In modern Linux udev is responsible for managing device nodes (including USB keyboards) in the /dev tree. It uses /run/udev/data which isn't available inside your container even with -privileged option.
So you need to mount that folder explicitly using -v /run/udev/data:/run/udev/data like this:
docker run -i -t -v /tmp/.X11-unix:/tmp/.X11-unix --rm --privileged -v /run/udev/data:/run/udev/data ubuntu startx

Resources