Docker (www.docker.io) looks terrific. However, after installing VirtualBox, Vagrant
... and finally Docker on a Mac, I'm finding it's not possible to access the service running in the Docker container from another computer (or from a terminal session on the Mac). The service I'm trying to access is Redis.
The problem appears to be that there's no route to the IP address assigned to the Docker container. In this case the container's IP is 172.16.42.2 while the Mac's IP is 196.168.0.3.
A couple notes:
It IS possible to access it - but only from within the VirtualBox session. This can be done using redis-cli -h 172.16.42.2 -p 6379.
I have added "config.vm.network :bridged" to the VagrantFile in an attempt to get the, but that didn't solve the problem.
The VM generated by vagrant is indeed isolated, in order to access it from your host, you can allocate a private network to it.
Instead of doing config.vm.network :bridged, try config.vm.network :private_network, ip: "192.168.50.4", It should do the trick
However, this will only allow you to access the VM itself, not the containers.
In order to do so, when running the container, you can add the -p option
ex: docker run -d -p 8989 base nc -lkp 8989
This will run a netcat listening on 8989 within a container and expose the port publicly. As it is also run with -d, the container will be in detached mode and the only output will be the container's ID
In order to expose the port, Docker do a simple NAT. In order to know the real port, you can
do docker port <ID of the container> 8989
Netcat will be available from the mac at 192.168.50.4:<result>
I just wrote a tutorial of how to use a host-only network and TCP routing to make this pretty easy. This way you don't have to map every specific port.
http://ispyker.blogspot.com/2014/04/accessing-docker-container-private.html
Important points ...
1) Add host-only network to Virtual Box
2) Tell the boot2docker VM to have an adapter on the host-only network
3) Add an IP for the new boot2docker VM host-only networking adapter
4) Route all Mac OS X traffic for the docker container subnet to that boot2docker VM host-only networking IP
Actual steps are on the blog with output so you can compare to what you see as you follow them.
I have installed tomcat from my Dockerfile and forwarded that to 6060 using vagrant`s port forwarding. These are the steps worked for me:
vagrant provision
vagrant up
vagrant ssh
box_name$ docker run -i -t -p 8080:8080 bsb_tomcat6 /bin/bash
Able to see tomcat up & running on localhost:6060, as I have done port forwarding to 6060 in my Vagrantfile
you also can define PRIVATE_NETWORK and FORWARD_DOCKER_PORTS environment variables to access your services that are running in docker containers:
$ vagrant halt
$ export PRIVATE_NETWORK=192.168.50.4
$ export FORWARD_DOCKER_PORTS=1
$ vagrant up
In my case i can access postgres from Mac using
$ telnet 192.168.50.4 49154
to find out actual application port you can use
$ sudo docker port 1854499c6547 5432
0.0.0.0:49154
Related
I just migrated to using Docker for Mac, from previously using Docker Toolbox with virtualbox for OSX.
I used to get the machine IP address with $(docker-machine ip default).
Is there a reliable way to get the Hyperkit IP address?
Thanks!
In opposition to Docker toolbox, Docker for Windows and Docker for Mac are designed to give you the feeling that Docker is running directly on your OS, so they use lightweight virtual machines running under lightweight hypervisors (instead of VirtualBox) handled directly by the docker executable.
Hence you won't see them with docker-machine and you won't see another IP address than localhost.
Docker for Windows relies on the HyperV hypervisor which allows a network connection to tcp://localhost:2375.
Docker for Mac relies on the xhyve hypervisor, the way it's implemented only provides a connection through the socket unix:///var/run/docker.sock.
Workaround
To provide a TCP connection for Docker for Mac:
Install socat. With brew:
brew install socat
Run this socat command to forward TCP requests to the socket
socat TCP-LISTEN:2375,reuseaddr,fork,bind=localhost UNIX-CONNECT:/var/run/docker.sock
Map what you want on tcp://localhost:2375
Up to you to run the socat command on startup, if necessary.
This was for instance useful to me to associate the Webstorm nodeJS debugger to a nodeJS container (since at the time of writing, docker debugging is supported by Webstorm docker integration plugin, but not through unix sockets).
Documentation on Docker for Mac limitations
https://docs.docker.com/docker-for-mac/networking/#/known-limitations-use-cases-and-workarounds
There is no docker0 bridge on macOS
Because of the way networking is implemented in Docker for Mac, you cannot see a docker0 interface in macOS. This interface is actually within HyperKit.
You could use docker image for socat which starts every time you start 'docker for mac'
docker run -d --restart=always -p 2376:2375 -v
/var/run/docker.sock:/var/run/docker.sock bobrik/socat
TCP4-LISTEN:2375,fork,reuseaddr UNIX-CONNECT:/var/run/docker.sock
Find your docker API ip address:
ifconfig | grep 'inet 192'| awk '{ print $2}'
There's no need for working with the xhyve VM's IP address directly like you would with docker-machine. All port mappings are directly mapped to localhost.
$ docker run -d -p 8080:80 nginx:latest
$ curl localhost:8080
Also see the official documentation:
When you run a container with the -p argument, for example: $ docker run -p 80:80 -d nginx Docker for Mac will make the container port available at localhost.
My current solution is to create the containers using Docker Machine (A linux VM which is available under another IP address) and route all the traffic of the containers to the docker machine VM.
sudo route -n add -net 172.18.0.0/16 192.168.99.100
You can get the network range of your docker containers using docker inspect and the IP address of your docker machine VM using docker-machine ip
Another workaround is to use sudo ifconfig lo0 alias 172.17.0.1 so you can still use the same static IP address (if your Linux-based colleagues or bash scripts insist on using that).
So I'm using Docker Toolbox because I don't have Hyper-V on my machine since it's not Windows 10 pro. Everything seems to work fine, but when I try to go on my browser 0.0.0.0:80 it always returns me: This site can’t be reached
But when I run the command: docker container ps I get the following: 0.0.0.0:80->80/tcp meaning that this address should work. I searched across stackoverflow and github issues. Now I'm stuck.
Am I missing something?
Thanks,
Mark
EDIT:
Using docker-machine ip default returns me 192.168.99.100. I run that on port 80. I still get the same result except that the address becomes the container id: https://fd677edg12
I run that command on cmd to find my ipv4: cmd /k ipconfig /all. Put the result with the port and it returns the same thing: https://fd677edg12
Docker Toolbox doesn't get as many conveniences as Docker for Windows, but you're correct in using it since you're on Home edition.
In Toolbox, nothing will be localhost, and will be 192.168.99.100 by default, since it's running a Linux VM in VirtualBox.
So if you run docker run -p 80:80 nginx
(notice I had to publish a port for 192.168.99.100 to listen on that port)
Then going to http://192.168.99.100 should work.
I initially had a few issues with accessing my Applications at localhost:8080 while using DockerToolBox and OracleVM VirtualBox.
In VirtualBox:
Click the appropriate machine (probably the one labeled "default")
Settings
Network > Adapter 1 > Advanced > Port Forwarding
Click "+" to add a new Rule
Set Host Port 8080 & Guest Port 8080; be sure to leave Host IP and Guest IP empty
Run the command:
docker run -p 8080:8080 ${image_id}
I was following docker for windows tutorial in https://docs.docker.com/docker-for-windows/#set-up-tab-completion-in-powershell and got stuck in step #6 when test nginx in the web browser. Seems I faced a similar problem since I also use Windows Home and don't have Hyper-V. My workaround is quite simple:
check your docker IP default
$ docker-machine ip default
192.168.99.100
Go to Oracle Virtual Machine to set for port forwarding. Make sure the network setting is NAT, and add port forwarding.
Host IP: 127.0.0.1, Guest IP: 192.168.99.100, port all set to 80
like this
Try again to your browser and run http://localhost or http://127.0.0.1 (can add the port 80 also). It should run.
The thing is that the nginx IP is meant to be accessible within the docker Virtual Machine, so that we need that port forwading setting in order to access it directly in the host machine's browser
You can use localhost instead of '192.168.99.100' by following the instructions:
Step #01:
docker-machine ip default
You will see the default IP
Step #02:
docker-machine stop default
Step #03:
Open VirtualBox Manager (from the start programs in windows search for VirtualBox Manager)
Select your Docker Machine VirtualBox image (e.g.: default)
Open Settings -> Network -> Advanced -> Port Forwarding
Add your app name, the desired host port and your guest port
i.e, app name : nginx, host: 127.0.0.1, host port: 80, guest port: 80
Step #04:
Now you’re ready to start your Docker Machine by executing the following:
docker-machine start default
Then just start your Docker container and you will be able to access it via localhost.
Have a look here for details.
To map the ports expected to localhost instead of hitting the docker-machine IP directly, you can use the VirtualBox CLI.
If the docker-machine VM (here called default) is running, add and delete rules like this:
> VBoxManage.exe controlvm "default" natpf1 "nginx,tcp,,8888,,8888"
> VBoxManage.exe controlvm "default" natpf1 delete nginx
If the VM is not running, or you want to stop before altering it:
> docker-machine stop
> VBoxManage.exe modifyvm "default" --natpf1 "nginx,tcp,,8888,,8888"
> VBoxManage.exe modifyvm "default" --natpf1 delete "nginx"
> docker-machine start
Where the format of the port forwarding rule is [<name>],tcp|udp,[<hostip>],<hostport>,[<guestip>], <guestport>.
Note that in VirtualBox, you want to map to the host port of Docker map, not the internal container port. You're mapping host -> VM, then Docker maps VM -> container.
See the VirtualBox docs.
This is another easy way to avoid typing the ip 192.168.99.100.
Go to C:\Windows\System32\drivers\etc\hosts and add at the end of the file:
192.168.99.100 docker.awesome or any name of your liking.
Save the file (You need to have admin rights so make sure you right click on the file and run as administrator to be able to save it when you edit it).
Go to your chosen domain name, docker.awesome:8080 in this case and there you have it.
After lot of trials, I was able to get this bulletin board.
The docker run command I used - docker run -p 4680:8080 --name bb bulletinboard:1.0
Here, 4680 is localhost port number.
8080 is container port number, the port at which the container will be listening. This port number is mentioned in the EXPOSE command in the Dockerfile.
Then, go to web-browser and type 192.168.99.100:4680
Here, 192.168.99.100 is the docker machine IP address (use command -> docker-machine ip)
After this, your browser page should open to -
Hope this helps you all!!
Using ubuntu tusty, there is a service running on a remote machine, that I can access via port forwarding through an ssh tunnel from localhost:9999.
I have a docker container running. I need to access that remote service via the host's tunnel, from within the container.
I tried tunneling from the container to the host with -L 9000:host-ip:9999 , then accessing the service through 127.0.0.1:9000 from within the container fails to connect. To check wether the port mapping was on, I tried
nc -luv -p 9999 # at host
nc -luv -p 9000 # at container
following this, parag. 2 but there was no perceived communication, even when doing
nc -luv host-ip -p 9000
at the container
I also tried mapping the ports via docker run -p 9999:9000 , but this reports that the bind failed because the host port is already in use (from the host tunnel to the remote machine, presumably).
So my questions are
1 - How will I achieve the connection? Do I need to setup an ssh tunnel to the host, or can this be achieved with the docker port mapping alone?
2 - What's a quick way to test that the connection is up? Via bash, preferably.
Thanks.
Using your hosts network as network for your containers via --net=host or in docker-compose via network_mode: host is one option but this has the unwanted side effect that (a) you now expose the container ports in your host system and (b) that you cannot connect to those containers anymore that are not mapped to your host network.
In your case, a quick and cleaner solution would be to make your ssh tunnel "available" to your docker containers (e.g. by binding ssh to the docker0 bridge) instead of exposing your docker containers in your host environment (as suggested in the accepted answer).
Setting up the tunnel:
For this to work, retrieve the ip your docker0 bridge is using via:
ifconfig
you will see something like this:
docker0 Link encap:Ethernet HWaddr 03:41:4a:26:b7:31
inet addr:172.17.0.1 Bcast:172.17.255.255 Mask:255.255.0.0
Now you need to tell ssh to bind to this ip to listen for traffic directed towards port 9000 via
ssh -L 172.17.0.1:9000:host-ip:9999
Without setting the bind_address, :9000 would only be available to your host's loopback interface and not per se to your docker containers.
Side note: You could also bind your tunnel to 0.0.0.0, which will make ssh listen to all interfaces.
Setting up your application:
In your containerized application use the same docker0 ip to connect to the server: 172.17.0.1:9000. Now traffic being routed through your docker0 bridge will also reach your ssh tunnel :)
For example, if you have a "DOT.NET Core" application that needs to connect to a remote db located at :9000, your "ConnectionString" would contain "server=172.17.0.1,9000;.
Forwarding multiple connections:
When dealing with multiple outgoing connections (e.g. a docker container needs to connect to multiple remote DB's via tunnel), several valid techniques exist but an easy and straightforward way is to simply create multiple tunnels listening to traffic arriving at different docker0 bridge ports.
Within your ssh tunnel command (ssh -L [bind_address:]port:host:hostport] [user#]hostname), the port part of the bind_address does not have to match the hostport of the host and, therefore, can be freely chosen by you. So within your docker containers just channel the traffic to different ports of your docker0 bridge and then create several ssh tunnel commands (one for each port you are listening to) that intercept data at these ports and then forward it to the different hosts and hostports of your choice.
on MacOS (tested in v19.03.2),
1) create a tunnel on host
ssh -i key.pem username#jump_server -L 3336:mysql_host:3306 -N
2) from container, you can use host.docker.internal or docker.for.mac.localhost or docker.for.mac.host.internal to reference host.
example,
mysql -h host.docker.internal -P 3336 -u admin -p
note from docker-for-mac official doc
I WANT TO CONNECT FROM A CONTAINER TO A SERVICE ON THE HOST
The host has a changing IP address (or none if you have no network access).
From 18.03 onwards our recommendation is to connect to the special DNS
name host.docker.internal, which resolves to the internal IP address
used by the host. This is for development purpose and will not work in
a production environment outside of Docker Desktop for Mac.
The gateway is also reachable as gateway.docker.internal.
I think you can do it by adding --net=host to your docker run. But see also this question: Forward host port to docker container
I'd like to share my solution to this. My case was as follows: I had a PostgreSQL SSH tunnel on my host and I needed one of my containers from the stack to connect to a database through it.
I spent hours trying to find a solution (Ubuntu + Docker 19.03) and I failed. Instead of doing voodoo magic with iptables, doing modifications to the settings of the Docker engine itself I came up with a solution and was shocked I didn't thought of this earlier. The most important thing was I didn't want to use the host mode: security first.
Instead of trying to allow a container to talk to the host, I simply added another service to the stack, which would create the tunnel, so other containers could talk to easily without any hacks.
After configuring a host inside my ~/.ssh/config:
Host project-postgres-tunnel
HostName remote.server.host
User sshuser
Port 2200
ForwardAgent yes
TCPKeepAlive yes
ConnectTimeout 5
ServerAliveCountMax 10
ServerAliveInterval 15
And adding a service to the stack:
postgres:
image: cagataygurturk/docker-ssh-tunnel:0.0.1
volumes:
- $HOME/.ssh:/root/ssh:ro
environment:
TUNNEL_HOST: project-postgres-tunnel
REMOTE_HOST: localhost
LOCAL_PORT: 5432
REMOTE_PORT: 5432
# uncomment if you wish to access the tunnel on the host
#ports:
# - 5432:5432
The PHP container started talking through the tunnel without any problems:
postgresql://user:password#postgres/db?serverVersion=11&charset=utf8
Just remember to put your public key inside that host if you haven't already:
ssh-copy-id project-postgres-tunnel
I'm pretty sure this will work regardless of the OS used (MacOS / Linux).
I agree with #hlobit that #B12Toaster answer should be the accepted answer.
In case anyone hits this problem but with a slightly different setup with the SSH tunnel, here are my findings. In my case, instead of creating a tunnel from Docker host machine to remote machine using ssh -L, I was creating remote forward SSH tunnel from remote machine to Docker host machine using ssh -L.
In this setup, by default sshd does NOT allow gateway ports, i.e. in file /etc/ssh/sshd_config on Docker host, the GatewayPorts no should be uncommented and set to GatewayPorts yes or GatewayPorts clientspecified. I configured GatewayPorts clientspecified and configured the remote forward SSH tunnel by ssh -L 172.17.0.1:dockerHostPort:localhost:sshClientPort user#dockerHost. Remember to restart sshd after changing /etc/ssh/sshd_config (sudo systemctl restart sshd).
Your Docker container should be able to connect to Docker host on 172.17.0.1:dockerHostPort and this in turn gets tunnelled back to SSH client's localhost:sshClientPort.
References:
https://www.ssh.com/ssh/tunneling/example
https://docs.docker.com/network/network-tutorial-host/
https://docs.docker.com/network/host/
My 2 cents for Ubuntu 18.04 - a very simple answer, no need for extra tunnels, extra containers, extra docker options or exposing host.
Simply, when creating a reverse tunnel make sure ssh binds to all interfaces as, by default, it binds ports of the reverse tunnel to localhost only. For example, in putty make sure that option Connection->SSH->Tunnels Remote ports do the same (SSH-2 only) is ticked.
This is more or less equivalent to specifying the binding address 0.0.0.0 for the remote part of the tunnel (more details here):
-R [bind_address:]port:host:hostport
However, this did not work for me unless I allowed the GatewayPorts option in my sshd server configuration. Many thanks to Stefan Seidel for his great answer.
In short: (1) you bind the reverse tunnel to 0.0.0.0, (2) you let the sshd server to accept such tunnels.
Once this is done I can access my remote server from my docker containers via the docker gateway 172.17.0.1 and port bound to the host.
On my side, running Docker in Windows Subsystem for Linux (WSL v1), I couldn't use docker0 connection approach. host.docker.internal also doesn't resolve (latest docker version).
However, I found out I could directly use the host-ip insider my docker container.
Get your Host IP (Windows cmd: ipconfig), e.g. 192.168.0.5
Bash into your Container and test if you can ping your host ip:
- docker exec -it d6b4be5b20f7 /bin/bash
- apt-get update && apt-get install iputils-ping
- ping 192.168.0.5
PING 192.168.0.5 (192.168.0.5) 56(84) bytes of data.
64 bytes from 192.168.0.5 : icmp_seq=1 ttl=37 time=2.17 ms
64 bytes from 192.168.0.5 : icmp_seq=2 ttl=37 time=1.44 ms
64 bytes from 192.168.0.5 : icmp_seq=3 ttl=37 time=1.68 ms
Apparently, in Windows, you can directly connect from within containers to the host using the official host ip.
In case anyone needs it (like I did), solution for Windows and WSL is same as #prayagupd mentioned for Mac OS
Create an SSH tunnel to your remote service with whatever tool you prefer to whatever port you prefer, for example 3300.
Then, from Docker container you can connect to, for example, MySQL DB on tunnel port 3300 using following command:
mysql -u user -p -h host.docker.internal -P 3300
An easy example to reproduce the situation and ssh to host
Run a container. Use --network="host
docker container run --network="host" --interactive --tty --rm ubuntu bash
Now you can access your host using localhost
Now your host machine is a Linux machine that has a public-private key file to ssh into it. So copy the contents of your private key file and reproduce the key file inside your host. (However, this is just a demonstration. This is not a good way to copy key files)
Now ssh into your host. Use localhost to access it.
ssh -i key_file.pem ec2-user#localhost
I'm running docker on top on vagrant and would like to debug application remotely using pycharm running on windows (which runs vagrant). Of course the docker host is then on vagrant - not the same machine pycharm is running on.
I have to specify the certificates folder and docker machine executable as a local files / directories. Does this mean I cannot debug applications using pycharm in this setup?
Of course I could ssh directly into the docker container but then I have no features pycharm gives me.
pycharm cannot remote debug because cannot connect with code in docker in vagrant
you need bridge port from docker with vagrant before this.
you need find vagrant ip and docker ip (by default, vagrant ip: 10.0.2.2, you can see when run vagrant ssh)
second determine port for debug( exam 21000)
use commant code in terminal
vagrant ssh
sudo iptables -t nat -A PREROUTING -p tcp --dport 21000 -j DNAT --to-destination 10.0.2.2:21000
sudo iptables -t nat -A POSTROUTING -j MASQUERADE
set code for python file:
change 172.19.0.1 with your docker ip (in vagrant)
import pydevd
pydevd.settrace('172.19.0.1', port=21000, suspend=False)
set on breakpoint on code and try to debug
It is possible however not recommended, it has the potential to introduce a number of problem spots longer term and brings a increased security risk.
as per the docker documentation ...
If you are okay with the security risk and if docker toolbox using boot2docker is not an option for your situation, then you will need to ensure:
Docker client/server versions are identical
Port forwarding on your local vagrant box is setup
Add the TCP binding for the docker server, either as a replacement to the default unix socket binding and/or in addition.
I'm developing locally with Docker on OSX using the latest Docker toolkit. I have a node server running in a docker container, bound to port 9999 of the VM. I can hit this server from a browser on my mac, and I would like to hit from another device on the same network. Is there a way to bind the VM to the machine's IP address? Or otherwise expose it?
I just figured this one out today! I am using docker-machine with virtualbox on Mac OS 10.10.5. The first thing I tried was to change the network interface from NAT to Bridged. This just breaks docker-machine's ability to communicate with the VM.
Instead I ADDED another network adapter running in bridged mode.
After starting the docker-machine I get this:
$ docker-machine ip redis-test
10.222.11.242
That is a local network address accessible from anyone else in my office or on my VPN.
Then if I run something like:
$ docker run -p 6379:6379 -d redis
I get a containerized redis service running on port 6379 of the 10.222.11.242 address.
So I can do this from anywhere else on the network:
$ telnet 10.222.11.242 6379
Trying 10.222.11.242...
Connected to 10.222.11.242.
Escape character is '^]'.
info
$1827
# Server
redis_version:2.8.19
...
And as a cool bonus of this we can remap the ports like this:
$ docker run -p 8080:6379 -d redis
e7cc53d9c157a658041c3bee5967dd3678b4d35e6146a02220a87bfebfc919ad
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e7cc53d9c157 redis "/entrypoint.sh redis" 7 seconds ago Up 6 seconds 0.0.0.0:8080->6379/tcp goofy_yonath
bf1dc6c7c6b5 redis "/entrypoint.sh redis" 51 minutes ago Up 51 minutes 0.0.0.0:6379->6379/tcp redis
Now I have two redis instances listening on different ports (6379 and 8080) of the same IP.
EDIT: Here are some details to help those confused about adding a NIC to the VM in VirtualBox. I have only used VirtualBox for this and cannot advise about other virtualization system configurations.
Stop the VM by selecting it in the VM Manager and using the right-click menu or pressing 'command-F'.
Click "Settings".
Click "Network".
Select one of the Adapters that is not currently enabled.
Enable it.
Select "Bridged Adapter" in the "Attached to" selection.
Click OK.
Start your VM and try it out.
NOTE: I am sure there are some clever command line options for doing this setup, but since I only ever needed to set it once I have never bothered to automate it.
You could setup port forwarding on the VirtualBox NAT adaptor.
Bridging it to the local network (in the answer above) is not the same as using the OSX IP address. Bridging can sometimes cause extra headaches if you are on laptop and move to different internet connections. The VM may not automatically pull a new IP from the new network, etc.
In the UI go to Settings --> Network --> Port Forwarding or from the commandline something like this:
VBoxManage controlvm "default" natpf1 "tcp-port9999,tcp,,9999,,9999";
where "default" is the name of the VM ("default" is normally used for docker-machine) and 9999 is the port you want to map.
More info at: https://github.com/boot2docker/boot2docker/blob/master/doc/WORKAROUNDS.md
https://www.virtualbox.org/manual/ch06.html (Configuring Port forwarding with NAT)
Did you put the expose command in Dockerfile?
EXPOSE 9999
Based on #e.thompsy answer, here is the command line version.
First, figure out the interface name you need using:
vboxmanage list bridgedifs
For me it was en1.
Then:
docker-machine create ...
docker-machine stop $VM
vboxmanage modifyvm "$VM" --nic3 bridged --bridgeadapter3 en1
docker-machine start $VM