We have a Jenkins server which is accessible only from within the VPC on the cloud. On Mac and Linux I use sshuttle to make a ssh connection to the bastion instance (to act a proxy) and open the Jenkins console in the browser. Everything works fine.
Now I'm on Windows and trying to do the same on WSL2. If I'm not mistaken previously, sshuttle didn't work on WSL1 (failed with some error message), but I managed to run it on WSL2 without any issue. The ssh connection is established and I can access my Jenkins (using curl).
Then I tried to access my Jenkins on Windows via WSL2:
1. I found the IP address of WSL2 and the port the ssh tunnle:
# lsof -i -n | grep ssh
sshuttle 1234 rad 5u IPv4 39270 0t0 TCP *:socks (LISTEN)
ssh 5678 rad 3u IPv4 40252 0t0 TCP 172.25.236.84:57578->bastion:ssh (ESTABLISHED)
2. I configured network proxy setting of Firefox (v77) to use my ssh tunnle:
Manual proxy configuration
SOCK host: 172.25.236.84
Port: 1080
SOCKS V5 (tested with V4 as well)
But loading the page fails with "The connection was reset" error on Firefox. I tested via Powershell that the SOCKS port is open and responding (using Test-NetConnection).
1. Any idea what the problem is? How to make it work?
2. If it's not gonna work, is there any other solution (e.g. Docker, etc)?
Thanks.
I'm not sure, but my guess is that sshuttle doesn't actually act as a SOCKS proxy and that's why the connection gets reset.
I managed to access my Jenkins on Windows machine using ssh SOCKS proxy: ssh -D 0.0.0.0:1080 rad#bastion and configured Firefox to use the SOCKS proxy.
Interestingly, for this you don't even need WSL. It seems Windows 10 has OpenSSH and you can use it. Just open CMD and type ssh -D 1080 rad#bastion and setup Firefox to use localhost as the proxy.
If there's any better solution or any comment/concern (apart from DNS over SOCKS) with this approach, please share.
Thanks.
As alternative on WSL(2) you can run a regular SSH tunnel.
Eg:
ssh -N -L 127.0.0.1:5432:some_domain_to_forward:5432 user#jumpbox_ip
and then just connect to 127.0.0.1:5432
Related
I am having trouble setting up an ssh tunnel on my mac machine. I have no problems setting up the tunnel on my ubuntu box. This is the command I run
ssh -nNT -L 172.18.0.1:4000:production-database-url:3306 jump-point
When I run this on my mac, I get the following error:
bind [172.18.0.1]:4000: Can't assign requested address
channel_setup_fwd_listener_tcpip: cannot listen to port: 4000 Could
not request local forwarding.
If I run without the bind_address (172.18.0.1), I am able to connect to the database via the tunnel.
If I bind to all interfaces (0.0.0.0), then tunnel is open, however, the connection to the database from inside the docker container does not work.
172.18.0.1 is the IP of docker's default bridge network gateway, not your host's IP.
You can run this command to check that.
$ docker network inspect bridge
Docker for Mac has limitations
There is no docker0 bridge on macOS (it's in the docker VM host on Mac and on Windows)
You cannot ping containers (without shaving a bunch of yaks)
Per-container IP addressing is not possible
Also note that this means the docker run option --net-host is not supported on Mac, but maybe that's a good thing
There is a workaround
These magic addresses resolve to the host's IP from within a container
docker.for.mac.localhost (deprecated)
docker.for.mac.host.internal (deprecated)
host.docker.internal
This resolves to the gateway of the host mac
gateway.docker.internal
Use the name host.docker.internal from within the container just like you would use localhost on the mac directly.
Don't worry about the bind address for the tunnel:
ssh -nNT -L 4000:production-database-url:3306 jump-point
You didn't mention which database but I take it from the port 3306 that it is MySQL.
To connect using the mysql cli from within a container, via an ssh tunnel on your host, to a remote mysql database server you can run:
mysql --host host.docker.internal [... other options go here]
Using ubuntu tusty, there is a service running on a remote machine, that I can access via port forwarding through an ssh tunnel from localhost:9999.
I have a docker container running. I need to access that remote service via the host's tunnel, from within the container.
I tried tunneling from the container to the host with -L 9000:host-ip:9999 , then accessing the service through 127.0.0.1:9000 from within the container fails to connect. To check wether the port mapping was on, I tried
nc -luv -p 9999 # at host
nc -luv -p 9000 # at container
following this, parag. 2 but there was no perceived communication, even when doing
nc -luv host-ip -p 9000
at the container
I also tried mapping the ports via docker run -p 9999:9000 , but this reports that the bind failed because the host port is already in use (from the host tunnel to the remote machine, presumably).
So my questions are
1 - How will I achieve the connection? Do I need to setup an ssh tunnel to the host, or can this be achieved with the docker port mapping alone?
2 - What's a quick way to test that the connection is up? Via bash, preferably.
Thanks.
Using your hosts network as network for your containers via --net=host or in docker-compose via network_mode: host is one option but this has the unwanted side effect that (a) you now expose the container ports in your host system and (b) that you cannot connect to those containers anymore that are not mapped to your host network.
In your case, a quick and cleaner solution would be to make your ssh tunnel "available" to your docker containers (e.g. by binding ssh to the docker0 bridge) instead of exposing your docker containers in your host environment (as suggested in the accepted answer).
Setting up the tunnel:
For this to work, retrieve the ip your docker0 bridge is using via:
ifconfig
you will see something like this:
docker0 Link encap:Ethernet HWaddr 03:41:4a:26:b7:31
inet addr:172.17.0.1 Bcast:172.17.255.255 Mask:255.255.0.0
Now you need to tell ssh to bind to this ip to listen for traffic directed towards port 9000 via
ssh -L 172.17.0.1:9000:host-ip:9999
Without setting the bind_address, :9000 would only be available to your host's loopback interface and not per se to your docker containers.
Side note: You could also bind your tunnel to 0.0.0.0, which will make ssh listen to all interfaces.
Setting up your application:
In your containerized application use the same docker0 ip to connect to the server: 172.17.0.1:9000. Now traffic being routed through your docker0 bridge will also reach your ssh tunnel :)
For example, if you have a "DOT.NET Core" application that needs to connect to a remote db located at :9000, your "ConnectionString" would contain "server=172.17.0.1,9000;.
Forwarding multiple connections:
When dealing with multiple outgoing connections (e.g. a docker container needs to connect to multiple remote DB's via tunnel), several valid techniques exist but an easy and straightforward way is to simply create multiple tunnels listening to traffic arriving at different docker0 bridge ports.
Within your ssh tunnel command (ssh -L [bind_address:]port:host:hostport] [user#]hostname), the port part of the bind_address does not have to match the hostport of the host and, therefore, can be freely chosen by you. So within your docker containers just channel the traffic to different ports of your docker0 bridge and then create several ssh tunnel commands (one for each port you are listening to) that intercept data at these ports and then forward it to the different hosts and hostports of your choice.
on MacOS (tested in v19.03.2),
1) create a tunnel on host
ssh -i key.pem username#jump_server -L 3336:mysql_host:3306 -N
2) from container, you can use host.docker.internal or docker.for.mac.localhost or docker.for.mac.host.internal to reference host.
example,
mysql -h host.docker.internal -P 3336 -u admin -p
note from docker-for-mac official doc
I WANT TO CONNECT FROM A CONTAINER TO A SERVICE ON THE HOST
The host has a changing IP address (or none if you have no network access).
From 18.03 onwards our recommendation is to connect to the special DNS
name host.docker.internal, which resolves to the internal IP address
used by the host. This is for development purpose and will not work in
a production environment outside of Docker Desktop for Mac.
The gateway is also reachable as gateway.docker.internal.
I think you can do it by adding --net=host to your docker run. But see also this question: Forward host port to docker container
I'd like to share my solution to this. My case was as follows: I had a PostgreSQL SSH tunnel on my host and I needed one of my containers from the stack to connect to a database through it.
I spent hours trying to find a solution (Ubuntu + Docker 19.03) and I failed. Instead of doing voodoo magic with iptables, doing modifications to the settings of the Docker engine itself I came up with a solution and was shocked I didn't thought of this earlier. The most important thing was I didn't want to use the host mode: security first.
Instead of trying to allow a container to talk to the host, I simply added another service to the stack, which would create the tunnel, so other containers could talk to easily without any hacks.
After configuring a host inside my ~/.ssh/config:
Host project-postgres-tunnel
HostName remote.server.host
User sshuser
Port 2200
ForwardAgent yes
TCPKeepAlive yes
ConnectTimeout 5
ServerAliveCountMax 10
ServerAliveInterval 15
And adding a service to the stack:
postgres:
image: cagataygurturk/docker-ssh-tunnel:0.0.1
volumes:
- $HOME/.ssh:/root/ssh:ro
environment:
TUNNEL_HOST: project-postgres-tunnel
REMOTE_HOST: localhost
LOCAL_PORT: 5432
REMOTE_PORT: 5432
# uncomment if you wish to access the tunnel on the host
#ports:
# - 5432:5432
The PHP container started talking through the tunnel without any problems:
postgresql://user:password#postgres/db?serverVersion=11&charset=utf8
Just remember to put your public key inside that host if you haven't already:
ssh-copy-id project-postgres-tunnel
I'm pretty sure this will work regardless of the OS used (MacOS / Linux).
I agree with #hlobit that #B12Toaster answer should be the accepted answer.
In case anyone hits this problem but with a slightly different setup with the SSH tunnel, here are my findings. In my case, instead of creating a tunnel from Docker host machine to remote machine using ssh -L, I was creating remote forward SSH tunnel from remote machine to Docker host machine using ssh -L.
In this setup, by default sshd does NOT allow gateway ports, i.e. in file /etc/ssh/sshd_config on Docker host, the GatewayPorts no should be uncommented and set to GatewayPorts yes or GatewayPorts clientspecified. I configured GatewayPorts clientspecified and configured the remote forward SSH tunnel by ssh -L 172.17.0.1:dockerHostPort:localhost:sshClientPort user#dockerHost. Remember to restart sshd after changing /etc/ssh/sshd_config (sudo systemctl restart sshd).
Your Docker container should be able to connect to Docker host on 172.17.0.1:dockerHostPort and this in turn gets tunnelled back to SSH client's localhost:sshClientPort.
References:
https://www.ssh.com/ssh/tunneling/example
https://docs.docker.com/network/network-tutorial-host/
https://docs.docker.com/network/host/
My 2 cents for Ubuntu 18.04 - a very simple answer, no need for extra tunnels, extra containers, extra docker options or exposing host.
Simply, when creating a reverse tunnel make sure ssh binds to all interfaces as, by default, it binds ports of the reverse tunnel to localhost only. For example, in putty make sure that option Connection->SSH->Tunnels Remote ports do the same (SSH-2 only) is ticked.
This is more or less equivalent to specifying the binding address 0.0.0.0 for the remote part of the tunnel (more details here):
-R [bind_address:]port:host:hostport
However, this did not work for me unless I allowed the GatewayPorts option in my sshd server configuration. Many thanks to Stefan Seidel for his great answer.
In short: (1) you bind the reverse tunnel to 0.0.0.0, (2) you let the sshd server to accept such tunnels.
Once this is done I can access my remote server from my docker containers via the docker gateway 172.17.0.1 and port bound to the host.
On my side, running Docker in Windows Subsystem for Linux (WSL v1), I couldn't use docker0 connection approach. host.docker.internal also doesn't resolve (latest docker version).
However, I found out I could directly use the host-ip insider my docker container.
Get your Host IP (Windows cmd: ipconfig), e.g. 192.168.0.5
Bash into your Container and test if you can ping your host ip:
- docker exec -it d6b4be5b20f7 /bin/bash
- apt-get update && apt-get install iputils-ping
- ping 192.168.0.5
PING 192.168.0.5 (192.168.0.5) 56(84) bytes of data.
64 bytes from 192.168.0.5 : icmp_seq=1 ttl=37 time=2.17 ms
64 bytes from 192.168.0.5 : icmp_seq=2 ttl=37 time=1.44 ms
64 bytes from 192.168.0.5 : icmp_seq=3 ttl=37 time=1.68 ms
Apparently, in Windows, you can directly connect from within containers to the host using the official host ip.
In case anyone needs it (like I did), solution for Windows and WSL is same as #prayagupd mentioned for Mac OS
Create an SSH tunnel to your remote service with whatever tool you prefer to whatever port you prefer, for example 3300.
Then, from Docker container you can connect to, for example, MySQL DB on tunnel port 3300 using following command:
mysql -u user -p -h host.docker.internal -P 3300
An easy example to reproduce the situation and ssh to host
Run a container. Use --network="host
docker container run --network="host" --interactive --tty --rm ubuntu bash
Now you can access your host using localhost
Now your host machine is a Linux machine that has a public-private key file to ssh into it. So copy the contents of your private key file and reproduce the key file inside your host. (However, this is just a demonstration. This is not a good way to copy key files)
Now ssh into your host. Use localhost to access it.
ssh -i key_file.pem ec2-user#localhost
I'm using Boot2Docker to manage Docker containers in Windows and I have a container running an IBM Liberty server (I guess is the same for any other server), I can access the server home screen in the host machine using only the ip (which I get using the command boot2docker ip), but if I try to access the server using the https port, like this xx.xx.xx.xx:9443 the connection fails.
I tried forwarding the port in VirtualBox like this:
And then access the server using the ports 1000 or 1001, but it fails too.
Am I missing something?
BTW, I'm using default NAT connection.
https uses port 443 (not 9443) by default.
Make both "Host Port" and "Guest Port" 443 and try again.
My lab runs RStudio on a server. A couple weeks ago, from my cousin's house, I successfully ssh'd into the server and pulled up the server-side RStudio through my local Firefox browser. Now when I try to access the server RStudio from home (via my own router), it doesn't work. I need help troubleshooting, and I'm guessing it's some problem on the router. I'm running Mac OSX 10.6.8. No idea what the university server's running, but I don't think it's a server-side problem.
Here's how it worked the first time I did it, at my cousin's house: first, I VPN into the university network; then I call SSH with port forwarding; then I open a Firefox browser, connect to my localhost port, and it opens up RStudio on the server side which I can access through my local browser window.
Here's the problem I'm having right now when I try to log-in from my home network:
I can make the VPN connection successfully. I can also set up SSH successfully with this command:
ssh -v -L 8783:localhost:8783 myacct#server.com
Here are the last several lines of the verbose output from the successful ssh command:
debug1: Authentication succeeded (password).
debug1: Local connections to LOCALHOST:8783 forwarded to remote address localhost:8783
debug1: Local forwarding listening on 127.0.0.1 port 8783.
debug1: channel 0: new [port listener]
debug1: Local forwarding listening on ::1 port 8783.
debug1: channel 1: new [port listener]
debug1: channel 2: new [client-session]
debug1: Entering interactive session.
Last login: Mon Sep 2 04:02:40 2013 from vpnipaddress
So I think I'm still succeeding at the VPN and SSH stage (though I don't know why it says my last login was Sep 2 when I've logged in a few times since then).
Next, I open Firefox, and I type localhost:8783, and instead of getting an RStudio server app through my browser window, I get the following errors:
In the Firefox browser window, it says: Server not found, Firefox can't find the server at www.localhost.com, Check the address for typing errors etc.
In the terminal window, it says:
debug1: Connection to port 8783 forwarding to localhost port 8783 requested.
debug1: channel 3: new [direct-tcpip]
channel 3: open failed: connect failed: Connection refused
debug1: channel 3: free: direct-tcpip: listening port 8783 for localhost port 8783, connect from 127.0.0.1 port 50420, nchannels 4
I'm not sure what I've got wrong. I haven't changed anything on my laptop since my last successful connection. I'm on my own router (instead of my cousin's), so maybe I need to mess with the firewall? I already allowed ports 22 and 8783 to come through the firewall to my laptop (I'm not even sure I needed to do that though). Help?
ssh -v -L 8783:localhost:8783 myacct#server.com
...
channel 3: open failed: connect failed: Connection refused
When you connect to port 8783 on your local system, that connection is tunneled through your ssh link to the ssh server on server.com. From there, the ssh server makes TCP connection to localhost port 8783 and relays data between the tunneled connection and the connection to target of the tunnel.
The "connection refused" error is coming from the ssh server on server.com when it tries to make the TCP connection to the target of the tunnel. "Connection refused" means that a connection attempt was rejected. The simplest explanation for the rejection is that, on server.com, there's nothing listening for connections on localhost port 8783. In other words, the server software that you were trying to tunnel to isn't running, or else it is running but it's not listening on that port.
Posting this to help someone.
Symptom:
channel 2: open failed: connect failed: Connection refused
debug1: channel 2: free: direct-tcpip:
listening port 8890 for 169.254.76.1 port 8890,
connect from ::1 port 52337 to ::1 port 8890, nchannels 8
My scenario; i had to use the remote server as a bastion host to connect elsewhere. Final Destination/Target: 169.254.76.1, port 8890. Through intermediary server with public ip: ec2-54-162-180-7.compute-1.amazonaws.com
SSH local port forwarding command:
ssh -i ~/keys/dev.tst -vnNT -L :8890:169.254.76.1:8890
glue#ec2-54-162-180-7.compute-1.amazonaws.com
What the problem was:
There was no service bound on port 8890 in the target host. i had forgotten to start the service.
How did i trouble shoot:
SSH into bastion host and then do curl.
Hope this helps.
Note: localhost is the hostname for an address using the local (loopback) network interface, and 127.0.0.1 is its IP in the IPv4 network standard (it's ::1 in IPv6). 0.0.0.0 is the IPv4 standard "current network" IP address.
I experienced this error with a Docker setup. I had a Docker container running on an external server, and I'd (correctly) mapped its ports out as 127.0.0.1:9232:9232. By port-forwarding ssh remote -L 9232:127.0.0.1:9232, I'd expected to be able to communicate with the remote server's port 9232 as if it were my own local port.
It turned out that the Docker container was internally running its process on 127.0.0.1:9232 rather than 0.0.0.0:9232, and so even though I'd specified the container's port-mappings correctly, they weren't on the correct interface for being mapped out.
In my case, it worked after running the vncserver on linux.
Entered this on linux command line : sudo ssh -L 5901:localhost:5901 -i <ssh_private_key> <username>#<public-IP-address>
Type there vncserver
Go to VncViewer application and connect using localhost:5901
I used to meet the similar problem because 'localhost' was not available on server when it restarted network service, e.g. 'ifdown -a' but followed by only 'ifup -eo1'. Besides server is not listening to the port, you can also check 'localhost' is available or not.
ps: Post it just hope someone who has the similar problem may benefit.
I had this problem when I wanted to make a vnc connection via a tunnel.
But the vncserver was not running.
I solved it by opening the channel on the remote machine with vncserver :3.
In my case, it worked after checking the correct IP address of the user credentials
previously I was using the wrong IP of the server
ssh -NfL 127.0.0.1:8084:127.0.0.1:8888 user#ip_address_of_server
after correcting it, works fine.
Encountered with the same error.
In my case, I found the problem was in the config file of jupyter.
Let's say there are 3 computers named A, B, and C, and A can access B but can't access C; B can access C.
To access jupyter-notebook service of C from A, first I established ssh tunnel from A to C through B, then I access jupyter-notebook by typing localhost:port_number, then I got the error.
Finally the problem was solved by writing the "c.NotebookApp.ip = '0.0.0.0'" in jupyter-notebook's config file, where '0.0.0.0' allows the access of other IPs.
Hope someone in a similar situation may benefit.
I had the same error when I was trying to tunnel my mlflow ui over ssh to view remotely. As mentioned in the first answer, the error arises because nothing on the server is listening for the port. This, for me, is because I forgot to start the mlflow app on my remote machine! So in general – make sure the app you're trying to access remotely is running.
Just replace localhost with 127.0.0.1.
(The answer is based on answers of other people on this page.)
This means the remote vm is not listening to current port i solved this by adding the port in the vm server
I am trying to run putty on Windows 7.
I need to run some SSH commands to upgrade Magento. Every time putty gives me this fatal error:
network error: connection refused
I have even tried to use it through winSCP. Note that putty is running fine on my client's side, but for me it's giving me that error every time.
I tried using port 22, 23 and a few others without success. I have also tried with Windows Firewall both enabled and disabled.
Thank you.
Install open-ssh or ssh in target Linux machine solved my problem .
Use below commands:
sudo apt install ssh
sudo systemctl status ssh
sudo systemctl enable ssh
The following are the list of possibilities of this error:
1) Common cause is - Your IP is blocked or not whitelisted in the server which can be done by editing the file at /etc/csf/csf.conf
2) Disable the windows firewall settings and check..
The above two things should work in most of the cases
3) This is very unusual and when the above two options fails it is very difficult to find the solution
In this case the traceroute to the server host or IP works usually but putty fails
port 22 looks normal and working but connection fails and restarting SSH (service ssh restart) also does not work
reboot or restart the server but still fails
Adding IP in firewalls or whitelisting the local IP may not workout
This case generally happens when the server is migrated or new server. IP of the old server might be whitelisted locally, but don't have port 22 open, or some other local filtering is occurring for this new IP but not the old server IP..
In this case you need to change the port to some 522 or 530 some unusual port will help in sorting it out.. The same port you have to use for the SFTP connection also .. where as FTP connections works normally..
or you have to whitelist the IP locally or open the TCP in and out to port 22 if you want to use the port 22..