I regularly have to connect to several systems via ssh using multiple hops. It also happens often that I then want to copy a file from either the destination system to my local system or the other way around in a simple way (my current work flow is copy the file to an external location both machines can see so that it saves me a few hops or if the file is not binary cat it and copy/paste it to the other terminal window).
Is there an easy way to do such a thing?
I am using OSX and iterm2 (obviously I woudn't mind changing the latter).
So the connection is something like (local machine) -> (portal A) -> (machine B) -> (portal C) -> (machine D)
So I would like to copy files from machine A to machine D in a simple way (without copying the file via all hops or creating four tunnels).
It's not quite what you're asking for, but there are some tricks you can play with SSH proxying that simplify this sort of thing enormously. The first thing to get familiar with is proxying multihop SSH connections over netcat. If you have OpenSSH version 5.4 or later on the various hosts, add something like this to your ~/.ssh/config:
Host B
ProxyCommand ssh A -W %h:%p
Host C
ProxyCommand ssh B -W %h:%p
Host D
ProxyCommand ssh D -W %h:%p
If any of the intermediates don't have a new enough version, but do have netcat (nc), you can use something like this instead:
Host D
ProxyCommand ssh C nc %h %p
This'll make ssh D automatically open a tunnel to C to run the connection over, which will automatically open a tunnel to B, ... You'll have to authenticate 4 times (to A, then B, etc) (unless you have public-key authentication set up), but other than that it's transparent. Which means you can use it with sftp D, scp D:/path/to/file, etc.
Now, there's one significant limitation on this for what you describe. You can certainly copy files from e.g. A to D like this:
scp A:/path/to/file D:/path/to/file
...but the file's contents will travel the path A -> your computer -> A -> B -> C -> D. They won't be stored anywhere on that path, but if the network link between you and A is slow (e.g. you're working from home), this'll be a bottleneck. In this case, it'd be best to copy the ~/.ssh/config entries for C and D onto computer A, ssh into A normally, then use scp /path/to/file D:/path/to/file and cut out the extra hops.
BTW, if you want to get fancy, you can add this to your ~/.ssh/config:
Host */*
ProxyCommand ssh $(dirname %h) -W $(basename %h):%p
And then use ssh A/B/C/D etc to built the tunnel path on the spot. See the OpenSSH cookbook for details.
I had to think about this for some time, but if you have set up passwordless authentication using keys, it is possible to do the thing like this:
$ cat test | ssh f21 "tee | ssh f20 \"tee test\""
encrypted ssh key doesn't matter. For transferring through one hop it is quite straightforward, for more hops it can get messy ...
Related
So, there exists the first SSH server, called A for simplicity, and the files and workspace I want to get to are on another SSH server, called B. Problem is, B is only accessible by SSH'ing to it from A.
So if I were to do this normally, I'd ssh into A, and from there ssh into B, or just ssh -t A "ssh B" which becomes mildly inefficient, if I were to code in B using vim.
So I don't know exactly how to sshfs B onto my local machine, but I can sshfs A. Sshfs'ing B onto A isn't possible, as A doesn't have it installed. Is there a way to sshfs B?
A and B are both Ubuntu Ubuntu 18.04.3 LTS and my computer is a Macbook Pro 2015.
I've tried the SSH-FS extension with VSCode. It only connects to A.
I also tried the Remote-SSH extension, and again, only gets as far as A. I even used -t for the connection command, but it doesn't seem to make any changes to the Remote-SSH config file.
I also tried ssh-fs while tunneled into A. No results there either.
Sorry for the trouble. This is a really niche problem.
You must use .ssh/config to have the ability to connect to B in on command from your Macbook Pro .
a example :
Host serverb.via.servera
HostKeyAlias serverb
User account_on_b
ProxyCommand ssh account_on_a#servera -W serverb:22
before using sshfs , you must test the setting by running :
ssh serverb.via.servera
and if you have your shell , so you can run
sshfs serverb.via.servera:/yourdir/ /tmp/localdir/
remarks:
serverb.via.servera is a arbritrary string , that you must use instead of the host
account_on_a and account_on_b must be replace by the login you use on each servers
I wonder if there is a way for me to SCP the file from remote2 host directly from my local machine by going through a remote1 host.
The networks only allow connections to remote2 host from remote1 host. Also, neither remote1 host nor remote2 host can scp to my local machine.
Is there something like:
scp user1#remote1:user2#remote2:file .
First window: ssh remote1, then scp remot2:file ..
Second shell: scp remote1:file .
First window: rm file; logout
I could write a script to do all these steps, but if there is a direct way, I would rather use it.
Thanks.
EDIT: I am thinking something like opening SSH tunnels but i'm confused on what value to put where.
At the moment, to access remote1, i have the following in $HOME/.ssh/config on my local machine.
Host remote1
User user1
Hostname localhost
Port 45678
Once on remote1, to access remote2, it's the standard local DNS and port 22. What should I put on remote1 and/or change on localhost?
I don't know of any way to copy the file directly in one single command, but if you can concede to running an SSH instance in the background to just keep a port forwarding tunnel open, then you could copy the file in one command.
Like this:
# First, open the tunnel
ssh -L 1234:remote2:22 -p 45678 user1#remote1
# Then, use the tunnel to copy the file directly from remote2
scp -P 1234 user2#localhost:file .
Note that you connect as user2#localhost in the actual scp command, because it is on port 1234 on localhost that the first ssh instance is listening to forward connections to remote2. Note also that you don't need to run the first command for every subsequent file copy; you can simply leave it running.
Double ssh
Even in your complex case, you can handle file transfer using a single command line, simply with ssh ;-)
And this is useful if remote1 cannot connect to localhost:
ssh user1#remote1 'ssh user2#remote2 "cat file"' > file
tar
But you loose file properties (ownership, permissions...).
However, tar is your friend to keep these file properties:
ssh user1#remote1 'ssh user2#remote2 "cd path2; tar c file"' | tar x
You can also compress to reduce network bandwidth:
ssh user1#remote1 'ssh user2#remote2 "cd path2; tar cj file"' | tar xj
And tar also allows you transferring a recursive directory through basic ssh:
ssh user1#remote1 'ssh user2#remote2 "cd path2; tar cj ."' | tar xj
ionice
If the file is huge and you do not want to disturb other important network applications, you may miss network throughput limitation provided by scp and rsync tools (e.g. scp -l 1024 user#remote:file does not use more than 1 Mbits/second).
But, a workaround is using ionice to keep a single command line:
ionice -c2 -n7 ssh u1#remote1 'ionice -c2 -n7 ssh u2#remote2 "cat file"' > file
Note: ionice may not be available on old distributions.
This will do the trick:
scp -o 'Host remote2' -o 'ProxyCommand ssh user#remote1 nc %h %p' \
user#remote2:path/to/file .
To SCP the file from the host remote2 directly, add the two options (Host and ProxyCommand) to your ~/.ssh/config file (see also this answer on superuser). Then you can run:
scp user#remote2:path/to/file .
from your local machine without having to think about remote1.
With openssh version 7.3 and up it is easy. Use ProxyJump option in the config file.
# Add to ~/.ssh/config
Host bastion
Hostname bastion.client.com
User userForBastion
IdentityFile ~/.ssh/bastion.pem
Host appMachine
Hostname appMachine.internal.com
User bastion
ProxyJump bastion # openssh 7.3 version new feature ProxyJump
IdentityFile ~/.ssh/appMachine.pem. #no need to copy pem file to bastion host
Commands to run to login or copy
ssh appMachine # no need to specify any tunnel.
scp helloWorld.txt appMachine:. # copy without intermediate jumphost/bastion host copy.**
ofcourse you can specify bastion Jump host using option "-J" to ssh command, if not configured in config file.
Note scp does not seems to support "-J" flag as of now. (i could not find in man pages. However above scp works with config file setting)
There is a new option in scp that add recently for exactly this same job that is very convenient, it is -3.
TL;DR For the current host that has authentication already set up in ssh config files, just do:
scp -3 remote1:file remote2:file
Your scp must be from recent versions.
All other mentioned technique requires you to set up authentication from remote1 to remote2 or vice versa, which not always is a good idea.
Argument -3 means you want to move files from two remote hosts by using current host as intermediary, and this host actually does the authentication to both remote hosts, so they don't have to have access to each other.
You just have to setup authentication in ssh config files, which is fairly easy and well documented, and then just run the command in TL;DR
The source for this answer is https://superuser.com/a/686527/713762
This configuration works nice for me:
Host jump
User username
Hostname jumphost.yourorg.intranet
Host production
User username
Hostname production.yourorg.intranet
ProxyCommand ssh -q -W %h:%p jump
Then the command
scp myfile production:~
Copies myfile to production machine.
a simpler way:
scp -o 'ProxyJump your.jump.host' /local/dir/myfile.txt remote.internal.host:/remote/dir
I was wondering if it is possible to edit a file with Sublime Text 3 through multi-hop SSH tunnel. In my particular case I have my Mac (let's call it A) and two Linux Machines: B and C. The files are located in C, and I access them with my machine like this:
A -> B -> C
I found these articles that can help but they only talk about editing files in B.
How to open remote files in sublime text 3
Editing files remotely via SSH on SublimeText 3
According to these articles, I can edit files in B installing rsub in the remote machine and a plugin in Sublime at A. I tried to do that in C (yes, i know it is not so useful, but who knows) but I got the error:
user#remote-C:~$ rsub
/usr/local/bin/rsub: connect: Connection refused
/usr/local/bin/rsub: line 327: /dev/tcp/localhost/52698: Connection refused
Unable to connect to TextMate on localhost:52698
I would be happy to know if there is a way to achieve this. Thanks in advance.
I will answer to myself. The solution is to do a SSH tunnelling from A to C with B in between using the ProxyCommand in the ssh config file at ~/.ssh/config.
I added these lines:
Host myMachineC
HostName NAME_OF_MACHINE_C
ProxyCommand ssh USER_IN_B#NAME_OF_MACHINE_B nc %h %p
User USER_IN_C
RemoteForward 52698 localhost:52698 # this is required by rsub
Host defines an alias for the real hostname which is written after the HostName directive. ProxyCommand is a command that is executed when you try to log in myMachineC. nc is a command that...
...by default creates a TCP socket either in listening mode (server socket) or a socket that is used in order to connect to a server (client mode) [1]
Now the machine C is accessible from A by only typing:
$ ssh myMachineC
It is recommendable that you already allowed password-less logins. To achieve this you need to have installed the public key from your home computer into the ~/.ssh/authorized_keys of each host along the way. [2]
In conclusion: With all this procedure, there will be a normal SSH connection to the intermediary machine B and then nc will be used to extend the connection to C. Using this tunnelling, the client can act as if the connection were direct using ssh. That will be useful to use with rsub.
Then, you should install and use rsub as normal and it will work like a charm.
I tried this in OSX Yosemite, but should run in almost any *nix system. I hope it will be useful for you.
Netcat Explanation and Examples
Transparent Multihop in SSH
The accepted solution didn't work for me because I use Host B as a SSH server where my SSH keys are stored. Also my SSH keys have passwords so the ProxyCommand command won't work.
But There's an easier way to do this.
You can add the following to the .ssh/config file on Host B;
Host *
RemoteForward 52698 localhost:52698
You can define a specific host or give the * wildcard for all hosts. This will forward port 52698 for all SSH sessions from Host B.
I need to transfer around 4.2 GB of files from my local computer to a server B. However to ssh into server B, I need to ssh into server A.
Currently I'm copying files from my local computer to server A and then from server A to server B.
So the flow goes like this:
rsync -avz --del ~/Desktop/abc/ <my-user-name>#<server-A>:~/abc
rsync -avz --del ~/Desktop/abc/ <my-user-name>#<server-B>:~/abc
This is slow and copies 4.2 gb of data two times instead of one!
Can I transfer files with rsync from my local computer to directly server B ?
You can always use ssh with proxy command, which allows you to transfer files transparently. Using this config (~/.ssh/config):
Host <server-A>
User <user-A>
Host <server-B>
User <user-B>
ProxyCommand ssh <server-A> -W %h:%p
You can call your rsync:
rsync -avz --del ~/Desktop/abc/ <server-B>:~/abc
The data will be only "routed" over the middle host.
What you want is to use port-forwarding to forward the ssh/rsync port (generally port 22) from server B to alternate ports on server A so when you call rsync -e "ssh -p altport" serverA:/sourcedir /destdir, you are actually invoking rsync from serverB.
There are many good howtos available on StackExchange and other sites. For example:
How to forward a port from one machine to
another?
or
How To Forward Ports through a Linux Gateway with
Iptables
will get you started. Using port-forwarding, you are essentially using serverA as a pass-through host so you will only have to transfer your 4.2G once.
Yes, you can copy the files (and even folders) directly without making any intermediate copies on the contact/login server, which is by default the machine known to the outside world, or contacted to get access to a specific local network.
Below is a simple demonstration using scp without any unnecessary complications. On the local machine, simply do the following:
$ scp -r -o ProxyCommand="ssh -W %h:%p your_username#contact-server.de" your_username#machine_name:/file/path/on/this/machine ~/destination/path/to/save/the/copied/folder
-r option instructs scp to copy the contents of the entire folder.
your_username need not be the same on both machines.
If it is successful, you'll be asked for your passwords on both machines for authentication.
In the above command it is assumed that the typical way to access the machine named as "machine_name" would be via the contact server.
Note:
The above command also works for transferring data from a source remote machine (e.g. s) to a target remote machine (say t). In such a scenario, first ssh to the source remote machine (s) and navigate to the path where the data resides. After that you can simply think of/treat that remote machine as a local/source machine and then simply use the same scp command listed above for copying folders.
For copying individual files, just remove the -r option and provide the path to the specific file that you want to copy.
I have a small local network. Only one of the machines is available to the outside world (this is not easily changeable). I'd like to be able to set it up such that ssh requests that don't come in on the standard port go to another machine. Is this possible? If so, how?
Oh and all of these machines are running either Ubuntu or OS X.
Another way to go would be to use ssh tunneling (which happens on the client side).
You'd do an ssh command like this:
ssh -L 8022:myinsideserver:22 paul#myoutsideserver
That connects you to the machine that's accessible from the outside (myoutsideserver) and creates a tunnel through that ssh connection to port 22 (the standard ssh port) on the server that's only accessible from the inside.
Then you'd do another ssh command like this (leaving the first one still connected):
ssh -p 8022 paul#localhost
That connection to port 8022 on your localhost will then get tunneled through the first ssh connection taking you over myinsideserver.
There may be something you have to do on myoutsideserver to allow forwarding of the ssh port. I'm double-checking that now.
Edit
Hmmm. The ssh manpage says this: **Only the superuser can forward privileged ports. **
That sort of implies to me that the first ssh connection has to be as root. Maybe somebody else can clarify that.
It looks like superuser privileges aren't required as long as the forwarded port (in this case, 8022) isn't a privileged port (like 22). Thanks for the clarification Mike Stone.
#Mark Biek
I was going to say that, but you beat me to it! Anyways, I just wanted to add that there is also the -R option:
ssh -R 8022:myinsideserver:22 paul#myoutsideserver
The difference is what machine you are connecting to/from. My boss showed me this trick not too long ago, and it is definitely really nice to know... we were behind a firewall and needed to give external access to a machine... he got around it by ssh -R to another machine that was accessible... then connections to that machine were forwarded into the machine behind the firewall, so you need to use -R or -L based on which machine you are on and which you are ssh-ing to.
Also, I'm pretty sure you are fine to use a regular user as long as the port you are forwarding (in this case the 8022 port) is not below the restricted range (which I think is 1024, but I could be mistaken), because those are the "reserved" ports. It doesn't matter that you are forwarding it to a "restricted" port because that port is not being opened (the machine is just having traffic sent to it through the tunnel, it has no knowledge of the tunnel), the 8022 port IS being open and so is restricted as such.
EDIT: Just remember, the tunnel is only open so long as the initial ssh remains open, so if it times out or you exit it, the tunnel will be closed.
(In this example, I am assuming port 2222 will go to your internal host. $externalip and $internalip are the ip addresses or hostnames of the visible and internal machine, respectively.)
You have a couple of options, depending on how permanent you want the proxying to be:
Some sort of TCP proxy. On Linux, the basic idea is that before the incoming packet is processed, you want to change its destination—i.e. prerouting destination NAT:
iptables -t nat -A PREROUTING -p tcp -i eth0 -d $externalip --dport 2222 --sport
1024:65535 -j DNAT --to $internalip:22
Using SSH to establish temporary port forwarding. From here, you have two options again:
Transparent proxy, where the client thinks that your visible host (on port 2222) is just a normal SSH server and doesn't realize that it is passing through. While you lose some fine-grained control, you get convenience (especially if you want to use SSH to forward VNC or X11 all the way to the inner host).
From the internal machine: ssh -g -R 2222:localhost:22 $externalip
Then from the outside world: ssh -p 2222 $externalip
Notice that the "internal" and "external" machines do not have to be on the same LAN. You can port forward all the way around the world this way.
Forcing login to the external machine first. This is true "forwarding," not "proxying"; but the basic idea is this: You force people to log in to the external machine (so you control on who can log in and when, and you get logs of the activity), and from there they can SSH through to the inside. It sounds like a chore, but if you set up simple shell scripts on the external machine with the names of your internal hosts, coupled with password-less SSH keypairs then it is very straightforward for a user to log in. So:
On the external machine, you make a simple script, /usr/local/bin/internalhost which simply runs ssh $internalip
From the outside world, users do: ssh $externalip internalhost and once they log in to the first machine, they are immediately forwarded through to the internal one.
Another advantage to this approach is that people don't get key management problems, since running two SSH services on one IP address will make the SSH client angry.
FYI, if you want to SSH to a server and you do not want to worry about keys, do this
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no
I have an alias in my shell called "nossh", so I can just do nossh somehost and it will ignore all key errors. Just understand that you are ignoring security information when you do this, so there is a theoretical risk.
Much of this information is from a talk I gave at Barcamp Bangkok all about fancy SSH tricks. You can see my slides, but I recommend the text version as the S5 slides are kind of buggy. Check out the section called "Forward Anything: Simple Port Forwarding" for info. There is also information on creating a SOCKS5 proxy with OpenSSH. Yes, you can do that. OpenSSH is awesome like that.
(Finally, if you are doing a lot of traversing into the internal network, consider setting up a VPN. It sounds scary, but OpenVPN is quite simple and runs on all OSes. I would say it's overkill just for SSH; but once you start port-forwarding through your port-forwards to get VNC, HTTP, or other stuff happening; or if you have lots of internal hosts to worry about, it can be simpler and more maintainable.)
You can use Port Fowarding to do this. Take a look here:
http://portforward.com/help/portforwarding.htm
There are instructions on how to set up your router to port forward request on this page:
http://www.portforward.com/english/routers/port_forwarding/routerindex.htm
In Ubuntu, you can install Firestarter and then use it's Forward Service feature to forward the SSH traffic from a non standard port on your machine with external access to port 22 on the machine inside your network.
On OS X you can edit the /etc/nat/natd.plist file to enable port fowarding.
Without messing around with firewall rules, you can set up a ~/.ssh/config file.
Assume 10.1.1.1 is the 'gateway' system and 10.1.1.2 is the 'client' system.
Host gateway
Hostname 10.1.1.1
LocalForward 8022 10.1.1.2:22
Host client
Hostname localhost
Port 8022
You can open an ssh connection to 'gateway' via:
ssh gateway
In another terminal, open a connection to the client.
ssh client