I've had a look on google and here on stack but can't find a good example on how to do this.
All I basically want to do is SSH into a server copy all the site files and paste them into a folder on my computer?
I normally use git but this is an old site which has not been setup with git so I just wanted to know a quick way to copy from the server as FTP sucks!
A simple process with commands for terminal would be great!
Check out rsync. It has the capability to operate over ssh. You might also want to look into ssh aliases (which it also honors) when copying files over, and it's what git uses to only sync the differences between two repositories.
The advantage of rsync over SCP or SFTP is that it can resume download if interrupted, takes little bandwidth to sync since it sends change sets instead of entire files (unless the file doesn't yet exist on one side), and can do one- or two-way sync depending on your preference.
ssh USER#SERVER "tar zcvf - /DUMP_DIR" | cat > /OUT_DIR/FILE_NAME_OF_ARCH
or
(rsync -avz --delete /DUMP_DIR USER#SERVER:/OUT_DIR &)
Look at SCP.
scp username#remotehost.com:/directoryname/* /some/local/directory
Use scp
scp -P 2222 json-serde-1.1.8-SNAPSHOT-jar-with-dependencies.jar root#127.0.0.1:
For Example.
Hope that helps!
Related
I have a local folder and a remote one on a server with ssh connection. I don't have admin privileges so installing new packages are not possible to use unison for example. I have to sync these two folders quite often and they are also big. From here I know that to sync in both sides I have to rsync twice. once from server to local:
rsync -Przzuve ssh user#server:/path/to/remote/folder/* /path/to/local/folder
and then the other way around, from local to server
rsync -Przzuve ssh /path/to/local/folder/* user#server:/path/to/remote/folder
What I want to have is a single command like:
rsyncb /path/to/local/folder user#server:/path/to/remote/folder
To just sync the content of two folders in both directions in one command without worrying about the -* options and /* at the end of the first path...
I found this about making a bash function with given arguments but I do not understand how to implement what I want. I would appreciate if you could help me with this.
Just define a function.
rsyncb() {
rsync -Przzuve ssh "$1"/* "$2"
rsync -Przzuve ssh "$2"/* "$1"
}
Then use it like this:
rsyncb user#server:/path/to/remote/dir /path/to/local/dir
I need to transfer around 4.2 GB of files from my local computer to a server B. However to ssh into server B, I need to ssh into server A.
Currently I'm copying files from my local computer to server A and then from server A to server B.
So the flow goes like this:
rsync -avz --del ~/Desktop/abc/ <my-user-name>#<server-A>:~/abc
rsync -avz --del ~/Desktop/abc/ <my-user-name>#<server-B>:~/abc
This is slow and copies 4.2 gb of data two times instead of one!
Can I transfer files with rsync from my local computer to directly server B ?
You can always use ssh with proxy command, which allows you to transfer files transparently. Using this config (~/.ssh/config):
Host <server-A>
User <user-A>
Host <server-B>
User <user-B>
ProxyCommand ssh <server-A> -W %h:%p
You can call your rsync:
rsync -avz --del ~/Desktop/abc/ <server-B>:~/abc
The data will be only "routed" over the middle host.
What you want is to use port-forwarding to forward the ssh/rsync port (generally port 22) from server B to alternate ports on server A so when you call rsync -e "ssh -p altport" serverA:/sourcedir /destdir, you are actually invoking rsync from serverB.
There are many good howtos available on StackExchange and other sites. For example:
How to forward a port from one machine to
another?
or
How To Forward Ports through a Linux Gateway with
Iptables
will get you started. Using port-forwarding, you are essentially using serverA as a pass-through host so you will only have to transfer your 4.2G once.
Yes, you can copy the files (and even folders) directly without making any intermediate copies on the contact/login server, which is by default the machine known to the outside world, or contacted to get access to a specific local network.
Below is a simple demonstration using scp without any unnecessary complications. On the local machine, simply do the following:
$ scp -r -o ProxyCommand="ssh -W %h:%p your_username#contact-server.de" your_username#machine_name:/file/path/on/this/machine ~/destination/path/to/save/the/copied/folder
-r option instructs scp to copy the contents of the entire folder.
your_username need not be the same on both machines.
If it is successful, you'll be asked for your passwords on both machines for authentication.
In the above command it is assumed that the typical way to access the machine named as "machine_name" would be via the contact server.
Note:
The above command also works for transferring data from a source remote machine (e.g. s) to a target remote machine (say t). In such a scenario, first ssh to the source remote machine (s) and navigate to the path where the data resides. After that you can simply think of/treat that remote machine as a local/source machine and then simply use the same scp command listed above for copying folders.
For copying individual files, just remove the -r option and provide the path to the specific file that you want to copy.
How do I copy a folder from remote to local host using scp?
I use ssh to log in to my server.
Then, I would like to copy the remote folder foo to local /home/user/Desktop.
How do I achieve this?
scp -r user#your.server.example.com:/path/to/foo /home/user/Desktop/
By not including the trailing '/' at the end of foo, you will copy the directory itself (including contents), rather than only the contents of the directory.
From man scp (See online manual)
-r Recursively copy entire directories
To use full power of scp you need to go through next steps:
Public key authorisation
Create SSH aliases
Then, for example if you have this ~/.ssh/config:
Host test
User testuser
HostName test-site.example
Port 22022
Host prod
User produser
HostName production-site.example
Port 22022
you'll save yourself from password entry and simplify scp syntax like this:
scp -r prod:/path/foo /home/user/Desktop # copy to local
scp -r prod:/path/foo test:/tmp # copy from remote prod to remote test
More over, you will be able to use remote path-completion:
scp test:/var/log/ # press tab twice
Display all 151 possibilities? (y or n)
For enabling remote bash-completion you need to have bash-shell on both <source> and <target> hosts, and properly working bash-completion. For more information see related questions:
How to enable autocompletion for remote paths when using scp?
SCP filename tab completion
To copy all from Local Location to Remote Location (Upload)
scp -r /path/from/local username#hostname:/path/to/remote
To copy all from Remote Location to Local Location (Download)
scp -r username#hostname:/path/from/remote /path/to/local
Custom Port where xxxx is custom port number
scp -r -P xxxx username#hostname:/path/from/remote /path/to/local
Copy on current directory from Remote to Local
scp -r username#hostname:/path/from/remote .
Help:
-r Recursively copy all directories and files
Always use full location from /, Get full location/path by pwd
scp will replace all existing files
hostname will be hostname or IP address
if custom port is needed (besides port 22) use -P PortNumber
. (dot) - it means current working directory, So download/copy from server and paste here only.
Note: Sometimes the custom port will not work due to the port not being allowed in the firewall, so make sure that custom port is allowed in the firewall for incoming and outgoing connection
What I always use is:
scp -r username#IP:/path/to/server/source/folder/ .
. (dot): it means current folder. so copy from server and paste here only.
IP: can be an IP address like 125.55.41.311 or it can be host like ns1.mysite.example.
Better to first compress catalog on remote server:
tar czfP backup.tar.gz /path/to/catalog
Secondly, download from remote:
scp user#your.server.example.com:/path/to/backup.tar.gz .
At the end, extract the files:
tar -xzvf backup.tar.gz
Typical scenario,
scp -r -P port username#ip:/path-to-folder .
explained with an sample,
scp -r -P 27000 abc#10.70.12.12:/tmp/hotel_dump .
where,
port = 27000
username = "abc" , remote server username
path-to-folder = tmp/hotel_dump
. = current local directory
And if you have one hell of a files to download from the remote location and if you don't much care about security, try changing the scp default encryption (Triple-DES) to something like 'blowfish'.
This will reduce file copying time drastically.
scp -c blowfish -r user#your.server.example.com:/path/to/foo /home/user/Desktop/
Go to Files on your unity toolbar
Press Ctrl + l and write here_goes_your_user_name#192.168.10.123
The 192.168.1.103 is the host that you want to connect.
The here one example
In case you run into "Too many authentication failures", specify the exact SSH key you have added to your severs ssh server:
scp -r -i /path/to/local/key user#remote.tld:/path/to/folder /your/local/target/dir
The question was how to copy a folder from remote to local with scp command.
$ scp -r userRemote#remoteIp:/path/remoteDir /path/localDir
But here is the better way for do it with sftp - SSH File Transfer Protocol (also Secure File Transfer Protocol, or SFTP) is a network protocol that provides file access, file transfer, and file management over any reliable data stream.(wikipedia).
$ sftp user_remote#remote_ip
sftp> cd /path/to/remoteDir
sftp> get -r remoteDir
Fetching /path/to/remoteDir to localDir 100% 398 0.4KB/s 00:00
For help about sftp command just type help or ?.
I don't know why but I was had to use local folder before source server directive . to make it work
scp -r . root#888.888.888.888:/usr/share/nginx/www/example.org/
For Windows OS, we used this command.
pscp -r -P 22 hostname#IP:/path/to/Downloads ./
The premise of the question is incorrect. The idea is, once logged into ssh, how to move files from the logged-in machine back to the client that is logged in. However, scp is not aware of nor can it use the ssh connection. It is making its own connections. So the simple solution is create a new terminal window on the local workstation, and run scp that transfers files from the remote server to local machine. E.g., scp -i key user#remote:/remote-dir/remote-file /local-dir/local-file
I want to install a software library (SWIG) on a list of computers (Jenkins nodes). I'm using the following script to automate this somewhat:
NODES="10.8.255.70 10.8.255.85 10.8.255.88 10.8.255.86 10.8.255.65 10.8.255.64 10.8.255.97 10.8.255.69"
for node in $NODES; do
scp InstallSWIG.sh root#$node:/root/InstallSWIG.sh
ssh root#$node sh InstallSWIG.sh
done
This way it's automated, except for the password request that occur for both the scp and ssh commands.
Is there a way to enter the passwords programmatically?
Security is not an issue. I’m looking for solutions that don’t involve SSH keys.
Here’s an expect example that sshs in to Stripe’s Capture The Flag server and enters the password automatically.
expect <<< 'spawn ssh level01#ctf.stri.pe; expect "password:"; send "e9gx26YEb2\r";'
With SSH the right way to do it is to use keys instead.
# ssh-keygen
and then copy the *~/.ssh/id_rsa.pub* file to the remote machine (root#$node) into the remote user's .ssh/authorized_keys file.
You can perform the task using empty, a small utility from sourceforge. It's similar to expect but probably more convenient in this case. Once you have installed it, your first scp will be accomplished by following two commands:
./empty -f scp InstallSWIG.sh root#$node:/root/InstallSWIG.sh
echo YOUR_SECRET_PASSWORD | ./empty -s -c
The first one starts your command in the background, tricking it into thinking it's running in interactive mode on a terminal. The other one sends it data from stdin. Of course, putting your password anywhere on command line is risky due to shell history being preserved, users being able to see it in ps results etc. Not secure either, but a bit better thing would be to store the password in a file and redirect the second command's input from that file instead of using echo and a pipe.
After copying to the server, you can run the script in a similar manner:
./empty -f ssh root#$node sh InstallSWIG.sh
echo YOUR_SECRET_PASSWORD | ./empty -s -c
You could look into setting up passwordless ssh keys for that. Establishing Batch Mode Connections between OpenSSH and SSH2 is a starting point, you'll find lots of information on this topic on the web.
Wes' answer is the correct one but if you're keen on something dirty and slow, you can use expect to automate this.
Is there a way to publish a web site from Visual Studio 2008 using SCP or SFTP? I know it is possible to publish to my local filesystem and then perform the transfer with SCP, but I'd like something more seamless (e.g. part of Visual Studio). Does this feature exist? An addin perhaps?
The built in system for publishing pages is a little bit limited.
One thing that I find useful is with WinSCP, there is a featured called "Keep Remote Directory up to Date". What it will do is set a bunch of file system watchers for your local system and if you change something locally, it will auto upload it. Using that and publishing to a local directory makes things easy.
If you have Windows 10 and bash/linux subsystem installed and a Linux/BSD server you can:
Combine ssh and rsync
I prefer to use rsync through a ssh pipe, since it won't upload files that werer not modified, it's more efficient.
from visual studio, publish in a folder, say I:/www/WebProject
use this command that updates changes only, and delete files that were deleted/absent from publish folder thanks to --delete
bash -c "rsync -avH --delete --progress /mnt/i/www/WebProject -e ssh server:/var/www/"
Where I:/www/WebProject is the local folder where the project was published, and /var/www the remote directory of the web application.
Preparation (to do once)
You need to work a bit to allow ssh to work without password but with keys.
let's say your bash username is also the same on the server; if not, just use username#server
name your server:
add xx.xx.xx.xx server to the file c:/windows/system32/drivers/etc/hosts)
add your server to hosts from bash with sudo echo "xx.xx.xx.xx server" >> /etc/hosts
from bash, generate your keys:
ssh --keygen
then [enter] (no passphrase)
send your public key to the server, in your home folder:
scp ~/.ssh/id_rsa.pub servername:~/
from your server (ssh server then password):
cat id_rsa.pub >> .ssh/authorized_keys && rm id_rsa.pub
Now you can ssh and scp without password. IMO this is way better than filezilla or just scp.