I have the following script:
#!/bin/sh
...
rsync -e 'ssh -i "$SSHKeyPath"'
The error is:
Warning: Identity file $SSHKeyPath not accessible: No such file or directory.
How can I get $SSHKeyGen evaluated before rsync gets called?
UPDATE:
fwiw, this is on OSX.
There are several solutions that I believe are easier:
Use ssh-agent(1) to unlock the private portion of the key for ssh(1) processes as they need it. This is by far the easiest mechanism to use.
Use ~/.ssh/config to select a different private key based on hostname:
host backuphost
IdentityFile ~/.ssh/different_key
Then there is no need to specify a key on the command line.
Update
Given that you're trying to separate the key from the individual user, this makes a lot more sense to me now. If you use another variable in sh you can make your original approach work:
$ cat foo.sh
#!/bin/sh
SSHKeyPath=/home/sarnold/.ssh/id_rsa
KEYARG="ssh -i $SSHKeyPath"
rsync -e "$KEYARG" /tmp/pointless localhost:/tmp/new_pointless
$ ./foo.sh
Enter passphrase for key '/home/sarnold/.ssh/id_rsa':
skipping directory pointless
You're using single quotes, which cannot substitute variable name with it's value. It will be ok with douuble quotes.
For example
$> path_to_file="/tmp/file"; rsync -e "$( ssh -i "$path_to_file" )"
Warning: Identity file /tmp/file not accessible: No such file or directory.
Try this:
rsync -e "ssh -i ""$SSHKeyPath"
Related
I'm implementing agent script bash to pull files from the remote server with SFTP service.
The script must:
connect SFTP
file listing
cycling on files found
get every file and copy agent side
after that files copied must be deleted
The script is followed:
#!/bin/bash
SFTP_CONNECTION="sftp -oIdentityFile=/home/account_xxx/.ssh/service_ssh user#host"
DEST_DATA=/tmp/test/data/
# GET list file by ls command ###############
$SFTP_CONNECTION
$LIST_FILES_DATA_OSM1 = $("ls fromvan/test/data/test_1")
echo $LIST_FILES_DATA_OSM1
for file in "${LIST_FILES_DATA_OSM1[#]}"
do
$SFTP_CONNECTION get $file $DEST_DATA
$SFTP_CONNECTION rm $file
done
I tried the script but it seems that the connection and command execution (ls) are distinct on thread separated.
How can I provide command sequential as described above ?
Screenshoot:
Invalid find command
SSH it seem not available
RSYNC result to take the files is the followed:
Thanks
First of all, I would recommend the following syntax changes:
#!/bin/bash
sftp_connection() {
sftp -oIdentityFile=/home/account_xxx/.ssh/service_ssh user#host "$#";
}
Dest_Data=/tmp/test/data/
# GET list file by ls command ###############
sftp_connection
List_Files_D_OSM1=$("ls fromvan/test/data/test_1")
echo "$LIST_FILES_DATA_OSM1"
for file in "${LIST_FILES_DATA_OSM1[#]}"
do
sftp_connection get "$file" $Dest_Data
sftp_connection rm "$file"
done
Quoting $file and $List_Files_D_OSM1 to prevent globbing and word splitting.
Assignments can't start with a $, otherwise bash will try to execute List_Files_D_OSM1 and will complain with a command not found
No white spaces in assignments like List_Files_D_OSM1 = $("ls fromvan/test/data/test_1")
You can use ShellCheck to catch this kind of errors.
Having said that, it is in general not a good idea to use ls in such way.
What you can use instead is something like find. For example:
find . -type d -exec echo '{}' \;
Use a different client. lftp supports sftp as a transport, and has a subcommand for mirroring which will do the work of listing the remote directory and iterating over files for you.
Assuming your ~/.ssh/config contains an entry like:
Host myhost
IdentityFile /home/account_xxx/.ssh/service_ssh
...you can run:
lftp -e 'mirror -R fromvan/test/data/test_1 /tmp/test/data' sftp://user#myhost
I am trying to rename all files in a remote directory over SSH or SFTP. The rename should convert the file into a date extension, for example .txt into .txt.2016-05-25.
I have the following command to loop each .txt file and try to rename, but am getting an error:
ssh $user#$server "for FILENAME in $srcFolder/*.txt; do mv $FILENAME $FILENAME.$DATE; done"
The error I am getting is:
mv: missing destination file operand after `.20160525_1336'
I have also tried this over SFTP with no such luck. Any help would be appreciated!
You need to escape (or single-quote) the $ of variables in the remote shell. It's also recommended to quote variables that represent file paths:
ssh $user#$server "for FILENAME in '$srcFolder'/*.txt; do mv \"\$FILENAME\" \"\$FILENAME.$DATE\"; done"
Try this:
By using rename (perl tool):
ssh user#host /bin/sh <<<$'
rename \047use POSIX;s/$/strftime(".%F",localtime())/e\047 "'"$srcFolder\"/*.txt"
To prepare/validate your command line, replace ssh...bin/sh by cat:
cat <<<$'
rename \047use POSIX;s/$/strftime(".%F",localtime())/e\047 "'"$srcFolder\"/*.txt"
will render something like:
rename 'use POSIX;s/$/strftime(".%F",localtime())/e' "/tmp/test dir"/*.txt
And you could localy try (ensuring $srcFolder contain a path to a local test folder):
/bin/sh <<<$'
rename \047use POSIX;s/$/strftime(".%F",localtime())/e\047 "'"$srcFolder\"/*.txt"
Copy of your own syntax:
ssh $user#$server /bin/sh <<<'for FILENAME in "'"$srcFolder"'"/*.txt; do
mv "$FILENAME" "$FILENAME.'$DATE'";
done'
Again, you could locally test your inline script:
sh <<<'for FILENAME in "'"$srcFolder"'"/*.txt; do
mv "$FILENAME" "$FILENAME.'$DATE'";
done'
or preview by replacing sh by cat.
When using/sending variables over SSH, you need to be careful what is a local variable and which is a remote variable. Remote variables must be escaped; otherwise they will be interpreted locally versus remotely as you intended. Other characters also need to be escaped such as backticks. The example below should point you in the right direction:
Incorrect
user#host1:/home:> ssh user#host2 "var=`hostname`; echo \$var"
host1
Correct
user#host1:/home:> ssh user#host2 "var=\`hostname\`; echo \$var"
host2
i need to exchange public key between two systems A and B.
These are the steps am following
copy the content of id_rsa.pub from /root/.ssh directory and save it in variable 'key'
ssh to B as ubuntu user . ssh -i key_file ubuntu#B
Move to root login by sudo su
Append the variable $key to /root/.ssh/authorized_keys
But the file authorized_keys is owned by root. Hence i get the permission error.
I cannot directory connect to system B as root. Only way is to connect as ubuntu and change to root.
I tried the following shell script
# Get all the Ips from the source file
sudo grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' $1 | sort -u > /tmp/list_of_servers.txt
# Get the public key
pubkey=$(sudo cat /root/.ssh/id_rsa.pub)
# For each server
while read ip;
do
(echo "$ip"
# ssh to the server
ssh -i $2 $3#$ip
# append key to autorized_keys file
sudo -c "echo $pubkey >> /root/.ssh/authorized_keys" root
echo "done $ip" )
done < /tmp/list_of_servers.txt
but i didnt work. its giving me permission error.
Can someone help me in the last step.
A fully paranoid approach to the mechanics of the SSH connection might be something like this:
# generate a shell-escaped version of the public key (spaces, wildcards, etc)
printf -v pubkey_q '%q' "$pubkey"
# generate a shell command using that quoted form
cmd="echo $pubkey_q >>/root/.ssh/authorized_keys"
# generate a shell-quoted sudo command invoking the above in a shell
printf -v cmd_q '%q ' sudo bash -c "$cmd"
# ...and execute it on the other end of a ssh connection.
ssh -i "$2" "$3#$ip" "$cmd_q"
printf %q is a bash extension which escapes a string in such a way that being parsed by a shell -- whether in a string that's eval'd, passed to ssh with bash as the remote shell, or passed to bash -c -- evaluates back to the original data. (For regular whitespace its output is safe for sh -c as well, but for any content where bash prefers $'' to escape nonprintable characters, this output may not be POSIX compliant).
This code doesn't do what you think it does:
# ssh to the server
ssh -i $2 $3#$ip
# append key to autorized_keys file
sudo -c "echo $pubkey >> /root/.ssh/authorized_keys" root
The ssh command there would normally open an interactive remote shell, but since we are in a script, an interactive shell is not possible. So the remote shell immediately exits, without actually doing anything at all.
The sudo command that follows is incorrect syntax, it cannot work that way with the -c flag. Check the man page of sudo. And since you are not actually in the remote she'll as you may have believed, the command is running in your local system, not the remote one where you want to append your key.
To run sudo remotely, use something like this:
ssh -i $2 $3#$ip sudo echo hello
The echo is just an example for testing of course.
However, this whole attempt of appending a public key to the authorized list of root is deeply flawed in terms of security. Sudo should be configured to ask for the password of the user, and there is no good way to do that in a script. Or if the user can run sudo without entering a password, that's just unacceptable from a security perspective.
Trying to copy a remote file to my local system using scp in bash
I've obtained the filename that i want and assigned to variable, $lastModifiedFile,
but the problem is it contains spaces in the filename.
To use this variable with scp the spaces need to be escaped with backslashes.
Is there an easy way to format this variable and insert the correct escape character where necessary i.e on spaces?
#!/bin/bash
lastModifiedFile=$(sshpass -p 'passw0rd' ssh user#server 'ls -tr /path/output*| tail -n 1')
echo "$lastModifiedFile"
sshpass -p 'passw0rd' scp user#server:"$lastModifiedFile" /root/
This is the script output ..
[user#host ~]# ./script.sh
/path/outputSat Mar 09 151905 GMT 2013.html
scp: /path/outputSat: No such file or directory
scp: Mar: No such file or directory
scp: 09: No such file or directory
scp: 151905: No such file or directory
scp: GMT: No such file or directory
scp: 2013.html: No such file or directory
I'm looking for something like below, or even a simpler solution? ..
escapedFilename=""
for letter in $lastModifiedFile
if $letter == " "
$escapedFilename += "\ "
else
$escapedFilename += $letter
With a bit of leaning toothpick syndrome:
param=user#server:${lastModifiedFile// /\\ /}
sshpass -p 'passw0rd' scp "$param" /root/
EDIT: It seems scp does not like me. I needed an additional level of variable in testing ... :)
EDIT 2: According to OP's feedback the exact solution appears to consist of using ${lastModifiedFile// /\\ \\}
I just hope there are no other characters than space that need escaping in some other filenames :)
Use single quotes around the filename passed to the remote system so that it is not subject to word splitting.
lastModifiedFile=$(sshpass -p 'passw0rd' ssh user#server 'ls -tr /path/output*| tail -n 1')
echo "$lastModifiedFile"
sshpass -p 'passw0rd' scp user#server:"'$lastModifiedFile'" /root/
or
sshpass -p 'passw0rd' scp "user#server:'$lastModifiedFile'" /root/
Just do it like this:
sshpass -p 'passw0rd' scp 'user#server:$lastModifiedFile' /root/
Here are a couple of methods that should handle almost anything (not just spaces) in the filename. First, bash's printf builtin has a %q format that adds quotes/escapes/whatever to the string:
sshpass -p 'passw0rd' scp user#server:"$(printf %q "$lastModifiedFile")" /root/
Note, however, that this quotes/escapes/etc it suitably for interpretation by bash. If the remote computer's default shell is something else, this may not work in all cases.
Option two is simpler in principle (but a bit messy in practice), and should be compatible with more remote shells. Here, I enclose the filename in single-quotes, which should work for anything other than single-quotes within the filename. For those, I substitute '\'' (which ends the single-quoted string, adds an escaped single-quote, then restarts the single-quoted string):
repl="'\''" # Have to store this in a variable to work around a bash parsing oddity
sshpass -p 'passw0rd' scp user#server:"'${lastModifiedFile//\'/$repl}'" /root/
I'm writing a cron to backup some stuffs on a server.
Basically I'm sending specific files form a local directory using scp.
I'm using a public key to avoid authentication.
For reusability I'm passing the local directory and the server url by arguments to my bash script.
How I set my parameters:
#!/bin/bash
DIR="$1"
URL="$2"
FILES="$DIR*.ext"
My problem is about formatting the url.
Without formatting
How I send files to the server:
#!/bin/bash
for F in $FILEs
do
scp $F $URL;
if ssh $URL stat $(basename "$F")
then
rm $F
else
echo "Fails to copy $F to $URL"
fi
done
If I try to copy at user's home on the server I do:
$ ~/backup /path/to/local/folder/ user#server.com:
If I try to copy at a specific directory on the server I do:
$ ~/backup /path/to/local/folder/ user#server.com:/path/to/remote/folder/
In all cases it gives me the well known error (and my custom echo):
ssh: Could not resolve hostname user#server.com: nodename nor [...]
Can't upload /path/to/local/folder/file.ext to user#server.com
And it works anyway (the file is copied). But that's not a solution, cause as scp fails (seems to), the file is never deleted.
With formatting
I tried sending files using this method:
#!/bin/bash
for F in $FILES
do
scp $F "$URL:"
done
I no longer get an error, and it works for copying at user's home directory then deleting the local file:
$ ~/backup /path/to/local/folder/ user#server.com
But, of course, sending to a specific directory don't work at all.
Finally
So I think that my first method is more appropriate, but how can I get rid of that error?
Your mistake is that you can scp to user#server.com: but not ssh to it : you need to remove the trailing : character (and possible path after it). You can do it easily like this with bash parameter expansion :
ssh "${URL%:*}" stat "$(basename "$F")"
RECOMMENDATIONS
"USE MORE QUOTES!" They are vital. Also, learn the difference between ' and " and `. See http://mywiki.wooledge.org/Quotes and http://wiki.bash-hackers.org/syntax/words
if you have spaces in filenames, your code will breaks things up. Better use while IFS= read -r line; do #stuff with $line; done < file.txt
See bash parameter expansion