I am deeply confused by Bash's Heredoc construct behaviour.
Here is what I am doing:
#!/bin/bash
user="some_user"
server="some_server"
address="$user"#"$server"
printf -v user_q '%q' "$user"
function run {
ssh "$address" /bin/bash "$#"
}
run << SSHCONNECTION1
sudo dpkg-query -W -f='${Status}' nano 2>/dev/null | grep -c "ok installed" > /home/$user_q/check.txt
softwareInstalled=$(cat /home/$user_q/check.txt)
SSHCONNECTION1
What I get is
cat: /home/some_user/check.txt: No such file or directory
This is very bizarre, because the file exists if I was to connect using SSH and check the following path.
What am I doing wrong? File is not executable, just a text file.
Thank you.
If you want the cat to run remotely, rather than locally during the heredoc's evaluation, escape the $ in the $(...):
softwareInstalled=\$(cat /home/$user_q/check.txt)
Of course, this only has meaning if some other part of your remote script then refers to "$softwareInstalled" (or, since it's in an unquoted heredoc, "\$softwareInstalled").
Related
I am trying to rename all files in a remote directory over SSH or SFTP. The rename should convert the file into a date extension, for example .txt into .txt.2016-05-25.
I have the following command to loop each .txt file and try to rename, but am getting an error:
ssh $user#$server "for FILENAME in $srcFolder/*.txt; do mv $FILENAME $FILENAME.$DATE; done"
The error I am getting is:
mv: missing destination file operand after `.20160525_1336'
I have also tried this over SFTP with no such luck. Any help would be appreciated!
You need to escape (or single-quote) the $ of variables in the remote shell. It's also recommended to quote variables that represent file paths:
ssh $user#$server "for FILENAME in '$srcFolder'/*.txt; do mv \"\$FILENAME\" \"\$FILENAME.$DATE\"; done"
Try this:
By using rename (perl tool):
ssh user#host /bin/sh <<<$'
rename \047use POSIX;s/$/strftime(".%F",localtime())/e\047 "'"$srcFolder\"/*.txt"
To prepare/validate your command line, replace ssh...bin/sh by cat:
cat <<<$'
rename \047use POSIX;s/$/strftime(".%F",localtime())/e\047 "'"$srcFolder\"/*.txt"
will render something like:
rename 'use POSIX;s/$/strftime(".%F",localtime())/e' "/tmp/test dir"/*.txt
And you could localy try (ensuring $srcFolder contain a path to a local test folder):
/bin/sh <<<$'
rename \047use POSIX;s/$/strftime(".%F",localtime())/e\047 "'"$srcFolder\"/*.txt"
Copy of your own syntax:
ssh $user#$server /bin/sh <<<'for FILENAME in "'"$srcFolder"'"/*.txt; do
mv "$FILENAME" "$FILENAME.'$DATE'";
done'
Again, you could locally test your inline script:
sh <<<'for FILENAME in "'"$srcFolder"'"/*.txt; do
mv "$FILENAME" "$FILENAME.'$DATE'";
done'
or preview by replacing sh by cat.
When using/sending variables over SSH, you need to be careful what is a local variable and which is a remote variable. Remote variables must be escaped; otherwise they will be interpreted locally versus remotely as you intended. Other characters also need to be escaped such as backticks. The example below should point you in the right direction:
Incorrect
user#host1:/home:> ssh user#host2 "var=`hostname`; echo \$var"
host1
Correct
user#host1:/home:> ssh user#host2 "var=\`hostname\`; echo \$var"
host2
i need to exchange public key between two systems A and B.
These are the steps am following
copy the content of id_rsa.pub from /root/.ssh directory and save it in variable 'key'
ssh to B as ubuntu user . ssh -i key_file ubuntu#B
Move to root login by sudo su
Append the variable $key to /root/.ssh/authorized_keys
But the file authorized_keys is owned by root. Hence i get the permission error.
I cannot directory connect to system B as root. Only way is to connect as ubuntu and change to root.
I tried the following shell script
# Get all the Ips from the source file
sudo grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' $1 | sort -u > /tmp/list_of_servers.txt
# Get the public key
pubkey=$(sudo cat /root/.ssh/id_rsa.pub)
# For each server
while read ip;
do
(echo "$ip"
# ssh to the server
ssh -i $2 $3#$ip
# append key to autorized_keys file
sudo -c "echo $pubkey >> /root/.ssh/authorized_keys" root
echo "done $ip" )
done < /tmp/list_of_servers.txt
but i didnt work. its giving me permission error.
Can someone help me in the last step.
A fully paranoid approach to the mechanics of the SSH connection might be something like this:
# generate a shell-escaped version of the public key (spaces, wildcards, etc)
printf -v pubkey_q '%q' "$pubkey"
# generate a shell command using that quoted form
cmd="echo $pubkey_q >>/root/.ssh/authorized_keys"
# generate a shell-quoted sudo command invoking the above in a shell
printf -v cmd_q '%q ' sudo bash -c "$cmd"
# ...and execute it on the other end of a ssh connection.
ssh -i "$2" "$3#$ip" "$cmd_q"
printf %q is a bash extension which escapes a string in such a way that being parsed by a shell -- whether in a string that's eval'd, passed to ssh with bash as the remote shell, or passed to bash -c -- evaluates back to the original data. (For regular whitespace its output is safe for sh -c as well, but for any content where bash prefers $'' to escape nonprintable characters, this output may not be POSIX compliant).
This code doesn't do what you think it does:
# ssh to the server
ssh -i $2 $3#$ip
# append key to autorized_keys file
sudo -c "echo $pubkey >> /root/.ssh/authorized_keys" root
The ssh command there would normally open an interactive remote shell, but since we are in a script, an interactive shell is not possible. So the remote shell immediately exits, without actually doing anything at all.
The sudo command that follows is incorrect syntax, it cannot work that way with the -c flag. Check the man page of sudo. And since you are not actually in the remote she'll as you may have believed, the command is running in your local system, not the remote one where you want to append your key.
To run sudo remotely, use something like this:
ssh -i $2 $3#$ip sudo echo hello
The echo is just an example for testing of course.
However, this whole attempt of appending a public key to the authorized list of root is deeply flawed in terms of security. Sudo should be configured to ask for the password of the user, and there is no good way to do that in a script. Or if the user can run sudo without entering a password, that's just unacceptable from a security perspective.
Hey guys I'm trying to run this code:
#!/bin/bash
sudo /usr/local/bin/sshpass -p pwd ssh -o stricthostkeychecking=no -p 11022 admin#$1.test.com<<EOI
i=1
while read line
do
location="sudo sed -n ${i}p /Users/Shared/$1.txt"
number="sudo sed -n ${i}p /Users/Shared/$2n.txt"
my_array=("${my_array[i]}" $line)
sudo cp /Applications/TEPS\ OS\ X\ Share\ Folder/MAIN\ IMAGES\ FOLDER\ ƒ/${location}${number} /Users/Shared/FYP/$number
sudo sips -Z 256 /Users/Shared/FYP/$number /Users/Shared/FYP/$number
((i++))
done </Users/Shared/$2.txt
exit
EOI
basically it reads a text file which gives the location of certain images, and will create a thumbnail of those images, which can be downloaded later. The problem is that I need the value of $i to set the values of $location and $number, but when I set the variable within the while loop the variables are not set. I've tried setting it locally and globally with single quotes, double quotes, passing through with the sshpass, exporting it -This works as a test but $i is of course unknown- tried placing brackets, curly braces, parentheses, escaping $, at this point I have exhausted my ideas, it's probably something incredibly simple, but I could use a fresh pair of eyes, any help is greatly appreciated!
EDIT:
Thanks to Charles Duffy for helping me clean it up so this is what I have now:
#!/bin/bash
sudo /usr/local/bin/sshpass -p support ssh -o stricthostkeychecking=no -p 11022 admin#$1.noerrpole.com<<'EOI'
i=1
while read -r line
do
location=sudo sed -n ${i}p "/Users/Shared/$1.txt"
number=sudo sed -n ${i}p "/Users/Shared/$2n.txt"
my_array+=( "$line" )
sudo cp "/Applications/TEPS\ OS\ X\ Share\ Folder/MAIN\ IMAGES\ FOLDER\ ƒ/${location}${number}" "/Users/Shared/FYP/$number"
sudo sips -Z 256 "/Users/Shared/FYP/$number" "/Users/Shared/FYP/$number"
((i++))
exit
done <"/Users/Shared/$2.txt"
EOI
But now $2 isn't getting passed through to the loop here's what I get back
1:bin Photo$ bash -x thumb npco2 20131216154714
+ sudo /usr/local/bin/sshpass -p support ssh -o stricthostkeychecking=no -p 11022 admin#npco2.noerrpole.com
Pseudo-terminal will not be allocated because stdin is not a terminal.
SHPA_12-16-2013/
sed: /Users/Shared/n.txt: No such file or directory
cp: /Applications/TEPS OS X Share Folder/MAIN IMAGES FOLDER ƒ/ is a directory (not copied).
Warning: /Users/Shared/FYP/ not a valid file - skipping
Warning: /Users/Shared/FYP/ not a valid file - skipping
Error 4: no file was specified
Try 'sips --help' for help using this tool
So where $2 should equal 20131216154714 it's returning an empty string like this
sed: /Users/Shared/n.txt: No such file or directory
The correct command would be
sed: /Users/Shared/20131216154714n.txt
The rest is just failing because $2 isn't passed.
Again thanks for the help!
ssh ... <<EOI does expansion on the local end, before starting ssh. Use ssh ... <<'EOI' to do expansions on the remote end.
If you want to pass arguments, use printf '%q ' to quote them so they survive remote unescaping intact:
printf -v quoted_args '%q ' "$one" "$two"
ssh user#host "bash -s - ${quoted_args}" <<<'EOI'
...
EOI
I have a set of scripts that I use to download files via FTP and then delete them from the server.
It works as follows:
for dir in `ls /volume1/auto_downloads/sync-complete`
do
if [ "x$dir" != *"x"* ]
then
echo "DIR: $dir"
echo "Moving out of complete"
# Soft delete from server so they don't get downloaded again
ssh dan#172.19.1.15 mv -v "'/home/dan/Downloads/complete/$dir'" /home/dan/Downloads/downloaded
Now $dir could be "This is a file" which works fine.
The problem I'm having is with special characters eg:
"This is (a) file"
This is a file & stuff"
tend to error:
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `mv -v '/home/dan/Downloads/complete/This is (a) file' /home/dan/Downloads/downloaded'
I can't work out how to escape it so both the variable gets evaluated and the command gets escaped properly. I've tried various combinations of escape characters, literal quotes, normal quotes, etc
If both sides are using bash, you can escape the arguments using printf '%q ', eg:
ssh dan#172.19.1.15 "$(printf '%q ' mv -v "/home/dan/Downloads/complete/$dir" /home/dan/Downloads/downloaded)"
You need to quote the whole expression ssh user#host "command":
ssh dan#172.19.1.15 "mv -v /home/dan/Downloads/complete/$dir /home/dan/Downloads/downloaded"
I'm confused, because your code as written works for me:
> dir='foo & bar (and) baz'
> ssh host mv -v "'/home/dan/Downloads/complete/$dir'" /home/dan/Downloads/downloaded
mv: cannot stat `/home/dan/Downloads/complete/foo & bar (and) baz': No such file or directory
For debugging, use set -vx at the top of the script to see what's going on.
Will Palmer's suggestion of using printf is great but I think it makes more sense to put the literal parts in printf's format.
That way, multi-command one-liners are more intuitive to write:
ssh user#host "$(printf 'mkdir -p -- %q && cd -- "$_" && tar -zx' "$DIR")"
One can use python shlex.quote(s) to
Return a shell-escaped version of the string s
docs
The script uses scp to upload a file. That works.
Now I want to log in with ssh, cd to the directory that holds the uploaded file, do an md5sum on the file. The script keeps telling me that md5sum cannot find $LOCAL_FILE. I tried escaping: \$LOCAL_FILE. Tried quoting the EOI: <<'EOI'. I'm partially understanding this, that no escaping means everything happens locally. echo pwd unescaped gives the local path. But why can I do "echo $MD5SUM > $LOCAL_FILE.md5sum", and it creates the file on the remote machine, yet "echo md5sum $LOCAL_FILE > md5sum2" does not work? And if it the local md5sum, how do I tell it to work on the remote?
scp "files/$LOCAL_FILE" "$i#$i.567.net":"$REMOTE_FILE_PATH"
ssh -T "$i#$i.567.net" <<EOI
touch I_just_logged_in
cd $REMOTE_DIRECTORY_PATH
echo `date` > I_just_changed_directories
echo `whoami` >> I_just_changed_directories
echo `pwd` >> I_just_changed_directories
echo "$MD5SUM" >> I_just_changed_directories
echo $MD5SUM > $LOCAL_FILE.md5sum
echo `md5sum $LOCAL_FILE` > md5sum2
EOI
You have to think about when $LOCAL_FILE is being interpreted. In this case, since you've used double-quotes, it's being interpreted on the sending machine. You need instead to quote the string in such a way that $LOCAL_FILE is in the command line on the receiving machine. You also need to get your "here document" correct. What you show just sends the output to touch to the ssh.
What you need will look something like
ssh -T address <'EOF'
cd $REMOTE_DIRECTORY_PATH
...
EOF
The quoting rules in bash are somewhat arcane. You might want to read up on them in Mendel Cooper's Advanced Guide to Bash Scripting.