I'm working through Red Hat Academy and one of the sample scripts provided differed from mine in a key way that I haven't been able to understand:
#!/usr/bin/bash
#
USR='student'
OUT='/home/student/output'
#
for SRV in servera serverb
do
ssh ${USR}#${SRV} "hostname -f" > ${OUT}-${SRV}
echo "#####" >> ${OUT}-${SRV}
ssh ${USR}#${SRV} "lscpu | grep '^CPU'" >> ${OUT}-${SRV}
echo "#####" >> ${OUT}-${SRV}
ssh ${USR}#${SRV} "grep -v '^$' /etc/selinux/config|grep -v '^#'" >> ${OUT}-${SRV}
echo "#####" >> ${OUT}-${SRV}
ssh ${USR}#${SRV} "sudo grep 'Failed password' /var/log/secure" >> ${OUT}-${SRV}
echo "#####" >> ${OUT}-${SRV}
done
On each line after the variable expansion, they've wrapped most of the rest of the command in double quotes. Now, I understand the usage of double quotes and single quotes when it comes to suppressing expansion/substitution, but the main takeaway I've learned is straight from the RH Academy:
It is recommended practice to use single quotation marks to
encapsulate the regular expression to protect any shell metacharacters
(such as the $, *, and {} characters). Encapsulating the regular
expression ensures that the characters are interpreted by the intended
command and not by the shell.
I can see that they're doing that here to make sure the regular expressions are properly passed to grep in a couple of lines, but I can't figure out the purpose of the double quotes. My questions are:
Why are they there?
Why are they only wrapped around a portion of
each line?
Is this a common practice?
As pointed out by others, the example provided has issues of being malformed for the intended purpose. Specifically, the 3rd ssh requires the "$" in '^$' to be escaped, to avoid the shell's attempt at substitution at your end. Otherwise, the remote host will see '^' !!!
Their usage of double-quotes is intended to "encapsulate" a construct meant to be executed as is on the remote server, hence around everything after the target host reference.
Issues arise if you need to expand variable values before sending to remote ... AND ... when you need such expansion to NOT occur. Single-quotes are meant to contain literals meant to be passed as-is, but if that is contained within another set of double-quotes, the contents of those are not "protected" and the shell attempts the usual substitutions whenever "$" is encountered.
For simple constructs, one-line commands are OK.
BUT ... when things get more complex (with multiple levels of "\" for escaping single-/double-quotes), it might be best to create a job-script, massage that into what is required for the remote host before calling ssh, then have ssh reference that in the following manner:
ssh user#remote 'bash -s' < local_script.sh
As for the line doing the check for failed password, I would never leave that to the end. I would have that as the first line, check the output, and abandon if the password failed, avoiding the likely failed attempts for the other commands, which at that point serves no purpose.
Related
I have a bash script that runs another script in a screen on a remote computer. The environment variable GITLAB_CI_TOKEN is set on the host machine and is defined properly. However, the script configure.sh on the remote machine tells that this environment variable is empty when it is executed, even if it is defined on the same line as the script...
Here is the command I am using:
ssh -o "StrictHostKeyChecking=accept-new" "${COMPUTERS_IPS[i]}" \
screen -S "deploy_${COMPUTERS_IPS[i]}" -dm " \
GITLAB_CI_TOKEN=${GITLAB_CI_TOKEN} \
bash \"${REMOTE_FOLDER}/configure.sh\" \"${REMOTE_FOLDER}\" > ${LOG_FILE} 2>&1;
"
Additionally, the logs are not being written to LOG_FILE, but are being displayed on the console of the screen. I have been pulling my hair out over this for the past two days... Any help or guidance would be greatly appreciated :)
Why GITLAB_CI_TOKEN is "empty":
Passing a command to a remote host over ssh is very similar to running it through eval. For example in your case, escaped newlines on the first evaluation become unescaped newlines on a subsequent evaluation. Consider this very simple program named args (place it in bin or somewhere else on your path to demo):
#!/bin/bash
for arg ; do
echo "|$arg|"
done
And these two use cases:
args "\
Hello \
World"
# prints:
# |Hello World|
ssh host args "\
Hello \
World"
# prints:
# |Hello|
# |World|
As you can see, when we run this via ssh the newline we attempted to escape splits our data into two separate lines even though we tried to keep it all on one line. This means your assignment of GITLAB_CI_TOKEN is just a regular shell variable instead of a scoped environment variable for your bash command. A scoped environment variable requires the declaration and the command to happen on the same line.
The easiest thing to do is likely to just export the variable explicitly with export GITLAB_CI_TOKEN=${GITLAB_CI_TOKEN}.
For similar reasons, the output of your command is going to the screen and not the logfile because the outer quotes of screen -dm "commands >output" are getting stripped on the first evaluation, and then the remote host is parsing screen -dm commands >output and assigning the output redirection to screen instead of commands. That means your configure.sh is writing to the screen, and it's the screen program that's writing its own output to a logfile.
To send complex commands to a remote host, you may want to look into tools like printf %q which can produce escaped output suitable for being safely evaluated in an eval-like context. Take a look at BashFAQ/096 for an example.
I sometimes use ssh to run multiple commands on a remote host like this:
ssh my_user#my_host "command1; command2"
utilizing double quotes to send both commands to the remote machine.
Now I would like to access an environment variable in one or both of the commands.
Here is a simplified example that illustrates what I have tried:
ssh my_user#my_host "echo $USER; echo $HOSTNAME"
This echos my user/hostname on the local machine, whereas I would like to see the names on the remote machine instead.
Using single quotes I can achieve what I want for a single command as follows:
ssh my_user#my_host echo '$USER'
but I don't get the behavior I want when I use single quotes inside the double quotes like this:
ssh my_user#my_host "echo '$USER'; echo '$HOSTNAME'"
How can I get the behavior I want with both commands being executed from a single ssh commmand?
(Please note that my actual commands are more complicated... I do realize that I can get both variables in a single command for my toy example as below, but this does not solve my real problem):
ssh my_user#my_host echo '$USER' '$HOST'
Just escape the $ using \.
ssh my_user#my_host "echo \$USER; echo \$HOSTNAME"
You can put the entire command to be run in single quotes, like so:
ssh my_user#my_host 'echo $USER; echo $HOSTNAME'
That will prevent your shell from substituting the environment variable values before passing them to the remote machine, but when they get to the remote machine they are unquoted so they get evaluated there.
Thanks to Guilherme Richter. That solution works! I will accept it once the 5 minutes has passed. I wanted to document that in the meantime I found another solution that avoids the double quotes instead of avoiding the single quotes by escaping the semicolon:
ssh user#host echo '$USER' \; echo '$USER'
I'm writing a script that would give me an ability to execute other local bash scripts remotely over SSH without uploading them. Some of my servers have Linux, some FreeBSD with csh as default. So far, I've came to the following:
ssh remote_server 'bash -c '\'"$(cat local_script.sh)"\'' script_parameters'
This allows me to execute local_script.sh on remote_server, supports interactivity (for example, "read" command will work in local_script.sh), and I can transfer positional parameters to the script. The problem is that if a local_script.sh has multiple lines the code above works for remote servers with bash default shell only. On FreeBSD sshd starts csh first to execute "bash -c" command and tries to pass the script code (the result of $(cat local_script.sh)) to it as the first parameter. But because this code is multiline and csh wants the closing quote be on the same line as the opening quote I get the "Unmatched '." error. So csh just can't parse this escaped multiline parameter to pass it to the bash process.
The only solution I can see now is to parse the local_script.sh and automatically rewrite it into a one-liner using ";" delimiters before passing it via SSH. But this requires creating another script and seems to be not so easy. So I'd like to ask if this csh miltiline parsing problem can be resolved somehow?
What about using standard input /dev/stdin ?
ssh remote_server bash /dev/stdin script_parameters < local_script.sh
Using two different commands and temporary file
bash
ssh remote_server $'tmp=`mktemp tmp.XXXXXX`;cat <<\'HEREDOC_END\' >"$tmp"
'"$(cat local_script.sh)"'
HEREDOC_END
bash "$tmp" script_parameters ;rm "$tmp"'
csh
ssh remote_server 'set tmp=`mktemp tmp.XXXXXX`;cat <<"HEREDOC_END" >"$tmp"
'"$(cat local_script.sh)"'
"HEREDOC_END"
bash "$tmp" script_parameters ;rm "$tmp"'
In this particular case, there is no need to keep the outermost quotes.
ssh remote_server bash -c "\"$(cat local_script.sh)\"" script_parameters
This is not entirely robust; for example, single quotes in local_script.sh will not quote text verbatim. So the following example
echo 'fnord "$(echo yes)"'
will have the command substitution evaluated remotely even though the string is in single quotes. In the general case, you'd have to replace cat with, basically, a shell parser which identifies constructs which need escaping.
I'm trying to send this command via ssh:
ssh <user1>#<ip1> ssh <user2>#<ip2> /opt/user/bin -f /opt/user/slap.conf -l /home/admin/`date +%Y%m%d`_Export_file$nr.gz -s "ou=multi" -a "(& (entry=$nr)(serv=PS))" -o wrap=no
this command is customized so do not confuse with this...
But it's not executed, smth like: unexpected '(
If i log in to the server and i give this command it gets executed correctly. So i think it should be something with bracket and parentheses rules.
Please can someone help me?
thank you in advance.
You will need to escape the quotes, possibly twice, since each invocation of ssh will involve stripping a layer off. Put escaped single quotes round the entire command, and then nested unescaped single quotes round the inner command:
ssh <user1>#<ip1> \'ssh <user2>#<ip2> '/opt/user/bin -f /opt/user/slap.conf -l /home/admin/`date +%Y%m%d`_Export_file$nr.gz -s "ou=multi" -a "(& (entry=$nr)(serv=PS))" -o wrap=no'\'
This assumes, by the way, that you want the backticks to be unpacked and the command executed on ip2, rather than beforehand on your source machine, and similarly with the decoding of the $nr variable. It's not clear how you want them interpreted.
I am writing a simple bash script (checkServs.sh) that will ssh into a list of servers and perform a health check on them.
I keep getting errors on the following line:
SERVERS=(blah1.example.com blah2.example.com blah3.example.com blah4.example.com)
Error is:
checkServs.sh: 3: checkServs.sh: Syntax error: "(" unexpected
I've checked online examples and this seems correct, isn't it? Thanks in advance!
I don't know about the syntax error, but this should work:
SERVERS="blah1.example.com blah2.example.com blah3.example.com blah4.example.com"
for server in $SERVERS
do
echo $server
done
EDIT: As noted by Jonathan Leffler in a comment, maybe you are not running the script with bash. Other shells, such as dash, may not recognize the array syntax. If that's the case, you can do:
SERVERS=(blah1.example.com blah2.example.com blah3.example.com blah4.example.com)
for i in $(seq 0 3)
do
echo ${SERVERS[$i]}
done
But if you just want to loop through the names and run an SSH command (ie if having an array won't provide useful functionality), the first method is more straightforward.
Your remote server probably calls a different shell when executing commands. Try to add bash -c to your arguments:
ssh user#server bash -c "<your commands>"
Or:
ssh user#server bash < yourscript.sh ## None in yourscript.sh must read input though.
An opening parenthesis starts a subshell, which is not a correct thing to have on the right side of an equals sign. It expects a string expression, not a command.
Quotation marks are used to keep a string expression together.