I want to do a substitution in /etc/bashrc remotely via SSH on multiple hosts.
ssh myhost sed 's/\\u\#\\h/\\u\#\\h (myhost)/' /etc/bashrc
gives me this error:
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `sed s/\\u\#\\h/\\u\#\\h (myhost)/ /etc/bashrc'
It runs without error directly on myhost.
The single quotes are removed by the local shell, so that the remote host sees the following command line:
sed s/\\u\#\\h/\\u\#\\h (myhost)/ /etc/bashrc
You need an extra layer of quoting on the local side, which requires extra escaping of the backslashes as well.
ssh myhost "sed 's/\\\\u\\#\\\\h/\\\\u\\#\\\\h (myhost)/' /etc/bashrc"
try adding
(
before the brackets also you don't have to use:
sed 's/something/another/'
you can use anything like:
sed 's:something:another:'
sed 's|something|another|'
or a few others to do the separation which gets rid of having to use backslashes for when using
//folder/
Related
I am trying to add an argument to the flag batch start. This is the error it gives me. Any idea on how to fix this?
$ sed -i "s/batch_start.*/batch_start\ 1111/" /tmp/runfile
sed: -e expression #1, char 27: unterminated `s' command
The root problem is that the command is being sent over ssh; that means it's running through two levels of shell parsing, one on the local computer, then another on the remote computer. That means it goes through quote/escape parsing, application, and removal twice, so you need two "layers" of quoting/escaping.
The command in the last comment doesn't parse (mismatched quotes), but I can reproduce the error message with this command:
ssh remoteHost "sudo sed -i "s/batch_start.*/batch_start\ 1111/" /tmp/runfile"
This sort-of has two levels of quotes, but quotes don't nest, so it doesn't work. The local shell parses this as a double-quoted string "sudo sed -i ", then an unquoted section s/batch_start.*/batch_start\ 1111/ (which contains an escaped space, so it'll remove the escape), then another double-quoted section: " /tmp/runfile". Since there are no spaces between them, they all get passed to ssh as a single argument. You can see the post-parsing string by replacing ssh remoteHost with echo:
$ echo "sudo sed -i "s/batch_start.*/batch_start\ 1111/" /tmp/runfile"
sudo sed -i s/batch_start.*/batch_start 1111/ /tmp/runfile
...so that's the command the remote shell will execute. Since there's a space between s/batch_start.*/batch_start and 1111/, they get passed to sed as separate arguments, and it treats the first as the command to execute (which is missing a close /) and the second as a filename.
Solution: there are many ways to correct the quoting. You can use the echo trick to see what'll get sent to the remote shell. I tend to prefer single-quotes around the entire command, and then just quote normally inside that (as long as the inner command doesn't itself contain single-quotes). In this case, that means:
ssh remoteHost 'sudo sed -i "s/batch_start.*/batch_start 1111/" /tmp/runfile'
which executes this on the remote computer:
sudo sed -i "s/batch_start.*/batch_start 1111/" /tmp/runfile
(note that I removed the escape on the space.)
I am trying to add a user "demouser" to /etc/sudoers of a remote server and I want to pass the username from a variable.
This works, but I want to use a variable $USERNAME instead of demouser
ssh centos#$remote_host -t 'sudo sed -i "\$ademouser ALL=(ALL) NOPASSWD:ALL" /etc/sudoers'
I tried using this but it's not working.
export USERNAME=demouser
ssh centos#remote_host bash -c "'sudo sed -i "\$a$USERNAME ALL=(ALL) NOPASSWD:ALL" /etc/sudoers'"
Error: -bash: syntax error near unexpected token `('
Parameters will not expand in single quotes, one can close them, and expand in double quotes instead:
ssh user#host 'sed "s/'"$localVar"'/replacement/" file'
^^
|Enter double quotes to avoid word splitting and globbing
Exit single quotes to expand on client side.
You should however know that the command send to the server is:
sed "s/abc/replacement/" file
Which might cause problems as we are now using double quotes on the server, one can send single quotes as well, but it quickly becomes as mess:
ssh user#host 'sed '\''s/'"$localVar"'/replacement/'\'' file'
^ ^
| Escaped remote single quote
Close local single quote
This will become:
sed 's/abc/replacement' file
I have a command like this:
ssh user#hostname 'sed -e "s|foo|${bar}|" /home/data/base_out.sql > /home/data/out.sql'
The sed command is working in local shell. But it is not expanding the variable over ssh command. Thanks!
The rule is that within single quotes, parameters are not expanded. You have single quotes around the entire command.
Try this:
ssh user#hostname "sed -e 's|foo|$bar|' /home/data/base_out.sql > /home/data/out.sql"
Now $bar is expanded before the command string is passed as an argument to ssh, which is what you want.
I removed the curly braces around ${bar} because I believe they offer a false sense of security. In this case, they are not protecting you against any of the issues associated using shell variables in sed commands.
I need pass $var_path to bash script inside single quotes and then get commnd executed on remote host. I know that single quotes do not allow passing variables, so tried double quoting, but getting error in sed. Assume this happens because its template uses " as well.
var="Test text"
var_path="/etc/file.txt"
echo "$var"|ssh root#$host 'cat - > /tmp/test.tmp && sed -n "/]/{:a;n;/}/b;p;ba}" $var_path > /tmp/new.conf.tmp'
so with ""
var="Test text"
var_path="/etc/file.txt"
echo "$var"|ssh root#$host "cat - > /tmp/test.tmp && sed -n "/]/{:a;n;/}/b;p;ba}" $var_path > /tmp/new.conf.tmp"
Errors from sed
sed: -e expression #1, char 0: unmatched `{'
./script.sh: line 4: n: command not found
./script.sh: line 4: /}/b: No such file or directory
./script.sh: line 4: p: command not found
If $var_path used directly without substitution script works as expected.
Arguments parsed as part of the command to send to the remote system in SSH in the are concatenated with spaces and then passed to the remote shell (in a manner similar to "${SHELL:-sh}" -c "$*"). Fortunately, bash has the built-in printf %q operation (an extension, so not available in all other POSIX shells) to make strings eval-safe, and thus ssh-safe, if your remote SHELL is bash; see the end of this answer for a workaround using Python's shlex module to generate a command safe in non-bash shells.
So, if you have a command that works locally, the first step is to put it into an array (notice also the quotes around the expansion of "$var_path" -- these are necessary to have an unambiguous grouping):
cmd=( sed -n '/]/{:a;n;/}/b;p;ba}' "$var_path" )
...which you can run locally to test:
"${cmd[#]}"
...or transform into an eval-safe string:
printf -v cmd_str '%q ' "${cmd[#]}"
...and then run it locally with ssh...
ssh remote_host "$cmd_str"
...or test it locally with eval:
eval "$cmd_str"
Now, your specific use case has some exceptions -- things that would need to be quoted or escaped to use them as literal commands, but which can't be quoted if you want them to retain their special meaning. &&, | and > are examples. Fortunately, you can work around this by building those parts of the string yourself:
ssh remote_host "cat - >/tmp/test.tmp && $cmd_str >/tmp/new.conf.tmp"
...which is equivalent to the local array expansion...
cat - >/tmp/test.tmp && "${cmd[#]}" >/tmp/new.conf.tmp
...or the local eval...
eval "cat - >/tmp/test.tmp && $cmd_str >/tmp/new.conf.tmp"
Addendum: Supporting Non-Bash Remote Shells
One caveat: printf %q is guaranteed to quote things in such a way that bash (or ksh, if you're using printf %q in ksh locally) will evaluate them to exactly match the input. If you had a target system with a shell which didn't support extensions to POSIX such as $'', this guarantee would no longer be available, and more interesting techniques become necessary to guarantee robustness in the face of the full range of possible inputs.
Fortunately, the Python standard library includes a function to escape arbitrary strings for any POSIX-compliant shell:
quote_string() {
python -c '
import sys
try:
from pipes import quote # Python 2
except ImportError:
from shlex import quote # Python 3
print(" ".join([quote(x) for x in sys.argv[1:]]))
' "$#"
}
Thereafter, when you need an equivalent to printf '%q ' "$#" that works even when the remote shell is not bash, you can run:
cmd_str=$(quote_string "${cmd[#]}")
Once you use double quotes the embedded command can simply use single quotes. This works because double quote variable substitutions treat embedded single quotes as regular characters... nothing special.
var="Test text"
var_path="/etc/file.txt";
echo "$var"|ssh root#$host "cat - > /tmp/test.tmp && sed -n '/]/{:a;n;/}/b;p;ba}' $var_path > /tmp/new.conf.tmp"
If you wanted to assign and use variables within the double quotes that would be more tricky.
root#anywhere is dangerous... is there no other way? If you are using certificates I hope they restrict root to specific commands.
This question already has answers here:
Escape a string for a sed replace pattern
(17 answers)
Closed 9 years ago.
I have a sed command in a bash script like this:
sed -i 's/db.url=.*/db.url='$URL'/' config.properties
and URL var is assigned as:
$URL=jdbc\:oracle\:thin\:#\/\/hostname\:12345\/XYZ
When I run this bash script on the host it exists on it work as intended, replacing the url with the one specified in URL. However, when I add a command in the bash script to do this on my other host like this:
ssh user#host02 <<EOF
sed -i 's/db.url=.*/db.url='$URL'/' config.properties
exit
EOF
I get this error:
sed: -e expression #1, char 47: unknown option to `s'
Anyone know what may be going on here?
You've properly quote the sed expression if it were running on the local host, but the string is then passed to the shell on the remote host, where the * is now unquoted and expanded as a glob. The simplest thing to do is to pipe the command to the remote shell via standard input, so you don't have to worry about quoting:
echo "sed -i 's/db.url=.*/db.url=$URL/' config.properties" | ssh user#host02 bash
With multiple commands, you may consider using a here document:
ssh user#host02 bash <<EOF
command1
sed -i 's/db.url=.*/db.url=$URL/' config.properities
command2
EOF
The solution was to use double single quotes (') around $URL like this: ''$URL''