Prevent shell from escaping single quotes - bash

Script:
#!/bin/sh -x
ARGS=""
CMD="./run_this_prog"
. . .
ARGS="-first_args '-A select[val]' "
. . .
$CMD $ARGS
I want the commandline to be expanded like this when I run this shell script:
./run_this_prog -first_args '-A select[val]'
Instead what shell does (note the added '\' before each single quote):
+ ARGS=
+ CMD='./run_this_prog'
+ ARGS='-first_args '\''-A select[val]'\'' '
and what it ran on commandline (escaped every special char - Not what I want):
./run_this_prog -first_args \'\-A select\[val\]\'
I tried escaping single quotes like :
ARGS="-first_args \'-A select[val]\' "
But that resulted in (added '\' after each backslash):
+ ARGS=
+ CMD='./run_this_prog'
+ ARGS='-first_args \'\''-A select[val]\'\'' '
I did my googling but couldn't find anything relevant. What am I missing here?
I am using sh-3.2 on rel6 centOS.

Once a quote is inside a string, it will not work the way you want: Inside a string quotes are not syntactic elements, they are just literal characters. This is one reason why bash offers arrays.
Replace:
#!/bin/sh -x
...
ARGS="-first_args '-A select[val]' "
$CMD $ARGS
With:
#!/bin/bash -x
...
ARGS=(-first_args '-A select[val]')
"$CMD" "${ARGS[#]}"
For a much more detailed discussion of this issue, see: "I'm trying to put a command in a variable, but the complex cases always fail!"

Related

How do you get GNU Parallel to parse quoted command line arguments?

This is one sample program in the GNU parallel documentation for executing via the shell script shebang.
#!/usr/bin/parallel --shebang-wrap --colsep " " /bin/bash
echo Arguments: $#
The output for
./bash_echo.parl gracias 'buenos dias'
is
gracias
buenos
dias
The above script does not handle command line arguments that are quoted and contain spaces. The arguments are expanded instead and treated as individual inputs.
How do I obtain the correct output as for the bash script below?
#!/usr/bin/env bash
for i in "$#"; do
echo "$i"
done
This, obviously, handles quoted command line args.
Output:
gracias
buenos dias
I've tried using the option 'colseps' setting the separator to ' ' but that isn't the solution.
You have found a bug: --shebang-wrap was never tested with spaces.
Possible fix:
diff --git a/src/parallel b/src/parallel
index 69adfdac..e7c0d930 100755
--- a/src/parallel
+++ b/src/parallel
## -3302,9 +3302,10 ## sub read_options() {
#options = shift #ARGV;
}
my $script = Q(shift #ARGV);
+ my #args = map{ Q($_) } #ARGV;
# exec myself to split $ARGV[0] into separate fields
- exec "$0 --_pipe-means-argfiles #options #parser $script ".
- "::: #ARGV";
+ exec "$0 --_pipe-means-argfiles #options #parser $script ".
+ "::: #args";
}
}
if($ARGV[0] =~ / --shebang(-?wrap)? /) {
It seems to fix your issue, but it may introduce others.
For updates: Follow https://savannah.gnu.org/bugs/index.php?63703

Escape double quotes in a Jenkins pipeline file's shell command

Below is a snippet from my Jenkins file -
stage('Configure replication agents') {
environment {
AUTHOR_NAME="XX.XX.XX.XX"
PUBLISHER_NAME="XX.XX.XX.XX"
REPL_USER="USER"
REPL_PASSWORD="PASSWORD"
AUTHOR_PORT="4502"
PUBLISHER_PORT="4503"
AUTHOR="http://${AUTHOR_NAME}:${AUTHOR_PORT}"
PUBLISHER="http://${PUBLISHER_NAME}:${PUBLISHER_PORT}"
S_URI= "${PUBLISHER}/bin/receive?sling:authRequestLogin=1"
}
steps {
sh 'curl -u XX:XX --data "status=browser&cmd=createPage&label=${PUBLISHER_NAME}&title=${PUBLISHER_NAME}&parentPath =/etc/replication/agents.author&template=/libs/cq/replication/templates/agent" ${AUTHOR}/bin/wcmcommand'
}
The above command, in Jenkins console, is printed as
curl -u XX:XX --data status=browser&cmd=createPage&label=XXXX&title=XXX&parentPath =/etc/replication/agents.author&template=/libs/cq/replication/templates/agent http://5XXXX:4502/bin/wcmcommand
Note how the double quotes "" are missing.
I need to preserve the double quotes after --data in this command. How do I do it?
I tried using forward slashes but that didnt work.
Cheers
To expand on my comment, a quick test revealed its the case.
You need to escape twice, once the quote for the shell with a slash, and once that slash with a slash for groovy itself.
node() {
sh 'echo "asdf"'
sh 'echo \"asdf\"'
sh 'echo \\"asdf\\"'
}
Result
[Pipeline] {
[Pipeline] sh
+ echo asdf
asdf
[Pipeline] sh
+ echo asdf
asdf
[Pipeline] sh
+ echo "asdf"
"asdf"
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
After long time of struggling and googling, this is what has worked for me on similar use case:
sh("ssh root#my.server.com \"su user -c \\\"mkdir ${newDirName}\\\"\"")
Update: How I think it gets interpreted
1] sh extension strips first escaping (\" becomes " and \\ becomes \, first and last " are not part of input)
ssh root#my.server.com "su user -c \"mkdir ${newDirName}\""
2] ssh command strips second level of escaping (\" becomes ", while outer " also not part of input)
su user -c "mkdir ${newDirName}"
I had double quotes inside the variable, so escaped single quotes worked for me:
sh "git commit -m \'${ThatMayContainDoubleQuotes}\'"
I needed the output to be with trailing \\ so I had to do something like this
echo 'key1 = \\\\"__value1__\\\\"' > auto.file
File looks like
cat auto.file
key1 = \\"__value1__\\"
Dependent Script
export value1="some-value"
var=${value1}
# Read in template one line at the time, and replace variables
tmpfile=$(mktemp)
sed -E 's/__(([^_]|_[^_])*)__/${\\1}/g' auto.file > ${tmpfile}
while read auto
do
eval echo "$auto"
done < "${tmpfile}" > autoRendered.file
rm -f ${tmpfile}
Rendered File looks like
cat autoRendered.file
key1 = "some-value"
For anyone who comes looking for a fix to a similar issue with quoting numbers during helm install/upgrade, you can use --set-string instead of --set
Ref: https://helm.sh/docs/chart_best_practices/values/#consider-how-users-will-use-your-values

How can i use system() with rxrepl in WinCC OA?

I try to use:
string result;
string path = "C:/winccoa.projects/filters/bin/tools/rxrepl.exe";
string cmd = "'opcki' | " + path + " -s 'op' -r 'tata'";
system(cmd, result);
DebugN(result);
But in LogViewer i see nothing, instead ["tatacki"]
Why? What i doing wrong?
In PowerShell that works fine:
PS C:\> 'opcki' | C:/winccoa.projects/filters/bin/tools/rxrepl.exe -s "op" -r "tata"
tatacki
I'm assuming that WinCC's system() function targets cmd.exe, not powershell.exe (which is typical, because historically cmd.exe has been the default shell, and APIs are unlikely to change, so as to maintain backward compatibility).
Therefore, formulate your command for cmd.exe:
string cmd = "echo opcki | " + path + " -s op -r tata";
Not the use of echo to produce output and the omission of single-quoting ('...'), which cmd.exe doesn't recognize.
If embedded quoting were needed, you'd have to use `" inside "..." PowerShell strings (or use '...' PowerShell strings (whose content is taken literally) and embed " chars. as-is).

bash script - unable to set variable with double quotes in value

Need help in fixing this bash script to set a variable with a value including double quotes. Somehow I am defining this incorrectly as my values foo and bar are not enclosed in double quotes as needed.
My script thus far:
#!/usr/local/bin/bash
set -e
set -x
host='127.0.0.1'
db='mydev'
_account="foo"
_profile="bar"
_version=$1
_mongo=$(which mongo);
exp="db.profile_versions_20170420.find({account:${_account}, profile:${_profile}, version:${_version}}).pretty();";
${_mongo} ${host}/${db} --eval "$exp"
set +x
Output shows:
+ host=127.0.0.1
+ db=mydev
+ _account=foo
+ _profile=bar
+ _version=201704112004
++ which mongo
+ _mongo=/usr/local/bin/mongo
+ exp='db.profile_versions_20170420.find({account:foo, profile:bar, version:201704112004}).pretty();'
+ /usr/local/bin/mongo 127.0.0.1/mydev --eval 'db.profile_versions_20170420.find({account:foo, profile:bar, version:201704112004}).pretty();'
MongoDB shell version: 3.2.4
connecting to: 127.0.0.1/mydev
2017-04-22T15:32:55.012-0700 E QUERY [thread1] ReferenceError: foo is not defined :
#(shell eval):1:36
What i need is account:"foo", profile:"bar" to be enclosed in double quotes.
In bash (and other POSIX shells), the following 2 states are equivalent:
_account=foo
_account="foo"
What you want to do is to preserve the quotations, therefore you can do the following:
_account='"foo"'
Since part of what you're doing here is forming JSON, consider using jq -- which will guarantee that it's well-formed, no matter what the values are.
host='127.0.0.1'
db='mydev'
_account="foo"
_profile="bar"
_version=$1
json=$(jq -n --arg account "$_account" --arg profile "$_profile" --arg version "$_version" \
'{$account, $profile, version: $version | tonumber}')
exp="db.profile_versions_20170420.find($json).pretty();"
mongo "${host}/${db}" --eval "$exp"
This makes jq responsible for adding literal quotes where appropriate, and will avoid attempted injection attacks (for instance, via a version passed in $1 containing something like 1, "other_argument": "malicious_value"), by replacing any literal " in a string with \"; a literal newline with \n, etc -- or, with the | tonumber conversion, failing outright with any non-numeric value.
Note that some of the syntax above requires jq 1.5 -- if you have 1.4 or prior, you'll want to write {account: $account, profile: $profile} instead of being able to write {$account, $profile} with the key names inferred from the variable names.
When you need to use double quotes inside a double quoted string, escape them with backslashes:
$ foo="acount:\"foo\"" sh -c 'echo $foo'
acount:"foo"
I needed to enquote something already in a variable and stick that in a variable. Expanding on Robert Seaman's answer, I found this worked:
VAR='"'$1'"'
(single quote, double quote, single quote, variable,single quote, double quote, single quote)

How to preserve single and double quotes in shell script arguments WITHOUT the ability to control how they pass

I need to receive arguments I have no control over into a shell script, and preserve any single or double quotes. For instance, a script that simply outputs the given arguments should act as follows:
> my_script.sh "double" 'single' none
"double" 'single' none
I don't have the privilege of augmenting the arguments such as in:
> my_script.sh \"double\" \'single\' none
or
> my_script.sh '"double"' "'single'" none
And neither "$#" nor "$*" work.
I also thought of reading from STDIN and try something like:
> echo "double" 'single' none | my_script.sh
thinking it may help, but no breakthrough so far.
Any suggestions?
CSH / PERL solutions are welcomed.
This is not possible (without escaping), because the shell processes the arguments and removes the quotes before your script is called. As a result, your script does not know about the quotes specified on the command line.
You cannot recover the single/double quotes exactly as they were, because the shell 'eats' them. If you need to call some other script from your script, you can e.g. single quote the arguments again. Here is a PERL solution I use:
sub args2shell
{
local (#argv) = #_;
local $" = '\' \'';
local (#margv);
#margv = map { s/'/'\\''/g; $_ } #argv;
return "\'#margv\'" if #margv;
return undef;
}
Example usage:
$args = args2shell #ARGV;
open F, "find -follow $args ! -type d |";
...

Resources