Iterate over a list of quoted strings - bash

Question

I'm trying to run a for loop over a list of strings where some of them are quoted and others are not like so:
STRING='foo "bar_no_space" "baz with space"'
for item in $STRING; do
echo "$item"
done
Expected result:
foo
bar_no_space
baz with space
Actual result:
foo
"bar_no_space"
"baz
with
space"
I can achieve the expected result by running the following command:
bash -c 'for item in '"$STRING"'; do echo "$item"; done;'
I would like to do this without spawning a new bash process or using eval because I do not want to take the risk of having random commands executed.
Please note that I do not control the definition of the STRING variable, I receive it through an environment variable. So I can't write something like:
array=(foo "bar_no_space" "baz with space")
for item in "${array[#]}"; do
echo "$item"
done
If it helps, what I am actually trying to do is split the string as a list of arguments that I can pass to another command.
I have:
STRING='foo "bar_no_space" "baz with space"'
And I want to run:
my-command --arg foo --arg "bar_no_space" --arg "baz with space"

Answer 1

Use an array instead of a normal variable.
arr=(foo "bar_no_space" "baz with space")
To print the values:
print '%s\n' "${arr[#]}"
And to call your command:
my-command --arg "${arr[0]}" --arg "${arr[1]}" --arg "{$arr[2]}"

Answer 2

Can you try something like this:
sh-4.4$ echo $string
foo "bar_no_space" "baz with space"
sh-4.4$ echo $string|awk 'BEGIN{FS="\""}{for(i=1;i<NF;i++)print $i}'|sed '/^ $/d'
foo
bar_no_space
baz with space

Answer 3

Solved: xargs + subshell
A few years late to the party, but...
Malicious Input:
SSH_ORIGINAL_COMMAND='echo "hello world" foo '"'"'bar'"'"'; sudo ls -lah /; say -v Ting-Ting "evil cackle"'
Note: I originally had an rm -rf in there, but then I realized that would be a recipe for disaster when testing variations of the script.
Converted perfectly into safe args:
# DO NOT put IFS= on its own line
IFS=$'\r\n' GLOBIGNORE='*' args=($(echo "$SSH_ORIGINAL_COMMAND" \
| xargs bash -c 'for arg in "$#"; do echo "$arg"; done'))
echo "${args[#]}"
See that you can indeed pass these arguments just like $#:
for arg in "${args[#]}"
do
echo "$arg"
done
Output:
hello world
foo
bar;
sudo
rm
-rf
/;
say
-v
Ting-Ting
evil cackle
I'm too embarrassed to say how much time I spent researching this to figure it out, but once you get the itch... y'know?
Defeating xargs
It is possible to fool xargs by providing escaped quotes:
SSH_ORIGINAL_COMMAND='\"hello world\"'
This can make a literal quote part of the output:
"hello
world"
Or it can cause an error:
SSH_ORIGINAL_COMMAND='\"hello world"'
xargs: unmatched double quote; by default quotes are special to xargs unless you use the -0 option
In either case, it doesn't enable arbitrary execution of code - the parameters are still escaped.

Answer 4

Pure bash parser
Here's a quoted-string parser written in pure bash (what terrible fun)!
Caveat: just like the xargs example above, this errors in the case of an escaped quoted.
Usage
MY_ARGS="foo 'bar baz' qux * "'$(dangerous)'" sudo ls -lah"
# Create array from multi-line string
IFS=$'\r\n' GLOBIGNORE='*' args=($(parseargs "$MY_ARGS"))
# Show each of the arguments array
for arg in "${args[#]}"; do
echo "$arg"
done
Output:
$#: foo bar baz qux *
foo
bar baz
qux
*
Parse Argument Function
Literally going character-by-character and adding to the current string, or adding to the array.
set -u
set -e
# ParseArgs will parse a string that contains quoted strings the same as bash does
# (same as most other *nix shells do). This is secure in the sense that it doesn't do any
# executing or interpreting. However, it also doesn't do any escaping, so you shouldn't pass
# these strings to shells without escaping them.
parseargs() {
notquote="-"
str=$1
declare -a args=()
s=""
# Strip leading space, then trailing space, then end with space.
str="${str## }"
str="${str%% }"
str+=" "
last_quote="${notquote}"
is_space=""
n=$(( ${#str} - 1 ))
for ((i=0;i<=$n;i+=1)); do
c="${str:$i:1}"
# If we're ending a quote, break out and skip this character
if [ "$c" == "$last_quote" ]; then
last_quote=$notquote
continue
fi
# If we're in a quote, count this character
if [ "$last_quote" != "$notquote" ]; then
s+=$c
continue
fi
# If we encounter a quote, enter it and skip this character
if [ "$c" == "'" ] || [ "$c" == '"' ]; then
is_space=""
last_quote=$c
continue
fi
# If it's a space, store the string
re="[[:space:]]+" # must be used as a var, not a literal
if [[ $c =~ $re ]]; then
if [ "0" == "$i" ] || [ -n "$is_space" ]; then
echo continue $i $is_space
continue
fi
is_space="true"
args+=("$s")
s=""
continue
fi
is_space=""
s+="$c"
done
if [ "$last_quote" != "$notquote" ]; then
>&2 echo "error: quote not terminated"
return 1
fi
for arg in "${args[#]}"; do
echo "$arg"
done
return 0
}
I may or may not keep this updated at:
https://git.coolaj86.com/coolaj86/git-scripts/src/branch/master/git-proxy
Seems like a rather stupid thing to do... but I had the itch... oh well.

Answer 5

Here is a way without an array of strings or other difficulties (but with bash calling and eval):
STRING='foo "bar_no_space" "baz with space"'
eval "bash -c 'while [ -n \"\$1\" ]; do echo \$1; shift; done' -- $STRING"
Output:
foo
bar_no_space
baz with space
If You want to do with the strings something more difficult then just echo You can split Your script:
split_qstrings.sh
#!/bin/bash
while [ -n "$1" ]
do
echo "$1"
shift
done
Another part with more difficult processing (capitalizing of a characters for example):
STRING='foo "bar_no_space" "baz with space"'
eval "split_qstrings.sh $STRING" | while read line
do
echo "$line" | sed 's/a/A/g'
done
Output:
foo
bAr_no_spAce
bAz with spAce