Is there a method in shell to ignore escape characters? - shell

In C#, there is a verbatim string so that,
string c = "hello \t world"; // hello world
string d = #"hello \t world"; // hello \t world
I am new to shell script, is there a similar method in shell?
Because I have many folders with the name like "Apparel & Accessories > Clothing > Activewear", I want to know if there is a easy way to process the escape characters without write so many .
test.sh
director="Apparel & Accessories > Clothing > Activewear"
# any action to escape spaces, &, > ???
hadoop fs -ls $director

For definining the specific string in your example, Apparel & Accessories > Clothing > Activewear, either double quotes or single quotes will work; referring to it later is a different story, however:
In the shell (any POSIX-compatible shell), how you refer to a variable is just as important as how you define it.
To safely refer to a previously defined variable without side-effects, enclose it in double quotes, e.g., "$directory".
To define [a variable as] a literal (verbatim) string:
(By contrast, to define a variable with embedded variable references or embedded command substitutions or embedded arithmetic expressions, use double quotes (").)
If your string contains NO single quotes:
Use a single-quoted string, e.g.:
directory='Apparel & Accessories > Clothing > Activewear'
A single-quoted string is not subject to any interpretation by the shell, so it's generally the safest option for defining a literal. Note that the string may span multiple lines; e.g.:
multiline='line 1
line 2'
If your string DOES contain single quotes (e.g., I'm here.) and you want a solution that works in all POSIX-compatible shells:
Break the string into multiple (single-quoted) parts and splice in single-quote characters:
Note: Sadly, single-quoted strings cannot contain single quotes, not even with escaping.
directory='I'\''m here.'
The string is broken into into single-quoted I, followed by literal ' (escaped as an unquoted string as \'), followed by single-quoted m here.. By virtue of having NO spaces between the parts, the result is a single string containing a literal single quote after I.
Alternative: if you don't mind using a multiline statement, you can use a quoted here document, as described at the bottom.
If your string DOES contain single quotes (e.g., I'm here.) and you want a solution that works in bash, ksh, and zsh:
Use ANSI-C quoting:
directory=$'I\'m here.'
Note: As you can see, ANSI-C quoting allows for escaping single quotes as \', but note the additional implications: other \<char> sequences are subject to interpretation, too; e.g., \n is interpreted as a newline character - see http://www.gnu.org/software/bash/manual/bash.html#ANSI_002dC-Quoting
Tip of the hat to #chepner, who points out that the POSIX-compatible way of directly including a single quote in a string to be used verbatim is to use read -r with a here document using a quoted opening delimiter (the -r option ensures that \ characters in the string are treated as literals).
# *Any* form of quoting, not just single quotes, on the opening EOF will work.
# Note that $HOME will by design NOT be expanded.
# (If you didn't quote the opening EOF, it would.)
read -r directory <<'EOF'
I'm here at $HOME
EOF
Note that here documents create stdin input (which read reads in this case). Therefore, you cannot use this technique to directly pass the resulting string as an argument.

use strong quotes i.e. 'string', allowing escape char or special char for string.
e.g. declare director='Apparel & Accessories > Clothing > Activewear'
also using declare is a good practice while declaring variable.

Related

Having a single quote inside a single quoted string

My question targets Zsh, but from what I tried, it seems to apply to POSIX shell and bash as well:
I want to write a string containing literal $ characters (no interpolation intended) and single quotes. Since the chapter on QUOTING in the Zsh man page says about single-quoted strings:
A literal ' character can be included in the string by using the \' escape.
I tried something like this (in an interactive zsh, before doing it in a script):
echo 'a$b\'c'
I expected that this would print a$b'c, but zsh tells me that I have an unclosed quote.
I am aware that I can use as a workaround
echo 'a$'"b'C"
but I still would like to know, why my original attempt failed.
The problem was my interpretation of the man-page, and I must blame myself that I have left out of my citation of the page one part which I thought is unimportant, but actually is relevant for this case. Here again the full sentence of the man page:
A string enclosed between '$'' and ''' is processed the same way as the
string arguments of the print builtin, and the resulting string is
considered to be entirely quoted. A literal ''' character can be
included in the string by using the '\'' escape.
Since Zsh has two ways to write a single-quoted String (one where the quotation, like in bash, starts with ' and one where it starts with $'), I understood the between ... and part that it meant "in $' and in ' -quoted strings, the \' escape works.
This interpretation is incorrect. What the man page means is that a quoted string which starts with $' and ends with ', can use that escape.
Hence my example can written as
echo $'a$b\'c'

Reason for inconsistent behavior of single quotes during BASH parameter expansion?

When using BASH Parameter Expansion, string that variable expands into can be quoted/escaped, which works fine, except when the single quotes are used and the whole variable is escaped in double quotes:
$ echo "${var:-\\a}"
\a # ok
$ echo "${var:-"\\a"}"
\a # ok
$ echo "${var:-$'\\a'}"
\a # ok
$ echo "${var:-'\a'}"
'\a' # wtf?
Interestingly, $' ' quotes work normally, while ' ' don't. Single quotes start working correctly if the variable itself is not quoted:
$ echo ${var:-'\a'}
\a
But, that can lead to other issues if $var itself contains whitespace characters.
Is there any good reason for this inconsistency?
I think this is the most relevant quote from the source code (y.tab.c):
/* Based on which dolstate is currently in (param, op, or word),
decide what the op is. We're really only concerned if it's % or
#, so we can turn on a flag that says whether or not we should
treat single quotes as special when inside a double-quoted
${...}. This logic must agree with subst.c:extract_dollar_brace_string
since they share the same defines. */
/* FLAG POSIX INTERP 221 */
[...]
/* The big hammer. Single quotes aren't special in double quotes. The
problem is that Posix used to say the single quotes are semi-special:
within a double-quoted ${...} construct "an even number of
unescaped double-quotes or single-quotes, if any, shall occur." */
/* This was changed in Austin Group Interp 221 */
It's not exactly clear to me why single quotes aren't special, but it seems like a conscious choice made after long (and I've been told contentious) debate preceding the change. But the fact is (if I am summarizing this correctly), single quotes here are just regular characters, not syntactic quotes, and are treated literally.

how to escape paths to be executed with $( )?

I have program whose textual output I want to directly execute in a shell. How shall I format the output of this program such that the paths with spaces are accepted by the shell ?
$(echo ls /folderA/folder\ with\ spaces/)
Some more info: the program that generates the output is coded in Haskell (source). It's a simple program that keeps a list of my favorite commands. It prints the commands with 'cmdl -l'. I can then choose one command to execute with 'cmdl -g12' for command number 12. Thanks for pointing out that instead of $( ) use 'cmdl -g12 | bash', I wasn't aware of that...
How shall I format the output of this program such that the paths with
spaces are accepted by the shell ?
The shell cannot distinguish between spaces that are part of a path and spaces that are separator between arguments, unless those are properly quoted. Moreover, you actually need proper quoting using single quotes ('...') in order to "shield" all those characters combinations that might otherwise have special meaning for the shell (\, &, |, ||, ...).
Depending the language used for your external tool, their might be a library available for that purpose. As as example, Python has pipes.quote (shlex.quote on Python 3) and Perl has String::ShellQuote::shell_quote.
I'm not quite sure I understand, but don't you just want to pipe through the shell?
For a program called foo
$ foo | sh
To format output from your program so Bash won't try to space-separate them into arguments either update, probably easiest just to double-quote them with any normal quoting method around each argument, e.g.
mkdir "/tmp/Joey \"The Lips\" Fagan"
As you saw, you can backslash the spaces alternatively, but I find that less readable ususally.
EDIT:
If you may have special shell characters (&|``()[]$ etc), you'll have to do it the hard/proper way (with a specific escaper for your language and target - as others have mentioned.
It's not just spaces you need to worry about, but other characters such as [ and ] (glob a.k.a pathname-expansion characters) and metacharacters such as ;, &, (, ...
You can use the following approach:
Enclose the string in single quotes.
Replace existing single quotes in the string with '\'' (which effectively breaks the string into multiple parts with spliced in \-escaped single quotes; the shell then reassembles the parts into a single string).
Example:
I'm good (& well[1];) would encode to 'I'\''m good (& well[1]);'
Note how single-quoting allows literal use of the glob characters and metacharacters.
Since single quotes themselves can never be used within single-quoted strings (there's not even an escape), the splicing-in approach described above is needed.
As described by #mklement0, a safe algorithm is to wrap every argument in a pair of single quotes, and quote single quotes inside arguments as '\''. Here is a shell function that does it:
function quote {
typeset cmd="" escaped
for arg; do
escaped=${arg//\'/\'\\\'\'}
cmd="$cmd '$escaped'"
done
printf %s "$cmd"
}
$ quote foo "bar baz" "don't do it"
'foo' 'bar baz' 'don'\''t do it'

Take in escaped input in Ruby command line app

I'm writing a Ruby command line application in which the user has to enter a "format string" (much like Date.strptime/strftime's strings).
I tried taking them in as arguments to the command, eg
> torque "%A\n%d\n%i, %u"
but it seems that bash actually removes all backslashes from input before it can be processed (plus a lot of trouble with spaces). I also tried the highline gem, which has some more advanced input options, but that automatically escapes backslashes "\" -> "\\" and provides no alternate options.
My current solution is to do a find-and-replace: "\\n" -> "\n". This would take care of the problem, but it also seems hacky and awful.
I could have users write the string in a text file (complicated for the user) or treat some other character, like "&&", as a newline (still complicated for the user).
What is the best way for users to input escaped characters on the command line?
(UPDATE: I checked the documentation for strptime/strftime, and the format strings for those functions replace newline characters with "%n", tabs with "%t", etc. So for now I'm doing that, but any other suggestions are welcome)
What you're looking for is using single quotes instead of double quotes.
Thus:
> torque '%A\n%d\n%i, %u'
Any string quoted in single quotes 'eg.' is does not go through any expansions and is used as is.
More details can be found in the Quoting section of man bash.
From man bash:
Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
p eval("\"#{gets.chomp}\"")
Example use:
\n\b # Input by the user from the keyboard
"\n\b" # Value from the script
%A\n%d\n%i, %u # Input by the user from the keyboard
"%A\n%d\n%i, %u" # Value from the script

how to show single quote in single-quoted text

I have a bash script,
echo 'abcd'
in shell, I want to show ab'c'd and I have tried following approach but without success
echo 'ab\'c\'d'
I am asking is it possible to show single quote in single quoted text?
From the bash manual section on Single Quotes:
A single quote may not occur between single quotes, even when preceded by a backslash.
You'll need to use double quotes instead. It's not pretty, but the following gives the output you are looking for:
echo 'ab'"'"'c'"'"'d'
A bash-specific feature, not part of POSIX, is a $'...'-quoted string:
echo $'ab\'c\'d'
Such a string behaves identically to a single-quoted string, but does allow for a selection of \-escaped characters (such as \n, \t, and yes, \').

Resources