For generating a string command for using in uiop:run-command I need to have a string which contains other strings (as required by the command in question). However the command (Fontforg's legacy scripting language) requires that some strings are enclosed in double quotes, e.g.:
"fontforge -lang=ff -c 'Print("A Doublequote String")'"
How can I get such a string with the "A Doublequote String" being literally part of the command-string, i.e. no doublequote being escaped?
Update
To be more concrete, the command I want to send to uiop:run-command is fontforge -lang=ff -c 'Open($1);SelectAll();foreach Print(GlyphInfo("Name")); endloop' haydn-11.svg, where the "Name" argument to GlyphInfo should be written with double quotes. Using escape characters backslash to preserve the quotes
(uiop:run-program
(format nil
"fontforge -lang=ff -c 'Open($1);SelectAll();foreach Print(GlyphInfo(\"Name\")); endloop' haydn-11.svg")
:output t)
the subprocess command exits with error code 1:
Subprocess with command "fontforge -lang=ff -c 'Open($1);SelectAll();foreach Print(GlyphInfo(\"Name\")); endloop' haydn-11.svg"
exited with error code 1
[Condition of type UIOP/RUN-PROGRAM:SUBPROCESS-ERROR]
I suppose the command is seeing the escape characters too, and that is why it fails to perform, since otherwise the command is syntactically correct.
Use Single Escape Character:
Backslash is a single escape character in standard syntax.
I.e., what you are looking for is
"fontforge -lang=ff -c 'Print(\"A Doublequote String\")'"
Note that by default *print-escape* is t, i.e., the above string will be printed with backslashes even though the string itself does not contain it:
(defparameter s (string #\"))
s
==> "\""
(length s)
==> 1
(char s 0)
==> #\"
I don't have fontforge, but I can do the equivalent. First of all the way you are doing it is not one, but two levels of language-in-a-string: the shell language is in a string in CL, and whatever fontforge language is in a string in the shell language. So let's minimise that by avoiding the whole shell language altogether. We still have one level of language-in-a-string, but that's a whole lot better than two.
I also don't understand why you're using (format nil <fixed string>) to make ... a fixed string. I'm guessing that eventually you're intending to replace the filename using format, but this is not needed since we're no longer going to use the shell at all.
So instead, do this
(defun runit (file)
(uiop:run-program
(list "echo" ; because I don't have fontforge
"fontforge" "-lang=ff" "-c"
"Open($1); SelectAll(); foreach Print(GlyphInfo(\"Name\")); endloop"
file)
:output t
:force-shell nil))
Note that the command here is echo: because I don't have fontforge I'll just get echo to print the command line. I've also added some spaces into the big chunk of whatever-language-fontforge-uses to make it clearer.
So now
> (runit "foo.svg")
fontforge -lang=ff -c Open($1); SelectAll(); foreach Print(GlyphInfo("Name")); endloop foo.svg
nil
nil
0
And this is fine, although it's not clear what the individual arguments are because echo doesn't do that. Well, I have a little utility called argv whose whole job is to print its argv clearly, so rewriting runit as
(defun runit (file)
(uiop:run-program
(list "argv" ; because I don't have fontforge
"fontforge" "-lang=ff" "-c"
"Open($1); SelectAll(); foreach Print(GlyphInfo(\"Name\")); endloop"
file)
:output t
:force-shell nil))
We get
> (runit "foo.svg")
"fontforge"
"-lang=ff"
"-c"
"Open($1); SelectAll(); foreach Print(GlyphInfo("Name")); endloop"
"foo.svg"
nil
nil
0
Note that argv isn't smart enough to escape the quotes inside the long argument: it's just a very tiny Perl script I use for debugging things.
Related
In the first part of my question I will provide some background info as a
service to the community. The second part contains the actual question.
Part I
Assume I've created the following alias:
alias ls='ls -r'
I know how to temporarily unalias (i.e., override this alias) in the following
ways, using:
1) the full pathname of the command: /bin/ls
2) command substitution: $(which ls)
3) the command builtin: command ls
4) double quotation marks: "ls"
5) single quotation marks: 'ls'
6) a backslash character: \ls
Case 1 is obvious and case 2 is simply a variation. The command builtin in case 3 was designed to ignore shell functions, but apparently it also works for circumventing aliases. Finally, cases 4 and 5 are consistent with both the POSIX standard (2.3.1):
"a resulting word that is identified
to be the command name word of a
simple command shall be examined to
determine whether it is an unquoted,
valid alias name."
and the Bash Reference Manual (6.6):
"The first word of each simple
command, if unquoted, is checked to
see if it has an alias."
Part II
Here's the question: why is case 6 (overriding the alias by saying \ls)
considered quoting the word? In keeping with the style of this question, I am looking for references to the "official" documentation.
The documentation says that a backslash only escapes the following
character, as opposed to single and double quotation marks, which quote a
sequence of characters. POSIX standard (2.2.1):
"A backslash that is not quoted shall
preserve the literal value of the
following character, with the
exception of a < newline >"
Bash Reference Manual (3.1.2.1):
"A non-quoted backslash ‘\’ is the
Bash escape character. It preserves
the literal value of the next
character that follows, with the
exception of newline."
(BTW, isn't "the next character that follows" a bit of overkill?)
A possible answer might be that this situation isn't that special: it is
similar to a few cases in ANSI-C quoting, e.g. \nnn. However, that is still
escaping a single character (the eight-bit character whose value is the octal
value nnn), not a sequence of characters.
Historically, and maintained by POSIX, quoting any part of the word causes the entire word to be considered quoted for the purposes of functions and alias expansion. It also applies to quoting the end token for a here document:
cat << \EOF
this $text is fully quoted
EOF
Just for completion, here's yet another way to suppress alias & function lookups (by clearing the entire shell environment for a single command):
# cf. http://bashcurescancer.com/temporarily-clearing-environment-variables.html
env -i ls
\ls only quotes the first character rather than the whole word. It's equivalent to writing 'l's.
You can verify it like this:
$ touch \?l
$ \??
bash: ?l: command not found
If \?? quoted the whole word it would say ?? not found rather than ?l not found.
I.e. it has the same effect as:
$ '?'?
bash: ?l: command not found
rather than:
$ '??'
bash: ??: command not found
I am trying to write a Makefile for GNU make. I can't figure out what the problem is here:
foo := this|works
bar := "I lost my 'single quotes'"
baz := 'make|will|not|accept|this|with|the|single|quotes'
whatIWant := "A string with its 'single|quoted|regex|alternatives'"
this-almost-works: #But the single quotes are lost.
#printf '%s ' "$(whatIWant)"
this-fails-horribly:
#printf '$(whatIWant)'
I get the following error message
/bin/sh: 1: quoted: not found
/bin/sh: 1: /bin/sh: 1: regex: not foundalternatives": not found
blah blah Error 127
Why is it trying to run parts of this string in the shell?
How can I define a variable to contain exactly the contents of whatIWant?
Might be worth looking in detail at the expansion.
When defining variables,
just about the only character that has an effect is $.
Everything else is taken literally.
It's worth nothing that white space around the assignment operator (= or :=) is ignored.
foo := this|works
foo is assigned the literal text this|works.
Similarly,
baz := 'make|will|not|accept|this|with|the|single|quotes'
assigns the literal text 'make|will|not|accept|this|with|the|single|quotes' to baz.
Fine and dandy.
Now, when make decides to build this-fails-horribly
(possibly because you said to the shell make this-fails-horribly)
it expands the block of commands before doing anything.
Not unreasonably,
$(whatIWant) is replaced by "A string with its 'single|quoted|regex|alternatives'".
Again, fine and dandy.
What is left is passed verbatim, one line at a time, to the shell.
The shell sees
printf '"A string with its 'single|quoted|regex|alternatives'"'
(which make would have helpfully echoed to you if you had left off the # prefix).
Now we are in the land of shell quoting.
The printf command is passed one parameter: "A string with its single:
'"A string with its ' is a single quoted string. The shell strips the 's and is left with the text "A string with its.
single has no metacharacters in it, so the shell leaves this alone.
The output is piped to the quoted command
The output is piped to the regex command
The output is piped to the alternatives" command
The shell sees the single quoted string '=', strips the quotes leaving you with a literal = which it appends to the word alternatives
No syntax error.
When the shell attempts to set up the pipeline it looks for the alternatives" command.
It doesn't find one in the directories it its $PATH, so it stops with the message /bin/sh: 1: /bin/sh: 1: regex: not foundalternatives": not found.
One possible encoding:
.PHONY: this-workes-nicely
this-workes-nicely:
echo $(whatIWant)
though you'll probably find it's cleaner to leave the quotes outside the variable definition in the first place.
Am trying to print a bunch of strings in a script (in zsh) and it doesn't seem to work. The code would work if I place the array in a variable and use it instead. Any ideas why this doesn't work otherwise?
for string in (some random strings to print) ; echo $string
The default form of the for command in zsh does not use parentheses (if there are any they are not interpreted as part of the for statement):
for string in some random strings to show
do
echo _$string
done
This results in the following output:
_some
_random
_strings
_to
_show
So, echo _$string was run for each word after in. The list ends with the newline.
It is possible to write the whole statement in a single line:
for string in some random strings to show; do echo _$string; done
As usual when putting multiple shell commands in the same line, newlines just need to be replaced by ;. The exception here is the newline after do; while zsh allows a ; to be placed after do, it is usually not done, and in bash it would be a syntax error.
There are also several short forms available for for, all of which are equivalent to the default form above and produce the same output:
for single commands (to be exact: single pipelines or multiple pipelines linked with && or ||, where a pipeline can also be just a single command), there are two options:
the default form, just without do or done:
for string in some random strings to show ; echo _$string
without in but with parentheses, also without do or done
for string (some random strings to show) ; echo _$string
for a list of commands (like in the default form), foreach instead of for, no in, with parentheses and terminated by end:
foreach string (some random strings to show) echo _$string ; end
In your case, you mixed the two short forms for single commands. Due to the presence of in, zsh did not take the parentheses as a syntactic element of the for command. Instead they are interpreted as a glob qualifier. Aside from the fact that you did not intend any filename expansions, this fails for two reasons:
there is no pattern (with or without actual globs) before the glob qualifier. So any matching filename would have to exactly match an empty string, which is just not possible
but mainly "some random strings to print" is not a valid glob qualifier. You probably get an error like "zsh: unknown file attribute: i" (at least with zsh 5.0.5, it may depend on the zsh version).
Check the zsh forloop documentation:
for x (1 2 3); do echo $x; done
for x in 1 2 3; do echo $x; done
You are probably trying to do this:
for string in some random strings to print ;do
echo $string
done
I have program whose textual output I want to directly execute in a shell. How shall I format the output of this program such that the paths with spaces are accepted by the shell ?
$(echo ls /folderA/folder\ with\ spaces/)
Some more info: the program that generates the output is coded in Haskell (source). It's a simple program that keeps a list of my favorite commands. It prints the commands with 'cmdl -l'. I can then choose one command to execute with 'cmdl -g12' for command number 12. Thanks for pointing out that instead of $( ) use 'cmdl -g12 | bash', I wasn't aware of that...
How shall I format the output of this program such that the paths with
spaces are accepted by the shell ?
The shell cannot distinguish between spaces that are part of a path and spaces that are separator between arguments, unless those are properly quoted. Moreover, you actually need proper quoting using single quotes ('...') in order to "shield" all those characters combinations that might otherwise have special meaning for the shell (\, &, |, ||, ...).
Depending the language used for your external tool, their might be a library available for that purpose. As as example, Python has pipes.quote (shlex.quote on Python 3) and Perl has String::ShellQuote::shell_quote.
I'm not quite sure I understand, but don't you just want to pipe through the shell?
For a program called foo
$ foo | sh
To format output from your program so Bash won't try to space-separate them into arguments either update, probably easiest just to double-quote them with any normal quoting method around each argument, e.g.
mkdir "/tmp/Joey \"The Lips\" Fagan"
As you saw, you can backslash the spaces alternatively, but I find that less readable ususally.
EDIT:
If you may have special shell characters (&|``()[]$ etc), you'll have to do it the hard/proper way (with a specific escaper for your language and target - as others have mentioned.
It's not just spaces you need to worry about, but other characters such as [ and ] (glob a.k.a pathname-expansion characters) and metacharacters such as ;, &, (, ...
You can use the following approach:
Enclose the string in single quotes.
Replace existing single quotes in the string with '\'' (which effectively breaks the string into multiple parts with spliced in \-escaped single quotes; the shell then reassembles the parts into a single string).
Example:
I'm good (& well[1];) would encode to 'I'\''m good (& well[1]);'
Note how single-quoting allows literal use of the glob characters and metacharacters.
Since single quotes themselves can never be used within single-quoted strings (there's not even an escape), the splicing-in approach described above is needed.
As described by #mklement0, a safe algorithm is to wrap every argument in a pair of single quotes, and quote single quotes inside arguments as '\''. Here is a shell function that does it:
function quote {
typeset cmd="" escaped
for arg; do
escaped=${arg//\'/\'\\\'\'}
cmd="$cmd '$escaped'"
done
printf %s "$cmd"
}
$ quote foo "bar baz" "don't do it"
'foo' 'bar baz' 'don'\''t do it'
In the first part of my question I will provide some background info as a
service to the community. The second part contains the actual question.
Part I
Assume I've created the following alias:
alias ls='ls -r'
I know how to temporarily unalias (i.e., override this alias) in the following
ways, using:
1) the full pathname of the command: /bin/ls
2) command substitution: $(which ls)
3) the command builtin: command ls
4) double quotation marks: "ls"
5) single quotation marks: 'ls'
6) a backslash character: \ls
Case 1 is obvious and case 2 is simply a variation. The command builtin in case 3 was designed to ignore shell functions, but apparently it also works for circumventing aliases. Finally, cases 4 and 5 are consistent with both the POSIX standard (2.3.1):
"a resulting word that is identified
to be the command name word of a
simple command shall be examined to
determine whether it is an unquoted,
valid alias name."
and the Bash Reference Manual (6.6):
"The first word of each simple
command, if unquoted, is checked to
see if it has an alias."
Part II
Here's the question: why is case 6 (overriding the alias by saying \ls)
considered quoting the word? In keeping with the style of this question, I am looking for references to the "official" documentation.
The documentation says that a backslash only escapes the following
character, as opposed to single and double quotation marks, which quote a
sequence of characters. POSIX standard (2.2.1):
"A backslash that is not quoted shall
preserve the literal value of the
following character, with the
exception of a < newline >"
Bash Reference Manual (3.1.2.1):
"A non-quoted backslash ‘\’ is the
Bash escape character. It preserves
the literal value of the next
character that follows, with the
exception of newline."
(BTW, isn't "the next character that follows" a bit of overkill?)
A possible answer might be that this situation isn't that special: it is
similar to a few cases in ANSI-C quoting, e.g. \nnn. However, that is still
escaping a single character (the eight-bit character whose value is the octal
value nnn), not a sequence of characters.
Historically, and maintained by POSIX, quoting any part of the word causes the entire word to be considered quoted for the purposes of functions and alias expansion. It also applies to quoting the end token for a here document:
cat << \EOF
this $text is fully quoted
EOF
Just for completion, here's yet another way to suppress alias & function lookups (by clearing the entire shell environment for a single command):
# cf. http://bashcurescancer.com/temporarily-clearing-environment-variables.html
env -i ls
\ls only quotes the first character rather than the whole word. It's equivalent to writing 'l's.
You can verify it like this:
$ touch \?l
$ \??
bash: ?l: command not found
If \?? quoted the whole word it would say ?? not found rather than ?l not found.
I.e. it has the same effect as:
$ '?'?
bash: ?l: command not found
rather than:
$ '??'
bash: ??: command not found