weird handling of double quotes when using cmd /c "echo" - cmd

I'm trying to escape some calls in bat file and I found out that I don't understand even on simple example the weird handling of double quotes:
Try
cmd /c "echo " "%TEMP%" rem this gives: " "C:\Users\xyz\AppData\Local\Temp
cmd /c "echo" "%TEMP%" rem this gives: The filename, directory name, or volume label syntax is incorrect.
cmd /c "echo beg "TEMP" %TEMP%" rem this gives: beg "TEMP" C:\Users\xyz\AppData\Local\Temp
cmd /c "echo beg ^"TEMP^" %TEMP%" rem this gives: beg "TEMP" C:\Users\xyz\AppData\Local\Temp
cmd /c "echo beg \"TEMP\" %TEMP%" rem this gives: beg \"TEMP\" C:\Users\xyz\AppData\Local\Temp
Just open command prompt (cmd.exe) and paste code.
The results are the same when copy/paste to cmd.exe or running a bat file.
What I would expect is that cmd /c "echo beg \"TEMP\" %TEMP%" should work correcctly. At least according to http://daviddeley.com/autohotkey/parameters/parameters.htm#WIN .
But I don't understand output from all the samples. Anybody could explain me that behaviour?
Edit:
The site I'm referencing just explains how arguments are parsed on Windows. E.g. http://daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULES
What is my expected output?
I just want to know how that passing of double quotes work.
Later I'd like to construct command lines like this one:
pwsh -noprofile -command "produceSomeStringsToFind | % { rg -g testfile* \" $_ some string that ends with quote\\\"" . }"
where rg is https://github.com/BurntSushi/ripgrep - where the \" $_ some string that ends with quote\\\"" part is regular expression where I need to escape the quotes.

From the help text of cmd /?:
[...]
If /C or /K is specified, then the remainder of the command line after
the switch is processed as a command line, where the following logic is
used to process quote (") characters:
1. If all of the following conditions are met, then quote characters
on the command line are preserved:
- no /S switch
- exactly two quote characters
- no special characters between the two quote characters,
where special is one of: &<>()#^|
- there are one or more whitespace characters between the
two quote characters
- the string between the two quote characters is the name
of an executable file.
2. Otherwise, old behavior is to see if the first character is
a quote character and if so, strip the leading character and
remove the last quote character on the command line, preserving
any text after the last quote character.
[...]
It becomes clear that you are facing the situation explained in section 2., because echo is not an executable file but an internal command. So if the first character is a ", it becomes removed and so becomes the last ", and all the other characters are preserved.
Now let us go through your command lines. So after being parsed and processed by cmd /c:
cmd /c "echo " "%TEMP%" becomes echo " "C:\Users\xyz\AppData\Local\Temp.
cmd /c "echo" "%TEMP%" becomes echo" "C:\Users\xyz\AppData\Local\Temp; then the command echo" is tried to be executed, but which is not a valid one, hence it fails.
cmd /c "echo beg "TEMP" %TEMP%" becomes echo beg "TEMP" C:\Users\xyz\AppData\Local\Temp.
cmd /c "echo beg ^"TEMP^" %TEMP%" becomes echo beg "TEMP" C:\Users\xyz\AppData\Local\Temp, because escaping with ^ happens even before cmd /c is executed, so it receives the already escaped literal " characters.
cmd /c "echo beg \"TEMP\" %TEMP%" becomes echo beg \"TEMP\" C:\Users\xyz\AppData\Local\Temp, because the \ is nothing special to cmd.
If you want to output the text "%TEMP%" (including the quotes), you could do this:
rem // This preserves all quotes, because the first character is not such:
cmd /c echo "%TEMP%"
rem // This removes the outer-most pair of quotes, because the first character is such:
cmd /c "echo "%TEMP%""
rem /* This removes the outer-most pair of quotes too, since `cmd /c` receives the already
rem escaped `"`; however, this could be useful to hide these quotes from the hosting
rem `cmd` instance, which avoids issues when `%TEMP%` contains special characters
rem (like `&` or `^`), which you would otherwise have to individually escape: */
cmd /c ^"echo "%TEMP%"^"

Related

Get result of command with quoted arguments within single quote command evaluation in Windows batch

How do you pass quoted arguments to an executable in a single-quoted evaluation such as FOR /F "delims=" %%i IN ('"executable path" arg1 "arg2 which contains spaces"') do ...?
As per many of the answers here, I'm trying to use the output of a console app in a Windows batch script, using the single quotes to get the console to evaluate it.
However, I need to quote some of the arguments I want to pass to that executable, which also needs to be quoted, as the path contains spaces as well.
But when I do that, the quoting around the executable path breaks.
Here is the would-be line:
FOR /F "delims=" %%i IN ('"!PathToExe!" action^=sanitize string^="!album!"') DO set "falbum=%%i"
(Both !PathToExe! and !album! contain spaces. It seems like I need to escape the equal signs here, hence the circumflexes. Delayed expansion is on)
The above results in "Outcome A": Quoting is broken for the exe path
'<Part of path to exe>' is not recognized as an internal or external command, operable program or batch file.
I've tried different combinations of different quote usages and escapings, but haven't found a way to make it work.
Here are some attempts and their results:
FOR /F "delims=" %%i IN ('"!PathToExe!" action^=sanitize string^=!album!') DO set "falbum=%%i")
No quotes around !album! results in "Outcome B": As expected, only the first word gets passed along with string=, all the other words get scattered as individual arguments.
FOR /F "delims=" %%i IN ('"!PathToExe!" action^=sanitize string^=^"!album!^"') DO set "falbum=%%i")
FOR /F "delims=" %%i IN ('^"!PathToExe!^" action^=sanitize string^=^"!album!^"') DO set "falbum=%%i")
Trying to escape the quotes for the string= argument or both exe path and string arg: Outcome A (still breaks the quoting for the exe path, gives:)
'<Part of path to exe>' is not recognized as an internal or external command, operable program or batch file.
FOR /F "delims=" %%i IN ('"!PathToExe!" action^=sanitize string^=^'!album!^'') DO set "falbum=%%i")
Using escaped single quotes: Outcome B again
FOR /F "delims=" %%i IN ('"!PathToExe!" action^=sanitize string^='"!album!") DO set "falbum=%%i")
Ending the single quote before the string= value and simply having it in quotes after that seems to result in everything being taken as a single first argument (command/path):
The system cannot find the file '"<path to exe>" action=sanitize string='"<Album var with spaces and whatnot>".
It does not matter whether the quotes are part of the variables or literally typed in the IN ('...') line.
Simple testing:
You could test this behavior if you copied %SystemRoot%\System32\cmd.exe to a directory with spaces, e.g. C:\folder with spaces\ and pasted the following script there:
#echo off
setlocal EnableDelayedExpansion
set PathToExe="%~dp0cmd.exe"
REM Toggle the next line to compare between quoted path with spaces and path without quotes or spaces:
REM set PathToExe=cmd.exe
set string=%~dp0
FOR /F "delims=" %%i IN ('!PathToExe! /C CD C:\windows\system32 ^& dir !string!') DO set "fstring=%%i"
echo !fstring!
pause
This should illustrate the challenge of having two quoted sections in one statement.
If the !string! variable remains unquoted, you'll get "The system cannot find the file specified.".
If the quotes of the !PathToExe! variable break, you'll see something like "'C:\folder' is not recognized as an internal or external command, operable program or batch file.".
The for /F command, when used to capture and parse command output, actually uses cmd /C to execute that command, which handles quotation marks in a particular way. From its help (cmd /?):
If /C or /K is specified, then the remainder of the command line after
the switch is processed as a command line, where the following logic is
used to process quote (") characters:
1. If all of the following conditions are met, then quote characters
on the command line are preserved:
- no /S switch
- exactly two quote characters
- no special characters between the two quote characters,
where special is one of: &<>()#^|
- there are one or more whitespace characters between the
two quote characters
- the string between the two quote characters is the name
of an executable file.
2. Otherwise, old behavior is to see if the first character is
a quote character and if so, strip the leading character and
remove the last quote character on the command line, preserving
any text after the last quote character.
This means that:
for /F "delims=" %%I in ('"executable path" arg1 "arg2 which contains spaces"') do (…)
actually tries to execute the command line:
cmd /C "executable path" arg1 "arg2 which contains spaces"
leading to:
executable path" arg1 "arg2 which contains spaces
which is obviously invalid syntax.
To overcome this issue provide an additional pair of quotes:
for /F "delims=" %%I in ('^""executable path" arg1 "arg2 which contains spaces"^"') do (…)
It is a good idea to escape these additional quotes (by ^"), so there is no need to alter any potential escape sequences.

Opening local (file) URL with parameters in batch file? [duplicate]

How do I escape ampersands in a batch file (or from the
Windows command line) in order to use the start command to
open web pages with ampersands in the URL?
Double quotes will not work with start; this starts a new
command-line window instead.
Update 1: Wael Dalloul's solution works. In addition, if
there are URL encoded characters (e.g. space is encoded as
%20) in the URL and it is in a batch file then '%' must be
encoded as '%%'. This is not the case in the example.
Example, from the command line (CMD.EXE):
start http://www.google.com/search?client=opera&rls=en&q=escape+ampersand&sourceid=opera&ie=utf-8&oe=utf-8
will result in
http://www.google.com/search?client=opera
being opened in the default browser and these errors in the command line window:
'rls' is not recognized as an internal or external command,
operable program or batch file.
'q' is not recognized as an internal or external command,
operable program or batch file.
'sourceid' is not recognized as an internal or external command,
operable program or batch file.
'ie' is not recognized as an internal or external command,
operable program or batch file.
'oe' is not recognized as an internal or external command,
operable program or batch file.
Platform: Windows XP 64 bit SP2.
& is used to separate commands. Therefore you can use ^ to escape the &.
From a cmd:
& is escaped like this: ^& (based on #Wael Dalloul's answer)
% does not need to be escaped
An example:
start http://www.google.com/search?client=opera^&rls=en^&q=escape+ampersand%20and%20percentage+in+cmd^&sourceid=opera^&ie=utf-8^&oe=utf-8
From a batch file
& is escaped like this: ^& (based on #Wael Dalloul's answer)
% is escaped like this: %% (based on the OPs update)
An example:
start http://www.google.com/search?client=opera^&rls=en^&q=escape+ampersand%%20and%%20percentage+in+batch+file^&sourceid=opera^&ie=utf-8^&oe=utf-8
You can enclose it in quotes, if you supply a dummy first argument.
Note that you need to supply a dummy first argument in this case, as start will treat the first argument as a title for the new console windows, if it is quoted. So the following should work (and does here):
start "" "http://www.google.com/search?client=opera&rls=en&q=escape+ampersand&sourceid=opera&ie=utf-8&oe=utf-8"
explorer "http://www.google.com/search?client=opera&rls=...."
The command
echo this ^& that
works as expected, outputing
this & that
The command
echo this ^& that > tmp
also works, writing the string to file "tmp". However, before a pipe
echo this ^& that | clip
the ^ is interpreted completely differently. It tries to write the output of the two commands "echo this" and "that" to the pipe. The echo will work then "that" will give an error. Saying
echo this ^& echo that | clip
will put the strings "this" and "that" on the clipboard.
Without the ^:
echo this & echo that | clip
the first echo will write to the console and only the second echo's output will be piped to clip (similarly for "> tmp" redirection). So, when output is being redirected, the ^ does not quote the & but instead causes it to be applied before the redirection rather than after.
To pipe an &, you have to quote it twice
echo this ^^^& that | clip
If you put the string in a variable
set m=this ^& that
then
set m
will output
m=this & that
but the obvious
echo %m%
fails because, after Windows substitutes the variable, resulting in
echo this & that
it parses this as a new command and tries to execute "that".
In a batch file, you can use delayed expansion:
setlocal enableDelayedExpansion
echo !m!
To output to a pipe, we have to replace all &s in the variable value with ^&, which we can do with the %VAR:FROM=TO% syntax:
echo !m:^&=^^^&! | clip
On the command line, "cmd /v" enables delayed expansion:
cmd /v /c echo !m!
This works even when writing to a pipe
cmd /v /c echo !m! | clip
Simple.
If you need to echo a string that contains an ampersand, quotes won't help, because you would see them on the output as well. In such a case, use for:
for %a in ("First & Last") do echo %~a
...in a batch script:
for %%a in ("First & Last") do echo %%~a
or
for %%a in ("%~1") do echo %%~a
If you have spaces in the name of the file and you have a character you need to escape:
You can use single AND double quotes to avoid any misnomers in the command.
scp ./'files name with spaces/internal folder with spaces/"text & files stored.txt"' .
The ^ character escapes the quotes otherwise.
For special characters like '&' you can surround the entire expression with quotation marks
set "url=https://url?retry=true&w=majority"

Missing putting quote " symbol into file from windows command line or batch

I am trying to write different things to a text file from a .bat file. At every attempt to insert one quote " it results in avoiding this command, well, it looks like because it is missing " the symbol in an output file.
Here is what I am trying to do:
echo ' " '>> file.txt
It does not even work if we simply try it from CLI
echo ' " '> file.txt
I had the same issue with printf and echo from MinGW.
What is wrong here?
You need to escape some special characters to treat them literally. Particularly, " double quote toggles the quote flag, if the quote flag is active, the following special characters are no longer special: ^ & | < > ( ).
==> echo ' " '>> file.txt
' " '>> file.txt
==> type file.txt
The system cannot find the file specified.
==> echo ' ^" '>> file.txt
==> type file.txt
' " '
Next script shows some escaping rules for ECHO command; note different output (and rules for caret and exclamation mark) if Delayed Expansion is disabled or enabled:
#cls
#setlocal disabledelayedexpansion
#Call :ouputOnly
#endlocal
#Echo .
#setlocal enabledelayedexpansion
#Call :ouputOnly
#endlocal
#GOTO :eof
:ouputOnly
#Echo ^# - At Symbol: be less verbose
#Echo ^~ - Tilde: Parameter Expansion as in Call subroutines, FOR loops etc.
#Echo ^& - Single Ampersand: used as a command separator
#Echo ^&^& - Double Ampersand: conditional command separator (if errorlevel 0)
#Echo ^|^| - Double Pipe: conditional command separator (if errorlevel ^> 0)
#Echo ^:^: - Double Colon: alternative to "rem" for comments outside of code blocks
#Echo ^^ - Caret: general escape character in batch
#Echo ^" - Double Quote: surrounding a string in double quotes
#Echo escapes all of the characters contained within it
#Echo ^() - Parentheses: used to make "code blocks" of grouped commands
#Echo %% - Percentage Sign: are used to mark three of the four variable types
#Echo ^^! - Exclamation Mark: to mark delayed expansion environment variables ^^!var^^!
#Echo ^* - Asterisk: wildcard matches any number or any characters
#Echo ^? - Question Mark: matches any single character
#Echo ^. - Single dot: represents the current directory
#Echo ^.. - Double dot: represents the parent directory of the current directory
#Echo ^\ - Backslash: represent the root directory of a drive dir ^\
#Echo ^| - Single Pipe: redirects the std.output of one command
#Echo into the std.input of another
#Echo ^NUL (File like device): is like a bottomless pit
#Echo ^CON (File like device): is a file like device that represents the console
#Echo ^> - Single Greater Than: redirects output to either a file or file like device
#Echo ^>^> - Double Greater than: output will be added to the very end of the file
#Echo ^< - Less Than: redirect the contents of a file to the std.input of a command
#Echo Stream redirection: regarding the less and greater than symbols
#echo caret^^ "caret^"
#echo caret^^^^ bang^^! "caret^^ bang^!"
#exit /B
#rem based on (dead link nowadays) http://judago.webs.com/batchoperators.htm

How do I escape ampersands in batch files?

How do I escape ampersands in a batch file (or from the
Windows command line) in order to use the start command to
open web pages with ampersands in the URL?
Double quotes will not work with start; this starts a new
command-line window instead.
Update 1: Wael Dalloul's solution works. In addition, if
there are URL encoded characters (e.g. space is encoded as
%20) in the URL and it is in a batch file then '%' must be
encoded as '%%'. This is not the case in the example.
Example, from the command line (CMD.EXE):
start http://www.google.com/search?client=opera&rls=en&q=escape+ampersand&sourceid=opera&ie=utf-8&oe=utf-8
will result in
http://www.google.com/search?client=opera
being opened in the default browser and these errors in the command line window:
'rls' is not recognized as an internal or external command,
operable program or batch file.
'q' is not recognized as an internal or external command,
operable program or batch file.
'sourceid' is not recognized as an internal or external command,
operable program or batch file.
'ie' is not recognized as an internal or external command,
operable program or batch file.
'oe' is not recognized as an internal or external command,
operable program or batch file.
Platform: Windows XP 64 bit SP2.
& is used to separate commands. Therefore you can use ^ to escape the &.
From a cmd:
& is escaped like this: ^& (based on #Wael Dalloul's answer)
% does not need to be escaped
An example:
start http://www.google.com/search?client=opera^&rls=en^&q=escape+ampersand%20and%20percentage+in+cmd^&sourceid=opera^&ie=utf-8^&oe=utf-8
From a batch file
& is escaped like this: ^& (based on #Wael Dalloul's answer)
% is escaped like this: %% (based on the OPs update)
An example:
start http://www.google.com/search?client=opera^&rls=en^&q=escape+ampersand%%20and%%20percentage+in+batch+file^&sourceid=opera^&ie=utf-8^&oe=utf-8
You can enclose it in quotes, if you supply a dummy first argument.
Note that you need to supply a dummy first argument in this case, as start will treat the first argument as a title for the new console windows, if it is quoted. So the following should work (and does here):
start "" "http://www.google.com/search?client=opera&rls=en&q=escape+ampersand&sourceid=opera&ie=utf-8&oe=utf-8"
explorer "http://www.google.com/search?client=opera&rls=...."
The command
echo this ^& that
works as expected, outputing
this & that
The command
echo this ^& that > tmp
also works, writing the string to file "tmp". However, before a pipe
echo this ^& that | clip
the ^ is interpreted completely differently. It tries to write the output of the two commands "echo this" and "that" to the pipe. The echo will work then "that" will give an error. Saying
echo this ^& echo that | clip
will put the strings "this" and "that" on the clipboard.
Without the ^:
echo this & echo that | clip
the first echo will write to the console and only the second echo's output will be piped to clip (similarly for "> tmp" redirection). So, when output is being redirected, the ^ does not quote the & but instead causes it to be applied before the redirection rather than after.
To pipe an &, you have to quote it twice
echo this ^^^& that | clip
If you put the string in a variable
set m=this ^& that
then
set m
will output
m=this & that
but the obvious
echo %m%
fails because, after Windows substitutes the variable, resulting in
echo this & that
it parses this as a new command and tries to execute "that".
In a batch file, you can use delayed expansion:
setlocal enableDelayedExpansion
echo !m!
To output to a pipe, we have to replace all &s in the variable value with ^&, which we can do with the %VAR:FROM=TO% syntax:
echo !m:^&=^^^&! | clip
On the command line, "cmd /v" enables delayed expansion:
cmd /v /c echo !m!
This works even when writing to a pipe
cmd /v /c echo !m! | clip
Simple.
If you need to echo a string that contains an ampersand, quotes won't help, because you would see them on the output as well. In such a case, use for:
for %a in ("First & Last") do echo %~a
...in a batch script:
for %%a in ("First & Last") do echo %%~a
or
for %%a in ("%~1") do echo %%~a
If you have spaces in the name of the file and you have a character you need to escape:
You can use single AND double quotes to avoid any misnomers in the command.
scp ./'files name with spaces/internal folder with spaces/"text & files stored.txt"' .
The ^ character escapes the quotes otherwise.
For special characters like '&' you can surround the entire expression with quotation marks
set "url=https://url?retry=true&w=majority"

Escape angle brackets in a Windows command prompt

I need to echo a string containing angle brackets (< and >) to a file on a Windows machine. Basically what I want to do is the following:
echo some string < with angle > brackets >>myfile.txt
This doesn't work since the command interpreter gets confused with the angle brackets. I could quote the whole string like this:
echo "some string < with angle > brackets" >>myfile.txt
But then I have double quotes in my file that I don't want.
Escaping the brackets ala unix doesn't work either:
echo some string \< with angle \> brackets >>myfile.txt
Ideas?
The Windows escape character is ^, for some reason.
echo some string ^< with angle ^> brackets >>myfile.txt
True, the official escape character is ^, but be careful because sometimes you need three ^ characters. This is just sometimes:
C:\WINDOWS> echo ^<html^>
<html>
C:\WINDOWS> echo ^<html^> | sort
The syntax of the command is incorrect.
C:\WINDOWS> echo ^^^<html^^^> | sort
<html>
C:\WINDOWS> echo ^^^<html^^^>
^<html^>
One trick out of this nonsense is to use a command other than echo to do the output and quote with double quotes:
C:\WINDOWS> set/p _="<html>" <nul
<html>
C:\WINDOWS> set/p _="<html>" <nul | sort
<html>
Note that this will not preserve leading spaces on the prompt text.
There are methods that avoid ^ escape sequences.
You could use variables with delayed expansion. Below is a small batch script demonstration
#echo off
setlocal enableDelayedExpansion
set "line=<html>"
echo !line!
Or you could use a FOR /F loop. From the command line:
for /f "delims=" %A in ("<html>") do #echo %~A
Or from a batch script:
#echo off
for /f "delims=" %%A in ("<html>") do echo %%~A
The reason these methods work is because both delayed expansion and FOR variable expansion occur after special operators like <, >, &, |, &&, || are parsed. See How does the Windows Command Interpreter (CMD.EXE) parse scripts? for more info.
sin3.14 points out that pipes may require multiple escapes. For example:
echo ^^^<html^^^>|findstr .
The reason pipes require multiple escapes is because each side of the pipe is executed in a new CMD process, so the line gets parsed multiple times. See Why does delayed expansion fail when inside a piped block of code? for an explanation of many awkward consequences of Window's pipe implementation.
There is another method to avoid multiple escapes when using pipes. You can explicitly instantiate your own CMD process, and protect the single escape with quotes:
cmd /c "echo ^<html^>"|findstr .
If you want to use the delayed expansion technique to avoid escapes, then there are even more surprises (You might not be surprised if you are an expert on the design of CMD.EXE, but there is no official MicroSoft documentation that explains this stuff)
Remember that each side of the pipe gets executed in its own CMD.EXE process, but the process does not inherit the delayed expansion state - it defaults to OFF. So you must explicitly instantiate your own CMD.EXE process and use the /V:ON option to enable delayed expansion.
#echo off
setlocal disableDelayedExpansion
set "line=<html>"
cmd /v:on /c echo !test!|findstr .
Note that delayed expansion is OFF in the parent batch script.
But all hell breaks loose if delayed expansion is enabled in the parent script. The following does not work:
#echo off
setlocal enableDelayedExpansion
set "line=<html>"
REM - the following command fails
cmd /v:on /c echo !test!|findstr .
The problem is that !test! is expanded in the parent script, so the new CMD process is trying to parse unprotected < and >.
You could escape the !, but that can get tricky, because it depends on whether the ! is quoted or not.
If not quoted, then double escape is required:
#echo off
setlocal enableDelayedExpansion
set "line=<html>"
cmd /v:on /c echo ^^!test^^!|findstr .
If quoted, then a single escape is used:
#echo off
setlocal enableDelayedExpansion
set "line=<html>"
cmd /v:on /c "echo ^!test^!"|findstr .
But there is a surprising trick that avoids all escapes - enclosing the left side of the pipe prevents the parent script from expanding !test! prematurely:
#echo off
setlocal enableDelayedExpansion
set "line=<html>"
(cmd /v:on /c echo !test!)|findstr .
But I suppose even that is not a free lunch, because the batch parser introduces an extra (perhaps unwanted) space at the end when parentheses are used.
Aint batch scripting fun ;-)
In order to use special characters, such as '>' on Windows with echo, you need to place a special escape character before it.
For instance
echo A->B
will no work since '>' has to be escaped by '^':
echo A-^>B
See also escape sequences.
There is a short batch file, which prints a basic set of special character and their escape sequences.
Escaping the brackets ala unix doesn't
work either:
echo some string \< with
angle \> brackets >>myfile.txt
The backslash would be considered the start of a absolute pathname.
You can also use double quotes to escape special characters...
echo some string "<" with angle ">" brackets >>myfile.txt

Resources