Why does this .BAT line split with caret fail? - windows-7

Why does this .BAT line split with caret fail?

Because not only does ^ continue the line by ignoring the linefeed, it also escapes the first character of the next line. So the > is treated as a literal instead of as redirection.
You can get the exact same error result using:
type C:\temp.txt ^> C:\temp2.txt
You can fix the multiline version by putting a space before the redirection
type C:\temp.txt ^
> C:\temp2.txt
The rules are actually a bit more complex than what I have described. See jeb's explanation of how caret works at end of line

Related

How can I ignore a line in a batch file that uses carets for line continuation?

Take this example that uses carets for line continuation:
echo ^
aaa ^
bbb ^
ccc ^
ddd
How can I effectively "comment out" the bbb line so that it is ignored when running the batch file?
echo aaa <nul ^
%=bbb =%< nul ^
ccc ^
ddd
Looks odd, but it's necessary because the caret at the end of a line has also the effect, that the first character of the next line is also escaped.
But redirections are able to remove the effect, and they can also remove single characters or even complete tokens from the output
That is the problem if you try:
echo aaa ^
%=bbb =%^
ccc
The percent expression expands to nothing, so the code it's equal to.
echo aaa^
^
ccc
That is equal to:
echo aaa^^
ccc
And now ccc is treated as a new command
First change the start line to the following.
echo^
aaa^
bbb^
ccc^
ddd
This way it is always guaranteed that the caret will escape the space on the following line and not something that should not be escaped.
If everything before the caret is hidden from cmd then the caret from the previous line will escape that caret and line-continuation gets broken.
So there needs to be something to be escaped by the caret of the aaa line.
This is why the space is necessary.
But an escaped space is still a space and visible with echo.
This space can be hidden in the echo by adding a backspace.
Method 1: transform line to a non-existing variable.
This method makes 'bbb' disappear by changing it to a non-existant variable. This method has been mentioned by others.
So change [SPACE]bbb^
to [SPACE][BS]%=bbb=%^
Method 2: make the line disappear by inserting a NUL character.
This method does something similar but turns the bbb-line into a sort of end-of-line-comment by inserting a NUL character.
So change [SPACE]bbb^
to [SPACE][BS][NUL]bbb^
How does this work?
The interpreter stops processing characters when it encounters a null character and skips the null and everything following it including the next CRLF.
Both methods have their pro's and con's.
The first is generally editor-friendly (editor needs to support backspace), but fails with 'b%b'.
The second is very editor-unfriendly (editor needs to support backspace and null-character), but will work on all text replacing 'bbb'.
The first can be used in command blocks and with with redirections.
The second might fail with command blocks and redirections, but i have done only minimal tests with these.
Both methods work ok for the in between lines of the continuation, but not for the first and last.
#ECHO OFF
SETLOCAL
SET "bbb=bbb"
echo ^
aaa ^
%bbb% ^
ccc ^
ddd
SET "bbb="
echo ^
aaa ^
%bbb% ^
ccc ^
ddd
GOTO :EOF
The character set in the second example is a Backspace (^H). This doesn't work for the very first element (aaa)

Spaces in batch script without breaking the syntax

Is there a way in a batch script to add multiple spaces at the beginning of each line without breaking the syntax and keeping the spaces on each line as they are as per example below?
gacutil /u ^
Microsoft.IdentityModel.Clients.ActiveDirectory,^
Version=2.28.0.725,^
Culture=neutral,^
PublicKeyToken=31bf3856ad364e35
The goal is to allow better readability not only when utilizing gacutil.exe but whenever writing a batch command
Every executable has its own rule set for splitting up the string after the executable file name into argument values being processed further by the executable.
The windows command processor cmd.exe interprets a horizontal tab, a normal space, a comma, a semicolon, an equal sign and an OEM encoded no-break space (byte with hexadecimal value FF) outside a double quoted string as argument separators as it can be seen on running a batch file with name test.cmd with just the command line #echo %0 $1 $2 $3 $4 from within a command prompt window with
test argument1,,,"argument 2";;;"argument 3 contains a comma ',' a semicolon ';' an equal sign '=' and multiple spaces"===argument4
resulting in the output
test argument1 "argument 2" "argument 3 contains a comma ',' a semicolon ';' an equal sign '=' and multiple spaces" argument4
,,, and ;;; and === are interpreted as argument string separators.
The global assembly cache tool gacutil.exe splits up the string passed to it to argument strings different in comparison to cmd.exe. Commas and equal signs are not interpreted as argument separators. Only the normal space character is interpreted as argument separator, except a normal space is inside a double quoted argument string.
It is common for Windows executables that the number of argument separators between argument strings does not matter. So if there is used just one space or multiple spaces between two argument strings does not matter.
What does happen on using in a batch file following lines?
gacutil /u ^
Microsoft.IdentityModel.Clients.ActiveDirectory,^
Version=2.28.0.725,^
Culture=neutral,^
PublicKeyToken=31bf3856ad364e35
Please read first: How does the Windows Command Interpreter (CMD.EXE) parse scripts?
The Windows command processor cmd.exe reads a batch file line by line with replacing carriage return + line-feed by just a line-feed on reading a line.
The caret character ^ at end of a line is interpreted by cmd.exe as an escape character for the newline character line-feed resulting in concatenating the next line with the current line which is repeated until either reaching end of the batch file or finding a line-feed not escaped with ^. There are no characters removed during the concatenation of lines in a batch file.
The result for the lines above in the batch file with three leading normal spaces on each line is:
gacutil /u Microsoft.IdentityModel.Clients.ActiveDirectory, Version=2.28.0.725, Culture=neutral, PublicKeyToken=31bf3856ad364e35
The three leading spaces left to gacutil at beginning of the command line are removed by cmd.exe, but all other spaces are kept because of cmd.exe cannot know if they are important for the program/script to run next or not. So there are four spaces after /u and three spaces between the other strings specified on separate lines in the batch file.
gacutil.exe interprets the command line now as list of following arguments:
/u
Microsoft.IdentityModel.Clients.ActiveDirectory,
Version=2.28.0.725,
Culture=neutral,
PublicKeyToken=31bf3856ad364e35
The result is an error message because of gacutil.exe must be run in this case with just two arguments:
The option /u to instruct the global assembly cache tool to uninstall an assembly from the global assembly cache
and the assembly name Microsoft.IdentityModel.Clients.ActiveDirectory,Version=2.28.0.725,Culture=neutral,PublicKeyToken=31bf3856ad364e35.
So the indenting spaces used in the batch file between the individual parts of the assembly name argument string not removed by cmd.exe makes the command line invalid for gacutil.exe.
For that reason it is not possible in this case to define the assembly name argument string on multiple lines with indenting spaces because of how cmd.exe concatenates the lines with keeping the spaces and how gacutil.exe splits up the string passed to it from cmd.exe into arguments.
A different executable could do the string splitting to a list of argument values different and for that reason writing the arguments of the executable on multiple lines in a batch file with indent spaces could work for this executable.
But the following syntax can be used in a batch file for this use case:
gacutil /u ^
^"Microsoft.IdentityModel.Clients.ActiveDirectory,^
Version=2.28.0.725,^
Culture=neutral,^
PublicKeyToken=31bf3856ad364e35^"
The global assembly cache tool has no problems with one or more spaces after a comma in assembly name as long as the entire assembly name argument string is enclosed in double quotes. For that reason there is a " at beginning and one more " at the end of the assembly name string. But the two double quotes must be escaped with ^ for the Windows command processor as otherwise cmd.exe would interpret the caret character ^ at end of each line as literal character and not as an escape character.
The command line spread over multiple lines must be interpreted by cmd.exe as a line with no argument string in double quotes to get each ^ interpreted as escape character while the assembly name is in real enclosed in double quotes on running finally gacutil.exe with the command line:
gacutil /u "Microsoft.IdentityModel.Clients.ActiveDirectory, Version=2.28.0.725, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
Yes, that is strange.

Why does CMD ignore the character `;`?

I'm just wondering. When I type ; in cmd, it will just ignore it.
I can type ;;;;;;;;;;;;;;; and it will do the same thing but, if I do ;a it will say error.
Why is that?
; is a delimiter.
Delimiters separate one parameter from the next - they split the command line up into words.
More info on https://ss64.com/nt/syntax-esc.html
The semicolon is not ignored by cmd.exe; rather is it even particularly recognised, namely as a token separator, which are used to separate commands from its arguments and arguments from each other. Here are all such characters:
SPACE (code 0x20)
TAB (horizontal tabulator, code 0x09)
, (comma, code 0x2C)
; (semicolon, code 0x3B)
= (equal-to sign, code 0x3D)
VTAB (vertical tabulator, code 0x0B)
FF (form-feed or page-break, code 0x0C)
NBSP (non-breaking space, code 0xFF)
Note that multiple consecutive token separators are collapsed to a single one.
Command prompt does not ignore the character ";", ";" is a delimeter and cmd recognizes it as so so it doesn't "ignore" the character, but reads it similar to a space so nothing appears when you write it alone.

Windows CMD - Why Are Both Carets In Front of Exclamation Mark Removed When Delayed Expansion Is DISABLED?

I am trying to understand more about how the Windows CMD parser works. I have been reading several posts about the CMD parser, including this one, but I can't seem to figure out why both carets (^) in the following code are stripped when delayed expansion is DISABLED:
#echo off
setlocal disabledelayedexpansion
set $test_var=This is text with escaped delayed expansion syntax - ^^!$var1^^! and ^^!$var2^^!
echo $test_var = %$test_var%
echo.
pause
I expected the result of running the code to produce the following output:
$test_var = This is text with escaped delayed expansion syntax - ^!$var1^! and ^!$var2^!
Instead, ALL carets are removed:
$test_var = This is text with escaped delayed expansion syntax - !$var1! and !$var2!
From reading the post about the parser, it is my understanding that Phase 2 removes special characters, which includes the caret (^) escape character. From my reading it seems that only one (1) caret character should be removed. Why are both carets removed?
Thanks For Your Help!
The key to the answer is the fact that normal (%-)expansion (phase 1) occurs before recognition of special characters (in phase 2) — refer to the accepted answer to: How does the Windows Command Interpreter (CMD.EXE) parse scripts?
The first carets (^) become removed by the command interpreter in the line
set $test_var=This is text with escaped delayed expansion syntax - ^^!$var1^^! and ^^!$var2^^!
when phase 2 is done, so every instance of ^^ becomes one literal ^. You can prove this when you change #echo off to #echo on, so each parsed command line becomes echoed before being executed, or when you place the following command in the next line (thanks to user jeb for the hint):
rem // This displays the actual content of the variable:
set $test_var
In your next line
echo $test_var = %$test_var%
removal of the remaining carets happens, because – as already said – %-expansion (phase 1) happens first, resulting in a command line like
echo $test_var = This is text with escaped delayed expansion syntax - ^!$var1^! and ^!$var2^!
and then phase 2 follows, where the remaining carets are recognised and removed, resulting in this final text
$test_var = This is text with escaped delayed expansion syntax - !$var1! and !$var2!
You can protect the carets (as well as any other special characters) in the set command line when using the quoted syntax, like this:
set "$test_var=This is text with escaped delayed expansion syntax - ^!$var1^! and ^!$var2^!"
So you can save one level of escaping. However, this only works when the command extensions are enabled, but this is the default setting of the command interpreter anyway.
For the echo command line however, you cannot use such a method, because quotation marks became returned too.
I spotted that you used echo. to output an empty line. You should better use echo/ or echo( (refer to the external resource ECHO. FAILS to give text or blank line - Instead use ECHO/ to find out why).
By the way, your displayed text does not match the actual situation, because you have got delayed expansion disabled.

Long commands split over multiple lines in Windows with quotes

Using the caret to split lines,
dir ^
*.bat ^
/w
works as expected, but
dir ^
"*.bat" ^
won't let me enter the "/w". I guess the caret does not work after a double quote. Is this a bug? Or if this is a feature, what is it's use and how can I get around it?
I found the answer myself:
dir ^
^"*.bat^" ^
/w
works as I want. In the second line there must be spaces before the first caret and after the last. (using Vista SP2)
An alternative solution that has worked for me (Windows Server 2012) is to include a tab on the subsequent line. For example:
dir ^
"*.bat" ^
/w
In my experience a tab character can replace a space when delimiting arguments to a command or script. I also feel that this has a cleaner look.
However, I haven't spent time testing it under every possible batch-file syntax scenario and it's possible there are some where it won't apply or work properly.

Resources