This question is originally coming from Escape percent signs in given variables. I do not want to disarrange the good answer over there. But my issue changed a little bit...
Let's assume there is a given string variable enclosed by double quotes which may include one or more percent signes. It is not possible to switch to enabled delayed expansion permanently (other code is already usable). Calling a function including the string variable as a parameter is necessary. This is what I determined so far:
#echo off & setlocal ENABLEEXTENSIONS
SET AlbumArtist=%1
CALL :EscapePoisonChars %AlbumArtist% AlbumArtist_VDN
echo %AlbumArtist_VDN%
CALL :EscapePoisonChars %%AlbumArtist%% AlbumArtist_VDN
echo %AlbumArtist_VDN%
endlocal &GOTO:EOF
:EscapePoisonChars
#echo off & setlocal ENABLEEXTENSIONS
SET TmpString="%~1"
SET TmpString=%TmpString:&=^^^&%
SET TmpString=%TmpString:(=^^^(%
SET TmpString=%TmpString:)=^^^)%
endlocal&SET %2=%TmpString:~1,-1%&GOTO :EOF
I know that this is probably not a "clean solution". But I would like to understand why when the routine is invoked by CALL :EscapePoisonChars %AlbumArtist% AlbumArtist_VDN the percent sign disappears. When called with the string variable %%AlbumArtist%% enclosed by doubled percent signs it gives the wanted output:
D:\Batch>PercentTwins.bat "100% Rock & Roll"
100 Rock & Roll
100% Rock & Roll
D:\Batch>
Why there is a different result if %AlbumArtist% is expanded in- or outside the function :EscapePoisonChars? With echo on I see that the percent sign just disappears with SET TmpString="~1". Any explanations will help me to improve my further cmd techniques. Thanks!
Anyone correct me if I am wrong, but I think when single percent signs in one command are passed on to another command, they will disappear. If it is just used within the same command, it normally will not disappear.
(There is probably a more 'correct' way of saying this, or this idea might be wrong altogether)
(some info on percent signs at Microsoft Support website)
The call command initiates the % parsing phase a second time (reference the accepted answer to the question post How does the Windows Command Interpreter (CMD.EXE) parse scripts?); this lets the % signs disappear. You could double the % signs in advance, but you would need to enable delayed expansion for that, like this:
#echo off & setlocal ENABLEEXTENSIONS EnableDelayedExpansion
SET AlbumArtist=%1
set AlbumArtist=!AlbumArtist:%%=%%%%!
CALL :EscapePoisonChars %AlbumArtist% AlbumArtist_VDN
echo %AlbumArtist_VDN%
endlocal &GOTO:EOF
In your second sub-routine call, the two parsing phases do not ever see the % signs of the original string; the first phase expands %%AlbumArtist%% to %AlbumArtist% literally, the second phase expands it to 100% Rock & Roll.
But:
There is no need for all that, you do not need the sub-routine at all, when you stick to the only safe set syntax and ensure to always have the string quoted properly:
set "AlbumArtist=%~1"
%1 would return the string as given; %~1 removes potential surrounding quotation marks. Enclosing the entire assignment expression within quotation marks makes the syntax robust against poisonous characters by not considering the "" as part of the assigned string itself.
When using/returning the string (by echo as in the example here), you need to enclose the string within quotes again to maintain safety, in case you are using normal (immediate) % expansion:
set "AlbumArtist=%~1"
echo "%AlbumArtist%"
Alternatively, delayed expansion makes reading variables safe even without quotation marks:
set "AlbumArtist=%~1"
setlocal EnableDelayedExpansion
echo !AlbumArtist!
endlocal
Related
I'm trying to write a text to a file using a batch file:
start https://www.google.com/search?q=%clipboard%%"
#echo off
set "Text=start https://www.google.com/search?q=%clipboard%%"
set "Text=%Text:^%=%%%"
echo %Text%>>lala.txt
But what is writen to the text file is only:
start https://www.google.com/search?q=%
The idea is to duplicate the percent signs so they are escaped and seen as text only but I guess for some reaseon the batch still things it's a variable. I know I could just use:
echo start https://www.google.com/search?q=%%clipboard%%%%"
But let's say I don't know what the line looks like I only know it has percent signs in it how should I proceed? The only thing I want to do is write the line as is into a text file without any manipulation.
When you define a literal string in a batch file, like set "Text=A single %%-sign" or echo A single %%-sign, you always have to manual double %-signs; otherwise, the %-expansion phase consumes them and tries to expand variables (refer to this answer for more details).
However, when the input text comes from somewhere else, like user input (set /P Text="Enter text: ") or from a file (e. g., read by for /F) you do not have to manually double %-signs, because the %-expansion phase is already completed when the text arrives.
This is the original answer before I recognised the real problem:
Well, the following line in your code cannot work:
set "Text=%Text:^%=%%%"
Because, besides the fact that it would actually replace ^% rather than %, the %-sign behind the =-sign finishes this sub-string substitution expression, and the remaining %% become then replaced by a single literal % (refer to this answer for more details).
To double %-signs in an arbitrary string you need to enable delayed expansion, because this uses ! instead of % to mark variables, which do not interfere with the literal %-signs you want to replace/double:
#echo off
rem /* You have to double `%`-signs when you put a literal string here; so
rem this literaly sets `https://www.google.com/search?q=%clipboard%`: */
set "Text=https://www.google.com/search?q=%%clipboard%%"
rem // Enable delayed expansion:
setlocal EnableDelayedExpansion
rem // Literal `%`-signs still have to be doubled here:
set "Text=!Text:%%=%%%%!"
rem // Return the string with `%`-signs doubled:
echo Delayed expansion: !Text!
echo Normal expansion: %Text%
set Text & rem // (avoiding `echo` here to review the true variable value)
rem // Variables set/changed since `setlocal` become lost past this point:
endlocal
Currently I have a loop that runs through a list of items and copies them to a directory (archive). However, one of the items in the list (which has a global variable in the path-name) is being interpreted 'literally' (as text instead of code).
I know usually you can just escape the line via (^^) to have it interpreted as code instead of text, but evidently I'm doing something wrong here, because it's not working...
The item in the my.list (with the escape in it) that is having issues is:
location\foo^^%date:~0,3%*.zip
The code I'm using is...
for /f "delims=" %%a in (my.list) do (
echo "%%a"
)
Echo's
"location\foo^^%date:~0,3%*.zip"
Instead of
location\fooMON*.zip
Any help would be greatly appreciated.
You are confused as to when you need to escape a character.
Some characters have special meaning ("code" as you describe it). Often times you can escape the character such that it is interpretted as a literal (text) instead of "code".
The most frequent method to escape a character within Windows CMD.EXE is to prefix it with a single ^ character. Sometimes a string is parsed twice, which can require an escape sequence of ^^^, (or perhaps ^^ when dealing with ! when delayed expansion is enabled). More rounds of parsing require ever more ^ characters. It can quickly become confusing, and requires practice to get the hang of it.
But your situation is completely different - It cannot be solved by escaping. You have "code" within your FOR variable, and you want it to be interpreted as such. But instead, it is being interpreted as text. In order to understand why, you must understand the order in which various stages of batch parsing occur. You could refer to How does the Windows Command Interpreter (CMD.EXE) parse scripts?, but it is pretty advanced stuff that takes time to digest.
Here is a crude synopsis showing when various types of expansion occur. (Note - these step numbers do not match up exactly with the phase numbers of the linked answer)
1) Parameter expansion - %1
2) Normal variable expansion - %var%
3) FOR variable expansion - %%A
4) Delayed variable expansion - !var!
5) CALL expansion - repeat steps 1) and 2) if CALL involved
You want your %date:~0,3% string to undergo normal (percent) expansion. Your FOR loop reads the line of text verbatim, without any expansion. The first time the parser sees your "code" is at step 3) when the %%a FOR variable is expanded. You can see that this is already too late to get
%date:~0,3% to expand the way you want.
You have two choices to solve your problem. But beware - each of these solutions potentially add new issues that may need to be solved.
I am assuming the ^^ is your naive attempt to force expansion of the embedded "code". The ^^ should be removed from your list file.
Option 1: Add an extra round of normal expansion by using CALL
for /f "delims=" %%a in (my.list) do call echo "%%a"
But now you have a potential problem that you might have a % literal in your list that you do not want to be expanded. Percents within batch scripts cannot be escaped with ^. Instead you escape a percent by doubling it as %%. So if you have percent literals in your list, they must be doubled.
Note that the original code that was posted with the question was significantly more complicated. It included an IF statement that referenced %%a. You cannot CALL an IF or FOR command. The solution is to CALL a subroutine, passing the value, and include the complex logic in the subroutine.
for /f "delims=" %%a in (my.list) do call :processValue "%%a" >>Logs\xfer.log
exit /b
:processValue
echo Attempting to archive %1...
if exist "c:\%~1" (
echo f | xcopy "c:\%%a" "c:\Lucas\archive\%~1" /E /C /H /R /Y
if %errorlevel%==0 (
echo ...%1 added to archive for transfer
echo.
) else (
echo ERROR: %1 not added to archive
echo.
)
) else (
echo ERROR: %1 Not found on client computer
echo.
)
Option 2: Use delayed expansion
Enable delayed expansion, and change your list to use !date:~0,3! instead of %date:~0,3%. Delayed expansion occurs after FOR variable expansion, so it will be expanded properly.
setlocal enableDelayedExpansion
for /f "delims=" %%a in (my.list) do echo "%%a"
But now you have a potential problem that you might have a ! literal in your list that you do not want to be expanded. You can preserve ! literals by escaping them as ^!.
My first post, most questions already solved using this friendly provided knowldge here. But now I run out of ideas, again with a question about handling of poison characters in cmd.exe.
Let's assume there is a given string variable enclosed in double quotes. Most poison characters has already been replaced by common chars before, the left ones disturbing the script are "&", "(", ")" and "%". The string must be echoed to a file without quotes afterwards. So I had the idea to escape the poison characters tripled:
#echo off & setlocal ENABLEEXTENSIONS
SET AlbumArtist=%1
CALL :EscapePoisonChars %AlbumArtist% AlbumArtist_VDN
SET "FlacHyperLink==hyperlink^("file://%AlbumArtist_VDN%"^;"LossLess"^)")
echo %FlacHyperLink%
echo %AlbumArtist_VDN%
endlocal &GOTO:EOF
:EscapePoisonChars
#echo off & setlocal ENABLEEXTENSIONS
SET TmpString=%1
SET TmpString=%TmpString:&=^^^&%
SET TmpString=%TmpString:(=^^^(%
SET TmpString=%TmpString:)=^^^)%
endlocal&SET %2=%TmpString:~1,-1%&GOTO :EOF
When I call my script above I get the expected output - apart from the missing percent sign:
G:\YAET\20130204_Work>TryAmper.bat "100% Rock & Roll (7' UpMix)"
=hyperlink("file://100 Rock & Roll (7' UpMix)";"LossLess")
100 Rock & Roll (7' UpMix)
G:\YAET\20130204_Work>
I know that the percent can be escaped by itself. So "%%" will normally lead to a single literal "%". But it was not possible for me to find a working replace procedure for percent signs because cmd always interprets it as a variable and tries to expand it. Is this the complete wrong direction to handle this issue or just misunderstanding of variable expansion? Any hints welcome! Thanks!
Cheers, Martin
Edit
Removed own code, see below Jeb's answer for clean solution.
Thanks for help, Martin
Nice question!
At first, yes you can replace even percent signs, but not within a percent expansion, you need a delayed expansion here.
Setlocal EnableDelayedExpansion
set tmpstr=!tmpstr:%=%%!
But if you use the delayed expansion, you don't need the escapes anymore, as the delayed expansion is the last phase of the batch parser and all characters lose any special meaning.
You only need to echo with delayed expansion.
Echo !tmpvar!
EDIT: Clean solution
#echo off
setlocal DisableDelayedExpansion
REM * More or less secure getting the parameter
SET "AlbumArtist=%~1"
setlocal EnableDelayedExpansion
SET "FlacHyperLink==hyperlink("file://!AlbumArtist!";"LossLess")"
echo !FlacHyperLink!
echo !FlacHyperLink!> hugo.txt
You need disableDelayedExpansion first, to get even exclamation marks from %1.
After that, you should switch to delayed expansion and use it anywhere.
Contents of test.bat are:
setlocal EnableExtensions EnableDelayedExpansion
set param1=%~1
echo %param1%
Can someone explain why test.bat "^^!^^^&^^^^" makes the cmd window crash but test.bat "^^^&^^^^" has an expected result of setting &^ to variable param1?
I can do test.bat "pass^^!word" and I get the expected result of pass!word.
Update: test.bat "^^!^^^^^&^^^^^^^^" works. But I'm not completely sure why. This gets interpreted to set param1=^!^^&^^^^. Why does ^ need ^^^ in front of it?
You got many problems, as the special characters will be evaluated mulitple times in your case.
First in the set command, the special character phase will reduce your string "^^!^^^&^^^^" to
^!^&^^
But as delayed expansion is enabled and your string contains an exclamation mark,
another phase will be enabled to reduce the carets again to.
!&^
At this point param1 contains !&^, you can test it with set param1
But as you try to echo the value with echo %param1% another expansion will be executed.
And now you get a problem, as %param1% will expand to !&^,
The exclamation mark will be removed, as the second exlamation mark is missing for expanding a variable,
the ampersand will be treated as new command separator and
the ending caret will be treated as multiline character.
echo ! & ^<next line>
It's much safer to use the delayed expansion here, as this never change the content, as this phase is the last one of the parser.
setlocal EnableDelayedExpansion
set param1=%~1
set param1
echo !param1!
And all these explanations can be found at How does CMD.EXE parse scripts?
It is because the escape character for the Windows shell is ^, so:
"^^!^^^^^&^^^^^^^^"
Each ^^ becomes ^
Each ^& becomes &
So finally you will get:
"^!^^&^^^^"
This code
#echo off
setlocal EnableDelayedExpansion
set myvar=first
set first=second
echo myvar:!myvar!
set myvar=!myvar!
echo myvar:!myvar!
gives
myvar:first
myvar:first
on Windows Vista SP2.
The output I had expected is
myvar:first
myvar:second
Why the difference and how to obtain desired effect?
The problem is that set myvar=!myvar! expands to set myvar=first,
you set it with the same content, and then you ask echo myvar:!myvar! to show the content of myvar.
I will try to add some more explanations, even if Aacini and shf301 already answered the question.
Both showed the double expansion with the !%var%! construct, and Aacini explained why it can work, and why the reversed version %!var!% can't work.
IMHO there are four different expansions.
Delayed Expansion:
As Aacini explained the delayed expansion is safe against any special characters in the content (it can handle ALL characters from 0x01 to 0xFF).
Percent Expansion:
The percent expansion can't handle or removes some characters (even with escaping).
It can be useful for simple content, as it can expand variables after an endlocal barrier.
setlocal
set "myVar=simple content"
(
endlocal
set result=%myVar%
)
FOR-Loop-Parameters expansion:
It is safe, if the delayed expansion is disabled, else the delayed expansion phase is executed after the expansion of the %%a variables.
It can be useful, as it can expand variables after an endlocal barrier
setlocal EnableDelayedExpansion
set "var=complex content &<>!"
for /F "delims=" %%A in ("!var!") DO (
endlocal
set "result=%%A"
)
SET Expansion:
set var expands also a variable, and it is always safe and works independent of the delayed expansion mode.
Aacini just explained how the call %%%var%%% construct work, I only want to give some additional remarks.
call is stackable, you can use many of them and each restarts the parser.
set "var=%%var%%#"
call call call call call echo %var%
results to %var%######
But call have many disadvantages/side effects!
Each call double all carets ^
You can say: "Hey I've tested it and I can't see any doubling"
call call call call echo ^^
result ^
Nevertheless it's true, but it's mostly hidden, as each restart also have a special character phase where carets escapes the next character, but you can see the doubling effect with
call call call call echo "^^"^^
result "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^"^
Even if a call expansion restarts the parser, you can never use the delayed expansion in any phase (only in the first one).
call stops working if it detects unescaped special characters.
echo you ^& me
call echo you & me
call echo you ^& me
call echo you ^^& me
call echo you ^^^& me
Only the first results to the output you & me, all the others fails.
Another problem is that call is extremly slow, a call set var=content is ~50times slower than set var=content, the cause is that call try to start an external program.
#echo off
setlocal
(
echo echo *** External batch, parameters are '%%*'
) > set.bat
set "var="
call set var=hello
set var
I hope it was interesting a bit ...
And if you want to go more in depth you can read CALL me, or better avoid call
and How does the Windows Command Interpreter (CMD.EXE) parse scripts?
This problem is not directly related to Delayed variable Expansion, but to the fact that two value expansions are required: the first one give the variable name and the second one must replace this name by its value. The direct way to do that is via two expansions in the same line as shown in the previous answer: set myvar=!%myvar%! that works because %var% expansion is done before the command-line is analyzed for execution whereas !var! expansion is done later, just before the command is executed (hence the "delayed" name). This mean that %var% expansion may provide parts of the command and may cause syntax errors, but !var! not. For example if %var%==value ... cause an error if var is empty or have spaces, but if !var!==value ... never cause a syntax error.
The double expansion of values may be achieved in other ways that does not involve Delayed variable Expansion. For example, we may create an auxiliary Batch file that do the second expansion:
echo myvar:%myvar%
echo set myvar=%%%myvar%%%> auxiliary.bat
call auxiliary
echo myvar:%myvar%
Previous method may be used to do a third or even deeper level expansions, and even be combined with Delayed Expansions to create very complex value managements. This matter is not just a curiosity, but the key to access array elements or linked lists. For example:
set month[1]=January
set month[2]=February
. . .
set month[12]=December
for /f "tokens=1-3 delims=/" %%a in ("%date%") do echo Today is !month[%%a]! %%b, %%c
What you're trying to do won't work - delayed expansion only changes the variable expansion behavior of a variable inside of a block. It doesn't allow you the aliasing/nesting (for a lack of a better word) that you are attempting.
set myvar=first sets the variable myvar to the text "first". set first=second sets the variable first to the text "second. There is no link between those two lines. myvar will never evaluate to something that it wasn't explicitly set to.
I don't believe there is anyway to accomplish what you are trying to do here.
* Edit *
OK after taking a look at your answer I seeing how that works, you can get your desired output with this:
#echo off
setlocal EnableDelayedExpansion
set myvar=first
set first=second
echo myvar:%myvar%
set myvar=!%myvar%!
echo myvar:%myvar%
So the magic seems to happen because of the way that standard and delayed expansion occur. The line set myvar=!%myvar%! is seems be expanded first by the standard expander to set myvar=!first! (you'll see this if you run the script with echo on). Then the delayed expander runs and expands !first to "second" and set's myvar to that.
I have no idea if this is documented behavior as to how standard and delayed expansion should work or just an implementation detail (which means it could break in the future)