While checking the details of the axis2server.bat file in Axis2 binary distribution I see one of the line containing text something like:
FOR %%c in ("%AXIS2_HOME%\lib\*.jar") DO set AXIS2_CLASS_PATH=!AXIS2_CLASS_PATH!;%%c
What does the part below with 2 exclamation marks mean?
!AXIS2_CLASS_PATH!
Names with in % mean variables, not sure what ! mark mean in a batch file.
When you enable delayed expansion and change or set a variable within a loop then the !variable! syntax allows you to use the variable within the loop.
A drawback is that ! becomes a poison character for delayed expansion.
As foxidrive mentioned, this is related to delayed expansion. You can find more information by running help set in a cmd prompt, which has the following explanation:
Finally, support for delayed environment variable expansion has been
added. This support is always disabled by default, but may be
enabled/disabled via the /V command line switch to CMD.EXE. See CMD /?
Delayed environment variable expansion is useful for getting around
the limitations of the current expansion which happens when a line
of text is read, not when it is executed. The following example
demonstrates the problem with immediate variable expansion:
set VAR=before
if "%VAR%" == "before" (
set VAR=after
if "%VAR%" == "after" #echo If you see this, it worked
)
would never display the message, since the %VAR% in BOTH IF statements
is substituted when the first IF statement is read, since it logically
includes the body of the IF, which is a compound statement. So the
IF inside the compound statement is really comparing "before" with
"after" which will never be equal. Similarly, the following example
will not work as expected:
set LIST=
for %i in (*) do set LIST=%LIST% %i
echo %LIST%
in that it will NOT build up a list of files in the current directory,
but instead will just set the LIST variable to the last file found.
Again, this is because the %LIST% is expanded just once when the
FOR statement is read, and at that time the LIST variable is empty.
So the actual FOR loop we are executing is:
for %i in (*) do set LIST= %i
which just keeps setting LIST to the last file found.
Delayed environment variable expansion allows you to use a different
character (the exclamation mark) to expand environment variables at
execution time. If delayed variable expansion is enabled, the above
examples could be written as follows to work as intended:
set VAR=before
if "%VAR%" == "before" (
set VAR=after
if "!VAR!" == "after" #echo If you see this, it worked
)
set LIST=
for %i in (*) do set LIST=!LIST! %i
echo %LIST%
I wanted to hand over a string containing a "!" as parameter (for imageMagick) and the ! was of course interpreted as syntax which broke my script. The solution was for me, change my string from
"Hello World!"
to (just added a ^ before the !):
"Hello World^!"
I found this trick by reading here: https://www.robvanderwoude.com/escapechars.php
Related
I'm over my head with this - spent too much time searching already - evidently I don't understand the basics of CMD variables etc. - and it always gives me such a headache
why wouldn't this work?
for %a in (*) do ( set tmpx=%a & echo %tmpx% )
the above code outputs the value of %tmpx% in some other scope - and it is always constant
yes, i run setlocal ENABLEDELAYEDEXPANSION
basically i need to do a simple rename of all files in folder from constantstring_somenameXX.tif to somenameXX.tif, where i.e. constantstring=0000000005
i had to use set because other posts rightly suggested that %a in a for loop has a special behaviour, and the substitutions wouldn't work for it as it is.
i would prefer not to use scripts and/or powershell - unless not using them is impossible
thank you
for %a in (*) do ( set tmpx=%a & echo %tmpx% )
The problem with the previous code is delayed expansion. Yes, you enabled it, but you have not used it, and depending on how you enabled it, it will not work
In cmd, when a line or block of lines (code inside parenthesis) is reached, it is first parsed and then executed. During the parse phase, variable read operations are removed from the command, replaced with the value in the variable before the command starts to execute. So, if you change the value of a variable inside a line/block you can not retrieve the changed value inside the same line/block as there are no variable reads (they were replaced)
setlocal enabledelayedexpansion allows to replace (where needed) the variable read syntax from %var% to !var!, indicating to the parser that the read operation will be delayed until the execution phase.
So, in your case, your code should have been something like
setlocal enabledelayedexpansion & for %a in (*) do ( set "tmpx=%a" & echo !tmpx! )
BUT this will not work (in default configured environments).
cmd has two execution modes: batch file and command line. In your case, you are using command line (no escaped percent sign in for loop) and in command line mode the setlocal enabledelayedexpansion will not work. It is intended for batch files (see setlocal /?)
How to make it work from the command line? By default cmd is started with delayed expansion disabled and you can not enable it if not inside a batch file. But you can start cmd with delayed expansion enabled and run your command in this started instance (see cmd /?)
cmd /v:on /c "for %a in (*) do ( set "tmpx=%a" & echo !tmpx! )"
Anyway, to solve your rename problem, delayed expansion is not needed
for %a in (*_*.tif) do for /f "tokens=1,* delims=_" %b in ("%~nxa") do echo ren "%a" "%c"
That is, for each tif file with an underscore, take the name and extension of the file (%~nxa) as a string, and using the underscore as a delimiter between tokens, retrieve the first token (the text on the left of the first underscore) in %b and the rest of the text (to the right of the underscore) into %c. Now, just rename the original file name (stored in %a) to the contents of %c (the text on the right of the underscore)
In this code rename operations are only echoed to console. If the output is correct, remove the echo command.
! is the character to use rather than % when wanting execution time value. % does when it's read value.
CMD was written by IBM engineers and they were trying to make MSDos a programming language while making sure Dos commands ran the same. So we get a hodge podge.
& seperates commands on a line.
&& executes this command only if previous command's errorlevel is 0.
|| (not used above) executes this command only if previous command's errorlevel is NOT 0
> output to a file
>> append output to a file
< input from a file
| output of one command into the input of another command
^ escapes any of the above, including itself, if needed to be passed to a program
" parameters with spaces must be enclosed in quotes
+ used with copy to concatinate files. E.G. copy file1+file2 newfile
, used with copy to indicate missing parameters. This updates the files modified date. E.G. copy /b file1,,
%variablename% a inbuilt or user set environmental variable
!variablename! a user set environmental variable expanded at execution time, turned with SelLocal EnableDelayedExpansion command
%<number> (%1) the nth command line parameter passed to a batch file. %0 is the batchfile's name.
%* (%*) the entire command line.
%<a letter> or %%<a letter> (%A or %%A) the variable in a for loop. Single % sign at command prompt and double % sign in a batch file.
Currently I have a loop that runs through a list of items and copies them to a directory (archive). However, one of the items in the list (which has a global variable in the path-name) is being interpreted 'literally' (as text instead of code).
I know usually you can just escape the line via (^^) to have it interpreted as code instead of text, but evidently I'm doing something wrong here, because it's not working...
The item in the my.list (with the escape in it) that is having issues is:
location\foo^^%date:~0,3%*.zip
The code I'm using is...
for /f "delims=" %%a in (my.list) do (
echo "%%a"
)
Echo's
"location\foo^^%date:~0,3%*.zip"
Instead of
location\fooMON*.zip
Any help would be greatly appreciated.
You are confused as to when you need to escape a character.
Some characters have special meaning ("code" as you describe it). Often times you can escape the character such that it is interpretted as a literal (text) instead of "code".
The most frequent method to escape a character within Windows CMD.EXE is to prefix it with a single ^ character. Sometimes a string is parsed twice, which can require an escape sequence of ^^^, (or perhaps ^^ when dealing with ! when delayed expansion is enabled). More rounds of parsing require ever more ^ characters. It can quickly become confusing, and requires practice to get the hang of it.
But your situation is completely different - It cannot be solved by escaping. You have "code" within your FOR variable, and you want it to be interpreted as such. But instead, it is being interpreted as text. In order to understand why, you must understand the order in which various stages of batch parsing occur. You could refer to How does the Windows Command Interpreter (CMD.EXE) parse scripts?, but it is pretty advanced stuff that takes time to digest.
Here is a crude synopsis showing when various types of expansion occur. (Note - these step numbers do not match up exactly with the phase numbers of the linked answer)
1) Parameter expansion - %1
2) Normal variable expansion - %var%
3) FOR variable expansion - %%A
4) Delayed variable expansion - !var!
5) CALL expansion - repeat steps 1) and 2) if CALL involved
You want your %date:~0,3% string to undergo normal (percent) expansion. Your FOR loop reads the line of text verbatim, without any expansion. The first time the parser sees your "code" is at step 3) when the %%a FOR variable is expanded. You can see that this is already too late to get
%date:~0,3% to expand the way you want.
You have two choices to solve your problem. But beware - each of these solutions potentially add new issues that may need to be solved.
I am assuming the ^^ is your naive attempt to force expansion of the embedded "code". The ^^ should be removed from your list file.
Option 1: Add an extra round of normal expansion by using CALL
for /f "delims=" %%a in (my.list) do call echo "%%a"
But now you have a potential problem that you might have a % literal in your list that you do not want to be expanded. Percents within batch scripts cannot be escaped with ^. Instead you escape a percent by doubling it as %%. So if you have percent literals in your list, they must be doubled.
Note that the original code that was posted with the question was significantly more complicated. It included an IF statement that referenced %%a. You cannot CALL an IF or FOR command. The solution is to CALL a subroutine, passing the value, and include the complex logic in the subroutine.
for /f "delims=" %%a in (my.list) do call :processValue "%%a" >>Logs\xfer.log
exit /b
:processValue
echo Attempting to archive %1...
if exist "c:\%~1" (
echo f | xcopy "c:\%%a" "c:\Lucas\archive\%~1" /E /C /H /R /Y
if %errorlevel%==0 (
echo ...%1 added to archive for transfer
echo.
) else (
echo ERROR: %1 not added to archive
echo.
)
) else (
echo ERROR: %1 Not found on client computer
echo.
)
Option 2: Use delayed expansion
Enable delayed expansion, and change your list to use !date:~0,3! instead of %date:~0,3%. Delayed expansion occurs after FOR variable expansion, so it will be expanded properly.
setlocal enableDelayedExpansion
for /f "delims=" %%a in (my.list) do echo "%%a"
But now you have a potential problem that you might have a ! literal in your list that you do not want to be expanded. You can preserve ! literals by escaping them as ^!.
Context: I need to call a Windows batch script which would update my PATH by adding another path 'xxx' at the end of it, but:
without any duplicate
(if I add 'xxx' to a PATH like 'aaa;xxx;bbb', I need an updated PATH like 'aaa;bbb;xxx')
without any aggregation
(I can call the script repeatedly without ending up with 'aaa;bbb;xxx;xxx;xxx;...')
What I have tried:
The following function takes care of any duplicate and does the job
:cleanAddPath -- remove %~1 from PATH, add it at the end of PATH
SETLOCAL ENABLEDELAYEDEXPANSION
set PATH=!PATH:%~2=!
set PATH=!PATH:;;=;!
set PATH=%PATH%;%~2
set P=!P:;;=;!
echo %PATH%
echo -------------
ENDLOCAL
exit /b
But, it needs delayed expansion local mode, which means: at the end of the script (or here, at the end of the function cleanAddPath), whatever has been set for %PATH% is thrown away.
I could ask the users (for which I write the script) to launch their cmd with a cmd /V:ON option (activating the delayed expansion, otherwise off by default), but that is not practical.
How can I modify the PATH variable the way I described above, and still have it updated in my current DOS session after calling said script?
The page "DOS - Function Collection" gives great example on how a function can return a value in DOS, even when using delayed expansion mode:
The following function will update any variable you want with an addition PATH:
:cleanAddPath -- remove %~2 from %~1, add it at the end of %~1
SETLOCAL ENABLEDELAYEDEXPANSION
set P=!%~1!
set P=!P:%~2=!
set P=!P:;;=;!
set P=!P!;%~2
set P=!P:;;=;!
(ENDLOCAL & REM.-- RETURN VALUES
SET "%~1=%P%"
)
exit /b
Note the concatenation of paths using. As jeb comments:
The line set P=%P%;%~2 is critical if your path contains ampersands like in C:\Documents&Settings.
Better change to set "P=!P!;%~2".
The SET "%~1=%P%" is the part which allows to memorize (in the variable represented by %~1) the value you have set using delayed expansion features.
I initially used SET "%~1=%P%" !, but jeb comments:
The command SET "%~1=%P%" ! could be simplified to SET "%~1=%P%" as the trailing exclamation mark has only a (good) effect in delayed expansion mode and if you prepared %P% before.
To update your PATH variable, you would call your function with:
call :cleanAddPath PATH "C:\my\path\to\add"
And it will persists after leaving that script, for your current DOS session.
dbenham's answer points to a more robust answer (upvoted), but in my case this script is enough.
The problem isn't as simple as you think. There are a number of issues that can break your code before it ever gets to the end where it needs to return the updated value across the ENDLOCAL barrier.
I already answered this question as an extension to an answer I provided for a similar question. See How to check if directory exists in %PATH%?. In that answer I provide a large list of issues that complicate the problem.
The code at the bottom of the linked answer shows how to reliably add a path if it does not exist in PATH already, and it also demonstrates how to reliably return the value across the ENDLOCAL barrier.
The following edits are from VonC in an attempt to actually put the answer here instead of just a link to the answer. I'll preserve the edit, but I find it difficult to follow without the context of the full linked answer.
[The answer demonstrates how to reliably return the value] using the set "%~1=%var%" ! trick (with the trailing '!')
That thread includes:
That's not clear to me. How can an exclamation mark behind the last quote influence the variable content?
The simple rule for delayed expansion is:
For each character in the line do:
If it is a caret (^) the next character has no special meaning, the caret itself is removed
If it is an exclamation mark, search for the next exclamation mark (carets are not observed here), then expands to the content of the variable
If no exclamation mark is found in this phase, the result is discarded, the result of the phase before is used instead (important for the carets)
So, at this point the difference should be clear, the carets are removed even if the exclamation mark have no other effect in a line.
Example:
#echo off
setlocal EnableDelayedExpansion
echo one caret^^
echo none caret^^ !
set "var1=one caret^"
set "var2=none caret^" !
echo !var1!
echo !var2!
----- OUTPUT ----
one caret^
none caret
one caret^
one caret
Yay! Finally got this working with the following test code:
#echo off
Setlocal enabledelayedexpansion
Set p="hello world"
( endlocal & rem return
Set "a1=%p%"
)
Set a1
This outputs:
a1="hello world"
The reason I used delayed expansion in the test without using any !'s is because it still effects how set works and the batchs I'm testing this for all have delayed expansion.
Thanks for the help guys :o)
PS I tried using the same variable name for both local and external environments but this broke the code. Hence the 2 names used.
I am trying to create a batch string like this: >abcd_
I have a variable called soeid, with value as abcd. So this is what i am doing, but it does not work.
set soeid=abcd
set "val1=>"
set "val2=_"
set "str=%val1%%soeid%%val2%"
echo %str%
I'm sure it is working just fine. To prove it, add SET STR after you define the value, and you will see the correct value.
The problem you are having is when you try to echo the value, the line that is executing becomes: echo >abcd_. The > is not quoted or escaped, so it is simply taking the ouput of ECHO with no arguments and redirecting it to a file named "abcd_"
If you don't mind seeing quotes, then change your line to echo "%str%" and it will work.
The other option is to enable and use delayed expansion (I'm assuming this is a batch script code, and not executing on the command line)
setlocal enableDelayedExpansion
set soeid=abcd
set "val1=>"
set "val2=_"
set "str=%val1%%soeid%%val2%"
echo !str!
Normal %var% expansion occurs early on while the interpreter is parsing the line. Delayed !var! expansion occurs at the end just before it is executed. The redirection is detected somewhere in the middle. That is why the normal expansion doesn't work - the interpreter sees the expanded > character and interprets it as the output redirection operator. The delayed expansion hides the > character from the interpreter until after redirection is parsed.
For more info about delayed expansion, type SET /? from the command line and read starting with the paragraph that starts with "Finally, support for delayed environment variable expansion...".
This code
#echo off
setlocal EnableDelayedExpansion
set myvar=first
set first=second
echo myvar:!myvar!
set myvar=!myvar!
echo myvar:!myvar!
gives
myvar:first
myvar:first
on Windows Vista SP2.
The output I had expected is
myvar:first
myvar:second
Why the difference and how to obtain desired effect?
The problem is that set myvar=!myvar! expands to set myvar=first,
you set it with the same content, and then you ask echo myvar:!myvar! to show the content of myvar.
I will try to add some more explanations, even if Aacini and shf301 already answered the question.
Both showed the double expansion with the !%var%! construct, and Aacini explained why it can work, and why the reversed version %!var!% can't work.
IMHO there are four different expansions.
Delayed Expansion:
As Aacini explained the delayed expansion is safe against any special characters in the content (it can handle ALL characters from 0x01 to 0xFF).
Percent Expansion:
The percent expansion can't handle or removes some characters (even with escaping).
It can be useful for simple content, as it can expand variables after an endlocal barrier.
setlocal
set "myVar=simple content"
(
endlocal
set result=%myVar%
)
FOR-Loop-Parameters expansion:
It is safe, if the delayed expansion is disabled, else the delayed expansion phase is executed after the expansion of the %%a variables.
It can be useful, as it can expand variables after an endlocal barrier
setlocal EnableDelayedExpansion
set "var=complex content &<>!"
for /F "delims=" %%A in ("!var!") DO (
endlocal
set "result=%%A"
)
SET Expansion:
set var expands also a variable, and it is always safe and works independent of the delayed expansion mode.
Aacini just explained how the call %%%var%%% construct work, I only want to give some additional remarks.
call is stackable, you can use many of them and each restarts the parser.
set "var=%%var%%#"
call call call call call echo %var%
results to %var%######
But call have many disadvantages/side effects!
Each call double all carets ^
You can say: "Hey I've tested it and I can't see any doubling"
call call call call echo ^^
result ^
Nevertheless it's true, but it's mostly hidden, as each restart also have a special character phase where carets escapes the next character, but you can see the doubling effect with
call call call call echo "^^"^^
result "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^"^
Even if a call expansion restarts the parser, you can never use the delayed expansion in any phase (only in the first one).
call stops working if it detects unescaped special characters.
echo you ^& me
call echo you & me
call echo you ^& me
call echo you ^^& me
call echo you ^^^& me
Only the first results to the output you & me, all the others fails.
Another problem is that call is extremly slow, a call set var=content is ~50times slower than set var=content, the cause is that call try to start an external program.
#echo off
setlocal
(
echo echo *** External batch, parameters are '%%*'
) > set.bat
set "var="
call set var=hello
set var
I hope it was interesting a bit ...
And if you want to go more in depth you can read CALL me, or better avoid call
and How does the Windows Command Interpreter (CMD.EXE) parse scripts?
This problem is not directly related to Delayed variable Expansion, but to the fact that two value expansions are required: the first one give the variable name and the second one must replace this name by its value. The direct way to do that is via two expansions in the same line as shown in the previous answer: set myvar=!%myvar%! that works because %var% expansion is done before the command-line is analyzed for execution whereas !var! expansion is done later, just before the command is executed (hence the "delayed" name). This mean that %var% expansion may provide parts of the command and may cause syntax errors, but !var! not. For example if %var%==value ... cause an error if var is empty or have spaces, but if !var!==value ... never cause a syntax error.
The double expansion of values may be achieved in other ways that does not involve Delayed variable Expansion. For example, we may create an auxiliary Batch file that do the second expansion:
echo myvar:%myvar%
echo set myvar=%%%myvar%%%> auxiliary.bat
call auxiliary
echo myvar:%myvar%
Previous method may be used to do a third or even deeper level expansions, and even be combined with Delayed Expansions to create very complex value managements. This matter is not just a curiosity, but the key to access array elements or linked lists. For example:
set month[1]=January
set month[2]=February
. . .
set month[12]=December
for /f "tokens=1-3 delims=/" %%a in ("%date%") do echo Today is !month[%%a]! %%b, %%c
What you're trying to do won't work - delayed expansion only changes the variable expansion behavior of a variable inside of a block. It doesn't allow you the aliasing/nesting (for a lack of a better word) that you are attempting.
set myvar=first sets the variable myvar to the text "first". set first=second sets the variable first to the text "second. There is no link between those two lines. myvar will never evaluate to something that it wasn't explicitly set to.
I don't believe there is anyway to accomplish what you are trying to do here.
* Edit *
OK after taking a look at your answer I seeing how that works, you can get your desired output with this:
#echo off
setlocal EnableDelayedExpansion
set myvar=first
set first=second
echo myvar:%myvar%
set myvar=!%myvar%!
echo myvar:%myvar%
So the magic seems to happen because of the way that standard and delayed expansion occur. The line set myvar=!%myvar%! is seems be expanded first by the standard expander to set myvar=!first! (you'll see this if you run the script with echo on). Then the delayed expander runs and expands !first to "second" and set's myvar to that.
I have no idea if this is documented behavior as to how standard and delayed expansion should work or just an implementation detail (which means it could break in the future)