I have been translating some shell code to MS-DOS Batch. In my code, I have the sample:
for %%i in (%*) do set "clargs=!clargs! %%i"
If I input the argument "-?" (without the quotation marks), it is not added to clargs. I assume it is because '?' is a wildcard character. Is there anything I can do to ensure that for does not do special things because of the question mark being located in the argument?
You are correct, the wild card characters * and ? are always expanded when used within a FOR IN() clause. Unfortunately, there is no way to prevent the wildcard expansion.
You cannot use a FOR loop to access all parameters if they contain wildcards. Instead, you should use a GOTO loop, along with the SHIFT command.
set clargs=%1
:parmLoop
if "%~1" neq "" (
set clargs=%clargs% %1
shift /1
goto :parmLoop
)
Although your sample is quite silly, since the resultant clargs variable ends up containing the same set of values that were already in %*. If you simply want to set a variable containing all values, simply use set clargs=%*
More typically, an "array" of argument variables is created.
set argCnt=0
:parmLoop
if "%~1" equ "" goto :parmsDone
set /a argCnt+=1
set arg%argCnt%=%1
shift /1
goto :parmLoop
:parmsDone
:: Working with the "array" of arguments is best done with delayed expansion
setlocal enableDelayedExpansion
for /l %%N in (1 1 %argCnt%) do echo arg%%N = !arg%%N!
See Windows Bat file optional argument parsing for a robust method to process unix style arguments passed to a Windows batch script.
#ECHO OFF
SETLOCAL
SET dummy=%*
FOR /f "tokens=1*delims==" %%f IN ('set dummy') DO CALL :addme %%g
ECHO %clargs%
GOTO :eof
:addme
IF "%~1"=="" GOTO :EOF
IF DEFINED clargs SET clargs=%clargs% %1
IF NOT DEFINED clargs SET clargs=%1
SHIFT
GOTO addme
I severly doubt you'll get a completely bullet-proof solution. The above solution will drop separators (comma, semicolon, equals) for instance. Other solutions may have problems with close-parentheses; there's the perpetual % and ^ problems - but it will handle -?
But for your purposes, from what you've shown, what's wrong with
set clargs=%clargs% %*
(No doubt you'll want to process further, but ve haff vays...)
Related
In a complex batch file I want to read in files with paths, among other things, to read them into a variable one after the other separated by spaces.
This works with the following code so far quite well - but only if the path does not contain an exclamation mark.
Even using the setlocal command (enabledelayedexpansion / disabledelayedexpansion) I did not succeed in processing exclamation marks.
Does anyone here have a clever idea to the problem?
The following example batch creates a text file in the current directory and then reads it in a for /F loop.
At the end all three paths from the text file should be in the variable %Output%. But with the exclamation mark.
#echo off
setlocal enabledelayedexpansion
echo This is an example^^! > "textfile.txt"
echo This is a second example^^! >> "textfile.txt"
echo And this line have an ^^! exclamation mark in the middle >> "textfile.txt"
for /F "usebackq tokens=* delims=" %%a in (textfile.txt) do (
set "Record=%%a"
set "Output=!Output!!Record! - "
)
)
echo %Output%
echo !Output!
endlocal
The Output is like this:
This is an example - This is a second example - And this line have an exclamation mark in the middle
But should be like this:
This is an example! - This is a second example! - And this line have an ! exclamation mark in the middle
It is advisable not using delayed variable expansion on processing files and directories, lines in a text file, strings not defined by the batch file itself, or output captured from the execution of a program or a command line. If it is for some reasons necessary to make use of delayed variable expansion inside a FOR loop, there should be first assigned the file/directory name, the line, or the string to process to an environment variable while delayed expansion is disabled and then enable delayed expansion temporary inside the FOR loop.
Here is a batch file demo which can be simply run from within a command prompt window or by double clicking on the batch file. It creates several files for demonstration in the directory for temporary files, but deletes them all before exiting.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
echo This is an example!> "%TEMP%\TextFile.tmp"
echo This is a second example!>> "%TEMP%\TextFile.tmp"
echo And this line has an exclamation mark ! in the middle.>> "%TEMP%\TextFile.tmp"
set "Output="
(for /F usebackq^ delims^=^ eol^= %%I in ("%TEMP%\TextFile.tmp") do set "Line=%%I" & call :ConcatenateLines) & goto ContinueDemo
:ConcatenateLines
set "Output=%Output% - %Line%" & goto :EOF
:ContinueDemo
cls
echo/
echo All lines concatenated are:
echo/
echo %Output:~3%
set "Output="
del "%TEMP%\TextFile.tmp"
echo File with name ".Linux hidden file!">"%TEMP%\.Linux hidden file!"
echo File with name "A simple test!">"%TEMP%\A simple test!"
echo File with name " 100%% Development & 'Test' (!).tmp">"%TEMP%\ 100%% Development & 'Test(!)'.tmp"
echo/
echo Files with ! are:
echo/
for /F "eol=| tokens=* delims=" %%I in ('dir "%TEMP%\*!*" /A-D /B /ON 2^>nul') do (
set "NameFile=%%I"
set "FileName=%%~nI"
set "FileExtension=%%~xI"
set "FullName=%TEMP%\%%I"
setlocal EnableDelayedExpansion
if defined FileName (
if defined FileExtension (
echo File with ext. !FileExtension:~1!: !NameFile!
) else (
echo Extensionless file: !NameFile!
)
) else echo Extensionless file: !NameFile!
del "!FullName!"
endlocal
)
endlocal
echo/
#setlocal EnableExtensions EnableDelayedExpansion & for /F "tokens=1,2" %%G in ("!CMDCMDLINE!") do #endlocal & if /I "%%~nG" == "cmd" if /I "%%~H" == "/c" set /P "=Press any key to exit the demo . . . "<nul & pause >nul
The output of this batch file is:
All lines concatenated are:
This is an example! - This is a second example! - And this line has an exclamation mark ! in the middle.
Files with ! are:
File with ext. tmp: 100% Development & 'Test(!)'.tmp
Extensionless file: .Linux hidden file!
Extensionless file: A simple test!
The text file example with concatenating lines makes use of a subroutine called from within the FOR loop processing the lines in the text file. The syntax used here is for maximum performance by getting the subroutine as near as possible to the FOR command line. That is important if the FOR loop has to process hundreds or even thousands of items.
The example processing file names enables and disables delayed expansion inside the FOR loop after having assigned all parts of the currently processed file to environment variables. It could be useful to reduce the list of environment variables before processing thousands of files for a better performance on using this method.
Another method is shown in Magoo´s answer using the command CALL to get a command line with referenced environment variables (re)defined inside the loop parsed a second time. I used that method also in the past quite often, but don't that anymore as it is not fail-safe and not efficient. call set results in searching by cmd.exe in current directory and next in all directories of environment variable PATH for a file with name set and a file extension of environment variable PATHEXT. So it results in lots of file system accesses in the background on each iteration of the FOR loop and if there is by chance a file set.exe, set.bat, set.cmd, etc. found by cmd.exe somewhere, the batch file does not work anymore as expected because of running the executable or calling the batch file instead of the (re)definition of the environment variable.
The following answers written by me could be also helpful:
How to read and print contents of text file line by line?
It explains in full details how to process all lines of a text file.
How to pass environment variables as parameters by reference to another batch file?
It explains in full details what the commands SETLOCAL and ENDLOCAL do.
How to pass a command that may contain special characters (such as % or !) inside a variable to a for /f loop?
This is an example of a batch file designed to process video files with any valid file name on any Windows computer very efficient, safe and secure with full explanation.
Well, the main trick is to enable delayed expansion only when it is actually needed and to disable it otherwise. Since you are accumulating multiple strings in a single variable inside of a loop, it becomes a bit more difficult, because you should have delayed expansion disabled during expansion of for meta-variables (like %%a), but enabled when joining the string, leading to setlocal and endlocal statements inside of the loop. The major purpose of these commands is environment localisation, hence any variable changes become lost past endlocal, so a method of tansfering the value beyond endlocal is required, which is incorporated in the following code:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem /* At this point delayed expansion is disabled, so there is no need to
rem escape exclamation marks; moreover a redirected block prevents
rem superfluous file close/reopen operations, and there is no more
rem trailing space written to the file (as in your original approach): */
> "textfile.txt" (
echo This is an example!
echo This is a second example!
echo And this line have an ! exclamation mark in the middle
)
rem // Let us initialise the output variable:
set "Output= - "
rem // Using `usebackq` only makes sense when you want to quote a file path:
for /F "usebackq tokens=* delims=" %%a in ("textfile.txt") do (
rem // Remember that delayed expansion is still disabled at this point:
set "Record=%%a"
rem // For concatenation we need delayed expansion to be enabled:
setlocal EnableDelayedExpansion
set "Output=!Output!!Record! - "
rem /* We need to terminate the environment localisation of `setlocal`
rem inside of the loop, but we would lose any changes in `Output`;
rem therefore let us (mis-)use `for /F`, which is iterated once: */
for /F "delims=" %%b in ("!Output!") do endlocal & set "Output=%%b"
rem /* An often used method to transfer a variable beyond `endlocal` is
rem the line `endlocal & set "Output=%Output%`, but this only works
rem outside of a parenthesised block because of percent expansion. */
)
rem /* Echo out text with delayed expansion enabled is the only safe way;
rem surrounding separators ` - ` are going to be removed; since `Output`
rem was initialised with something non-empty, we do not even need to skip
rem sub-string expansion for the problematic case of an empty string: */
setlocal EnableDelayedExpansion
echo(!Output:~3,-3!
endlocal
endlocal
exit /B
Pew. I finally got it to work.
It works via a workaround using a second text file.
Not pretty, not performant, but it works and is sufficient for my purposes.
#Magoo, thanks for your post.
This is my solution:
#echo off
setlocal enableextensions enabledelayedexpansion
echo This is an example^^!> "textfile.txt"
echo This is a second example^^!>> "textfile.txt"
echo And this line have an ^^! exclamation mark in the middle>> "textfile.txt"
echo.
echo Content of the textfile:
type "textfile.txt"
set output=
del "textfile2.txt" 1> nul 2>&1
setlocal disabledelayedexpansion
for /f "usebackq tokens=* delims=" %%a IN ("textfile.txt") do (
rem Write each line without a newline character into a new text file
echo|set /p "dummy=%%a, ">>"textfile2.txt"
)
endlocal
rem Loading the content of the new text file into the variable
set /p output=<"textfile2.txt"
del "textfile2.txt" 1> nul 2>&1
echo.
echo --------------------------------------------
echo Content of the variable:
set out
endlocal
The output looks like this:
Content of the textfile:
This is an example!
This is a second example!
And this line have an ! exclamation mark in the middle
--------------------------------------------
Content of the variable:
output=This is an example!, This is a second example!, And this line have an ! exclamation mark in the middle,
It's delayedexpansion mode that appears to raise this problem.
#ECHO OFF
setlocal enabledelayedexpansion
echo This is an example^^^! > "textfile.txt"
echo This is a second example^^^! >> "textfile.txt"
echo And this line have an ^^^! exclamation mark in the middle >> "textfile.txt"
TYPE "textfile.txt"
SETLOCAL disabledelayedexpansion
for /F "usebackq tokens=* delims=" %%a in (textfile.txt) do (
set "Record=%%a"
CALL set "Output2=%%Output2%%%%record%% - "
CALL set "Output=%%Output%%%%a - "
SET out
)
)
endlocal&SET "output=%output%"
echo %Output%
echo !Output!
SET out
I've no doubt that with delayedexpansion off, there would be the same problem with %. Just special characters, I suppose.
Note that with endlocal&SET "output=%output%", the set is executed in delayedexpansion mode.
The problem is echo !out! show all lines but echo %data% which assigned from !out! only show first line
#echo off
call :f
echo %data%
:f
setlocal EnableDelayedExpansion
set out=
set NL=^
for /f "delims=" %%i in ('dir /b') do (
if defined out set "out=!out!!NL!"
set "out=!out!%%i"
)
echo !out!
set data=!out!
exit /b 0
What is the correct way to pass the value entirely to other variable which i need to use freely after endlocal?
Thank you #aschipfl's for your answer that clarify that it's impossible to do it without end up in for-in loop again. Actually all that I want is to simplify my code without have to write for-in loop many times by turn it into a subroutine. I've solved it by passing the subroutine to the subroutine instead.
As derived from this thread, you cannot echo a multi-line string using immediate (%-)expansion, because everything after the first line-break is ignored.
To make your script working you need to correct two issues:
before the line endlocal & set data=%out% you must replace every new-line in variable out by an escaped new-line, that is the sequence ^ plus new-line plus new-line, which is exactly the same that you are using for defining the variable NL;
echo %data% truncates the displayed string at the first occurrence of a new-line in the value of variable data, so you need to use set data to show the actual content of the variable (or more precisely said, of all variables whose names begin with data);
Both of these items are commented (rem) in the following code:
#echo off
setlocal EnableDelayedExpansion
set out=
set NL=^
for /F "delims= eol=|" %%i in ('dir /b') do (
if defined out set "out=!out!!NL!"
set "out=!out!%%i"
)
echo * Original variable content:
set out
rem // Replace every new-line by an escaped new-line:
set out=!out:^%NL%%NL%=^^^%NL%%NL%^%NL%%NL%!
echo * Modified variable content:
set out
endlocal & set data=%out%
rem // Do not use `echo` to show true content of variable:
echo * Returned variable content:
set data
echo * Mistaken variable content:
echo %data%
exit /B 0
Although the variable value is now correctly passed over the endlocal barrier, this approach is not exactly brilliant, because it does not allow you to use variable %data% (again because everything after the first line-break is ignored as initially mentioned), unless you have got delayed expansion enabled in the hosting cmd instance, which would permit to use !data!.
Another remaining problem is that special characters in the multi-line string (like ^, &, (, ) and ", <, >, |) may cause syntax errors or other unexpected issues. However, this can be avoided by using a for meta-variable rather than a normal environment variable for passing the variable value beyond the endlocal barrier, because the former are expanded after special character recognition, in contrast to the latter, which are expanded before:
#echo off
setlocal EnableDelayedExpansion
set out=
set NL=^
for /F "delims=" %%i in ('dir /b') do (
if defined out set "out=!out!!NL!"
set "out=!out!%%i"
)
echo # Original variable content:
set out
rem /* Use a `for` meta-variable rather than a normal environment variable to
rem pass the variable value beyond the `endlocal` barrier;
rem a standard `for` loop can be used here, because there are not going to be
rem wildcards `?` and `*` in the variable value since they have already been
rem resolved by `dir`; `for /F` cannot be used here due to the new-lines: */
for %%j in ("!out!") do endlocal & set "data=%%~j"
rem // Do not use `echo` to show true content of variable:
echo # Returned variable content:
set data
echo # Mistaken variable content:
echo %data%
exit /B 0
The problem not being able to use variable %data% remains though.
To be able to use variable %data% with immediate expansion you could however simply store escaped new-lines rather than literal ones in the variable, because upon expansion you will have the intended literal new-line:
#echo off
setlocal EnableDelayedExpansion
set out=
set NL=^
for /F "delims=" %%i in ('dir /b') do (
if defined out set "out=!out!!NL!"
set "out=!out!%%i"
)
echo # Original variable content:
set out
rem // Replace every new-line by an escaped new-line:
set out=!out:^%NL%%NL%=^^^%NL%%NL%^%NL%%NL%!
echo # Modified variable content:
set out
rem /* Use a `for` meta-variable rather than a normal environment variable to
rem pass the variable value beyond the `endlocal` barrier;
rem a standard `for` loop can be used here, because there are not going to be
rem wildcards `?` and `*` in the variable value since they have already been
rem resolved by `dir`; `for /F` cannot be used here due to the new-lines: */
for %%j in ("!out!") do endlocal & set "data=%%~j"
rem // Do not use `echo` to show true content of variable:
echo # Actual variable content:
set data
echo # Parsed variable content:
echo %data%
exit /B 0
But regard that this is only going to work when %data% does not appear within quoted ("") strings.
This question already has answers here:
Make an environment variable survive ENDLOCAL
(8 answers)
Closed 3 years ago.
I'm trying to use a FOR loop to read the lines in a text file, but I also need to keep track of some variables and evaluate them. The easiest way to do that is by enabling DelyaedExpansion. Actually, it seems to be the ONLY way as everything else I've tried in relation to variables fails miserably if I don't use it. Unfortunately, this means that if any of the lines of text in the file contain exclamation points, they will be stripped out.
I thought I had found a solution by reading a line of text and putting it into a variable, THEN enabling DelayedExpansion, doing the variable operations, and finally using ENDLOCAL & SET VARIABLE=%VARIABLE% to preserve the value. Unfortunately that doesn't seem to work if the ENDLOCAL statement is inside a loop.
For example;
echo off
for /F "delims=" %%F in (test.txt) do (
set Line=%%F
setlocal enabledelayedexpansion
set /a Count=Count+1
echo !Count! - !Line!
endlocal & set Count=%Count%
)
echo Total: %Count%
Each time the loop repeats, the value of "Count" is reset to zero.
If I move the SETLOCAL command before the FOR command, it will strip any "!" from the text, which is unacceptable.
Please note: The example above is only a small part of a much larger script that does many things with the variables inside the loop. I have boiled the problem down to the bare minimum to make it easy to understand. I need to preserve "!" in text read from a file while also being able to perform multiple variable operations within each loop.
So I either need a way to read text from a file, one line at a time, with DeleyedExpansion enabled AND preserve any "!" in the text, or preserve the value of variables that are defined within the SETLOCAL/ENDLOCAL commands within a loop.
With Help from dbenham and his answer here, There is a Solution that exists for this Scenario.
The key, as Dave has Shown, is in Setting the variables PRIOR to using SetlocalEnableDelayedExpansion so that ! is preserved.
#echo off
Set "count=0"
For /F "delims=" %%F in (test.txt) do (
Call :LineParse "%%~F"
)
REM The Below Loop demonstrates Preservation of the Values
Setlocal EnableDelayedExpansion
For /L %%a in (1,1,!count!) DO (
ECHO(!line[%%a]!
)
Endlocal
pause
exit
:LineParse
Set /a count+=1
Set "Line[%count%]=%~1"
Setlocal EnableDelayedExpansion
ECHO(!Line[%count%]!
ECHO(Total: !count!
(
ENDLOCAL
)
GOTO :EOF
There are still a few characters that will not be parsed as desired with this Method, noted in test.txt:
test.txt
Safe Characters: ! > * & ` ' . ] [ ~ # # : , ; ~ } { ) ( / \ ? > < = - _ + $ |
problem Characters: ^ "" %%
problem examples:
line disappears " from single doublequote
but not "" from escaped doublequote
%% will not display unless escaped. % unescaped Percent Symbols will attempt to expand %
caret doubles ^ ^^ ^^^
Don't need to complicate...
Just replace:
echo/ to set /p
setlocal enabledelayedexpansion to cmd /v /c
#echo off
for /F "delims=" %%F in ('type test.txt')do set /a "Count+=1+0" && (
(echo/ & cmd /v/s/c "set/p "'=!Count! - %%F"<nul")>>".\newfile.txt")
cmd /v /c echo/ Total: !Count! && call set "Count="<nul && goto :EOF
This code snippet below is stripped of all the extra junk, down to just the error-generating code,
)"" was unexpected at this time.
#echo off
SETLOCAL ENABLEDELAYEDEXPANSION ENABLEEXTENSIONS
Set "regex="(Test_Health=(?!100))""
echo Regex is: %regex%
FOR /L %%I IN (1,1,5) DO (
Set "to_call=call crv.exe "%%I" %regex%"
echo About to call: !to_call!
)
Basically, in the real script, I'm trying to call a command-line tool that takes a complex string with potentially special chars in it, as well as a regex.
I figured out a workaround, which was to add a single caret (^) before %%I's last quote:
Set "to_call=call crv.exe "%%I^" %regex%"
But that feels like a dirty hack. What am I doing wrong, and what should I do to get the desired behavior without a dirty hack?
To fix your problem without a hack:
Make sure that the ! char. in your regex variable value is recognized as a literal:
Set "regex=(Test_Health=(?^!100))"
Due to setlocal enabledelayedexpansion, literal ! chars. inside "..." must be escaped as ^!.
Note that the <name>=<value> token is double-quoted as a whole, to prevent additional interpretation of the value.
Reference variable regex delayed inside the for loop body:
Use !regex! instead of %regex%.
To make the resulting command line more robust - even though it's not needed in this specific case - ensure that the value of regex is enclosed in double quotes (note that %%I - as a mere number - does not need quoting):
Set "to_call=call crv.exe %%I "!regex!""
To put it all together:
#echo off
setlocal enabledelayedexpansion enableextensions
Set "regex=(Test_Health=(?^!100))"
echo Regex is: %regex%
FOR /L %%I IN (1,1,5) DO (
Set "to_call=call crv.exe %%I "!regex!""
echo About to call: !to_call!
)
yields:
Regex is: (Test_Health=(?100))
call crv.exe 1 "(Test_Health=(?!100))"
call crv.exe 2 "(Test_Health=(?!100))"
call crv.exe 3 "(Test_Health=(?!100))"
call crv.exe 4 "(Test_Health=(?!100))"
call crv.exe 5 "(Test_Health=(?!100))"
As for what you did wrong:
%<name>%-style variable references - except for the loop variable (%%I in this case) - are expanded inside the loop body ((...)) before the loop is even parsed, so the values of such variable references can break the loop.
Here's a minimal example that demonstrates the problem:
#echo off
Set "regex=))"
FOR %%I IN ("dummy") DO (
rem !! breaks, because the up-front %regex% expansion causes a syntax error.
echo %regex%
)
Delaying the expansion - by enclosing the variable name in !...!, assuming setlocal enabledelayedexpansion is in effect - bypasses this problem:
#echo off
setlocal enabledelayedexpansion
Set "regex=))"
FOR %%I IN ("dummy") DO (
rem OK - outputs "))"
echo !regex!
)
#ECHO OFF
SETLOCAL
Set "regex="(Test_Health=(?^^^^!100^^)^^)""
SETLOCAL ENABLEDELAYEDEXPANSION ENABLEEXTENSIONS
echo Regex is: %regex%
FOR /L %%I IN (1,1,5) DO (
Set "to_call=call crv.exe "%%I" %regex%"
echo About to call: !to_call!
)
GOTO :EOF
Sort of depends on what your "desired behaviour" is. Unfortunately, you don't specify.
It's a matter of understanding how cmd works - by substitution, using escape characters and the sequence that this occurs.
The echo reporting the regex won't yield the correct result. Within the for however, each pair of carets is interpreted as a single caret, so the required escapes are as required for the expected output, presumably call crv.exe "5" "(Test_Health=(?!100))" and the like...
Greetings, dear Experts!
I want to check the existense of parameter (or argument) sting at all in my batch script:
if "%*"=="" findstr "^::" "%~f0"&goto :eof
This works fine if none of parameters is enclosed in double quotes. For example:
test.bat par1 par2 ... --- works
but
test.bat "par 1" par2 ... --- fails
My question are:
1) Is there any way to overcome this instead of requirement for use to use non-double-quoted symbol to specify "long" arguments and then use string substitution?
2) Can I ever use "if" to compare two strings containing both double quotes and spaces?
Your prompt and clear reply would be very much appreciated.
~ will strip the double quotes, but does not work on %*, but if you just want to know if there are no parameters, just checking %1 should be enough
if "%~1"==""
You might want to call setlocal ENABLEEXTENSIONS first to make sure extensions are on (required for ~)
Thank you, Andres.
Here are two more pieces of code to check the existence and number of passed parameters:
set ArgumentString=%*
if not defined ArgumentString findstr "^::" "%~f0"&goto :eof
if "%ArgumentString:"=%"=="" findstr "^::" "%~f0"&goto :eof
set NumberOfArguments=0
for /l %%N in (1,1,9) do (
call set CurrentArgument=%%%%N
if defined CurrentArgument set /a NumberOfArguments += 1
)
if %NumberOfArguments% NEQ %N% findstr "^::" "%~f0"&goto :eof
Here variable N contains needed number of parameters.
Hope this helps for somebody!
Since Andrey's answer fails to correctly count arguments, here is a way that does work, even with more than 9 arguments (and it even perserves the original ones, so %1 still points to the first one by moving the shifting into a subroutine):
#echo off
call :get_num_args %*
echo %NumArgs%
goto :eof
:get_num_args
set NumArgs=0
:loop
if not [%1]==[] (set /a NumArgs+=1) else (goto :eof)
shift
goto loop
goto :eof