Batch file variable filenames and editing - windows

Fairly new to the game and have spent many hours looking for help and examples. I've had a version of this working, although I want to streamline the process by having a single input file 'computer_listc.txt'. I basically want to read in a series of folders paths (from computer_listc.txt), and use parts of the path to write further cmd expressions.
My problem is that fname and comp_id are not being set properly. I want to make them equal to T (the filename), and then edit them.
fname needs to be the filepath with the first 12 characters removed. While comp_id needs to equal the computername specified in the filepath e.g.
fname = c_modelling\model\slave1
comp_id = VOSTRO460-1
Heres the script. Can you see my error??
for /f "usebackq tokens=*" %%T in ("C:\c_modelling\Model\cal21_cal\Computer_ListC.TXT") do (
set fname = %%T
ren fname "%fname%" "C:\%fname%:~12%"
set comp_id = %%T
%comp_id% = %{comp_id:0:12}
mkdir %%T > NUL 2>&1
echo cd %fname% >\\%%T\beo_insC.cmd
echo beo >>\\%%T\beo_insC.cmd
robocopy c:\c_modelling\model\cal21_cal \\%%T\ /s /e
start cmd.exe /c psexec \\%comp_id% -c c:\c_modelling\model\cal21_cal\beo_insC.cmd
del \\%%T\beo_insC.cmd
)
Computer_ListC.txt contains the following file names: This will eventually have a long list of computer folder paths.
VOSTRO460-1\c_modelling\model\slave1
VOSTRO460-1\c_modelling\model\slave2
I'm creating a *.cmd file for use with 'psexec', as when I direct the process to a local computer, the path defaults to \system32.
Any assistance would be very helpful!!

First issue is that SPACES in string SET statements are significant. The variablename AND the value assigned will both include any spaces, so set fname = %%T will set a variable named "fname ", not "fname" - and its value will contain the leading space before the value of %%T
Next is the perpetual delayed-expansion issue. Within a block (a sequence of parenthesised statements) any %var% will be replaced by its value at PARSE time - before execution of the loop begins.
To access the RUN-TIME value, you need to execute a SETLOCAL ENABLEDELAYEDEXPANSION statement, and while delayedexpansion is invoked, !var! accesses the RUN-time variable value.
Note however that you cannot simply invoke delayedexpansion within a loop with impunity. There is a limit to the number you can have open at any one time, so you need to 'close the bracket' with an ENDLOCAL command (reaching logical End-Of-File is equivalent to an ENDLOCAL) and the downside is that ENDLOCAL restores the environment to its value when the matching SETLOCAL was invoked. For this reason, SETLOCAL ENABLEDELAYEDEXPANSION is normally executed at the start of the batch, usually after the #echo off
Next issue is
ren fname "%fname%" "C:\%fname%:~12%"
The rename command's syntax is ren sourcename newname - exactly two arguments, enclose any argument containing spaces in "rabbit's ears" Further, the destination name is a NAME only - not a path.
Then there's this:
%comp_id% = %{comp_id:0:12}
Assuming you've fixed this to remove the spaces in variablenames and that comp_id has been assigned the value fred and you've realised that you are dealing with the RUN-time value of comp_id then this would now appear as
!comp_id!=%{comp_id:0:12}
which Batch would interpret as
fred=%{comp_id:0:12}
and promptly give up.
What you are probably expecting to do is
set comp_id=!comp_id:~0,12!
that is, set comp_id to the RUN-time value of comp_id, from the first character (0) for 12 characters.

Related

How to pass a command that may contain special characters (such as % or !) inside a variable to a for /f loop?

I have a few nested loops in my code and in some point, they're divided by a call to a label like this:
#echo off
chcp 65001
for /r %%a in (*.mkv *.mp4 *.avi *.mov) do (
echo Processing "%%~a"
call :innerloop "%%a" "%%~fa"
)
:: Instead of goto :eof, I chose cmd /k because I want the command prompt to still be open after the script is done, not sure if this is correct though
cmd /k
:innerloop
setlocal EnableExtensions EnableDelayedExpansion
for /f "delims=" %%l in ('mkvmerge.exe -i "%~1"') do (
:: Probably this would be a safer place for setlocal, but I believe that would mean that I wouldn't get to keep a single, different !propeditcmd! per processed file
echo Processing line "%%~l"
for /f "tokens=1,4 delims=: " %%t in ("%%l") do (
:: This section checks for mkv attachments. There are similar checks for chapters and global tags, all of those are handled by mkvpropedit.exe
if /i "%%t" == "Attachment" (
if not defined attachments (
set /a "attachments=1"
) else (
set /a "attachments+=1"
)
if not defined propeditcmd (
set "propeditcmd= --delete-attachment !attachments!"
) else (
set "propeditcmd=!propeditcmd! --delete-attachment !attachments!"
)
)
)
)
:: Since !propeditcmd! (which contains the parameters to be used with the executable) is called after all lines are processed, I figured setlocal must be before the first loop in this label
if defined propeditcmd (
mkvpropedit.exe "%~f1" !propeditcmd!
)
endlocal
goto :eof
The script works for most files and is divided like that to allow breaking the inner loop without breaking the outer when a pass is reached. While it works for most files, I noticed it can't handle filenames containing parenthesis % in their names, likely due to EnableDelayedExtensions.
Normally, I know I would have to escape these characters with a caret (^), but I don't know how I can do it if the special characters are inside a variable (%~1).
Is there a way to do it?
Update: I've been working a way to separate the section that needs delayed expansion from the one that needs it off just find in the end of my code the line mkvpropedit.exe "%~f1" !propeditcmd!, which both needs it off and on due to "%~f1" and !propeditcmd! respectively. I think this means there's no way around the question and escaping will be necessary.
Continuing my research, this answer seem to suggest this could be achieved with something like set filename="%~1:!=^^!". Nevertheless, this doesn't seem to be the proper syntax according to SS64. I'm also unsure if this will replace all occurrences of ! with ^! and I'm also concerned this kind of substitution could create an infinite loop and if wouldn't it be more adequate to perform this by first replacing ! with, say, ¬ before replacing it ^!.
While I intend to do testing soon to determine all of this, I'm worried I may not cover it all, so more input would definitely be appreciated.
PS: full code (88 lines) is available here if more context is needed, although I'll edit the snippet in this question as it may be requested!
Edit: I didn't think it was relevant at first, but now I think it helps to know what is an standard output from mkvmerge.exe -i:
File 'test.mkv': container: Matroska
Track ID 0: video (AVC/H.264/MPEG-4p10)
Track ID 1: audio (Opus)
Track ID 2: subtitles (SubRip/SRT)
Attachment ID 1: type 'image/jpeg', size 30184 bytes, file name 'test.jpg'
Attachment ID 2: type 'image/jpeg', size 30184 bytes, file name 'test2.jpg'
Attachment ID 3: type 'image/jpeg', size 30184 bytes, file name 'test3.jpg'
Chapters: 5 entries
Global tags: 3 entries
There is not really a need for a subroutine. Delayed variable expansion is needed finally, but it is possible to first assign the fully qualified file name to an environment variable like FileName to avoid troubles with file names containing an exclamation mark.
The rewritten code according to the code posted in the question with some comments:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "WindowTitle=%~n0"
rem Find out if the batch file was started with a double click which means
rem with starting cmd.exe with option /C and the batch file name appended
rem as argument. In this case start one more Windows command processor
rem with the option /K and the batch file name to keep the Windows command
rem processor running after finishing the processing of this batch file
rem and exit the current command processor processing this batch file.
rem This code does nothing if the batch file is executed from within a
rem command prompt window or it was restarted with the two options /D /K.
setlocal EnableDelayedExpansion
for /F "tokens=1,2" %%G in ("!CMDCMDLINE!") do (
if /I "%%~nG" == "cmd" if /I "%%~H" == "/c" (
endlocal
start %SystemRoot%\System32\cmd.exe /D /K %0
if not errorlevel 1 exit /B
setlocal EnableDelayedExpansion
)
)
rem Set the console window title to the batch file name.
title !WindowTitle!
endlocal
set "WindowTitle="
rem Get the number of the current code page and change the code page
rem to 65001 (UTF-8). The initial code page is restored at end.
for /F "tokens=*" %%G in ('%SystemRoot%\System32\chcp.com') do for %%H in (%%G) do set "CodePage=%%~nH"
%SystemRoot%\System32\chcp.com 65001 >nul 2>&1
for /R %%G in (*.mkv *.mp4 *.avi *.mov) do (
echo(
echo Processing "%%~G"
set "Attachments="
for /F "delims=" %%L in ('mkvmerge.exe -i "%%G"') do (
rem echo Processing line "%%L"
for /F "delims=: " %%I in ("%%L") do if /I "%%I" == "Attachment" set /A Attachments+=1
)
if defined Attachments (
set "FileName=%%G"
setlocal EnableDelayedExpansion
set "propeditcmd=--delete-attachment 1"
for /L %%I in (2,1,!Attachments!) do set "propeditcmd=!propeditcmd! --delete-attachment %%I"
mkvpropedit.exe "!FileName!" !propeditcmd!
endlocal
)
)
rem Restore the initial code page.
%SystemRoot%\System32\chcp.com %CodePage% >nul
endlocal
Why is the window title passed to TITLE with using delayed expansion?
An argument string must be enclosed in " if it contains after the expansion of dynamic variable, environment variable, loop variable or batch file argument references a space or one of these characters &()[]{}^=;!'+,`~<>| if all these characters should be interpreted literally by the Windows command processor cmd.exe. For that reason the third line encloses the argument string WindowTitle=%~n0 in double quotes because of %~n0 references the batch file name without file extension and without path which could contain, for example, an ampersand although that would be a very usual file name for a batch file.
See also: How does the Windows Command Interpreter (CMD.EXE) parse scripts?
The command TITLE is like the command ECHO regarding to ". It always interprets double quotes as literal characters and do not remove them from the argument string. So the usage of title "%WindowTitle%" would result in having a title for the console window which starts and ends with a double quote. That would not look nice. Therefore the batch file name as window title should be passed to the cmd.exe internal command TITLE without double quotes. But that is problematic in case of the batch file name contains a character with a special meaning for cmd.exe processing the command line before executing the command TITLE like &. For that reason delayed variable expansion is enabled and used here to reference the batch file name assigned to the environment variable WindowTitle which makes it possible to get the window title really set according to the batch file name.
Why is current code page determined and restored at end?
A good written batch file for usage by many people changing something on execution environment should always restore the initial execution environment, except the batch file is explicitly designed to define the execution environment for applications and scripts executed after batch file execution finished.
What does that mean for batch file development?
The following properties of the execution environment should be unmodified after finishing the execution of a batch file in comparison to the property values on starting the batch file:
the list of environment variables and their values;
the status of command extensions;
the status of delayed expansion;
the current directory;
the command prompt;
text and background color;
the code page to use for character encoding;
the number of rows and columns of the console window.
The first four properties of the execution environment are unmodified on using at top of the batch file SETLOCAL and optionally at bottom also ENDLOCAL. An explicit ENDLOCAL at bottom of a batch file is optional because of cmd.exe calls it implicit for each SETLOCAL without an executed matching ENDLOCAL before exiting the processing of a batch file independent on the cause of exiting the batch file processing.
See also: How to pass environment variables as parameters by reference to another batch file?
It explains in full details what happens on each execution of SETLOCAL and ENDLOCAL.
For each successfully executed PUSHD should be executed also a POPD to restore the initial current directory.
The command prompt needs to be restored only on changing it with command PROMPT which most batch files don't do at all.
The usage of CHCP to change the code page should result in using CHCP once again at end of a batch file to restore the original code page. The same should be done on using the command COLOR to change the text color and the background color and command MODE to change the rows and the columns of the console window.
See DosTips forum topic [Info] Saving current codepage, especially the post written by Compo, for an explanation about getting current code page number assigned to an environment variable which is used at end of the batch file to restore the initial code page.
It is a bit difficult to understand why getting the current code page number is done with two FOR loops whereby the second one uses the modifier %~n although the output of chcp.com is definitely not a file name. So let us look on what happens on a German Windows on which the command CHCP outputs the string:
Aktive Codepage: 850.
The dot at end of the output is not wanted, just the code page number like on English Windows on which the output is:
Active code page: 850
See the referenced DosTips topic for other variants depending on the language of Windows.
The output of chcp.com is first assigned completely to the loop variable G with removing leading normal spaces and horizontal tabs if chcp.com would output the code page information with leading spaces/tabs. The second FOR loop processes this list of words with using normal space, comma, semicolon, equal sign and OEM encoded no-break space as word delimiters.
The second FOR loop runs the command SET for German code page information three times with the strings:
Aktive
Codepage:
850.
The usage of the modifier %~n results now three times in accessing the file system by cmd.exe and searching in current directory for a file with the string assigned to the loop variable H as file name. There is most likely no file Aktive. Codepage: with the colon at end is an invalid file name, and a file 850 with trailing dot removed by the Windows file IO API functions is most likely also not found in current directory. However, it does not really matter if there is by chance a file system entry matching one of the three strings or not because of %~n results in using just the string from beginning to the character before the last dot. So the command SET is first executed with Aktive, a second time with Codepage: and finally a third time with 850. So the environment variable CodePage is defined finally with just the number 850.
Description of the main FOR loops processing the video files
The most outer FOR assigns the name of the found file always with full path without surrounding " to the specified loop variable G because of using option /R. For that reason just "%%G" is used instead of "%%~G" wherever the fully qualified file name must be referenced to speed up the processing of the file names.
echo( outputs an empty line, see the DosTips forum topic ECHO. FAILS to give text or blank line - Instead use ECHO/
If an undefined environment variable like Attachments is referenced in an arithmetic expression evaluated by SET, the value 0 is used as explained by the usage help output on running set /? in a command prompt window. For that reason set /A Attachments+=1 can be used to either define the variable with 1 on first execution or increment the value of environment variable Attachments by one on all further executions for the current file.
The final value of environment variable Attachments is evaluated after processing all lines output by mkvmerge. If there are attachments, the file name is assigned to the environment variable FileName with still disabled delayed variable expansion and for that reason ! is interpreted as literal character. The environment variable propeditcmd is created next dynamically according to the number of attachments.
Optimized code for the entire video files processing task
I have installed neither mkvmerge.exe nor mkvpropedit, but I looked also on the referenced full code. Here is a rewritten optimized version of your full code without any comment which I could not really test completely.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "WindowTitle=%~n0"
setlocal EnableDelayedExpansion
for /F "tokens=1,2" %%G in ("!CMDCMDLINE!") do (
if /I "%%~nG" == "cmd" if /I "%%~H" == "/c" (
endlocal
start %SystemRoot%\System32\cmd.exe /D /K %0
if not errorlevel 1 exit /B
setlocal EnableDelayedExpansion
)
)
title !WindowTitle!
endlocal
for /F delims^=^=^ eol^= %%G in ('set ^| %SystemRoot%\System32\findstr.exe /B /I /L /V "ComSpec= PATH= PATHEXT= SystemRoot= TEMP= TMP="') do set "%%G="
if exist "%~dp0mkvmerge.exe" (set "ToolsPath=%~dp0") else if exist mkvmerge.exe (set "ToolsPath=%CD%") else for %%I in (mkvmerge.exe) do set "ToolsPath=%%~dp$PATH:I"
if not defined ToolsPath echo ERROR: Could not find mkvmerge.exe!& exit /B 2
if "%ToolsPath:~-1%" == "\" set "ToolsPath=%ToolsPath:~0,-1%"
if not exist "%ToolsPath%\mkvpropedit.exe" echo ERROR: Could not find mkvpropedit.exe!& exit /B 2
for /F "tokens=*" %%G in ('%SystemRoot%\System32\chcp.com') do for %%H in (%%G) do set /A "CodePage=%%H" 2>nul
%SystemRoot%\System32\chcp.com 65001 >nul 2>&1
del /A /F /Q Errors.txt ExtraTracksList.txt 2>nul
(
set "ToolsPath="
set "CodePage="
for /F "delims=" %%G in ('dir *.mkv /A-D-H /B /S 2^>nul') do (
echo --^> Processing file "%%G" ...
setlocal
set "FullFileName=%%G"
for /F "tokens=1,4 delims=: " %%H in ('^""%ToolsPath%\mkvmerge.exe" -i "%%G" --ui-language en^"') do (
if /I "%%I" == "audio" (
set /A AudioTracks+=1
setlocal EnableDelayedExpansion
if !AudioTracks! == 2 echo !FullFileName!>>ExtraTracksList.txt
endlocal
) else if not defined SkipFile if /I "%%I" == "subtitles" (
echo --^> "%%~nxG" has subtitles
"%ToolsPath%\mkvmerge.exe" -o "%%~dpnG.nosubs%%~xG" -S -M -T -B --no-global-tags --no-chapters --ui-language en "%%G"
if not errorlevel 1 (
echo --^> Deleting old file ...
del /F "%%G"
echo --^> Renaming new file ...
ren "%%~dpnG.nosubs%%~xG" "%%~nxG"
) else (
echo Warnings/errors generated during remuxing, original file not deleted, check Errors.txt
"%ToolsPath%\mkvmerge.exe" -i --ui-language en "%%G">>Errors.txt
del "%%~dpnG.nosubs%%~xG" 2>nul
)
set "SkipFile=1"
) else if /I "%%H" == "Attachment" (
set /A Attachments+=1
) else if /I "%%H" == "Global" (
set "TagsAll=--tags all:"
) else if /I "%%H" == "Chapters" (
set "Chapters=--chapters """
)
)
if not defined SkipFile (
set "OnlyFileName=%%~nxG"
setlocal EnableDelayedExpansion
if defined Attachments (
set "PropEditOptions= --delete-attachment 1"
for /L %%H in (2,1,!Attachments!) do set "PropEditOptions=!PropEditOptions! --delete-attachment %%H"
)
if defined TagsAll set "PropEditOptions=!PropEditOptions! !TagsAll!"
if defined Chapters set "PropEditOptions=!PropEditOptions! !Chapters!"
if defined PropEditOptions (
echo --^> "!OnlyFileName!" has extras ...
"%ToolsPath%\mkvpropedit.exe" "!FullFileName!"!PropEditOptions!
)
endlocal
)
echo(
echo ##########
echo(
endlocal
)
for /F "delims=" %%G in ('dir *.avi *.mp4 *.mov /A-D-H /B /S 2^>nul') do (
echo Processing file "%%G" ...
"%ToolsPath%\mkvmerge.exe" -o "%%~dpnG.mkv" -S -M -T -B --no-global-tags --no-chapters --ui-language en "%%G"
if not errorlevel 1 (
echo --^> Deleting old file ...
del /F "%%G"
) else (
echo --^> Warnings/errors generated during remuxing, original file not deleted.
"%ToolsPath%\mkvmerge.exe" -i --ui-language en "%%G">>Errors.txt
del "%%~dpnG.mkv" 2>nul
)
echo(
echo ##########
echo(
)
if exist Errors.txt for %%G in (Errors.txt) do if %%~zG == 0 del Errors.txt 2>nul
%SystemRoot%\System32\chcp.com %CodePage% >nul
)
endlocal
Removal of not needed environment variables in local environment
The batch file has to process perhaps hundreds or even thousands of files using multiple environment variables.
There is at least once per MKV file used SETLOCAL and ENDLOCAL creating a copy of current environment variables list which is discarded after finishing processing of the current MKV file.
There are also other programs executed for each video files on which the Windows kernel library function CreateProcess creates also a copy of the current list of environment variables of current process.
For that reason it is helpful to use a local environment variables list which contains only the environment variables really needed during processing of the video files.
The first FOR after setting the window title runs in background one more cmd.exe as follows:
C:\Windows\System32\cmd.exe /c set | C:\Windows\System32\findstr.exe /B /I /L /V "ComSpec= PATH= PATHEXT= SystemRoot= TEMP= TMP="
There is output by set of started cmd.exe in background the same list of environment variables with their values as the command process currently uses which processes the batch file. The lines are passed to findstr which searches case-insensitive (/I) and literally (/L) for the space separated strings at beginning of each line (/B) and outputs the inverted result (/V) which means all lines NOT beginning with one of the space separated strings. So there are output all the environment variables separated with a = from their values, except those searched for and found by findstr.
The captured lines are processed by FOR with using the equal sign as string delimiter and no character as end of line character to process even an environment variable of which name starts with a semicolon and assigns to the loop variable G just the variable name which is used to remove the variable from the current environment variables list.
So there are only remaining the environment variables ComSpec, PATH, PATHEXT, SystemRoot, TEMP and TMP.
Use fully qualified file names to avoid unnecessary file system accesses
Most people use in batch files just the file names of executables without file extension and without file path which forces cmd.exe to search in current directory and next in all directories as specified in environment variable PATH for the file with a file extension as specified in environment variable PATHEXT. That results in thousands of file system accesses on processing hundreds of files in a loop calling executables on each file.
All these file system accesses can be avoided by specifying each executable with its fully qualified file name in the batch file. That does not mean that a batch file must contain already the fully qualified file name for each executable as the code above demonstrates because of the full file names of the executables can be determined also once at beginning of the batch file.
The batch file first checks if mkvmerge.exe is in the directory of the batch file and defines the environment variable ToolsPath with the full batch file path if that file check is positive. Otherwise there is searched in the current directory for the executable mkvmerge.exe and the current directory path is assigned to ToolsPath if there is a file system entry (hopefully a file and not a directory) with the name mkvmerge.exe. Last there is searched for mkvmerge.exe in the directories of environment variable PATH and if found this directory path is assigned to ToolsPath.
The batch file outputs an error message, restores the initial environment and exits on executable mkvmerge.exe or the other one mkvpropedit.exe could not be found at all.
%~dp0 and %%~dp$PATH:I expand to a path string always ending with a backslash. %CD% expands to a path string not ending with a backslash, except the current directory is the root directory of a drive. For that reason an IF condition with a string comparison is used to check if the path string assigned to ToolsPath ends with a backslash in which case the environment variable is redefined with this backslash removed. The backslash is added in the code below on referencing the path string of ToolsPath.
Determination of current code page using a different method
This time the first solution developed by Compo is used to determine the number of the current code page. It is similar to the other solution as using the same two FOR loops, but the command SET executed by the second FOR loop evaluates now an arithmetic expression to get on last iteration the code page number without the dot assigned to the environment variable CodePage.
Let us look again what happens on processing the string: Aktive Codepage: 850.
There is first executed set /A "CodePage=Aktive" which results in environment variable CodePage is defined with value 0 because of Aktive is interpreted as environment variable name and there is no such environment variable. Next is executed set /A "CodePage=Codepage:" with the same interpretation and the same result 0. And last is executed set /A "CodePage=850." which results in the error message Missing operator. to handle STDERR redirected to the device NUL to suppress it. However, the value assigned to the environment variable CodePage is 850 as wanted.
The advantage of this solution is the usage of %%H inside the arithmetic expression which does not result in any file system access. So this solution is in general better in my opinion.
How to avoid batch file accesses during processing the video files?
I recommend reading Why is a GOTO loop much slower than a FOR loop and depends additionally on power supply?
Conclusion: It is a good idea to put the entire code required to process hundreds or thousands of files into one command block which the Windows command processors reads and parses just once.
The problem is in most cases how to handle variables of which values changes within the command block without using all the time delayed expansion as that affects processing of strings like file names. That is in most cases not easy, but it is often possible as it can be seen on the code above.
The environment variables ToolsPath and CodePage can be undefined immediately at beginning of the main code block because of the command processor replaced already all %ToolsPath% and %CodePage% by the appropriate path and code page number strings before executing the first command set "ToolsPath=". So the current environment variables list on execution of the first main FOR loop contains just the five environment variables found by findstr.
The Windows command processor does not access anymore the batch file until having finished processing all video files and restored the original code page.
Other special information about the code in second batch file
The two text files with information collected during processing of the video files are always deleted first using the command DEL if the file system does not prevent the deletion of the files.
There is used twice for /F instead of for /R as main FOR loops to get first all file names of video files to process with full path loaded into memory of the Windows command processor and then process the video files instead of iterating over the current file system entries as done by for /R. This makes a big difference for the loop processing *.mkv files, especially on video files being stored on a FAT32 or exFAT formatted drive on which the file allocation table does not only change on processing an MKV file as also on NTFS formatted drives, but are not updated in file allocation table in a local alphabetic sort as on an NTFS formatted drive. The usage of for /R could result on a FAT32 or exFAT formatted drive in either processing an MKV file more than once or skipping unexpected one or more MKV files due to the file allocation table changes caused by the execution of mkvmerge or mkvpropedit on an MKV file.
The commands SETLOCAL and ENDLOCAL are used to quickly restore always the minimal environment variables list defined outside of the main FOR loops for each MKV file which results in discarding always all the changes made on the environment variables list on processing an MKV file.
The execution of mkvmerge.exe with its full path with option -i and the full name of current MKV file by one more cmd.exe started with /c and the specified command line is a bit tricky on taking into account that %ToolsPath% and %%G could contain also characters like & to be interpreted as literal characters by cmd.exe processing the batch file and also by cmd.exe started in background.
It is necessary to enclose the entire command line to execute by cmd.exe in background in double quotes to be correct processed by this cmd.exe instance. But the cmd.exe instance processing the batch file must interpret these two " as literal characters and not as beginning or end of an argument string. Otherwise "" at beginning would be interpreted by cmd.exe processing the batch file as the beginning and the end of an empty argument string. Therefore the tools path string would be not anymore enclosed in double quotes for cmd.exe processing the batch file which of course is problematic on containing & or ' or ).
For that reason the two double quotes to enclose the entire command line in " are specified in the batch file with the caret character ^ to be escaped which results in cmd.exe processing the batch file is interpreting these two double quotes as literal characters and not as beginning/end of an argument string.
The result is that "%ToolsPath%\mkvmerge.exe" and "%%G" are interpreted by both cmd.exe as double quoted argument strings and therefore can contain all characters interpreted as literal characters which would otherwise be interpreted with a special meaning.
The information about audio tracks are processed always independent in which order mkvmerge.exe outputs the information data about the current MKV file. But all other information are not further processed once the environment variable SkipFile is defined because of the current MKV file has subtitles.
The file Errors.txt is deleted on being created, but has finally a size of 0 bytes.
Usage help for the used Windows commands
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
chcp /?
cmd /?
dir /?
del /?
echo /?
endlocal /?
exit /?
findstr /?
for /?
if /?
rem /?
set /?
setlocal /?
start /?
title /?
See also Issue 7: Usage of letters ADFNPSTXZadfnpstxz as loop variable and the other chapters about general issues made by beginners in batch file coding.

Why does my batch script not interpret a string correctly

In my second if statement, I want to filter out "tool" or "tool.bat" from the final list of filenames. However, the final list of filenames includes "tool" and total_bags is being incremented. I was wondering what I did incorrectly that's causing the program to not catch this case.
set /A total_bags=0
set target=%~1
if "%target%"=="" set target=%cd%
set LF=^
rem Previous two lines deliberately left blank for LF to work.
for /f "tokens=1 delims=. " %%i in ('dir /b /s /a:-d "%target%"') do (
set current_file=%%~ni
echo !unique_files! | find "!current_file!:" > nul
if NOT !ERRORLEVEL! == 0 (
if NOT !current_file! == "tool.bat" (
set /A total_bags=total_bags+1
set unique_files=!unique_files!!current_file!:
)
)
)
echo %unique_files::=!LF!%
echo %total_bags%
endlocal
The condition if NOT "%current_file%" == "tool.bat" as initially used does not work because of %current_file% is replaced already by current string of the environment variable current_file respectively an empty string on Windows command processor is processing the entire command block starting with ( and ending with matching ) before executing command FOR. That can be seen on debugging the batch file. See also Variables are not behaving as expected for a very good and short example explaining how the Windows command interpreter (CMD.EXE) parses scripts.
It is in general not advisable to assign the string already assigned to a loop variable to an environment variable which is not further modified inside a FOR loop. It would be better to use %%~ni everywhere in your code on which the current file name needs to be referenced.
The usage of delayed expansion requires enabling it with setlocal EnableDelayedExpansion (or with setlocal EnableExtensions EnableDelayedExpansion to enable explicitly also the command extensions enabled by default) as it is not enabled by default in comparison to the command extensions. Then the Windows command processor parses each command line a second time and expands !current_file! on execution of command IF.
But even if NOT !current_file! == "tool.bat" evaluates always to true for the batch file with name tool.bat because of set current_file=%%~ni results in assigned to the environment variable current_file only the string tool (file name without file extension) and the left string is not enclosed in double quotes while the right string is always enclosed in double quotes. The command IF does not remove the double quotes from right string before comparing the two strings.
The batch file in question misses also set unique_files= above the FOR loop to undefine explicitly the environment variable unique_files in case of being already defined by chance on starting the batch file, for example from a previous execution within a command prompt window.
Another problem with the batch file in question is that maximum string length of variable name + equal sign + string assigned to the environment variable is 8191 characters which is a problem on several thousands of file names are concatenated to a long string assigned to one environment variable like unique_files.
I suggest to use this batch file with comments explaining it.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem Delete all environment variables of which name starts very unusual
rem with a question mark existing already by chance (with exception of
rem those environment variables with multiple question marks in name).
for /F "delims=?" %%I in ('set ? 2^>nul') do set "?%%I?="
rem Search with the string passed as first argument or simply within current
rem directory recursively for all files and define for each file name an
rem environment variable with a question mark at beginning and one more at
rem end of the variable name. A file name cannot contain a question mark.
rem The value assigned to the environment variable does not matter. As it
rem is not possible to define multiple environment variables with same name
rem and environment variable names are case-insensitive, there is just one
rem environment variable defined on multiple files have same file name.
rem The batch file itself is ignored because of the IF condition.
for /F "delims=" %%I in ('dir "%~1" /A-D /B /S 2^>nul') do if not "%%I" == "%~f0" set "?%%~nI?=1"
rem Initialize the file counting environment variable.
set "FileCount=0"
rem Output all file names which are the environment variable names sorted
rem alphabetically with the question marks removed and additionally count
rem the number of file names output by this loop.
for /F "eol=| delims=?" %%I in ('set ? 2^>nul') do set /A "FileCount+=1" & echo %%I
rem Output finally the number of unique file names excluding file extensions.
echo %FileCount%
rem Restore initial execution environment which results also in the
rem deletion of all environment variables defined during batch execution.
endlocal
It does not use delayed expansion and for that reason works also for file names containing one or more ! in file name which would be processed wrong on enabling delayed expansion on line set current_file=%%~ni because of the exclamation mark(s) in file name would be interpreted as begin/end of a delayed expanded environment variable reference.
There is defined an environment variable for each unique file name. The number of environment variables is limited only by the total available memory for environment variables which is 64 MiB. That should be enough even for several thousands of unique file names in the directory tree.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /? ... explains %~f0 which references full name of argument 0 which is the full qualified file name of the currently processed batch file and %~1 referencing first argument with perhaps existing surrounding " removed from argument string.
dir /?
echo /?
endlocal /?
for /?
if /?
rem /?
set /?
setlocal /?
Read the Microsoft documentation about Using command redirection operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on the FOR command lines to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir or set command line with using a separate command process started in background with %ComSpec% /c and the command line within ' appended as additional arguments.

Batch File - Read specific line, and save a specific string in that line as a variable

Is there any way to get for /f loop (or anything else) to read a specific line?
Here is the code I have so far, it reads first word of every line.
#echo off
set file=readtest.txt
for /f "tokens=1 delims= " %%A in (%file%) do (echo %%A)
pause
If someone can point me in the right direction, it'd be much appreciated.
Thanks
Additional Information: I want to make a batch file which will rename a TXT file to a string within that TXT file, located at a specific location. I have figured out how to rename files, all I need to learn to do is to retrieve a string (located at a specific location) with in the file which will go into the name of that TXT file.
Since you haven't fully defined what you mean by "a specific location", I'll make some (reasonable, in my opinion) assumptions, though the method I present is equally valid no matter what your definition turns out to be.
You can get arbitrary lines and arbitrary words on that line by using a line counter variable in conjunction with tokens.
Let's assume your text file name can be found as the second argument on the fourth line of the infile.txt file. You can get that with something like:
#setlocal enableextensions enabledelayedexpansion
#echo off
set /a "line = 0"
for /f "tokens=2 delims= " %%a in (infile.txt) do (
set /a "line = line + 1"
if !line!==4 set thing=%%a
)
endlocal & set thing=%thing%
echo %thing%
This actually uses a few "tricks" which warrant further explanation:
the line counter to ensure you only grab what you want from a specific line, though you could change the test !line!==4 into anything you need such as a line beginning with #, the fifth line containing the string xyzzy and so on.
the use of setlocal/endlocal to effectively give you a scope from which variables cannot leak. This is good programming practice even for a language often not normally associated with such things :-)
the use of endlocal & set to bypass that scope so that thing is the only thing that does actually leak (as it should).
the use of delayed expansion and !..! variables to ensure they're correct within the for loop. Without this, the %..% will always be expand to the value they were set to when the for loop started.
Those last two bullet points are actually related. %..% variables are expanded when the command is read rather than when it is executed.
For a for loop, the command is the entire thing from the for to the final ). That means, if you use %line% within the loop, that will be evaluated before the loop starts running, which will result in it always being 0 (the variable itself may change but the expansion of it has already happened). However, !line! will be evaluated each time it is encountered within the loop so will have the correct value.
Similarly, while endlocal would normally clear out all variables created after the setlocal, the command:
endlocal & set thing=%thing%
is a single command in the context of expansion. The %thing% is expanded before endlocal is run, meaning it effectively becomes:
endlocal & set thing=whatever_thing_was_set_to_before_endlocal
That's why the use of setlocal and endlocal & set is a very useful way to limit variables "escaping" from a scope. And, yes, you can chain multiple & set stanzas to allow more variables to escape the scope.

I need to match or replace an asterisk * in a batch environmental variable using only native Windows commands. Is this possible?

I'm trying to remove an asterisk from an environmental variable string, but can't seem to do it.
I'm creating an m3u file based around search strings, so for instance I if I want to make an m3u file containing every song with the word love in it, I would enter:
m3u *Love*
And m3u.bat would create the file:
xLovex.m3u
But the regular method of replacing characters does not work with an asterisk. (Though I don't have that problem with the question mark.)
set nam=%nam:*=x%.m3u
Instead creates the filename
x.m3u
The easy answer is no.
The problem that you're encountering stems from the fact that the asterisk * is a special character when used with the SET search and replace method. It matches multiple characters in a limited, but still useful, way. You can learn about that here.
The hard answer is Yes!
I will provide you with two solutions. One an incomplete solution but elegent,
the other complete and inelegent.
Both methods will search for * and replace it with an x.
Both methods will both search and modify the following string:
*love*
The first method that comes to mind is using a 'FOR /L' statement, and requires that you know how many characters long the environmental variable is.
::Major Edit::
I thought I knew the various maximum size strings of environmental variables, but dbenham has taken me to school, shown me a kick-in-the-behind length function, and in the mean time completely reversed my opinions of the two solutions I'm presenting.
Other than for the Windows 95/98/ME limitation of a 256 Character maximum environmental variable size. It seems that all versions of Windows using CMD.EXE have a limitation of 8,192 characters, well below what the documentation suggests.
Both versions require delayed environmental variable expansion, but for two different reasons. One because I'm operating inside a FOR statement. The other because you cannot put a % pair inside another % pair because the command processor matches the second % that it encounters to the first one it encounters, but we need to use a variable inside another variable expression. (You'll see.)
This solution uses the strLen function (in line 3) from DosTips.com that can be found Here. Just slap it into a file called strLen.bat and be amazed at it's speed!
Solution 1: (FOR /L Solution) :: Preferred Solution ::
setlocal ENABLEDELAYEDEXPANSION
set nam=*love*
rem calling strLen
call :strLen nam len
for /l %%x in (0,1,%len%) do if not "!nam:~%%x,1!"=="" if "!nam:~%%x,1!"=="*" (
set /a plusone=%%x+1
for /l %%y in (!plusone!, 1, !plusone!) do (
set nam=!nam:~0,%%x!x!nam:~%%y!
)
)
echo %nam%
ENDLOCAL
I think this is a quick and elegant solution It could be sped up by adding the contents of strLen.bat to the routine, but I wanted no confusion as to the author.
If you, for some reason, do not wish to use strLen, then the next quickest method would probably use a GOTO loop.
Solution 2: (Goto Solution)
setlocal ENABLEDELAYEDEXPANSION
set nam=*love*
set num=0
:loop
set /a plusone=%num%+1
if "!nam:~%num%,1!"=="*" set nam=!nam:~0,%num%!x!nam:~%plusone%!
set /a num=%num%+1
if not "!nam:~%num%,1!"=="" goto :loop
echo %nam%
EndLocal
Special thanks to dbenham for pointing out the strLen function. It works faster than any batch based function has a right to!
Although there were already some very good and robust ways explained here, I'd still like to add another option for the sake of completion.
It's not as good as the other options but I personally use it in some cases where I'd like to keep the code clean and where I know that it will suffice:
The way it works is by using for /f's delims to cut the string into two parts, which are then put back together, getting rid of the * in the process:
for /f "tokens=1,* delims=*" %%a in ("a*b") do (set string=%%a%%b)
>>> string=ab
Obviously, the downside to this is that it can only be used to remove one *.
To remove more, we can either just use more tokens...
for /f "tokens=1-3,* delims=*" %%a in ("a*b*c*d") do (set string=%%a%%b%%c%%d)
>>> string=abcd
... or we can put the first line in a for /l-loop:
setlocal enableDelayedExpansion
set string=a*b*c*d
for /l %%a in (1, 1, 3) do (
for /f "tokens=1,* delims=*" %%b in ("!string!") do (set string=%%b%%c)
)
>>> string=abcd
Another thing to note is that you can define more than one character in delims, and they will all be removed at once:
for /f "tokens=1,* delims=+-*/" %%a in ("a*-/+b") do (set string=%%a%%b)
>>> string=ab
Another solution to the stated problem is to use a PowerShell replace command within your batch script.
set var=*Love*
echo %var%>var.txt | powershell -command "((get-content var.txt) -replace '[\x2A]','x') -replace '.{1}$' | set-content var.txt"
set /p var=<var.txt
set var=%var%.m3u
echo %var%
In the above code, the second line
writes your string into a text file
calls a PowerShell command to get the contents of that file
replaces the * character with null
overwrites the text file with the new value
Once that is done, you read the value back into your variable.
To further explain the replace command, the first single quotes is what you are searching for. We are using square brackets to identify the * character as a hex character (\x2A is the hex value for *). After the comma, the second set of single quotes contains no value so that the searched object is removed. To prevent a space between xLovex and the .m3u, we have to use -replace '.{1}$' before writing the result to the text file.
Once you are done with the text file, enter a line to delete it.
if exist var.txt del var.txt
Here is an approach that does not walk through all characters of a string, but it uses a for /F loop to split the string at every occurrence of a (sequence of a) certain character. The actual functionality is packed into a sub-routine for easy reuse, so the main section of the following script just contains some code to test:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
::This is the main routine of the script holding code for test and demonstration:
rem // Definition of some sample text to test (note that `%%` becomes one literal `%`):
set "DATA=some text,"^&"&;0'#%%~#`$:wild**card*?.re<dir>=|+([{parens}])-^/equal==to=!_"
echo/
call :REPL_CHAR TEXT DATA "*" "?"
setlocal EnableDelayedExpansion
echo(In: !DATA!
echo(Out:!TEXT!
echo/
echo(In: !TEXT!
call :REPL_CHAR TEXT TEXT "=" "/"
echo(Out:!TEXT!
endlocal
endlocal
exit /B
:REPL_CHAR
::This function replaces in a string every occurrence of a sequence of a certain character
::by another character or a string. It even correctly handles the characters `*` and `=`.
:: USAGE:
:: call :REPL_CHAR ref_output_string ref_input_string val_search_char val_replace_char
:: PARAMETERS:
:: ref_output_string reference to (name of) variable to receive the resulting string;
:: ref_input_string reference to variable that holds the original string; if empty
:: (`""`), the variable referenced by `ref_output_string` is used;
:: val_search_char single character that is to be replaced;
:: val_replace_char character or string to replace every sequence of `val_search_char`
:: with; this may even be empty;
rem // Localise environment and detect whether delayed expansion is enabled (needed later):
setlocal & set "$NDX=!"
setlocal DisableDelayedExpansion
rem // Fetch arguments and verify them:
set "#RET=%~1" & if not defined #RET endlocal & endlocal & exit /B 2
set "#STR=%~2" & if not defined #STR set "#STR=%#RET%"
set "CHR=%~3"
if not defined CHR endlocal & endlocal & exit /B 1
set "RPL=%~4"
setlocal EnableDelayedExpansion
rem // Initialise several auxiliary variables:
set "TST=!%#STR%!" & set "CHR=!CHR:~,1!" & set "INS="
if "!CHR!"=="_" (set "BUF=#" & set "WRK=!TST!#") else (set "BUF=_" & set "WRK=!TST!_")
:REPL_CHAR_LOOP
rem // Check whether the end of the string has been reached:
if not defined TST set "BUF=!BUF:~1,-1!" & goto :REPL_CHAR_NEXT
rem // Split the string at the next sequence of search characters:
for /F tokens^=1*^ delims^=^%CHR%^ eol^=^%CHR% %%S in ("!BUF!!INS!!WRK!") do (
rem // Store the portions before and after the character sequence:
endlocal & set "BUF=%%S" & set "TST=%%T" & set "WRK=%%T" & setlocal EnableDelayedExpansion
)
rem // Loop back and find the next character sequence:
set "INS=!RPL!" & goto :REPL_CHAR_LOOP
:REPL_CHAR_NEXT
rem // Return the resulting string with all special characters properly handled:
if not defined $NDX if defined BUF set "BUF=!BUF:"=""!^"
if not defined $NDX if defined BUF set "BUF=!BUF:^=^^^^!"
if not defined $NDX if defined BUF set "BUF=%BUF:!=^^^!%" !
if not defined $NDX if defined BUF set "BUF=!BUF:""="!^"
for /F "delims=" %%S in (^""!BUF!"^") do endlocal & endlocal & endlocal & set "%#RET%=%%~S" !
exit /B
The input and output data of this script (let us call it repl_char_demo.bat) are:
>>> repl_char_demo.bat
In: some text,"&"&;0'#%~#`$:wild**card*?.re<dir>=|+([{parens}])-^/equal==to=!_
Out:some text,"&"&;0'#%~#`$:wild?card??.re<dir>=|+([{parens}])-^/equal==to=!_
In: some text,"&"&;0'#%~#`$:wild?card??.re<dir>=|+([{parens}])-^/equal==to=!_
Out:some text,"&"&;0'#%~#`$:wild?card??.re<dir>/|+([{parens}])-^/equal/to/!_
This is a script that uses for /L loops to walk through all characters of the string, to check each character against a predefined one and replaces it as specified. This method replaces every single matching character rather than sequences. Again the functionality is put into a sub-routine (the main section is dismissed this time):
:REPL_CHAR
::This function replaces in a string every occurrence of one certain character by another
::character or a string. It even correctly handles the characters `*` and `=`, as well as
::sequences of search characters so that every single one becomes replaced.
:: USAGE:
:: call :REPL_CHAR ref_output_string ref_input_string val_search_char val_replace_char
:: PARAMETERS:
:: ref_output_string reference to (name of) variable to receive the resulting string;
:: ref_input_string reference to variable that holds the original string; if empty
:: (`""`), the variable referenced by `ref_output_string` is used;
:: val_search_char single character that is to be replaced;
:: val_replace_char character or string to replace every single `val_search_char`
:: with; this may even be empty;
rem // Localise environment and detect whether delayed expansion is enabled (needed later):
setlocal & set "$NDX=!"
setlocal DisableDelayedExpansion
rem // Fetch arguments and verify them:
set "#RET=%~1" & if not defined #RET endlocal & endlocal & exit /B 2
set "#STR=%~2" & if not defined #STR set "#STR=%#RET%"
set "CHR=%~3"
if not defined CHR endlocal & endlocal & exit /B 1
set "RPL=%~4"
setlocal EnableDelayedExpansion
rem // Initialise several auxiliary variables:
set "WRK=!%#STR%!" & set "CHR=!CHR:~,1!" & set "BUF="
rem // Loop through all characters and check for match:
if defined WRK for /L %%J in (0,1,63) do for /L %%I in (0,1,127) do (
set /A "POS=%%J*64+%%I" & for %%P in (!POS!) do (
set "TST=!WRK:~%%P,1!" & if not defined TST goto :REPL_CHAR_QUIT
rem // Store character or replacement depending on whether there is a match:
if "!TST!"=="!CHR!" (set "BUF=!BUF!!RPL!") else (set "BUF=!BUF!!TST!")
)
)
:REPL_CHAR_QUIT
rem // Return the resulting string with all special characters properly handled:
if not defined $NDX if defined BUF set "BUF=!BUF:"=""!^"
if not defined $NDX if defined BUF set "BUF=!BUF:^=^^^^!"
if not defined $NDX if defined BUF set "BUF=%BUF:!=^^^!%" !
if not defined $NDX if defined BUF set "BUF=!BUF:""="!^"
for /F "delims=" %%S in (^""!BUF!"^") do endlocal & endlocal & endlocal & set "%#RET%=%%~S" !
exit /B
There are actually two nested for /L loops rather than a single one, both of which become broken as soon as the end of the string is reached, using the goto command. Breaking a for /L loop means that it completes iterating in the background although its body is no longer executed. Therefore, using a single loop takes much more time to finish after being broken rather than two nested ones.
The input and output data of this script (with the same main section as above) are:
>>> repl_char_demo.bat
In: some text,"&"&;0'#%~#`$:wild**card*?.re<dir>=|+([{parens}])-^/equal==to=!_
Out:some text,"&"&;0'#%~#`$:wild??card??.re<dir>=|+([{parens}])-^/equal==to=!_
In: some text,"&"&;0'#%~#`$:wild??card??.re<dir>=|+([{parens}])-^/equal==to=!_
Out:some text,"&"&;0'#%~#`$:wild??card??.re<dir>/|+([{parens}])-^/equal//to/!_
See this answer, and with set-ast.bat you'll want to put set-ast nam "x" in your file where needed.
set-ast takes the parameters <variable-to-modify> <string-to-replace-asterisks-with>

batch script variable unset in for loop has no effect

Below is my script. I am trying to look into folders one level below and pick out only those folders, hence the ~-9 which extracts the last 9 chars from the path. But the set var= does not unset the variable because the output comes back with the same folder name repeated # times. Also batch doesn't allow me to do this extract trick directly on %%i, hence the need for the local variable.
How do I clear this variable so that it takes the new value in the next iteration?
#echo off
for /d %%i in (%1\*) do (
set var=%%i
echo %var:~-9%
set "var="
)
http://judago.webs.com/variablecatches.htm has an explanation for my problem. The magic lines were setlocal enabledelayedexpansion and calling var as echo !var:~-9!. ! vs % ...wow! cmd still amazes me.
You found the source of your problem, as well as the solution - delayed expansion.
But using FOR while delayed expansion is enabled can cause problems if any of the filenames contain the ! character. The expansion of the for variable %%i will be corrupted if the value contains ! and delayed expansion is enabled. This is not a frequent problem, but it happens.
The solution is to toggle delayed expansion on and off within the loop
#echo off
setlocal disableDelayedExpansion
for /d %%i in (%1\*) do (
set var=%%i
setlocal enableDelayedExpansion
echo !var:~-9!
endlocal
)
I'm also wondering what you mean by "I am trying to look into folders one level below and pick out only those folders, hence the ~-9 which extracts the last 9 chars from the path". I suspect your are trying to get the name of the child folder, without the leading path information. If that is so, then using the substring operation is not a good solution because the length of folder names varies.
There is a very simple method to get the name of the folder without the leading path info:
for /d %%i in (%1\*) do echo %%~nxi

Resources