I am trying to rename the episodes in a directory in an incremental way, but there are exclamation marks in some of the episodes. It will skip those files. I tried doing delayed expansion, but it didn't work.
#echo off
setlocal DisableDelayedExpansion
set /a num=0
for %%a in (*.mkv) do (
set filename=%%a
setlocal EnableDelayedExpansion
ren "!filename!" "Soul Eater Episode 0!num!.mkv"
set /a num=!num!+1
)
pause
endlocal
Try this:
#echo off
setlocal EnableDelayedExpansion
set /a num=0
for %%a in (*.mkv) do (
set filename=%%a
ren "!filename!" "Soul Eater Episode 0!num!.mkv"
set /a num=!num!+1
)
endlocal
pause
The following batch file code could be used to rename the files containing one or more ! in file name.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "num=0"
for /F "eol=| delims=" %%I in ('dir *.mkv /A-D /B /ON 2^>nul ^| %SystemRoot%\System32\findstr.exe /B /I /L /V /C:"Soul Eater Episode"') do (
set "filename=%%I"
setlocal EnableDelayedExpansion
ren "!filename!" "Soul Eater Episode 0!num!.mkv"
endlocal
set /A num+=1
)
pause
endlocal
There is no need to use an arithmetic expression to define the environment variable num with the value 0.
It is very advisable on running renames on a list of file names in a directory using a wildcard pattern like *.mkv to get first the list of file names loaded into memory of Windows command processor and then rename one file after the other as done by this code using a for /F loop. Otherwise the result of the file renames is unpredictable as depending on file system (NTFS, FAT32, exFAT) and current names of the files matched by the wildcard pattern.
The additional FINDSTR is used to filter out all file names beginning already case-insensitive with the string Soul Eater Episode although the batch file would most likely fail to rename some files if there are already files with a file name matched by Soul Eater Episode 0*.mkv in the current directory on execution of the batch file.
Read the Microsoft documentation about Using command redirection operators for an explanation of 2>nul and |. The redirection operators > and | must be escaped with caret character ^ on FOR command line to be interpreted as literal characters when Windows command interpreter processes this command line before executing command FOR which executes the embedded command line with dir and findstr with using a separate command process started in background with %ComSpec% /c and the specified command line appended as additional arguments.
The file name is first assigned as output by DIR filtered by FINDSTR to the environment variable filename with delayed expansion disabled as otherwise the double processing of this command line on enabled delayed expansion would result in interpreting ! in file name assigned to loop variable I as beginning/end of a delayed expanded environment variable reference.
Then delayed expansion is enabled to be able to do the rename with referencing the environment variable num using delayed expansion and of course also the file name assigned to environment variable filename.
Next delayed expansion is disabled again before an arithmetic expression is used using the preferred syntax to increment the value of an environment variable by one which always works independent on disabled or enabled delayed expansion.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
dir /?
echo /?
endlocal /?
findstr /?
for /?
pause /?
ren /?
set /?
setlocal /?
Please read this answer with details on what happens in background on every execution of SETLOCAL and ENDLOCAL.
Related
In a complex batch file I want to read in files with paths, among other things, to read them into a variable one after the other separated by spaces.
This works with the following code so far quite well - but only if the path does not contain an exclamation mark.
Even using the setlocal command (enabledelayedexpansion / disabledelayedexpansion) I did not succeed in processing exclamation marks.
Does anyone here have a clever idea to the problem?
The following example batch creates a text file in the current directory and then reads it in a for /F loop.
At the end all three paths from the text file should be in the variable %Output%. But with the exclamation mark.
#echo off
setlocal enabledelayedexpansion
echo This is an example^^! > "textfile.txt"
echo This is a second example^^! >> "textfile.txt"
echo And this line have an ^^! exclamation mark in the middle >> "textfile.txt"
for /F "usebackq tokens=* delims=" %%a in (textfile.txt) do (
set "Record=%%a"
set "Output=!Output!!Record! - "
)
)
echo %Output%
echo !Output!
endlocal
The Output is like this:
This is an example - This is a second example - And this line have an exclamation mark in the middle
But should be like this:
This is an example! - This is a second example! - And this line have an ! exclamation mark in the middle
It is advisable not using delayed variable expansion on processing files and directories, lines in a text file, strings not defined by the batch file itself, or output captured from the execution of a program or a command line. If it is for some reasons necessary to make use of delayed variable expansion inside a FOR loop, there should be first assigned the file/directory name, the line, or the string to process to an environment variable while delayed expansion is disabled and then enable delayed expansion temporary inside the FOR loop.
Here is a batch file demo which can be simply run from within a command prompt window or by double clicking on the batch file. It creates several files for demonstration in the directory for temporary files, but deletes them all before exiting.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
echo This is an example!> "%TEMP%\TextFile.tmp"
echo This is a second example!>> "%TEMP%\TextFile.tmp"
echo And this line has an exclamation mark ! in the middle.>> "%TEMP%\TextFile.tmp"
set "Output="
(for /F usebackq^ delims^=^ eol^= %%I in ("%TEMP%\TextFile.tmp") do set "Line=%%I" & call :ConcatenateLines) & goto ContinueDemo
:ConcatenateLines
set "Output=%Output% - %Line%" & goto :EOF
:ContinueDemo
cls
echo/
echo All lines concatenated are:
echo/
echo %Output:~3%
set "Output="
del "%TEMP%\TextFile.tmp"
echo File with name ".Linux hidden file!">"%TEMP%\.Linux hidden file!"
echo File with name "A simple test!">"%TEMP%\A simple test!"
echo File with name " 100%% Development & 'Test' (!).tmp">"%TEMP%\ 100%% Development & 'Test(!)'.tmp"
echo/
echo Files with ! are:
echo/
for /F "eol=| tokens=* delims=" %%I in ('dir "%TEMP%\*!*" /A-D /B /ON 2^>nul') do (
set "NameFile=%%I"
set "FileName=%%~nI"
set "FileExtension=%%~xI"
set "FullName=%TEMP%\%%I"
setlocal EnableDelayedExpansion
if defined FileName (
if defined FileExtension (
echo File with ext. !FileExtension:~1!: !NameFile!
) else (
echo Extensionless file: !NameFile!
)
) else echo Extensionless file: !NameFile!
del "!FullName!"
endlocal
)
endlocal
echo/
#setlocal EnableExtensions EnableDelayedExpansion & for /F "tokens=1,2" %%G in ("!CMDCMDLINE!") do #endlocal & if /I "%%~nG" == "cmd" if /I "%%~H" == "/c" set /P "=Press any key to exit the demo . . . "<nul & pause >nul
The output of this batch file is:
All lines concatenated are:
This is an example! - This is a second example! - And this line has an exclamation mark ! in the middle.
Files with ! are:
File with ext. tmp: 100% Development & 'Test(!)'.tmp
Extensionless file: .Linux hidden file!
Extensionless file: A simple test!
The text file example with concatenating lines makes use of a subroutine called from within the FOR loop processing the lines in the text file. The syntax used here is for maximum performance by getting the subroutine as near as possible to the FOR command line. That is important if the FOR loop has to process hundreds or even thousands of items.
The example processing file names enables and disables delayed expansion inside the FOR loop after having assigned all parts of the currently processed file to environment variables. It could be useful to reduce the list of environment variables before processing thousands of files for a better performance on using this method.
Another method is shown in Magoo´s answer using the command CALL to get a command line with referenced environment variables (re)defined inside the loop parsed a second time. I used that method also in the past quite often, but don't that anymore as it is not fail-safe and not efficient. call set results in searching by cmd.exe in current directory and next in all directories of environment variable PATH for a file with name set and a file extension of environment variable PATHEXT. So it results in lots of file system accesses in the background on each iteration of the FOR loop and if there is by chance a file set.exe, set.bat, set.cmd, etc. found by cmd.exe somewhere, the batch file does not work anymore as expected because of running the executable or calling the batch file instead of the (re)definition of the environment variable.
The following answers written by me could be also helpful:
How to read and print contents of text file line by line?
It explains in full details how to process all lines of a text file.
How to pass environment variables as parameters by reference to another batch file?
It explains in full details what the commands SETLOCAL and ENDLOCAL do.
How to pass a command that may contain special characters (such as % or !) inside a variable to a for /f loop?
This is an example of a batch file designed to process video files with any valid file name on any Windows computer very efficient, safe and secure with full explanation.
Well, the main trick is to enable delayed expansion only when it is actually needed and to disable it otherwise. Since you are accumulating multiple strings in a single variable inside of a loop, it becomes a bit more difficult, because you should have delayed expansion disabled during expansion of for meta-variables (like %%a), but enabled when joining the string, leading to setlocal and endlocal statements inside of the loop. The major purpose of these commands is environment localisation, hence any variable changes become lost past endlocal, so a method of tansfering the value beyond endlocal is required, which is incorporated in the following code:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem /* At this point delayed expansion is disabled, so there is no need to
rem escape exclamation marks; moreover a redirected block prevents
rem superfluous file close/reopen operations, and there is no more
rem trailing space written to the file (as in your original approach): */
> "textfile.txt" (
echo This is an example!
echo This is a second example!
echo And this line have an ! exclamation mark in the middle
)
rem // Let us initialise the output variable:
set "Output= - "
rem // Using `usebackq` only makes sense when you want to quote a file path:
for /F "usebackq tokens=* delims=" %%a in ("textfile.txt") do (
rem // Remember that delayed expansion is still disabled at this point:
set "Record=%%a"
rem // For concatenation we need delayed expansion to be enabled:
setlocal EnableDelayedExpansion
set "Output=!Output!!Record! - "
rem /* We need to terminate the environment localisation of `setlocal`
rem inside of the loop, but we would lose any changes in `Output`;
rem therefore let us (mis-)use `for /F`, which is iterated once: */
for /F "delims=" %%b in ("!Output!") do endlocal & set "Output=%%b"
rem /* An often used method to transfer a variable beyond `endlocal` is
rem the line `endlocal & set "Output=%Output%`, but this only works
rem outside of a parenthesised block because of percent expansion. */
)
rem /* Echo out text with delayed expansion enabled is the only safe way;
rem surrounding separators ` - ` are going to be removed; since `Output`
rem was initialised with something non-empty, we do not even need to skip
rem sub-string expansion for the problematic case of an empty string: */
setlocal EnableDelayedExpansion
echo(!Output:~3,-3!
endlocal
endlocal
exit /B
Pew. I finally got it to work.
It works via a workaround using a second text file.
Not pretty, not performant, but it works and is sufficient for my purposes.
#Magoo, thanks for your post.
This is my solution:
#echo off
setlocal enableextensions enabledelayedexpansion
echo This is an example^^!> "textfile.txt"
echo This is a second example^^!>> "textfile.txt"
echo And this line have an ^^! exclamation mark in the middle>> "textfile.txt"
echo.
echo Content of the textfile:
type "textfile.txt"
set output=
del "textfile2.txt" 1> nul 2>&1
setlocal disabledelayedexpansion
for /f "usebackq tokens=* delims=" %%a IN ("textfile.txt") do (
rem Write each line without a newline character into a new text file
echo|set /p "dummy=%%a, ">>"textfile2.txt"
)
endlocal
rem Loading the content of the new text file into the variable
set /p output=<"textfile2.txt"
del "textfile2.txt" 1> nul 2>&1
echo.
echo --------------------------------------------
echo Content of the variable:
set out
endlocal
The output looks like this:
Content of the textfile:
This is an example!
This is a second example!
And this line have an ! exclamation mark in the middle
--------------------------------------------
Content of the variable:
output=This is an example!, This is a second example!, And this line have an ! exclamation mark in the middle,
It's delayedexpansion mode that appears to raise this problem.
#ECHO OFF
setlocal enabledelayedexpansion
echo This is an example^^^! > "textfile.txt"
echo This is a second example^^^! >> "textfile.txt"
echo And this line have an ^^^! exclamation mark in the middle >> "textfile.txt"
TYPE "textfile.txt"
SETLOCAL disabledelayedexpansion
for /F "usebackq tokens=* delims=" %%a in (textfile.txt) do (
set "Record=%%a"
CALL set "Output2=%%Output2%%%%record%% - "
CALL set "Output=%%Output%%%%a - "
SET out
)
)
endlocal&SET "output=%output%"
echo %Output%
echo !Output!
SET out
I've no doubt that with delayedexpansion off, there would be the same problem with %. Just special characters, I suppose.
Note that with endlocal&SET "output=%output%", the set is executed in delayedexpansion mode.
Let's say I have a couple of images and I need to rename them and on every iteration add an incremented number.
For this situation I have three images no matter how they name is and I want to rename them like this.
1239.jpg => file1.jpg
file.jpg => file2.jpg
image.jpg => file3.jpg
My commands executed in a command prompt window for this task are:
setlocal EnableDelayedExpansion
set filename=file
set counter=1
for /f "usebackq delims=*" %i in ('dir /b *.jpg') do (set /a counter+=1 ren "%i" "%filename%!counter!.jpg")
But this results in the error message Missing operator.
Can anyone help me with this?
The commands SETLOCAL and ENDLOCAL can be used only in a batch file. Please read this answer for details about the commands SETLOCAL and ENDLOCAL. These two commands do nothing on being executed in a command prompt window. It is necessary to start cmd.exe with option /V:ON to use delayed expansion in a command prompt window as explained by the help output on running cmd /? in a command prompt window.
The usage of usebackq requires enclosing the command line to be executed in ` instead of ' as usual. usebackq is mainly used for processing the lines of a text file of which name is specified in the round brackets enclosed in ".
The following command line with the two commands SET and REN is not of valid syntax. The command SET interprets everything after /a as arithmetic expression to evaluate. In this case the expression misses an operator between 1 and ren whereby ren would be interpreted here as name of an environment variable and not as command to execute next after set.
(set /a counter+=1 ren "%i" "%filename%!counter!.jpg")
The valid command line would be:
set /A "counter+=1" & ren "%i" "%filename%!counter!.jpg"
Enclosing the arithmetic expression in double quotes makes it clear for command SET where the arithmetic expression starts and where it ends. The conditional execution operator & is interpreted by Windows command processor before executing the command SET and results in execution of command REN after command SET even on SET would fail to evaluate the arithmetic expression.
A file renaming task done with Windows command processor is no easy to achieve if
the file extension of the files should not change and
files with any name including those with one or more &()[]{}^=;!'+,`~ should be supported and
there can be already files in the directory with one of the new file names.
For testing the batch file below I created first in a directory following files:
file.jpg
file1.jpg
file2.jpg
file3.jpg
file 4.jpg
File8.jpg
hello!.jpg
image.jpg
The directory was on a FAT32 drive. The file systems FAT16, FAT32 and exFAT return a list of matching directory entries not sorted by name as NTFS which means the list output by command DIR in the main FOR loop in code below is in an unsorted and therefore unpredictable order.
It would be of course possible to append the DIR option /ON to get the list of file names ordered by DIR according to name, but in fact that is not real help in this case, especially because of DIR makes a strict alphabetical sort and not an alphanumeric sort.
A strict alphabetic sort returns a list of ten file names as file1.jpg, file10.jpg, file2.jpg, file3.jpg, ..., file9.jpg while an alphanumeric sort returns a list of ten file names as file1.jpg, file2.jpg, file3.jpg, ..., file9.jpg, file10.jpg.
So here is the commented batch file for this file rename task:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FileName=file"
rem The user can run this batch file with a folder path in which all *.jpg
rem files should be renamed with an incremented number. Otherwise the
rem directory of the batch file is search for *.jpg files to rename.
if not "%~1" == "" (
pushd "%~1" || exit /B
) else (
pushd "%~dp0" || exit /B
)
set "FileCount=0"
set "DelayedLoopCount=0"
set "DelayedRenameCount=0"
rem Remove all existing environment variables in local environment of which
rem name starts with DelayedRename_ whereby the underscore is very important
rem because there is also the environment variable DelayedRenameCount.
for /F "delims==" %%I in ('set DelayedRename_ 2^>nul') do set "%%I="
rem Get a captured list of all *.jpg files in current directory and then
rem rename one file after the other if that is possible on no other file
rem has by chance already the new file name for the current file.
for /F "eol=| delims=" %%I in ('dir *.jpg /A-D /B 2^>nul') do call :RenameFile "%%I"
goto DelayedRenameLoop
:RenameFile
set /A FileCount+=1
set "NewName=%FileName%%FileCount%%~x1"
rem Has the file case-sensitive already the right name?
if %1 == "%NewName%" goto :EOF
rem Is the new file name the same as the current name
rem with exception of the case of one or more letters?
if /I %1 == "%NewName%" (
echo Rename %1 to "%NewName%"
ren %1 "%NewName%"
goto :EOF
)
rem Is there no other file which has already the new name?
if not exist "%NewName%" (
echo Rename %1 to "%NewName%"
ren %1 "%NewName%"
goto :EOF
)
rem Another file or folder has already the new name. Remember the name
rem of this file and the new file name with an environment variable for
rem a delayed rename after all other files have been renamed as far as
rem possible.
set /A DelayedRenameCount+=1
set "DelayedRename_%DelayedRenameCount%=|%~1|%NewName%"
goto :EOF
rem It could happen that "file15.jpg" should be renamed to "file3.jpg"
rem while "file3.jpg" exists already which should be renamed to "file12.jpg"
rem while "file12.jpg" exists already which should be renamed to "file20.jpg".
rem This extra loop is used for such worst case scenarios which is executed
rem in a loop until all files have been renamed with a maximum of 50 loop
rem runs in case of one file cannot be renamed and therefore blocking
rem renaming of another file. An endless running loop should be avoided.
rem A file cannot be renamed if a folder has by chance the new file name.
rem A file cannot be renamed if an application has opened the file with
rem a sharing access mode preventing the rename of the file as long as
rem being opened by this application.
:DelayedRenameLoop
if %DelayedRenameCount% == 0 goto EndBatch
for /F "tokens=1-3 delims=|" %%I in ('set DelayedRename_ 2^>nul') do if not exist "%%K" (
echo Rename "%%J" to "%%K"
ren "%%J" "%%K"
set "%%I"
set /A DelayedRenameCount-=1
)
set /A DelayedLoopCount+=1
if not %DelayedLoopCount% == 50 goto DelayedRenameLoop
:EndBatch
popd
endlocal
This batch file output on execution:
Rename "file3.jpg" to "file4.jpg"
Rename "file 4.jpg" to "file5.jpg"
Rename "File8.jpg" to "file6.jpg"
Rename "hello!.jpg" to "file7.jpg"
Rename "image.jpg" to "file8.jpg"
Rename "file2.jpg" to "file3.jpg"
Rename "file1.jpg" to "file2.jpg"
Rename "file.jpg" to "file1.jpg"
The files in the directory were finally:
file1.jpg
file2.jpg
file3.jpg
file4.jpg
file5.jpg
file6.jpg
file7.jpg
File8.jpg
What about the last file?
It has the file name File8.jpg instead of file8.jpg although executed was ren "image.jpg" "file8.jpg". Well, FAT32 is a bit problematic regarding to updates of the file allocation table on a table entry changes only in case of one or more letters.
The solution is using this batch file with two extra FOR loops with # as loop variable and optimized by removing the comments.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FileName=file"
if not "%~1" == "" (pushd "%~1" || exit /B) else (pushd "%~dp0" || exit /B)
set "FileCount=0"
set "DelayedLoopCount=0"
set "DelayedRenameCount=0"
for /F "delims==" %%I in ('set DelayedRename_ 2^>nul') do set "%%I="
for /F "eol=| delims=" %%I in ('dir *.jpg /A-D /B 2^>nul') do call :RenameFile "%%I"
goto DelayedRenameLoop
:RenameFile
set /A FileCount+=1
set "NewName=%FileName%%FileCount%%~x1"
if %1 == "%NewName%" goto :EOF
if /I %1 == "%NewName%" (
echo Rename %1 to "%NewName%"
ren %1 "%NewName%"
goto :EOF
)
if not exist "%NewName%" (
echo Rename %1 to "%NewName%"
ren %1 "%NewName%"
for %%# in ("%NewName%") do if not "%%~nx#" == "%NewName%" ren "%%~nx#" "%NewName%"
goto :EOF
)
set /A DelayedRenameCount+=1
set "DelayedRename_%DelayedRenameCount%=|%~1|%NewName%"
goto :EOF
:DelayedRenameLoop
if %DelayedRenameCount% == 0 goto EndBatch
for /F "tokens=1-3 delims=|" %%I in ('set DelayedRename_ 2^>nul') do if not exist "%%K" (
echo Rename "%%J" to "%%K"
ren "%%J" "%%K"
for %%# in ("%%K") do if not "%%~nx#" == "%%K" ren "%%~nx#" "%%K"
set "%%I"
set /A DelayedRenameCount-=1
)
set /A DelayedLoopCount+=1
if not %DelayedLoopCount% == 50 goto DelayedRenameLoop
:EndBatch
popd
endlocal
The result of this enhanced batch file is even on FAT32:
file1.jpg
file2.jpg
file3.jpg
file4.jpg
file5.jpg
file6.jpg
file7.jpg
file8.jpg
The reason for using | as string separator on execution of
set "DelayedRename_%DelayedRenameCount%=|%~1|%NewName%"
resulting, for example, in execution of
set "DelayedRename_1=|file.jpg|file1.jpg"
set "DelayedRename_2=|file1.jpg|file2.jpg"
set "DelayedRename_3=|file2.jpg|file3.jpg"
is that the vertical bar is not allowed in a file folder name. So it is a very good character to separate the name of the environment variable with the equal sign appended from current file name and from new file name. This makes it possible to use later delims=| for renaming the file and deleting the environment variable.
See also the Microsoft documentations:
Naming Files, Paths, and Namespaces
Using command redirection operators
The equal sign is allowed in a file name. It is even possible that a *.jpg file has as file name =My Favorite Picute=.jpg which is another reason for using | to get executed for example
set "DelayedRename_4=|=My Favorite Picute=.jpg|file9.jpg"
which later results in assigned DelayedRename_4= to loop variable I, =My Favorite Picute=.jpg to loop variable J and file9.jpg to loop variable K in the FOR loop doing the delayed file renames.
Note: Each FOR loop with '...' in the round brackets results
in starting in background one more command process with %ComSpec% /c '...' and
capturing the output written to handle STDOUT like the output of the cmd.exe internal commands DIR and SET
while cmd.exe processing the batch file waits until started cmd.exe terminated (closed) itself after execution of the command line
and then processing the captured lines one after the other by FOR with ignoring empty lines and lines starting with the defined end of line character after doing the string delimiting which is the reason why eol=| is used on main FOR loop as a file name can start with default end of line character ; and which of course should not be ignored here.
The redirection operator > must be escaped with caret character ^ on those FOR command lines to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir or set command line in the separate command process started in background.
The batch file does not use delayed expansion as this would cause troubles on a file name having one or more exclamation marks which would be interpreted as beginning/end of a delayed expanded environment variable reference on command lines like ren "%%J" "%%K". Therefore a subroutine is used for the main file rename loop on which it is necessary to access the two incremented counter values.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
dir /?
echo /?
endlocal /?
exit /?
goto /?
if /?
popd /?
pushd /?
rem /?
ren /?
set /?
setlocal /?
I suggest further to look on:
Microsoft documentation for the Windows Commands
SS64.com - A-Z index of Windows CMD commands
Where does GOTO :EOF return to?
Single line with multiple commands using Windows batch file
Symbol equivalent to NEQ, LSS, GTR, etc. in Windows batch files
How does the Windows Command Interpreter (CMD.EXE) parse scripts?
Why is no string output with 'echo %var%' after using 'set var = text' on command line?
I have a folder with four to five text files in it.
My overall aim is the following: Create one big file which has the content of the separate files, but in the right order.
I can use the time-stamp of each file to start with the oldest file up to the youngest.
My process right now looks like this:
Order the files in this folder by date.
Create a temporary file and write the content from the separate files into this file.
Output the temporary file.
In code I do something like this:
set temp_concat=%temp_dir%\temp_concat.log
echo %temp_concat%
echo aiu_logs > %temp_concat%
for /f "delims=" %%? in ('dir /b /o:d %Folder%*') do (
for /f "delims=" %%K in (%Folder%%%?) do (
echo %%K >>%temp_concat%
)
)
The above code seems to work as my temp_concat is very large.
However, this takes much much longer than expected. I have to wait about 40 seconds just to merge three files in my case.
Is there some better way of merging some amount of files, but keep them in the correct order by date?
This batch file uses the suggestion posted by Sqashman to use a FOR loop to create the arguments string for command COPY used to concatenate the file contents into a single file in the order of oldest modified file first and newest modified file last.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
set "Folder=%~dp0"
if not "%~1" == "" set "Folder=%~1"
set "Folder=%Folder:/=\%"
if not "%Folder:~-1%" == "\" set "Folder=%Folder%\"
set "ResultsFile=%Folder%Results.log"
del "%ResultsFile%" 2>nul
set "Arguments="
for /F "eol=| delims=" %%I in ('dir /A-D-H /B /O:D "%Folder%*" 2^>nul') do if not "%%~fI" == "%~f0" set "Arguments=!Arguments! + "%%I""
if defined Arguments (
echo aiu_logs>"%ResultsFile%"
copy /B "%ResultsFile%"%Arguments% "%ResultsFile%" >nul
)
endlocal
The batch file as is does not work if either the folder path or one of the file names contains one or more exclamation marks ! because of an enabled delayed environment variable expansion.
Further the command line length is limited and so this batch file does not work on too many files must be concatenated depending on length of the file path of each file and the length of the file names.
A better solution would be using following batch file:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "Folder=%~dp0"
if not "%~1" == "" set "Folder=%~1"
set "Folder=%Folder:/=\%"
pushd "%Folder%" 2>nul
if errorlevel 1 goto EndBatch
set "ResultsFile=Results.log"
del "%ResultsFile%" 2>nul
set "Arguments="
for /F "eol=| delims=" %%I in ('dir /A-D-H /B /O:D * 2^>nul') do if not "%%~fI" == "%~f0" call set "Arguments=%%Arguments%% + "%%I""
if defined Arguments (
echo aiu_logs>"%ResultsFile%"
copy /B "%ResultsFile%"%Arguments% "%ResultsFile%" >nul
)
popd
:EndBatch
endlocal
A folder path with one or more exclamation marks is no problem anymore. Also the file names can contain ! because of delayed expansion is not used by this batch file which is a bit slower than the first batch file.
The folder with the files to concatenate is made the current directory by this batch file. For that reason more file names can be specified as arguments on COPY command line in comparison to first batch file because of the file names are specified without path. But the number of file contents which can be merged with this batch file is nevertheless limited by the maximum length of a Windows command line respectively the maximum length of an environment variable value.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
copy /?
dir /?
echo /?
endlocal /?
for /?
goto /?
if /?
popd /?
pushd /?
set /?
setlocal /?
Read also the Microsoft article about Using command redirection operators for an explanation of > and 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command line to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir command line with using a separate command process started in background with %ComSpec% /c and the DIR command line between the two ' appended as further arguments.
The second FOR /F does not contain a command. It contains a filename. I have not tested this, but perhaps:
set temp_concat=%temp_dir%\temp_concat.log
echo %temp_concat%
echo aiu_logs > "%temp_concat%"
for /f "delims=" %%? in ('dir /b /o:d "%Folder%"') do (
if not "%%~f?" == "%~f0" (
type %%? >>"%temp_concat%"
)
)
This will concatenate all files in the "%Folder%" directory. Paths should be quoted in case there are special characters in them.
I need a windows batch file to create a folder based on part of a file name (the part before an underscore) and move any files that start with the folder name into the folder.
I'm not familiar with windows batch files. I've googled and tinkered a solution which works except that I cannot substring the file name at the underscore.
(Yes there are a few similar threads but nothing I could use to exactly answer my question)
FWIW my unsuccessful solution:
#ECHO OFF
setlocal enabledelayedexpansion
SETLOCAL
SET "sourcedir=C:\Development\test"
PUSHD %sourcedir%
FOR /f "tokens=1*" %%a IN (
'dir /b /a-d "TTT*_*.*"'
) DO (
ECHO MD NEED FILE NAME BEFORE UNDERSCORE HERE
ECHO MOVE "%%a" .\NEED FILE NAME BEFORE UNDERSCORE HERE\
)
(Ideally I'd remove the leading 'TTT' from files too but if necessary can create the files without this.)
Try this batch file code:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "SourceDir=C:\Development\test"
set "DestDir=C:\Development\test"
for /F "eol=| delims=" %%A in ('dir /B /A-D-H "%SourceDir%\TTT*_*" 2^>nul') do (
for /F "eol=| tokens=1 delims=_" %%B in ("%%~nA") do (
md "%DestDir%\%%B" 2>nul
move /Y "%SourceDir%\%%A" "%DestDir%\%%B\"
)
)
endlocal
The first FOR executes in a separate command process started with cmd.exe /C in background the command line:
dir /B /A-D-H "C:\Development\test\TTT*_*" 2>nul
DIR searches in specified directory for
just non-hidden files because of /A-D-H (attribute not directory and not hidden)
matching the wildcard pattern TTT*_* which could be also just *_*
and outputs to handle STDOUT in bare format because of /B just the file names with file extension, but without file path.
The error message output by DIR to handle STDERR if the specified directory does not exist at all or there is no file matching the pattern is suppressed by redirecting it with 2>nul to device NUL.
Read also the Microsoft documentation about Using Command Redirection Operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command line to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir command line with using a separate command process started in background.
FOR captures everything written to STDOUT of started command process and processes the captured output line by line.
FOR ignores by default all empty lines (do not occur here) and all lines starting with a semicolon. A file name could begin with a semicolon. For that reason option eol=| is used to redefine end of line character to vertical bar which a file name can't contain, see Microsoft documentation Naming Files, Paths, and Namespaces. In this case on using TTT*_* as wildcard pattern it is not possible that a file name starts with a semicolon, but it would be possible on usage of *_* as wildcard pattern.
FOR would split up also each line into substrings (tokens) using space/tab as delimiters and would assign just the first space/tab separated string to specified loop variable A. This splitting behavior is not wanted here as file names can contain one or more space characters. Therefore the option delims= is used to define an empty list of delimiters which disables line splitting completely and results in assigning entire file name with extension to loop variable A.
The inner FOR processes just the file name (without extension) as string. This time the file name is split up using the underscore as delimiter because of delims=_ with assigning just first underscore delimited string to loop variable B because of tokens=1. Well, tokens=1 is the default on using for /F and so this option string could be removed from code.
So the outer FOR assigns to A for example TTTxy_test & example!.txt and the inner FOR processes TTTxy_test & example! and assigns to B the string TTTxy.
The command MD creates in set destination directory a subdirectory for example with name TTTxy. An error message is output also on directory already existing. This error message is suppressed by redirecting it to device NUL.
Then the file is moved from source to perhaps just created subdirectory in destination directory with overwriting an existing file with same name in target directory of the file.
The inner FOR loop could be optimized away when there are never files starting with an underscore or which have more than one underscore after first part of file name up to first underscore.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "SourceDir=C:\Development\test"
set "DestDir=C:\Development\test"
for /F "eol=| tokens=1* delims=_" %%A in ('dir /B /A-D-H "%SourceDir%\TTT*_*" 2^>nul') do (
md "%DestDir%\%%A" 2>nul
move /Y "%SourceDir%\%%A_%%B" "%DestDir%\%%A\"
)
endlocal
Option tokens=1* results in assigning first underscore delimited part of file name to loop variable A and rest of file name to next loop variable B according to ASCII table without further splitting up on underscores.
But please take into account that the optimized version does not work for file names like
_TTTxy_test & example!.txt ... underscore at beginning (ignored by pattern), or
TTTxy__test & example!.txt ... more than one underscore after first part.
The optimized version can be further optimized to a single command line:
#for /F "eol=| tokens=1* delims=_" %%A in ('dir /B /A-D-H "C:\Development\test\TTT*_*" 2^>nul') do #md "C:\Development\test\%%A" 2>nul & move /Y "C:\Development\test\%%A_%%B" "C:\Development\test\%%A\"
Well, the not optimized version could be also written as even longer single command line:
#for /F "eol=| delims=" %%A in ('dir /B /A-D-H "C:\Development\test\TTT*_*" 2^>nul') do #for /F "eol=| tokens=1 delims=_" %%B in ("%%~nA") do #md "C:\Development\test\%%B" 2>nul & move /Y "C:\Development\test\%%A" "C:\Development\test\%%B\"
See also Single line with multiple commands using Windows batch file for an explanation of operator &.
For additionally removing TTT from file name on moving the file the first batch code is modified with using two additional commands SET and CALL:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "SourceDir=C:\Development\test"
set "DestDir=C:\Development\test"
for /F "eol=| delims=" %%A in ('dir /B /A-D-H "%SourceDir%\TTT*_*" 2^>nul') do (
for /F "eol=| tokens=1 delims=_" %%B in ("%%~nA") do (
md "%DestDir%\%%B" 2>nul
set "FileName=%%A"
call move /Y "%SourceDir%\%%A" "%DestDir%\%%B\%%FileName:~3%%"
)
)
endlocal
The file name is assigned to an environment variable FileName. The value of this environment variable cannot be referenced with just using %FileName% because of all references of environment variable values using percent signs are substituted by Windows command processor in entire command block starting with first ( and ending with matching ) before FOR is executed at all. Delayed expansion is usually used in such cases, but that would result here in file names containing one or more exclamation marks would not be corrected processed by the batch file.
The solution is using %% on both sides of FileName environment variable reference instead of % and force a double parsing of the command line by using command CALL.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
dir /?
echo /?
endlocal /?
for /?
md /?
move /?
set /?
setlocal /?
It is really very simple:
#echo off
for /f "tokens=1-2 delims=_" %%i in ('dir /b /a-d "TTT*_*"') do (
if not exist "%%i" mkdir "%%i"
move "%%i_%%j" "%%i\%%j"
)
We split by _ into 2 tokens, %%i everything before _ and %%j everything after.
We simply create folder (if it does not exist) then move the file with only the name after the _ into the new folder.
So as an example file TTT123_File1.txt will create a folder called TTT123 and place the file into it but rename it as File1.txt
You might consider using Tcl/Tk. Tcl/Tk is an open source script language. You can call it as a stand-alone or execute it from a windows batch file. You will need to install it first if you don't have it yet.
The following Tcl script does what you want:
cd "C:/Development/test"
# glob is a tcl command to list all functions that match the requirements
set files [glob TTT*_*]
foreach f $files {
# use the underscore as a separator to split f and store the parts in dir and fnew
lassign [split $f "_"] dir fnew
if {![file exist $dir]} {
file mkdir $dir
}
file rename $f [file join $dir $fnew]
}
In my opinion, this is a very readable script, even if you don't know tcl.
You can call this script from a batch file as:
tclsh script.tcl
if you have saved the script as script.tcl
In windows 10, I need to make a batch with a loop on the subfolder names in a folder, I did the following, but the problem is the %%I doesn't manage the folder name with spaces, it takes only the first part:
#echo off
FOR /F %%I IN ('dir /b C:\Users\Thomas\Music') DO (
ECHO %%I)
If the folder "Music" contains the folder "My music", then echo %%I will print only "My".
FOR /F "delims=" %%I IN ('dir /b /ad C:\Users\Thomas\Music') DO (
... and use "%%I" where you want to use the name-containing-spaces (ie. quote the constructed string) - a principle that applies wherever batch uses strings containing separators like Space
The /ad selects directorynames instead of filenames.
Adding a further switch, /s will scan the entire subdirectory-tree.
Assignment of string values to variables is best done with
set "var=%variablefrom%"
or in the case of a metavariable (eg the loop-control variable %%I in your code) you need
set "var=%%I"
BUT you should investigate the topic of delayed expansion (many items here) if you want to use the value of the variable assigned (var) within the loop.
My best-practice concept shown in a commented .bat script:
#ECHO OFF
SETLOCAL EnableExtensions DisableDelayedExpansion
FOR /F "delims=" %%I IN ('dir /b /AD "%UserProfile%\Music" 2^>NUL') DO (
rem process a FOR-loop variable directly
ECHO For_variable "%%~I"
rem set a FOR-loop variable as an environment variable value
set "_myVar=%%~I" see tilde which would strip double quotes
rem process an environment variable INside a FOR-loop body
SETLOCAL EnableDelayedExpansion
echo Env_variable "!_myVar!"
ENDLOCAL
rem process an environment variable OUTside a FOR-loop body
call :ProcessMyVar
)
ENDLOCAL
goto :eof
:ProcessMyVar
rem process an environment variable OUTside a FOR-loop body
echo Env_var_Call "%_myVar%"
goto :eof
Output shows that even names with cmd-poisonous characters like percent or exclamation (etc. etc.) are processed properly:
==> tree "%UserProfile%\Music"
Folder PATH listing
Volume serial number is 0000005F F2DF:F89D
C:\USERS\USER\MUSIC
├───100% My Tunes!
└───My Songs!
==> D:\bat\SO\39697872.bat
For_variable "100% My Tunes!"
Env_variable "100% My Tunes!"
Env_var_Call "100% My Tunes!"
For_variable "My Songs!"
Env_variable "My Songs!"
Env_var_Call "My Songs!"
==>
Resources (required reading, incomplete):
(command reference) An A-Z Index of the Windows CMD command line
(helpful particularities) Windows CMD Shell Command Line Syntax
(%I, %~I etc. special page) Command Line arguments (Parameters)
(special page) EnableDelayedExpansion
(>, & etc. special page) Redirection
(%% doubled percent sign, ^ caret, double quotes) Escape Characters, Delimiters and Quotes