I want to trigger a windows command over multiple files inside a directory at the same time. 'FOR' loop and other recursive methods are triggering the commands one after another. I need an alternative that can run the same command on all files at the same time. The present code I have is
#echo off
call :scan
goto :eof
:scan
for %%f in (*.txt) do *mycommand* -i %%f
for /D %%d in (*) do (
cd %%d
call :scan
cd ..
)
exit /b
The easiest way to have slow tasks running in parallel on Windows is by using the start command and calling another batch file, e.g.:
scan.bat
#echo off
echo Processing %1
dir %1
exit
main.bat
for %%f in (*.txt) do start scan.bat %%f
This will create a new window for each scan.bat instance. If you want to use the current window, use start /b scan.bat.
The exit command at the end of scan.bat is important so that the command processor exists (so that the window is closed).
In case you want to limit the number of tasks running in parallel, you should use something more powerful like gnu make (which can be used on Windows with cygwin or mingw; see Limiting the number of spawned processes in batch script for a solution using batch files (from #LotPings).
Related
For example, I have a simple batch script get_path.bat
#echo off
echo C:\Software\dt
Also, I have another simple batch script switch_dir.bat
#echo off
get_path.bat > target.tmp
set /p TARGET=<target.tmp
cd %TARGET%
Now, what I want to accomplish is that when I invoke in cmd.exe the batch file switch_dir.bat, my current working directory changes to C:\Software\dt.
As of now, the scripts work but they run in a process spawned from cmd.exe so my current working directory stays the same. What is missing? Basically, we need Unix-like source or . here.
Well, there are many possible solutions:
Start your script with cmd /c:
All you have to write in cmd is:
cmd /c switch_dir.bat
Using popd/pushd in your batch file:
In your switch_dir.bat add:
#echo off
pushd dir\you\want\to\remain\
get_path.bat > target.tmp
set /p TARGET=<target.tmp
cd %TARGET%
rem [code...]
popd
An additional note: A better way to find directory specified in get_path.bat is to use a for /f loop like this:
#echo off
pushd dir\you\want\to\remain\
for /f "delims=" %%A IN ('get_path.bat') do set TARGET=%%A
cd %TARGET%
rem [code...]
popd
In my efforts to understand the for..do loops syntax and their use of %% variables. I have gone through 2 specific examples/implementations where the one for loop does not use DELAYEDEXPANSION and another where it does use DELAYEDEXPANSION with the ! notation. The 1st for loop appears to be compatible with older OSs like the Windows XP whereas the 2nd for loop example does not.
Specifically, the 1st for loop example is taken from this answer (which is related to this) and the 2nd for loop example is taken from this answer.
Modified code for both examples copied below:
1st for loop
for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a"
set "YY=%dt:~2,2%"
set "YYYY=%dt:~0,4%"
set "MM=%dt:~4,2%"
set "DD=%dt:~6,2%"
set "HH=%dt:~8,2%"
set "Min=%dt:~10,2%"
set "Sec=%dt:~12,2%"
set "datestamp=%YYYY%%MM%%DD%"
set "timestamp=%HH%%Min%%Sec%"
echo datestamp: "%datestamp%"
echo timestamp: "%timestamp%"
2nd for loop
SETLOCAL ENABLEDELAYEDEXPANSION
set "path_of_folder=C:\folderA\folderB"
for /f "skip=5 tokens=1,2,4 delims= " %%a in (
'dir /ad /tc "%path_of_folder%\."') do IF "%%c"=="." (
set "dt=%%a"
set vara=%%a
set varb=%%b
echo !vara!, !varb!
set day=!vara:~0,2!
echo !day!
)
Since I have been reading and seeing issues where delayed expansion (or the ! notation) is not compatible with older OSs (e.g. Windows XP), I would like to see how to write the 2nd loop like the 1st loop; i.e. without the use of DELAYEDEXPANSION.
I explain in detail what aschipfl wrote already absolutely right in his comment.
Both batch files work also on Windows 2000 and Windows XP using also cmd.exe as command processor. The batch files do not work on MS-DOS, Windows 95 and Windows 98 using very limited command.com as command interpreter.
A command can be executed with parameter /? in a command prompt window to get output the help for this command. When in help is written with enabled command extensions it means supported only by cmd.exe on Windows NT based Windows versions and not supported by MS-DOS or Windows 9x using command.com. That means, for example, for /F or if /I or call :Subroutine are not available on Windows 9x, or on Windows NT based Windows with command extensions explicitly disabled. On Windows 9x it is not even possible to use "%~1" or "%~nx1".
The first batch file executes in FOR loop only one command exactly once:
set "dt=%%a"
That command line requires already enabled command extensions. All other commands below are executed after the FOR loop finished. In other words the FOR loop in first batch file does not use a command block to run multiple commands within the FOR loop.
Whenever the Windows command processor detects the beginning of a command block on a command line, it processes the entire command block before executing the command on this command line the first time.
This means for second batch file all variable references using %Variable% are expanded already before the command FOR is executed and then the commands in the command block are executed with the values of the variables as defined above FOR command line. This can be seen by removing #echo off from first line of batch file or change it to #echo ON and run the batch file from within a command prompt window because now it can be seen which command lines respectively entire command blocks defined with ( ... ) are really executed after preprocessing them by the Windows command processor.
So whenever an environment variable is defined or modified within a command block and its value is referenced in same command block it is necessary to use delayed expansion or use workarounds.
One workaround is demonstrated below:
setlocal EnableExtensions DisableDelayedExpansion
set "FolderPath=%SystemRoot%\System32"
for /F "skip=5 tokens=1,2,4 delims= " %%a in ('dir /AD /TC "%FolderPath%\."') do if "%%c"=="." (
set "VarA=%%a"
set "VarB=%%b"
call echo %%VarA%%, %%VarB%%
call set "Day=%%VarA:~0,2%%
call echo %%Day%%
)
endlocal
pause
As there is no #echo off at top of this batch code it can be seen on executing the batch file what happens here. Each %% is modified on processing the command block to just %. So executed are the command lines.
call echo %VarA%, %VarB%
call set "Day=%VarA:~0,2%
call echo %Day%
The command CALL is used to process the rest of the line a second time to run the ECHO and the SET commands with environment variable references replaced by their corresponding values without or with string substitution.
The disadvantage of this solution is that CALL is designed primary for calling a batch file from within a batch file. For that reason the command lines above result in searching first in current directory and next in all directories of environment variable PATH for a file with name echo respectively set with a file extension of environment variable PATHEXT. That file searching behavior causes lots of file system accesses, especially on running those command lines in a FOR loop. If there is really found an executable or script file with file name echo or set, the executable respectively the script interpreter of the script file would be executed instead of the internal command of cmd.exe as usually done on using such a command line. So this solution is inefficient and not fail-safe on execution of the batch file.
Another workaround to avoid delayed expansion is using a subroutine:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FolderPath=%SystemRoot%\System32"
for /F "skip=5 tokens=1,2,4 delims= " %%a in ('dir /AD /TC "%FolderPath%\."') do if "%%c"=="." call :ProcessCreationDate "%%a" "%%b"
endlocal
pause
exit /B
:ProcessCreationDate
echo %~1, %~2
set "Day=%~1"
set "Day=%Day:~0,2%
echo %Day%
goto :EOF
A subroutine is like another batch file embedded in current batch file.
The command line with exit /B avoids a fall through to the code of the subroutine.
The command line with goto :EOF would not be necessary if the line above is the last line of the batch file. But it is recommended to use it nevertheless in case of more command lines are ever added later below like a second subroutine.
The second batch file is for getting the day on which the specified folder was created. It would be possible to code this batch file without usage of delayed expansion and any workarounds.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FolderPath=%SystemRoot%\System32"
for /F "skip=5 tokens=1,2,4 delims= " %%a in ('dir /ad /tc "%FolderPath%\." 2^>nul') do if "%%c"=="." set "CreationDate=%%a, %%b" & goto OutputDateAndDay
echo Failed to get creation date of "%FolderPath%"
endlocal
pause
exit /B
:OutputDateAndDay
echo %CreationDate%
set "Day=%CreationDate:~0,2%
echo %Day%
endlocal
pause
Once the line of interest with the creation date of specified folder is found, the creation date/time is assigned to an environment variable and the FOR loop is exited with using command GOTO to continue execution on a label below. For the meaning of operator & see single line with multiple commands using Windows batch file.
This solution is better than all other methods because the FOR loop executes the single command line with the three commands IF, SET and GOTO only once which makes this solution the fastest. And it outputs an error message when it was not possible to determine the creation date of the directory because of the directory does not exist at all.
Of course it would be possible to add a GOTO command also on the other solutions to exit FOR loop once the creation date of the directory was determined and output. The last solution is nevertheless the fastest and in my point of view best one for this task.
BTW: All posted batch file examples were tested on Windows XP and produced the expected output.
I'm stuck with an idea that I just can't get off the ground and it's because I don't understand FOR loops very well.
Essentially though, what I am trying to do is a FOR loop that will go through its current directory and for every folder that is 7 random numbers long, it'll move another file into that directory and call it. I'll make a diagram here.
%cd%
\1234567\
\2345671\
\3456712\
\Release\
filetomove.bat
So the intent is that the file to move will end up only in the numbered directories.
%cd%
\1234567\filetomove.bat
\2345671\filetomove.bat
\3456712\filetomove.bat
\Release\
filetomove.bat
Then once it is in those directories, it will call the bat in each of them.
%cd%
CALL \1234567\filetomove.bat
CALL \2345671\filetomove.bat
CALL \3456712\filetomove.bat
\Release\
filetomove.bat
I would make it more convoluted with an IF statement to only move a certain .bat if certain files are or aren't present, but I want to be able to get down moving the file first. I think I might be good on the IF statement.
Does anyone know what the heck I'd want to do to make this possible? I thought for file directories I could have done like
FOR /D %%G in ("%cd%\[0-9][0-9][0-9][0-9][0-9][0-9][0-9]\") DO COPY...
but apparently something like this doesn't work the way I intend. The only way I could actually see that's finding each file is to do something like...
FOR /D %%G in ("%cd%\*") DO ECHO %%G
and have the code return to me the directories string. What am I doing wrong, just don't know, or just am not understanding?
I'm having trouble understanding your entire idea, but here's how you can iterate over all the folders that have 7 digit names and then run batch's equivalent of a function to handle the output. Note that to use the %% variable syntax this needs to be run from inside a batch file. The command line uses only a single % in front of the iterator variable.
for /f "tokens=*" %%a in ('dir /b /aD * ^|findstr /R "^[0-9][0-9][0-9][0-9][0-9][0-9][0-9]$"') do call :junk "%%a"
REM stuff here runs after the for loop finishes
REM the goto :EOF here makes sure we aren't going to accidentally run :junk again when we didn't mean to.
goto :EOF
:junk
REM This runs each time 'for' spits out a new directory name.
set yourDirName=%~1
REM do other stuff here with this dir name, like copying or whatever.
REM If for instance you wanted to run a batch file c:\mybatfile.bat in
REM each dir, you could do that here:
pushd "%yourDirName%"
REM Now you're in the %yourDirName% directory and you can run your command:
c:\mybatfile.bat
popd
REM popd is the opposite of pushd.
goto :EOF
Here's an explanation of some of the syntax in the order it's used in the script.
for /f can parse command output. That's why it's always my favorite.
"tokens=*" causes the output from that iteration of the for-loop to all end up in one variable (%%a). Otherwise you would get the first delimited token in %%a, the next in %%b, etc.
Single-quote your command. I do it because it works, not because I know why.
For dir /b removes the columns of info you don't need and just gives the name. /aD gives you the files that have the Directory /attribute
You need to escape the | character with a ^ when it's part of a command you're giving to for
Use findstr /R so you can filter the names based on a regular expression.
call allows you to jump to a block of commands and pass arguments to that block (as %1, %2, etc.) so you can execute a set of commands without having to cram them into a set of parentheses since that gives you weird evaluation behavior.
When you use call, it's like running another script that just happens to be part of your current script. To return from that "other script"'s execution context, you use goto :EOF. It's distinct from exit because it doesn't completely terminate your program; it just does a goto on the meta-label :EOF which is short for End-Of-File. Here we want the script to act as if it has run out of commands (where it will keep executing a higher context if there was one) rather than exiting all the way out to the command line using exit.
The flow here is:
Start at the for loop
For each iteration of the for loop:
call :junk
Execute all of the commands under :junk until we hit the goto :EOF
goto :EOF at the bottom of :junk causes execution control to return to the for loop so it can iterate again.
When the for runs out of data to iterate over, it tries to execute more lines of the script. We don't want it to execute :junk again, so there's a goto :EOF above :junk.
Simple, right?
My problem is that two FOR loops are working separately, but don't want to work one after another.
The goal is:
The first loop creates XML files and only when the creation has already been done the second loop starts and counts the size of created XML files and writes it into .txt file.
#echo off
Setlocal EnableDelayedExpansion
for /f %%a in ('dir /b /s C:\Users\NekhayenkoO\test\') do (
echo Verarbeite %%~na
jhove -m PDF-hul -h xml -o C:\Users\NekhayenkoO\outputxml\%%~na.xml %%a
)
for /f %%i in ('dir /b /s C:\Users\NekhayenkoO\outputxml\') do (
echo %%~ni %%~zi >> C:\Users\NekhayenkoO\outputxml\size.txt
)
pause
This question can be answered easily when knowing what jhove is.
So I searched in world wide web for jhove, found very quickly the homepage JHOVE | JSTOR/Harvard Object Validation Environment and downloaded also jhove-1_11.zip from SourceForge project page of JHOVE.
All this was done by me to find out that jhove is a Java application which is executed on Linux and perhaps also on Mac using the shell script jhove and on Windows the batch file jhove.bat for making it easier to use by users.
So Windows command interpreter searches in current directory and next in all directories specified in environment variable PATH for a file matching the file name pattern jhove.* having a file extension listed in environment variable PATHEXT because jhove.bat is specified without file extension and without path in the batch file.
But the execution of a batch file from within a batch file without usage of command CALL results in script execution of current batch file being continued in the other executed batch file without ever returning back to the current batch file.
For that reason Windows command interpreter runs into jhove.bat on first file found in directory C:\Users\NekhayenkoO\test and never comes back.
This behavior can be easily watched by using two simple batch files stored for example in C:\Temp.
Test1.bat:
#echo off
cd /D "%~dp0"
for %%I in (*.bat) do Test2.bat "%%I"
echo %~n0: Leaving %~f0
Test2.bat:
#echo %~n0: Arguments are: %*
#echo %~n0: Leaving %~f0
On running from within a command prompt window C:\Temp\Test1.bat the output is:
Test2: Arguments are: "Test1.bat"
Test2: Leaving C:\Temp\Test2.bat
The processing of Test1.bat was continued on Test2.bat without coming back to Test1.bat.
Now Test1.bat is modified to by inserting command CALL after do.
Test1.bat:
#echo off
cd /D "%~dp0"
for %%I in (*.bat) do call Test2.bat "%%I"
echo Leaving %~f0
The output on running Test1.bat from within command prompt window is now:
Test2: Arguments are: "Test1.bat"
Test2: Leaving C:\Temp\Test2.bat
Test2: Arguments are: "Test2.bat"
Test2: Leaving C:\Temp\Test2.bat
Test1: Leaving C:\Temp\Test1.bat
Batch file Test1.bat calls now batch file Test2.bat and therefore the FOR loop is really executed on all *.bat files found in directory of the two batch files.
Therefore the solution is using command CALL as suggested already by Squashman:
#echo off
setlocal EnableDelayedExpansion
for /f %%a in ('dir /b /s "%USERPROFILE%\test\" 2^>nul') do (
echo Verarbeite %%~na
call jhove.bat -m PDF-hul -h xml -o "%USERPROFILE%\outputxml\%%~na.xml" "%%a"
)
for /f %%i in ('dir /b /s "%USERPROFILE%\outputxml\" 2^>nul') do (
echo %%~ni %%~zi>>"%USERPROFILE%\outputxml\size.txt"
)
pause
endlocal
A reference to environment variable USERPROFILE is used instead of C:\Users\NekhayenkoO.
All file names are enclosed in double quotes in case of any file found in the directory contains a space character or any other special character which requires enclosing in double quotes.
And last 2>nul is added which redirects the error message output to handle STDERR by command DIR on not finding any file to device NUL to suppress it. The redirection operator > must be escaped here with ^ to be interpreted on execution of command DIR and not as wrong placed redirection operator on parsing already the command FOR.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
cd /?
dir /?
echo /?
for /?
And read also the Microsoft article Using command redirection operators.
You need to use the START command with the /WAIT flag when you launch an external application.
I believe it would look something like this:
START /WAIT jhove -m PDF-hul -h xml -o C:\Users\NekhayenkoO\outputxml\%%~na.xml %%a
That should cause the batch file to pause and wait for the external application to finish before proceeding.
I use start.bat to call multiple other batch files. I don't know how to do it right.
cd "C:\Users\server\Desktop\MAG8000_CSV\MAG8000_606402H016"
call mergingcsv.bat
I use this code and it works on my start.bat. It calls my mergingcsv.bat.
What do I need to call many mergingcsv.bat?
I need to open 41 mergingcsv.bat whereby each is in a separate folder.
When I do it like this
cd "C:\Users\server\Desktop\MAG8000_CSV\MAG8000_606402H016"
call mergingcsv.bat
cd "C:\Users\server\Desktop\MAG8000_CSV\MAG8000_606302H016"
call mergingcsv2.bat
cd "C:\Users\server\Desktop\MAG8000_CSV\MAG8000_606202H016"
call mergingcsv3.bat
cd "C:\Users\server\Desktop\MAG8000_CSV\MAG8000_606102H016"
call mergingcsv4.bat
after the first call it pauses and I need to click to continue:
And how to make it automated?
Try this commented batch code for StartMergingCsv.bat:
#echo off
rem Store full path of current directory on stack. Might be removed
rem if there is no need to restore the current directory after
rem processing all the batch files for merging CSV files.
pushd
rem For each subdirectory in C:\Users\server\Desktop\MAG8000_CSV
rem check if the subdirectory contains at least 1 mergingcsv*.bat.
rem If this condition is true, change the current directory to the
rem subdirectory containing the mergingcsv*.bat file(s) and call
rem each batch file matching this file name pattern.
for /D %%F in ("C:\Users\server\Desktop\MAG8000_CSV\*") do (
if exist "%%F\mergingcsv*.bat" (
cd /D "%%F"
echo Merging CSV files in directory %%F ...
for %%B in ("%%F\mergingcsv*.bat") do call "%%B"
)
)
rem Restore initial current directory from stack. Might be removed, see above.
popd
Don't use start.bat as name for your batch file because START is a Windows standard command which you would replace by your batch file with name start.bat.
There should be no pause between each call of a batch file, except the mergingcsv*.bat contains a command like PAUSE which waits for a key press. mergingcsv*.bat should also not contain command EXIT without parameter /B as otherwise command processor would be exited and therefore there is no return to StartMergingCsv.bat.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
cd /?
echo /?
for /?
if /?
pushd /?
popd /?
rem /?
Run in command prompt window also:
exit /?
pause /?
start /?