how to run several .cmd files in parallel - cmd

I have 4 .cmd files. I want to run then in parallel taking 2 at one time.
say my files are : 1.cmd, 2.cmd, 3.cmd, 4.cmd
i want to run 1.cmd and 2.cmd in parallel. Now when any of these ends , i want to run 3.cmd and then 4.cmd. In short, at any given time i want 2 of the .cmd files to run.
I am using the Start command for parallel execution. But I am new to scripting and I am getting confused on how to furmulate the above mentioned way of running the cmd files.
Any help would be appreciated.
Thanks
Debjani

I have given an answer to “Parallel execution of shell processes” once, quoted here:
Sounds more like you want to use
Powershell 2. However, you can spawn
new cmd windows (or other processes)
by using start, see also this
answer. Although you probably have to
use some other tools and a little
trickery to create something like a
"process pool" (to have only a maximum
of n instances running at a time).
You could achieve the latter by using
tasklist /im and counting how many
are already there (for loop or wc,
if applicable) and simply wait (ping
-n 2 ::1 >nul 2>&1) and re-check again whether you can spawn a new
process.
I have cobbled together a little test
batch for this:
#echo off
for /l %%i in (1,1,20) do call :loop %%i
goto :eof
:loop
call :checkinstances
if %INSTANCES% LSS 5 (
rem just a dummy program that waits instead of doing useful stuff
rem but suffices for now
echo Starting processing instance for %1
start /min wait.exe 5 sec
goto :eof
)
rem wait a second, can be adjusted with -w (-n 2 because the first ping
returns immediately;
rem otherwise just use an address that's unused and -n 1)
echo Waiting for instances to close ...
ping -n 2 ::1 >nul 2>&1
rem jump back to see whether we can spawn a new process now
goto loop
goto :eof
:checkinstances
rem this could probably be done better. But INSTANCES should contain
the number of running instances
afterwards.
for /f "usebackq" %%t in (tasklist /fo csv /fi "imagename eq
wait.exe"^|wc -l) do set
INSTANCES=%%t
goto :eof
It spawns a maximum of four new
processes that execute in parallel and
minimized. Wait time needs to be
adjusted probably, depending on how
much each process does and how long it
is running. You probably also need to
adjust the process name for which
tasklist is looking if you're doing
something else.
There is no way to properly count the
processes that are spawned by this
batch, though. One way would be to
create a random number at the start of
the batch (%RANDOM%) and create a
helper batch that does the processing
(or spawns the processing program) but
which can set its window title to a
parameter:
#echo off
title %1
"%2" "%3"
This would be a simple batch that sets
its title to the first parameter and
then runs the second parameter with
the third as argument. You can then
filter in tasklist by selecting only
processes with the specified window
title (tasklist /fi "windowtitle eq
..."). This should work fairly
reliable and prevents too many false
positives. Searching for cmd.exe
would be a bad idea if you still have
some instances running, as that limits
your pool of worker processes.
You can use %NUMBER_OF_PROCESSORS%
to create a sensible default of how
many instances to spawn.
You can also easily adapt this to use
psexec to spawn the processes
remotely (but wouldn't be very viable
as you have to have admin privileges
on the other machine as well as
provide the password in the batch).
You would have to use process names
for filtering then, though.

I have not tried this, but I assume you can do this with PowerShell. Use this type of structure:
http://www.dougfinke.com/blog/index.php/2008/01/30/run-your-powershell-loops-in-parallel/
Within this example you should be able to execute cmd/bat files.
Check out the following thread for some ideas (possible duplicate?)
Parallel execution of shell processes

Related

How to kill a single process on multiple workstations

I'm trying to write a .bat file that can look at clients that are in a file (hosts.txt) and see if calculator is running and kill it or them but it keeps say there are not instances found.
#echo off
cd \
for /F "tokens=1-2" %%a in ("hosts.txt") do (
wmic /node:%%a process where name='Calculator.exe' call terminate
)
What would be a better way of doing this or what am I missing?
My comment as an answer, (one line only):
#WMIC /Node:#hosts.txt Process Where "Name='calc.exe'" Call Terminate
The answer assumes you have privileges to do so, you may need to include /user:privilegedusername and /password:privilegeduserpassword.

What's the best way to stop a batch script after a specified amount of time?

I'm writing a batch script where I want the user to be able to control how long the script runs. When running it from the command line, the user will pass in a switch like this:
./myscript --stop-after 30
which means that the script will keep doing its job, and check every iteration how much time has passed. If more than a half a minute has passed, then it'll quit. How would I implement this in a batch script?
For reference, here is the code I have so far:
:parseArgs
if "%~1" == "" goto doneParsing
if /i "%~1" == "--stop-after" (
shift
set "duration=%~1"
)
:: Parse some other options...
shift
goto parseArgs
:doneParsing
:: Now do the actual work (psuedocode)
set "start=getCurrentTime"
set "end=%start% + %duration%"
while getCurrentTime < %end% (
:: Do some lengthy task...
)
How would I go about implementing the latter part of the script, after parsing the options?
Thanks for helping.
This is not that trivial. You'll have to do a lot of calculation within your script to cover all cases of full minute, full hour, or even new day. I can think of two different ways. Both are based on two batch files:
1. Termination via taskkill
starter.bat:
#echo off
if "%1"=="" (
set duration=5
) else (
set duration=%1
)
start "myscript" script.bat
ping 127.0.0.1 -n %duration% -w 1000 > nul
echo %duration% seconds are over. Terminating!
taskkill /FI "WINDOWTITLE eq myscript*"
pause
script.bat:
#echo off
:STARTLOOP
echo doing work
ping 127.0.0.1 -n 2 -w 1000 > nul
goto STARTLOOP
For this solution, it's important that you give the window executing your script a unique name inside the line start "myscript" script.bat. In this example, the name is myscript. taskkill /FI "WINDOWTITLE eq myscript*" uses myscript to identify which process to terminate.
However, this might be a bit dangerous. Your script will be killed after x seconds, no matter if an iteration is done or not. So, e.g., write access would be a bad idea.
2. Termination via flag file
starter.bat:
#echo off
if "%1"=="" (
set duration=5
) else (
set duration=%1
)
if exist terminationflag.tmp del terminationflag.tmp
start script.bat
ping 127.0.0.1 -n %duration% -w 1000 > nul
echo %duration% seconds are over. Setting termination flag!
type NUL>terminationflag.tmp
script.bat:
#echo off
:STARTLOOP
echo doing work
ping 127.0.0.1 -n 2 -w 1000 > nul
if not exist terminationflag.tmp goto STARTLOOP
del terminationflag.tmp
echo terminated!
Here, it's important to ensure that your script is allowed to create/delete a file at the current location. This solution is safer. The starter script will wait the given amount of time and then create the flag file. Your script will check after each full iteration whether the flag is there or not. If it's not, it will go on—if it is, it will delete the flag file and terminate safely.
In both solutions ping is used as timeout function. You could also use timeout/t <TimeoutInSeconds> if you are on Windows 2000 or later. However, timeout doesn't always work. It will fail in some scheduled tasks, on build servers, and many other cases. You'd be well advised to stick to ping.

Running / killing executable using .bat file

Batch file scripting is brand new to me, so please be patient with me...
In order to run many (e.g. 5076) calculations on a windows (10, 64-bit) environment, I use the .bat file:
gulp.exe < input-1.dat > output-1.out
gulp.exe < input-2.dat > output-2.out
gulp.exe < input-3.dat > output-3.out
.
.
.
gulp.exe < input-5076.dat > output-5076.out
Unfortunately, some of the calculations will hang (for unknown reasons)...currently, when this occurs, I manually kill the whole batch command...which, means I have to keep an eye on how the calculations are progressing and cannot simply leave them running (e.g. overnight)
Thus, I am looking for a way of automatically killing the gulp.exe executable if it has been running for 30mins, but in a way that will mean the next calculation in my list still runs.
i.e. if calculation 2 hangs, kill it and run calculation 3
A search [1, 2]
indicates that taskkill might be the command I am looking for, but am a little confused as to whether it kills the .bat script or the executable and how to apply it to more than 1 calculation in the list.
Thus, I would really appreciate some pointers...
Thanks
The calculations consume CPU time so you can analyze it in 1 second samples with built-in typeperf, assuming there can only be one GULP process running at a time.
#echo off
set INTERVAL=1
set CPUTHRESHOLD=0.1
call :runGulp "input-1.dat" "output-1.out"
call :runGulp "input-2.dat" "output-2.out"
call :runGulp "input-3.dat" "output-3.out"
pause
exit /b
:runGulp
start /b gulp <"%~1" >"%~2"
:wait
for /f "delims=, tokens=2 skip=2" %%a in ('
typeperf "\Process(GULP)\%% Processor Time" -si %INTERVAL% -sc 1
') do (
if %%~a GEQ %CPUTHRESHOLD% goto wait
if %%~a==-1 exit /b
)
taskkill /f /im gulp.exe
exit /b

how to send each iteration of a loop in a batch script to a new cmd window and continue with loop

Disclaimer: I'm an engineer not a programmer, so while I do have technical knowledge, please bear with me if I am using the wrong terminology or asking the wrong questions.
I am trying to write a windows batch script that will allow me to submit multiple finite element simulations as a batch with all of the same settings. I currently have a script that works after a fashion, but it is not as efficient as I would like. My current script steps through the directories and runs the simulation in the command window before moving on to the next directory and repeating the loop. This script can be seen below:
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
title BEST FRM Runs batch submission
echo This script will step through the run# directories and run the blast.k file in each run# folder. The .cmd file needs to be placed in the "FRMRuns" directory. You may need to change the reference to the LSDYNA solver if the version has changed.
SET /P ANSWER=Do you want to continue (y/n)?
if /i {%ANSWER%}=={y} (goto :yes)
if /i {%ANSWER%}=={n} (goto :no)
:yes
SET /P i=Enter input file name (e.g blast.k)
SET /P n=Enter number of CPUs (e.g. 2)
SET /P m=Enter memory (e.g 500m)
FOR /D %%D IN (run*) DO (
echo %%D
cd %%D
set lstc_license=network
set lstc_license_server=###.##.##.##
::the solver reference may need to be changed as we move on to new versions
c:\LSDYNA\program\ls971_s_R5.1.1_winx64_p.exe i=%i% ncpu=%n% memory=%m%
cd ..
)
exit /b
:no
exit /b
Our network licensing for LSDYNA allows for queuing of jobs, so ideally I would like to run through the entire loop and have the jobs run simultaneously rather than run one after another. I think this would be possible if I could send each iteration of the loop to a new command window and have it execute independently while the loop in batch script continues.
I'm not sure if I am searching for the wrong things, or if this is a unique request, but I have not been able to find anything that really helps with what I am trying to do. I have tried various things using start /b cmd /k, but I have not been able to pass the loop commands to the new window and have the loop continue in the original window. I have managed to get the new window to open, but not to actually execute any commands, and the code does not continue until the new window is closed.
I would appreciate any suggestions that you might have for how to accomplish my goal.
Thank you!
This starts each command in it's own process, with the start "" at the beginning.
start "" c:\LSDYNA\program\ls971_s_R5.1.1_winx64_p.exe i=%i% ncpu=%n% memory=%m%

Possible to launch multiple threads of commands in cmd?

I have about 290 files that I need to optimize in a short period of time.
When I do optipng *.png it takes about 10 minutes to complete the transaction.
However when I do optipng a*.png and optipng m*.png in two separate command line it gets the work done in 5 minutes.
Now is there a way I can launch about 20 processes at the same time which will get the work done faster and not take up all of the space on my Desktop?
I have written a batch file that executes only a maximum number of commands a while ago: Parallel execution of shell processes:
#echo off
for /l %%i in (1,1,20) do call :loop %%i
goto :eof
:loop
call :checkinstances
if %INSTANCES% LSS 5 (
rem just a dummy program that waits instead of doing useful stuff
rem but suffices for now
echo Starting processing instance for %1
start /min wait.exe 5 sec
goto :eof
)
rem wait a second, can be adjusted with -w (-n 2 because the first ping returns immediately;
rem otherwise just use an address that's unused and -n 1)
echo Waiting for instances to close ...
ping -n 2 ::1 >nul 2>&1
rem jump back to see whether we can spawn a new process now
goto loop
goto :eof
:checkinstances
rem this could probably be done better. But INSTANCES should contain the number of running instances afterwards.
for /f "usebackq" %%t in (`tasklist /fo csv /fi "imagename eq wait.exe"^|wc -l`) do set INSTANCES=%%t
goto :eof
It spawns a maximum of four new processes that execute in parallel and minimized. Wait time needs to be adjusted probably, depending on how much each process does and how long it is running. You probably also need to adjust the process name for which tasklist is looking if you're doing something else.
There is no way to properly count the processes that are spawned by this batch, though. One way would be to create a random number at the start of the batch (%RANDOM%) and create a helper batch that does the processing (or spawns the processing program) but which can set its window title to a parameter:
#echo off
title %1
"%2" "%3"
This would be a simple batch that sets its title to the first parameter and then runs the second parameter with the third as argument. You can then filter in tasklist by selecting only processes with the specified window title (tasklist /fi "windowtitle eq ..."). This should work fairly reliable and prevents too many false positives. Searching for cmd.exe would be a bad idea if you still have some instances running, as that limits your pool of worker processes.
You can use %NUMBER_OF_PROCESSORS% to create a sensible default of how many instances to spawn.
You can also easily adapt this to use psexec to spawn the processes remotely (but wouldn't be very viable as you have to have admin privileges on the other machine as well as provide the password in the batch). You would have to use process names for filtering then, though.
Looks like you can write a batch file and run your commands asynchronously from that file.
Running Windows batch file commands asynchronously

Resources