Running a cmd .bat file with loops and conditions - windows

Question

Hi I am trying to run a scrapy spider using start command from a cmd batch file. I want to run the spider parallely for 10 names from a csv file which contains more than 500 names. So my thought is to basically add some conditions in the loop from 1 to 500 that checks if the 10 command windows have closed, if not then wait for them to close (they auto close after spider is finished). Once the 10 windows which had opened, are closed, open the next 10 and so on. Following is the code I have, i am pretty sure it has big syntax errors. Could you help me debug? Thanks
cd /d "C:\Users\xyz"
for /f "tokens=1,*" %%m in ('tasklist ^| find /I /C "conhost.exe"') do (set var1=%%m)
set counter=1
for /f "usebackq tokens=1 delims=," %%n in ("test.csv") do (
(START /MIN "" scrapy crawl xyz_scraper -a query="%%n" -a pages=20)
set /a counter=counter+1
for /f "tokens=1,*" %%p in ('tasklist ^| find /I /C "conhost.exe"') do (set var2=%%p)
SET /A _result=counter%%10
echo %_result%
IF _result EQU 0 (
:abcd
timeout /t 10
if var2 EQU var1 (
goto bcde
)
ELSE (
goto abcd)
)
:bcde
)
pause
EDIT: deleted the for loop one. Edited the above code based on some suggestions. I don't understand where would i use quotes for variables and where not and how to print a variables value to cmd.
The updated code below:
cd /d "C:\Users\sodhian\sodhi-scraper"
for /f "tokens=1,*" %%m in ('tasklist ^| find /I /C "conhost.exe"') do (set var1=%%m)
echo %var1%
set counter=1
for /f "usebackq tokens=1 delims=," %%n in ("test comp.csv") do (
(START /MIN "" scrapy crawl ind_scraper -a query="%%n" -a pages=20)
set /a "counter=counter+1"
echo %counter%
SET /A _result="counter%%2"
echo %_result%
IF "%_result%" EQU "0" (
:abcd
timeout /t 10
for /f "tokens=1,*" %%p in ('tasklist ^| find /I /C "conhost.exe"') do (set var2=%%p)
echo %var2%
if var2==var1 (
goto bcde
)
ELSE (
goto abcd)
)
:bcde
)
pause
Edit 2:
Based on Stephan's answer. Tried to accomplish what i mentioned in comment of the answer:
setlocal enabledelayedexpansion
set counter=0
for /f "usebackq tokens=1 delims=," %%n in ("test comp.csv") do (
set /a counter+=1
start /MIN "MySpider!counter!" scrapy crawl ind_scraper -a query="%%n" -a pages=20
for /f "tokens=1,*" %%b in ('tasklist /v ^| find /I /C "MySpider"') do (set var1=%%b)
if !var1! geq 5 call :wait
)
:wait
timeout /t 5
for /f "tokens=1,*" %%p in ('tasklist /v ^| find /I /C "MySpider"') do (set var2=%%p)
if !var2! geq 5 call :wait
goto :eof
Changed it to the following: (the /v (verbose option in tasklist was making the above slow)
setlocal enabledelayedexpansion
set counter=0
set max_scrappers=7
for /f "tokens=1,*" %%a in ('tasklist ^| find /C "conhost"') do (set var1=%%a)
for /f "usebackq tokens=1 delims=," %%n in ("test comp.csv") do (
set /a counter+=1
start /min "MySpider!counter!" scrapy crawl ind_scraper -a query="%%n" -a pages=20
for /f "tokens=1,*" %%b in ('tasklist ^| find /C "conhost"') do (set var2=%%b)
set /a var3=!var2!-!var1!
if !var3! geq !max_scrappers! call :wait
)
:wait
for /f "tokens=1,*" %%p in ('tasklist ^| find /C "conhost"') do (set var4=%%p)
set /a var5=!var4!-!var1!
if !var5! geq !max_scrappers! call :wait
goto :eof

Answer 1

As already noted in the comments, labels inside a code block don't work. But you can call a "function", where goto and labels are no problem:
I choosed other numbers and another command to make it work on every system (and faster). Replacing the timeout command with your spider and adapting the numbers should be no problem.
#echo off
setlocal enabledelayedexpansion
REM next line just for generating a testfile:
>"test comp.csv" (for /l %%i in (1,1,10) do echo !random:~-1!)
set counter=0
for /f "usebackq tokens=1 delims=," %%n in ("test comp.csv") do (
set /a counter+=1
start "MySpider!counter!" timeout %%n
if !counter! geq 3 call :wait
)
:wait
tasklist /v|find "MySpider">nul && goto :wait
set counter=0
goto :eof
Searching for conhost is not a good idea, because there could be other processes. Choose an unique window title instead (MySpider in my example) and look for that title (tasklist /v)