Batch script read a file that's continously written - windows

Question

I have a file that's being continuously written with information like this
HU/gen/tcfg/target/jacinto/starter/starter_b1.cfg
32.77 34% 141.59kB/s 0:00:00
94.64 100% 405.35kB/s 0:00:00 (xfer#60, to-check=1002/1097)
It is the output of rsync tool, that copies a folder to another path
I'm trying to write a batch script that reads this file and calculates the total amount of data being copied, focusing on this line
94.64 100% 405.35kB/s 0:00:00 (xfer#60, to-check=1002/1097)
the number 94.64 being the file size in bytes, so I'm guessing I should extract whatever is before "100%" from the line and add it
But I don't know how to continuously read the file while it's being written at the same time
Can somebody help?
Thanks

Answer 1

Here is an example that shows how to read a file while it is being written by another process. This is very similar to what jeb shows, except I have added a generic test to see if the process is complete. It assumes the process maintains a lock on the file throughout the entire execution, and it also assumes 20 consecutive empty lines indicates either the end of file, or else we are waiting for more output from the process. The 20 empty line threshold could be set to any number that works for you.
This solution may not work properly if the process writes partial lines to the file. I believe it is reliable if the the process always writes each line in its entirety in a single operation. Also, the lines must be less than or equal to 1021 bytes long, and they must be terminated by carriage return, linefeed.
#echo off
setlocal enableDelayedExpansion
set "file=test.txt"
set emptyCount=0
del "%file%
:: For this test, I will start an asynchronous process in a new window that
:: writes a directory listing to an output file 3 times, with a 5 second pause
:: between each listing.
start "" cmd /c "(for /l %%N in (1 1 3) do #dir & ping localhost -n 6 >nul) >"%file%""
:wait for the output file to be created and locked
if not exist "%file%" goto :wait
:: Now read and process the file, as it is being written.
call :read <"%file%"
exit /b
:read
set "ln="
set /p "ln="
if defined ln (
REM Process non-empty line here - test if correct line, extract size, and add
REM I will simply echo the line instead
echo(!ln!
REM Reset emptyCount for end of file test and read next line
set /a emptyCount=0
goto :read
) else ( REM End of file test
REM Count how many consecutive empty lines have been read.
set /a emptyCount+=1
REM Assume that 20 consecutive empty lines signifies either end of file
REM or else waiting for output. If haven't reached 20 consectutive
REM empty lines, then read next line.
if !emptyCount! lss 20 goto :read
REM Test if output file is still locked by attempting to redirect an unused
REM stream to the file in append mode. If the redirect fails, then the file
REM is still locked, meaning we are waiting for the process to finish and have
REM not reached the end. So wait 1 sec so as not to consume 100% of CPU, then
REM reset count and try to read the next line again.
REM If the file is not locked, then we are finished, so simply fall through
REM and exit.
(echo off 9>>"%file%")2>nul || (
set emptyCount=0
ping localhost -n 2 >nul
goto :read
)
)
exit /b

Answer 2

Like dbenham said, it can be done, but batch isn't the first choice.
Attention, the sample will run in an endless loop.
You need to add a break condition, depending of the file content.
I add the ping ... command to avoid, that the process consumes 100% CPU-time
#echo off
setlocal disabledelayedexpansion
setlocal enableDelayedExpansion
< rsyncOutput.tmp (
call :readData
)
echo ready
exit /b
:readDataX
set "line="
set /p line=
if defined line (
for /f "tokens=1,2 delims= " %%a in ("!line!") do (
if "%%b"=="100%%" echo Bytes %%a
)
)
ping localhost -n 2 > nul
goto :readDataX
exit /b