Issue with output redirection in batch - cmd

Question

I have a script a.cmd that calls another script b.cmd, and redirects its output. the called script, starts an executable that is never terminated. The output of the executable is redirected to its own log file. Simplified code:
a.cmd:
[1] #ECHO OFF
[2] SET LOG_FILE_NAME="log.txt"
[3] REM Start the b.cmd redirecting all output
[4] CALL b.cmd >> %LOG_FILE_NAME% 2>&1
[5] ECHO returned to a.cmd >> %LOG_FILE_NAME% 2>&1
[6] EXIT /B 0
b.cmd:
[1] #ECHO OFF
[2] SET ANOTHER_LOG_FILE_NAME="log2.txt"
[4] ECHO RunForEver.exe redirecting all output
[5] START CMD /C "RunForEver.exe >> %ANOTHER_LOG_FILE_NAME% 2>&1"
[6] ECHO b.cmd execution complete
[7] EXIT /B 0
(Line numbers were added for convenience)
The problem I'm encountering is that line 4 in b.cmd seems to grab a handle on the initial log file (LOG_FILE_NAME) because all b.cmd output is redirected to it, and the handle is not released while the executable (and the cmd that launched it) are running.
I didn't except this behavior because I thought only the output of the start command itself will be redirected to the LOG_FILE_NAME log file, and the output from the other process that is actually running the RunForEver.exe executable will be written to the ANOTHER_LOG_FILE_NAME.
As a result, line 5 in a.cmd errors out with access denied to LOG_FILE_NAME.
Could someone explain what's going on? Is there a way to avoid this?
I tried doing the output redirection to LOG_FILE_NAME from inside b.cmd, but then I get the access denied error in line 2 of b.cmd.
Thanks in advance!

Answer 1

Wow! That is a fascinating and disturbing discovery.
I don't have an explanation, but I do have a solution.
Simply avoid any additional redirection to log.txt after the never ending process has started. That can be done by redirecting a parenthesized block of code just once.
#ECHO OFF
SET LOG_FILE_NAME="log.txt"
>>%LOG_FILE_NAME% 2>&1 (
CALL b.cmd
ECHO returned to a.cmd
)
EXIT /B 0
Or by redirecting the output of a CALLed subroutine instead.
#ECHO OFF
SET LOG_FILE_NAME="log.txt"
call :redirected >>%LOG_FILE_NAME% 2>&1
EXIT /B 0
:redirected
CALL b.cmd
ECHO returned to a.cmd
exit /b
If you need to selectively redirect output in a.cmd, then redirect a non-standard stream to your file just once, and then within the block, selectively redirect output to the non-standard stream.
#ECHO OFF
SET LOG_FILE_NAME="log.txt"
3>>%LOG_FILE_NAME% (
echo normal output that is not redirected
CALL b.cmd >&3 2>&1
ECHO returned to a.cmd >&3 2>&1
)
EXIT /B 0
Again, the same technique could be done using a CALL instead of a parenthesized block.
I've developed a simple, self contained TEST.BAT script that anyone can run to demonstrate the problem. I called it TEST.BAT on my machine.
#echo off
del log*.txt 2>nul
echo begin >>LOG1.TXT 2>&1
call :test >>LOG1.TXT 2>&1
echo end >>LOG1.TXT 2>&1
exit /b
:test
echo before start
>nul 2>&1 (
echo ignored output
start "" cmd /c "echo start result >LOG2.TXT 2>&1 & pause >con"
)
echo after start
pause >con
exit /b
Both the master and the STARTed process are paused, thus allowing me to choose which process finishes first. If the STARTed process terminates before the master, then everything works as expected, as evidenced by the following output from the main console window.
C:\test>test
Press any key to continue . . .
C:\test>type log*
LOG1.TXT
begin
before start
after start
end
LOG2.TXT
start result
C:\test>
Here is an example of what happens if I allow the main process to continue before the STARTed process terminates:
C:\test>test
Press any key to continue . . .
The process cannot access the file because it is being used by another process.
C:\test>type log*
LOG1.TXT
begin
before start
after start
LOG2.TXT
start result
C:\test>
The reason I find the behavior disturbing is that I can't fathom how the STARTed process has any relationship with LOG1.TXT. By the time the START command executes, all standard output has been redirected to nul, so I don't understand how the new process knows about LOG1.TXT, let alone how it establishes an exclusive lock on it. The fact that echo ignored output has no detectable output is proof that the standard output has been successfully redirected to nul.