file flush and sync with other program - windows

Question

I am running a Python script that is meant to launch a .bat file on Windows 7. The batch file runs a sort of Monte Carlo program for 2 minutes. At the end of those 2 minutes the program must update a data file that I will read later in the same Python script, do some data analysis on the output and relaunch the same .bat file if necessary.
I do this process an unlimited amount of time until I get a satisfactory output.
The Python code looks like this
import os
import numpy as np
os.system('myBatch.bat > messages.txt')
while True:
# force flushing
fd = open("output.txt",'a')
fd.flush()
os.fsync(fd.fileno())
fd.close()
# read data
data = np.loadtxt("output.txt")
# some data analysis ...
# testing
if dataSatisfactory:
break
else:
os.system('myBatch.bat > messages.txt')
As you can see I am trying to force flushing the output.txt file that must be written by the program in the batch file. As I have no control over this compiled executable program the output file is not getting updated after the command os.system('myBatch.bat > messages.txt'), but only after I kill the whole python process.
I tried to add the 4 lines after # force flushing, but apparently still not working.
For the sake of clarification, the input of the executable is the output.txt as well as the output, so the executable must over-write the file output.txt.
The file messages.txt is to redirect all the messages of the executable from the stdout to messages.txt.
Any advice or a hint?
NB: The batch file looks like this
.\TheExecutable %output.txt

Answer 1

if your batch file does nothing but calling your monte carlo program, get rid of it and call the program directly from python: os.system("TheExecutable.exe output.txt")
If you can make your program write to standard output instead of a file, you could use subprocess.check_output("TheExecutable.exe") which just returns that standard output. See the documentation of numpy.loadtxt() for functions loading from string instead of a filename.
This way, you completely avoid writing & reading to/from disk.