I am writing a WinDBG extension to debug a device driver, and need to call an external binary to debug the device's firmware. I would like to show the output of this binary in the WinDBG console.
My initial idea was to simply pipe the output of the binary to a buffer and print that buffer with ControlledOutput. However, I get a 'broken pipe' error when I try to read from the pipe in my extension.
Here is how I create the external process in my extension:
SECURITY_ATTRIBUTES sAttr;
HANDLE childOutRead = NULL;
HANDLE childOutWrite = NULL;
PROCESS_INFORMATION childProcInfo;
STARTUPINFO childStartInfo;
char buf[4096];
sAttr.nLength = sizeof(SECURITY_ATTRIBUTES);
sAttr.bInheritHandle = TRUE;
sAttr.lpSecurityDescriptor = NULL;
CreatePipe(&childOutRead, &childOutWrite, &sAttr, 0);
// don't inherit read end
SetHandleInformation(childOutRead, HANDLE_FLAG_INHERIT, 0);
ZeroMemory(&childProcInfo, sizeof(PROCESS_INFORMATION));
ZeroMemory(&childStartInfo, sizeof(STARTUPINFO));
childStartInfo.cb = sizeof(STARTUPINFO);
childStartInfo.hStdError = childOutWrite;
childStartInfo.hStdOut = childOutWrite;
childStartInfo.hStdIn = GetStdHandle(STD_INPUT_HANDLE);
childStartInfo.dwFlags |= STARTF_USESTDHANDLES;
CreateProcessA(NULL, "myBinary.exe someArgs",
NULL, NULL, TRUE, 0, NULL, NULL,
&childStartInfo, &childProcInfo);
// close the handle not used in parent
CloseHandle(childOutWrite);
// read output
while (1) {
DWORD read;
BOOL r;
DWORD error;
r = ReadFile(childOutRead, buf, sizeof(buf), &read, NULL);
if (!r) {
error = GetLastError();
windbgPrintf("got error 0x%x\n", error);
break;
}
if (read == 0) break;
windbgPrint(buf, read);
}
ReadFile fails with error 0x6D, BROKEN_PIPE. This makes me suspect that the pipe is somehow not being inherited.
I have nearly identical code working in a test outside of WinDBG, so it must be doing something differently. How do I get pipes working in this way inside WinDBG?
Related
I am working on my library which needs to capture and process the standard output (and err) of a child process as it runs. The problem arises when ReadFile is used to read the output, it does not return once the process ends (gets killed or exits).
It looks like ReadFile is not able to detect that the other end of the pipe (the write handle) is closed. According to the documentation it should return FALSE and set the last error to ERROR_BROKEN_PIPE:
If an anonymous pipe is being used and the write handle has been closed, when ReadFile attempts to read using the pipe's corresponding read handle, the function returns FALSE and GetLastError returns ERROR_BROKEN_PIPE.
Here is my code, I have stripped out the irrelevant bits: (NOTE: I have updated the allium_start to follow the suggested changes, I am keeping the original for reference, please use the newer function code to find flaws)
bool allium_start(struct TorInstance *instance, char *config, allium_pipe *output_pipes) {
// Prepare startup info with appropriate information
SecureZeroMemory(&instance->startup_info, sizeof instance->startup_info);
instance->startup_info.dwFlags = STARTF_USESTDHANDLES;
SECURITY_ATTRIBUTES pipe_secu_attribs = {sizeof(SECURITY_ATTRIBUTES), NULL, true};
HANDLE pipes[2];
if (output_pipes == NULL) {
CreatePipe(&pipes[0], &pipes[1], &pipe_secu_attribs, 0);
output_pipes = pipes;
}
instance->startup_info.hStdOutput = output_pipes[1];
instance->startup_info.hStdError = output_pipes[1];
instance->stdout_pipe = output_pipes[0]; // Stored for internal reference
// Create the process
bool success = CreateProcessA(
NULL,
cmd,
NULL,
NULL,
config ? true : false,
0,
NULL,
NULL,
&instance->startup_info,
SecureZeroMemory(&instance->process, sizeof instance->process)
);
// Return on failure
if (!success) return false;
}
char *allium_read_stdout_line(struct TorInstance *instance) {
char *buffer = instance->buffer.data;
// Process the input
unsigned int read_len = 0;
while (true) {
// Read data
unsigned long bytes_read;
if (ReadFile(instance->stdout_pipe, buffer, 1, &bytes_read, NULL) == false || bytes_read == 0) return NULL;
// Check if we have reached end of line
if (buffer[0] == '\n') break;
// Proceed to the next character
++buffer; ++read_len;
}
// Terminate the new line with null character and return
// Special handling for Windows, terminate at CR if present
buffer[read_len >= 2 && buffer[-1] == '\r' ? -1 : 0] = '\0';
return instance->buffer.data;
}
The allium_start creates the pipe for output redirection (it uses the same pipe for both stdout and stderr to get merged streams) and then creates the child process. The other allium_read_stdout_line function is responsible for reading the output from the pipe and returning it when it encounters a new line.
The issue occurs at the ReadFile function call, it never returns if there is nothing to read after the process exits, from my understanding all the handles of a process are closed by Windows when it ends, so it looks like ReadFile is not able to detect the fact that the pipe (write handle) at the other end has been closed.
How do I fix this? I have been searching for a solution but I have found none so far, one potential option is to use multi-threading and put ReadFile in a separate thread so that it doesn't block the whole program, by using that method I can check if the process still exists periodically while I wait for the reading to finish... or kill/stop the thread if the process is gone.
I do prefer fixing the issue instead of opting for a workaround, but I am open to any other solutions to make it work. Thanks in advance!
Edit: After reading #RemyLebeau's answer and #RbMm's comments in that answer, it is pretty clear that my understand of how handle inheritance works is fundamentally flawed. So I incorporated their suggestions (SetHandleInformation to disable inheritance of read handle and closing it after creating the child process) into my allium_start function:
bool allium_start(struct TorInstance *instance, char *config, allium_pipe *output_pipes) {
// Prepare startup info with appropriate information
SecureZeroMemory(&instance->startup_info, sizeof instance->startup_info);
instance->startup_info.dwFlags = STARTF_USESTDHANDLES;
SECURITY_ATTRIBUTES pipe_secu_attribs = {sizeof(SECURITY_ATTRIBUTES), NULL, true};
HANDLE pipes[2];
if (output_pipes == NULL) {
CreatePipe(&pipes[0], &pipes[1], &pipe_secu_attribs, 0);
output_pipes = pipes;
}
SetHandleInformation(output_pipes[0], HANDLE_FLAG_INHERIT, 0);
instance->startup_info.hStdOutput = output_pipes[1];
instance->startup_info.hStdError = output_pipes[1];
instance->stdout_pipe = output_pipes[0]; // Stored for internal reference
// Create the process
bool success = CreateProcessA(
NULL,
cmd,
NULL,
NULL,
config ? true : false,
0,
NULL,
NULL,
&instance->startup_info,
SecureZeroMemory(&instance->process, sizeof instance->process)
);
// Close the write end of our stdout handle
CloseHandle(output_pipes[1]);
// Return on failure
if (!success) return false;
}
(The below text was originally here before edit 2)
But sadly it still doesn't work :(
Edit 2 (after accepting answer): It does work! See my last comment on the accepted answer.
You are not managing your pipes correctly, or more specifically, you are not controlling the inheritance of your pipe handles. DO NOT let the child process inherit the reading handle of your pipe (output_pipes[0]), otherwise the pipe will not break correctly when the child process ends.
Read MSDN for more details:
Creating a Child Process with Redirected Input and Output
The case of the redirected standard handles that won’t close even though the child process has exited
Use SetHandleInformation() or PROC_THREAD_ATTRIBUTE_LIST to prevent CreateProcess() from passing output_pipes[0] to the child process as an inheritable handle. The child process does not need access to that handle, so there is no need to pass it over the process boundary anyway. It only needs access to the writing handle of your pipe (output_pipes[1]).
For anonymous pipelines, the read process and the write process will have the handler of hRead and hWrite, each of process has its own handler(copy after inheritance). So after your child process exit and close the handler in it, anther hWrite still in parent process. We must pay attention to close hRead in the write process, close hWrite in the read process.
I can reproduce this ReadFile issue, and if closing write handler after setting child's hStdOutput and hStdError, the ReadFile will return 0 after the child process exit.
Here is my code sample,
Parent.cpp:
#include <windows.h>
#include <iostream>
#include <stdio.h>
HANDLE childInRead = NULL;
HANDLE W1 = NULL;
HANDLE W2 = NULL;
HANDLE R2 = NULL;
HANDLE R1 = NULL;
#define BUFSIZE 4096
void CreateChildProcess() {
TCHAR applicationName[] = TEXT("kids.exe");
PROCESS_INFORMATION pi;
STARTUPINFO si;
BOOL success = FALSE;
ZeroMemory(&pi, sizeof(PROCESS_INFORMATION));
ZeroMemory(&si, sizeof(STARTUPINFO));
si.cb = sizeof(STARTUPINFO);
si.hStdError = W1;
si.hStdOutput = W1;
si.hStdInput = R2;
si.dwFlags |= STARTF_USESTDHANDLES;
success = CreateProcess(NULL, applicationName, NULL, NULL, TRUE, CREATE_NEW_CONSOLE, NULL, NULL, &si, &pi);
if (!success) {
printf("Error creating child process \n");
}
else {
printf("Child process successfuly created \n");
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
}
}
int main()
{
printf("Parent process running.... \n");
DWORD dRead, dWritten;
CHAR chBuf[BUFSIZE] = { 0 };
BOOL bSuccess = FALSE;
SECURITY_ATTRIBUTES secAttr;
secAttr.nLength = sizeof(SECURITY_ATTRIBUTES);
secAttr.bInheritHandle = TRUE;
secAttr.lpSecurityDescriptor = NULL;
printf("Creating first pipe \n");
if (!CreatePipe(&R1, &W1, &secAttr, 0)) {
printf("\n error creating first pipe \n");
}
printf("Creating second pipe \n");
if (!CreatePipe(&R2, &W2, &secAttr, 0)) {
printf("\n error creating second pipe \n");
}
if (!SetHandleInformation(R1, HANDLE_FLAG_INHERIT, 0)) {
printf("\n R1 SetHandleInformation \n");
}
if (!SetHandleInformation(W2, HANDLE_FLAG_INHERIT, 0)) {
printf("\n W1 SetHandleInformation \n");
}
printf("\n Creating child process..... \n");
HANDLE hStdOut = GetStdHandle(STD_OUTPUT_HANDLE);
HANDLE hStdIn = GetStdHandle(STD_INPUT_HANDLE);
CreateChildProcess();
CloseHandle(W1);
CloseHandle(R2);
for (;;) {
printf("Inside for loop \n");
//1. read from stdin
printf("read from stdin:\n");
bSuccess = ReadFile(hStdIn, chBuf, BUFSIZE, &dRead, NULL);
if (!bSuccess) {
printf("error reading \n");
break;
}
//2. write to Pipe2
printf("write to Pipe2...\n");
bSuccess = WriteFile(W2, chBuf, 100, &dWritten, NULL);
if (!bSuccess) {
printf("error reading \n");
break;
}
//3. read from Pipe1
printf("read from Pipe1...\n");
bSuccess = ReadFile(R1, chBuf, BUFSIZE, &dRead, NULL);
if (!bSuccess)
{
printf("error reading :%d \n", GetLastError());
break;
}
//4. write to stdout
printf("write to stdout:\n");
bSuccess = WriteFile(hStdOut, chBuf, 100, &dWritten, NULL);
if (!bSuccess) {
printf("error reading \n");
break;
}
}
getchar();
return 0;
}
Kids.cpp:
#include <windows.h>
#include <stdio.h>
#define BUFSIZE 4096
int main()
{
DWORD dRead, dWritten;
CHAR chBuf[BUFSIZE];
BOOL success = FALSE;
HANDLE stdIn = GetStdHandle(STD_INPUT_HANDLE);
HANDLE stdOut = GetStdHandle(STD_OUTPUT_HANDLE);
printf("Child process running....");
if (stdIn == INVALID_HANDLE_VALUE || stdOut == INVALID_HANDLE_VALUE) {
ExitProcess(1);
}
//for (;;) {
success = ReadFile(stdIn, chBuf, BUFSIZE, &dRead, NULL);
//if (!success || dRead == 0) break;
success = WriteFile(stdOut, chBuf, dRead, &dWritten, NULL);
//if (!success) break;
//}
return 0;
}
I'm able to create a child process and pipe its stdin and stdout. It works fine when it is in text mode.
However, when I try to set the I/O in the child process to be in the binary format (i.e. no 0x0A to 0x0D 0x0A translation) the child process fails. _setmode returns -1 which has been documented to indicate failure. Why is that and how can it be fixed?
Parent code resembles the following:
const std::string path; // = "path_to.exe"
PROCESS_INFORMATION info;
SECURITY_ATTRIBUTES sec_attr;
//in-out from the CHILD process' perspective
HANDLE out_r = nullptr;
HANDLE out_w = nullptr;
HANDLE in_r = nullptr;
HANDLE in_w = nullptr;
sec_attr.nLength = sizeof(SECURITY_ATTRIBUTES);
sec_attr.lpSecurityDescriptor = NULL;
sec_attr.bInheritHandle = TRUE; //inherit by child processes
if (!CreatePipe(&out_r, &out_w, &sec_attr, 0))
throw std::exception();
if (!SetHandleInformation(out_r, HANDLE_FLAG_INHERIT, 0))
throw std::exception();
if (!CreatePipe(&in_r, &in_w, &sec_attr, 0))
throw std::exception();
if (!SetHandleInformation(in_r, HANDLE_FLAG_INHERIT, 0))
throw std::exception();
if (out_r && out_w && in_r && in_w)
{
startup_info.hStdError = out_w;
startup_info.hStdOutput = out_w;
startup_info.hStdInput = in_r;
startup_info.dwFlags = STARTF_USESTDHANDLES;
}
if (CreateProcessA(path.c_str(), (char*)cmd, NULL, NULL, TRUE, 0, NULL, NULL, &startup_info, &info)
!= TRUE)
{
DWORD error = GetLastError();
//error handling
}
// ... read data using ReadFile
Child code resembles the following:
int status = _setmode(_fileno(stdin), _O_BINARY);
if (status == -1)
throw std::exception();
status = _setmode(_fileno(stdout), _O_BINARY);
if (status == -1)
throw std::exception();
puts("hello from the child process");
I'm looking at the MSDN article Creating a Child Process with Redirected Input and Output that provides a reference implementation for nearly what you're trying to do here.
One subtle difference I see is, where in your parent code is written (the second call to SetHandleInformation()):
if (!SetHandleInformation(in_r, HANDLE_FLAG_INHERIT, 0))
the MSDN example (translated to use your variable names) would be written:
if (!SetHandleInformation(in_w, HANDLE_FLAG_INHERIT, 0))
Here are a couple more SO pages that would be worth looking at as well, if this small change doesn't help:
What is the simplest way to write to stdout in binary mode?
Win32 changing to binary mode child's Stdout (pipe)
I'm learning how to use in Winapi
And I'm trying to read a file from My Computer
But for some reason it doesn't work ...
HANDLE hFile;
//PVOID First_Bytes[2048];
char First_Bytes[2048];
DWORD dbr = 0;
hFile = CreateFile(L"d:\\My-File",GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL , NULL);
if (hFile == INVALID_HANDLE_VALUE) {
printf("Error %x", GetLastError());
return 1;
}
if (ReadFile(hFile, &First_Bytes, 512, &dbr, NULL) == 0) {
printf("ReadFile error: %x", GetLastError());
return 1;
}
printf("%s", First_Bytes);
CloseHandle(hFile);
The console doesn't print anything.
What am I doing wrong?
I edited the code and add that errors checks.
But still consul does not print anything
The logical conclusion is that the first byte in your file is a zero. You treat the buffer as a null-terminated string, and so nothing is printed.
Do note that there is no guarantee that your buffer is null terminated so you potentially have undefined behaviour.
I am creating a child process, and reading its output. My code works fine when the child process creates output (cmd /c echo Hello World), however ReadFile will hang if process does not create output (cmd /c echo Hello World > output.txt). I am only reading after the process has terminated.
Am I doing something horribly wrong? Is there anyway to do this with synchronous mode, or do I have to use asynchronous mode? All of this is happening in a seperate thread, so I dont think asynchronous mode would offer any benefit to me, unless it is the only way to get this to work. Thanks a lot!
saAttr.nLength = sizeof(SECURITY_ATTRIBUTES);
saAttr.bInheritHandle = TRUE;
saAttr.lpSecurityDescriptor = NULL;
CreatePipe(&g_hChildStd_OUT_Rd, &g_hChildStd_OUT_Wr, &saAttr, 0);
SetHandleInformation(g_hChildStd_OUT_Rd, HANDLE_FLAG_INHERIT, 0);
memset(&piProcInfo, 0, sizeof(PROCESS_INFORMATION));
memset(&siStartInfo, 0, sizeof(STARTUPINFO));
siStartInfo.cb = sizeof(STARTUPINFO);
siStartInfo.hStdError = g_hChildStd_OUT_Wr;
siStartInfo.hStdOutput = g_hChildStd_OUT_Wr;
siStartInfo.dwFlags |= STARTF_USESTDHANDLES;
CreateProcess(NULL, commandWideString, NULL, NULL, TRUE, 0, NULL, NULL, &siStartInfo, &piProcInfo);
while(1)
{
GetExitCodeProcess(piProcInfo.hProcess, &processExitCode);
if(processExitCode != STILL_ACTIVE)
break;
else
Sleep(1);
}
*output = (char *)calloc(32, sizeof(char));
processOutputSize = 0;
while(1)
{
bSuccess = ReadFile( g_hChildStd_OUT_Rd, processOutputTemp, 32, &dwRead, NULL);
if(!bSuccess || !dwRead)
break;
memcpy(*output + processOutputSize, processOutputTemp, dwRead);
processOutputSize += dwRead;
if(dwRead == 32)
*output = (char *)realloc(*output, processOutputSize + 32);
else
{
memset(*output + processOutputSize, 0, 1);
break;
}
}
CloseHandle(piProcInfo.hProcess);
CloseHandle(piProcInfo.hThread);
CloseHandle(g_hChildStd_OUT_Rd);
CloseHandle(g_hChildStd_OUT_Wr);
You should close the write end of the output pipe before you read from it, as #Marcus suggested in the comment.
CloseHandle(g_hChildStd_OUT_Wr);
For me this is the real answer.
You redirect the output of the process to a pipe, start the process, wait till it exits, and then read the output.
The problem is that windows buffers just a limited amount of data. So you must read the pipe while the process is still running, otherwise the process will be blocked because it cannot write any more data to the pipe.
You could use PeekNamedPipe in a loop like this:
for (;;)
{
DWORD bytesAvail = 0;
if (!PeekNamedPipe(stdoutPipeRead, NULL, 0, NULL, &bytesAvail, NULL)) {
std::cout << "Failed to call PeekNamedPipe" << std::endl;
}
if (bytesAvail) {
CHAR buf[BUFSIZE];
DWORD n;
BOOL success = ReadFile(stdoutPipeRead, buf, BUFSIZE, &n, NULL);
if (!success || n == 0) {
std::cout << "Failed to call ReadFile" << std::endl;
}
std::cout << std::string(buf, buf + n);
}
}
I am trying to get a list of all the executable paths of running processes
The do-while loop (shown below) starts off and after about 90 something iterations it fails with a ERROR_INSUFFICIENT_BUFFER error. I presume thats the pBuffer, I tried with a very large buffer and it still failed. The ProcessEntry struct on the failed iteration has garbage in szExeFile. Please advise (I close the handles, not shown below)
Code:
// Retrieve a handle to the process snapshot
HANDLE hProcessSnapshot(CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0));
if (INVALID_HANDLE_VALUE == hProcessSnapshot) ERROR;
// Retrieve information about the first process and exit if unsuccessful
PROCESSENTRY32 oProcessEntry;
memset(&oProcessEntry, 0x00, sizeof(oProcessEntry));
oProcessEntry.dwSize = sizeof(PROCESSENTRY32);
BOOL bFileFound(Process32First(hProcessSnapshot, &oProcessEntry));
DWORD dwError;
if(!bFileFound) {
dwError = GetLastError();
if(ERROR_NO_MORE_FILES == dwError) return TPathList();
// Error
ERROR;
}
// Walk the snapshot of processes
TCHAR pBuffer[MAX_PATH];
TPathList lExecutablePaths;
do {
// Get handle to process
HANDLE hProcess(OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ,
FALSE,
oProcessEntry.th32ProcessID));
if(INVALID_HANDLE_VALUE == hProcess) ERROR;
if (!hProcess) continue;
// Get the module path
if(GetModuleFileNameEx(hProcess, 0, pBuffer, MAX_PATH) == 0) ERROR;
lExecutablePaths.push_back(CPath(pBuffer));
}
// Get next process
while(Process32Next(hProcessSnapshot, &oProcessEntry));
// If we ran out of files return what has been found
dwError = GetLastError();
if(ERROR_NO_MORE_FILES == dwError) return lExecutablePaths;
ERROR;
You can use like this:
CString sBuffer;
DWORD dwSize = MAX_PATH + 1, dwError = 0;
do
{
GetModuleFileName( NULL, sBuffer.GetBuffer( dwSize ), dwSize );
// Retrieve the last error. If we've succeeded ERROR_SUCCESS
// will be returned; otherwise, we'll get an ERROR_INSUFFICIENT_BUFFER
// error.
dwError = ::GetLastError( );
// Buffer may not be big enough so double its size
dwSize *= 2;
}
while( dwError == ERROR_INSUFFICIENT_BUFFER
&& dwError != ERROR_SUCCESS );
// Release the buffer (turn the string back to const)
sBuffer.ReleaseBuffer( );
I hope it will work.
Also you can prefer How can I calculate the complete buffer size for GetModuleFileName?