How to capture process output asynchronously in powershell? - events

Question

I want to capture stdout and stderr from a process that I start in a Powershell script and display it asynchronously to the console. I've found some documentation on doing this through MSDN and other blogs.
After creating and running the example below, I can't seem to get any output to be displayed asynchronously. All of the output is only displayed when the process terminates.
$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = "cmd.exe"
$ps.StartInfo.UseShellExecute = $false
$ps.StartInfo.RedirectStandardOutput = $true
$ps.StartInfo.Arguments = "/c echo `"hi`" `& timeout 5"
$action = { Write-Host $EventArgs.Data }
Register-ObjectEvent -InputObject $ps -EventName OutputDataReceived -Action $action | Out-Null
$ps.start() | Out-Null
$ps.BeginOutputReadLine()
$ps.WaitForExit()
In this example, I was expecting to see the output of "hi" on the commandline before the end of program execution because the OutputDataReceived event should have been triggered.
I've tried this using other executables - java.exe, git.exe, etc. All of them have the same effect, so I'm left to think that there is something simple that I'm not understanding or have missed. What else needs to be done to read stdout asynchronously?

Answer 1

Unfortunately asynchronous reading is not that easy if you want to do it properly. If you call WaitForExit() without timeout you could use something like this function I wrote (based on C# code):
function Invoke-Executable {
# Runs the specified executable and captures its exit code, stdout
# and stderr.
# Returns: custom object.
param(
[Parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[String]$sExeFile,
[Parameter(Mandatory=$false)]
[String[]]$cArgs,
[Parameter(Mandatory=$false)]
[String]$sVerb
)
# Setting process invocation parameters.
$oPsi = New-Object -TypeName System.Diagnostics.ProcessStartInfo
$oPsi.CreateNoWindow = $true
$oPsi.UseShellExecute = $false
$oPsi.RedirectStandardOutput = $true
$oPsi.RedirectStandardError = $true
$oPsi.FileName = $sExeFile
if (! [String]::IsNullOrEmpty($cArgs)) {
$oPsi.Arguments = $cArgs
}
if (! [String]::IsNullOrEmpty($sVerb)) {
$oPsi.Verb = $sVerb
}
# Creating process object.
$oProcess = New-Object -TypeName System.Diagnostics.Process
$oProcess.StartInfo = $oPsi
# Creating string builders to store stdout and stderr.
$oStdOutBuilder = New-Object -TypeName System.Text.StringBuilder
$oStdErrBuilder = New-Object -TypeName System.Text.StringBuilder
# Adding event handers for stdout and stderr.
$sScripBlock = {
if (! [String]::IsNullOrEmpty($EventArgs.Data)) {
$Event.MessageData.AppendLine($EventArgs.Data)
}
}
$oStdOutEvent = Register-ObjectEvent -InputObject $oProcess `
-Action $sScripBlock -EventName 'OutputDataReceived' `
-MessageData $oStdOutBuilder
$oStdErrEvent = Register-ObjectEvent -InputObject $oProcess `
-Action $sScripBlock -EventName 'ErrorDataReceived' `
-MessageData $oStdErrBuilder
# Starting process.
[Void]$oProcess.Start()
$oProcess.BeginOutputReadLine()
$oProcess.BeginErrorReadLine()
[Void]$oProcess.WaitForExit()
# Unregistering events to retrieve process output.
Unregister-Event -SourceIdentifier $oStdOutEvent.Name
Unregister-Event -SourceIdentifier $oStdErrEvent.Name
$oResult = New-Object -TypeName PSObject -Property ([Ordered]#{
"ExeFile" = $sExeFile;
"Args" = $cArgs -join " ";
"ExitCode" = $oProcess.ExitCode;
"StdOut" = $oStdOutBuilder.ToString().Trim();
"StdErr" = $oStdErrBuilder.ToString().Trim()
})
return $oResult
}
It captures stdout, stderr and exit code. Example usage:
$oResult = Invoke-Executable -sExeFile 'ping.exe' -cArgs #('8.8.8.8', '-a')
$oResult | Format-List -Force
For more info and alternative implementations (in C#) read this blog post.

Answer 2

Based on Alexander Obersht's answer I've created a function that uses timeout and asynchronous Task classes instead of event handlers.
According to Mike Adelson
Unfortunately, this method(event handlers) provides no way to know
when the last bit of data has been received. Because everything is
asynchronous, it is possible (and I have observed this) for events to
fire after WaitForExit() has returned.
function Invoke-Executable {
# from https://stackoverflow.com/a/24371479/52277
# Runs the specified executable and captures its exit code, stdout
# and stderr.
# Returns: custom object.
# from http://www.codeducky.org/process-handling-net/ added timeout, using tasks
param(
[Parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[String]$sExeFile,
[Parameter(Mandatory=$false)]
[String[]]$cArgs,
[Parameter(Mandatory=$false)]
[String]$sVerb,
[Parameter(Mandatory=$false)]
[Int]$TimeoutMilliseconds=1800000 #30min
)
Write-Host $sExeFile $cArgs
# Setting process invocation parameters.
$oPsi = New-Object -TypeName System.Diagnostics.ProcessStartInfo
$oPsi.CreateNoWindow = $true
$oPsi.UseShellExecute = $false
$oPsi.RedirectStandardOutput = $true
$oPsi.RedirectStandardError = $true
$oPsi.FileName = $sExeFile
if (! [String]::IsNullOrEmpty($cArgs)) {
$oPsi.Arguments = $cArgs
}
if (! [String]::IsNullOrEmpty($sVerb)) {
$oPsi.Verb = $sVerb
}
# Creating process object.
$oProcess = New-Object -TypeName System.Diagnostics.Process
$oProcess.StartInfo = $oPsi
# Starting process.
[Void]$oProcess.Start()
# Tasks used based on http://www.codeducky.org/process-handling-net/
$outTask = $oProcess.StandardOutput.ReadToEndAsync();
$errTask = $oProcess.StandardError.ReadToEndAsync();
$bRet=$oProcess.WaitForExit($TimeoutMilliseconds)
if (-Not $bRet)
{
$oProcess.Kill();
# throw [System.TimeoutException] ($sExeFile + " was killed due to timeout after " + ($TimeoutMilliseconds/1000) + " sec ")
}
$outText = $outTask.Result;
$errText = $errTask.Result;
if (-Not $bRet)
{
$errText =$errText + ($sExeFile + " was killed due to timeout after " + ($TimeoutMilliseconds/1000) + " sec ")
}
$oResult = New-Object -TypeName PSObject -Property ([Ordered]#{
"ExeFile" = $sExeFile;
"Args" = $cArgs -join " ";
"ExitCode" = $oProcess.ExitCode;
"StdOut" = $outText;
"StdErr" = $errText
})
return $oResult
}

Answer 3

I couldn't get either of these examples to work with PS 4.0.
I wanted to run puppet apply from an Octopus Deploy package (via Deploy.ps1) and see the output in "real time" rather than wait for the process to finish (an hour later), so I came up with the following:
# Deploy.ps1
$procTools = #"
using System;
using System.Diagnostics;
namespace Proc.Tools
{
public static class exec
{
public static int runCommand(string executable, string args = "", string cwd = "", string verb = "runas") {
//* Create your Process
Process process = new Process();
process.StartInfo.FileName = executable;
process.StartInfo.UseShellExecute = false;
process.StartInfo.CreateNoWindow = true;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = true;
//* Optional process configuration
if (!String.IsNullOrEmpty(args)) { process.StartInfo.Arguments = args; }
if (!String.IsNullOrEmpty(cwd)) { process.StartInfo.WorkingDirectory = cwd; }
if (!String.IsNullOrEmpty(verb)) { process.StartInfo.Verb = verb; }
//* Set your output and error (asynchronous) handlers
process.OutputDataReceived += new DataReceivedEventHandler(OutputHandler);
process.ErrorDataReceived += new DataReceivedEventHandler(OutputHandler);
//* Start process and handlers
process.Start();
process.BeginOutputReadLine();
process.BeginErrorReadLine();
process.WaitForExit();
//* Return the commands exit code
return process.ExitCode;
}
public static void OutputHandler(object sendingProcess, DataReceivedEventArgs outLine) {
//* Do your stuff with the output (write to console/log/StringBuilder)
Console.WriteLine(outLine.Data);
}
}
}
"#
Add-Type -TypeDefinition $procTools -Language CSharp
$puppetApplyRc = [Proc.Tools.exec]::runCommand("ruby", "-S -- puppet apply --test --color false ./manifests/site.pp", "C:\ProgramData\PuppetLabs\code\environments\production");
if ( $puppetApplyRc -eq 0 ) {
Write-Host "The run succeeded with no changes or failures; the system was already in the desired state."
} elseif ( $puppetApplyRc -eq 1 ) {
throw "The run failed; halt"
} elseif ( $puppetApplyRc -eq 2) {
Write-Host "The run succeeded, and some resources were changed."
} elseif ( $puppetApplyRc -eq 4 ) {
Write-Warning "WARNING: The run succeeded, and some resources failed."
} elseif ( $puppetApplyRc -eq 6 ) {
Write-Warning "WARNING: The run succeeded, and included both changes and failures."
} else {
throw "Un-recognised return code RC: $puppetApplyRc"
}
Credit goes to T30 and Stefan Goßner

Answer 4

The examples here are all useful, but didn't completely suit my use case. I didn't want to invoke the command and exit. I wanted to open a command prompt, send input, read the output, and repeat. Here's my solution for that.
Create Utils.CmdManager.cs
using System;
using System.Diagnostics;
using System.Text;
using System.Threading;
namespace Utils
{
public class CmdManager : IDisposable
{
const int DEFAULT_WAIT_CHECK_TIME = 100;
const int DEFAULT_COMMAND_TIMEOUT = 3000;
public int WaitTime { get; set; }
public int CommandTimeout { get; set; }
Process _process;
StringBuilder output;
public CmdManager() : this("cmd.exe", null, null) { }
public CmdManager(string filename) : this(filename, null, null) { }
public CmdManager(string filename, string arguments) : this(filename, arguments, null) { }
public CmdManager(string filename, string arguments, string verb)
{
WaitTime = DEFAULT_WAIT_CHECK_TIME;
CommandTimeout = DEFAULT_COMMAND_TIMEOUT;
output = new StringBuilder();
_process = new Process();
_process.StartInfo.FileName = filename;
_process.StartInfo.RedirectStandardInput = true;
_process.StartInfo.RedirectStandardOutput = true;
_process.StartInfo.RedirectStandardError = true;
_process.StartInfo.CreateNoWindow = true;
_process.StartInfo.UseShellExecute = false;
_process.StartInfo.ErrorDialog = false;
_process.StartInfo.Arguments = arguments != null ? arguments : null;
_process.StartInfo.Verb = verb != null ? verb : null;
_process.EnableRaisingEvents = true;
_process.OutputDataReceived += (s, e) =>
{
lock (output)
{
output.AppendLine(e.Data);
};
};
_process.ErrorDataReceived += (s, e) =>
{
lock (output)
{
output.AppendLine(e.Data);
};
};
_process.Start();
_process.BeginOutputReadLine();
_process.BeginErrorReadLine();
_process.StandardInput.AutoFlush = true;
}
public void RunCommand(string command)
{
_process.StandardInput.WriteLine(command);
}
public string GetOutput()
{
return GetOutput(null, CommandTimeout, WaitTime);
}
public string GetOutput(string endingOutput)
{
return GetOutput(endingOutput, CommandTimeout, WaitTime);
}
public string GetOutput(string endingOutput, int commandTimeout)
{
return GetOutput(endingOutput, commandTimeout, WaitTime);
}
public string GetOutput(string endingOutput, int commandTimeout, int waitTime)
{
string tempOutput = "";
int tempOutputLength = 0;
int amountOfTimeSlept = 0;
// Loop until
// a) command timeout is reached
// b) some output is seen
while (output.ToString() == "")
{
if (amountOfTimeSlept >= commandTimeout)
{
break;
}
Thread.Sleep(waitTime);
amountOfTimeSlept += waitTime;
}
// Loop until:
// a) command timeout is reached
// b) endingOutput is found
// c) OR endingOutput is null and there is no new output for at least waitTime
while (amountOfTimeSlept < commandTimeout)
{
if (endingOutput != null && output.ToString().Contains(endingOutput))
{
break;
}
else if(endingOutput == null && tempOutputLength == output.ToString().Length)
{
break;
}
tempOutputLength = output.ToString().Length;
Thread.Sleep(waitTime);
amountOfTimeSlept += waitTime;
}
// Return the output and clear the buffer
lock (output)
{
tempOutput = output.ToString();
output.Clear();
return tempOutput.TrimEnd();
}
}
public void Dispose()
{
_process.Kill();
}
}
}
Then from PowerShell add the class and use it.
Add-Type -Path ".\Utils.CmdManager.cs"
$cmd = new-object Utils.CmdManager
$cmd.GetOutput() | Out-Null
$cmd.RunCommand("whoami")
$cmd.GetOutput()
$cmd.RunCommand("cd")
$cmd.GetOutput()
$cmd.RunCommand("dir")
$cmd.GetOutput()
$cmd.RunCommand("cd Desktop")
$cmd.GetOutput()
$cmd.RunCommand("cd")
$cmd.GetOutput()
$cmd.RunCommand("dir")
$cmd.GetOutput()
$cmd.Dispose()
Don't forget to call the Dispose() function at the end to clean up the process that is running in the background. Alternatively, you could close that process by running something like $cmd.RunCommand("exit")

Answer 5

I came here looking for a solution to create a wrapper that logs the process, and outputs it to screen. None of these worked for me. I made this code, which seemed to work fine.
The PSDataCollection allows you to continue out with your script, without having to wait for process to complete.
Using namespace System.Diagnostics;
Using namespace System.Management.Automation;
$Global:Dir = Convert-Path "."
$Global:LogPath = "$global:Dir\logs\mylog.log"
[Process]$Process = [Process]::New();
[ProcessStartInfo]$info = [ProcessStartInfo]::New();
$info.UseShellExecute = $false
$info.Verb = "runas"
$info.WorkingDirectory = "$Global:Dir\process.exe"
$info.FileName = "$Global:Dir\folder\process.exe"
$info.Arguments = "-myarg yes -another_arg no"
$info.RedirectStandardOutput = $true
$info.RedirectStandardError = $true
$Process.StartInfo = $info;
$Process.EnableRaisingEvents = $true
$Global:DataStream = [PSDataCollection[string]]::New()
$Global:DataStream.add_DataAdded(
{
$line = $this[0];
[IO.File]::AppendAllLines($LogPath, [string[]]$line);
[Console]::WriteLine($line)
$this.Remove($line);
}
)
$script = {
param([Object]$sender, [DataReceivedEventArgs]$e)
$global:Datastream.Add($e.Data)
}
Register-ObjectEvent -InputObject $Process -Action $script -EventName 'OutputDataReceived' | Out-Null
Register-ObjectEvent -InputObject $Process -Action $script -EventName 'ErrorDataReceived' | Out-Null
$Process.Start()
$Process.BeginOutputReadLine()
$Process.BeginErrorReadLine()

Answer 6

If you just want to dynamically dump it to the PowerShell console do this:
my.exe | Out-Default
I can't claim to have figured it out.
See the bottom of this technet post: https://social.technet.microsoft.com/Forums/windowsserver/en-US/b6691fba-0e92-4e9d-aec2-47f3d5a17419/start-process-and-redirect-output-to-powershell-window?forum=winserverpowershell
which also refers to this stackoverflow post.
$LASTEXITCODE was also populated with the exit code from my exe which was also what I needed.