I am running an application which opens CMD and connect via API service. Throughout the day new stuff will show up in the CMD and I would like to export that information to txt somewhere and Everytime something new shows up append to the same file, or create a new one. It doesn't really matter
App.exe > /file.txt doesn't really work
Redirection examples
command > filename # Redirect command output to a file (overwrite)
command >> filename # APPEND into a file
command 2> filename # Redirect Errors from operation to a file(overwrite)
command 2>> filename # APPEND errors to a file
command 2>&1 # Add errors to results
command 1>&2 # Add results to errors
command | command # This is the basic form of a PowerShell Pipeline
# In PowerShell 3.0+
command 3> warning.txt # Write warning output to warning.txt
command 4>> verbose.txt # Append verbose.txt with the verbose output
command 5>&1 # Writes debug output to the output stream
command *> out.txt # Redirect all streams (output, error, warning, verbose, and debug) to out.txt
You are not showing any code as to how you are starting/using cmd.exe for your use case. Which just leaves folks trying to help you, to guess. So, redirect of cmd.exe, for example:
$MyOutputFile = C:\MyOutputFile.txt
Start-Process -FilePath c:\windows\system32\cmd.exe -ArgumentList '/c C:\YourCommand.bat' -Wait -NoNewWindow -RedirectStandardOutput $MyOutputFile
Lastly, since you've left us to guess. If you’re launching Process A from PowerShell, but it, Process A is, in turn, launching Process B, then it would be up to Process A to capture or redirect the output of Process B. There’s no way for PowerShell to sub-capture if Process A isn’t doing it.
Resources
About Redirection
How-to: Redirection
PowerShell Redirection Operators
Understanding Streams, Redirection, and Write-Host in PowerShell
Use PowerShell Redirection Operators for Script Flexibility
Related
Hello so I have been making a small cross platform script I can curl and pipe into bash and Powershell. The basic idea is the server sends a command to the interpreter and then it gives a command to redirect all output after to stdout. An example in bash is
#some commands
aplay rick.wav
cat -
random text
that will be redirected to stdout by cat...
bash will never see this
I would then pipe this to stdin of bash
But for Powershell I can do cat test.ps1 | iex or cat test.ps1 | powershell -
But can't redirect stdin to stdout continuously in one command like cat - because cat doesn't look from stdin.
Also some side notes after trying a lot of random things it seems like there are many stdin types for Windows, one being keyboard and another being pipes
You can pipe lines of text to powershell.exe, the Windows PowerShell CLI, via -Command - (-c -), and it will interpret them one by one.
Here's an interactive demonstration from inside PowerShell; it works the same with input piped (provided via stdin) from the outside:
# Repeatedly prompt for a line of input and execute it as a PowerShell command.
# Press Ctrl-C to exit.
& { while ($true) { Read-Host } } | powershell -noprofile -c -
Note:
-Command - has problematic aspects, notably with commands that span multiple lines (an additional Enter keystroke / newline is then needed for the command to be recognized) and so does -File -, whose behavior is even stranger - see this answer and GitHub issue #3223.
Another demonstration, simulating outside stdin input via 2 lines piped to powershell -c -:
'get-date', 'get-item /' | powershell -noprofile -c -
The two commands are executed and their output is printed; powershell.exe then exits, because no more stdin input is available; however, with indefinite stdin input (analogous to cat - on Unix-like platforms) the PowerShell process would be kept alive indefintely too.
I have a problem with the nohup command.
When I run my job, I have a lot of data. The output nohup.out becomes too large and my process slows down. How can I run this command without getting nohup.out?
The nohup command only writes to nohup.out if the output would otherwise go to the terminal. If you have redirected the output of the command somewhere else - including /dev/null - that's where it goes instead.
nohup command >/dev/null 2>&1 # doesn't create nohup.out
Note that the >/dev/null 2>&1 sequence can be abbreviated to just >&/dev/null in most (but not all) shells.
If you're using nohup, that probably means you want to run the command in the background by putting another & on the end of the whole thing:
nohup command >/dev/null 2>&1 & # runs in background, still doesn't create nohup.out
On Linux, running a job with nohup automatically closes its input as well. On other systems, notably BSD and macOS, that is not the case, so when running in the background, you might want to close input manually. While closing input has no effect on the creation or not of nohup.out, it avoids another problem: if a background process tries to read anything from standard input, it will pause, waiting for you to bring it back to the foreground and type something. So the extra-safe version looks like this:
nohup command </dev/null >/dev/null 2>&1 & # completely detached from terminal
Note, however, that this does not prevent the command from accessing the terminal directly, nor does it remove it from your shell's process group. If you want to do the latter, and you are running bash, ksh, or zsh, you can do so by running disown with no argument as the next command. That will mean the background process is no longer associated with a shell "job" and will not have any signals forwarded to it from the shell. (A disowned process gets no signals forwarded to it automatically by its parent shell - but without nohup, it will still receive a HUP signal sent via other means, such as a manual kill command. A nohup'ed process ignores any and all HUP signals, no matter how they are sent.)
Explanation:
In Unixy systems, every source of input or target of output has a number associated with it called a "file descriptor", or "fd" for short. Every running program ("process") has its own set of these, and when a new process starts up it has three of them already open: "standard input", which is fd 0, is open for the process to read from, while "standard output" (fd 1) and "standard error" (fd 2) are open for it to write to. If you just run a command in a terminal window, then by default, anything you type goes to its standard input, while both its standard output and standard error get sent to that window.
But you can ask the shell to change where any or all of those file descriptors point before launching the command; that's what the redirection (<, <<, >, >>) and pipe (|) operators do.
The pipe is the simplest of these... command1 | command2 arranges for the standard output of command1 to feed directly into the standard input of command2. This is a very handy arrangement that has led to a particular design pattern in UNIX tools (and explains the existence of standard error, which allows a program to send messages to the user even though its output is going into the next program in the pipeline). But you can only pipe standard output to standard input; you can't send any other file descriptors to a pipe without some juggling.
The redirection operators are friendlier in that they let you specify which file descriptor to redirect. So 0<infile reads standard input from the file named infile, while 2>>logfile appends standard error to the end of the file named logfile. If you don't specify a number, then input redirection defaults to fd 0 (< is the same as 0<), while output redirection defaults to fd 1 (> is the same as 1>).
Also, you can combine file descriptors together: 2>&1 means "send standard error wherever standard output is going". That means that you get a single stream of output that includes both standard out and standard error intermixed with no way to separate them anymore, but it also means that you can include standard error in a pipe.
So the sequence >/dev/null 2>&1 means "send standard output to /dev/null" (which is a special device that just throws away whatever you write to it) "and then send standard error to wherever standard output is going" (which we just made sure was /dev/null). Basically, "throw away whatever this command writes to either file descriptor".
When nohup detects that neither its standard error nor output is attached to a terminal, it doesn't bother to create nohup.out, but assumes that the output is already redirected where the user wants it to go.
The /dev/null device works for input, too; if you run a command with </dev/null, then any attempt by that command to read from standard input will instantly encounter end-of-file. Note that the merge syntax won't have the same effect here; it only works to point a file descriptor to another one that's open in the same direction (input or output). The shell will let you do >/dev/null <&1, but that winds up creating a process with an input file descriptor open on an output stream, so instead of just hitting end-of-file, any read attempt will trigger a fatal "invalid file descriptor" error.
nohup some_command > /dev/null 2>&1&
That's all you need to do!
Have you tried redirecting all three I/O streams:
nohup ./yourprogram > foo.out 2> foo.err < /dev/null &
You might want to use the detach program. You use it like nohup but it doesn't produce an output log unless you tell it to. Here is the man page:
NAME
detach - run a command after detaching from the terminal
SYNOPSIS
detach [options] [--] command [args]
Forks a new process, detaches is from the terminal, and executes com‐
mand with the specified arguments.
OPTIONS
detach recognizes a couple of options, which are discussed below. The
special option -- is used to signal that the rest of the arguments are
the command and args to be passed to it.
-e file
Connect file to the standard error of the command.
-f Run in the foreground (do not fork).
-i file
Connect file to the standard input of the command.
-o file
Connect file to the standard output of the command.
-p file
Write the pid of the detached process to file.
EXAMPLE
detach xterm
Start an xterm that will not be closed when the current shell exits.
AUTHOR
detach was written by Robbert Haarman. See http://inglorion.net/ for
contact information.
Note I have no affiliation with the author of the program. I'm only a satisfied user of the program.
Following command will let you run something in the background without getting nohup.out:
nohup command |tee &
In this way, you will be able to get console output while running script on the remote server:
sudo bash -c "nohup /opt/viptel/viptel_bin/log.sh $* &> /dev/null" &
Redirecting the output of sudo causes sudo to reask for the password, thus an awkward mechanism is needed to do this variant.
If you have a BASH shell on your mac/linux in-front of you, you try out the below steps to understand the redirection practically :
Create a 2 line script called zz.sh
#!/bin/bash
echo "Hello. This is a proper command"
junk_errorcommand
The echo command's output goes into STDOUT filestream (file descriptor 1).
The error command's output goes into STDERR filestream (file descriptor 2)
Currently, simply executing the script sends both STDOUT and STDERR to the screen.
./zz.sh
Now start with the standard redirection :
zz.sh > zfile.txt
In the above, "echo" (STDOUT) goes into the zfile.txt. Whereas "error" (STDERR) is displayed on the screen.
The above is the same as :
zz.sh 1> zfile.txt
Now you can try the opposite, and redirect "error" STDERR into the file. The STDOUT from "echo" command goes to the screen.
zz.sh 2> zfile.txt
Combining the above two, you get:
zz.sh 1> zfile.txt 2>&1
Explanation:
FIRST, send STDOUT 1 to zfile.txt
THEN, send STDERR 2 to STDOUT 1 itself (by using &1 pointer).
Therefore, both 1 and 2 goes into the same file (zfile.txt)
Eventually, you can pack the whole thing inside nohup command & to run it in the background:
nohup zz.sh 1> zfile.txt 2>&1&
You can run the below command.
nohup <your command> & > <outputfile> 2>&1 &
e.g.
I have a nohup command inside script
./Runjob.sh > sparkConcuurent.out 2>&1
I have a shell script that can enable ble device scan with the following command
timeout 10s hcitool lescan
By executing this script (say ble_scan), I can see the nearby devices shown on the terminal.
However, when I redirect it to the file and terminal
./ble_scan | tee test.log
I can't see the nearby devices shown on the screen anymore and log file as well.
./ble_scan 2>&1 | tee test.log
The above redirection also doesnt help, anything I go wrong here?
If the command behaves differently with file output, you can run it within script.
script test.log
#=> Script started, output file is test.log
./ble_scan
# lots of output here
exit
#=> Script done, output file is test.log
Note that the file will include terminal-specific characters like carriage returns not normally captured in output redirects.
I'm using a tool that tests hard disks, fstest.exe. It runs fine from the command line, displaying how long it took to do various file-creation/-deletion/-mangling tasks. The usual output, when run from the command line as fstest.exe otherParams, looks like this:
---
CPU Usage: 0.0%
Disk reads/sec: 0
Disk writes/sec: 0
Disk bytes/read: 0
Disk bytes/write: 0
Test duration: 0 milliseconds, 1153 ticks (3507177 ticks/sec)
---
The trouble is that when I redirect the output to file, it doesn't display anything:
fstest.exe otherParams > out.txt creates an empty out.txt file, even though the command otherwise executed just fine (and created a few test-files as part of its execution).
How can I force this application to redirect output to a file? I've tried looking at it more closely with PowerShell (via Start-Process), and both the standard-out and standard-error streams are just empty.
Other things I've tried:
cmd /c "fstest.exe otherParams > out.txt"
fstest.exe otherParams 2>&1 >> out.txt
fstest.exe otherParams | sort
powershell Start-Process -FilePath .\fstest.exe -ArgumentList #("create2", "-openexisting") -RedirectStandardOutput out.txt -RedirectStandardError err.txt -wait
(That creates both out.txt and err.txt, both empty.)
What would cause an application to change its output depending on whether it's redirected, and is there any way I can make it redirect to file?
UPDATE: I've gotten my hands on the source code. It's C++, and the output is just straightforward printf statements.
It turns out the program in question wasn't flushing stdout after doing a printf to it. Apparently, cmd is willing to flush that buffer when printing to the console, but not when redirecting output to a file. (Or perhaps the file handle is closed before the console could force a flush.) The program was exiting normally (via return, exit code always 0).
To answer the question: I had to fix the program; there was nothing I could do with command-line switches or redirects to change it.
CMD can handle up to 10 file descriptors. Try redirecting them to separate files to identify the descriptor your program writes to:
fstest.exe {params} 0>out0.txt 3>out3.txt 4>out4.txt 5>out5.txt ...
If it's writing to standard error, instead of standard output, redirect thusly:
fstest.exe otherParams 2> out.txt
Why does PowerShell show the surprising behaviour in the second example below?
First, an example of sane behaviour:
PS C:\> & cmd /c "echo Hello from standard error 1>&2"; echo "`$LastExitCode=$LastExitCode and `$?=$?"
Hello from standard error
$LastExitCode=0 and $?=True
No surprises. I print a message to standard error (using cmd's echo). I inspect the variables $? and $LastExitCode. They equal to True and 0 respectively, as expected.
However, if I ask PowerShell to redirect standard error to standard output over the first command, I get a NativeCommandError:
PS C:\> & cmd /c "echo Hello from standard error 1>&2" 2>&1; echo "`$LastExitCode=$LastExitCode and `$?=$?"
cmd.exe : Hello from standard error
At line:1 char:4
+ cmd <<<< /c "echo Hello from standard error 1>&2" 2>&1; echo "`$LastExitCode=$LastExitCode and `$?=$?"
+ CategoryInfo : NotSpecified: (Hello from standard error :String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
$LastExitCode=0 and $?=False
My first question, why the NativeCommandError?
Secondly, why is $? False when cmd ran successfully and $LastExitCode is 0? PowerShell's documentation about automatic variables doesn't explicitly define $?. I always supposed it is True if and only if $LastExitCode is 0, but my example contradicts that.
Here's how I came across this behaviour in the real-world (simplified). It really is FUBAR. I was calling one PowerShell script from another. The inner script:
cmd /c "echo Hello from standard error 1>&2"
if (! $?)
{
echo "Job failed. Sending email.."
exit 1
}
# Do something else
Running this simply as .\job.ps1, it works fine, and no email is sent. However, I was calling it from another PowerShell script, logging to a file .\job.ps1 2>&1 > log.txt. In this case, an email is sent! What you do outside the script with the error stream affects the internal behaviour of the script. Observing a phenomenon changes the outcome. This feels like quantum physics rather than scripting!
[Interestingly: .\job.ps1 2>&1 may or not blow up depending on where you run it]
(I am using PowerShell v2.)
The '$?' variable is documented in about_Automatic_Variables:
$?
Contains the execution status of the last operation
This is referring to the most recent PowerShell operation, as opposed to the last external command, which is what you get in $LastExitCode.
In your example, $LastExitCode is 0, because the last external command was cmd, which was successful in echoing some text. But the 2>&1 causes messages to stderr to be converted to error records in the output stream, which tells PowerShell that there was an error during the last operation, causing $? to be False.
To illustrate this a bit more, consider this:
> java -jar foo; $?; $LastExitCode
Unable to access jarfile foo
False
1
$LastExitCode is 1, because that was the exit code of java.exe. $? is False, because the very last thing the shell did failed.
But if all I do is switch them around:
> java -jar foo; $LastExitCode; $?
Unable to access jarfile foo
1
True
... then $? is True, because the last thing the shell did was print $LastExitCode to the host, which was successful.
Finally:
> &{ java -jar foo }; $?; $LastExitCode
Unable to access jarfile foo
True
1
...which seems a bit counter-intuitive, but $? is True now, because the execution of the script block was successful, even if the command run inside of it was not.
Returning to the 2>&1 redirect.... that causes an error record to go in the output stream, which is what gives that long-winded blob about the NativeCommandError. The shell is dumping the whole error record.
This can be especially annoying when all you want to do is pipe stderr and stdout together so they can be combined in a log file or something. Who wants PowerShell butting in to their log file??? If I do ant build 2>&1 >build.log, then any errors that go to stderr have PowerShell's nosey $0.02 tacked on, instead of getting clean error messages in my log file.
But, the output stream is not a text stream! Redirects are just another syntax for the object pipeline. The error records are objects, so all you have to do is convert the objects on that stream to strings before redirecting:
From:
> cmd /c "echo Hello from standard error 1>&2" 2>&1
cmd.exe : Hello from standard error
At line:1 char:4
+ cmd &2" 2>&1
+ CategoryInfo : NotSpecified: (Hello from standard error :String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
To:
> cmd /c "echo Hello from standard error 1>&2" 2>&1 | %{ "$_" }
Hello from standard error
...and with a redirect to a file:
> cmd /c "echo Hello from standard error 1>&2" 2>&1 | %{ "$_" } | tee out.txt
Hello from standard error
...or just:
> cmd /c "echo Hello from standard error 1>&2" 2>&1 | %{ "$_" } >out.txt
This bug is an unforeseen consequence of PowerShell's prescriptive design for error handling, so most likely it will never be fixed. If your script plays only with other PowerShell scripts, you're safe. However if your script interacts with applications from the big wide world, this bug may bite.
PS> nslookup microsoft.com 2>&1 ; echo $?
False
Gotcha! Still, after some painful scratching, you'll never forget the lesson.
Use ($LastExitCode -eq 0) instead of $?
(Note: This is mostly speculation; I rarely use many native commands in PowerShell and others probably know more about PowerShell internals than me)
I guess you found a discrepancy in the PowerShell console host.
If PowerShell picks up stuff on the standard error stream it will assume an error and throw a NativeCommandError.
PowerShell can only pick this up if it monitors the standard error stream.
PowerShell ISE has to monitor it, because it is no console application and thus a native console application has no console to write to. This is why in the PowerShell ISE this fails regardless of the 2>&1 redirection operator.
The console host will monitor the standard error stream if you use the 2>&1 redirection operator because output on the standard error stream has to be redirected and thus read.
My guess here is that the console PowerShell host is lazy and just hands native console commands the console if it doesn't need to do any processing on their output.
I would really believe this to be a bug, because PowerShell behaves differently depending on the host application.
Update: The problems have been fixed in v7.2 - see this answer.
A summary of the problems as of v7.1:
The PowerShell engine still has bugs with respect to 2> redirections applied to external-program calls:
The root cause is that using 2> causes the stderr (standard error) output to be routed via PowerShell's error stream (see about_Redirection), which has the following undesired consequences:
If $ErrorActionPreference = 'Stop' happens to be in effect, using 2> unexpectedly triggers a script-terminating error, i.e. aborts the script (even in the form 2>$null, where the intent is clearly to ignore stderr lines). See GitHub issue #4002.
Workaround: (Temporarily) set $ErrorActionPreference = 'Continue'
Since 2> currently touches the error stream, $?, the automatic success-status variable is invariably set to $False if at least one stderr line was emitted, and then no longer reflects the true success status of the command. See this GitHub issue.
Workaround, as recommended in your answer: only ever use $LASTEXITCODE -eq 0 to test for success after calls to external programs.
With 2>, stderr lines are unexpectedly recorded in the automatic $Error variable (the variable that keeps a log of all errors that occurred in the session) - even if you use 2>$null. See this GitHub issue.
Workaround: Short of keeping track how many error records were added and removing them with $Error.RemoveAt() one by one, there is none.
Generally, unfortunately, some PowerShell hosts by default route stderr output from external programs via PowerShell's error stream, i.e. treat it as error output, which is inappropriate, because many external programs use stderr also for status information, or more generally, for anything that is not data (git being a prime example): Not every stderr line can be assumed to represent an error, and the presence of stderr output does not imply failure.
Affected hosts:
The obsolescent Windows PowerShell ISE and possibly other, older GUI-based IDEs other than Visual Studio Code.
When executing external programs via PowerShell remoting or in a background job (these two invocation mechanisms share the same infrastructure and use the ServerRemoteHost host that ships with PowerShell).
Hosts that DO behave as expected in non-remoting, non-background invocations (they pass stderr lines through to the display and print them normally):
Terminals (consoles), including Windows Terminal.
Visual Studio Code with the PowerShell extension; this cross-platform editor (IDE) is meant to supersede the Windows PowerShell ISE.
This inconsistency across hosts is discussed in this GitHub issue.
For me it was an issue with ErrorActionPreference.
When running from ISE I've set $ErrorActionPreference = "Stop" in the first lines and that was intercepting everything event with *>&1 added as parameters to the call.
So first I had this line:
& $exe $parameters *>&1
Which like I've said didn't work because I had $ErrorActionPreference = "Stop" earlier in file (or it can be set globally in profile for user launching the script).
So I've tried to wrap it in Invoke-Expression to force ErrorAction:
Invoke-Expression -Command "& `"$exe`" $parameters *>&1" -ErrorAction Continue
And this doesn't work either.
So I had to fallback to hack with temporary overriding ErrorActionPreference:
$old_error_action_preference = $ErrorActionPreference
try
{
$ErrorActionPreference = "Continue"
& $exe $parameters *>&1
}
finally
{
$ErrorActionPreference = $old_error_action_preference
}
Which is working for me.
And I've wrapped that into a function:
<#
.SYNOPSIS
Executes native executable in specified directory (if specified)
and optionally overriding global $ErrorActionPreference.
#>
function Start-NativeExecutable
{
[CmdletBinding(SupportsShouldProcess = $true)]
Param
(
[Parameter (Mandatory = $true, Position = 0, ValueFromPipelinebyPropertyName=$True)]
[ValidateNotNullOrEmpty()]
[string] $Path,
[Parameter (Mandatory = $false, Position = 1, ValueFromPipelinebyPropertyName=$True)]
[string] $Parameters,
[Parameter (Mandatory = $false, Position = 2, ValueFromPipelinebyPropertyName=$True)]
[string] $WorkingDirectory,
[Parameter (Mandatory = $false, Position = 3, ValueFromPipelinebyPropertyName=$True)]
[string] $GlobalErrorActionPreference,
[Parameter (Mandatory = $false, Position = 4, ValueFromPipelinebyPropertyName=$True)]
[switch] $RedirectAllOutput
)
if ($WorkingDirectory)
{
$old_work_dir = Resolve-Path .
cd $WorkingDirectory
}
if ($GlobalErrorActionPreference)
{
$old_error_action_preference = $ErrorActionPreference
$ErrorActionPreference = $GlobalErrorActionPreference
}
try
{
Write-Verbose "& $Path $Parameters"
if ($RedirectAllOutput)
{ & $Path $Parameters *>&1 }
else
{ & $Path $Parameters }
}
finally
{
if ($WorkingDirectory)
{ cd $old_work_dir }
if ($GlobalErrorActionPreference)
{ $ErrorActionPreference = $old_error_action_preference }
}
}