Why does descriptor redirection seem inconsistent? - bash

Most of us know that to redirect STDERR to STDOUT we do 2>&1
We also know about FILE redirection using ">" and process redirection using "|"
What I always wondered about was about the combination of the above two
If you want to redirect both STDERR and STDOUT of prog1 to prog2 you place the 2>&1 prior to the |prog2 pipe. On the other hand, if you are redirecting STDERR and STDOUT of prog1 to a file (file.txt), the 2>&1 goes after the > file.txt.
So I know HOW to do it, I am just wondering WHY it is done like that. To me it seems inconsistent, but I may be looking at it the wrong way
Thanks

They are processed in order.
So if you do
progname 2>&1 1>out.txt
That diverts stderr from the program to the current destination of the program's stdout, which is the stdout stream of the shell, and diverts stdout of the program to out.txt.
if you do
progname 1>out.txt 2>&1
That diverts the stdout of the program to out.txt, then diverts the stderr from the program to the current destination of the program's stdout, which is out.txt.

It helps if you don't think of a pipe as redirection. Using 2>&1, you're redirecting stderr to stdout. Only stdout goes through a pipe. If you redirect stdout before a pipe, then nothing goes through.

Related

Redirect stderr and stdout to one file and stderr to another using cmd

I want to start a program and redirect stdout and stderr in one file and stderr to another. I read a lot about using tee but this seems not to work for cmd.
This already works but I need stderr in a second file as well.
programm >> combined.log 2>&1
I have tried sth like this but it didnt work.
program >> combined.log 2>&1 2>> error.log
It would be nice to have your cake and eat it too. That is not always possible. The stderr log can be captured, then appended to the combined log.
program >>combined.log 2>err.log
type err.log >>combined.log

Redirecting stdout and stderr while duplicating stderr to another file in bash

I'm trying to redirect some bash script outputs.
What I whould like to do is :
./some_script.sh 2> error.log >> all_output.log 2>&1
I whould like to put the stderr in a file, and both stderr and stdout on another file.
In addition I want to append at the end of all_output.log (for error.log that doesn't matter).
But I'm not getting the right syntax, I've been trying lot of things and I wasn't able to find out the right thing to do.
Thanks for you help ! :)
Redirection statements (like > foo or 2> bar or 1>&2) are best read like assignments to file descriptors, executed from left to right. Your code does this:
2> error.log
Means: fd2 = open_for_writing('error.log')
>> all_output.log
Means: fd1 = open_for_appending('all_output.log')
2>&1
Means: fd2 = fd1
By this you can understand that the first statement (2> error.log) will have no effect besides maybe creating the (empty) error.log.
What you want to achieve is duplicate one stream into two different targets. That is not done by a mere redirect of anything. For that you need a process which reads one thing and writes it into two different streams. That's best done using tee(1).
Unfortunately passing streams to other processes is done via pipes and they only pass stdout, not stderr. To achieve your goals you have to swap stderr and stdout first.
The complete resulting call could look like this:
(./some_script.sh 3>&2 2>&1 1>&3 | tee error.log) >> all_outputlog 2>&1

How to redirect error output to both stdout and to stderr?

I'm putting together a complex pipeline, where I want to include stderr in the program output for recordkeeping purposes but I also want errors to remain on stderr so I can spot problems.
I found this question that asks how to direct stdout+stderr to a file and still get stderr on the terminal; it's close, but I don't want to redirect stdout to a file yet: The program's output will be consumed by other scripts, so I'd like it to remain on stdout (and same for stderr). So, to summarize:
Script produces output in fd 1, errors in fd 2.
I want the calling program to rearrange things so that output+errors appear in fd 1, errors in fd 2.
Also, errors should be interleaved with output (as much as their own buffering allows), not saved and added at the end.
Due-diligence notes: Capturing stderr is easy enough with 2>&1. Saving and viewing stdout is easy enough by piping through tee. I also know how to divert stdout to a file and direct stderr through a pipe: command 2>&1 1>fileA | tee fileB. But how do I duplicate stderr and put stdout back in fd 1?
As test to generate both stdout and stderr, let's use the following:
{ echo out; echo err >&2; }
The following code demonstrates how both stdout and stderr can be sent to the next step in the pipeline while also sending stderr to the terminal:
$ { echo out; echo err >&2; } 2> >(tee /dev/stderr) | cat >f
err
$ cat f
out
err
How it works
2>
This redirects stderr to the (pseudo) file which follows.
>(tee /dev/stderr)
This is process substitution and its acts as a pseudo-file that receives input from stderr. Any input it receives is sent to the tee command which sends it both to stderr and to stdout.

Order of redirection operators in Windows

foo.exe > out.txt 2>&1 will redirect both stdout and stderr to a file.
However foo.exe 2>&1 > out.txt will redirect stdout to a file and stderr to console. Why do they behave differently? Aren't they both saying the same thing (send stderr to stdout and send stdout to file)?
You need to set up the redirection from STDOUT first, and then set the 2>&1 redirection. It redirects STDERR to the place that STDOUT is going.

Redirecting stdout/stderr to multiple files

I was wondering how to redirect stderr to multiple outputs. I tried it with this script, but I couldn't get it to work quite right. The first file should have both stdout and stderr, and the 2nd should just have errors.
perl script.pl &> errorTestnormal.out &2> errorTest.out
Is there a better way to do this? Any help would be much appreciated. Thank you.
perl script.pl 2>&1 >errorTestnormal.out | tee -a errorTestnormal.out > errorTest.out
Will do what you want.
This is a bit messy, lets go through it step by step.
We say what used to go to STDERR will now go STDOUT
We say what used to go to STDOUT will now go to errorTestnormal.out.
So now, STDOUT gets printed to a file, and STDERR gets printed to STDOUT. We want put STDERR into 2 different files, which we can do with tee. tee appends the text it's given to a file, and also echoes to STDOUT.
We use tee to append to errorTestnormal.out, so it now contains all the STDOUT and STDERR output of script.pl.
Then, we write STDOUT of tee (which contains STDERR from script.pl) into errorTest.out
After this, errorTestnormal.out has all the STDOUT output, and then all the STDERR output. errotTest.out contains only the STDERR output.
I had to mess around with this for a while as well. In order to get stderr in both files, while only putting stdout into a single file (e.g. stderr into errors.log and output.log and then stdout into just output.log) AND in the order that they happen, this command is better:
((sh test.sh 2>&1 1>&3 | tee errors.log) 3>&1 | tee output.log) > /dev/null 2>&1
The last /dev/nul 2>&1 can be omitted if you want the stdout and stderr to still be output onto the screen.
I guess in case of the 2nd ">" you try to send the error output of errorTestnormal.out (and not that of script.pl) to errorTest.out.

Resources