I need to redirect stdout to console along with stdout and stderr redirected to a file. This needs to be done inside a shell script.
I found the below code to redirect both to console and log file, now I need to remove stderr to console.
exec > >(tee -i "output.log") 2>&1
Could you please help me here?
Derived from this answer:
How do I get both STDOUT and STDERR to go to the terminal and a log file?
I tested the following code:
echo '' > out.log
exec 1> >(tee -a -i out.log) 2> >(tee -a -i out.log > /dev/null)
>&2 echo yay
echo nay
The STDERR content goes to file and not to console, and STDOUT goes to both.
I think you should launch it like this:
your-command 2>&1 | tee -i "output.log"
It will pipe both stdin and stderr to tee -i output.log, which will echo text before writing.
Or, to ignore stderr (pipe to /dev/null):
your-command 2>/dev/null | tee -i "output.log"
Or your command modified with /dev/null redirection:
exec > >(tee -i "output.log") 2>/dev/null
It'd be something like this, but you don't have to tee that second one.
exec 1> >( tee stdout.txt ) 2> stderr.txt
output 1 is tee'd so you see it on screen, and in stdout.txt
errors 2 just goes straight to stderr.txt
Related
I've a Bash command with runs a python script and produce outputs on stdout and stderror.
Normal and error logs are written to separate files with this command:
python3 file.py >> normal.log 2>> error.log
Additionally, stdout and stderr shall be forwarded to an external program (e.g. logToTelegram.sh):
python3 file.py 2>&1 | logToTelegram.sh
Is there a way to implement both during one execution, write log files (normal and error) and pipe stdout and stderror together to the program logToTelegram.sh?
This should do the trick:
( python3 file.py 2> >(tee -a error.log) > >(tee -a normal.log) ) | logToTelegram.sh
Note that 2> >(tee ...) has to be placed before > >(tee ...) for this to work correctly. For simplicity, both tee commands output to stdout, eliminating the need for 2>&1 before piping to logToTelegram.sh.
Or, to stay closer to the original code and to be more precise:
( python3 file.py > >(tee -a normal.log) 2> >(tee -a error.log >&2) ) 2>&1 | logToTelegram.sh
Here, the first tee outputs to stdout, while the second tee outputs to stderr, thus 2>&1 is required to send all output to logToTelegram.sh.
I know that it is possible to redirect both to a specific file:
./command 1> out.log 2> err.log
or
./command 1>test.log 2>&1
to write both to a file. But I don't know a way to write both to the same file (preserving the output order) while printing just one of both. tee isn't very helpful because it prints both file descriptors.
Writing standard output to one file and both to another is fairly simple with tee:
{ cmd | tee stdout.log; } &> both.log
Both descriptors of the compound command are redirected to both.log, but the standard output of cmd is first passed through tee to stdout.log before also being written to both.log.
Writing standard error to one and both to another is trickier.
{ foo 2>&1 1>&3 | tee stderr.log ; } 3>&1 | tee both.log > /dev/null
It's a little tricky to describe correctly. First, the command group's standard error is ignored; it's standard output is the pipe to tee both.log. But 3>&1 also copies its fd 3 to its standard output. So the question is, what gets written to that?
Inside the command group, foo's standard output is the pipe to tee stderr.log. The 2>&1 copies tee's standard error to that descriptor, and 1>&3 copies foo's standard output to the inherited fd 3.
It will be better to write them to different files, in this case it will be very easy:
cmd 2>/stderr.log| tee -a stdout.log
But if you want a single file, you will need some tricks here and additional process running for redirection.
You can use several redirections:
foo () { echo 1 ; echo 2 >&2 ; }
(( foo | tee >(cat) >&3) &>log ) 3>&1
The tee command sends the stdout to file descriptors 1 (through the process substitution) and 3. Both stdout and stderr are redirected to the log. In the end, 3, the copy of stdout, is sent back to the terminal.
Alternatively, you can do without the process substitution, redirect the output directly to log, but use -a and >> for append. You need to clear the log beforehand.
: > log; (( foo | tee -a log >&3) 2>> log ) 3>&1
stdout to the console and stdout and error to a file?
{ cmd | tee /dev/tty; } &> /tmp/both.log
To answer the question :
stderr to the console and stdout and error to a file?
{ cmd 3>&1 1>&2 2>&3 | tee /dev/tty; } &> /tmp/both.log
3>&1 1>&2 2>&3 means swapping stdout and stderr
The script should redirect all the output (stdout and stderr) to a log file, and only display stderr to the screen (notifying user if an error happens). The command tee may help but don't know how to write it.
Thanks.
P.S., thanks lihao and konsolebox for the answer, but is there a way to keep the output in order. For example:
$ cat test.sh
echo "to stdout..1"
echo "to stderr..1" >&2
echo "to stdout..2"
echo "to stderr..2" >&2
$ sh test.sh 2>&1 >test.log | tee -a test.log
to stderr..1
to stderr..2
$ cat test.log
to stdout..1
to stdout..2
to stderr..1
to stderr..2
Command: { sh test.sh 2> >(tee /dev/fd/4); } 4>&1 >test.log has the same output.
how about the following:
cmd args 2>&1 >logfile | tee -a logfile
You should map normal stdout to another file descriptor (4), make the file the default output, then use tee to redirect output to the new file descriptor through /dev/fd. Of course you'd need process substitution to pass stderr output to tee:
{ cmd args 2> >(exec tee /dev/fd/4); } 4>&1 >file
If you want to make a general redirection for the script, place this at the beginning of it:
exec 4>&1 >file 2> >(exec tee /dev/fd/4)
You can restore normal output with:
exec >&4 4>&-
I'm running a pipeline of commands that have STDERR and STDOUT outputs. I want to save both outputs in a single log file.
This are my attempts to do it:
bash my_script.sh > log.txt #Only save STDOUT
bash my_script.sh > >(tee log.txt) 2> >(tee log.txt >&2) #The STDERR overwrite the STDOUT
I hope you can provide a simple solution to do this.
Thanks for your time!
How about just
bash my_script.sh > >(tee log.txt) 2>&1
Also if you want to append output if log.txt already exists, add -a option to tee
bash my_script.sh > >(tee -a log.txt) 2>&1
It's actually equivalent to bash my_script.sh 2>&1 | tee log.txt or bash my_script.sh 2>&1 | tee -a log.txt
bash my_script.sh > log.txt 2>&1
where 2>&1 redirects stderr to stdout
Redirecting stdout+stderr such that both get written to a file while still outputting to stdout is simple enough:
cmd 2>&1 | tee output_file
But then now both stdout/stderr from cmd are coming on stdout. I'd like to write stdout+stderr to the same file (so ordering is preserved assuming cmd is single threaded) but then still be able to also separately redirect them, something like this:
some_magic_tee_variant combined_output cmd > >(command-expecting-stdout) 2> >(command-expecting-stderr)
So combined_output contains the both with order preserved, but the command-expecting-stdout only gets stdout and command-expecting-stderr only gets stderr. Basically, I want to log stdout+stderr while still allowing stdout and stderr to be separately redirected and piped. The problem with the tee approach is it globs them together. Is there a way to do this in bash/zsh?
From what I unterstand this is what you are looking for. First I made a litte script to write on stdout and stderr. It looks like this:
$ cat foo.sh
#!/bin/bash
echo foo 1>&2
echo bar
Then I ran it like this:
$ ./foo.sh 2> >(tee stderr | tee -a combined) 1> >(tee stdout | tee -a combined)
foo
bar
The results in my bash look like this:
$ cat stderr
foo
$ cat stdout
bar
$ cat combined
foo
bar
Note that the -a flag is required so the tees don't overwrite the other tee's content.
{ { cmd | tee out >&3; } 2>&1 | tee err >&2; } 3>&1
Or, to be pedantic:
{ { cmd 3>&- | tee out >&3 2> /dev/null; } 2>&1 | tee err >&2 3>&- 2> /dev/null; } 3>&1
Note that it's futile to try and preserve order. It is basically impossible. The only solution would be to modify "cmd" or use some LD_PRELOAD or gdb hack,
Order can indeed be preserved. Here's an example which captures the standard output and error, in the order in which they are generated, to a logfile, while displaying only the standard error on any terminal screen you like. Tweak to suit your needs.
1.Open two windows (shells)
2.Create some test files
touch /tmp/foo /tmp/foo1 /tmp/foo2
3.In window1:
mkfifo /tmp/fifo
</tmp/fifo cat - >/tmp/logfile
4.Then, in window2:
(ls -l /tmp/foo /tmp/nofile /tmp/foo1 /tmp/nofile /tmp/nofile; echo successful test; ls /tmp/nofile1111) 2>&1 1>/tmp/fifo | tee /tmp/fifo 1>/dev/pts/1
Where /dev/pts/1 can be whatever terminal display you want. The subshell runs some "ls" and "echo" commands in sequence, some succeed (providing stdout) and some fail (providing stderr) in order to generate a mingled stream of output and error messages, so that you can verify the correct ordering in the log file.
Here's how I do it:
exec 3>log ; example_command 2>&1 1>&3 | tee -a log ; exec 3>&-
Worked Example
bash$ exec 3>log ; { echo stdout ; echo stderr >&2 ; } 2>&1 1>&3 | \
tee -a log ; exec 3>&-
stderr
bash$ cat log
stdout
stderr
Here's how that works:
exec 3>log sets up file descriptor 3 to redirect into the file called log, until further notice.
example_command to make this a working example, I used { echo stdout ; echo stderr >&2 ; }. Or you could use ls /tmp doesnotexist to provide output instead.
Need to jump ahead to the pipe | at this point because bash does it first. The pipe sets up a pipe and redirects the file descriptor 1 into this pipe. So now, STDOUT is going into the pipe.
Now we can go back to where we were next in our left-to-right interpretation: 2>&1 this says errors from the program are to go to where STDOUT currently points, i.e. into the pipe we just set up.
1>&3 means STDOUT is redirected into file descriptor 3, which we earlier set up to output to the log file. So STDOUT from the command just goes into the log file, not to the terminal's STDOUT.
tee -a log takes it's input from the pipe (which you'll remember is now the errors from the command), and outputs it to STDOUT and also appends it to the log file.
exec 3>&- closes the file descriptor 3.
Victor Sergienko's comment is what worked for me, adding exec to the front of it makes this work for the entire script (instead of having to put it after individual commands)
exec 2> >(tee -a output_file >&2) 1> >(tee -a output_file)