This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
osx/linux: pipes into two processes?
Is there a way to pipe the output from one command into the input of two other commands, running them simultaneously?
Something like this:
$ echo 'test' |(cat) |(cat)
test
test
The reason I want to do this is that I have a program which receives an FM radio signal from a USB SDR device, and outputs the audio as raw PCM data (like a .wav file but with no header.) Since the signal is not music but POCSAG pager data, I need to pipe it to a decoder program to recover the pager text. However I also want to listen to the signal so I know whether any data is coming in or not. (Otherwise I can't tell if the decoder is broken or there's just no data being broadcast.) So as well as piping the data to the pager decoder, I also need to pipe the same data to the play command.
Currently I only know how to do one - either pipe it to the decoder and read the data in silence, or pipe it to play and hear it without seeing any decoded text.
How can I pipe the same data to both commands, so I can read the text and hear the audio?
I can't use tee as it only writes the duplicated data to a file, but I need to process the data in real-time.
It should be ok if you use both tee and mkfifo.
mkfifo pipe
cat pipe | (command 1) &
echo 'test' | tee pipe | (command 2)
Recent bash present >(command) syntax:
echo "Hello world." | tee >(sed 's/^/1st: /') >(sed 's/^/2nd cmd: /') >/dev/null
May return:
2nd cmd: Hello world.
1st: Hello world.
download somefile.ext, save them, compute md5sum and sha1sum:
wget -O - http://somewhere.someland/somepath/somefile.ext |
tee somefile.ext >(md5sum >somefile.md5) | sha1sum >somefile.sha1
or
wget -O - http://somewhere.someland/somepath/somefile.ext |
tee >(md5sum >somefile.md5) >(sha1sum >somefile.sha1) >somefile.ext
Old answer
There is a way to do that via unnamed pipe (tested under linux):
(( echo "hello" |
tee /dev/fd/5 |
sed 's/^/1st occure: /' >/dev/fd/4
) 5>&1 |
sed 's/^/2nd command: /'
) 4>&1
give:
2nd command: hello
1st occure: hello
This sample will let you download somefile.ext, save them, compute his md5sum and compute his sha1sum:
(( wget -O - http://somewhere.someland/somepath/somefile.ext |
tee /dev/fd/5 |
md5sum >/dev/fd/4
) 5>&1 |
tee somefile.ext |
sha1sum
) 4>&1
Maybe take a look at tee command. What it does is simply print its input to a file, but it also prints its input to the standard output. So something like:
echo "Hello" | tee try.txt | <some_command>
Will create a file with content "Hello" AND also let "Hello" (flow through the pipeline) end up as <some_command>'s STDIN.
Related
This question already has answers here:
write to a file after piping output from tail -f through to grep
(4 answers)
Closed 5 years ago.
I am having an issue with filtering a log file that is being written and writing the output to another file (if possible using tee, so I can see it working as it goes).
I can get it to output on stdout, but not write to a file, either using tee or >>.
I can also get it to write to the file, but only if I drop the -f options from tail, which I need.
So, here is an overview of the commands:
tail -f without writing to file: tail -f test.log | sed 's/a/b/' works
tail writing to file: tail test.log | sed 's/a/b/' | tee -a a.txt works
tail -f writing to file: tail -f test.log | sed 's/a/b/' | tee -a a.txt doesn't output on stdout nor writes to file.
I would like 3. to work.
It's the sed buffering. Use sed -u. man sed:
-u, --unbuffered
load minimal amounts of data from the input files and flush the
output buffers more often
And here's a test for it (creates files fooand bar):
$ for i in {1..3} ; do echo a $i ; sleep 1; done >> foo &
[1] 12218
$ tail -f foo | sed -u 's/a/b/' | tee -a bar
b 1
b 2
b 3
Be quick or increase the {1..3} to suit your skillz.
As an example, taking one single program's stdout, obtaining two copies of it with tee and sending them both (one or preferably both able to be piped through other programs) back into vimdiff.
Bonus points if it can be done without having to create a file on disk.
I know how to direct input into a program that takes two inputs, like this
vimdiff <(curl http://google.com) <(curl http://archives.com/last_night/google.com)
and with tee for making two output streams
echo "abc" | tee >(sed 's/a/zzz/') >(sed 's/c/zzz/')
but I do not know how to connect the pipes back together into a diamond shape.
It's not so hard if you can use a fifo:
test -e fifo || mkfifo fifo
echo abc | tee >(sed s/a/zzz/ > fifo) | sed s/c/zzz/ | diff - fifo
Just as a side note, to have this work under ZSH an extra ">" is needed after tee (multios option should be set):
$ setopt multios
$ test -e fifo || mkfifo fifo
$ echo abc | tee > >(sed s/a/zzz/ > fifo) | sed s/c/zzz/ | diff - fifo
I have a process that is writing to standard out, and I want to be able to monitor the output by grepping for various strings while running tail -f. One way to do this is to write to a normal file and then tail the file grepping for one string, then tail it grepping for another. However, I don't want to write to a file because this will fill up the disk, and I don't want to deal with the hassle of rotating log files.
I thought I could achieve this using fifos. E.g.,
mkfifo myfifo
foo > myfifo &
tail -f myfifo | grep bar
tail -f myfifo | grep baz
Unfortunately, this doesn't seem to work. In fact, the only way I see any output when tailing is when I first execute tail -f myfifo and then foo > mfifo, but I don't want to restart foo (that's the whole point, otherwise I can just grep standard out directly and restart the process to grep for a different string). Does anyone know why this is happening or have a suggestion for how to achieve this?
This is happening because a fifo is a data stream. Once a piece of data is read from a FIFO, it's removed from the FIFO. In your case, the output of foo that's stored in myfifo is being read by the first tail -f that greps for "bar", leaving nothing for the second tail -f.
But you don't need to send the output of foo to a file at all (FIFO or otherwise). You can just send its output directly into tee and have it send that output to as many processes as you want. For example:
$ foo | tee >(grep -o bar) >(grep -o baz) >/dev/null
Or if you're just using grep, you can use -e as many times as you want on the output:
$ foo | grep -e bar -e baz
You can use a different grep syntax:
tail -f myfifo | grep -e 'pattern1' -e 'pattern2' -e 'pattern3'
I think you want both to happen at the same time on the same output stream, right?
Otherwise you could play with tee
tail -f myfifo | tee -a somefile | grep bar
grep foo somefile
I would like to pipe standard output of a program while keeping it on screen.
With a simple example (echo use here is just for illustration purpose) :
$ echo 'ee' | foo
ee <- the output I would like to see
I know tee could copy stdout to file but that's not what I want.
$ echo 'ee' | tee output.txt | foo
I tried
$ echo 'ee' | tee /dev/stdout | foo but it does not work since tee output to /dev/stdout is piped to foo
Here is a solution that works at on any Unix / Linux implementation, assuming it cares to follow the POSIX standard. It works on some non Unix environments like cygwin too.
echo 'ee' | tee /dev/tty | foo
Reference: The Open Group Base Specifications Issue 7
IEEE Std 1003.1, 2013 Edition, ยง10.1:
/dev/tty
Associated with the process group of that process, if any. It is
useful for programs or shell procedures that wish to be sure of
writing messages to or reading data from the terminal no matter how
output has been redirected. It can also be used for applications that
demand the name of a file for output, when typed output is desired and
it is tiresome to find out what terminal is currently in use. In each process, a synonym for the controlling terminal
Some environments like Google Colab have been reported not to implement /dev/tty while still having their tty command returning a usable device. Here is a workaround:
tty=$(tty)
echo 'ee' | tee $tty | foo
or with an ancient Bourne shell:
tty=`tty`
echo 'ee' | tee $tty | foo
Another thing to try is:
echo 'ee' | tee >(foo)
The >(foo) is a process substitution.
Edit:
To make a bit clearer, (.) here start a new child process to the current terminal, where the output is being redirected to.
echo ee | tee >(wc | grep 1)
# ^^^^^^^^^^^^^^ => child process
Except that any variable declarations/changes in child process do not reflect in the parent, there is very few of concern with regard to running commands in a child process.
Try:
$ echo 'ee' | tee /dev/stderr | foo
If using stderr is an option, of course.
Access to "/dev/stdout" is denied on some systems, but access to the user terminal is given by "/dev/tty".
Using "wc" for "foo", the above examples work OK (on linux, OSX, etc.) as:
% echo 'Hi' | tee /dev/tty | wc
Hi
1 1 3
To add a count at the bottom of a list of matching files, I use something like:
% ls [A-J]* | tee /dev/tty | wc -l
To avoid having to remember all this, I define aliases:
% alias t tee /dev/tty
% alias wcl wc -l
so that I can simply say:
% ls [A-J]* | t | wcl
POSTSCRIPT: For the younger set, who might titter at its pronunciation as "titty", I might add that "tty" was once the common
abbreviation for a "teletype" terminal, which used a roll of yellow
paper and had round keys that often stuck.
first you need to figure out the terminal associated with your screen (or whichever screen you want the output to display on):
tty
then you can tee the output to that terminal and pipe the other copy through your foo program:
echo ee | tee /dev/pty/2 | foo
As I build *nix piped commands I find that I want to see the output of one stage to verify correctness before building the next stage but I don't want to re-run each stage. Does anyone know of a program that will help with that? It would keep the output of the last stage automatically to use for any new stages. I usually do this by sending the result of each command to a temporary file (i.e. tee or run each command one at a time) but it would be nice for a program to handle this.
I envision something like a tabbed interface where each tab is labeled with each pipe command and selecting a tab shows the output (at least a hundred lines) of applying that command to to the previous result.
Use 'tee' to copy the intermediate results out to some file as well as pass them on to the next stage of the pipe, like so:
cat /var/log/syslog | tee /tmp/syslog.out | grep something | tee /tmp/grep.out | sed 's/foo/bar/g' | tee /tmp/sed.out | cat >>/var/log/syslog.cleaned
You can also use pipes if you need bidirectional communication (i.e. with netcat):
mknod backpipe p
nc -l -p 80 0<backpipe | tee -a inflow | nc localhost 81 | tee -a outflow 1>backpipe
(via)
There's also the "pv" command - available in debian / ubuntu repostitories which shows you the throughput of your pipes.
An example from the man page :
Transferring a file from another process and passing the expected size to pv:
cat file | pv -s 12345 | nc -w 1 somewhere.com 3000
tee(1) is your friend. It sends its input to both the specified file and stdout.
Stick it between your pipes. For example:
ls | tee /tmp/out1 | sort | tee /tmp/out2 | sed 's/foo/bar/g'