As an example, taking one single program's stdout, obtaining two copies of it with tee and sending them both (one or preferably both able to be piped through other programs) back into vimdiff.
Bonus points if it can be done without having to create a file on disk.
I know how to direct input into a program that takes two inputs, like this
vimdiff <(curl http://google.com) <(curl http://archives.com/last_night/google.com)
and with tee for making two output streams
echo "abc" | tee >(sed 's/a/zzz/') >(sed 's/c/zzz/')
but I do not know how to connect the pipes back together into a diamond shape.
It's not so hard if you can use a fifo:
test -e fifo || mkfifo fifo
echo abc | tee >(sed s/a/zzz/ > fifo) | sed s/c/zzz/ | diff - fifo
Just as a side note, to have this work under ZSH an extra ">" is needed after tee (multios option should be set):
$ setopt multios
$ test -e fifo || mkfifo fifo
$ echo abc | tee > >(sed s/a/zzz/ > fifo) | sed s/c/zzz/ | diff - fifo
Related
I already know how to use tee with process substitution to send output to various commands, and stdout, eg
command0 | tee >(command1) >(command2)
With the above line, stdout will be composed of interleaved lines from command0, command1, and command2.
Is there a way to prevent tee from writing to stdout, without removing the output of any commands it pipes to? So for the example above, for stdout to only have output from command1 and command2?
Most answers relating to teeing without stdout are only writing directly to files, and recommend using something like this:
command0 | tee file1 file2 >/dev/null
But with process substitution, that would consume all output from the other commands too.
command0 | tee >(command1) >(command2) >/dev/null
Is there some way to tell tee not to print to stdout, or to only consume the output directly from tee?
Try this:
( command0 | tee >(command1 1>&3 ) | command2 ) 3>&1
It redirects the stdout of command1 to pipe 3, so that command2 sees only the original source. At end, you redirect pipe 3 to stdout again.
Use this to test it:
( echo test | tee >( sed 's/^/1 /' >&3 ) | sed 's/^/2 /' ) 3>&1
The output is unordered and in my case:
2 test
1 test
I have seen a comment and an answer that use an extra >, but don't really explain why it does what it does. It seems like it is redirecting output somewhere but all I can tell so far is that it does what I'm looking for. This works:
command0 | tee > >(command1) >(command2)
command0 | tee >(command1) > >(command2)
it appears not to matter where the extra > is, so long as it is before at least one of the arguments to tee. So this will not work:
command0 | tee >(command1) >(command2) >
Without knowing what this is called, and with no further leads, I can't explain further.
Construct the pipe to execute the following job.
"Output of ls should be displayed on the screen and from this output the lines
containing the word ‘poem’ should be counted and the count should be
stored in a file.”
If bash is allowed, use a process substitution as the receiver for tee
ls | tee >( grep -c poem > number.of.poetry.files)
Your attempt was close:
ls | tee /dev/tty | grep poem | wc -l >number_of_poems
The tee /dev/tty copies all ls output to the terminal. This satisfies the requirement that "Output of ls should be displayed on the screen." while also sending ls's output to grep's stdin.
This can be further simplified:
ls | tee /dev/tty | grep -c poem >number_of_poems
Note that neither of these solutions require bash. Both will work with lesser shells and, in particular, with dash which is the default /bin/sh under debian-like systems.
This sounds like a homework assignment :)
#!/bin/bash
ls
ls -l | grep -c poem >> file.txt
The first ls will display the output on the screen
The next line uses a series of pipes to output the number of files/directories containing "poem"
If there were 5 files with poem in them, file.txt would read 5. If file.txt already exists, the new count will be appended to the end. If you want overwrite file each time, change the line to read ls -l | grep -c poem > file.txt
I have a process that is writing to standard out, and I want to be able to monitor the output by grepping for various strings while running tail -f. One way to do this is to write to a normal file and then tail the file grepping for one string, then tail it grepping for another. However, I don't want to write to a file because this will fill up the disk, and I don't want to deal with the hassle of rotating log files.
I thought I could achieve this using fifos. E.g.,
mkfifo myfifo
foo > myfifo &
tail -f myfifo | grep bar
tail -f myfifo | grep baz
Unfortunately, this doesn't seem to work. In fact, the only way I see any output when tailing is when I first execute tail -f myfifo and then foo > mfifo, but I don't want to restart foo (that's the whole point, otherwise I can just grep standard out directly and restart the process to grep for a different string). Does anyone know why this is happening or have a suggestion for how to achieve this?
This is happening because a fifo is a data stream. Once a piece of data is read from a FIFO, it's removed from the FIFO. In your case, the output of foo that's stored in myfifo is being read by the first tail -f that greps for "bar", leaving nothing for the second tail -f.
But you don't need to send the output of foo to a file at all (FIFO or otherwise). You can just send its output directly into tee and have it send that output to as many processes as you want. For example:
$ foo | tee >(grep -o bar) >(grep -o baz) >/dev/null
Or if you're just using grep, you can use -e as many times as you want on the output:
$ foo | grep -e bar -e baz
You can use a different grep syntax:
tail -f myfifo | grep -e 'pattern1' -e 'pattern2' -e 'pattern3'
I think you want both to happen at the same time on the same output stream, right?
Otherwise you could play with tee
tail -f myfifo | tee -a somefile | grep bar
grep foo somefile
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
osx/linux: pipes into two processes?
Is there a way to pipe the output from one command into the input of two other commands, running them simultaneously?
Something like this:
$ echo 'test' |(cat) |(cat)
test
test
The reason I want to do this is that I have a program which receives an FM radio signal from a USB SDR device, and outputs the audio as raw PCM data (like a .wav file but with no header.) Since the signal is not music but POCSAG pager data, I need to pipe it to a decoder program to recover the pager text. However I also want to listen to the signal so I know whether any data is coming in or not. (Otherwise I can't tell if the decoder is broken or there's just no data being broadcast.) So as well as piping the data to the pager decoder, I also need to pipe the same data to the play command.
Currently I only know how to do one - either pipe it to the decoder and read the data in silence, or pipe it to play and hear it without seeing any decoded text.
How can I pipe the same data to both commands, so I can read the text and hear the audio?
I can't use tee as it only writes the duplicated data to a file, but I need to process the data in real-time.
It should be ok if you use both tee and mkfifo.
mkfifo pipe
cat pipe | (command 1) &
echo 'test' | tee pipe | (command 2)
Recent bash present >(command) syntax:
echo "Hello world." | tee >(sed 's/^/1st: /') >(sed 's/^/2nd cmd: /') >/dev/null
May return:
2nd cmd: Hello world.
1st: Hello world.
download somefile.ext, save them, compute md5sum and sha1sum:
wget -O - http://somewhere.someland/somepath/somefile.ext |
tee somefile.ext >(md5sum >somefile.md5) | sha1sum >somefile.sha1
or
wget -O - http://somewhere.someland/somepath/somefile.ext |
tee >(md5sum >somefile.md5) >(sha1sum >somefile.sha1) >somefile.ext
Old answer
There is a way to do that via unnamed pipe (tested under linux):
(( echo "hello" |
tee /dev/fd/5 |
sed 's/^/1st occure: /' >/dev/fd/4
) 5>&1 |
sed 's/^/2nd command: /'
) 4>&1
give:
2nd command: hello
1st occure: hello
This sample will let you download somefile.ext, save them, compute his md5sum and compute his sha1sum:
(( wget -O - http://somewhere.someland/somepath/somefile.ext |
tee /dev/fd/5 |
md5sum >/dev/fd/4
) 5>&1 |
tee somefile.ext |
sha1sum
) 4>&1
Maybe take a look at tee command. What it does is simply print its input to a file, but it also prints its input to the standard output. So something like:
echo "Hello" | tee try.txt | <some_command>
Will create a file with content "Hello" AND also let "Hello" (flow through the pipeline) end up as <some_command>'s STDIN.
I would like to pipe standard output of a program while keeping it on screen.
With a simple example (echo use here is just for illustration purpose) :
$ echo 'ee' | foo
ee <- the output I would like to see
I know tee could copy stdout to file but that's not what I want.
$ echo 'ee' | tee output.txt | foo
I tried
$ echo 'ee' | tee /dev/stdout | foo but it does not work since tee output to /dev/stdout is piped to foo
Here is a solution that works at on any Unix / Linux implementation, assuming it cares to follow the POSIX standard. It works on some non Unix environments like cygwin too.
echo 'ee' | tee /dev/tty | foo
Reference: The Open Group Base Specifications Issue 7
IEEE Std 1003.1, 2013 Edition, §10.1:
/dev/tty
Associated with the process group of that process, if any. It is
useful for programs or shell procedures that wish to be sure of
writing messages to or reading data from the terminal no matter how
output has been redirected. It can also be used for applications that
demand the name of a file for output, when typed output is desired and
it is tiresome to find out what terminal is currently in use. In each process, a synonym for the controlling terminal
Some environments like Google Colab have been reported not to implement /dev/tty while still having their tty command returning a usable device. Here is a workaround:
tty=$(tty)
echo 'ee' | tee $tty | foo
or with an ancient Bourne shell:
tty=`tty`
echo 'ee' | tee $tty | foo
Another thing to try is:
echo 'ee' | tee >(foo)
The >(foo) is a process substitution.
Edit:
To make a bit clearer, (.) here start a new child process to the current terminal, where the output is being redirected to.
echo ee | tee >(wc | grep 1)
# ^^^^^^^^^^^^^^ => child process
Except that any variable declarations/changes in child process do not reflect in the parent, there is very few of concern with regard to running commands in a child process.
Try:
$ echo 'ee' | tee /dev/stderr | foo
If using stderr is an option, of course.
Access to "/dev/stdout" is denied on some systems, but access to the user terminal is given by "/dev/tty".
Using "wc" for "foo", the above examples work OK (on linux, OSX, etc.) as:
% echo 'Hi' | tee /dev/tty | wc
Hi
1 1 3
To add a count at the bottom of a list of matching files, I use something like:
% ls [A-J]* | tee /dev/tty | wc -l
To avoid having to remember all this, I define aliases:
% alias t tee /dev/tty
% alias wcl wc -l
so that I can simply say:
% ls [A-J]* | t | wcl
POSTSCRIPT: For the younger set, who might titter at its pronunciation as "titty", I might add that "tty" was once the common
abbreviation for a "teletype" terminal, which used a roll of yellow
paper and had round keys that often stuck.
first you need to figure out the terminal associated with your screen (or whichever screen you want the output to display on):
tty
then you can tee the output to that terminal and pipe the other copy through your foo program:
echo ee | tee /dev/pty/2 | foo