How can I define a function bar in a Bash script such that echo foo | bar in that script will read an input from the script's stdin and not the pipe? In other words, if bar is:
function bar(){
read ZOO
}
I want it to wait for my input rather than setting ZOO to "foo"
The idea of a pipe is to connect stdout of the process on the left side of the pipe with stdin of the process on the right side of the pipe. So foo is in this case piped into stdin of the function bar().
If you want to read explicitly from the current terminal, then pass the special device /dev/tty to stdin of read:
function bar() {
read ZOO < /dev/tty
}
Related
In a Bash loop, I have a mix of commands that append directly to a file, and commands that output to stdout, and then the whole loop is appended to the same file, like this:
for i in {1..3}; do
echo foo >> mylog
echo bar
done >> mylog
The result is that the lines in the mylog file are interleaved, as if the output from the echo bar is appended to mylog in real time:
foo
bar
foo
bar
foo
bar
However, if I pipe the output of the loop through any command (I use the trivial cat command here for simplicity, but it could be any other command where I take each input line and output a processed line), and then pipe it to the file, like this:
for i in {1..3}; do
echo foo >> mylog
echo bar
done | cat >> mylog
the output is not interleaved, with all the bars printed after all the foos, as if it waited until all the output was done from the cat before appending the lines to the file:
foo
foo
foo
bar
bar
bar
My questions are:
Why does this happen?
How can I modify my second code (assuming I still want to have each line processed through a custom command) so that the output is interleaved?
Here is what happens:
loop repeat 3 times:
echo foo >> mylog: Open file mylog, append foo, close file mylog.
echo bar: Append bar to the already opened stdout file handle.
whole loop as a single block:
done |: redirect the whole stdout for the loop to the pipe which has a small buffer for until the receiving process is ready to listen and read stdin data from the pipe.
cat >> mylog: The sub-process running the cat command, open file mylog, append what has been read from stdin, close file mylog.
Technically it is:
Repeat 3 times:
Inefficiently append foo to mylog.
Append bar to buffer.
Content of mylog at this stage:
foo
foo
foo
Then:
Read buffer content and append it all to mylog
Content of mylog at this stage:
foo
foo
foo
bar
bar
bar
If more content was generated in the loop, it would probably block or interleave content once the cat sub-process starts appending concurrently to the file.
I have tow bash files.
File Child
echo some text | tee /dev/tty
File Parent
my_variable=$(./Child)
Executing ./Child prints "some text", executing ./Parent prints "some text" twice. So far so good.
However, if I modify Child file to:
a=$(echo some text | tee /dev/tty)
Then Child still prints "some text" but Parent only prints once, and my_variable is an empty variable.
Is there a way to capture the value from /dev/tty on the parent?
I'm trying to find a way to do something like this:
# script.sh:
cmd0
set_prepend "some text"
cmd1
cmd2
cmd3
unset_prepend
cmd4
Then whatever stdout generated by cmd1, 2 and 3, every line will be prepended by "some text". There is no relationship between the commands, and the commands can be anything (ls, cat, awk, whatever):
$ script.sh
cmd0 line1
...
cmd0 lineN0
some text cmd1 line1
some text ...
some text cmd1 lineN1
some text cmd2 line1
some text ...
some text cmd2 lineN2
some text cmd3 line1
some text ...
some text cmd3 lineN3
cmd4 line1
...
cmd4 lineN4
The only way I can think of is far from elegant:
script.sh | prepender
and for each line received by prepender, it checks the existence of a file; if the file exists, the contents are the text to prepend; and set_prepend would create that file and unset_prepend would remove it. However buffering would interfere with this, so it would have to be turned off, and I'm not sure how to garantee that a line going to stdout will be processed by prepender before the next script cmd is execute (otherwise race condition possible).
Use exec to redirect output to a pipe. You'll need to save the old stdout in another FD so you can restore it later.
set_prepend() {
exec 3>&1 | sed "s/^/$1 /"
}
unset_prepend() {
exec >&3 3>&-
}
Can someone please explain how these process substitutions are working.
(echo "YES")> >(read str; echo "1:${str}:first";)> >(read sstr; echo "2:$sstr:two")> >(read ssstr; echo "3:$ssstr:three")
Output
1:2:3:YES:three:two:first
I've figured out, that the 'ssstr'-Substitution got FD 60, sstr FD 61 and str FD 62. (right to left)
But how is (echo "YES") connected to input of FD60, and output of FD60 with input FD61 and so on and finally FD62 prints out on Terminal ?
All against the direction of the two redirections.
How are they nested, and how connected ?
Makes me crazy.
Ty.
First off, don't actually write code like this :)
The process substitutions are the constructs >(...). The (...)> isn't a specific construct; it's just a subshell followed by an output redirection.
This example is a single command (echo "YES") followed by three output redirections
> >(read str; echo "1:${str}:first";)
> >(read sstr; echo "2:$sstr:two")
> >(read ssstr; echo "3:$ssstr:three")
The last one is the one that is actually applied to the original command; something like echo word >foo >bar >baz would create all three files, but the output of echo would only be written to baz.
Similarly, all three process substitutions start a new process, but the output YES is only written to the last one. So read ssstr gets its input from echo YES.
At this point, I think you are seeing what amounts to undefined behavior. The three process substitutions run in the reverse order they were created, as if the OS pushed each process on to a stack as the next one is created, then schedules them by popping them off the stack, but I don't think that order is guaranteed by anything.
In each case, though, the standard input of each process substitution is fixed to the standard output of the command, which is whichever other process substitution just ran. In other words, the command ends up being similar to
echo YES | {
read ssstr
echo "3:$ssstr:three" | {
read sstr
echo "2:$sstr:two" | {
read str
echo "1:$str:one"
}
}
}
I have some set of bash functions which output some information:
find-modelname-in-epson-ppds
find-modelname-in-samsung-ppds
find-modelname-in-hp-ppds
etc ...
I've been writing functions which read output and filter it:
function filter-epson {
find-modelname-in-epson-ppds | sed <bla-blah-blah>
}
function filter-hp {
find-modelname-in-hp-ppds | sed <the same bla-blah-blah>
}
etc ...
But the I thought that it would be better do something like this:
function filter-general {
(somehow get input) | sed <bla-blah-blah>
}
and then call in another high-level functions:
function high-level-func {
# outputs filtered information
find-modelname-in-hp/epson/...-ppds | filter-general
}
How can I achieve that with the best bash practices?
If the question is How do I pass stdin to a bash function?, then the answer is:
Shellscript functions take stdin the ordinary way, as if they were commands or programs. :)
input.txt:
HELLO WORLD
HELLO BOB
NO MATCH
test.sh:
#!/bin/sh
myfunction() {
grep HELLO
}
cat input.txt | myfunction
Output:
hobbes#metalbaby:~/scratch$ ./test.sh
HELLO WORLD
HELLO BOB
Note that command line arguments are ALSO handled in the ordinary way, like this:
test2.sh:
#!/bin/sh
myfunction() {
grep "$1"
}
cat input.txt | myfunction BOB
Output:
hobbes#metalbaby:~/scratch/$ ./test2.sh
HELLO BOB
To be painfully explicit that I'm piping from stdin, I sometimes write
cat - | ...
A very simple means to get stdin into a variable is to use read. By default, it reads file descriptor "0", i.e. stdin i.e., /dev/stdin.
Example Function:
input(){ local in; read in; echo you said $in; }
Example implementation:
echo "Hello World" | input
Result:
you said Hello World
Additional info
You don't need to declare a variable as being local, of course. I just included that for the sake of good form. Plain old read in does what you need.
So you understand how read works, by default it reads data off the given file descriptor (or implicit stdin) and blocks until it encounters a newline. Much of the time, you'll find that will implicitly be attached to your input, even if you weren't aware of it. If you have a function that seems to hang with this mechanism just keep this detail in mind (there are other ways of using read to deal with that...).
More robust solutions
Adding on to the basic example, here's a variation that lets you pass the input via a stdin OR an argument:
input()
{
local in=$1; if [ -z "$in" ]; then read in; fi
echo you said $in
}
With that tweak, you could ALSO call the function like:
input "Hello World"
How about handling an stdin option plus other arguments? Many standard nix utilities, especially those which typically work with stdin/stdout adhere to the common practice of treating a dash - to mean "default", which contextually means either stdin or stdout, so you can follow the convention, and treat an argument specified as - to mean "stdin":
input()
{
local a=$1; if [ "$a" == "-" ]; then read a; fi
local b=$2
echo you said $a $b
}
Call this like:
input "Hello" "World"
or
echo "Hello" | input - "World"
Going even further, there is actually no reason to only limit stdin to being an option for only the first argument! You might create a super flexible function that could use it for any of them...
input()
{
local a=$1; if [ "$a" == "-" ]; then read a; fi
local b=$2; if [ "$b" == "-" ]; then read b; fi
echo you said $a $b
}
Why would you want that? Because you could formulate, and pipe in, whatever argument you might need...
myFunc | input "Hello" -
In this case, I pipe in the 2nd argument using the results of myFunc rather than the only having the option for the first.
Call sed directly. That's it.
function filter-general {
sed <bla-blah-blah>
}