When I do this:
$ /bin/echo 123 | /bin/echo
I get no o/p. Why is that?
You ask why it doesn't work. In fact, it does work; it does exactly what you told it to do. Apparently that's not what you expected. I think you expected it to print 123, but you didn't actually say so.
(Note: "stdin" is standard input; "stdout" is standard output.)
/bin/echo 123 | /bin/echo
Here's what happens. The echo command is executed with the argument 123. It writes "123", followed by a newline, to its stdout.
stdout is redirected, via the pipe (|) to the stdin of the second echo command. Since the echo command ignores its stdin, the output of the first echo is quietly discarded. Since you didn't give the second echo command any arguments, it doesn't print anything. (Actually, /bin/echo with no arguments typically prints a single blank line; did you see that?)
Normally pipes (|) are used with filters, programs that read from stdin and write to stdout. cat is probably the simplest filter; it just reads its input and writes it, unchanged, to its output (which means that some-command | cat can be written as just some-command).
An example of a non-trivial filter is rev, which copies stdin to stdout while reversing the characters in each line.
echo 123 | rev
prints
321
rev is a filter; echo is not. echo does print to stdout, so it makes sense to have it on the left side of a pipe, but it doesn't read from stdin, so it doesn't make sense to use it on the right side of a pipe.
"echo" reads from command line, not standard input. So pipeline is not working here.
From bash manual:
echo [-neE] [arg ...]
Output the args, separated by spaces, terminated with a newline.
So your first echo did print "123" to stand output, however, the second echo didn't make use of it so "123" is dropped. Then an empty line is printed as if you run "echo".
You can use cat as Keith Thompson suggested:
echo 123|cat
/bin/echo 123 < /bin/echo
the pipe = concate
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-4.html
Pipes let you use the output of a program as the input of another one
Related
The description of STDIN of sh in SUSv4 2016 edition says
It shall not read ahead in such a manner that any characters intended to be read by the invoked command are consumed by the shell
I did an experiment to see it in action, by redirecting the following script file into sh -s:
#!/bin/sh
./rewind # a c program that reads up its stdin, prints it to stdout, before SEEK_SET its standard input to 0.
echo 1234-1234
And it keeps printing out "echo 1234-1234". (Had shell consumed whole block of file, it would only print out "1234-1234")
So obviously, shells (at least my ones) do read at line boundaries.
But however, when I examined the FreeBSD ash "input.c" source codes, it reads in BUFSIZ-byte blocks, and I don't understand how it preserves line boundaries.
What I want to know is: How does shells preserve line boundaries when their source codes apparently shows that they read in blocks?
Standard input isn't seekable in some cases, for example if it is redirected from a pipe or from a terminal. E.g. having a file called rew with a content:
#!/bin/bash
echo 123
perl -E 'seek(STDIN,0,0) or die "$!"' #the rewind
and using it as
bash -s < rew
prints
123
123
...
123
^C
so, when the STDIN is seekable it will work as expected, but trying the same from a pipe, such:
cat rew | bash -s #the cat is intentional here :)
will print
123
Illegal seek at -e line 1.
So, your c-program rewind should print an error, when it is trying to seek in un-seekable input.
You can demonstrate this (surprising) behaviour with the following script:
$ cat test.sh
cut -f 1
printf '%s\t%s\n' script file
If you pass it to standard input of your shell, line 2 onward should become the standard input of the cut command:
$ sh < test.sh
printf '%s\t%s\n' script file
If you instead run it normally and input literally foo, followed by Tab, bar, Enter, and Ctrl-d, you should instead see standard input being linked as normal:
$ sh test.sh
foo bar
foo
script file
Annotated:
$ sh test.sh # Your command
foo bar # Your input to `cut`
foo # The output of `cut`
script file # The output of `printf`
When I compiled the FreeBSD ash with NO_HISTORY macro defined to 1 in "shell.h", it consumes the whole file and outputs only 1234-1234 on my rewind testing program. Apparently, the FreeBSD ash relies on libedit to preserve IO line boundaries.
This question already has answers here:
How to pass command output as multiple arguments to another command
(5 answers)
Closed 5 years ago.
When I run the following Bash script, I would expect it to print Hello. Instead, it prints a blank line and exits.
echo 'Hello' | echo
Why doesn't piping output from echo to echo work?
echo prints all of its arguments. It does not read from stdin. So the second echo prints all of its arguments (none) and exits, ignoring the Hello on stdin.
For a program that reads its stdin and prints that to stdout, use cat:
$ echo Hello | cat
Hello
In this case the pipe you are using are more correctly known as anonymous pipes, because they have no name (there are also named pipes). Anonymous pipes only work between related processes, for example processes with the same parent.
Pipes are part of the IO system resulting from the C runtime-library. These streams are buffered (there is an exception) by default. Basically a pipe is just connecting the output buffer from one process to the input buffer of another.
The first three streams used (called file descriptors) are numbered 0, 1, and 2. The first, 0, is known as standard input, or stdin (the name used in C). By default this is connected to the keyboard, but it can be redirected either using the < symbol or the program name being on the right side of a pipe.
The second, 1, is known as standard output, or stdout. By default this is connected to the terminal screen, but can be redirected by using the > symbol or the program name being on the left side of a pipe.
So:
echo 'Hello' | echo
takes the standard output from echo and passes it to the standard input of echo. But echo does not read stdin! So nothing happens.
Filter programs process the filenames specified on the command-line. If no filenames are given then they read stdin. Examples include cat, grep, and sed, but not echo. For example:
echo 'Hello' | cat
will display 'Hello', and the cat is useless (it often is).
echo 'Hello' | cat file1
will ignore the output from echo and just display the contents of file1. Remember that stdin is only read if no filename is given.
What do you think this displays?
echo 'Hello' | cat < file1 file2
and why?
Finally, the third stream, 2, is called standard error, or stderr, and this one is unbuffered. It is ignored by pipes, because they only operate between stdin and stdout. However, you can redirect stderr to use stdout (see man dup2):
myprog 2>&1 | anotherprog
The 2>&1 means "redirect file descriptor 2 to the same place as fie descriptor 1".
The above is normal behaviour, however a program can override all that if it wants to. It could read from file descriptor 2, for example. I have omitted a lot of other detail, including other forms of redirection such as process substitution and here documents.
Piping can be done only for commands taking inputs from stdin. But echo does not takes from stdin. It will take input from argument and print it. So this wont work. Inorder to echo you can do something like echo $(echo 'hello')
It is because echo (both builtin and /bin/echo) don't read anything from stdin.
Use cat instead:
echo 'Hello' | cat
Hello
Or without pipes:
cat <<< 'Hello'
Why does cat exit a shell script, but only when it's fed by a pipe?
Case in point, take this shell script called "foobar.sh":
#! /bin/sh
echo $#
echo $#
cat $1
sed -e 's|foo|bar|g' $1
And a text file called "foo.txt" which contains only one line:
foo
Now if I type ./foobar.sh foo.txt on the command line, then I'll get this expected output:
1
foo.txt
foo
bar
However if I type cat foo.txt | ./foobar.sh then surprisingly I only get this output:
0
foo
I don't understand. If the number of arguments reported by $# is zero, then how can cat $1 still return foo? And, that being the case, why doesn't sed -e 's|foo|bar|g' $1 return anything since clearly $1 is foo?
This seems an awful lot like a bug, but I'm assuming it's magic instead. Please explain!
UPDATE
Based on the given answer, the following script gives the expected output, assuming a one-line foo.txt:
#! /bin/sh
if [ $# ]
then
yay=$(cat $1)
else
read yay
fi
echo $yay | cat
echo $yay | sed -e 's|foo|bar|g'
No, $1 is not "foo". $1 is
ie, undefined/nothing.
Unlike a programming language, variables in the shell are quite dumbly and literally replaced, and the resulting commands textually executed (well, sorta kinda). In this case, "cat $1" becomes just "cat ", which will take input from stdin. That's terribly convenient to your execution since you've kindly provided "foo" on stdin via your pipe!
See what's happening?
sed likewise will read from stdin, but is already on end of stream, so exits.
When you don't give an argument to cat, it reads from stdin. When $1 isn't given the cat $1 is the same as a simple cat, which reads the text you piped in (cat foo.txt).
Then the sed command runs, and same as cat, it reads from stdin because it has no filename argument. cat has already consumed all of stdin. There's nothing left to read, so sed quits without printing anything.
Is it possible to make changes to a line written to STDOUT in shell, similar to the way many programs such as scp do?
The point would be to allow me to essentially have a ticker, or a monitor of some sort, without it scrolling all over the screen.
You can manipulate the terminal with control characters and ANSI escape codes. For example \b returns the cursor one position back, and \r returns it to the beginning of the line. This can be used to make a simple ticker:
for i in $(seq 10)
do
echo -en "Progress... $i\r" # -e is needed to interpret escape codes
sleep 1
done
echo -e "\nDone."
With ANSI escape codes you can do even more, like clear part of the screen, jump to any position you want, and change the output color.
You can overwrite the last printed line by printing the \r character.
For instance this:
for i in `seq 1 10`; do
echo -n $i;
sleep 1;
echo -n -e "\r" ;
done
Will print 1 then update it with 2 and so on until 10.
You can do modify the output of stdout using another program in a pipeline. When you run the program you use | to pipe the input into the next program. The next program can do whatever it wants with the output. A general purpose program for modifying the output of a program is sed, or you could write something yourself that modifies the data from the previous program.
A shell program would be something like:
while read line; do
# do something with $line and output the results
done
so you can just:
the_original_program | the_above_program
i have a file called test which contains the word "hello" in it.
shouldn't
echo test | cat
output hello? since its taking the output from the echo test, which is test, as the input for cat. so essentially im doing cat test.
but the actual output is test, im really confused.
Your pipes sends test to cat as the input, not as the argument. You could do:
cat `echo test`
to control the argument to cat with echo.
echo prints its arguments. cat prints a file which is by default standard input. When you pipe echo's standard output is connected to cat's standard input.
Correct is simply cat test.
From cat --help
If no FILE or when FILE is -, read standard input.
In your case, cat reads from stdin, which is test and outputs that.
In some cases you might want the argument to be passed through the pipe. This is how you would do that:
echo test | xargs cat
which will output the contents of the file named "test".