Are these two bash snippets equivalent and if not which is better? - bash

Question

I am interested on which is better in the sense of either being more flexible/general/extensible or faster or less error prone or uses less memory.
Snippet1:
grep '^+[^+]' /tmp/p0rn.lst | while read x;do
wget $x
done
Snippet2:
while read x;do
wget $x
done < <(grep '^+[^+]' /tmp/p0rn.lst)

Answer 1

The first uses a pipeline, the second process substitution. Process substitution is usually better.
Piping input to a loop means the loop runs in a subshell. Variable changes inside a subshell are lost when the subshell finishes, i.e. when the loop ends.
For example, if you wanted to count the number of URLs processed by incrementing a variable each iteration you'd have to use process substitution:
count=0
while read x; do
wget "$x"
((++count))
done < <(grep '^+[^+]' /tmp/p0rn.lst)
echo "processed $count urls"
If you used a pipeline it would always print "processed 0 urls":
count=0
grep '^+[^+]' /tmp/p0rn.lst | while read x; do
wget "$x"
((++count))
done
# Doesn't work!
echo "processed $count urls"

Answer 2

In addition to #john-kugelman 's answer I would like to point out that the bash holds a variable called PIPESTATUS which is an array of all the exit codes of the last pipe:
(exit 1) | (exit 2) | (exit 3) | (exit 4); echo "${PIPESTATUS[#]}"
This will print
1 2 3 4
There is no similar mechanism if this is rewritten using process substitutions:
(exit 4) < <(
(exit 3) < <(
(exit 2) < <(
(exit 1)
)
)
)
In this case, the inner exit codes are just lost, unless they are explicitly stored somehow (e. g. dumped in a file or similar).
The overall exit code of a pipe is the exit code of the last pipe element, on default, so true | false is like false. In many cases this is desirable, e. g.
if curl "$url" | grep -q "searchterm"
then
# do something in case the searchterm was found
In this usage the exit value of the grep clearly is what is wanted.
There are other uses, however, in which you would rather react on the exit value of the left element:
curl "$url" | sed 's/password.*/###/g' > x || { # incorrect usage!
echo "Curl failed." 1>&2
}
This will not work because the pipe's exit value is the exit value of sed (which in this case is not the relevant part). Moving the check deeper inside can solve the issue:
(
curl "$url" || echo "Curl failed." 1>&2
) | sed 's/password.*/###/g' > x
Or also:
sed 's/password.*/###/g' < <(
curl "$url" || echo "Curl failed." 1>&2
) > x
But neither allows e. g. exiting the surrounding shell properly (because the check is done in a nested scope).
You can, however, use the pipefail option of the shell in which case a pipe's exit value is the exit value of the rightmost non-zero exit value:
set -o pipefail
(exit 1) | (exit 2) | (exit 3) | (exit 0); echo $?
This will print 3 instead of 0. Applied to our last example:
set -o pipefail
if ! curl "$url" | sed 's/password.*/###/g' > x
then
echo "Curl failed (or sed)." 1>&2
# Now we can even exit this shell here if we like.
fi
(You might want to scope the pipefail option to avoid influencing the rest of the script.)
You can, however, achieve a similar thing using process substitutions for the output:
if ! curl "$url" > >(sed 's/password.*/###/g' > x)
then
echo "Curl failed." 1>&2
# Now we can even exit this shell here if we like.
fi
But this will ignore failures in the right commands of the pipe (sed in this case). Using pipefail you can react on failures in any of the piped commands. There is no (simple and recommendable) way of achieving this using process substitutions.