This question already has answers here:
Variables getting reset after the while read loop that reads from a pipeline
(3 answers)
Closed 7 years ago.
I have 3 lines in file /root/backuplist.txt.
The first echo prints perfectly, but the last one prints an empty line; I'm not sure why. Somehow, the $DIRS value is getting unset.
#!/bin/bash
cat /root/backuplist.txt |
while read line
do
DIRS+="$line "
echo $DIRS
done
echo $DIRS
Problem is use of pipe here, which is forking a sub-shell for your while loop and thus changes to DIRS are being made in the child shell that are not visible in the parent shell. Besides cat is unnecessary here.
Have it this way:
#!/bin/bash
while read -r line
do
DIRS+="$line "
echo "$DIRS"
done < /root/backuplist.txt
echo "$DIRS"
Related
This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 1 year ago.
I can't find my answer anywhere. The counter in my code doesn't work. Why and what to do ?
count=0;
for file1 in folder1;
do
cat $file1 | while read line
do
echo $line;
((count++));
done
done
echo "Count : ${count}";
When using a pipeline, the commands are executed in a subshell. Changes in a subshell don't propagate to the parent shell, so the counter is never incremented.
You don't need a pipeline here. Use a redirection instead:
count=0
while read line
do
echo $line;
((count++));
done < "$file1"
This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 2 years ago.
I'm trying to set the path for the PERL5LIB using a simple config file like this:
cat /Users/me/.local/bash.d/perl5lib_dirs | while read line
do
case ":$PERL5LIB:" in
*:$line:*) :;;
*) echo 'Adding path: '$line; PERL5LIB="$line:";;
esac
done
Apparently, since the loop is opened in a subshell, the value of the PERL5LIB variable is lost when the loop exits.
So how else can I accomplish this?
Rewrite the loop to read from the file with input redirection (thus avoiding the subshell):
while read line; do
case ":$PERL5LIB:" in
*:$line:*) :;;
*) echo 'Adding path: '$line; PERL5LIB="$line:";;
esac
done < /Users/me/.local/bash.d/perl5lib_dirs
Instead of running the game try sourcing the file.
for example
abc.sh
abc=123
sh abc.sh
echo $abc
source abc.sh
echo $abc
123
This question already has answers here:
Shell script while read loop executes only once
(6 answers)
While loop stops reading after the first line in Bash
(5 answers)
Bash script stops execution of ffmpeg in while loop - why?
(3 answers)
Closed 5 years ago.
I have a while loop that should iterate over a text file but stops at the first line and I can't figure out why. My code is below.
while read hadoop_accounts; do
if ! grep "no lock no remove"; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >> missing_no_lock_no_remove.txt
fi
done < hadoop_accounts.txt
When grep is run with no explicit redirection or file to read, it reads stdin. All of stdin. The same stdin your while read loop is reading from.
Thus, with all your stdin consumed by grep, there's nothing left for the next read command to consume.
The easy approach (and much better for performance) is to do the substring check internal to the shell, and not bother starting up a new copy of grep per line processed at all:
while IFS= read -r hadoop_accounts; do
if ! [[ $hadoop_accounts = *"no lock no remove"* ]]; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
fi
done < hadoop_accounts.txt 3>> missing_no_lock_no_remove.txt
Note also that we're only opening the output file once, not re-opening it every single time we want to write a single line.
If you really want to call grep over and over and over with only a single line of input each time, though, you can do that:
while IFS= read -r hadoop_accounts; do
if ! grep "no lock no remove" <<<"$hadoop_accounts"; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
fi
done < hadoop_accounts.txt 3>> missing_no_lock_no_remove.txt
Or, even better than either of the above, you can just run grep a single time over the entire input file and read its output in the loop:
while IFS= read -r hadoop_accounts; do
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
done < <(grep -v 'no lock no remove' <hadoop_accounts.txt) 3>>missing_flags.txt
This question already has answers here:
bash: piping output from a loop seems to change the scope within the loop - why?
(2 answers)
Closed 6 years ago.
I've read that you can redirect I/O into and out of various constructs in Bash (such as "if" and "while"). When I was trying this out, I noticed that using a "|" at the end of an if construct prevented any variables that I overwrote inside the if construct from taking effect; but if I use ">" instead, then the variable modifications take effect.
#!/bin/bash
VAR_0='Unmodified'
if true
then
VAR_0='Changed'
fi | cat
echo "VAR_0: $VAR_0"
###########
VAR_1='Unmodified'
if true
then
VAR_1='Changed'
fi > tmpFile
rm tmpFile
echo "VAR_1: $VAR_1"
Running Bash version 4.3.11 on 64-bit Linux produces the following output:
VAR_0: Unmodified
VAR_1: Changed
Note the only difference is how I'm redirecting stdout from the if construct. Why is the "|" preventing VAR_0 from being changed?
TL;DR
Check out Bash FAQ 24.
It answers your question along with provided examples!
You're creating a SubShell which in turn being discarded along with its variables copy of the parent shell.
Take the following common mistake as an example:
var=0
some-command | while read -r line; do
printf 'var value inside subshell is: %s\n' "$((++var))"
done
printf 'But in the parent shell it stays: %s\n' "$var"
Output:
var value inside subshell is: 1
But in the parent shell it stays: 0
When this behvior is not desired, the solution is often to use Process Substitution to keep the varaibles updated. So, in our case ( or as can be seen generally in Bash FAQ 24 ) we do the following:
var=0
while read -r line; do
printf 'var value inside subshell is: %s\n' "$((++var))"
done < <(some-command)
printf 'And it stays the same: %s\n' "$var"
Output:
var value inside subshell is: 1
And it stays the same: 1
This occurs because pipelines use subshells; by piping to cat, you've made the first if-then block execute in a subshell. Hence, VAR_0='Changed' is changed, but only in the subshell.
Try:
#!/bin/bash
VAR_0='Unmodified'
if true
then
VAR_0='Changed'
echo "VAR_0: $VAR_0"
fi | cat
echo "VAR_0: $VAR_0"
Redirecting to a file (">") does not create a subshell; thus that variable persists.
This question already has answers here:
While-loop subshell dilemma in Bash
(4 answers)
Closed 8 years ago.
I occured a problem and I can't find why does it run.
The follow codes is both used to count the line of file 'file.in' , but the first can't change the value of $line_count
The first code is :
#!/bin/bash
line_count=0
cat file.in | while read line; do
let ++line_count
done
echo $line_count
the second code is :
#!/bin/bash
line_count=0
while read line; do
let ++line_count
done < file.in
echo $line_count
Due to use of pipe your first code sample is executing while loop in a sub-shell hence changes made in line_count variable get lost after sub shell exits.