This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 4 years ago.
dpkg --list |grep linux-image |grep "ii " | while read line
do
arr=(${line})
let i=i+1
_constr+="${arr[2]} "
done
echo $i
echo ${_constr}
The echo statements outside of the loop do not display the expected variables.
How should I make the contents of the variable propagate outside the loop?
The problem is the pipe, not the loop. Try it this way
let i=0
arr=()
_constr=
while read -r line ; do
arr=("${line}")
let i=i+1
_constr+="${arr[2]} "
done < <(dpkg --list | grep linux-image | grep 'ii ')
echo "$i"
echo "${_constr}"
Pipes are executed in a subshell, as noted by Blagovest in his comment. Using process substitution instead (this is the < <(commands) syntax) keeps everything in the same process, so changes to global variables are possible.
Incidentally, your pipeline could be improved as well
dpkg --list | grep '^ii.*linux-image'
One less invocation of grep to worry about.
This somewhat by-passes your question (and it's a good question), but you can achieve the same results using simply:
_constr=($(dpkg --list | awk '/^ii.*linux-image/{print $2}'))
The ($(cmd)) construct initialises a bash array using the output of the command within.
[me#home]$ echo ${_constr[*]}
linux-image-2.6.35-22-generic linux-image-2.6.35-28-generic linux-image-generic
[me#home]$ echo ${_constr[2]}
linux-image-generic
and you can get the number of elements using ${#_constr[*]}.
[me#home]$ echo ${#_constr[*]}
3
Alternatively, you can move the echo statements inside the subshell:
dpkg --list |grep linux-image |grep "ii " | (
let i=0
declare -a arr
while read line
do
arr=(${line})
let i=i+1
_constr+="${arr[2]} "
done
echo $i
echo ${_constr}
)
Note the insertion of the parenthesis to explicitly define where the subshell begins and ends.
Related
Bash allows to use: cat <(echo "$FILECONTENT")
Bash also allow to use: while read i; do echo $i; done </etc/passwd
to combine previous two this can be used: echo $FILECONTENT | while read i; do echo $i; done
The problem with last one is that it creates sub-shell and after the while loop ends variable i cannot be accessed any more.
My question is:
How to achieve something like this: while read i; do echo $i; done <(echo "$FILECONTENT") or in other words: How can I be sure that i survives while loop?
Please note that I am aware of enclosing while statement into {} but this does not solves the problem (imagine that you want use the while loop in function and return i variable)
The correct notation for Process Substitution is:
while read i; do echo $i; done < <(echo "$FILECONTENT")
The last value of i assigned in the loop is then available when the loop terminates.
An alternative is:
echo $FILECONTENT |
{
while read i; do echo $i; done
...do other things using $i here...
}
The braces are an I/O grouping operation and do not themselves create a subshell. In this context, they are part of a pipeline and are therefore run as a subshell, but it is because of the |, not the { ... }. You mention this in the question. AFAIK, you can do a return from within these inside a function.
Bash also provides the shopt builtin and one of its many options is:
lastpipe
If set, and job control is not active, the shell runs the last command of a pipeline not executed in the background in the current shell environment.
Thus, using something like this in a script makes the modfied sum available after the loop:
FILECONTENT="12 Name
13 Number
14 Information"
shopt -s lastpipe # Comment this out to see the alternative behaviour
sum=0
echo "$FILECONTENT" |
while read number name; do ((sum+=$number)); done
echo $sum
Doing this at the command line usually runs foul of 'job control is not active' (that is, at the command line, job control is active). Testing this without using a script failed.
Also, as noted by Gareth Rees in his answer, you can sometimes use a here string:
while read i; do echo $i; done <<< "$FILECONTENT"
This doesn't require shopt; you may be able to save a process using it.
Jonathan Leffler explains how to do what you want using process substitution, but another possibility is to use a here string:
while read i; do echo "$i"; done <<<"$FILECONTENT"
This saves a process.
This function makes duplicates $NUM times of jpg files (bash)
function makeDups() {
NUM=$1
echo "Making $1 duplicates for $(ls -1 *.jpg|wc -l) files"
ls -1 *.jpg|sort|while read f
do
COUNT=0
while [ "$COUNT" -le "$NUM" ]
do
cp $f ${f//sm/${COUNT}sm}
((COUNT++))
done
done
}
Bash allows to use: cat <(echo "$FILECONTENT")
Bash also allow to use: while read i; do echo $i; done </etc/passwd
to combine previous two this can be used: echo $FILECONTENT | while read i; do echo $i; done
The problem with last one is that it creates sub-shell and after the while loop ends variable i cannot be accessed any more.
My question is:
How to achieve something like this: while read i; do echo $i; done <(echo "$FILECONTENT") or in other words: How can I be sure that i survives while loop?
Please note that I am aware of enclosing while statement into {} but this does not solves the problem (imagine that you want use the while loop in function and return i variable)
The correct notation for Process Substitution is:
while read i; do echo $i; done < <(echo "$FILECONTENT")
The last value of i assigned in the loop is then available when the loop terminates.
An alternative is:
echo $FILECONTENT |
{
while read i; do echo $i; done
...do other things using $i here...
}
The braces are an I/O grouping operation and do not themselves create a subshell. In this context, they are part of a pipeline and are therefore run as a subshell, but it is because of the |, not the { ... }. You mention this in the question. AFAIK, you can do a return from within these inside a function.
Bash also provides the shopt builtin and one of its many options is:
lastpipe
If set, and job control is not active, the shell runs the last command of a pipeline not executed in the background in the current shell environment.
Thus, using something like this in a script makes the modfied sum available after the loop:
FILECONTENT="12 Name
13 Number
14 Information"
shopt -s lastpipe # Comment this out to see the alternative behaviour
sum=0
echo "$FILECONTENT" |
while read number name; do ((sum+=$number)); done
echo $sum
Doing this at the command line usually runs foul of 'job control is not active' (that is, at the command line, job control is active). Testing this without using a script failed.
Also, as noted by Gareth Rees in his answer, you can sometimes use a here string:
while read i; do echo $i; done <<< "$FILECONTENT"
This doesn't require shopt; you may be able to save a process using it.
Jonathan Leffler explains how to do what you want using process substitution, but another possibility is to use a here string:
while read i; do echo "$i"; done <<<"$FILECONTENT"
This saves a process.
This function makes duplicates $NUM times of jpg files (bash)
function makeDups() {
NUM=$1
echo "Making $1 duplicates for $(ls -1 *.jpg|wc -l) files"
ls -1 *.jpg|sort|while read f
do
COUNT=0
while [ "$COUNT" -le "$NUM" ]
do
cp $f ${f//sm/${COUNT}sm}
((COUNT++))
done
done
}
This question already has answers here:
bash: piping output from a loop seems to change the scope within the loop - why?
(2 answers)
Closed 6 years ago.
I've read that you can redirect I/O into and out of various constructs in Bash (such as "if" and "while"). When I was trying this out, I noticed that using a "|" at the end of an if construct prevented any variables that I overwrote inside the if construct from taking effect; but if I use ">" instead, then the variable modifications take effect.
#!/bin/bash
VAR_0='Unmodified'
if true
then
VAR_0='Changed'
fi | cat
echo "VAR_0: $VAR_0"
###########
VAR_1='Unmodified'
if true
then
VAR_1='Changed'
fi > tmpFile
rm tmpFile
echo "VAR_1: $VAR_1"
Running Bash version 4.3.11 on 64-bit Linux produces the following output:
VAR_0: Unmodified
VAR_1: Changed
Note the only difference is how I'm redirecting stdout from the if construct. Why is the "|" preventing VAR_0 from being changed?
TL;DR
Check out Bash FAQ 24.
It answers your question along with provided examples!
You're creating a SubShell which in turn being discarded along with its variables copy of the parent shell.
Take the following common mistake as an example:
var=0
some-command | while read -r line; do
printf 'var value inside subshell is: %s\n' "$((++var))"
done
printf 'But in the parent shell it stays: %s\n' "$var"
Output:
var value inside subshell is: 1
But in the parent shell it stays: 0
When this behvior is not desired, the solution is often to use Process Substitution to keep the varaibles updated. So, in our case ( or as can be seen generally in Bash FAQ 24 ) we do the following:
var=0
while read -r line; do
printf 'var value inside subshell is: %s\n' "$((++var))"
done < <(some-command)
printf 'And it stays the same: %s\n' "$var"
Output:
var value inside subshell is: 1
And it stays the same: 1
This occurs because pipelines use subshells; by piping to cat, you've made the first if-then block execute in a subshell. Hence, VAR_0='Changed' is changed, but only in the subshell.
Try:
#!/bin/bash
VAR_0='Unmodified'
if true
then
VAR_0='Changed'
echo "VAR_0: $VAR_0"
fi | cat
echo "VAR_0: $VAR_0"
Redirecting to a file (">") does not create a subshell; thus that variable persists.
I am trying to push some variables into a bash array. For some reasons I cant understand, my script find the variable templates_age directly but not in the loop.
You can try the code on BASH Shell Online.
script:
templates_age="42"
templates_name="foo"
echo "age=${templates_age}"
echo "name=${templates_name}"
readarray GREPPED < <($(compgen -A variable | grep "templates_"))
for item in "${GREPPED[#]}"
do
echo "${item}"
done
output:
age=42
name=foo
./main.sh: line 32: templates_age: command not found
I tried different kind of echo "${item}" without success.
To convert from grep to array, I am using this logic.
To correctly populate array from a command's output use process substitution without $(...) which is called command substitution:
readarray -t grepped < <(compgen -A variable | grep "templates_")
Also note use of -t to trim newlines.
Full script:
templates_age="42"
templates_name="foo"
echo "age=${templates_age}"
echo "name=${templates_name}"
readarray -t grepped < <(compgen -A variable | grep "templates_")
declare -p grepped
for item in "${grepped[#]}"
do
printf "%s=%s\n" "${item}" "${!item}"
done
I'm not sure why you want to use compgen and grep here. Wouldn't this be enough?
for item in "${!templates_#}"; do
printf '%s=%s\n' "$item" "${!item}"
done
If you really want to populate an array, it's as simple as:
grepped=( "${!templates_#}" )
See Shell Parameter Expansion in the reference manual.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Unix for loop help please?
I am trying to list the names of all the files in a directory separated by a blank line. I was using a for loop but after trying a few examples, none really work by adding blank lines in between. Any ideas?
Is there any command which outputs only the first line of a file in unix? How could I only display the first line?
for i in ls
do
echo "\n" && ls -l
done
for i in ls
do
echo "\n"
ls
done
Use head or sed 1q to display only the first line of a file. But in this case, if I'm understanding you correctly, you want to capture and modify the output of ls.
ls -l | while read f; do
printf '%s\n\n' "$f"
# alternately
echo "$f"; echo
done
IFS="
"
for i in $(ls /dir/name/here/or/not)
do
echo -e "$i\n"
done
To see the first part of a file use head and for the end of a file use tail (of course). The command head -n 1 filename will display the first line. Use man head to get more options. (I know how that sounds).
Use shell expansion instead of ls to list files.
for file in *
do
echo "$file"
echo
if [ -f "$file" ];then
read firstline < "$file"
echo "$firstline" # read first line
fi
done