bash: accessing global variables from a command pipeline [duplicate] - bash

This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 6 years ago.
I want to fill an associative array in bash in somewhat non-trivial setup. I have a pipeline of commands to produce the required input for the array.
Here is a minimal/toy example:
declare -A mapping
seq 10 | while read i; do
key="key_$i"
val="val_$i"
echo "mapping[$key]=$val"
mapping["${key}"]="${val}"
done
echo "${mapping["key_1"]}"
echo "${mapping["key_2"]}"
In this example mapping is changed inside while, but these changes do not propagate into the global namespace. I think this is because while works inside a separate subshell, thus namespaces have diverged.
In order to avoid (what I suggest) the problem with subshells, I came up with the following:
declare -A mapping
while read i; do
key="key_$i"
val="val_$i"
echo "mapping[$key]=$val"
mapping["${key}"]="${val}"
done < <(seq 10)
echo "${mapping["key_1"]}"
echo "${mapping["key_2"]}"
Thus, the generation part explicitly goes into a subshell, while the while loop left at the top-level alone. The construction works.
My questions are: is there any better way to accomplish my goal? And, is my suggestion about subshells correct? If so, why bash uses a subshell in the first case, but not in the second?
EDIT: after little more digging, the question mostly is a duplicate of this one. A good list of options to handle the issue could be found at http://mywiki.wooledge.org/BashFAQ/024

Not sure if this is a better way than your second code snippet, but a way to solve the first one is to use sub shell { ... } right after the pipe:
declare -A mapping
seq 10 | {
while read i; do
key="key_$i"
val="val_$i"
echo "mapping[$key]=$val"
mapping["${key}"]="${val}"
done
echo "${mapping["key_1"]}"
echo "${mapping["key_2"]}"
}

Related

Extracting git commit information in GitHub action workflow- use of '$' symbol [duplicate]

This question already has answers here:
Backticks vs braces in Bash
(3 answers)
Brackets ${}, $(), $[] difference and usage in bash
(1 answer)
Closed 4 years ago.
I have two questions and could use some help understanding them.
What is the difference between ${} and $()? I understand that ()
means running command in separate shell and placing $ means passing
the value to variable. Can someone help me in understanding
this? Please correct me if I am wrong.
If we can use for ((i=0;i<10;i++)); do echo $i; done and it works fine then why can't I use it as while ((i=0;i<10;i++)); do echo $i; done? What is the difference in execution cycle for both?
The syntax is token-level, so the meaning of the dollar sign depends on the token it's in. The expression $(command) is a modern synonym for `command` which stands for command substitution; it means run command and put its output here. So
echo "Today is $(date). A fine day."
will run the date command and include its output in the argument to echo. The parentheses are unrelated to the syntax for running a command in a subshell, although they have something in common (the command substitution also runs in a separate subshell).
By contrast, ${variable} is just a disambiguation mechanism, so you can say ${var}text when you mean the contents of the variable var, followed by text (as opposed to $vartext which means the contents of the variable vartext).
The while loop expects a single argument which should evaluate to true or false (or actually multiple, where the last one's truth value is examined -- thanks Jonathan Leffler for pointing this out); when it's false, the loop is no longer executed. The for loop iterates over a list of items and binds each to a loop variable in turn; the syntax you refer to is one (rather generalized) way to express a loop over a range of arithmetic values.
A for loop like that can be rephrased as a while loop. The expression
for ((init; check; step)); do
body
done
is equivalent to
init
while check; do
body
step
done
It makes sense to keep all the loop control in one place for legibility; but as you can see when it's expressed like this, the for loop does quite a bit more than the while loop.
Of course, this syntax is Bash-specific; classic Bourne shell only has
for variable in token1 token2 ...; do
(Somewhat more elegantly, you could avoid the echo in the first example as long as you are sure that your argument string doesn't contain any % format codes:
date +'Today is %c. A fine day.'
Avoiding a process where you can is an important consideration, even though it doesn't make a lot of difference in this isolated example.)
$() means: "first evaluate this, and then evaluate the rest of the line".
Ex :
echo $(pwd)/myFile.txt
will be interpreted as
echo /my/path/myFile.txt
On the other hand ${} expands a variable.
Ex:
MY_VAR=toto
echo ${MY_VAR}/myFile.txt
will be interpreted as
echo toto/myFile.txt
Why can't I use it as bash$ while ((i=0;i<10;i++)); do echo $i; done
I'm afraid the answer is just that the bash syntax for while just isn't the same as the syntax for for.
your understanding is right. For detailed info on {} see bash ref - parameter expansion
'for' and 'while' have different syntax and offer different styles of programmer control for an iteration. Most non-asm languages offer a similar syntax.
With while, you would probably write i=0; while [ $i -lt 10 ]; do echo $i; i=$(( i + 1 )); done in essence manage everything about the iteration yourself

bash - loop through file contents and append to string [duplicate]

This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 8 months ago.
In my bash script, I'd like to read the contents of a file and append each line of the file to an empty string via a loop. This seems like it would be easy to do and I thought I had implemented it correctly based on some other posts on SO (like Concatenate inputs in string while in loop), but the end result still seems to be an empty string. I'm clearly doing something wrong, but I'm not very experienced with bash scripting so I could use a quick hand.
My bash script:
#!/bin/bash
SOME_STRING=""
cat .env | while read line
do
SOME_STRING+="$line"
done
echo "$SOME_STRING"
and the .env file it references:
FOO=bar
BAZ=bim
I'd expect the output to be FOO=barBAZ=bim, but it just writes an empty string. If I toss an echo "$SOME_STRING" inside of the loop, I do see the string building up as expected, though.
I'm going to assume that this has something to do with the way that I'm reading the file contents and/or looping through it - for example, I tried a for/in loop through a space-separated string instead of a while loop through the file contents, and that worked fine.
Thanks much!
By putting the read loop in a pipe, you are building the string in a subprocess. Instead, do something like:
#!/bin/bash
some_string=""
while read line; do
some_string+="$line"
done < .env
echo "$some_string"
But, really, don't do any of that. Instead, do:
some_string=$(tr -d \\n < .env)
It's worth noting that sometimes you want to keep the subprocess, but you need to be aware that the variables will lose their values at the end of the process. But it is sometimes very convenient to do things like:
#!/bin/bash
some_string=""
cmd | {
while read line; do
some_string+="$line"
done
echo "in pipe, some_string=$some_string"
}
echo "after pipe, some_string=$some_string"

Shell Script for loop stops after the first iteration (very simple code) [duplicate]

This question already has an answer here:
Expanding a bash array only gives the first element
(1 answer)
Closed 3 years ago.
I'm very new to shell script. I'm learning the basic of it. My very simple for loop is not working. It always stops at the first iteration. I already follow the document to create an array variable and using for loop with super simple code.
#!/bin/bash
LIST=()
LIST+=('aaa')
LIST+=('bbb')
LIST+=('ccc')
for i in $LIST
do
echo '----------'$i'----------'
done
It only show 'aaa' then stop the loop. I really have no idea. Please help.
$LIST expands to the first element in array LIST, it's basically the same thing as ${LIST[0]}. You need to use ${LIST[#]} in double-quotes to get each element as a separate word, like:
#!/bin/bash
LIST=()
LIST+=('aaa')
LIST+=('bbb')
LIST+=('ccc')
for i in "${LIST[#]}"
do
echo '----------'"$i"'----------'
done
c.f. Bash Reference Manual ยง Arrays

create multiple global variables from a loop in bash [duplicate]

This question already has answers here:
Indirect variable assignment in bash
(7 answers)
Dynamic variable names in Bash
(19 answers)
Closed 4 years ago.
So I have a loop that basically goes through all of the disks installed on the system, then it assigns a variable to a disk name. however, I cannot use those variables if it's not inside the loop, how do I make them available to be used in other functions or other parts of the script?
Here is the code
#!/bin/bash
dev=1
for disk in $(fdisk -l | grep -o '/dev/sd[a-z]'); do
set "DISK$dev=$disk"
dev=$((dev+1))
done
So if I do echo $DISK1 for example, it doesn't display anything.
But if I do echo $DISK1 INSIDE the loop, then ir displays the fist variable assignment. Can I export them and make them available outside of the loop?
set is not at all the right command to use here. You could pull this off with eval where you have set; but the proper way to solve this is simply to assign the values to an array.
disks=($(fdisk -l | grep -o '/dev/sd[a-z]'))
You can loop over the individual entries with ${disk[0]} through ${disk[n]} or retrieve the entire array at once with "${disk[#]}". The expression ${#disk[#]} evaluates to the number of elements in the array; though because array indexing is zero-based, the last index is one less than this value.
Of course, very often, you don't really need to keep the results in a variable explicitly. If you don't need random access (do you need to recall what you know about the first disk when processing the fifth? Really?) you should probably just loop over the values directly.
for disk in $(fdisk -l | grep -o '/dev/sd[a-z]'); do
: whatever you need to do with the current "$disk"
done
You cannot directly access the parent scope. Also with "export" you will be able to access the exported variable to sub-shell but not the parent one.
A work around for this can be put all these set into a file (appending the new sets) and execute it with after the loop:
dev=1
for disk in $(fdisk -l | grep -o '/dev/sd[a-z]'); do
echo "export \"DISK$dev=$disk\";" >> my_script_full_of_sets.sh
dev=$((dev+1))
done
. my_script_full_of_sets.sh

Bash "delayed expansion" and nested variables [duplicate]

This question already has answers here:
Dynamic variable names in Bash
(19 answers)
Closed 6 years ago.
I was able to do the following in batch, but for the life of me, I cannot figure out how to do this in bash and could use some help. Basically, I used a for loop and delayed expansion to set variables as the for loop iterated through an array. It looked something like this:
FOR /L %%A in (1,1,10) DO (
SET someVar = !inputVar[%%A]!
)
The brackets are merely for clarity.
I now have a similar problem in bash, but cannot figure out how "delayed expansion" (if that's even what it is called in bash) works:
for (( a=1; a<=10; a++ )); do
VAR${!a}= some other thing
done
Am I completely off base here?
Update:
So it seems that I was completely off base and #muru's hint of the XY problem made me relook at what I was doing. The easy solution to my real question is this:
readarray -t array < /filepath
I can now easily use the needed lines.
I think, that eval could help in this case. Not sure, if it's the best option, but could work.
INPUT_VAR=(fish cat elephant)
SOME_VAR=
for i in `seq 0 3`;do
SOME_VAR[$i]='${INPUT_VAR['"$i"']}'
done
echo "${SOME_VAR[2]}" # ${INPUT_VAR[2]}
eval echo "${SOME_VAR[2]}" # elephant
Nice eval explanation:
eval command in Bash and its typical uses
Working with arrays in bash, would be helpful too:
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_10_02.html
Note, that arrays are supported only at new version of bashs.

Resources