Using yq in for loop bash - bash

Question

I have a yaml array like below,
identitymappings:
- arn: "arn:aws:iam::12345567:role/AdmRole"
group: "system:masters"
user: "user1"
- arn: "arn:aws:iam::12345567:role/TestRole"
group: "system:masters"
user: "user2"
I am trying to parse this yaml in a bash script using for loop and yq.
for identityMapping in $(yq read test.yaml "identitymappings[*]"); do
roleArn=$identityMapping["arn"]
group=$identityMapping.group
user=$identityMapping.user
done
But I am not getting the expected results like not able to fetch the values of roleArn,group,user.
Please let me know how to fix this.

Answer 1

The way I would do it is:
# load array into a bash array
# need to output each entry as a single line
readarray identityMappings < <(yq e -o=j -I=0 '.identitymappings[]' test.yml )
for identityMapping in "${identityMappings[#]}"; do
# identity mapping is a yaml snippet representing a single entry
roleArn=$(echo "$identityMapping" | yq e '.arn' -)
echo "roleArn: $roleArn"
done
output:
roleArn: arn:aws:iam::12345567:role/AdmRole
roleArn: arn:aws:iam::12345567:role/TestRole
Disclaimer: I wrote yq

Answer 2

I wasn't able to comment on Charles Duffy's proper answer, but this works for yq v4 without the use of jq...
while IFS=$'\t' read -r roleArn group user _; do
echo "Role: $roleArn"
echo "Group: $group"
echo "User: $user"
done < <(yq e '.identitymappings[] | [.arn, .group, .user] | #tsv' test.yaml)

Answer 3

The easiest way to read from jq or yq into bash is to use a BashFAQ #1 while read loop to handle line-oriented data; in the below, we use #tsv to generate line-oriented output:
while IFS=$'\t' read -r roleArn group user _; do
echo "Role: $roleArn"
echo "Group: $group"
echo "User: $user"
done < <(yq -j read test.yaml \
| jq -r '.identitymappings[] | [.arn, .group, .user] | #tsv')
Note that if you were using the Python yq rather than the Go one, you could remove the yq -j read and just use yq -r '...' in place of jq -r '...'.

Answer 4

There is an improvement of #Rad4's answer that worked for me.
You can neatly loop through using latest yq and jq via:
for im in $(yq eval -o=j test.yaml | jq -cr '.identitymappings[]'); do
arn=$(echo $im | jq -r '.arn' -)
group=$(echo $im | jq -r '.group' -)
user=$(echo $im | jq -r '.user' -)
echo $arn $group $user
done
This loops through valid online jsons, which makes jq still work inside the loop.

Answer 5

The answer by #mike.f is a good one. However, it does not work on OSX machines, because readarray is not an available command.
You can read more about this here.
Here is the equivalent that would work on a mac:
# load array into a bash array
# need to output each entry as a single line
identitymappings=( $(yq e -o=j -I=0 '.identitymappings[]' test.yml ) )
for identityMapping in "${identityMappings[#]}"; do
# identity mapping is a yaml snippet representing a single entry
roleArn=$(echo "$identityMapping" | yq e '.arn' -)
echo "roleArn: $roleArn"
done

Answer 6

Get identitymappings length
Using index to access element of identitymappings
array_length=`yq e ". identitymappings | length - 1" test.yaml`
if [ $array_length -le 0 ] ; then
exit
fi
for element_index in `seq 0 $array_length`;do
arn=`yq e ".identitymappings[$element_index]. arn" test.yml`
group=`yq e ".identitymappings[$element_index]. group" test.yml`
user=`yq e ".identitymappings[$element_index]. user" test.yml`
done
How to get length of array in yq?
https://mikefarah.gitbook.io/yq/operators/length

Answer 7

I figured out..
for identityMapping in $(yq read test.yaml -j "identitymappings[*]"); do
echo $identityMapping
roleArn= echo $identityMapping | jq -r '.arn'
echo $roleArn
group= echo $identityMapping | jq -r '.group'
echo $group
user= echo $identityMapping | jq -r '.user'
echo $user