Iterate over JSON data for each specific key - bash

Question

I have a JSON with this format:
{
"instance-01": {
"instance_aws_az": "eu-west-1a",
"instance_aws_dns_name": "<ip>eu-west-1.compute.amazonaws.com",
"instance_ec2_tags": {
"Name": "2ed97a79c27be6bca883",
"Serial_number": "22000528",
},
"instance_sg_groups": [],
},
"instance-29": {
"instance_aws_az": "eu-west-1a",
"instance_aws_dns_name": "<ip>eu-west-1.compute.amazonaws.com",
"instance_ec2_tags": {
"Name": "2ed97a79c27be6bca773",
"Serial_number": "22000578",
},
"instance_sg_groups": [],
}
}
So, the main keys are "instance-01" and "instance-29". I can get that info with:
jq -c 'keys[]' instances_metadata.json
However, I want to use the key names I get with that to iterate over the nested info for each one, for example:
jq -c 'keys[]' instances_metadata.json | while read i; do
echo "Instance $i: Serial Number is: $i.instance_ec2_tags.Serial_number"
// Edit with more specific tasks I want to run
if [ "instance-01" = "$i" ]; then
echo "Instance $i: "
// Something like that to filter by key.. I tried several options but with no success
serial_number=$(jq -r '.[].instance_ec2_tags.Serial_number | select(. == "instance-01")' instances_metadata.json )
echo "Value: $serial_number"
aws s3 cp .... //using $serial_number and other values
fi
done
How can I do this? I just found questions to iterate but ignoring which key name it has and in this case I need to have that info inside the loop.

Answer 1

If the task is accomplishable within jq, there's no need for a bash loop and multiple jq calls, as you can do the iteration inside jq using variables with the as notation as in keys[] as $i | …. For example:
jq -r '
keys[] as $i
| "Instance \($i): Serial Number is: \(.[$i].instance_ec2_tags.Serial_number)"
'
Instance instance-01: Serial Number is: 22000528
Instance instance-29: Serial Number is: 22000578
Demo
If it is not, then you will have to loop in bash. Introduce the loop variable to the nested jq calls using --arg and reference the according object using it as .[$varname]. Here's your edited example with the modifications mentioned:
jq -r 'keys[]' instances_metadata.json | while read i; do
echo "Instance $i: Serial Number is: $(jq -r --arg i "$i" '.[$i].instance_ec2_tags.Serial_number' instances_metadata.json)"
// Edit with more specific tasks I want to run
if [ "instance-01" = "$i" ]; then
echo "Instance $i: "
// Something like that to filter by key.. I tried several options but with no success
serial_number="$(jq -r --arg i "$i" '.[$i].instance_ec2_tags.Serial_number' instances_metadata.json)"
echo "Value: $serial_number"
aws s3 cp .... //using $serial_number and other values
fi
done