I want write a script which is giving me the volumeId,instanceId and tags. But in tags just the "Name" is interest me.
I will elaborate.
today I am running with the next call
aws ec2 describe-volumes --filters Name=status,Values=available | jq -c '.Volumes[] | {State: .State, VolumeId: .VolumeId, Tags: .Tags}'
"State":"available","VolumeId":"vol-094c79bc5e641bd7e","Tags":[{"Key":"Name","Value":"Adi"},{"Key":"Team","Value":"SRE"}]}
2.{"State":"available","VolumeId":"vol-041485dd7394bbdd7","Tags":[{"Key":"Team","Value":"SRE"}]}
I want my response will include the just the tag "Name" and it's value.
If the tag "Name" does not exist , I want in my response just the "State","VolumeId".
For 1.State":"available","VolumeId":"vol-094c79bc5e641bd7e","Tags":[{"Key":"Name","Value":"Adi"}
For 2. {"State":"available","VolumeId":"vol-041485dd7394bbdd7"}
If you can live with "Tags": [] holding an empty array while still being present, a simple map(select(.Key == "Name")) will do:
jq -c '.Volumes[] | {State, VolumeId, Tags: (.Tags | map(select(.Key == "Name")))}'
{"State":"available","VolumeId":"vol-094c79bc5e641bd7e","Tags":[{"Key":"Name","Value":"Adi"}]}
{"State":"available","VolumeId":"vol-041485dd7394bbdd7","Tags":[]}
Demo
Otherwise you need to distinguish separately whether the field should be added or not. This can be achieved using an if statement:
jq -c '.Volumes[] | {State, VolumeId} + (
{Tags: (.Tags | map(select(.Key == "Name")))} | if .Tags == [] then {} else . end
)'
{"State":"available","VolumeId":"vol-094c79bc5e641bd7e","Tags":[{"Key":"Name","Value":"Adi"}]}
{"State":"available","VolumeId":"vol-041485dd7394bbdd7"}
Demo
Related
I want to get the index of a element from the array.
Link: https://api.github.com/repos/checkstyle/checkstyle/releases
I want to fetch the tag_name.
I generally use this command to get my latest release tag:
LATEST_RELEASE_TAG=$(curl -s https://api.github.com/repos/checkstyle/checkstyle/releases/latest \
| jq ".tag_name")
But now I want previous releases too so I want the index of a particular tag name. For eg: tag_name = "10.3.1"
Also, I am thinking to use mathematical reduction to get the other previous release if I get a particular index number like:
( 'index of 10.3.1' - 1) Any thought regarding this?
Just index of "checkstyle-10.3.1":
curl -s https://api.github.com/repos/checkstyle/checkstyle/releases | jq '[.[].tag_name] | to_entries | .[] | select(.value=="checkstyle-10.3.1") | .key'
Release before "checkstyle-10.3.1":
curl -s https://api.github.com/repos/checkstyle/checkstyle/releases | jq '([.[].tag_name] | to_entries | .[] | select(.value=="checkstyle-10.3.1") | .key) as $index | .[$index+1]'
Release "checkstyle-10.3.1"
curl -s https://api.github.com/repos/checkstyle/checkstyle/releases | jq '.[] | select(.tag_name=="checkstyle-10.3.1")'
Here's a general-purpose function to get the index of something in an array:
# Return the 0-based index of the first item in the array input
# for which f is truthy, else null
def index_by(f):
first(foreach .[] as $x (-1;
.+1;
if ($x|f) then . else empty end)) // null;
This can be used to retrieve the item immediately before a target as follows:
jq '
def index_by(f): first(foreach .[] as $x (-1; .+1; if ($x|f) then . else empty end)) // null;
index_by(.tag_name == "checkstyle-10.3.1") as $i
| if $i then .[$i - 1] else empty end
'
Given the following yaml:
charts:
# repository with Helm charts for creation namespaces
path: ns
pathMonitoringPrometheus: prom
namespaces:
first:
description: "Description of first"
enabled: false
branch: master
bootstrapChart: bootstrap
syncAccessGroups: []
namespace:
role: k8s-role-of-first
istio: disabled
public: view
sources: []
second:
description: "Description of second"
enabled: false
branch: HEAD
bootstrapChart: bootstrap
namespace:
role: k8s-role-of-second
istio: 1-13-2
labels:
label: second
sources:
- http://url.of.second
How could we get a list of namespaces and their istio value if it is different to "disabled".
We are trying to use "yq" tool, but I guess any approach would be ok, although "yq" would be a preferred approach.
second, 1-13-2
Using kislyuk/yq you can base your filter on jq.
to_entries splits up the object into an array of key-value pairs
select selects those items matching your criteria
String interpolation in combination with the -r option puts together your desired output
yq -r '
.namespaces
| to_entries[]
| select(.value.namespace.istio != "disabled")
| "\(.key), \(.value.namespace.istio)"
'
second, 1-13-2
Using mikefarah/yq the filter is quite similar.
to_entries[] has to be split up to_entries | .[]
String interpolation is replaced using join and an array
yq '
.namespaces
| to_entries | .[]
| select(.value.namespace.istio != "disabled")
| [.key, .value.namespace.istio] | join(", ")
'
second, 1-13-2
this will do:
cat /path/tp/your.yaml |yq -r '.namespaces | to_entries[] | "\(.key) \(.value.namespace.istio)"'`
will result:
first disabled
second 1-13-2
I'm trying to extract the ARN's (.StackId) of all AWS CloudFormation stacks which match a specific string in key StackName and StackStatus of "CREATE COMPLETE" or "UPDATE_COMPLETE".
Example:
{
"StackSummaries": [
{
"StackId": "arn:aws:cloudformation:us-east-1:AWS_ACCOUNT_ID:stack/some-service-name/9ad489b0-ab22-11eb-af8f-0a56fXXXX8ad",
"StackName": "some-service-name",
"CreationTime": "2021-05-02T08:44:28.106000+00:00",
"StackStatus": "CREATE_COMPLETE",
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
},
{
"StackId": "arn:aws:cloudformation:us-east-1:AWS_ACCOUNT_ID:stack/some-service-name/44239210-9703-11eb-b085-12daXXXX6186",
"StackName": "some-service-name",
"TemplateDescription": "some-service-name",
"CreationTime": "2021-04-06T18:09:45.470000+00:00",
"LastUpdatedTime": "2021-04-13T13:09:37.683000+00:00",
"StackStatus": "UPDATE_COMPLETE",
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
}
]
}
These are the commands I've tried:
aws cloudformation list-stacks --profile production |
jq -r '.StackSummaries[] |
select(.StackName | match("some-service-name";"i")) and
select(.StackStatus | match("UPDATE_COMPLETE";"i")) .StackId'
aws cloudformation list-stacks --profile production |
jq -r '.StackSummaries[] |
select(.StackName | contains("some-service-name")) and
select(.StackStatus | contains("UPDATE_COMPLETE")) .StackId'
aws cloudformation list-stacks --profile production |
jq -r '.StackSummaries[] |
select(.StackName | match("some-service-name";"i")) and
select(.StackStatus | match("UPDATE_COMPLETE";"i") or
select(.StackStatus | match("CREATE_COMPLETE";"i"))) .StackId'
All of the above commands return boolean value instead of the StackId itself. How can I get the StackId instead of a boolean value?
If you have two conditions to meet, filter twice
select(condition1) | select(condition2)
or and the conditions
select(condition1 and condition2)
As you discovered, and-ing the results of select doesn't make sense.
jq -r '
.StackSummaries[] |
select(
.StackName == "some-service-name" and (
.StackStatus == "CREATE_COMPLETE" or
.StackStatus == "UPDATE_COMPLETE"
)
) |
.StackId
'
jqplay
Note that | has very low precedence, so
a | b and c | d
means
a | ( b and c ) | d
Use parens if you mean
( a | b ) and ( c | d )
I use Jq to perform some filtering on a large json file using :
paths=$(jq '.paths | to_entries | map(select(.value[].tags | index("Filter"))) | from_entries' input.json)
and write the result to a new file using :
jq --argjson prefix "$paths" '.paths=$prefix' input.json > output.json
But this ^ fails as $paths has a very high line count (order of 100,000).
Error :
jq: Argument list too long
I also went through : /usr/bin/jq: Argument list too long error bash , understood the same problem there, but did not get the solution.
In general, assuming your jq allows it, you could use —argfile or —slurpfile but in your case you can simply avoid the issue by invoking jq just once instead of twice. For example, to keep things clear:
( .paths | to_entries | map(select(.value[].tags | index("Filter"))) | from_entries ) as $prefix
| .paths=$prefix
Even better, simply use |=:
.paths |= ( to_entries | map(select(.value[].tags | index("Filter"))) | from_entries)
or better yet, use with_entries.
I have a json stream of updates for products, and I'm trying to get the last X versions sorted by version (they are sorted by release date currently).
It looks like jq can't sort a stream of objects directly, sort_by only works on arrays, and I couldn't find a way to collect a stream into an array that doesn't involve piping the output of jq -c to jq -s.
My current solution:
< xx \
jq -c '.[] | select(.platform | contains("Unix"))' \
| jq -cs 'sort_by(.version) | reverse | .[]' \
| head -5 \
| jq -C . \
| less
I expected to be able to use
jq '.[] | select(...) | sort_by(.version) | limit(5) | reverse'
but I couldn't find a thing that limits and sort_by doesn't work on non arrays.
I am testing this on atlassian's json for releases: https://my.atlassian.com/download/feeds/archived/bamboo.json
In jq you can always contain the results to an array using the [..] that put the results to an array for the subsequent functions to operate on. Your given requirement could be simply done as
jq '[.[] | select(.platform | contains("Unix"))] | sort_by(.version) | limit(5;.[])'
See it working on jq playground tested on v1.6
and with added reverse() function, introduce an another level of array nesting. Use reverse[] to dump the objects alone
jq '[[.[] | select(.platform | contains("Unix"))] | sort_by(.version) | limit(5;.[]) ] | reverse'