I am unable to pass a variable in the tag-user cli command.
A=$(aws iam list-user-tags --user-name user --query 'Tags[].{Key:Key,Value:Value}' | grep -B2 "Description" | grep Value | awk -F ":" '{print $2}' | tr -d '",'| awk '$1=$1')
aws iam list-user-tags --user-name user --query 'Tags[].{Key:Key,Value:Value}' | grep -B2 "Description" | grep Value
"Value": "Used for SSO",
A=Used for SSO
passing the value of A to the below CLI :
aws iam tag-user --user-name azure-sso-user --tags "[{"Key": "own:team","Value": "test#test.com"},{"Key": "security","Value": "Service"},{"Key": "comment","Value": "$A"}]"
This is the error I get:
Error parsing parameter '--tags': Invalid JSON:
[{Key: own:team,Value: test#test.com},{Key: security,Value: Service},{Key: own:comment,Value: Used
This worked:
aws iam tag-user --user-name user --tags '[{"Key": "own:team","Value": "test#test.com"},{"Key": "security","Value": "Service"},{"Key": "own:comment","Value": "'"$A"'"}]'
That is, using the following:
[
{
"Key": "own:team",
"Value": "test#test.com"
},
{
"Key": "security",
"Value": "Service"
},
{
"Key": "own:comment",
"Value": "'"
$A
"'"
}
]
Related
I have two datasets:
data1='[
{ "bookings": 2984, "timestamp": 1675854900 },
{ "bookings": 2967, "timestamp": 1675855200 }
]'
data2='[
{ "errors": 51, "timestamp": 1675854900 },
{ "errors": 90, "timestamp": 1675855200 }
]'
I want the output to be:
combined='[
{ "errors": 51, bookings: 2984, "timestamp": 1675854900 },
{ "errors": 90, bookings: 2967, "timestamp": 1675855200 }
]'
Can this be achieved by shell scripting and jq command?
Assume that timestamp will always be present and will always have a common value across two datasets. Even the order is same.
This last paragraph just caught my attention:
Assume that timestamp will always be present and will always have a common value across two datasets. Even the order is same.
If this is truly the case then it is reasonable to assume that both arrays have the same length and their items are aligned respectively. Thus, there's no need to build up a hash-based INDEX as accessing the items by their numeric keys (positions within the arrays) can already be achieved in constant time.
jq -n --argjson data1 "$data1" --argjson data2 "$data2" '
$data1 | [keys[] | $data2[.] + $data1[.]]
'
[
{
"errors": 51,
"timestamp": 1675854900,
"bookings": 2984
},
{
"errors": 90,
"timestamp": 1675855200,
"bookings": 2967
}
]
A simple JOIN operation could do:
jq -n --argjson data1 "$data1" --argjson data2 "$data2" '
[JOIN(INDEX($data1[]; .timestamp); $data2[]; .timestamp | #text; add)]
'
[
{
"errors": 51,
"timestamp": 1675854900,
"bookings": 2984
},
{
"errors": 90,
"timestamp": 1675855200,
"bookings": 2967
}
]
I'm getting this error: jq: error: JOIN/4 is not defined at <top-level>, line 2: [JOIN(INDEX($data1[]; .timestamp); $data2[]; .timestamp | #text; add)] jq: 1 compile error
You are probably using an older version of jq. JOIN and INDEX were introduced in jq 1.6. Either define them yourself by taking their definitions from source, or take those definitions and modify them to fit your very use case (both work well with jq 1.5).
Definitions from source:
jq -n --argjson data1 "$data1" --argjson data2 "$data2" '
def INDEX(stream; idx_expr):
reduce stream as $row ({}; .[$row | idx_expr | tostring] = $row);
def JOIN($idx; stream; idx_expr; join_expr):
stream | [., $idx[idx_expr]] | join_expr;
[JOIN(INDEX($data1[]; .timestamp); $data2[]; .timestamp | #text; add)]
'
Adapted to your use case:
jq -n --argjson data1 "$data1" --argjson data2 "$data2" '
($data1 | with_entries(.key = (.value.timestamp | #text))) as $ix
| $data2 | map(. + $ix[.timestamp | #text])
'
In general, if you find JOIN a bit tricky to understand or use, then consider using INDEX for this type of problem. In the present case, you could get away with a trivially simple approach, e.g.:
jq -n --argjson data1 "$data1" --argjson data2 "$data2" '
INDEX($data1[]; .timestamp) as $dict
| $data2 | map( . + $dict[.timestamp|tostring])
Another way to do this is to build a map from timestamps to error counts, and perform a lookup in it.
jq -n '
input as $data1
| input as $data2
| ($data2
| map({ "key": (.timestamp | tostring), "value": .errors })
| from_entries
) as $errors_by_timestamp
| $data1 | map(.errors = $errors_by_timestamp[(.timestamp | tostring)])
' <<<"$data1 $data2"
By the way, I have trying to this answer from AI since morning and finally it also gave me correct solution this time
#!/bin/bash
data1='[
{ "bookings": 2984, "timestamp": 1675854900 },
{ "bookings": 2967, "timestamp": 1675855200 }
]'
data2='[
{ "errors": 51, "timestamp": 1675854900 },
{ "errors": 90, "timestamp": 1675855200 }
]'
combined=$(jq -n --argjson d1 "$data1" --argjson d2 "$data2" '
[ $d1, $d2 ] | transpose[] | group_by(.timestamp) | map(
reduce .[] as $i ({}; . * $i)
)
')
echo "$combined"
Just pasting it here for you guys in case you didn't think of this method
I want write a script which is giving me the volumeId,instanceId and tags. But in tags just the "Name" is interest me.
I will elaborate.
today I am running with the next call
aws ec2 describe-volumes --filters Name=status,Values=available | jq -c '.Volumes[] | {State: .State, VolumeId: .VolumeId, Tags: .Tags}'
"State":"available","VolumeId":"vol-094c79bc5e641bd7e","Tags":[{"Key":"Name","Value":"Adi"},{"Key":"Team","Value":"SRE"}]}
2.{"State":"available","VolumeId":"vol-041485dd7394bbdd7","Tags":[{"Key":"Team","Value":"SRE"}]}
I want my response will include the just the tag "Name" and it's value.
If the tag "Name" does not exist , I want in my response just the "State","VolumeId".
For 1.State":"available","VolumeId":"vol-094c79bc5e641bd7e","Tags":[{"Key":"Name","Value":"Adi"}
For 2. {"State":"available","VolumeId":"vol-041485dd7394bbdd7"}
If you can live with "Tags": [] holding an empty array while still being present, a simple map(select(.Key == "Name")) will do:
jq -c '.Volumes[] | {State, VolumeId, Tags: (.Tags | map(select(.Key == "Name")))}'
{"State":"available","VolumeId":"vol-094c79bc5e641bd7e","Tags":[{"Key":"Name","Value":"Adi"}]}
{"State":"available","VolumeId":"vol-041485dd7394bbdd7","Tags":[]}
Demo
Otherwise you need to distinguish separately whether the field should be added or not. This can be achieved using an if statement:
jq -c '.Volumes[] | {State, VolumeId} + (
{Tags: (.Tags | map(select(.Key == "Name")))} | if .Tags == [] then {} else . end
)'
{"State":"available","VolumeId":"vol-094c79bc5e641bd7e","Tags":[{"Key":"Name","Value":"Adi"}]}
{"State":"available","VolumeId":"vol-041485dd7394bbdd7"}
Demo
I'm trying to extract the ARN's (.StackId) of all AWS CloudFormation stacks which match a specific string in key StackName and StackStatus of "CREATE COMPLETE" or "UPDATE_COMPLETE".
Example:
{
"StackSummaries": [
{
"StackId": "arn:aws:cloudformation:us-east-1:AWS_ACCOUNT_ID:stack/some-service-name/9ad489b0-ab22-11eb-af8f-0a56fXXXX8ad",
"StackName": "some-service-name",
"CreationTime": "2021-05-02T08:44:28.106000+00:00",
"StackStatus": "CREATE_COMPLETE",
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
},
{
"StackId": "arn:aws:cloudformation:us-east-1:AWS_ACCOUNT_ID:stack/some-service-name/44239210-9703-11eb-b085-12daXXXX6186",
"StackName": "some-service-name",
"TemplateDescription": "some-service-name",
"CreationTime": "2021-04-06T18:09:45.470000+00:00",
"LastUpdatedTime": "2021-04-13T13:09:37.683000+00:00",
"StackStatus": "UPDATE_COMPLETE",
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
}
]
}
These are the commands I've tried:
aws cloudformation list-stacks --profile production |
jq -r '.StackSummaries[] |
select(.StackName | match("some-service-name";"i")) and
select(.StackStatus | match("UPDATE_COMPLETE";"i")) .StackId'
aws cloudformation list-stacks --profile production |
jq -r '.StackSummaries[] |
select(.StackName | contains("some-service-name")) and
select(.StackStatus | contains("UPDATE_COMPLETE")) .StackId'
aws cloudformation list-stacks --profile production |
jq -r '.StackSummaries[] |
select(.StackName | match("some-service-name";"i")) and
select(.StackStatus | match("UPDATE_COMPLETE";"i") or
select(.StackStatus | match("CREATE_COMPLETE";"i"))) .StackId'
All of the above commands return boolean value instead of the StackId itself. How can I get the StackId instead of a boolean value?
If you have two conditions to meet, filter twice
select(condition1) | select(condition2)
or and the conditions
select(condition1 and condition2)
As you discovered, and-ing the results of select doesn't make sense.
jq -r '
.StackSummaries[] |
select(
.StackName == "some-service-name" and (
.StackStatus == "CREATE_COMPLETE" or
.StackStatus == "UPDATE_COMPLETE"
)
) |
.StackId
'
jqplay
Note that | has very low precedence, so
a | b and c | d
means
a | ( b and c ) | d
Use parens if you mean
( a | b ) and ( c | d )
I am running aws cli
aws cloudwatch get-metric-statistics --metric-name CPUUtilization --start-time 2010-02-20T12:00:00 --end-time 2010-02-20T15:00:00 --period 60 --namespace AWS/EC2 --extended-statistics p80 --dimensions Name=InstanceId,Value=i-0b123423423
the output comes as
{
"Label": "CPUUtilization",
"Datapoints": [
{
"Timestamp": "2020-02-20T12:15:00Z",
"Unit": "Percent",
"ExtendedStatistics": {
"p80": 0.16587132264856133
}
},
How do i get the output in the below format's (2 Columns)
19.514049550078127 2020-02-13T20:15:00Z
12.721997782508938 2020-02-13T19:15:00Z
13.318820949213313 2020-02-13T18:15:00Z
15.994192991030545 2020-02-13T17:15:00Z
18.13096421299414 2020-02-13T16:15:00Z
with Heading as CPUUtilization (2 columns)
CPUUtilization
19.514049550078127 2020-02-13T20:15:00Z
12.721997782508938 2020-02-13T19:15:00Z
13.318820949213313 2020-02-13T18:15:00Z
15.994192991030545 2020-02-13T17:15:00Z
18.13096421299414 2020-02-13T16:15:00Z
And in single column
19.514049550078127
12.721997782508938
13.318820949213313
15.994192991030545
18.13096421299414
How can achieve this ?
Assuming the input file is input.json, then:
To output in the 2 columns format:
jq -r '.Datapoints[] | [.ExtendedStatistics.p80, .Timestamp] | #tsv' input.json | sort -nr
With Heading as CPUUtilization (2 columns):
echo CPUUtilization; jq -r '.Datapoints[] | [.ExtendedStatistics.p80, .Timestamp] | #tsv' input.json | sort -nr
And in single column:
jq -r '.Datapoints[] | [.ExtendedStatistics.p80] | #tsv' input.json | sort -nr
This question already has answers here:
jq not working on tag name with dashes and numbers
(2 answers)
Closed 4 years ago.
Whole file:https://1drv.ms/u/s!AizscpxS0QM4hJpEkp12VPHiKO_gBg
Using this command i get part bellow (get latest job)
jq '.|[ .executions[] | select(.job.name != null) | select(.job.name) ]
| sort_by(.id)
| reverse
| .[0] ' 1.json
{
"argstring": null,
"date-ended": {
"date": "2018-04-03T17:43:38Z",
"unixtime": 1522777418397
},
"date-started": {
"date": "2018-04-03T17:43:34Z",
"unixtime": 1522777414646
},
"description": "",
"executionType": "user",
"failedNodes": [
"172.30.61.88"
],
"href": "http://172.30.61.88:4440/api/21/execution/126",
"id": 126,
"job": {
"averageDuration": 4197,
"description": "",
"group": "",
"href": "http://172.30.61.88:4440/api/21/job/271cbcec-5042-4d52-b794-ede2056b2ab8",
"id": "271cbcec-5042-4d52-b794-ede2056b2ab8",
"name": "aa",
"permalink": "http://172.30.61.88:4440/project/demo/job/show/271cbcec-5042-4d52-b794-ede2056b2ab8",
"project": "demo"
},
"permalink": "http://172.30.61.88:4440/project/demo/execution/show/126",
"project": "demo",
"status": "failed",
"user": "administrator"
I managed to extract job name and status, now want to get date-ended.date ?
jq '.|[ .executions[] |select(.job.name != null) | select(.job.name) ]
| sort_by(.id)
| reverse
| .[0]
| "\(.status), \(.job.name)"' 1.json
With the "-r" command-line option, the following filter:
[.executions[] | select(.job.name != null)]
| sort_by(.id)
| reverse
| .[0]
| [.status, .job.name, ."date-ended".date]
| #csv
produces:
"failed","aa","2018-04-03T17:43:38Z"
An important point that you might have missed is that "-" is a "special" character in that it can signify negation or subtraction.
If your jq does not support the syntax ."date-ended".date, then you could fall back to the basic syntax: (.["date-ended"] | .date)
I guess you have troubles extracting .date-ended.date because the name contains a dash that is interpreted by jq as subtraction.
The solution is listed in the documentation:
If the key contains special characters, you need to surround it with double quotes like this: ."foo$", or else .["foo$"].
This means the last filter of your jq program should be:
"\(.status), \(.job.name), \(."date-ended".date)"