Hey I am using the next command
aws ec2 describe-volumes --filters Name=status,Values=available | jq '.Volumes[] | { State: .State, VolumeId: .VolumeId, Tags: .Tags}'
I get the next response:
{
"State": "available",
"VolumeId": "vol-03449dadd29f2067f",
"Tags": [
{
"Key": "sre",
"Value": "test"
}
]}
1.I want my response will be in 1 row regarding each volume.
2. I want to check if the volume has the tag "Name" if so I want the value, otherwise the tags are not interest me. How can I manipulate it ?
Use the -c option.
Use a query of the form select(any(Tags[]; CONDITION))
e.g. select(any(Tags[]; .Key == "Name"))
You can abbreviate expressions of the form {foo: .foo} to just {foo}
So, depending on your exact requirements, your invocation of jq could look like:
jq -c '.Volumes[] | {State, VolumeId, Tags} | select(any(.Tags[]; .Key == "Name"))'
Related
given the following (simplified) json file:
{
"data": [
{
"datum": "2023-01-11 00:00:00",
"prijs": "0.005000",
"prijsZP": "0.161550",
"prijsEE": "0.181484",
"prijsTI": "0.160970",
},
{
"datum": "2023-01-11 01:00:00",
"prijs": "0.000000",
"prijsZP": "0.155500",
"prijsEE": "0.175434",
"prijsTI": "0.154920",
}
]
}
I want to specify in my jq command which fields to retreive, i.e. only "datum" and "prijsTI". But on another moment this selection will be different.
I use the following command to gather all the fields, but would like to set the field selection via a variable:
cat data.json |jq -r '.data[]|[.datum, .prijsTI]|#csv'
I already tried using arguments, but this did not work :-(
myJQselect=".datum, .prijsTI"
cat data.json |jq -r --arg myJQselect "$myJQselect" '.data[$myHour |tonumber]|[$myJQselect]|#csv'
gives the following result: ".datum, .prijs" instead of the correct values.
Would this be possible?
Thanks,
Jeroen
You can use the --args option to provide a variable number of fields to query, then use the $ARGS.positional array to retrieve them:
jq -r '.data[] | [.[$ARGS.positional[]]] | #csv' data.json --args datum prijsTI
"2023-01-11 00:00:00","0.160970"
"2023-01-11 01:00:00","0.154920"
I have a json file which i have obtained using curl command and it looks as below.
{
"Storages": [
{
"Creation": "2020-04-21T14:01:54",
"Modified": "2020-04-21T14:01:54",
"Volume": "/dev/null",
"id": 10000,
"version": "20190925-230722"
},
{
"Creation": "2020-04-22T14:01:54",
"Modified": "2020-04-22T14:01:54",
"Volume": "/opt/home",
"id": 10001,
"version": "22a-20190925-230722"
},
{
"Creation": "2020-04-23T14:01:54",
"Modified": "2020-04-23T14:01:54",
"Volume": "/home/abcd",
"id": 10003,
"version": "21c-20190925-230722"
}
]
}
Now I need to loop thorough array and get id and volume values into 2 variables if version startswith 21a. No need to form another json
For educational purposes, here's a jq command that does both the things you want, but in 2 separate steps:
jq -r 'del(.Storages[] | select(.version | startswith("21a") | not))
.Storages[] | {id, version}'
The first part (del(.Storages[] | select(.version | startswith("21a") | not))) filters out the array elements that don't have a version starting with 21a. The second part (.Storages[] | {id, version}) drills and extracts the specific information you need.
You can use startswith builtin function such as
jq -r '.Storages[] | select(.version | startswith("21a")) | {id, Volume}'
Demo
Edit : Assuming the JSON embedded into a file(Storages.json), then you can assign the results into shell variables such as
$ readarray -t vars < <( jq -r '.Storages[] | select(.version|startswith("21a"))| .id, .Volume' Storages.json )
and display those variables as
$ declare -p vars
declare -a vars='([0]="10003" [1]="/home/abcd")'
I have an issue where I am trying to filter records with a CONTAINS, but it won't accept a variable that has spaces in it. I am including the JSON and the calls. I explain what works and the last one that does not work. I have looked High and Low but I can't make it work. I have seen and tried many (hundreds of ways taking into account the double quotes, escaped, not escaped, with, without, but no luck) can someone take a look and point me to something that might help.
JSON used to test
_metadatadashjson='{ "meta": { "provisionedExternalId": "" }, "dashboard": { "liveNow": false, "panels": [ { "collapsed": false, "title": "Gyrex Thread Count Gauges", "type": "row", "targets": [ { "expr": "jvm_threads_current{instance=\"192.1.50.22:8055\",job=\"prometheus_gyrex\"}", "refId": "B" } ] }, { "datasource": "Prometheus_16_Docker", "targets": [ { "exemplar": true, "expr": "jvm_threads_current{instance=\"10.32.0.4:8055\",job=\"prometheus_gyrex\"}" } ], "title": ".16 : 3279", "type": "gauge" }, { "description": "", "targets": [ { "expr": "jvm_threads_current{instance=\"10.32.0.7:8055\",job=\"prometheus_gyrex\"}", "refId": "B" } ], "title": ".16 : 3288", "type": "graph" }, { "description": "", "targets": [ { "expr": "jvm_threads_current{instance=\"192.168.2.16:3288\",job=\"prometheus_gyrex\"}", "refId": "C" } ], "title": ".16 : 3288", "type": "graph" } ], "version": 55 }}'
Set the string to search for in key "expr"
exprStrSearch="10.32.0.4:8055"
This works returns one record
echo "${_metadatadashjson}" | jq -r --arg EXPRSTRSEARCH "$exprStrSearch" '.dashboard.panels[] | select(.targets[].expr | contains($EXPRSTRSEARCH)) | .targets[].expr'
This works no problem returns two records.
echo "${_metadatadashjson}" | jq -r --arg EXPRSTRSEARCH "$exprStrSearch" '.dashboard.panels[] | select(.targets[].expr | contains("10.32.0.4:8055", "10.32.0.7:8055")) | .targets[].expr'
Change the value to include a space and another string
exprStrSearch="10.32.0.4:8055 10.32.0.7:8055"
Does not work.
echo "${_metadatadashjson}" | jq -r --arg EXPRSTRSEARCH "$exprStrSearch" '.dashboard.panels[] | select(.targets[].expr | contains($EXPRSTRSEARCH)) | .targets[].expr'
None of your data contains "10.32.0.4:8055 10.32.0.7:8055".
You could pass multiple strings to contains(), using a bash array:
strings=("10.32.0.4:8055" "10.32.0.7:8055")
echo "${_metadatadashjson}" |
jq -r --args '.dashboard.panels[] | select(.targets[].expr | contains($ARGS.positional[])) | .targets[].expr' "${strings[#]}"
But contains will evaluate to true for each match. Ie. if one expr contained both strings, it would be selected (and printed) twice.
With test, that won't happen. Here's how you can add the |s between multiple strings, and pass them in a single jq variable (as well as escape all the dots):
strings=("10.32.0.4:8055" "10.32.0.7:8055")
IFS=\|
echo "${_metadatadashjson}" |
jq -r --arg str "${strings[*]//./\\.}" '.dashboard.panels[] | select(.targets[].expr | test($str)) | .targets[].expr'
Both examples print this:
jvm_threads_current{instance="10.32.0.4:8055",job="prometheus_gyrex"}
jvm_threads_current{instance="10.32.0.7:8055",job="prometheus_gyrex"}
Update: I forgot to escape the dots for test. I edited the test example so that all the dots get escaped (with a single backslash). It's regex, so (unescaped) dots will match any character. The contains example matches the strings literally (not regex).
The problem is that the string with the space in it does not in fact occur in the given JSON. It's not too clear what you are trying to do but please note that contains is not symmetric:
"a" | contains("a b")
evaluates to false.
If you intended to write a boolean search criterion, you could use a boolean expression, or use jq's regular expression machinery, e.g.
test("10.32.0.4:8055|10.32.0.7:8055")
or probably even better:
test("\"(10[.]32[.]0[.]4:8055|10[.]32[.]0[.]7:8055)\"")
I don't understand what I'm doing wrong or why this does not work.
test.json file:
[
{
"Header": {
"Region": "US",
"Tenant": "Tenant1",
"Stage": "testing",
"ProductType": "old"
},
"Body": []
},
{
"Header": {
"Region": "EU",
"Tenant": "Tenant2",
"Stage": "development",
"ProductType": "new"
},
"Body": []
}
]
I want to display the values of the .Header.Tenant key. So the simple jq call does its job:
$ jq '[.[].Header.Tenant]' test.json
[
"Tenant1",
"Tenant2"
]
Now I want to assign that jq filter to a bash variable and use it with jq's --arg variable.
And I am getting this:
$ a=".[].Header.Tenant"; jq --arg xx "$a" '[$xx]' test.json
[
".[].Header.Tenant"
]
What is wrong?
jq does not have an eval function for evaluating arbitrary jq expressions, but it does provide functions that can be used to achieve much the same effect, the key idea being that certain JSON values can be used to specify query operations.
In your case, you would have to translate the jq query into a suitable jq operation, such as:
jq --argjson a '["Header","Tenant"]' '
getpath(paths|select( .[- ($a|length) :]== $a))
' test.json
Extending jq's JSON-based query language
More interestingly, you could write your own eval, e.g.
jq --argjson a '[[], "Header","Tenant"]' '
def eval($expr):
if $expr == [] then .
else $expr[0] as $op
| if $op == [] then .[] | eval($expr[1:])
else getpath([$op]) | eval($expr[1:])
end
end;
eval($a)
' test.json
With eval.jq as a module
If the above def of eval were put in a file, say ~/jq/eval.jq, then you could simply write:
jq -L ~/jq --argjson a '[[], "Header","Tenant"]' '
include "eval";
eval($a)' test.json
Or you could specify the search path in the jq program:
jq --argjson a '[[], "Header","Tenant"]' '
include "eval" { "search": "~/jq" };
eval($a)' input.json
Or you could use import ...
TLDR; The following code does the job:
$ a=".[].Header.Tenant"; jq -f <(echo "[$a]") test.json
[
"Tenant1",
"Tenant2"
]
One as well can add/modify the filter in the jq call, if needed:
$ a=".[].Header.Tenant"; jq -f <(echo "[$a]|length") test.json
2
Longer explanation
My ultimate goal was to figure out how I can define the lowest common denominator jq filter in a variable and use it when calling jq, plus add additional parameters if necessary. If you have a really complex jq filter spanning multiple lines that you call frequently, you probably want to template it somehow and use that template when calling jq.
While peak demonstrated how it can be done, I think it is overengineering the simple task.
However, using process substitution combined with the jq's -f option to read a filter from the file does solve my problem.
I'm working with jq 1.6 to get the last entry in an object. It should work like this:
data='{ "1": { "a": "1" }, "2": { "a": "2" }, "3": { "a": "3" } }'
result=`echo $data | jq 'myfilter'`
echo $result
{ "3": { "a": "3" } }
I tried these filters:
jq '. | last' # error: Cannot index object with number
How can I tell jq to quote the number?
jq '. | to_entries | last' # { "key": "3", "value": { "a": "3" } }
I guess I could munge this up by concatenating the key and value entries. Is there a simpler way?
The tutorial and the manual didn't help. No joy on SO either.
You can use the following :
jq 'to_entries | [last] | from_entries'
Try it here.
We can't use with_entries(last) because last returns a single element and from_entries requires an array, hence the [...] construct above.