given the following (simplified) json file:
{
"data": [
{
"datum": "2023-01-11 00:00:00",
"prijs": "0.005000",
"prijsZP": "0.161550",
"prijsEE": "0.181484",
"prijsTI": "0.160970",
},
{
"datum": "2023-01-11 01:00:00",
"prijs": "0.000000",
"prijsZP": "0.155500",
"prijsEE": "0.175434",
"prijsTI": "0.154920",
}
]
}
I want to specify in my jq command which fields to retreive, i.e. only "datum" and "prijsTI". But on another moment this selection will be different.
I use the following command to gather all the fields, but would like to set the field selection via a variable:
cat data.json |jq -r '.data[]|[.datum, .prijsTI]|#csv'
I already tried using arguments, but this did not work :-(
myJQselect=".datum, .prijsTI"
cat data.json |jq -r --arg myJQselect "$myJQselect" '.data[$myHour |tonumber]|[$myJQselect]|#csv'
gives the following result: ".datum, .prijs" instead of the correct values.
Would this be possible?
Thanks,
Jeroen
You can use the --args option to provide a variable number of fields to query, then use the $ARGS.positional array to retrieve them:
jq -r '.data[] | [.[$ARGS.positional[]]] | #csv' data.json --args datum prijsTI
"2023-01-11 00:00:00","0.160970"
"2023-01-11 01:00:00","0.154920"
Related
I don't understand what I'm doing wrong or why this does not work.
test.json file:
[
{
"Header": {
"Region": "US",
"Tenant": "Tenant1",
"Stage": "testing",
"ProductType": "old"
},
"Body": []
},
{
"Header": {
"Region": "EU",
"Tenant": "Tenant2",
"Stage": "development",
"ProductType": "new"
},
"Body": []
}
]
I want to display the values of the .Header.Tenant key. So the simple jq call does its job:
$ jq '[.[].Header.Tenant]' test.json
[
"Tenant1",
"Tenant2"
]
Now I want to assign that jq filter to a bash variable and use it with jq's --arg variable.
And I am getting this:
$ a=".[].Header.Tenant"; jq --arg xx "$a" '[$xx]' test.json
[
".[].Header.Tenant"
]
What is wrong?
jq does not have an eval function for evaluating arbitrary jq expressions, but it does provide functions that can be used to achieve much the same effect, the key idea being that certain JSON values can be used to specify query operations.
In your case, you would have to translate the jq query into a suitable jq operation, such as:
jq --argjson a '["Header","Tenant"]' '
getpath(paths|select( .[- ($a|length) :]== $a))
' test.json
Extending jq's JSON-based query language
More interestingly, you could write your own eval, e.g.
jq --argjson a '[[], "Header","Tenant"]' '
def eval($expr):
if $expr == [] then .
else $expr[0] as $op
| if $op == [] then .[] | eval($expr[1:])
else getpath([$op]) | eval($expr[1:])
end
end;
eval($a)
' test.json
With eval.jq as a module
If the above def of eval were put in a file, say ~/jq/eval.jq, then you could simply write:
jq -L ~/jq --argjson a '[[], "Header","Tenant"]' '
include "eval";
eval($a)' test.json
Or you could specify the search path in the jq program:
jq --argjson a '[[], "Header","Tenant"]' '
include "eval" { "search": "~/jq" };
eval($a)' input.json
Or you could use import ...
TLDR; The following code does the job:
$ a=".[].Header.Tenant"; jq -f <(echo "[$a]") test.json
[
"Tenant1",
"Tenant2"
]
One as well can add/modify the filter in the jq call, if needed:
$ a=".[].Header.Tenant"; jq -f <(echo "[$a]|length") test.json
2
Longer explanation
My ultimate goal was to figure out how I can define the lowest common denominator jq filter in a variable and use it when calling jq, plus add additional parameters if necessary. If you have a really complex jq filter spanning multiple lines that you call frequently, you probably want to template it somehow and use that template when calling jq.
While peak demonstrated how it can be done, I think it is overengineering the simple task.
However, using process substitution combined with the jq's -f option to read a filter from the file does solve my problem.
I have a json object with below element,
rsrecords="{
"ResourceRecords": [
{
"Value": "\"heritage=external-dns,external-dns/owner=us-east-1:sandbox,external-dns/resource=service/api""
}
],
"Type": "TXT",
"Name": "\\052.apiconsumer.alpha.sandbox.test.net.",
"TTL": 300
}"
And in my bash script,I have below code snippet,
jq -r '.[] | .Name ,.ResourceRecords[0].Value' <<< "$rsrecords" | \
while read -r name; read -r value; do
echo $name
Output is printed as,
\052.apiconsumer.alpha.sandbox.test.net.
But I am expecting it to print as \\052.apiconsumer.alpha.sandbox.test.net., which is , as it is "Name" from the json object..
How can this be done?
Before getting to the heart of the matter, please note that
the sample data as given is a bit of a mishmash, so I'll assume you meant something like:
rsrecords='
{
"ResourceRecords": [
{
"Value": "heritage=external-dns,external-dns/owner=us-east-1:sandbox,external-dns/resource=service/api"
}
],
"Type": "TXT",
"Name": "\\052.apiconsumer.alpha.sandbox.test.net.",
"TTL": 300
}
'
Your jq query does not match the above JSON, so I'll assume you intended the query to be simply:
.Name, .ResourceRecords[0].Value
In any case, with the above JSON, the bash commands:
jq -r '.Name, .ResourceRecords[0].Value' <<< "$rsrecords" |
while read -r name; read -r value; do
echo "$name"
done
yields:
\052.apiconsumer.alpha.sandbox.test.net.
This is correct, because the JSON string "\\X" is an encoding of the raw string: \X
If you want to see the JSON string, then invoke jq without the -r option. If you want to invoke jq with the -r option and want to see two backslashes, you will have to encode them as four backslashes in your JSON.
I would like to get the values from Json file. Which is working.
JsonFileToTest:
{
"permissions": [
{
"emailid": "test1#test.com",
"rights": "read"
},
{
"emailid": "test2#test.com",
"rights": "read"
}
]
}
readPermissions=($(jq -r '.permissions' JsonFileToTest))
# The command below works perfectly, But when I Put it in a loop, It does not.
#echo ${readPermissions[#]} | jq 'values[].emailid'
for vals in ${readPermissions[#]}
do
# I would like o extract the email id of the user. The loop is not working atm.
echo ${vals[#]} | jq 'values[].emailid'
done
what am I missing here?
thanks
If you really want to do it this way, that might look like:
readarray -t permissions < <(jq -c '.permissions[]' JsonFileToTest)
for permissionSet in "${permissions[#]}"; do
jq -r '.emailid' <<<"$permissionSet"
done
Note that we're telling jq to print one line per item (with -c), and using readarray -t to read each line into an array element (unlike the array=( $(...command...) ) antipattern, which splits not just on newlines but on other whitespace as well, and expands globs in the process).
But there's no reason whatsoever to do any of that. You'll get the exact same result simply running:
jq -r '.permissions[].emailid' JsonFileToTest
In my bash script, when I run the following jq against my curl result:
curl -u someKey:someSecret someURL 2>/dev/null | jq -r '.schema' | jq -r -c '.fields'
I get back a JSON array as follows:
[{"name":"id","type":"int","doc":"Documentation for the id field."},{"name":"test_string","type":"string","doc":"Documentation for the test_string field"}]
My goal is to do a call with jq applied to return the following (given the example above):
{"id":1234567890,"test_string":"xxxxxxxxxx"}
NB: I am trying to automatically generate templated values that match the "schema" JSON shown above.
So just to clarify, that is:
all array objects (there could be more than 2 shown above) returned in a single comma-delimited row
doc fields are ignored
the values for "name" (including their surrounding double-quotes) are concatenated with either:
:1234567890 ...when the "type" for that object is "int"
":xxxxxxxxxx" ...when the "type" for that object is "string"
NB: these will be the only types we ever get for now
Can someone show me how I can expand upon my initial jq to return this?
NB: I tried working down the following path but am failing beyond this...
curl -u someKey:someSecret someURL 2>/dev/null | jq -r '.schema' | jq -r -c '.fields' | "\(.name):xxxxxxxxxxx"'
If it's not possible in pure JQ (my preference) I'm also happy for a solution that mixes in a bit of sed/awk magic :)
Cheers,
Stan
Given the JSON shown, you could add the following to your pipeline:
jq -c 'map({(.name): (if .type == "int" then 1234567890 else "xxxxxxxxxx" end)})|add'
With that JSON, the output would be:
{"id":1234567890,"test_string":"xxxxxxxxxx"}
However, it would be far better if you combined the three calls to jq into one.
I am using the following curl command:
curl -s -v --user admin:orca --insecure -X GET https://insecure.registry.com/api/v0/repositories/authi-api/tags
Getting following output:
{
"name": "Dev_ReleaseRollout_Lane-3",
"inRegistry": true,
"hashMismatch": false,
"inNotary": false
},
{
"name": "Dev_ReleaseRollout_Lane-latest",
"inRegistry": true,
"hashMismatch": false,
"inNotary": false
},
{
"name": "Payments_Dev_Lane-267",
"inRegistry": true,
"hashMismatch": false,
"inNotary": false
}
I want to get only name values in a variable.
I need only Dev_ReleaseRollout_Lane-3 Dev_ReleaseRollout_Lane-latest Payments_Dev_Lane-267 in a variable
Assuming you actually have an array around the three objects:
$ curl ... | jq -r '.[].name'
Dev_ReleaseRollout_Lane-3
Dev_ReleaseRollout_Lane-latest
Payments_Dev_Lane-267
It's fairly simple, . is the array, [].name take name from each element in the array. -r is raw output.
--raw-output / -r:
With this option, if the filter’s result is a string then it will be written directly to standard output rather than being formatted as a JSON string with quotes. This can be useful for making jq filters talk to non-JSON-based systems.
If the cURL output is actually as mentioned above the following will work:
jq -rRs '"[\(.)]" | fromjson[].name' file.json
However I think there is a better way to wrap an array around input,
-R is raw input and -s is slurp. \(...) is string interpolation.
--slurp/-s:
Instead of running the filter for each JSON object in the input, read the entire input stream into a large array and run the filter just once.