I have a json with 3 equal lists:
{
"value": {
"list1": [
"el11",
"el12",
"el13",
"el14"
],
"list2": [
"el21",
"el22",
"el23",
"el24"
],
"list3": [
"el31",
"el32",
"el33",
"el34"
]
}
}
I'm trying to extract from each list the elements with the same index:
el11 el21 el31
...
el13 el23 el33
Ideally those need to be exported as ENV values. But I just want to know if it's possible to do that with jq. And how.
Thank you
You can use the transpose function for this. For example,
$ jq -r '[.value[]] | transpose[] | join (" ")' tmp.json
el11 el21 el31
el12 el22 el32
el13 el23 el33
el14 el24 el34
Related
Let's assume $commandGetEvents is an array of json objects. I use the following command to extract the event Id, which is a number from 1 - 65 and store it in currentEventId. Now let's assume I have another variable called startedEventId which holds the value I'm looking for, which is 22.
Here's an example of the data $commandGetEvents contains.
[
{
"eventId": 22,
"Name" : "Bob"
"Activity" : "Eat Food"
"startedEventId" : 15
},
{
"eventId": 21,
"Name" : "Smith"
"Activity" : "Ride a bike"
"startedEventId" : 13
},
{
"eventId": 20,
"Name" : "Tony"
"Activity": "Print paper"
"startedEventId" : 10
},
]
eventId is the unique identifier of the json object. & startedEventId is the identifier of json object that caused the current one to take place.
currentEventId=$(jq ".[$index].eventId" <<< ${commandGetEvents})
startedEventid=$(jq ".[${eventCounter}].startedEventId" <<< $commandGetEvents)
When i echo both statements in a while loop, I get the following output.
currentEventId = 1
startedEventId = 22
currentEventId = 2
startedEventId = 22
currentEventId = 3
startedEventId = 22
The while loop continues until all elements of currentEventId are exhausted.
My problem is when I compare both statements like this:
if [[ ${startedId} -eq ${currentEventId} ]] ;
then
echo "Equal"
fi
I get the following error message:
line 90: [[: 22: syntax error: operand expected (error token is "22")
The provided "JSON" is invalid as JSON. Please fix it.
When using jq at the bash command line, it's almost always best to enclose the jq program in single-quotation marks; bash shell variables can be passed in using --arg or --argjson. Consider for example the following snippet that assume `commandGetEvents' is valid JSON along the lines suggested in the Q:
index=0
currentEventId=$(jq --argjson index $index '.[$index].eventId' <<< ${commandGetEvents})
echo index=$index
echo currentEventId=$currentEventId
The part of the question involving eventCounter is somewhat obscure, but it looks like the example given immediately above will serve as a guide.
Rather than using bash constructs to iterate through the JSON array, it would almost certainly be better to use jq's support for iteration and selection. For example:
jq '.[] | select(.eventId == 22)' <<< ${commandGetEvents}
yields:
{
"eventId": 22,
"Name": "Bob",
"Activity": "Eat Food",
"startedEventId": 15
}
So if you just want the startedEventId value (or values) corresponding to .eventId == 22, you could write:
jq '.[] | select(.eventId == 22) | .startedEventId' <<< ${commandGetEvents}
This question already has answers here:
Parsing JSON with Unix tools
(45 answers)
Closed 3 years ago.
I have the below json content in my sample file:
{
"listingRequest": {
"id": "016a1050-82dc-1262-cc9b-4baf3e0b7123",
"uri": "http://localhost:9090/nifi-api/flowfile-queues/016a104a-82dc-1262-7d78-d84a704abfbf/listing-requests/016a1050-82dc-1262-cc9b-4baf3e0b7123",
"submissionTime": "04/28/2019 19:40:58.593 UTC",
"lastUpdated": "19:40:58 UTC",
"percentCompleted": 0,
"finished": false,
"maxResults": 100,
"state": "Waiting for other queue requests to complete",
"queueSize": {
"byteCount": 480,
"objectCount": 20
},
"sourceRunning": false,
"destinationRunning": false
}
}
I want to retrieve the value of the byte count i.e. byteCount. The result should be 480.
Using other tools like jq is not allowed to be installed in our ourganization due to restrictions.
How do I do it via sed/grep? I tried grep -Po '"byteCount":.*?[^\\]",' but did not get any output
$ sed -n 's/.*"byteCount": *\([0-9]*\).*/\1/p' file
480
More generally you could use this (using any POSIX awk) to convert your specific format of JSON to a flat file and then print whatever you want by it's tag hierarchy:
$ cat tst.awk
{ gsub(/^[[:space:]]+|[[:space:]]+$/,"") }
match($0,/^"[^"]+"/) {
subTag = substr($0,RSTART+1,RLENGTH-2)
$0 = substr($0,RSTART+RLENGTH)
}
!NF || /^{/ { next }
/^:[[:space:]]*{/ {
preTag = (preTag=="" ? "" : preTag ".") subTag
next
}
/^}/ {
sub(/\.[^.]+$/,"",preTag)
next
}
{
gsub(/^[[:space:]]*:[[:space:]]*|[[:space:]]*,[[:space:]]*$/,"")
tag = preTag "." subTag
val = $0
printf "%s=%s\n", tag, val
}
.
$ awk -f tst.awk file
listingRequest.id="016a1050-82dc-1262-cc9b-4baf3e0b7123"
listingRequest.uri="http://localhost:9090/nifi-api/flowfile-queues/016a104a-82dc-1262-7d78-d84a704abfbf/listing-requests/016a1050-82dc-1262-cc9b-4baf3e0b7123"
listingRequest.submissionTime="04/28/2019 19:40:58.593 UTC"
listingRequest.lastUpdated="19:40:58 UTC"
listingRequest.percentCompleted=0
listingRequest.finished=false
listingRequest.maxResults=100
listingRequest.state="Waiting for other queue requests to complete"
listingRequest.queueSize.byteCount=480
listingRequest.queueSize.objectCount=20
listingRequest.sourceRunning=false
listingRequest.destinationRunning=false
$ awk -f tst.awk file | awk -F'=' '$1=="listingRequest.queueSize.byteCount"{print $2}'
480
I think you could count the characters
a= your_json
b="byteCount"
strindex() {
x="${1%%$2*}"
[[ "$x" = "$1" ]] && echo -1 || echo "${#x}"
}
index= strindex "$a" "$b" #print 4
result ={your_json:(index+11)}
Sources:
https://www.tldp.org/LDP/abs/html/string-manipulation.html
Position of a string within a string using Linux shell script?
I am having a JSON object x and a variable requiredValue
let requiredValue = 5;
let x = [
{"score":1},
{"score":2},
{"score":3}
}
Here using jq first i want to extract all score values and then check if any score value in the object is greater than or equal to requiredValue.
Here is what I tried
jq -r '.[].score | join(",") | contains([requiredValue])'
Suppose if requiredValue is 5 then jq query should return false and if requiredValue is 2 it should return true.
If you split your inputs into two valid JSON documents, rather than having a JavaScript input which is not valid JSON, you could do the following:
requiredValue=5
x='
[{"score":1},
{"score":2},
{"score":3}]
'
jq -n \
--argjson requiredValue "$requiredValue" \
--argjson x "$x" '
[$x[].score | select(. == $requiredValue)] | any
'
The following has been tested with bash:
requiredValue=5
x='[
{"score":1},
{"score":2},
{"score":3}
]'
jq --argjson requiredValue $requiredValue '
any(.[].score; . >= $requiredValue)' <<< "$x"
The result:
false
I have the following input:
[
{"id": "first", "val": 1},
{"id": "second", "val": 2},
{"id": "second", "val": 3}
]
Using the jq filter : .[] | select(.id == "second")
I get following as output :
{
"id": "second",
"val": 2
}
{
"id": "second",
"val": 3
}
I want to get the result in the form of an array. Is it possible to get the multiple result values of select operation in an array?
Yes; wrap the filter in an array :)
$ jq '[.[] | select(.id == "second")]' tmp.json
[
{
"id": "second",
"val": 2
},
{
"id": "second",
"val": 3
}
]
Or, use map/1, which is predefined as [.[] | ...].
$ jq 'map(select(.id == "second"))' tmp.json
[same result]
To wrap the results in a bash array, use the -c option to output each result on a single line, and read the result with readarray.
$ readarray -t arr < <(jq -c '.[] | select(.id == "second")' tmp.json)
$ for r in "${arr[#]}"; do echo "Result: $r"; done
Result: {"id":"second","val":2}
Result: {"id":"second","val":3}
I have the following line:
for custom_field in $(jq -r '.issues[] | .fields.created' 1.json
); do
echo $custom_field
done
Output:
2018-03-06T21:24:41.000+0000
2018-03-06T22:48:47.000+0000
How to compare current datetime with each output and if it's older than 3 hours to print "old"?
Given input like
{ "issues":
[{"id": 1, "fields": {"created": "2018-03-06T21:24:41.000+0000"}},
{"id": 2, "fields": {"created": "2018-03-06T22:48:47.000+0000"}},
{"id": 3, "fields": {"created": "2018-03-09T22:48:47.000+0000"}}]}
you can use the built-in date manipulation functions to print the old records with something like
jq -r '(now-3600*3) as $when | .issues[] |
select(.fields.created | strptime("%Y-%m-%dT%H:%M:%S.000+0000") | mktime < $when) |
[.id, .fields.created, "old"]' 1.json
where the last line probably needs tweaking to produce exactly the output you want.
It is much easier to convert first and subtract the three hours.
Example below converts to seconds and prints true if condition is met.
date_in_seconds=$(date -d $custom_field +"%s");
[ ${date_in_seconds} -gt 259200 ] && echo true;
For non GNU versions of the date command you can use the following;
date_in_seconds=$(date -j -f '%Y-%m-%d %H:%M:%S' "2016-02-22 20:22:14" '+%s')
Keep in mind that the EPOCH will rollover 1 Jan 2036.