I am trying to update an value for a key based on a select filter. The list contains dictionaries with same key names, but different values. Based on value, I am filtering to select a dictionary and then update the sibling key in it.
The value update works, but the dictionaries move out of list as expected because of .[], but how do I add them back to the list. Or, how can I do it without using .[]?
Input List:
[
{
"key1": "a",
"key2": "b"
},
{
"key1": "c",
"key2": "d"
},
{
"key1": "d",
"key2": "e"
}
]
Command I am running:
jq --arg temp "f" '.[] | select( .key1 == "c").key2 |= $temp' test.json
output:
{
"key1": "a",
"key2": "b"
}
{
"key1": "c",
"key2": "f"
}
{
"key1": "d",
"key2": "e"
}
The objects are not part of list now.
Expected output:
[
{
"key1": "a",
"key2": "b"
},
{
"key1": "c",
"key2": "f"
},
{
"key1": "d",
"key2": "e"
}
]
How can we add the objects back to a list, or do it in-place.
Use map() to keep the original structure:
jq --arg temp "f" 'map(select( .key1 == "c").key2 |= $temp)' test.json
Online demo
You could also just use another pair of parentheses around the left hand side of the update assignment in order to retain the context:
Either way works:
jq --arg temp "f" '(.[] | select(.key1 == "c").key2) |= $temp' test.json
Demo
jq --arg temp "f" '(.[] | select(.key1 == "c")).key2 |= $temp' test.json
Demo
Related
I have a yaml file say sample.yaml with the following structure
items:
- version: v1
data:
file1.json: |-
{
"key1": val1,
"key2": val2
}
- version: v2
data:
file2.json: |-
{
"key3": val3,
"key4": val4
}
In order to achieve the following output by splitting the data into separate files, what commands/operators should I be using in yq4? The documentation is rather confusing.
f1.json:
{
"key1": val1,
"key2": val2
}
f2.json:
{
"key3": val3,
"key4": val4
}
With yq3, I was able to run
files=$(yq read sample.yaml "items[*].data.*" -pp)
for file in $files; do
yq read sample.yaml "$file" > f1.json
done
Any pointers would be greatly appreciated. Thanks!
Using mikefarah/yq v4.14.1+ with from_json and the -oj option to convert to JSON, and the -s option to split up into files
yq -oj -s '"f" + ($index + 1)' '.items[].data[] | from_json' sample.yaml
will produce f1.json:
{
"key1": "val1",
"key2": "val2"
}
and f2.json:
{
"key3": "val3",
"key4": "val4"
}
If you want to use the filenames from the keys instead, you need to employ to_entries to have access to .key and .value. This will produce the files file1.json and file2.json:
yq -oj -s 'parent | .key | sub("\.json$","")' \
'(.items[].data | to_entries)[].value | from_json' sample.yaml
So i make a curl command to a url which returns an array of objects
response= $(curl --locaiton --request GET "http://.....")
I need to iterate over the returned json and extract a single value..
The json is as follows:
{
data:[
{"name": "ABC", "value": 1},
{"name": "EFC", "value": 4},
{"name": "CEC", "value": 3}
]
}
Is there anyway in BASh i can extract the second object value.. by perhaps iterating and doing an IF
Use jq
A simple example to extract the 2nd entry of the array would be:
RESPONSE='{"data":[{"name":"ABC","value": 1},{"name":"EFC","value":4},{"name":"CEC","value":3}]}';
EXTRACTED=$(echo -n "$RESPONSE" | jq '.data[1]');
echo $EXTRACTED
jq is the most commonly used command/tool to parse JSON data from a shell script. Here is an example with your data:
#!/usr/bin/env sh
# JSON response
response='
{
"data": [
{"name": "ABC", "value": 1},
{"name": "EFC", "value": 4},
{"name": "CEC", "value": 3}]
}'
# Name of entry
name='EFC'
# Get value of entry
value=$( jq --null-input --raw-output --arg aName "$name" \
"$response"' | .data[] | select(.name == $aName) | .value')
# Print it out
printf 'value for %s is: %s\n' "$name" "$value"
Alternatively jq can be used to transform the whole JSON name value objects array, into a Bash associative array declaration:
#!/usr/bin/env bash
# JSON response
response='
{
"data": [
{"name": "ABC", "value": 1},
{"name": "EFC", "value": 4},
{"name": "CEC", "value": 3}]
}'
# Name of entry
name='EFC'
# Covert all JSON array name value entries into a Bash associative array
# shellcheck disable=SC2155 # safe generated declaration
declare -A entries="($( jq --null-input --raw-output \
"$response"' | .data[] | ( "[" + ( .name | #sh ) + "]=" + (.value | #sh) )'))"
# Print it out
printf 'value for %s is: %s\n' "$name" "${entries[$name]}"
I've got this json-file:
{
"name": "market",
"type": "grocery",
"shelves": {
"upper_one": [
"23423565",
"23552352",
"08789089"
]
}
}
I need to iterate over every element of an list (upper_one), and replace it with other value.
I've tried this code:
#/bin/bash
for product in $(cat first-shop.json| jq -r '.shelves.upper_one[]')
do
cat first-shop.json| jq --arg id "$((1 + $RANDOM % 10))" --arg product "$product" -r '.shelves.upper_one[]|select(. == $product)|= $id'
done
But I got this kind of output:
1
23552352
08789089
23423565
10
08789089
23423565
23552352
7
Is it possible to iterate over list with jq, replace values with value from another function (like $id in the code), and print the whole final json with substituted values?
I need this kind of output:
{
"name": "market",
"type": "grocery",
"shelves": {
"upper_one": [
"1",
"10",
"7"
]
}
}
not just elements of "upper_one" list thrice.
You could try the following script :
#!/usr/bin/env bash
for product in $(jq -r '.shelves.upper_one[]' input.json)
do
id="$((1 + $RANDOM % 10))"
newIds+=("$id")
done
jq '.shelves.upper_one = $ARGS.positional' input.json --args "${newIds[#]}"
IMHO its better to use some scripting language and manipulate objects programmatically. If bash and jq is your only option - this do the job though not nice
$ jq '.shelves.upper_one[] |= (sub("23423565";"1") | sub("23552352";"10") | sub("08789089";"7"))' your.json
{
"name": "market",
"type": "grocery",
"shelves": {
"upper_one": [
"1",
"10",
"7"
]
}
}
consider conversion to numbers with | tonumber
I'm working with jq 1.6 to get the last entry in an object. It should work like this:
data='{ "1": { "a": "1" }, "2": { "a": "2" }, "3": { "a": "3" } }'
result=`echo $data | jq 'myfilter'`
echo $result
{ "3": { "a": "3" } }
I tried these filters:
jq '. | last' # error: Cannot index object with number
How can I tell jq to quote the number?
jq '. | to_entries | last' # { "key": "3", "value": { "a": "3" } }
I guess I could munge this up by concatenating the key and value entries. Is there a simpler way?
The tutorial and the manual didn't help. No joy on SO either.
You can use the following :
jq 'to_entries | [last] | from_entries'
Try it here.
We can't use with_entries(last) because last returns a single element and from_entries requires an array, hence the [...] construct above.
I am totaly new to shell..... let me ut the proper use case.
Use case:-
I have written two get method in my shell script, and when a user calls that script I will perform some operation for many id's using a for loop. like below
test_get1(){
value1=//performing some operation and storing it
value2=//performing some operation and storing it
//below line I am converting the o/p of value1 and value2 in json
value=$JQ_TOOL -n --arg key1 "$value1" --arg key2 "$value2" '{"key1":"\($value1)","key2":"\($value2)"}'
}
test_get2(){
arr=(1,2,3)
local arr_values=()
for value in arr
do
// Calling test_get1 for each iteraion of this loop, like below
val=$(test_get1 $value)
//below line will store the values in array
arr_values+=("$val")
done
}
When I am doing echo for the above arr_values, I am getting the below output
Output.
arr_values={
"key1":"value1",
"key2":"value2"
}
{
"key1":"value1",
"key2":"value2"
}
I want to convert the above value in json format like below.
json_value=[
{
"key1":"value1",
"key2":"value2"
},
{
"key1":"value1",
"key2":"value2"
}
]
I tried to do it with JQ, but unable to get the proper result.
Use the slurp option:
jq -s . in.json > out.json
in.json
{
"key1": "value1",
"key2": "value2"
}
{
"key1": "value1",
"key2": "value2"
}
out.json
[
{
"key1": "value1",
"key2": "value2"
}
]
[
{
"key1": "value1",
"key2": "value2"
}
]
1) Your existing "value=" line can be simplified to:
value=$(jq -n --arg key1 "$value1" --arg key2 "$value2" '\
{key1: $value1, key2: $value2}')
because --arg always interprets the provided value as a string, and because jq expressions need not follow all the rules of JSON.
2) From your script, arr_value is a bash array of JSON values. To convert it into a JSON array, you should be able to use an incantation such as:
for r in "${a[#]}" ; do printf "%s" "$r" ; done | jq -s .
3) There is almost surely a much better way to achieve your ultimate goal. Perhaps it would help if you thought about calling jq just once.