i'm trying to build script that takes specific attribute value and store it in the array , this is the following JSON file:
[
{
"id": 1,
"name": "myna",
"description": "Simple Question",
"speaker": "USER",
},
{
"all_Id's": [
"11111"
],
"user": "me",
},
{
"id": 2,
"name": "mry",
"description": "Simple",
"speaker": "aaa",
}
]
as you see object in json file don't have the same attributes so i'm looking only on object has "name " attribute,the following script reads the Json file and return the values of attribute name only ,but i build something wrond as theERROR always on the "{" of the last object in file I don't know why , what i am i doing wrong?
the expected output is : [myna, mry]
#!/bin/bash
declare -a OB_I=()
declare counter1=0
jq -r '.name' file.json ; while read -r val ; do
if [[ ! $val ]]
then
OB_I[$counter]=$val ;
counter=$((counter+1));
fi
done;
$ printf '%s\n' "${OB_I[#]}"
The input of jq is a list, which doesn't have any keys, let alone one named name. You want
jq -r '.[].name'
instead.
Unrelated, you don't need the variable counter. You can simply append to your array with OB_I+=("$val").
Related
So i make a curl command to a url which returns an array of objects
response= $(curl --locaiton --request GET "http://.....")
I need to iterate over the returned json and extract a single value..
The json is as follows:
{
data:[
{"name": "ABC", "value": 1},
{"name": "EFC", "value": 4},
{"name": "CEC", "value": 3}
]
}
Is there anyway in BASh i can extract the second object value.. by perhaps iterating and doing an IF
Use jq
A simple example to extract the 2nd entry of the array would be:
RESPONSE='{"data":[{"name":"ABC","value": 1},{"name":"EFC","value":4},{"name":"CEC","value":3}]}';
EXTRACTED=$(echo -n "$RESPONSE" | jq '.data[1]');
echo $EXTRACTED
jq is the most commonly used command/tool to parse JSON data from a shell script. Here is an example with your data:
#!/usr/bin/env sh
# JSON response
response='
{
"data": [
{"name": "ABC", "value": 1},
{"name": "EFC", "value": 4},
{"name": "CEC", "value": 3}]
}'
# Name of entry
name='EFC'
# Get value of entry
value=$( jq --null-input --raw-output --arg aName "$name" \
"$response"' | .data[] | select(.name == $aName) | .value')
# Print it out
printf 'value for %s is: %s\n' "$name" "$value"
Alternatively jq can be used to transform the whole JSON name value objects array, into a Bash associative array declaration:
#!/usr/bin/env bash
# JSON response
response='
{
"data": [
{"name": "ABC", "value": 1},
{"name": "EFC", "value": 4},
{"name": "CEC", "value": 3}]
}'
# Name of entry
name='EFC'
# Covert all JSON array name value entries into a Bash associative array
# shellcheck disable=SC2155 # safe generated declaration
declare -A entries="($( jq --null-input --raw-output \
"$response"' | .data[] | ( "[" + ( .name | #sh ) + "]=" + (.value | #sh) )'))"
# Print it out
printf 'value for %s is: %s\n' "$name" "${entries[$name]}"
I've got this json-file:
{
"name": "market",
"type": "grocery",
"shelves": {
"upper_one": [
"23423565",
"23552352",
"08789089"
]
}
}
I need to iterate over every element of an list (upper_one), and replace it with other value.
I've tried this code:
#/bin/bash
for product in $(cat first-shop.json| jq -r '.shelves.upper_one[]')
do
cat first-shop.json| jq --arg id "$((1 + $RANDOM % 10))" --arg product "$product" -r '.shelves.upper_one[]|select(. == $product)|= $id'
done
But I got this kind of output:
1
23552352
08789089
23423565
10
08789089
23423565
23552352
7
Is it possible to iterate over list with jq, replace values with value from another function (like $id in the code), and print the whole final json with substituted values?
I need this kind of output:
{
"name": "market",
"type": "grocery",
"shelves": {
"upper_one": [
"1",
"10",
"7"
]
}
}
not just elements of "upper_one" list thrice.
You could try the following script :
#!/usr/bin/env bash
for product in $(jq -r '.shelves.upper_one[]' input.json)
do
id="$((1 + $RANDOM % 10))"
newIds+=("$id")
done
jq '.shelves.upper_one = $ARGS.positional' input.json --args "${newIds[#]}"
IMHO its better to use some scripting language and manipulate objects programmatically. If bash and jq is your only option - this do the job though not nice
$ jq '.shelves.upper_one[] |= (sub("23423565";"1") | sub("23552352";"10") | sub("08789089";"7"))' your.json
{
"name": "market",
"type": "grocery",
"shelves": {
"upper_one": [
"1",
"10",
"7"
]
}
}
consider conversion to numbers with | tonumber
I have this example test.json file:
{
"folder": {
"name": "Desktop",
"path": "Users/user/Desktop",
"executable": false
}
}
I am trying to read the object contained in 'folder' into a variable $var as an array. Desired output - $var contains the following:
[
{
"name": "Desktop",
"path": "Users/user/Desktop",
"executable": false
}
]
It seemed relatively straightforward. jq '.folder' test.json to read in the folder value, and then pipe the output using the -s flag to get it enclosed in an array. This command executes just fine:
jq '.folder' test.json | jq -s
However, when I try to save the result to a variable, it throws an error displaying the jq usage statement. What am I missing? Is there a different way to accomplish this?
var=$(jq '.folder' test.json | jq -s)
echo $var
I also found out that this works fine in zsh but throws an error in bash. Not sure if that's relevant and whether it has to do with different versions of jq
Is this what you were expecting ?
$ var="$(jq "[.folder]" test.json)"
$ echo "$var"
[
{
"name": "Desktop",
"path": "Users/user/Desktop",
"executable": false
}
]
I have a json object with below element,
rsrecords="{
"ResourceRecords": [
{
"Value": "\"heritage=external-dns,external-dns/owner=us-east-1:sandbox,external-dns/resource=service/api""
}
],
"Type": "TXT",
"Name": "\\052.apiconsumer.alpha.sandbox.test.net.",
"TTL": 300
}"
And in my bash script,I have below code snippet,
jq -r '.[] | .Name ,.ResourceRecords[0].Value' <<< "$rsrecords" | \
while read -r name; read -r value; do
echo $name
Output is printed as,
\052.apiconsumer.alpha.sandbox.test.net.
But I am expecting it to print as \\052.apiconsumer.alpha.sandbox.test.net., which is , as it is "Name" from the json object..
How can this be done?
Before getting to the heart of the matter, please note that
the sample data as given is a bit of a mishmash, so I'll assume you meant something like:
rsrecords='
{
"ResourceRecords": [
{
"Value": "heritage=external-dns,external-dns/owner=us-east-1:sandbox,external-dns/resource=service/api"
}
],
"Type": "TXT",
"Name": "\\052.apiconsumer.alpha.sandbox.test.net.",
"TTL": 300
}
'
Your jq query does not match the above JSON, so I'll assume you intended the query to be simply:
.Name, .ResourceRecords[0].Value
In any case, with the above JSON, the bash commands:
jq -r '.Name, .ResourceRecords[0].Value' <<< "$rsrecords" |
while read -r name; read -r value; do
echo "$name"
done
yields:
\052.apiconsumer.alpha.sandbox.test.net.
This is correct, because the JSON string "\\X" is an encoding of the raw string: \X
If you want to see the JSON string, then invoke jq without the -r option. If you want to invoke jq with the -r option and want to see two backslashes, you will have to encode them as four backslashes in your JSON.
In the below script, I am not able to successfully call the "repovar" variable in the jq command.
cat quayrepo.txt | while read line
do
export repovar="$line"
jq -r --arg repovar "$repovar" '.data.Layer| .Features[] | "\(.Name), \(.Version), $repovar"' severity.json > volume.csv
done
The script uses a text file to loop through the repo names
quayrepo.txt---> file has the list of names in this case the file has a value of "Reponame1"
sample input severity.json file:
{
"status": "scanned",
"data": {
"Layer": {
"IndexedByVersion": 3,
"Features": [
{
"Name": "elfutils",
"Version": "0.168-1",
"Vulnerabilities": [
{
"NamespaceName": "debian:9",
"Severity": "Medium",
"Name": "CVE-2016-2779"
}
]
}
]
}
}
}
desired output:
elfutils, 0.168-1, Medium, Reponame1
Required output: I need to retrieve the value of my environment variable as the last column in my output csv file
You need to surround $repovar with parenthesis, as the other values
repovar='qweqe'; jq -r --arg repovar "$repovar" '.data.Layer| .Features[] | "\(.Name), \(.Version), \($repovar)"' tmp.json
Result:
elfutils, 0.168-1, qweqe
There's no need for the export.
#!/usr/bin/env bash
while read line
do
jq -r --arg repovar "$line" '.data.Layer.Features[] | .Name + ", " + .Version + ", " + $repovar' severity.json
done < quayrepo.txt > volume.csv
with quayrepo.txt as
Reponame1
and severity.json as
{
"status": "scanned",
"data": {
"Layer": {
"IndexedByVersion": 3,
"Features": [
{
"Name": "elfutils",
"Version": "0.168-1",
"Vulnerabilities": [
{
"NamespaceName": "debian:9",
"Severity": "Medium",
"Name": "CVE-2016-2779"
}
]
}
]
}
}
}
produces volume.csv containing
elfutils, 0.168-1, Reponame1
To #peak's point, changing > to >> in ...severity.json >> volume.csv will create a multi-line csv instead of just overwriting until the last line
You don't need a while read loop in bash at all; jq itself can loop over your input lines, even when they aren't JSON, letting you run jq only once, not once per line in quayrepo.txt.
jq -rR --slurpfile inJson severity.json <quayrepo.txt >volume.csv '
($inJson[0].data.Layer | .Features[]) as $features |
[$features.Name, $features.Version, .] |
#csv
'
jq -R specifies raw input, letting jq directly read lines from quayrepo.txt into .
jq --slurpfile varname filename.json reads filename.json into an array of JSON objects parsed from that file. If the file contains only one object, one needs to refer to $varname[0] to refer to it.
#csv converts an array to a CSV output line, correctly handling data with embedded quotes or other oddities that require special processing.