I am having below code in JSON file.
{
"comment": {
"vm-updates": [],
"site-ops-updates": [
{
"comment": {
"message": "You can start maintenance on this resource"
},
"hw-name": "Machine has got missing disks. "
}
]
},
"object_name": "4QXH862",
"has_problems": "yes",
"tags": ""
}
I want to separate "hw-name" from this JSON file using jq. I've tried below combinations, but nothing worked.
cat jsonfile | jq -r '.comment[].hw-name'
cat json_file.json | jq -r '.comment[].site-ops-updates[].hw-name'
Appreciated help from StackOverflow!!!
It should be:
▶ cat jsonfile | jq -r '.comment."site-ops-updates"[]."hw-name"'
Machine has got missing disks.
Or better still:
▶ jq -r '.comment."site-ops-updates"[]."hw-name"' jsonfile
Machine has got missing disks.
From the docs:
If the key contains special characters, you need to surround it with double quotes like this: ."foo$", or else .["foo$"].
Related
I have the following json format, basically it is a huge file with several of such entries.
[
{
"id": "kslhe6em",
"version": "R7.8.0.00_BNK",
"hostname": "abacus-ap-hf-test-001:8080",
"status": "RUNNING",
},
{
"id": "2bkaiupm",
"version": "R7.8.0.00_BNK",
"hostname": "abacus-ap-hotfix-001:8080",
"status": "RUNNING",
},
{
"id": "rz5savbi",
"version": "R7.8.0.00_BNK",
"hostname": "abacus-ap-hf-test-005:8080",
"status": "RUNNING",
},
]
I wanted to fetch all the hostname values that starts with "abacus-ap-hf-test" and without ":8080" into a variable and then wanted to use those values for further commands over a for loop something like below. But, am bit confused how can I extract such informaion.
HOSTAME="abacus-ap-hf-test-001 abacus-ap-hf-test-005"
for HOSTANAME in $HOSTNAME
do
sh ./trigger.sh
done
The first line command update to this:
HOSTAME=$(grep -oP 'hostname": "\K(abacus-ap-hf-test[\w\d-]+)' json.file)
or if you sure that the hostname end with :8080", try this:
HOSTAME=$(grep -oP '(?<="hostname": ")abacus-ap-hf-test[\w\d-]+(?=:8080")' json.file)
you will find that abacus-ap-hf-test[\w\d-]+ is the regex, and other strings are the head or the end of the regex content which for finding result accuracy.
Assuming you have valid JSON, you can get the hostname values using jq:
while read -r hname ; do printf "%s\n" "$hname" ; done < <(jq -r .[].hostname j.json)
Output:
abacus-ap-hf-test-001:8080
abacus-ap-hotfix-001:8080
abacus-ap-hf-test-005:8080
I realize this is a simple question but I haven't been able to find the answer. Thank you to anyone who may be able to help me understand what I am doing wrong.
Goal: Search and replace a string in a specific key in a JSON file with a string in a Bash variable using jq.
For example, in the following JSON file:
"name": "Welcome - https://docs.mysite.com/",
would become
"name": "Welcome",
Input (file.json)
[
{
"url": "https://docs.mysite.com",
"name": "Welcome - https://docs.mysite.com/",
"Ocurrences": "679"
},
{
"url": "https://docs.mysite.com",
"name": "Welcome",
"Ocurrences": "382"
}
]
Failing script (using variable)
siteUrl="docs.mysite.com"
jq --arg siteUrl "$siteUrl" '.[].name|= (gsub(" - https://$siteUrl/"; ""))' file.json > file1.json`
Desired output (file1.json)
[
{
"url": "https://docs.mysite.com",
"name": "Welcome",
"Ocurrences": "679"
},
{
"url": "https://docs.mysite.com",
"name": "Welcome",
"Ocurrences": "382"
}
]
I've tried various iterations on removing quotes, changing between ' and ", and adding and removing backslashes.
Successful script (not using variable)
jq '.[].name|= (gsub(" - https://docs.mysite.com/"; ""))' file.json > file1.json
More specifically, if it matters, I am processing an export of a website's usage data from Azure App Insights. Unfortunately, the same page may be assigned different names. I sum the Ocurrences of the two objects with the newly identical url later. If it is better to fix this in App Insights I am grateful for that insight, although I know Bash better than Kusto queries. I am grateful for any help or direction.
Almost. Variables are not automatically expanded within a string. You must interpolate them explicitly with \(…):
jq --arg siteUrl 'docs.mysite.com' '.[].name |= (gsub(" - https://\($siteUrl)/"; ""))' file.json
Alternatively, detect a suffix match and extract the prefix by slicing:
jq --arg siteUrl 'docs.mysite.com' '" - https://\($siteUrl)/" as $suffix | (.[].name | select(endswith($suffix))) |= .[:$suffix|-length]' file.json
I'm trying to check that all instances attached to an AWS ELB are in a state of "InService",
For that, I created an AWS CLI command to check the status of the instances.
problem is that the JSON output returns the status of both instances.
So it is not that trivial to examine the output as I wish.
When I run the command:
aws elb describe-instance-health --load-balancer-name ELB-NAME | jq -r '.[] | .[] | .State'
The output is:
InService
InService
The complete JSON is:
{
"InstanceStates": [
{
"InstanceId": "i-0cc1e6d50ccbXXXXX",
"State": "InService",
"ReasonCode": "N/A",
"Description": "N/A"
},
{
"InstanceId": "i-0fc21ddf457eXXXXX",
"State": "InService",
"ReasonCode": "N/A",
"Description": "N/A"
}
]
}
What I've done so far is creating that one liner shell command:
export STR=$'InService\nInService'
if aws elb describe-instance-health --load-balancer-name ELB-NAME | jq -r '.[] | .[] | .State' | grep -q "$STR"; then echo 'yes'; fi
But I get "yes" as long as there is "InService" at the first command output
Is there a way I can get TRUE/YES only if I get twice "InService" as an output?
or any other way to determine that this is indeed what I got in return?
Without seeing an informative sample of the JSON it's not clear what the best solution would be, but the following meets the functional requirements as I understand them, without requiring any further post-processing:
jq -r '
def count(stream): reduce stream as $s (0; .+1);
if count(.[][] | select(.State == "InService")) > 1 then "yes" else empty end
'
This question already has answers here:
Add new element to existing JSON array with jq
(3 answers)
Closed 3 years ago.
I want to create valid json using jq in bash.
each time when bash script will execute "Add new element to existing JSON array" and if file is empty create new file.
I am using following jq command to create my json (which is incomplete, please help me to complete it)
$jq -n -s '{service: $ARGS.named}' \
--arg transcationId $TRANSACTION_ID_METRIC '{"transcationId":"\($transcationId)"}' \
--arg name $REALPBPODDEFNAME '{"name ":"\($name )"}'\
--arg lintruntime $Cloudlintruntime '{"lintruntime":"\($lintruntime)"}' \
--arg status $EXITCODE '{"status":"\($status)"}' \
--arg buildtime $totaltime '{"buildtime":"\($buildtime)"}' >> Test.json
which is producing output like
{
"service": {
"transcationId": "12345",
"name": "sdsjkdjsk",
"lintruntime": "09",
"status": "0",
"buildtime": "9876"
}
}
{
"service": {
"transcationId": "123457",
"servicename": "sdsjkdjsk",
"lintruntime": "09",
"status": "0",
"buildtime": "9877"
}
}
but I don't want output in this format
json should be created first time like
what should be jq command for creating below jason
{
"ServiceData":{
"date":"30/1/2020",
"ServiceInfo":[
{
"transcationId":"20200129T130718Z",
"name":"MyService",
"lintruntime":"178",
"status":"0",
"buildtime":"3298"
}
]
}
}
and when I next time execute the bash script element should be added into the array like
what is the jq command for getting json in this format
{
"ServiceData":{
"date":"30/1/2020",
"ServiceInfo":[
{
"transcationId":"20200129T130718Z",
"name":"MyService",
"lintruntime":"16",
"status":"0",
"buildtime":"3256"
},
{
"transcationId":"20200129T130717Z",
"name":"MyService",
"lintruntime":"16",
"status":"0",
"buildtime":"3256"
}
]
}
}
also I want "date " , "service data" , "service info"
fields in my json which are missing in my current one
You don't give a separate filter to each --arg option; it just defines a variable which can be used in the single filter argument. You just want to add new object to your input. jq doesn't do in-place file editing, so you'll have to write to a temporary file and replace your original after the fact.
jq --arg transactionId "$TRANSACTION_ID_METRIC" \
--arg name "$REALPBPODDEFNAME" \
--arg lintruntime "$Cloudlintruntime" \
--arg status "$EXITCODE" \
--arg buildtime "$totaltime" \
'.ServiceData.ServiceInfo += [ {transactionID: $transactionId,
name: $name,
lintruntime: $lintruntime,
status: $status,
buildtime: $buildtime
}]' \
Test.json > tmp.json &&
mv tmp.json Test.json
Here's the same command, but using an array to store all the --arg options and a variable to store the filter so the command line is a little simpler. (You also don't need explicit line continuations inside an array definition.)
args=(
--arg transactionId "$TRANSACTION_ID_METRIC"
--arg name "$REALPBPODDEFNAME"
--arg lintruntime "$Cloudlintruntime"
--arg status "$EXITCODE"
--arg buildtime "$totaltime"
)
filter='.ServiceData.ServiceInfo += [
{
transactionID: $transactionId,
name: $name,
lintruntime: $lintruntime,
status: $status,
buildtime: $buildtime
}
]'
jq "${args[#]}" "$filter" Test.json > tmp.json && mv tmp.json Test.json
I use the following code : https://github.com/sararob/ml-talk-demos/blob/master/speech/request.sh
to fit my own bash script.
cat <<EOF > $JSONFILENAME
{
"config": {
"encoding":"LINEAR16",
"sampleRateHertz":8000,
"languageCode": "nl-NL",
"speechContexts": {
"phrases": ['']
},
"maxAlternatives": 1
},
"audio": {
"content":
}
}
EOF
base64 $1 -w 0 > $SOUNDFILE.base64
#MYBASE64=$(base64 $1 -w 0)
sed -i $JSONFILENAME -e "/\"content\":/r $SOUNDFILE.base64"
#sed -i $JSONFILENAME -e "/\"content\":/r $MYBASE64"
curl -s -X POST -H "Content-Type: application/json" --data-binary #${JSONFILENAME} https://speech.googleapis.com/v1/speech:recognize?key=$API_KEY
The base64 output is correctly filled in by the sed command, however there are also newlines added.
This is the Google API response :
{
"error": {
"code": 400,
"message": "Invalid JSON payload received. Unexpected token.\n\": {\n \"content\":\nUklGRqTIAgBXQVZFZm10\n ^",
"status": "INVALID_ARGUMENT"
}
}
How can I make sure the "content" in my JSON-object is a continuous string of base64 ?
You should avoid updating JSON data with sed.
If you have valid JSON data (i.e you have to fix the lines "phrases": [] and "content": "", you could use jq instead:
jq ".audio.content = \"$(base64 -w 0 "$1")\"" "$JSONFILENAME"
I don't recommend sed, but in this case where a large entry must be appended, you could try this:
echo \"$(base64 -w 0 "$1")\" > "$SOUNDFILE.base64"
sed -i "$JSONFILENAME" -e "/\"content\":/r $SOUNDFILE.base64"
The google error you receive is likely due to the fact that the string is not double quoted.