I'm wondering how i can escape the newValue so jq replaces the value in :
#!/bin/sh
test="{ \"one.one\" : { \"version\" : \"0.0.1\" }, \"two.two\" : { \"version\" : \"0.0.2\" } }"
newvalue="3.3.3"
newjson=$(echo $test | jq '."two.two".version = "$newvalue" ')
echo $newjson
outputs:
{ "one.one": { "version": "0.0.1" }, "two.two": { "version": "$newvalue" } }
none of the escapes i tried \" \' around $newvalue seems to work
First of all, to use a variable, it can't be in quotes. Replace
"$newvalue"
with
$newvalue
Secondly, you never set $newvalue in the jq program. To achieve that, you can use the following:
--arg newvalue "$newvalue"
Related
I'm trying to create a json file via shellscript, and someone has mentioned jq, but I'm struggling a little bit to make it work:
The desire output is:
inboundurls{
"op": "add",
"path": "/support",
"apiSupports": [
{
"familyType": "EXAMPLE",
"healthCheckUris": "http://example.com"
}
],
"inboundurls": [
{
"healthCheckUris": "http://example.com"
}
]
}
Researching about I found a start point, but it's not working properly, I need some help, here is what I have:
script:
#!/bin/bash
apiSupports=$(jq -n --arg familyType EXAMPLE \
--arg healthCheckUris http://example.com \
'$ARGS.named'
)
final=$(jq -n --arg op "add" \
--arg path "/supportServices" \
--argjson apiSupports "[$apiSupports]" \
'$ARGS.named'
)
echo "$final"
the output of the script above:
{
"op": "add",
"path": "/supportServices",
"apiSupports": [
{
"familyType": "EXAMPLE",
"healthCheckUris": "http://example.com"
}
]
}
If anyone could help me I would be glad, or even suggesting Ideas, thank you in advance?
The following produces the valid JSON component of what is shown as the desired output:
jq -n --arg op "add" \
--arg path "/support" \
--arg familyType EXAMPLE \
--arg healthCheckUris http://example.com '
{$op, $path,
apiSupports: [ {$familyType, $healthCheckUris }],
inboundurls: [ {$healthCheckUris }]
}
'
I have a Bash script that uses jq and a for loop to iterate through an array, grab a directory that I need to be monitored by Amazon CloudWatch, and stick it into the latter's JSON configuration file. However, for some reason, only the last item in the array is actually being written. I assume there's something in my logic that is not appending my changes, and instead overwriting them in a particular place, but I can't quite figure out the fix.
Here is my code:
logPaths=("/shared/logs/application/application1"
"/shared/logs/application/application2"
"/shared/logs/application/application3")
# Loop through array to create stanzas and export them to the temp file
for i in ${logPaths[#]}; do
jq "
.logs.logs_collected.files.collect_list[-1] |= . + {
\"file_path\": \"$i\",
\"log_group_name\": \"/aws-account/aws/ec2/syslogs\",
\"log_stream_name\": \"$definedElsewhere\",
\"timestamp_format\": \"%b %d %H:%M:%S\"}" \
/opt/aws/amazon-cloudwatch-agent/amazon-cloudwatch-agent.json \
> /opt/aws/amazon-cloudwatch-agent/amazon-cloudwatch-agent.json.tmp \
&& cp /opt/aws/amazon-cloudwatch-agent/amazon-cloudwatch-agent.json.tmp /opt/aws/amazon-cloudwatch-agent/amazon-cloudwatch-agent.json
done
When this is executed, and I look at amazon-cloudwatch-agent.json, only a record for the 3rd entry in the array (/application3) appears in the configuration file.
I can't reproduce your bug -- but it's irrelevant, because if this were correctly written there wouldn't be any loop needed at all.
Using jq --args allows the logPaths array to be passed in as a set of positional arguments, and referred to from within the relevant jq code as $ARGS.positional. Thus:
#!/usr/bin/env bash
logPaths=("/shared/logs/application/application1"
"/shared/logs/application/application2"
"/shared/logs/application/application3")
# Make up some sample input, since the OP didn't provide any
cat >old.json <<'EOF'
{
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{"test": "make sure this old data is retained"}
]
}
}
}
}
EOF
jq --arg definedElsewhere "Other Value" '
($ARGS.positional | [
.[] | { "file_path": .,
"log_group_name": "/aws-account/aws/ec2/syslogs",
"log_stream_name": $definedElsewhere,
"timestamp_format": "%b %d %H:%M:%S"
}]) as $newLogSinks |
.logs.logs_collected.files.collect_list += $newLogSinks
' --args "${logPaths[#]}" <old.json >new.json && mv new.json old.json
...which correctly emits as output:
{
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"test": "make sure this old data is retained"
},
{
"file_path": "/shared/logs/application/application1",
"log_group_name": "/aws-account/aws/ec2/syslogs",
"log_stream_name": "Other Value",
"timestamp_format": "%b %d %H:%M:%S"
},
{
"file_path": "/shared/logs/application/application2",
"log_group_name": "/aws-account/aws/ec2/syslogs",
"log_stream_name": "Other Value",
"timestamp_format": "%b %d %H:%M:%S"
},
{
"file_path": "/shared/logs/application/application3",
"log_group_name": "/aws-account/aws/ec2/syslogs",
"log_stream_name": "Other Value",
"timestamp_format": "%b %d %H:%M:%S"
}
]
}
}
}
}
I have the following JSON file:
{
"1":
{
"media_content":"test3.xspf"
},
"2":
{
"media_content":"test3.xspf"
}
}
In the terminal, using bash as shell, I can execute the following commands:
export schedules="1"
echo $(jq '.[env.schedules]["media_content"]' json_file.json)
Which results in outputing this:
test3.xspf
So it works as expected, but when I place that jq command in a script and run it, it just returns null.
I did echo the values of schedules to make sure the value is non-null inside the script, and it is ok:
echo $schedules
But I did not manage to find the reason, why this command works when run directly in shell and does not work when run in script.
I run the script in the following ways:
bash script.sh
./script.sh
PS: yes, I did offer execute permission: chmod +x script.sh
HINT: env.schedules represents the environment variable 'schedules', and I did make sure that it is assigned in the script before calling jq.
EDIT: I am posting now a whole script, specifying the files tree.
There is one directory containing:
script.sh
json_file.json
static.json
script.sh:
export zone=$(cat static.json | jq '.["1"]');
echo "json block: "$zone
export schedules="$(echo $zone | jq '.schedules')"
echo "environment variable: "$schedules
export media_content=$(jq '.[env.schedules]["media_content"]' json_file.json)
echo "What I want to get: \"test3.xspf\""
echo "What I get: "$media_content
json_file.json:
{
"1":
{
"media_content":"test3.xspf"
},
"2":
{
"media_content":"test3.xspf"
}
}
static.json:
{
"1":
{
"x": "0",
"y": "0",
"width": "960",
"height": "540",
"schedules":"1"
}
}
If I run the script, it displays:
json block: { "x": "0", "y": "0", "width": "960", "height": "540", "schedules": "1" }
environment variable: "1"
What I want to get: "test3.xspf"
What I get: null
If I hardcode the variable:
export schedules="1"
The problem no longer occurs
The problem is simple.
It's not jq's fault.
It the unproper way the schedule's value is piped to the next command.
You have to remove the "s that surround the variable's value, add the second command that uses sed to do that:
export schedules="$(echo $zone | jq '.schedules')"
schedules=$( echo $schedules | sed s/\"//g )
Long answer
Let's see:
here schedules is a string and echo shows its value as being 1:
export schedules="1" ; echo $schedules
here even though double quotes are not mentioned:
export schedules=1 ; echo $schedules
But the result from this also generates additional "s:
export schedules=$(echo $zone | jq '.schedules')
If you print it now you will see additional "s:
echo $schedules # "1"
So just remove the "s from the value:
schedules=$( echo $schedules | sed s/\"//g )
This question already has answers here:
Add new element to existing JSON array with jq
(3 answers)
Closed 3 years ago.
I want to create valid json using jq in bash.
each time when bash script will execute "Add new element to existing JSON array" and if file is empty create new file.
I am using following jq command to create my json (which is incomplete, please help me to complete it)
$jq -n -s '{service: $ARGS.named}' \
--arg transcationId $TRANSACTION_ID_METRIC '{"transcationId":"\($transcationId)"}' \
--arg name $REALPBPODDEFNAME '{"name ":"\($name )"}'\
--arg lintruntime $Cloudlintruntime '{"lintruntime":"\($lintruntime)"}' \
--arg status $EXITCODE '{"status":"\($status)"}' \
--arg buildtime $totaltime '{"buildtime":"\($buildtime)"}' >> Test.json
which is producing output like
{
"service": {
"transcationId": "12345",
"name": "sdsjkdjsk",
"lintruntime": "09",
"status": "0",
"buildtime": "9876"
}
}
{
"service": {
"transcationId": "123457",
"servicename": "sdsjkdjsk",
"lintruntime": "09",
"status": "0",
"buildtime": "9877"
}
}
but I don't want output in this format
json should be created first time like
what should be jq command for creating below jason
{
"ServiceData":{
"date":"30/1/2020",
"ServiceInfo":[
{
"transcationId":"20200129T130718Z",
"name":"MyService",
"lintruntime":"178",
"status":"0",
"buildtime":"3298"
}
]
}
}
and when I next time execute the bash script element should be added into the array like
what is the jq command for getting json in this format
{
"ServiceData":{
"date":"30/1/2020",
"ServiceInfo":[
{
"transcationId":"20200129T130718Z",
"name":"MyService",
"lintruntime":"16",
"status":"0",
"buildtime":"3256"
},
{
"transcationId":"20200129T130717Z",
"name":"MyService",
"lintruntime":"16",
"status":"0",
"buildtime":"3256"
}
]
}
}
also I want "date " , "service data" , "service info"
fields in my json which are missing in my current one
You don't give a separate filter to each --arg option; it just defines a variable which can be used in the single filter argument. You just want to add new object to your input. jq doesn't do in-place file editing, so you'll have to write to a temporary file and replace your original after the fact.
jq --arg transactionId "$TRANSACTION_ID_METRIC" \
--arg name "$REALPBPODDEFNAME" \
--arg lintruntime "$Cloudlintruntime" \
--arg status "$EXITCODE" \
--arg buildtime "$totaltime" \
'.ServiceData.ServiceInfo += [ {transactionID: $transactionId,
name: $name,
lintruntime: $lintruntime,
status: $status,
buildtime: $buildtime
}]' \
Test.json > tmp.json &&
mv tmp.json Test.json
Here's the same command, but using an array to store all the --arg options and a variable to store the filter so the command line is a little simpler. (You also don't need explicit line continuations inside an array definition.)
args=(
--arg transactionId "$TRANSACTION_ID_METRIC"
--arg name "$REALPBPODDEFNAME"
--arg lintruntime "$Cloudlintruntime"
--arg status "$EXITCODE"
--arg buildtime "$totaltime"
)
filter='.ServiceData.ServiceInfo += [
{
transactionID: $transactionId,
name: $name,
lintruntime: $lintruntime,
status: $status,
buildtime: $buildtime
}
]'
jq "${args[#]}" "$filter" Test.json > tmp.json && mv tmp.json Test.json
I am having below code in JSON file.
{
"comment": {
"vm-updates": [],
"site-ops-updates": [
{
"comment": {
"message": "You can start maintenance on this resource"
},
"hw-name": "Machine has got missing disks. "
}
]
},
"object_name": "4QXH862",
"has_problems": "yes",
"tags": ""
}
I want to separate "hw-name" from this JSON file using jq. I've tried below combinations, but nothing worked.
cat jsonfile | jq -r '.comment[].hw-name'
cat json_file.json | jq -r '.comment[].site-ops-updates[].hw-name'
Appreciated help from StackOverflow!!!
It should be:
▶ cat jsonfile | jq -r '.comment."site-ops-updates"[]."hw-name"'
Machine has got missing disks.
Or better still:
▶ jq -r '.comment."site-ops-updates"[]."hw-name"' jsonfile
Machine has got missing disks.
From the docs:
If the key contains special characters, you need to surround it with double quotes like this: ."foo$", or else .["foo$"].