I'm trying to delete one specific key/value pair from a json file.
My json file is, for this example, params.json
[
{
"ParameterKey": "RTSMMinSize",
"ParameterValue": "1"
},
{
"ParameterKey": "RTSMReplicateDB",
"ParameterValue": "false"
},
{
"ParameterKey": "RTSMSnapshotID",
"ParameterValue": "snapID"
},
{
"ParameterKey": "RTSMEMAIL",
"ParameterValue": ""
}
]
I want to remove the RTSMSnapshotID key value pair entirely as part of my bash script. The file should look like this after:
[
{
"ParameterKey": "RTSMMinSize",
"ParameterValue": "1"
},
{
"ParameterKey": "RTSMReplicateDB",
"ParameterValue": "false"
},
{
"ParameterKey": "RTSMEMAIL",
"ParameterValue": ""
}
]
I thought this would be something as simple as
jq 'del(.RTSMSnapshotID)' params.json
but I'm getting
jq: error (at <filename>): Cannot index array with string "RTSMSnapshotID"
Clearly I don't understand how delete works. Any help?
del(.foo) expects there to be a top-level dictionary with a key named foo. That's not the case here; instead, you have a top-level list with ParameterKey having a series of values, with only one of which you want to remove the entire pair.
jq '[ .[] | select(.ParameterKey != "RTSMSnapshotID") ]'
...or...
jq 'map(select(.ParameterKey != "RTSMSnapshotID"))'
Related
The following command:
cat foo | jq -r '[.items[] | select(.metadata.random_field["foo/foo"] == "true") | .metadata.name]'
results in
[
"test_data"
]
I would like the following output to be:
[
{"name":"test", "data":"test_data"}
]
foo
{
"apiVersion": "v1",
"items":[
{
"metadata": {
"random_field": {
"foo/foo": "true"
},
"creationTimestamp": "2022-03-09T21:54:08Z",
"name": "test_data"
}
}
]
}
UPDATE: added test data so the point can be illustrated properly.
It's not entirely clear where the value for name in the output comes from in your question, but let's assume for a moment that it is the first part until the underscore of the name from the input. The the following produces your required output:
.items | map(
.metadata
| select(.random_field."foo/foo" == "true")
| { name: .name|split("_")|first, data: .name }
)
Output:
[
{
"name": "test",
"data": "test_data"
}
]
I have a question that is an extension/followup to a previous question I've asked:
How do I concatenate dummy values in JQ based on field value, and then CSV-aggregate these concatenations?
In my bash script, when I run the following jq against my curl result:
curl -u someKey:someSecret someURL 2>/dev/null | jq -r '.schema' | jq -r -c '.fields'
I get back a JSON array as follows:
[
{"name":"id", "type":"int"},
{
"name": "agents",
"type": {
"type": "array",
"items": {
"name": "carSalesAgents",
"type": "record"
"fields": [
{
"name": "agentName",
"type": ["string", "null"],
"default": null
},
{
"name": "agentEmail",
"type": ["string", "null"],
"default": null
},
{
"name": "agentPhones",
"type": {
"type": "array",
"items": {
"name": "SalesAgentPhone",
"type": "record"
"fields": [
{
"name": "phoneNumber",
"type": "string"
}
]
}
},
"default": []
}
]
}
},
"default": []
},
{"name":"description","type":"string"}
]
Note: line breaks and indentation added here for ease of reading. This is all in reality a single blob of text.
My goal is to do a call with jq applied to return the following, given the example above (again lines and spaces added for readability, but only need to return valid JSON blob):
{
"id":1234567890,
"agents": [
{
"agentName": "xxxxxxxxxx",
"agentEmail": "xxxxxxxxxx",
"agentPhones": [
{
"phoneNumber": "xxxxxxxxxx"
},
{
"phoneNumber": "xxxxxxxxxx"
},
{
"phoneNumber": "xxxxxxxxxx"
}
]
},
{
"agentName": "xxxxxxxxxx",
"agentEmail": "xxxxxxxxxx",
"agentPhones": [
{
"phoneNumber": "xxxxxxxxxx"
},
{
"phoneNumber": "xxxxxxxxxx"
},
{
"phoneNumber": "xxxxxxxxxx"
}
]
}
],
"description":"xxxxxxxxxx"
}
To summarise, I am trying to automatically generate templated values that match the "schema" JSON shown above.
So just to clarify, the values for "name" (including their surrounding double-quotes) are concatenated with either:
:1234567890 ...when the "type" for that object is "int"
":xxxxxxxxxx" ...when the "type" for that object is "string"
...and when type is "array" or "record" the appropriate enclosures are added {} or [] with the nested content inside.
if its an array of records, generate TWO records for the output
The approach I have started down to cater for parsing nested content like this is to have a series of if-then-else's for every combination of each possible jq type.
But this is fast becoming very hard to manage and painful. From my initial scratch efforts...
echo '[{"name":"id","type":"int"},{"name":"test_string","type":"string"},{"name":"string3ish","type":["string","null"],"default":null}]' | jq -c 'map({(.name): (if .type == "int" then 1234567890 else (if .type == "string" then "xxxxxxxxxx" else (if .type|type == "array" then "xxARRAYxx" else "xxUNKNOWNxx" end) end) end)})|add'
I was wondering if anyone knew of a smarter way to do this in bash/shell with JQ.
PS: I have found alternate solutions for such parsing using Java and Python modules, but JQ is preferable for a unique case of limitations around portability. :)
Thanks!
jq supports functions. Those functions can recurse.
#!/usr/bin/env jq -f
# Ignore all but the first type, in the case of "type": ["string", "null"]
def takeFirstTypeFromArray:
if (.type | type) == "array" then
.type = .type[0]
else
.
end;
def sampleData:
takeFirstTypeFromArray |
if .type == "int" then
1234567890
elif .type == "string" then
"xxxxxxxxxx"
elif .type == "array" then # generate two entries for any test array
[(.items | sampleData), (.items | sampleData)]
elif .type == "record" then
(.fields | map({(.name): sampleData}) | add)
elif (.type | type) == "array" then
(.type[] | sampleData)
elif (.type | type) == "object" then
(.type | sampleData)
else
["UNKNOWN", .]
end;
map({(.name): sampleData}) | add
in shell I have the following object
[
{
"ParameterKey":"a",
"ParameterValue":"1"
},
{
"ParameterKey":"b",
"ParameterValue":"2"
},
{
"ParameterKey":"c",
"ParameterValue":"3"
},
{
"ParameterKey":"d",
"ParameterValue":"4"
}
]
I need to find out how to access the ParameterKeys 'b' and 'd' in an optimal way (WITHOUT using array indexes) using e.g. jq so that the resulting array would be:
[
{
"ParameterKey":"a",
"ParameterValue":"1"
},
{
"ParameterKey":"b",
"ParameterValue":"5"
},
{
"ParameterKey":"c",
"ParameterValue":"3"
},
{
"ParameterKey":"d",
"ParameterValue":"6"
}
]
This will change a single ParameterValue for a ParameterName
jq '[.[] | select(.ParameterKey == "b") .ParameterValue |= "5"]'
[
{
"ParameterKey": "a",
"ParameterValue": "1"
},
{
"ParameterKey": "b",
"ParameterValue": "5"
},
{
"ParameterKey": "c",
"ParameterValue": "3"
},
{
"ParameterKey": "d",
"ParameterValue": "4"
}
]
Try it online!
If you do not want to hardcode the key and value, you can use --arg
jq --arg key "b" --arg value "5" '[.[] | select(.ParameterKey == $key) .ParameterValue |= $value]' </tmp/data.json
[
{
"ParameterKey": "a",
"ParameterValue": "1"
},
{
"ParameterKey": "b",
"ParameterValue": "5"
},
{
"ParameterKey": "c",
"ParameterValue": "3"
},
{
"ParameterKey": "d",
"ParameterValue": "4"
}
]
Hope this helps!
It's not clear exactly what you're asking but the following jq filter solves one interpretation of the question:
map(if .ParameterKey == "b" then .ParameterValue = "5"
elif .ParameterKey == "d" then .ParameterValue = "6"
else . end)
Here's an alternative approach using a single JSON object to define the key/value pairs defining the update:
map( {(.ParameterKey): .ParameterValue} )
| add + {b:"5", d:"6"}
| to_entries
| map( {ParameterKey: .key, ParameterValue: .value} )
I have to parse a JSON which includes an array of of class/name:
{
"_class": "model.ListView",
"jobs": [
{
"_class": "hudson.matrix.MatrixProject",
"name": "tests-different-node-full"
},
{
"_class": "hudson.matrix.MatrixProject",
"name": "tests-jms-activemq-full"
},
{
"_class": "hudson.matrix.MatrixProject",
"name": "tests-txpropag-jpa-full"
}
]
}
I need to retrieve the list of "name".
Looking at the examples I've found of jq, I have tried with:
cat jobs.json | jq '.[].name'
It fails with:
jq: error (at <stdin>:0): Cannot index string with string "name"
How should I reference the name element of the array?
Thanks
found it:
cat jobs.json | jq '.jobs[].name'
Have a big json like this
"envConfig": {
"environmentName": {
"versions": [
{
"name": "version1",
"value": "Dev"
},
{
"name": "version2",
"host": "qa"
}
],
"userRoles": [
{
"name": "Roles",
"entry": [
{
"name": "employees",
"value": "rwx"
},
{
"name": "customers",
"value": "rx"
}
]
}
]
}
},
I wanted to change the JSON attribute from "environmentName" to "prod". Below is the output i am expecting
"envConfig": {
"prod": {
"versions": [
...
],
"userRoles": [
...
]
}
}
Tried with sed command as below
sed "s/\('environmentName':\)/\1\"prod\"\,/g" version.json
Tried with jq as below but not working
cat version.json | jq ' with_entries(.value |= {"prod" : .environmentName} ) '
Any help here to replace the attribute/key of an json with desired value
You weren't too far off with the jq, how about this?
jq '.envConfig |= with_entries(.key |= sub("^environmentName$"; "prod"))'
Two differences: first off, we want to drill down to envConfig before doing a with_entries, and second, when we get there, the thing we want will be a key, not a value. In case there are any other keys besides environmentName they'll be preserved.
TL,TR
You can use the following command:
jq '(.envConfig |= (. + {"prod":.environmentName}|del(.environmentName)))' foo.json
Let's say you have the following json:
{
"foo": {
"hello" : "world"
}
}
You can rename the node foo to bar by first duplicating it and then remove the original node:
jq '. + {"bar":.foo}|del(.foo)' foo.json
Output:
{
"bar": {
"hello" : "world"
}
}
It get's a bit more complicated if you want to replace a child key somewhere in the tree. Let's say you have the following json:
{
"test": {
"foo": {
"hello": "world"
}
}
}
You can use the following jq command for that:
jq '(.test |= (. + {"bar":.foo}|del(.foo)))' foo.json
Note the additional parentheses and the use of the assignment operator |=.
Output:
{
"test": {
"bar": {
"hello": "world"
}
}
}
Using sed:
sed -i '/^ \"environmentName\":/ s/environmentName/prod/' <yourfile>
Keep in mind that -i will overwrite the file. You may want to make a backup first.