When I run a command I get a response like this
{
"status": "available",
"managed": true,
"name":vdisk7,
"support":{
"status": "supported"
},
"storage_pool": "pfm9253_pfm9254_new",
"id": "ff10abad"-2bf-4ef3-9038-9ae7f18ea77c",
"size":100
},
and hundreds of this type of lists or dictionaries
I want a command that does such sort of a thing
if name = "something",
get the id
Any links that would help me in learning such sort of commands would be highly appreciated
I have tried
awk '{if ($2 == "something") print $0;}'
But I think the response is in Json so the colum wise awk formatting is not working.
Also it's just a single command that I need to run so I would prefer not to use any external library.
JSON parser is better for this task
awk and sed are utilities to parse line-oriented text, but not json. What if your json formatting will change ? (some lines will go on one line ?).
You should use any standard json parser out there. Or use some powerful scripting language, such as PHP, Python, Ruby, etc.
I can provide you with example on how to do it with python.
What if I can't use powerful scripting language ?
If you totally unable to use python, then there is utility jq out there: link
If you have some recent distro, jq maybe already in repositories (example: Ubuntu 13.10 has it in repos).
I can use python!
I would do that using simple python inline script.
For example we have some some_command that returns json as a result.
We have to get value of data["name"].
Here we go:
some_command | python -c "import json, sys; print json.load(sys.stdin)['name']"
It will output vdisk7 in your case
For this to work you need to be sure, json is fully valid.
If you have a list of json objects:
[
{
...
"name": "vdisk17"
...
},
{
...
"name": "vdisk18"
...
},
{
...
"name": "vdisk19"
...
},
...
]
You could use some list comprehensions:
some_command | python -c "import json, sys; [sys.stdout.write(x['name'] + '\n') for x in json.load(sys.stdin)]"
It will output:
vdisk17
vdisk18
vdisk19
Related
I'd like to use jq to format the output of some known keys in my objects more succinctly.
Sample object:
// test.json
[
{
"target": "some-string",
"datapoints": [
[
123,
456
],
[
789,
101112
]
]
}
]
I'd like to use JQ (with some incantation) to change this to put all the datapoints objects on a single line. E.g.
[
{
"target": "some-string",
"datapoints": [[ 123, 456 ], [ 789, 101112 ]]
}
]
I don't really know if JQ allows this. I searched around - and found custom formatters like https://www.npmjs.com/package/perfect-json which seem to do what I want. I'd prefer to have a portable incantation for this using jq alone (and/or with standard *nix tools).
Use a two-pass approach. In the first, stringify the field using special markers so that in the second pass, they can be removed.
Depending on your level of paranoia, this second pass could be very simple or quite complex. On the simple end of the spectrum, choose markers that simply will not occur elsewhere, perhaps "<q>…</q>", or using some combination of non-ASCII characters. On the complex end of the spectrum, only remove the markers if they occur in the fields in which they are known to be markers.
Both passes could be accomplished with jq, along the lines of:
jq '.[].datapoints |= "<q>\(tojson)</q>"' |
jq -Rr 'sub("<q>(?<s>.*)</q>"; .s)'
Using jq and perl :
jq 'map(.datapoints |= "\u001b\(tojson)\u001b")
' test.json | perl -pe 's/"\\u001b(.*?)\\u001b"/$1/g'
I'm making a query into a rest api, from this result i got:
{ "meta": { "query_time": 0.004266858, "pagination": { "offset": 0, "limit": 00, "total": 4 }, "powered_by": "device-api", "trace_id": "foo" }, "resources": [ "foo/bar", "foo/bar/2", "foo/bar/3", "foo/bar/4" ], "errors": [] }
I want to take results only from resources like this:
"resources": [
"foo/bar",
"foo/bar/2",
"foo/bar/3",
"foo/bar/4"
],
Can we share some knowledge? thanks a lot!
PS: these results from resources are random
Don't use grep or other regular expression tools to parse JSON. JSON is structured data and should be processed by a tool designed to read JSON. On the command line jq is a great tool for this purpose. There are many powerful JSON libraries written in other languages if jq isn't what you need.
Once you've extracted the data you care about, you can use the shuf utility to select random lines, e.g. shuf -n 5 would sample five random lines from the input.
With the JSON you've provided this appears to do what I think you want:
jq --raw-output '.resources[]' | shuf -n 2
You may need to tweak the jq syntax slightly if the real JSON has a different structure.
I am trying to process some string which has special characters in it like abc123#45 or ab$123 or qwe&123.
I am trying to fetch it in shell like:
In json file : foo=qwe$123
foo=`cat tmp_json | jq -r '.keys.foo'`
But it is coming like :
foo=qwe23
JSON input
{
"metadata": {
"name": "xyz",
"version": 7,
"lastUpdated": 1585551422521
},
"keys": {
"abc": "qwe$123",
"foo": "qwe$123"
}
}
When shell strings contain special characters that you do not want to be interpreted specially by the shell, you have to quote them using single quotes, e.g. foo='qwe$123'
Using bash 4.x, the form
x=`...`
does not present any problems with respect to characters such $, #, or &, though it should be noted that the preferred form for such assignments is x=$(...)
However these forms should only be used with great care because of other special characters.
Generally, it would be better to use an idiom such as:
jq -r .... | while -r read line ; do .... ; done
Depending on your requirements, you might also wish to consider jq's #sh filter.
{
"assignedTo": [
"Daniel Raisor",
"Dalton Leslie",
"Logan Petro"
]
}
I want to extract Daniel Raisor Dalton Leslie and Logan Petro and assign these to a separate variable like assignedTo=Daniel Raisor,Dalton Leslie,Logan Petro
also my unix doesn't support jq ,I need to do this using grep or sed command
Use a proper JSON parser to parse JSON. Even if you don't have jq, there's a perfectly good one included with the Python interpreter, which the following example script uses:
#!/bin/ksh
# note: this was tested with ksh93u+ 2012-08-01
# to use with Python 3 rather than Python 2, replace "pipes" with "shlex"
json='{"assignedTo": ["Daniel \"The Man\" Raisor","Dalton Leslie","Logan Petro"]}'
getEntries() {
python -c '
import json, sys, pipes
content = json.load(sys.stdin)
for name, values in content.iteritems():
print("%s=%s" % (pipes.quote(name), pipes.quote(",".join(values))))
'
}
eval "$(getEntries <<<"$json")"
echo "$assignedTo"
...properly emits the following output:
Daniel "The Man" Raisor,Dalton Leslie,Logan Petro
I am using the following curl command:
curl -s -v --user admin:orca --insecure -X GET https://insecure.registry.com/api/v0/repositories/authi-api/tags
Getting following output:
{
"name": "Dev_ReleaseRollout_Lane-3",
"inRegistry": true,
"hashMismatch": false,
"inNotary": false
},
{
"name": "Dev_ReleaseRollout_Lane-latest",
"inRegistry": true,
"hashMismatch": false,
"inNotary": false
},
{
"name": "Payments_Dev_Lane-267",
"inRegistry": true,
"hashMismatch": false,
"inNotary": false
}
I want to get only name values in a variable.
I need only Dev_ReleaseRollout_Lane-3 Dev_ReleaseRollout_Lane-latest Payments_Dev_Lane-267 in a variable
Assuming you actually have an array around the three objects:
$ curl ... | jq -r '.[].name'
Dev_ReleaseRollout_Lane-3
Dev_ReleaseRollout_Lane-latest
Payments_Dev_Lane-267
It's fairly simple, . is the array, [].name take name from each element in the array. -r is raw output.
--raw-output / -r:
With this option, if the filter’s result is a string then it will be written directly to standard output rather than being formatted as a JSON string with quotes. This can be useful for making jq filters talk to non-JSON-based systems.
If the cURL output is actually as mentioned above the following will work:
jq -rRs '"[\(.)]" | fromjson[].name' file.json
However I think there is a better way to wrap an array around input,
-R is raw input and -s is slurp. \(...) is string interpolation.
--slurp/-s:
Instead of running the filter for each JSON object in the input, read the entire input stream into a large array and run the filter just once.