I am trying to write a rule but am running into an issue. I managed to extract the following from as my input:
myData:= [{"Key": "use", "Value": "1"}, {"Key": "use", "Value": "2"}, {"Key": "att1", "Value": "3"}]
I am trying to count the amount of times a key with the value use appears. However when I do:
p := {keep| keep:= myData[_]; myData.Key == "use"}
I assumed this would create a listing of all I would like to keep but the playground errors with:
1 error occurred: policy.rego:24: rego_type_error: undefined ref: data.play.myData.Key
data.play.myData.Key
I hoped I could list them in p and then do count(p) > 1 to check if more that one is listed.
In your set comprehension for p, you're iterating over the objects in myData, assigning each element to keep. Then, you assert something on myData.Key. I think what you're looking for is
p := {keep| keep := myData[_]; keep.Key == "use"}
Be aware that it's a set comprehension, so p would be the same for these two inputs:
myData:= [{"Key": "use", "Value": "1"}]
myData:= [{"Key": "use", "Value": "1"}, {"Key": "use", "Value": "1"}]
You could use an array comprehension (p := [ keep | keep := ... ]) if that's not what you want.
Related
I like to filter json files using jq:
jq . some.json
Given the json containing an array of objects:
{
"theList": [
{
"id": 1,
"name": "Horst"
},
{
"id": 2,
"name": "Fritz"
},
{
"id": 3,
"name": "Walter"
},
{
"id": 4,
"name": "Gerhart"
},
{
"id": 5,
"name": "Harmut"
}
]
}
I want to filter that list to only show the elements with id having the value 2 and 4, so the expected output is:
{
"id": 2,
"name": "Fritz"
},
{
"id": 4,
"name": "Gerhart"
}
How do I filter the json using jq? I have played around with select and map, yet didn't got any of those to work, e.g.:
$ jq '.theList[] | select(.id == 2) or select(.id == 4)' array.json
true
From the docs:
jq '.[] | select(.id == "second")'
Input [{"id": "first", "val": 1}, {"id": "second", "val": 2}]
Output {"id": "second", "val": 2}
I think you can do something like this:
jq '.theList[] | select(.id == 2 or .id == 4)' array.json
You could use select within map.
.theList | map(select(.id == (2, 4)))
Or more compact:
[ .theList[] | select(.id == (2, 4)) ]
Though written that way is a little inefficient since the expression is duplicated for every value being compared. It'll be more efficient and possibly more readable written this way:
[ .theList[] | select(any(2, 4; . == .id)) ]
Using select(.id == (2, 4)) here is generally inefficient (see below).
If your jq has IN/1, then it can be used to achieve a more efficient solution:
.theList[] | select( .id | IN(2,3))
If your jq does not have IN/1, then you can define it as follows:
def IN(s): first(select(s == .)) // false;
Efficiency
One way to see the inefficiency is to use debug. The following expression, for example, results in 10 calls to debug, whereas only 9 checks for equality are actually needed:
.theList[] | select( (.id == (2,3)) | debug )
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",true]
{
"id": 2,
"name": "Fritz"
}
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",true]
{
"id": 3,
"name": "Walter"
}
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",false]
index/1
In principle, using index/1 should be efficient, but as of this writing (October 2017), its implementation, though fast (it is written in C), is inefficient.
Here is a solution using indices:
.theList | [ .[map(.id)|indices(2,4)[]] ]
I have a list of dictionaries with each entry having the following structure
{
"id": 0,
"type": "notification",
"name": "jane doe",
"loc": {
"lat": 38.8239,
"long": 104.7001
},
"data": [
{
"type": "test",
"time": "Fri Aug 13 09:17:16 2021",
"df": 80000000,
"db": 1000000,
"tp": 92
},
{
"type": "real",
"time": "Sat Aug 14 09:21:30 2021",
"df": 70000000,
"db": 2000000,
"tp:": 97
}
]
}
I need to be able to sort this list by any of these keys: name, type, time, tp and return it in memory.
I understand how to sort by the top level keys sorted(json_list, key=lambda k:k['name']) or even nested keys. For instance by lat sorted(json_list, key=lambda k:k['loc']['lat'])
so currently I have a function that works for the case when sorting by name.
def sort_by(self, param, rev=False):
if param == NAME:
self.json_list = sorted(self.json_list, key=lambda k: k[param], reverse=rev)
else:
# need help here
I'm having trouble sorting by type, time, and tp. Notice the data key is also a list of dictionaries. I would like to leverage existing methods built into the standard lib if possible. I can provide more clarification if necessary
Update:
def sort_by(self, param, rev=False):
if param == NAME:
self.json_list = sorted(self.json_list, key=lambda k: k[param], reverse=rev)
else:
self.json_list = sorted(self.json_list, key=lambda k: k['data'][0][param], reverse=rev)
return self.json_list
This works fine if there is only one item in the data list
If json_list[i]['data'] (for each i) only contains one dict, then the following should work; otherwise modifications are required.
sorted(json_list, key = lambda k: (
k['name'], k['data']['type'], k['data']['time'], k['data']['tp']
))
I like to filter json files using jq:
jq . some.json
Given the json containing an array of objects:
{
"theList": [
{
"id": 1,
"name": "Horst"
},
{
"id": 2,
"name": "Fritz"
},
{
"id": 3,
"name": "Walter"
},
{
"id": 4,
"name": "Gerhart"
},
{
"id": 5,
"name": "Harmut"
}
]
}
I want to filter that list to only show the elements with id having the value 2 and 4, so the expected output is:
{
"id": 2,
"name": "Fritz"
},
{
"id": 4,
"name": "Gerhart"
}
How do I filter the json using jq? I have played around with select and map, yet didn't got any of those to work, e.g.:
$ jq '.theList[] | select(.id == 2) or select(.id == 4)' array.json
true
From the docs:
jq '.[] | select(.id == "second")'
Input [{"id": "first", "val": 1}, {"id": "second", "val": 2}]
Output {"id": "second", "val": 2}
I think you can do something like this:
jq '.theList[] | select(.id == 2 or .id == 4)' array.json
You could use select within map.
.theList | map(select(.id == (2, 4)))
Or more compact:
[ .theList[] | select(.id == (2, 4)) ]
Though written that way is a little inefficient since the expression is duplicated for every value being compared. It'll be more efficient and possibly more readable written this way:
[ .theList[] | select(any(2, 4; . == .id)) ]
Using select(.id == (2, 4)) here is generally inefficient (see below).
If your jq has IN/1, then it can be used to achieve a more efficient solution:
.theList[] | select( .id | IN(2,3))
If your jq does not have IN/1, then you can define it as follows:
def IN(s): first(select(s == .)) // false;
Efficiency
One way to see the inefficiency is to use debug. The following expression, for example, results in 10 calls to debug, whereas only 9 checks for equality are actually needed:
.theList[] | select( (.id == (2,3)) | debug )
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",true]
{
"id": 2,
"name": "Fritz"
}
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",true]
{
"id": 3,
"name": "Walter"
}
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",false]
index/1
In principle, using index/1 should be efficient, but as of this writing (October 2017), its implementation, though fast (it is written in C), is inefficient.
Here is a solution using indices:
.theList | [ .[map(.id)|indices(2,4)[]] ]
I'm new to ruby so please excuse any ignorance I may bear. I was wondering how to parse a JSON reponse for every value belonging to a specific key. The response is in the format,
[
{
"id": 10008,
"name": "vpop-fms-inventory-ws-client",
"msr": [
{
"key": "blocker_violations",
"val": 0,
"frmt_val": "0"
},
]
},
{
"id": 10422,
"name": "websample Maven Webapp",
"msr": [
{
"key": "blocker_violations",
"val": 0,
"frmt_val": "0"
}...
There's some other entries in the response, but for the sake of not having a huge block of code, I've shortened it.The code I've written is:
require 'uri'
require 'net/http'
require 'JSON'
url = URI({my url})
http = Net::HTTP.new(url.host, url.port)
request = Net::HTTP::Get.new(url)
request["cache-control"] = 'no-cache'
request["postman-token"] = '69430784-307c-ea1f-a488-a96cdc39e504'
response = http.request(request)
parsed = response.read_body
h = JSON.parse(parsed)
num = h["msr"].find {|h1| h1['key']=='blocker_violations'}['val']
I am essentially looking for the val for each blocker violation (the json reponse contains hundreds of entries, so im expecting hundreds of blocker values). I had hoped num would contain an array of all the 'val's. If you have any insight in this, it would be of great help!
EDIT! I'm getting a console output of
scheduler caught exception:
no implicit conversion of String into Integer
C:/dashing/test_board/jobs/issue_types.rb:20:in `[]'
C:/dashing/test_board/jobs/issue_types.rb:20:in `block (2 levels) in <top (requi
red)>'
C:/dashing/test_board/jobs/issue_types.rb:20:in `select'
I suspect that might have too much to do with the question, but some help is appreciated!
You need to do 2 things. Firstly, you're being returned an array and you're only interested in a subset of the elements. This is a common pattern that is solved by a filter, or select in Ruby. Secondly, the condition by which you wish to select these elements also depends on the values of another array, which you need to filter using a different technique. You could attempt it like this:
res = [
{
"id": 10008,
"name": "vpop-fms-inventory-ws-client",
"msr": [
{
"key": "blocker_violations",
"val": 123,
"frmt_val": "0"
}
]
},
{
"id": 10008,
"name": "vpop-fms-inventory-ws-client",
"msr": [
{
"key": "safe",
"val": 0,
"frmt_val": "0"
}
]
}
]
# define a lambda function that we will use later on to filter out the blocker violations
violation = -> (h) { h[:key] == 'blocker_violations' }
# Select only those objects who contain any msr with a key of blocker_violations
violations = res.select {|h1| h1[:msr].any? &violation }
# Which msr value should we take? Here I just take the first.
values = violations.map {|v| v[:msr].first[:val] }
The problem you may have with this code is that msr is an array. So theoretically, you could end up with 2 objects in msr, one that is a blocker violation and one that is not. You have to decide how you handle that. In my example, I include it if it has a single blocker violation through the use of any?. However, you may wish to only include them if all msr objects are blocker violations. You can do this via the all? method.
The second problem you then face is, which value to return? If there are multiple blocker violations in the msr object, which value do you choose? I just took the first one - but this might not work for you.
Depending on your requirements, my example might work or you might need to adapt it.
Also, if you've never come across the lambda syntax before, you can read more about it here
Here is my JSON file:
[{
"name": "chetan",
"age": 23,
"hobby": ["cricket", "football"]
}, {
"name": "raj",
"age": 24,
"hobby": ["cricket", "golf"]
}]
Here is the golang code I tried but didn't work as expected.
id:= "ket"
c.EnsureIndexKey("hobby")
err = c.Find(bson.M{"$hobby": bson.M{"$search": id,},}).All(&result)
It gives error:
$hobby exit status 1
From $search I'm assuming you're trying to use a text index/search, but in your case that wouldn't work. Text index doesn't support partials. You can still use regex to find those documents, but performance wise it wouldn't be a wise choice probably, unless you can utilize the index - which in your case wouldn't happen.
Still, you could achieve what you want with:
id := "ket"
regex := bson.M{"$regex": bson.RegEx{Pattern: id}}
err = c.Find(bson.M{"hobby": regex}).All(&result)