Iterate and search a JSON array for the element in the array - ruby

I have a JSON array that looks like this:
response = {
"items"=>[
{
"tags"=>[
"random"
],
"timestamp"=>12345,
"storage"=>{
"url"=>"https://example.com/example",
"key"=>"mykeys"
},
"envelope"=>{
},
"log-level"=>"info",
"id"=>"random_id_test_1",
"campaigns"=>[
],
"user-variables"=>{
},
"flags"=>{
"is-test-mode"=>false
},
"message"=>{
"headers"=>{
"to"=>"random#example.com",
"message-id"=>"foobar#example.com",
"from"=>"noreply#example.com",
"subject"=>"new subject"
},
"attachments"=>[
],
"recipients"=>[
"result#example.com"
],
"size"=>4444
},
"event"=>"stored"
},
{
"tags"=>[
"flowerPower"
],
"timestamp"=>567890,
"storage"=>{
"url"=>"https://yahoo.com",
"key"=>"some_really_cool_keys_go_here"
},
"envelope"=>{
},
"log-level"=>"info",
"id"=>"some_really_cool_ids_go_here",
"campaigns"=>[
],
"user-variables"=>{
},
"flags"=>{
"is-test-mode"=>false
},
"message"=>{
"headers"=>{
"to"=>"another_great#example.com",
"message-id"=>"email_id#example.com",
"from"=>"from#example.com",
"subject"=>"email_looks_good"
},
"attachments"=>[
],
"recipients"=>[
"example#example.com"
],
"size"=>2222
},
"event"=>"stored"
}]
}
I am trying to obtain the "storage" "url" based on the "to" email.
How do I iterate through this array where x is just the element in the array
response['items'][x]["message"]["headers"]["to"]
Once I find the specific email that I need, it will stop and return the value of x which is the element number.
I was going to use that value for x and call response['items'][x]['storage']['url']
which will return the string for the URL.
I thought about doing this but there's gotta be a better way:
x = 0
user_email = another_great#example.com
while user_email != response['items'][x]["message"]["headers"]["to"] do
x+=1
value = x
puts value
end

target =
response['items'].detect do |i|
i['message']['headers']['to'] == 'another_great#example.com'
end
then
target['storage']['url']

This is another option by creating Hash with key of to's email. And on basis of it fetch required information like this:
email_hash = Hash.new
response["items"].each do |i|
email_hash[i["message"]["headers"]["to"]] = i
end
Now if you want to fetch "storage" "url" then simply do:
user_email = "another_great#example.com"
puts email_hash[user_email]["storage"]["url"] if email_hash[user_email]
#=> "https://yahoo.com"

You can use it as #Satoru suggested. As a suggestion, if you use case involves complex queries on json data (more complex than this), then you can store your data in mongodb, and can elegantly query anything.

Related

DRY Strategy for looping over unknown levels of nested objects

My scenario is based on Gmail API.
I've learned that email messages can have their message parts deeply or shallowly nested based upon varying factors, but mostly the presence of attachments.
I'm using the Google API Ruby Client gem, so I'm not working with JSON, I'm getting objects with all the same information, but I think the JSON representation makes it easier to understand my issue.
A simple message JSON response looks like this (one parts array with 2 hashes inside it):
{
"id": "175b418b1ff69896",
"snippet": "COVID-19: Resources to help your business manage through uncertainty 20 Liters 500 PEOPLE FOUND YOU ON GOOGLE Here are the top search queries used to find you: 20 liters used by 146 people volunteer",
"payload": {
"parts": [
{
"mimeType": "text/plain",
"body": {
"data": "Hey, you found the body of the email! I want this!"
}
},
{
"mimeType": "text/html",
"body": {
"data": "<div>I actually don't want this</div>"
}
}
]
}
}
The value I want is not that hard to get:
response.payload.parts.each do |part|
#body_data = part.body.data if part.mime_type == 'text/plain'
end
BUT The JSON response of a more complex email message with attachments looks something like this (now parts nests itself 3 levels deep):
{
"id": "175aee26de8209d2",
"snippet": "snippet text...",
"payload": {
"parts": [
{
"mimeType": "multipart/related",
"parts": [
{
"mimeType": "multipart/alternative",
"parts": [
{
"mimeType": "text/plain",
"body": {
"data": "hey, you found me! This is what I want!!"
}
},
{
"mimeType": "text/html",
"body": {
"data": "<div>I actually don't want this one.</div>"
}
}
]
},
{
"mimeType": "image/jpeg"
},
{
"mimeType": "image/png"
},
{
"mimeType": "image/png"
},
{
"mimeType": "image/jpeg"
},
{
"mimeType": "image/png"
},
{
"mimeType": "image/png"
}
]
},
{
"mimeType": "application/pdf"
}
]
}
}
And looking at a few other messages, the object can vary from 1 to 5 levels (maybe more) of parts
I need to loop over an unknown number of parts and then loop over an unknown number of nested parts and the repeat this again until I reach the bottom, hopefully finding the thing I want.
Here's my best attempt:
def trim_response(response)
# remove headers I don't care about
response.payload.headers.keep_if { |header| #valuable_headers.include? header.name }
# remove parts I don't care about
response.payload.parts.each do |part|
# parts can be nested within parts, within parts, within...
if part.mime_type == #valuable_mime_part && part.body.present?
#body_data = part.body.data
break
elsif part.parts.present?
# there are more layers down
find_body(part)
end
end
end
def find_body(part)
part.parts.each do |sub_part|
if sub_part.mime_type == #valuable_mime_part && sub_part.body.present?
#body_data = sub_part.body.data
break
elsif sub_part.parts.present?
# there are more layers down
######### THIS FEELS BAD!!! ###########
find_body(sub_part)
end
end
end
Yep, there's a method calling itself. I know, that's why I'm here.
This does work, I've tested it on a few dozen messages, but... there has to be a better, DRY-er way to do this.
How do I recursively loop and then move down a level and loop again in a DRY fashion when I don't know how deep the nesting goes?
No need to go through all this pain. Just keep diving in the parts dictionary until you find the first value where there is no parts anymore. At this moment you have the final parts in your parts variable.
Code:
reponse = {"id" => "175aee26de8209d2","snippet" => "snippet text...","payload" => {"parts" => [{"mimeType" => "multipart/related","parts" => [{"mimeType" => "multipart/alternative","parts" => [{"mimeType" => "text/plain","body" => {"data" => "hey, you found me! This is what I want!!"}},{"mimeType" => "text/html","body" => {"data" => "<div>I actually don't want this one.</div>"}}]},{"mimeType" => "image/jpeg"}]},{"mimeType" => "application/pdf"}]}}
parts = reponse["payload"]
parts = (parts["parts"].send("first") || parts["parts"]) while parts["parts"]
data = parts["body"]["data"]
puts data
Output:
hey, you found me! This is what I want!!
You can compute the desired result using recursion.
def find_it(h, top_key, k1, k2, k3)
return nil unless h.key?(top_key)
recurse(h[top_key], k1, k2, k3)
end
def recurse(h, k1, k2, k3)
return nil unless h.key?(k1)
h[k1].each do |g|
v = g.dig(k2,k3) || recurse(g, k1 , k2, k3)
return v unless v.nil?
end
nil
end
See Hash#dig.
Let h1 and h2 equal the two hashes given in the example1. Then:
find_it(h1, :payload, :parts, :body, :data)
#=> "Hey, you found the body of the email! I want this!"
find_it(h2, :payload, :parts, :body, :data)
#=> "hey, you found me! This is what I want!!"
1. The hash h[:payload][:parts].last #=> { "mimeType": "application/pdf" } appears to contain hidden characters that are causing a problem. I therefore removed that hash from h2.

I have json data i need search `unique` if key exist or not

I have JSON data I need search unique if the key exists or not.
[
{
"key1" => []
},
{
"key" => []
},
{
"unique" => []
}
]
I can use loop but need an efficient way to check unique exist or not
You'll need to iterate through the array either way.
# You'll get found item or `nil`
data.find { |item| item.key?('unique') }
# You'll get `true` or `false`
data.any? { |item| item.key?('unique') }
Btw better to use a hash as an input instead of an array:
data = {
"key1" => [],
"key" => [],
"unique" => []
}
data.key?('unique')
=> true

How to query key values from a hash of arrays of hashes

I have a JSONB payload in my database. This payload is from a GraphQL query of the shopify_api.
For the shop_order below, I am trying to query for the name of the fourth order in the node.
shop_order = {"data":{"orders":{"edges":[{"node":{"id":"gid://shopify/Order/2228134674512","name":"#1001","createdAt":"2020-05-01T18:46:04Z","shippingAddress":{"address1":"1234 Long Avenue, 2N","address2":"","city":"Chicago","province":"Illinois","provinceCode":"IL","zip":"55555"}}},{"node":{"id":"gid://shopify/Order/2239643451472","name":"#1002","createdAt":"2020-05-05T14:40:36Z","shippingAddress":{"address1":"1234 Long Avenue","address2":"2N","city":"Chicago","province":"Illinois","provinceCode":"IL","zip":"55555"}}},{"node":{"id":"gid://shopify/Order/2239950323792","name":"#1003","createdAt":"2020-05-05T16:35:38Z","shippingAddress":{"address1":"1234 Long Avenue","address2":"2N","city":"Chicago","province":"Illinois","provinceCode":"IL","zip":"55555"}}},{"node":{"id":"gid://shopify/Order/2239959105616","name":"#1004","createdAt":"2020-05-05T16:38:27Z","shippingAddress":{"address1":"1234 Long Avenue","address2":"2N","city":"Chicago","province":"Illinois","provinceCode":"IL","zip":"55555"}}}]}},"casted_data":{},"errors":[]}
order = shop_order[:data][:orders][:edges][3]
puts order
response > {:node=>{:id=>"gid://shopify/Order/2239959105616", :name=>"#1004", :createdAt=>"2020-05-05T16:38:27Z", :shippingAddress=>{:address1=>"1234 Long Avenue", :address2=>"2N", :city=>"Chicago", :province=>"Illinois", :provinceCode=>"IL", :zip=>"55555"}}}
order_to_a = shop_order[:data][:orders][:edges][3].to_a
puts order_to_a
response > node
{:id=>"gid://shopify/Order/2239959105616", :name=>"#1004", :createdAt=>"2020-05-05T16:38:27Z", :shippingAddress=>{:address1=>"1234 Long Avenue", :address2=>"2N", :city=>"Chicago", :province=>"Illinois", :provinceCode=>"IL", :zip=>"55555"}}
How do I query and display a specific value from a key that is inside a node?
It's not entirely clear what your intent is, but your access of elements in a hash can be streamlined using dig:
shop_order = {
"data": {
"orders": {
"edges": [
{}, {}, {}, {
"node": {
"name": '#1004',
"shippingAddress": {
"zip": '55555'
}
}
}
]
}
}
}
Access data using:
order = shop_order.dig(:data, :orders, :edges)[3]
# => {:node=>{:name=>"#1004", :shippingAddress=>{:zip=>"55555"}}}
or:
order = shop_order.dig(:data, :orders, :edges, 3)
# => {:node=>{:name=>"#1004", :shippingAddress=>{:zip=>"55555"}}}
How do I query and display a specific value from a key that is inside a node?
Huh? If you want information inside order, do the same sort of thing:
order.dig(:node, :name) # => "#1004"
order.dig(:node, :shippingAddress, :zip) # => "55555"
or:
shop_order.dig(:data, :orders, :edges, 3, :node, :name) # => "#1004"
shop_order.dig(:data, :orders, :edges, 3, :node, :shippingAddress, :zip) # => "55555"
Many times when we're walking through a complex hash of arrays we point to the array in a variable and then work from that point. It's similar to putting your finger on a page in a recipe, so we can go back to it quickly. We do the same when parsing HTML/XML, parsed JSON and YAML, etc.

Logstash filter out values with null values for a key in a nested json array

I have quite an extensive Logstash pipeline ending in a Json as such:
{
"keyA": 1,
"keyB": "sample",
"arrayKey": [
{
"key": "data"
},
{
"key": null
}
]
}
What I want to achieve is to filter "arrayKey" and remove objects within with value for "key" is null.
Tried this to no luck:
filter {
ruby {
code => "
event.get('arrayKey').each do |key|
[key].delete_if do |keyCandidate|
if [keyCandidate][key] != nil
true
end
end
end
"
}
}
This gives no implicit converter found from |hash|:|Int| error. How do I achieve this? Is there and easier way to do this?
As Aleksei pointed out, you can create a copy of the array that does not contain entries where [key] is null using reject. You have to use event.set to overwrite the inital value of [arrayKey]
ruby {
code => '
a = event.get("arrayKey")
if a
event.set("arrayKey", a.reject { |x| x["key"] == nil })
end
'
}

Read a value from a JSON key when the key is unknown/random

My JSON response looks like this:
{
"item": {
"-LVShDSBr5tvs0wGkc0JJ": {
"text": "H"
}
},
"item": {
"-LEVSZndgiqwhgnytO3Kr": {
"text": "Hatem"
}
}
}
I can read each item object, but I need to reach the text value. To do that, I need to get through the random ID that's one level above that. How do I read what's within that key?
I have this:
items.each do |item|
# gets one item successfully
# but im unable to read the key within since it's unknown
text = item[:unknown_key][:text]
end
Use Hash#values:
texts =
items.map do |item|
item.values.first[:text]
end
If you expect more than one item, play around with mapping values to their [:text]s.

Resources