I am doing an mongodb aggregation using mongoid, using ModleName.collection.aggregate(pipeline) . The value returned is an array and not a Mongoid::Criteria, so if a do a first on the array, I get the first element which is of the type BSON::Document instead of ModelName. As a result, I am unable to use it as a model.
Is there a method to return a criteria instead of an array from the aggregation, or convert a bson document to a model instance?
Using mongoid (4.0.0)
I've been struggling with this on my own too. I'm afraid you have to build your "models" on your own. Let's take an example from my code:
class Searcher
# ...
def results(page: 1, per_page: 50)
pipeline = []
pipeline <<
"$match" => {
title: /#{#params['query']}/i
}
}
geoNear = {
"near" => coordinates,
"distanceField" => "distance",
"distanceMultiplier" => 3959,
"num" => 500,
"spherical" => true,
}
pipeline << {
"$geoNear" => geoNear
}
count = aggregate(pipeline).count
pipeline << { "$skip" => ((page.to_i - 1) * per_page) }
pipeline << { "$limit" => per_page }
places_hash = aggregate(pipeline)
places = places_hash.map { |attrs| Offer.new(attrs) { |o| o.new_record = false } }
# ...
places
end
def aggregate(pipeline)
Offer.collection.aggregate(pipeline)
end
end
I've omitted a lot of code from original project, just to present the way what I've been doing.
The most important thing here was the line:
places_hash.map { |attrs| Offer.new(attrs) { |o| o.new_record = false } }
Where both I'm creating an array of Offers, but additionally, manually I'm setting their new_record attribute to false, so they behave like any other documents get by simple Offer.where(...).
It's not beautiful, but it worked for me, and I could take the best of whole Aggregation Framework!
Hope that helps!
Related
How do I print the group_id from the returned object?
The following is returned from a function. I want to print the group_id or maybe return the group_id
{
:security_groups=>[
{
:description=>"Created By ManageIQ",
:group_name=>"MIQ_019",
:ip_permissions=>[
{
:from_port=>22,
:ip_protocol=>"tcp",
:ip_ranges=>[
{
:cidr_ip=>"0.0.0.0/0",
:description=>nil
}
],
:ipv_6_ranges=>[],
:prefix_list_ids=>[],
:to_port=>22,
:user_id_group_pairs=>[]
}
],
:owner_id=>"943755119718",
:group_id=>"sg-0c2c5f219f1bafc1a",
:ip_permissions_egress=>[
{
:from_port=>nil,
:ip_protocol=>"-1",
:ip_ranges=>[
{
:cidr_ip=>"0.0.0.0/0",
:description=>nil
}
],
:ipv_6_ranges=>[],
:prefix_list_ids=>[],
:to_port=>nil,
:user_id_group_pairs=>[]
}
],
:tags=>[],
:vpc_id=>"vpc-d817c1b3"
}
],
:next_token=>nil
}
This is the function: I want to return security_group.group_id
def describe_security_group (
group_name
)
ec2 = get_aws_client
security_group = ec2.describe_security_groups(
filters: [
{name: 'group-name', values: [ group_name ]}]
)
puts "Describing security group '#{group_name}' with ID " \
"'#{security_group}'"
return security_group
rescue StandardError => e
puts "Error describing security group: #{e.message}"
return
end
So, returning value seems like a hash, or you can make it hash exactly.
For case with one-element array you can simple use ruby dig method.
And according to your datum and comment below we can access needed element like this:
# from your ec2 api call
security_group = ec2.describe_security_groups(...)
# Result value is stored in `security_group` variable,
# and looks exactly like hash below
{
:security_groups=>[
{
:description=>"Created By ManageIQ",
:group_name=>"MIQ_019",
:ip_permissions=>[
{
:from_port=>22,
:ip_protocol=>"tcp",
:ip_ranges=>[
{
:cidr_ip=>"0.0.0.0/0",
:description=>nil
}
],
:ipv_6_ranges=>[],
:prefix_list_ids=>[],
:to_port=>22,
:user_id_group_pairs=>[]
}
],
:owner_id=>"943755119718",
:group_id=>"sg-0c2c5f219f1bafc1a",
:ip_permissions_egress=>[
{
:from_port=>nil,
:ip_protocol=>"-1",
:ip_ranges=>[
{
:cidr_ip=>"0.0.0.0/0",
:description=>nil
}
],
:ipv_6_ranges=>[],
:prefix_list_ids=>[],
:to_port=>nil,
:user_id_group_pairs=>[]
}
],
:tags=>[],
:vpc_id=>"vpc-d817c1b3"
}
],
:next_token=>nil
}
# And this is a target value, that you can store in another one,
# return from method or simply print to output
security_group.dig(:security_groups)
.try(:[], 0)
.dig(:group_id)
=> "sg-0c2c5f219f1bafc1a"
But if you need to search in array with multiple elements, methods from Ruby's Enumerable module could be helpful (like select or reject).
UPDATE with OpenStruct, if you prefer such method calls with dot notation:
json = security_group.to_json
os = JSON.parse(json, object_class: OpenStruct)
os.security_groups.first.group_id
=> "sg-0c2c5f219f1bafc1a"
When I create Collection::View instance with:
client = Mongo::Client.new('mongodb://127.0.0.1:27017/test')
view = client[:users].find( { name: "Sally" } )
=> #<Mongo::Collection::View:0x69824029475340 namespace='test.users' #filter={"name" => "Sally"} #options={}>
How I can change filter hash of this instance later? This does not work:
view.filter.merge!("age" => 30)
=> #FrozenError: can't modify frozen BSON::Document
I don't think you can. .filter is a method which takes arguments. It is not a hash.
See examples
and also search the code
However you might be able to do something like:
view = lambda { |hash| client[:users].find(hash) }
search_params = { name: "Sally" }
view.(search_params)
view.(search_params.merge!({foo: 'bar'}))
I'm receving events in Logstash containing measurement, values and tags. I do not know ahead of time what field there are and what tags. So i wanted to do something like this:
input {
http {}
}
filter {
ruby {
code => '
tags = event.get("stats_tags").split(",")
samples = event.get("stats_samples").split(" ")
datapoints = {}
samples.each {|s|
splat = s.split(" ")
datapoints[splat[0]] = splat[1]
}
event.set("[#metadata][stats-send-as-tags]", tags)
event.set("[#metadata][stats-datapoints]", datapoints)
'
}
}
output {
influxdb {
host => "influxdb"
db => "events_db"
measurement => measurement
send_as_tags => [#metadata][stats-send-as-tags]
data_points => [#metadata][stats-datapoints]
}
}
But this produce error. After much googling to no avail i'm starting to think this is imposible.
Is there a way to pass hash and array from event field to output/filter configuration?
EDIT: If i doublequote it, the error i'm getting is
output {
influxdb {
# This setting must be a hash
# This field must contain an even number of items, got 1
data_points => "[#metadata][stats-datapoints]"
...
}
}
MongoDb doesn't support keys with dot. I have a Ruby nested hash that has many keys with dot (.) character. Is there a configuration that can be used to specify a character replacement for . like an underscore _ while inserting such data to MongoDb
I'm using MongoDB with Ruby & mongo gem.
example hash is like below
{
"key.1" => {
"second.key" => {
"third.key" => "val"
}
}
}
If it isn't possible to use keys with . in Mongodb, you'll have to modify the input data :
hash = {
'key.1' => {
'second.key' => {
'third.key' => 'val.1',
'fourth.key' => ['val.1', 'val.2']
}
}
}
Transforming string keys
This recursive method transforms the keys of a nested Hash :
def nested_gsub(object, pattern = '.', replace = '_')
if object.is_a? Hash
object.map do |k, v|
[k.to_s.gsub(pattern, replace), nested_gsub(v, pattern, replace)]
end.to_h
else
object
end
end
nested_gsub(hash) returns :
{
"key_1" => {
"second_key" => {
"third_key" => "val.1",
"fourth_key" => [
"val.1",
"val.2"
]
}
}
}
Transforming keys and values
It's possible to add more cases to the previous method :
def nested_gsub(object, pattern = '.', replace = '_')
case object
when Hash
object.map do |k, v|
[k.to_s.gsub(pattern, replace), nested_gsub(v, pattern, replace)]
end.to_h
when Array
object.map { |v| nested_gsub(v, pattern, replace) }
when String
object.gsub(pattern, replace)
else
object
end
end
nested_gsub will now iterate on string values and arrays :
{
"key_1" => {
"second_key" => {
"third_key" => "val_1",
"fourth_key" => [
"val_1",
"val_2"
]
}
}
}
In mongoDB, there is no configuration to support dot in the key. You need to preprocess the JSON before inserting to MongoDB collection.
One approach is that you can replace the dot with its unicode equivalent U+FF0E before insertion.
Hope this helps.
I have this hash:
response = '{"librairies":[{"id":1,"books":[{"id":1,"qty":1},{"id":2,"qty":3}]},{"id":2,"books":[{"id":1,"qty":0},{"id":2,"qty":3}]}]}'
in which I'd like to delete every librairies where, at least, one of the book quantity is null.
For instance, with this given response, I'd expect this return:
'{"librairies":[{"id":1,"books":[{"id":1,"qty":1},{"id":2,"qty":3}]}]}'
I've tried this:
parsed = JSON.parse(response)
parsed["librairies"].each do |library|
library["books"].each do |book|
parsed.delete(library) if book["qty"] == 0
end
end
but this returns the exact same response hash, without having deleted the second library (the one with id => 2).
You can use Array#delete_if and Enumerable#any? for this
# Move through each array element with delete_if
parsed["librairies"].delete_if do |library|
# evaluates to true if any book hash in the library
# has a "qty" value of 0
library["books"].any? { |book| book["qty"] == 0 }
end
Hope this helps
To avoid changing the hash parsed, you could do the following.
Firstly, let's format parsed so we can see what we're dealing with:
parsed = { "libraries"=>[ { "id"=>1,
"books"=>[ { "id"=>1, "qty"=>1 },
{ "id"=>2, "qty"=>3 } ]
},
{ "id"=>2,
"books"=>[ { "id"=>1, "qty"=>0 },
{ "id"=>2, "qty"=>3 } ]
}
]
}
Later I want to show that parsed has not been changed when we create the new hash. An easy way of doing that is to compute a hash code on parsed before and after, and see if it changes. (While it's not 100% certain that different hashes won't have the same hash code, here it's not something to lose sleep over.)
parsed.hash
#=> 852445412783960729
We first need to make a "deep copy" of parsed so that changes to the copy will not affect parsed. One way of doing that is to use the Marshal module:
new_parsed = Marshal.load(Marshal.dump(parsed))
We can now modify the copy as required:
new_parsed["libraries"].reject! { |h| h["books"].any? { |g| g["qty"].zero? } }
#=> [ { "id"=>1,
# "books"=>[ { "id"=>1, "qty"=>1 },
# { "id"=>2, "qty"=>3 }
# ]
# }
# ]
new_parsed # => { "libraries"=>[ { "id"=>1,
"books"=>[ { "id"=>1, "qty"=>1},
{ "id"=>2, "qty"=>3}
]
}
]
}
And we confirm the original hash was not changed:
parsed.hash
#=> 852445412783960729