I have a collection of Person, stored in a legacy mongodb server (2.4) and accessed with the mongoid gem via the ruby mongodb driver.
If I perform a
Person.where(email: 'some.existing.email#server.tld').first
I get a result (let's assume I store the id in a variable called "the_very_same_id_obtained_above")
If I perform a
Person.find(the_very_same_id_obtained_above)
I got a
Mongoid::Errors::DocumentNotFound
exception
If I use the javascript syntax to perform the query, the result is found
Person.where("this._id == #{the_very_same_id_obtained_above}").first # this works!
I'm currently trying to migrate the data to a newever version. Currently mongodbrestore-ing on amazon documentdb to make tests (mongodb 3.6 compatible) and the issue remains.
One thing I noticed is that those object ids are peculiar:
5ce24b1169902e72c9739ff6 this works anyway
59de48f53137ec054b000004 this requires the trick
The small number of zeroes toward the end of the id seems to be highly correlated with the problem (I have no idea of the reason).
That's the default:
# Raise an error when performing a #find and the document is not found.
# (default: true)
raise_not_found_error: true
Source: https://docs.mongodb.com/mongoid/current/tutorials/mongoid-configuration/#anatomy-of-a-mongoid-config
If this doesn't answer your question, it's very likely the find method is overridden somewhere in your code!
Is find_each an acceptable replacement for the ActiveRecord function all in version 4.0.0 of ActiveRecord?
For example, previously I had:
all_users = User.all
which produces a warning stating something along the lines of ActiveRelation:all is deprecated.
As a replacement I came up with:
User.find_each do |user|
all_users += user
end
Is this acceptable, or should I be doing it another way?
I understand the reason behind using find_each is because of 'batching' which will allow the query to stop running if there is a very large dataset. Let's assume for this case that the dataset is small.
EDIT
It seems the deprecation error only comes up when you use conditons, eg:
User.all(:conditions => ["name like ?", "%bob%"])
produces:
DEPRECATION WARNING: Relation#all is deprecated. If you want to
eager-load a rel ation, you can call #load (e.g.
Post.where(published: true).load). If you want to get an array of
records from a relation, you can call #to_a (e.g. Post.wher
e(published: true).to_a). (called from irb_binding at (irb):8)
The correct replacement for the above appears to be:
User.where("name like ?", "%bob%")
Model.all isn't deprecated in Rails 4, but it has changed. Instead of returning an array of all of the records, it returns an ActiveRecord Relation which is significantly faster.
Model.all is designed to deprecate Model.scoped which previously lazy loaded the records. Model.all can now be used for easier chaining of methods instead of scoped.
You can read all about it here in this article.
I am using DataMapper to interface with MySql. Is there any check I can do with Datamapper to ensure that the database is up?
If you want to test whether you can actually do some work with your database, something like this could be helpful:
begin
DataMapper.repository(:default).adapter.execute('SHOW TABLES;')
rescue
puts "Problem!"
end
This will make sure that the server is up and that the database you chose is valid (that's why something like SELECT 1 wouldn't work).
We are using Datamapper in a Sinatra application and would like to use case insensitive like that works on both Sqlite (locally in development) and Postgresql (on Heroku in production).
We have statements like
TreeItem.all(:name.like =>"%#{term}%",:unique => true,:limit => 20)
If termis "BERL" we get the suggestion "BERLIN" from both the Sqlite and Postgresql backends. However if termis "Berl" we only get that result from Sqlite and not Postgresql.
I guess this has to do with the fact that both dm-postgres-adapter and dm-sqlite-adapter outputting a LIKE in the resulting SQL query. Since Postgresql has a case sensitive LIKE we get this (for us unwanted) behavior.
Is there a way to get case insensitive like in Datamapper without resorting to use a raw SQL query to the adapter or patching the adapter to use ILIKEinstead of LIKE?
I could of course use something in between, such as:
TreeItem.all(:conditions => ["name LIKE ?","%#{term}%"],:unique => true,:limit => 20)
but then we would be tied to the use of Postgresql within our own code and not just as a configuration for the adapter.
By writing my own data object adapter that overrides the like_operator method I managed to get Postgres' case insensitive ILIKE.
require 'do_postgres'
require 'dm-do-adapter'
module DataMapper
module Adapters
class PostgresAdapter < DataObjectsAdapter
module SQL #:nodoc:
private
# #api private
def supports_returning?
true
end
def like_operator(operand)
'ILIKE'
end
end
include SQL
end
const_added(:PostgresAdapter)
end
end
Eventually I however decided to port the application in question to use a document database.
For other people who happen to use datamapper wanting support for ilike as well as 'similar to' in PostgreSQL: https://gist.github.com/Speljohan/5124955
Just drop that in your project, and then to use it, see these examples:
Model.all(:column.ilike => '%foo%')
Model.all(:column.similar => '(%foo%)|(%bar%)')
Rails' ActiveRecord has a feature called Query Caching (ActiveRecord::QueryCache) which saves the result of SQL query for the life-span of a request. While I'm not very familiar with the internals of the implementation, I think that it saves the query results somewhere in the Rack env, which is discarded in the end of the request.
The Mongoid, unfortunately, doesn't currently provide such feature, and this is exacerbated by the fact, that some queries occur implicitly (references).
I'm considering to implement this feature, and I'm curious, where and how Mongoid (or, perhaps, mongo driver?) should be hooked in order to implement this.
Mongoid has caching, described under http://mongoid.org/en/mongoid/docs/extras.html
Also MongoDB itself has caching ability: http://www.mongodb.org/display/DOCS/Caching
The mongoid caching extra knows 2 different cases: Caching of all queries of a model or caching of a query.
Mongoid caching seems to work slightly different: it looks like mongoid delegates caching to mongodb. (In the sources of mongoid I only can find option settings for caching but no cache module.)
Finally would say, there is no real difference in the caching in general -- in memory is in fact in memory! No matter if it's in the app or in the database.
I don't prefer to implement an extra caching algorithm, because this seems to be redundant and a RAM killer.
BTW: If your really want to cache results in-app you could try Rails.cache or another cache gem as a workaround.
The other answer is obviously wrong. Not only mongoid or mongo driver doesn't cache the query, even if mongo would - it still might be on other machine across the network.
My solution was to wrap the receive_message in Mongo::Connection.
Pros: one definite place
Cons: deserialization still takes place
require 'mongo'
module Mongo
class Connection
module QueryCache
extend ActiveSupport::Concern
module InstanceMethods
# Enable the selector cache within the block.
def cache
#query_cache ||= {}
old, #query_cache_enabled = #query_cache_enabled, true
yield
ensure
clear_query_cache
#query_cache_enabled = old
end
# Disable the selector cache within the block.
def uncached
old, #query_cache_enabled = #query_cache_enabled, false
yield
ensure
#query_cache_enabled = old
end
def clear_query_cache
#query_cache.clear
end
def cache_receive_message(operation, message)
#query_cache[operation] ||= {}
key = message.to_s.hash
log = "[MONGO] CACHE %s"
if entry = #query_cache[operation][key]
Mongoid.logger.debug log % 'HIT'
entry
else
Mongoid.logger.debug log % 'MISS'
#query_cache[operation][key] = yield
end
end
def receive_message_with_cache(operation, message, log_message=nil, socket=nil, command=false)
if query_cache_enabled
cache_receive_message(operation, message) do
receive_message_without_cache(operation, message, log_message, socket, command)
end
else
receive_message_without_cache(operation, message, log_message, socket, command)
end
end
end # module InstanceMethods
included do
alias_method_chain :receive_message, :cache
attr_reader :query_cache, :query_cache_enabled
end
end # module QueryCache
end # class Connection
end
Mongo::Connection.send(:include, Mongo::Connection::QueryCache)
OK, Mongoid 4 supports QueryCache middleware.
Just add middleware in application.rb
config.middleware.use "Mongoid::QueryCache::Middleware"
And then profit:
MOPED: 127.0.0.1:27017 QUERY database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dabb6d61631e21d70000')}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.4397ms
MOPED: 127.0.0.1:27017 QUERY database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dacf6d61631e21dc0000')}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.4590ms
QUERY CACHE database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564c9596d61631e21d30000')}, "$orderby"=>{:_id=>1}}
QUERY CACHE database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dabb6d61631e21d70000')}, "$orderby"=>{:_id=>1}}
Source:
Mongoid changelog
https://github.com/mongoid/mongoid/blob/master/CHANGELOG.md#new-features-2
3410 Mongoid now has a query cache that can be used as a middleware in Rack applications. (Arthur Neves)
For Rails:
config.middleware.use(Mongoid::QueryCache::Middleware)
Mongoid 4.0+ now has a QueryCaching module: http://www.rubydoc.info/github/mongoid/mongoid/Mongoid/QueryCache
You can use it on finds by wrapping your lookups like so:
QueryCache.cache { MyCollection.find("xyz") }