I'm trying to insert a document that has multiple embedded documents but I have been unable to determine the structure for such a document.
I'm using Mongoid in most places but need to perform a batch document insert.
I've tried the following:
def build_records_array(records)
records.collect do |record|
record.raw_attributes["identifier"] = record.identifiers.collect { |identifier| identifier.raw_attributes }
record.raw_attributes
end
end # self.build_records_array
However the identifiers don't show up as embedded documents when I call insert. I just get a bunch of garbage in my parent document.
What is the proper structure for embedded documents?
So, I just had a typo. I wasn't thinking about Mongoid when looking at my problem. After playing around with the Mongo Driver to retrieve records Mongoid had created I discovered that I had everything right but the attribute name.
def build_records_array(records)
records.collect do |record|
record.raw_attributes["identifiers"] = record.identifiers.collect { |identifier| identifier.raw_attributes }
record.raw_attributes
end
end # build_records_array
Related
I'm using Anemone to store crawled pages into MongoDB. It mostly works, except for accessing the page headers when I retrieve a page from MongoDB.
When I call collection.find_one("http://stackoverflow.com") I'll get the correct object from the data store, but I can't acecss the headers.
Anemone stores the headers as a hash, so theoretically, after retreiving the document, I should be able to do something like
document["headers"]["content-type"]
but that doesn't work because document["headers"] is a BSON::Binary.
puts document["headers"]
displays a mixture of text and binary characters.
How can I create a usable ruby hash object from the binary data that comes back from MongoDB?
EDIT: I haven't solved the original problem, but was able to modify Anemone so that I can have it load the data for me, which seems to work:
class NewMongo < Anemone::Storage::MongoDB
def initialize(mongo_db, collection_name)
#db = mongo_db
#collection = #db[collection_name]
#Do not delete the collection! I need it!
##collection.remove
#collection.create_index 'url'
end
end
And then later on...
repo = NewMongo.new(db, "pages")
repo.each db |url, page|
puts page.content_type
end
If the data was stored in a Binary format by the Anemone storage backend there isn't much you can do unless you know the format or there is a deserializer they provide. It sounds like that would be a bad choice for storing the header as the hash would be a more natural form for it.
I want to take Google Custom Search API results and add them to a database using Datamapper.
I've managed to successfully set up and manually add some items to the database, so it seems like that is all set up correctly.
I'm using HTTParty to make the call the the Google API, which is returning JSON results. I then want to take that JSON and add each link into the database. I'mtrying to use .each as follows;
response["items"].each do |item|
i=DMapperModel.create(city: "London", link: item["link"])
puts i
puts i.saved?
end
Response is a variable holding the HTTParty::response, "items" and "link" are both subsets of the HTTParty::response.
puts i successfully puts the correct DataMapper resource (i.e. <#DMapperModel city: 'London', link: 'example.com'>)
puts i.saved? is a check to see if i saved to the database, at the moment this is returning false...
So it is successfully setting i to the DataMapper resource, but not saving it to the database for some reason, can anyone see where I'm going wrong?
Solved it myself!
I had the links parameter set as a string, which DataMapper sets at a default maximum length of 50 characters. All of the links I was trying to add were longer that 50 characters so the save was failing.
Resolved the problem by setting the maximum length of the link parameter to 2000 characters;
class CityLink
include DataMapper::Resource
property :id, Serial
property :city, String
property :link, String, *length: 2000*
end
I took the length of 2000 characters from this page on the DataMapper docs which is for the URI parameter type and links to this SO question.
Once I'd set this, the above .each method worked a trick.
I'm trying to embed a document inside an existing document using the Ruby Driver.
Here's what my primary document looks like:
db = Mongo::Connection.new.db("Portfolios")
project_collection = db.collection("Projects")
new_Project = { :url => 'http://www.tekfolio.me/billy/portfolio/focus', :author => 'Billy'}
project_collection.insert(new_Project)
After I've created my new_project and added it to my project_collection I may or may not add another collection to the same document later called assets. This is where I'm stuck. The following code doesn't seem to do anything:
new_asset = { :image_url => 'http://assets.tekfolio.me/portfolios/68fbb25a-8353-41a8-a779-4bd9762b00f2/projects/13/assets/20/focus2.PNG'}
new_Project.assest.insert(new_asset)
I'm certain I've butchered my understanding of Mongodb and the Ruby driver and the embeded document concept and would appreciate your help getting me out of this wet paper bag I can't seem to get out of ;)
Have you tried just setting the value of asset without insert and instead using update?
new_Project["asset"] = new_asset
project_collection.update({"_id" => new_Project["_id"]}, new_Project)
I think , are you trying to "update" the new_project record with the asset
it doesn't work because then you are only updating the hash in ruby, not in mongo, you have to first get the reference to the object in mongo, update it, and then save it, check this info:
http://www.mongodb.org/display/DOCS/Updating+Data+in+Mongo
(if you can, you can assign the asset before inserting, and it should work)
I've to add several fields in a mongoid model, I know there is not migration with MongoDB but if I go on without dropping the DB, making rails to "regenerate" the DB entirely, it doesn't display or use the new fields at all !
What's the best way to go here ? Is there something softer than drop/reopen mongodb ?
Thanks in advance
luca
In general it should be possible to update old documents with the new fields at runtime. There is no need for migrations in MongoDB.
You maybe want to write rake tasks to update your old documents with the new fields and default values.
You could find out these documents by checking those new fields which have per default a nil value.
Update
Easy style:
If you define a new field with a default value, this value should always be used as long as you set a new one:
app/models/my_model.rb
class MyModel
include Mongoid::Document
field :name, type: String
field :data, type: String
# NEW FIELD
field :note, type: String, default: "no note given so far!"
end
If you query your database you should get your default value for documents which haven't this field before your extension:
(rails console)
MyModel.first
#=> #<MyModel …other fields…, note: "no note given so far!">
I tested this with a fresh rails stack with a current mongoid on Ruby 1.9.2 - should work with other stacks, too.
More complicated/complex style:
If you didn't set a default value, you'll get nil for this new field.
app/models/my_model.rb
class MyModel
include Mongoid::Document
field :name, type: String
field :data, type: String
# NEW FIELD
field :note, type: String
end
(rails console)
MyModel.first
#=> #<MyModel …other fields…, note: nil>
Then you could set up a rake task and migration file like in this example:
lib/tasks/my_model_migration.rake:
namespace :mymodel do
desc "MyModel migration task"
task :migrate => :environment do
require "./db/migrate.rb"
end
end
db/migrate.rb:
olds = MyModel.where(note: nil)
# Enumerator of documents without a valid :note field (= nil)
olds.each do |doc|
doc.note = "(migration) no note given yet"
# or whatever your desired default value should be
doc.save! rescue puts "Could not modify doc #{doc.id}/#{doc.name}"
# the rescue is only a failsafe statement if something goes wrong
end
Run this migration with rake mymodel:migrate.
This is only a starting point and you can extend this to a full mongoid migration engine.
The task :migrate => :environment do … is necessary, otherwise rake won't load models.
It is a little ridiculous to say that you don't need migrations with mongodb or mongoid. Any sophisticated app needs to be refactored from time to time and that can mean pulling fields out of disparate documents into a new one.
Writing one off rake tasks is way less convenient and error prone than having migrations be part of your deploy script so that it always gets run on every environment.
https://github.com/adacosta/mongoid_rails_migrations brings AR style migrations to mongoid.
You might need them less often, but you will certainly need them as an app grows.
Below is a nice code example for data migration script with mongoid and the ruby mongo driver - to be used when your updated model no longer match production data.
http://pivotallabs.com/users/lee/blog/articles/1548-mongoid-migrations-using-the-mongo-driver
I whish we would stop using "no migrations with mongoid" as slogan. It'll turn people to MongoDB for the wrong reasons, and it's only partially true. No schema, true, but data still needs to be maintained, which IMO is harder with MongoDB than RDBMs. There are other, great reasons for choosing MongoDB and it depends on your problem.
Is there any way that I can fire a raw mongo query directly in Ruby instead of converting them to the native Ruby objects?
I went through Ruby Mongo Tutorial, but I cannot find such a method anywhere.
If it were mysql, I would have fired a query something like this.
ActiveRecord::Base.connection.execute("Select * from foo")
My mongo query is a bit large and it is properly executing in the MongoDB console. What I want is to directly execute the same inside Ruby code.
Here's a (possibly) better mini-tutorial on how to get directly into the guts of your MongoDB. This might not solve your specific problem but it should get you as far as the MongoDB version of SELECT * FROM table.
First of all, you'll want a Mongo::Connection object. If
you're using MongoMapper then you can call the connection
class method on any of your MongoMapper models to get a connection
or ask MongoMapper for it directly:
connection = YourMongoModel.connection
connection = MongoMapper.connection
Otherwise I guess you'd use the from_uri constructor to build
your own connection.
Then you need to get your hands on a database, you can do this
using the array access notation, the db method, or get
the current one straight from MongoMapper:
db = connection['database_name'] # This does not support options.
db = connection.db('database_name') # This does support options.
db = MongoMapper.database # This should be configured like
# the rest of your app.
Now you have a nice shiny Mongo::DB instance in your hands.
But, you probably want a Collection to do anything interesting
and you can get that using either array access notation or the
collection method:
collection = db['collection_name']
collection = db.collection('collection_name')
Now you have something that behaves sort of like an SQL table so
you can count how many things it has or query it using find:
cursor = collection.find(:key => 'value')
cursor = collection.find({:key => 'value'}, :fields => ['just', 'these', 'fields'])
# etc.
And now you have what you're really after: a hot out of the oven Mongo::Cursor
that points at the data you're interested in. Mongo::Cursor is
an Enumerable so you have access to all your usual iterating
friends such as each, first, map, and one of my personal
favorites, each_with_object:
a = cursor.each_with_object([]) { |x, a| a.push(mangle(x)) }
There are also command and eval methods on Mongo::DB that might do what you want.
In case you are using mongoid you will find the answer to your question here.
If you're using Mongoid 3, it provides easy access to its MongoDB driver: Moped. Here's an example of accessing some raw data without using Models to access the data:
db = Mongoid::Sessions.default
# inserting a new document
collection = db[:collection_name]
collection.insert(name: 'my new document')
# finding a document
doc = collection.find(name: 'my new document').first
# "select * from collection"
collection.find.each do |document|
puts document.inspect
end