Is there a Ruby, or Activerecord method that can write and read a hash to and from a database field?
I need to write a web utility to accept POST data and save it to a database, then later on pull it from the database in its original hash form. But ideally without 'knowing' what the structure is. In other words, my data store needs to be independent of any particular set of hash keys.
For example, one time the external app might POST to my app:
"user" => "Bill",
"city" => "New York"
But another time the external app might POST to my app:
"company" => "Foo Inc",
"telephone" => "555-5555"
So my utility needs to save an arbitrary hash to a text field in the database, then, later, recreate the hash from what was saved.
Rails 4 adds support for the Postgres hstore data type which will let you add hashes directly into your (postgres) database.
If you are using Rails 4 and Postgres, you can use hstore in your migration:
def up
execute "create extension hstore"
add_column :table, :column, :hstore
end
def down
remove_column :table, :column
end
That execute command will enable hstore in Postgres, so you only have to do that once.
This will enable you to store a hash in :column just like you would any other data type.
There are two ways to do this:
Serialize your hash and store it in a text field.
Split the hash and store each key in a separate row.
The problem with the first approach is that finding and manipulating is difficult and expensive. For example, prefix a "0" before the telephone number of all employees working in Foo Inc. will be a nightmare, compared to storing the data in regular tabular format.
Your schema would be:
employees (id, created_at, updated_at)
employee_details (id, employee_id, key, value)
So, to store
"company" => "Foo Inc",
"telephone" => "555-5555"
you would do:
employees: 1, 2012-01-01, 2012-01-01
employee_details (1, 1, "company", "Foo Inc"), (2, 1, "telephone", "555-5555")
Drawbacks of this approach: Rails does not natively support such kind of a schema.
You can use serialization with 3 options: Marshal in binary format, YAML and JSON human-readable formats of data store.
Once you are trying each of methods, do not forget to measure time to serialize and deserialize as well. If you need to pull data back in origin format, JSON is the good choice to use, because you don't need to deserialize it, but use it as a string itself.
You're looking for serialization. It will help you to do exactly what you want.
Rails 4 has a new feature called Store, so you can easily use it to solve your problem. You can define an accessor for it and it is recommended you declare the database column used for the serialized store as a text, so there's plenty of room. The original example:
class User < ActiveRecord::Base
store :settings, accessors: [ :color, :homepage ], coder: JSON
end
u = User.new(color: 'black', homepage: '37signals.com')
u.color # Accessor stored attribute
u.settings[:country] = 'Denmark' # Any attribute, even if not specified with an accessor
# There is no difference between strings and symbols for accessing custom attributes
u.settings[:country] # => 'Denmark'
u.settings['country'] # => 'Denmark'
Related
I generated the following using a Keyword extraction API
(I searched for "Hitler" because no one is trying to sell you things when you search for him.)
x=[{"relevance"=>"0.592174", "text"=>"eight-year-old Hitler"}, {"relevance"=>"0.589796", "text"=>"Hitler states"}, {"relevance"=>"0.589118", "text"=>"Hitler lived."}, {"relevance"=>"0.585102", "text"=>"Hitler learnt"}, {"relevance"=>"0.580278", "text"=>"Hitler Youth"}, {"relevance"=>"0.414842", "text"=>"German Workers"}, {"relevance"=>"0.413532", "text"=>"German military leaders"}, {"relevance"=>"0.404701", "text"=>"nazi party"}, {"relevance"=>"0.391943", "text"=>"Mein Kampf"}, {"relevance"=>"0.388932", "text"=>"Reichstag"}, ]
So the database should have :
Eight-year-old hitler
Hitler states
...
I've tried x.each do |x| and
x.to_a.each do
x["text"]
x
None of which work.
Since you haven't given the following information, I will make some assumptions:
Database table name
Schema for table
Model name
ORM
Let's rename x to search_results. Let's also assume that we have a table in the database called search_results, a model called SearchResult, that the search_results table has a column called text, and that we are using ActiveRecord as the ORM:
search_results.each do |data|
SearchResult.create(text: data[:text])
end
Or since you have an array of hashes, simply:
SearchResult.create(search_results)
I have Team and Players classes and want to return the data in one JSON string which contains Team info but at the same time it displays all the information about the players.
class Team < ActiveRecord::Base
has_many :players
end
class Players < ActiveRecord::Base
belongs_to :team
end
I know how to retrieve the information about team and players but not in the same query. Another problem is I don't how to merge the result JSONs in one JSON.
team = Team.last.to_json
player = team.players.to_json
How can I query the info about Team and Players in the same query. I tried:
#team = Team.includes(:players).where(players: {team_id: Team.last}).last.to_json
and it only returns me information about the team. I want a JSON like:
-id
-name
-players
-player
-player
In case it's impossible, how can I merge into one JSON all the information from the two queries.
You can write a "join" to incorporate the players in the team with the team information. At that point you'll have a structure that has the information needed to create the JSON. See "12 Joining Tables" from the Active Record documentation for more information.
Or, you can make two separate queries, then create a bit more complex JSON hash or array allowing you to output both sets of data into one larger serialized object. For instance:
require 'json'
team = {
'name' => 'bears'
}
players = {
'1' => 'fred',
'2' => 'joe'
}
puts ({
'team' => team,
'players' => players
}).to_json
Here's the output:
{"team":{"name":"bears"},"players":{"1":"fred","2":"joe"}}
Here's the data returned back to the Ruby object:
data = '{"team":{"name":"bears"},"players":{"1":"fred","2":"joe"}}'
JSON[data]
# => {"team"=>{"name"=>"bears"}, "players"=>{"1"=>"fred", "2"=>"joe"}}
Also, since you're using Sinatra, it's not necessary to use Active Record. Sequel is a very good ORM, and is my personal favorite when working with Sinatra. You might find it easier to work with.
Another option to manual serialization is to use ActiveModel::Serializer which allows you to define relationships between objects and gives you finer grained choices of what to include when you serialize, what to filter out and what related objects to preload. An alternative could also be Rabl which also has quite a nice API.
If you're just playing around with a small amount of JSON this might be overkill, but it's a nice practice to be more organized
I have a set of legacy database tables that i cannot normalize out to what should have been done in the first place. e.g one big table with 200 columns.
I'm building an API and would like to represent this data to the consumer in a better state, and perhaps address the database issues at a later stage, there are many backend systems that reply on the data and changes are not easy.
I wanted to represent the current database schema using Active Record, however perform a model transformation into a new model that will be used for presentation only to an API consumer as json data.
current database schema:
Products table (200 columns)
New Model:
Product
+ Pricing
+ Assets
+ Locations
+ Supplier
I could hard-code a json string in a template, but feel that would not be a very poor approach.
What approach or gem would you recommend to tackle this best?
I have looked at :
RABL
ActiveModel::Serializers
If you define an as_json method that returns a hash, ActiveRecord will take care of the serialization for you. E.g.
class Product < ActiveRecord::Base
def as_json options = {}
{
product: <product value>,
pricing: <pricing value>,
# ... etc.
}
end
end
Now you can do:
> Product.first.to_json
=> "{\"product\":<product_value> ... }"
You can even render these as json from the controllers via:
render json: #model
When I try to run the following code, DataMapper calls for 3 queries in just these two lines. Can anyone explain why it would do this?
#u = User.first(:uid => 1, :fields => [:uid, :name])
json #u
This calls the following queries:
SELECT "uid", "name" FROM "users" WHERE "uid" = 1 ORDER BY "uid" LIMIT 1
SELECT "uid", "email" FROM "users" WHERE "uid" = 1 ORDER BY "uid"
SELECT "uid", "accesstoken" FROM "users" WHERE "uid" = 1 ORDER BY "uid"
It is worth noting that datamapper has a validation on name for being => unique
Also, the accesstoken is lazily loaded so it should only be queried when asked for specifically, which must be happening when serializing it to a json object.
EDIT:
I have added my model class for clarification. I just want one query made for the uid and name without having to extract them individually from the object. Maybe this is the only way?
property :uid, Serial
property :name, String
property :email, String
property :accesstoken, Text
ANSWER:
Use the dm-serializer gem that has this support built-in
https://github.com/datamapper/dm-serializer
The first query is invoked by your User.first... call. Notice the fields it's selecting are what you requested - uid and name
The second and third queries are getting run in the json serialization, as it's lazy loading each property you didn't already load.
So you either need to do a custom serialization to only output uid and name for your users, or you should just remove the field selection from your initial query so it all gets loaded at once.
Update:
To do a custom serialization with datamapper, you can use the dm-serializer gem https://github.com/datamapper/dm-serializer and call #u.to_json(only: [:uid, :name])
Alternatively in this simple case you could just build the serialized object you want yourself, for which there are many examples: Rails3: Take controll over generated JSON (to_json with datamapper ORM)
I have a table named subs which has many articles. The articles table has a timestamp column called published.
Sub.select( "subs.*,MAX(articles.published) published").joins("LEFT OUTER JOIN articles ON subs.id=articles.sub_id").group("subs.id").first.published.class
=> String
Article.select("max(published) published").group("id").first.published.class
=> ActiveSupport::TimeWithZone
I want to get an ActiveSupport::TimeWithZone object back from the first query.
Rails 3
Rails determines how to type cast attributes based on their database column definitions. For example, say you have a created_at method on your Sub model. When a record is loaded read_attribute is used (ActiveRecord::AttributeMethods::Read). This uses type_cast_attribute which determines how to cast the value based on the column info. For example, if you are using PostgreSQL it may use:
Sub.columns.detect { |c| c.name == "created_at" }.type_cast_code("v")
=> "ActiveRecord::ConnectionAdapters::PostgreSQLColumn.string_to_time(v)"
But Rails doesn't know what to do with columns that aren't on the Sub model. So it just gives back a String. If you need to work with a ActiveSupport::TimeWithZone object, you can cast the value with:
published = Sub.select( "subs.*,MAX(articles.published) published").joins("LEFT OUTER JOIN articles ON subs.id=articles.sub_id").group("subs.id").first.published
published.present? ? Time.zone.parse(published) : nil
Rails 4
In Rails 4, Rails is smarter about this kind of type-casting. When the SQL is executed, ActiveRecord::Result is created and the column_types are passed to the initializer. In your example Sub.select query, the published column would be cast as a Time object.