I have a set of legacy database tables that i cannot normalize out to what should have been done in the first place. e.g one big table with 200 columns.
I'm building an API and would like to represent this data to the consumer in a better state, and perhaps address the database issues at a later stage, there are many backend systems that reply on the data and changes are not easy.
I wanted to represent the current database schema using Active Record, however perform a model transformation into a new model that will be used for presentation only to an API consumer as json data.
current database schema:
Products table (200 columns)
New Model:
Product
+ Pricing
+ Assets
+ Locations
+ Supplier
I could hard-code a json string in a template, but feel that would not be a very poor approach.
What approach or gem would you recommend to tackle this best?
I have looked at :
RABL
ActiveModel::Serializers
If you define an as_json method that returns a hash, ActiveRecord will take care of the serialization for you. E.g.
class Product < ActiveRecord::Base
def as_json options = {}
{
product: <product value>,
pricing: <pricing value>,
# ... etc.
}
end
end
Now you can do:
> Product.first.to_json
=> "{\"product\":<product_value> ... }"
You can even render these as json from the controllers via:
render json: #model
Related
When simply displaying large amounts of data (over 100k records) my code works well, and I paginate on the server.
However, when I need to sort this data I'm stuck. I'm only sorting on the page, and NOT sorting on ALL the records related to this one customer.
How can I paginate but also sort across all the records of my customer and NOT simply sort the records returned from the server side pagination?
I'm also using BootStrap Table to display all my data.
Here is my code that gets all the customers:
def get_customers
#data_to_return = []
#currency = current_shop.country_currency
customers = current_shop.customers.limit(records_limit).offset(records_offset)#.order("#{sort_by}" " " "#{sort_order}")
customers.each do |customer|
#data_to_return.push(
state: false,
id: customer.id,
email: customer.email,
accepts_marketing: customer.accepts_marketing,
customer_status: customer.customer_status,
tags: customer.tags)
end
sort_customers
end
And then this is the sort_customers method:
def sort_customers
fixed_data = data_to_return.sort_by {|hsh| hsh[sort_by]}
customer_size = current_shop.customers.length
if sort_order == "ASC"
fixed_data
else
fixed_data.reverse!
end
render json: {"total": customer_size, "rows": fixed_data}
end
In the above code you can see that data_to_return is coming from get_customers and its limited. But I don't want to return ALL the customers for many reasons.
How can I sort across all the records, but only return the paginated subset?
You should actually sort at the model/query level, not at the ruby level.
The difference is basically:
# sort in ruby
relation.sort_by { |item| foo(item) }
# sort in database - composes with pagination
relation.order('column_name ASC/DESC')
In the first case, the relation is implicitly executed, enumerated and converted to array before calling sort_by. If you did pagination (manually or with kaminari), you will get just that page of data.
In the second case, you are actually composing the limit, offset and where (limit and offset are anyways used under the hood by kaminari, where is implicit when you use associations) with a order so your database would execute
SELECT `customers`.`*` FROM `customers`
WHERE ...
OFFSET ...
LIMIT ...
ORDER BY ...
which will return the correct data.
A good option is to define scopes in the model, like
class Customer < ApplicationRecord
scope :sorted_by_email, ->(ascending = true) { order("email #{ascending ? 'ASC' : 'DESC'}") }
end
# in controller
customers = current_shop.customers.
limit(records_limit).
offset(records_offset).
sorted_by_email(false)
You can resolve sorting and paginate issue using Data Tables library, which is client side. It's a Jquery library. Using this you need to load all data into page, then it would work very well.
Below are the references please check.
Data tables jquery libray
Data tables gem for rails
You can try these, they will work very well. You can customise it as well
If the answer is helpful, you can accept it.
Is there a way to restrict the "columns" returned from a Realm Xamarin LINQ query?
For example, if I have a Customer RealmObject and I want a list of all customer names, do I have to query All<Customer> and then enumerate the results to build the names list? That seems cumbersome and inefficient. I am not seeing anything in the docs. Am I missing something obvious here? Thanks!
You have to remember that Realm is an object based store. In a RDBMS like Sqlite, restricting the return results to a sub-set of "columns" of an "record" makes sense, but in an object store, you would be removing attributes from the original class and thus creating a new dynamic class to then instantiate these new classes as objects.
Thus is you want just a List of strings representing the customer names you can do this:
List<string> names = theRealm.All<Customer>().ToList().Select(customer => customer.Name).ToList();
Note: That you take the Realm.All<> results to a List first and then using a Linq Select "filter" just the property that you want. Using a .Select directly on a RealmResults is not currently supported (v0.80.0).
If you need to return a complex type that is a subset of attributes from the original RealObject, assuming you have a matching POCO, you can use:
var custNames = theRealm.All<Customer>().ToList().Select((Customer c) => new Name() { firstName = c.firstName, lastName = c.lastName } );
Remember, once you convert a RealmResult to a static list of POCOs you do lose the liveliness of using RealmObjects.
Personally I avoid doing this whenever possible as Realm is so fast that using a RealmResult and thus the RealObjects directly is more efficient on processing time and memory overhead then converting those to POCOs everytime you need to new list...
I have Team and Players classes and want to return the data in one JSON string which contains Team info but at the same time it displays all the information about the players.
class Team < ActiveRecord::Base
has_many :players
end
class Players < ActiveRecord::Base
belongs_to :team
end
I know how to retrieve the information about team and players but not in the same query. Another problem is I don't how to merge the result JSONs in one JSON.
team = Team.last.to_json
player = team.players.to_json
How can I query the info about Team and Players in the same query. I tried:
#team = Team.includes(:players).where(players: {team_id: Team.last}).last.to_json
and it only returns me information about the team. I want a JSON like:
-id
-name
-players
-player
-player
In case it's impossible, how can I merge into one JSON all the information from the two queries.
You can write a "join" to incorporate the players in the team with the team information. At that point you'll have a structure that has the information needed to create the JSON. See "12 Joining Tables" from the Active Record documentation for more information.
Or, you can make two separate queries, then create a bit more complex JSON hash or array allowing you to output both sets of data into one larger serialized object. For instance:
require 'json'
team = {
'name' => 'bears'
}
players = {
'1' => 'fred',
'2' => 'joe'
}
puts ({
'team' => team,
'players' => players
}).to_json
Here's the output:
{"team":{"name":"bears"},"players":{"1":"fred","2":"joe"}}
Here's the data returned back to the Ruby object:
data = '{"team":{"name":"bears"},"players":{"1":"fred","2":"joe"}}'
JSON[data]
# => {"team"=>{"name"=>"bears"}, "players"=>{"1"=>"fred", "2"=>"joe"}}
Also, since you're using Sinatra, it's not necessary to use Active Record. Sequel is a very good ORM, and is my personal favorite when working with Sinatra. You might find it easier to work with.
Another option to manual serialization is to use ActiveModel::Serializer which allows you to define relationships between objects and gives you finer grained choices of what to include when you serialize, what to filter out and what related objects to preload. An alternative could also be Rabl which also has quite a nice API.
If you're just playing around with a small amount of JSON this might be overkill, but it's a nice practice to be more organized
Is there a Ruby, or Activerecord method that can write and read a hash to and from a database field?
I need to write a web utility to accept POST data and save it to a database, then later on pull it from the database in its original hash form. But ideally without 'knowing' what the structure is. In other words, my data store needs to be independent of any particular set of hash keys.
For example, one time the external app might POST to my app:
"user" => "Bill",
"city" => "New York"
But another time the external app might POST to my app:
"company" => "Foo Inc",
"telephone" => "555-5555"
So my utility needs to save an arbitrary hash to a text field in the database, then, later, recreate the hash from what was saved.
Rails 4 adds support for the Postgres hstore data type which will let you add hashes directly into your (postgres) database.
If you are using Rails 4 and Postgres, you can use hstore in your migration:
def up
execute "create extension hstore"
add_column :table, :column, :hstore
end
def down
remove_column :table, :column
end
That execute command will enable hstore in Postgres, so you only have to do that once.
This will enable you to store a hash in :column just like you would any other data type.
There are two ways to do this:
Serialize your hash and store it in a text field.
Split the hash and store each key in a separate row.
The problem with the first approach is that finding and manipulating is difficult and expensive. For example, prefix a "0" before the telephone number of all employees working in Foo Inc. will be a nightmare, compared to storing the data in regular tabular format.
Your schema would be:
employees (id, created_at, updated_at)
employee_details (id, employee_id, key, value)
So, to store
"company" => "Foo Inc",
"telephone" => "555-5555"
you would do:
employees: 1, 2012-01-01, 2012-01-01
employee_details (1, 1, "company", "Foo Inc"), (2, 1, "telephone", "555-5555")
Drawbacks of this approach: Rails does not natively support such kind of a schema.
You can use serialization with 3 options: Marshal in binary format, YAML and JSON human-readable formats of data store.
Once you are trying each of methods, do not forget to measure time to serialize and deserialize as well. If you need to pull data back in origin format, JSON is the good choice to use, because you don't need to deserialize it, but use it as a string itself.
You're looking for serialization. It will help you to do exactly what you want.
Rails 4 has a new feature called Store, so you can easily use it to solve your problem. You can define an accessor for it and it is recommended you declare the database column used for the serialized store as a text, so there's plenty of room. The original example:
class User < ActiveRecord::Base
store :settings, accessors: [ :color, :homepage ], coder: JSON
end
u = User.new(color: 'black', homepage: '37signals.com')
u.color # Accessor stored attribute
u.settings[:country] = 'Denmark' # Any attribute, even if not specified with an accessor
# There is no difference between strings and symbols for accessing custom attributes
u.settings[:country] # => 'Denmark'
u.settings['country'] # => 'Denmark'
How can I interact with objects I've created based on their given attributes in Ruby?
To give some context, I'm parsing a text file that might have several hundred entries like the following:
ASIN: B00137RNIQ
-------------------------Status Info-------------------------
Upload created: 2010-04-09 09:33:45
Upload state: Imported
Upload state id: 3
I can parse the above with regular expressions and use the data to create new objects in a "Product" class:
class Product
attr_reader :asin, :creation_date, :upload_state, :upload_state_id
def initialize(asin, creation_date, upload_state, upload_state_id)
#asin = asin
#creation_date = creation_date
#upload_state = upload_state
#upload_state_id = upload_state_id
end
end
After parsing, the raw text from above will be stored in an object that look like this:
[#<Product:0x00000101006ef8 #asin="B00137RNIQ", #creation_date="2010-04-09 09:33:45 ", #upload_state="Imported ", #upload_state_id="3">]
How can I then interact with the newly created class objects? For example, how might I pull all the creation dates for objects with an upload_state_id of 3? I get the feeling I'm going to have to write class methods, but I'm a bit stuck on where to start.
You would need to store the Product objects in a collection. I'll use an array
product_collection = []
# keep adding parse products into the collection as many as they are
product_collection << parsed_product_obj
#next select the subset where upload_state_ud = 3
state_3_products = product_collection.select{|product| product.upload_state_id == 3}
attr reader is a declarative way of defining properties/attributes on your product class. So you can access each value as obj.attribute like I have done for upload_state_id above.
select selects the elements in the target collection, which meet a specific criteria. Each element is assigned to product, and if the criteria evaluates to true is placed in the output collection.