I try to reduce SQL queries from my Rails application.
I have some controller like:
class Rest::MyController < Rest::BaseController
def show
render xml: some_model, status: :ok
end
private
def my_associations
[
:model2,
:model3,
:model4,
]
end
def some_model
#some_model ||= SomeModel.includes(my_associations).where(id: test_params[:id])
end
def test_params
params.permit(:id)
end
end
To avoid N + 1 I use includes, so basically when i try to execute some_model method, AR make lot of call's like that (SELECT ALL FROM):
SomeModel Load (1.7ms) SELECT `model2`.* FROM `model2` WHERE `model2`.`type` IN ('SomeModel') AND `model2`.`is_something` = 0 AND `model2`.`id` = 1
SomeModel Load (1.7ms) SELECT `model3`.* FROM `model3` WHERE `model3`.`type` IN ('SomeModel') AND `model3`.`is_something` = 0 AND `model3`.`id` = 1
SomeModel Load (1.7ms) SELECT `model4`.* FROM `model4` WHERE `model4`.`type` IN ('SomeModel') AND `model4`.`is_something` = 0 AND `model4`.`id` = 1
This is only example
Now, through my serializer I would like to get only selected columns for model2, model3 and model4
Unfortunately Active record make a call like SELECT model2.* FROM
For example, for model2 serializer i try to get only (:id, :name) columns.
Is it possible to make a call like ?
SELECT some_model.*, model2.id, model2.name FROM `some_model`
instead
SELECT `model2`.* FROM `model2` WHERE `model2`.`type` IN ('SomeModel')
If you want to use Rails's includes feature, then no, there isn't an especially good way to selectively control the columns from included models.
If you're looking for help to optimize the query, you'll need to provide more specifics about the data schema and the desired output.
Related
I have some working script filtering my results with Active Record Scoping. Everything works fine when i want to filter by comparing params with data from database.
But i have some function counting car price inside _car.html.erb partial, the result of this function depends on params result.
How can i scope search results by result of this function and show only cars which are under some price (defined in params).
Some code to make it more clear:
car.rb (model file)
scope :price_leasing, -> (price_leasing) { where('price_leasing <= ?', price_leasing) }
# for now price_leasing is getting price from database
scope :brand, -> (brand) { where brand: brand }
scope :car_model, -> (car_model) { where car_model: car_model }
scope :category, -> (category) { where category: category }
cars_controller.rb (controller file)
def index
#cars = Car.where(nil)
#cars = #cars.price_leasing(params[:price_leasing]) if params[:price_leasing].present?
#cars = #cars.brand(params[:brand]) if params[:brand].present?
#cars = #cars.car_model(params[:car_model]) if params[:car_model].present?
#cars = #cars.category(params[:category]) if params[:category].present?
#brands = Brand.all # importing all car brands into filters
end
in index.html.erb i have "render #cars" code
<%=
if #cars.size > 0
render #cars.where(:offer_status => 1)
else
render html: '<p>Nie znaleziono pasujących wyników.</p>'.html_safe
end
%>
inside _car.html.erb file i have function from helper
<h3 class="car-cell__price"><%= calculate_car_price(car.price, car.id) %> <span class="car-cell__light-text">zł/mc</span></h3>
my calculate_car_price() function inside helper
def calculate_car_price(car_price, car_id)
car = Car.find(car_id)
fullprice = car_price
if params[:price_leasing].present?
owncontribution = params[:price_leasing].to_i
else
owncontribution = car.owncontribution
end
pv = fullprice - owncontribution + (0.02 * fullprice)
if params[:period].present?
carperiod = params[:period].to_i
carprice = (Exonio.pmt(0.0522/60, carperiod, pv)) * -1
else
carprice = (Exonio.pmt(0.0522/60, 60, pv)) * -1
end
p number_with_precision(carprice, precision: 0)
end
i would love to scope by the result of this function. Is it possible?
The thing about scopes are that they are implemented at the database level. You want to select records that have a value
To do what you want at the DB level, you would need to a virtual column in the database, the implementation will change based on which database product you're using (I would not expect a virtual column definition in postgreSQL to be the same as a virtual column definition in mySQL)
So I'm not sure there's an optimal way to do this.
I would suggest you build your own class method and instance method in your model Car. It would be less performant but easier to understand and implement.
def self.car_price_under(target_price, params)
select { |v| v.car_price_under?(target_price, params) }
end
def car_price_under?(target_price, params)
full_price = price
if params[:price_leasing].present?
my_own_contribution = params[:price_leasing].to_i
else
my_own_contribution = owncontribution
end
pv = full_price - my_own_contribution + (0.02 * full_price)
if params[:period].present?
car_period = params[:period].to_i
new_car_price = (Exonio.pmt(0.0522/60, car_period, pv)) * -1
else
new_car_price = (Exonio.pmt(0.0522/60, 60, pv)) * -1
end
new_car_price <= target_price
end
This would let you do...
#cars = Car.where(nil)
#cars = #cars.brand(params[:brand]) if params[:brand].present?
#cars = #cars.car_model(params[:car_model]) if params[:car_model].present?
#cars = #cars.category(params[:category]) if params[:category].present?
#cars = #cars.car_price_under(target_price, params) if target_price.present?
Note that this is happening in rails, NOT in the database, so the car_price_under method should be called after all other scopes to minimise the number of records that need to be examined. You cannot chain additional scopes as #cars would be an array... if you want to be able to chain additional scopes (or you want an active record relation, not an array) you could do something like #cars = Car.where(id: #cars.pluck(:id))
This post should be a little more complex than usual.
We have created a new field for an account.invoice.line : pack_operation. With this field, we can print serial/lot number for each line on the PDF invoice (this part works well).
Many hours passed trying to write the domain to select the EXACT and ONLY stock pack operation for each invoice line.
In the code below, we used the domain [('id','=', 31)] to make our tests printing the PDF.
Ho to write this domain to be sure at 100 % that we will get the right stock pack operation for each invoice line?
I really need your help here... Too complex for my brain.
Our code :
class AccountInvoiceLine(models.Model):
_inherit = "account.invoice.line"
pack_operation = fields.Many2one(comodel_name='stock.pack.operation', compute='compute_stock_pack_operation_id')
def compute_stock_pack_operation_id(self):
stock_operation_obj = self.env['stock.pack.operation']
stock_operation = stock_operation_obj.search( [('id','=', 31)] )
self.pack_operation = stock_operation[0]
EDIT#1
I know that you won't like my code. But, this one seems to work. I take any comments and improvements with pleasure.
class AccountInvoiceLine(models.Model):
_inherit = "account.invoice.line"
pack_operation = fields.Many2one(comodel_name='stock.pack.operation', compute='compute_stock_pack_operation_id')#api.one
def compute_stock_pack_operation_id(self):
procurement_order_obj = self.env['procurement.order']
stock_operation_obj = self.env['stock.pack.operation']
all_picking_ids_for_this_invoice_line = []
for saleorderline in self.sale_line_ids:
for procurement in saleorderline.procurement_ids:
for stockmove in procurement.move_ids:
if stockmove.picking_id.id not in all_picking_ids_for_this_invoice_line
all_picking_ids_for_this_invoice_line.append(stockmove.picking_id.id)
all_picking_ids_for_this_invoice_line))
stock_operation = stock_operation_obj.search(
[ '&',
('picking_id','in',all_picking_ids_for_this_invoice_line),
('product_id','=',self.product_id.id)
]
)
self.pack_operation = stock_operation[0]
The pack_operation field is a computed field, that be default means that the field will not be saved on the database unless you set store=True when you define your field.
So, what you can do here is change:
pack_operation = fields.Many2one(comodel_name='stock.pack.operation', compute='compute_stock_pack_operation_id')
to:
pack_operation = fields.Many2one(comodel_name='stock.pack.operation', compute='compute_stock_pack_operation_id', store=True)
And try running your query again.
I would like to combine 2 querysets from 2 differents models, then I need to order them by date and finally my goal is to serialize it.
So far I did that :
last_actions = serializers.SerializerMethodField()
def get_last_actions(self, obj):
prc = obj.product_request_configs.all().order_by('modified_date')[:5]
psc = obj.product_send_configs.all().order_by('modified_date')[:5]
result_list = sorted(
chain(prc, psc),
key=attrgetter('modified_date'),
reverse=True)
But I don't know how to call my two django rest serializers so that I can return the right data.
If I could make a database view it coult be simpler I think.
Serializers are designed for match one model relationship, so we need to create a custom Model for the logic you are trying to achieve:
class CustomModel(models.Model):
def dictfetchall(self, cursor):
"""Returns all rows from a cursor as a dict"""
desc = cursor.description
return [dict(zip([col[0] for col in desc], row))
for row in cursor.fetchall()]
def yourMethod(self):
cursor = connection.cursor()
cursor.execute("""
select field1, field2 from app_table
where field1=%s and field2=%s group by field1
""",
[value1, value2,]
)
return self.dictfetchall(cursor)
class Meta:
abstract = True
This will return a dictionary and then you can serialize that response with a seializer like:
class CustomModelSerializer(serializers.Serializer):
field1 = serializers.IntegerField()
field2 = serializers.CharField()
Please note that on SQL you can use as keyword to rename some fields, the current name of fields must match var names in your serializer.
I have an hash object:
#chosen_opportunity = {"id"=>66480, "prize_id"=>4, "admin_user_id"=>1, "created_at"=>2015-09-20 18:37:29 +0200, "updated_at"=>2015-09-20 18:37:29 +0200, "opportunity_available"=>true}
How do I update the value of deal_available to false?
I tried this but it fails:
#chosen_opportunity['deal_available'] = false
#chosen_opportunity.save
controllers/deal_controller.rb:
def show_opportunity
#deal = Deal.friendly.find(params[:id])
#chosen_opportunity = Opoortunity.find_by_sql(
" SELECT \"opportunities\".*
FROM \"opportunities\"
WHERE (deal_id = #{#deal.id}
AND opportunity_available = true)
ORDER BY \"opportunities\".\"id\" ASC LIMIT 1"
)
# comes from http://apidock.com/rails/ActiveRecord/Base/find_by_sql/class
#chosen_opportunity[0].attributes['opportunity_available'] = false
#chosen_opportunity[0].save
respond_to do |format|
format.js
end
end
Can I update the value of opportunity_available from the Opportunity model inside a Deal controller? Is that why it's not working?
I know I could use Active Record but I need to use raw PostgreSQL for the first query. Thanks for your understanding of this non very Rails-y way.
You can change your code to:
#chosen_opportunity = Opportunity.find_by deal_id: deal.id, opportunity_available: true
#chosen_opportunity.opportunity_available = false
#chosen_opportunity.save!
This is much more Rails compatible. In addition, if I'm not mistaken, Rails won't let you save an object that you got through find_by_sql, so at the least you'd need to get a proper model object from the result. You can write (very ugly) code like:
Opportunity.where(id: #chosen_opportunity[0].attributes['id'])
.update_all(opportunity_available: false)
Warning: This will update the database, but not the #chosen_opportunity[0] object.
There is no method save on a hash. You have to do it on the model that holds that hash as an instance variable.
I have two models, User and Group, where groups contain many users. If I want to count the number of users in each group using a single query, I can use the following SQL:
select id, (select count(1) from users where group_id = groups.id) from groups
Is it possible to do this efficiently with ActiveRecord?
To be clear, this query will list all group ids, along with the number of users in each group.
You can use either to get count
using associations
group = Group.find(1) #find group with id = 1
group.users.count # count users whose group_id = 1 calls db everytime
or
group.users.size # get size from cache if group.users loaded
or directly
User.where(:group_id=>1).count
count helper fires a count(*) query on the database with specified conditions
check more options at
http://apidock.com/rails/ActiveRecord/Calculations/count
also I recommend you to go through rails guides
I found an efficient solution using a join:
Group.all(
:select => "groups.id, count(u.group_id) as users_count",
:joins => "LEFT OUTER JOIN users u ON u.group_id = groups.id",
:group => "groups.id"
)
First solution is simply translate your query to ActiveRecord and use the subquery:
subquery = User.where("users.group_id = groups.id").select('count(1)')
groups_with_count = Group.select(:id, "(#{subquery.to_sql}) as users_count")
Or use a sql grouping for the same result
groups_with_count = Group.joins(:users).select(:id, 'count(users.id) as users_count').group(:id)
in both case you can now have the result in ONE query with MINIMAL raw sql:
groups_with_count.each { |group| puts "#{group.id} => #{group.users_count}" }
Additional note
You can write the first subquery as subquery = User.via(:group).select('count(1)') which is more simple and maintainable imo, by using the following helper.
I've used this code on several projects in order to write nicer subquery:
class ApplicationRecord < ActiveRecord::Base
# transform Raw sql that references an association such as: Shift.where('shifts.id = checkins.shift_id')
# into a simpler version Shift.via(:checkin) if shift have the checkin relationship
# No support for polymorphic association
# Basic support for "through" reflection (using join)
def via(name)
association = reflect_on_association(name)
raise ArgumentError, "#{name} is not a valid association of #{self.class.name}" unless association
raise NotImplementedError if association.polymorphic?
join_keys = association.join_keys
table_column = arel_table[join_keys.foreign_key]
association_column = Arel::Table.new(association.table_name)[join_keys.key]
if association.through_reflection?
through_association = association.through_reflection
table_column = Arel::Table.new(through_association.table_name)[join_keys.foreign_key]
joins(through_association.name).where(table_column.eq(association_column))
else
where(table_column.eq(association_column))
end
end
end