Consequent attribute calculations with a queuing system - ruby

Question

For all of the following assume these:
rails v3.0
ruby v1.9
resque
We have 3 models:
Product belongs_to :sku, belongs_to :category
Sku has_many :products, belongs_to :category
Category has_many :products, has_many :skus
When we update the product (let's say we disable it) we need to have some things happen to the relevant sku and category. The same is true for when a sku is updated.
The proper way of achieving this is have an after_save on each model that triggers the other models' update events.
example:
products.each(&:disable!)
# after_save triggers self.sku.products_updated
# and self.category.products_updated (self is product)
Now if we have 5000 products we are in for a treat. The same category might get updated hundreds of times and hog the database while doing so.
We also have a nice queueing system, so the more realisting way of updating products would be products.each(&:queue_disable!) which would simply toss 5000 new tasks to the working queue. The problem of 5000 category updates still exists though.
Is there a way to avoid all those updates on the db?
How can we concatenate all the category.products_updated for each category in the queue?

Answer 1

You can ensure a single category update for all the products by using a couple Resque plugins: Resque Unique Job and Resque Scheduler.
Delay the execution of the job to update the category slightly (however long it takes to typically call all the product updates) and ensure each job is unique by including the Unique Job module. Unique Job uses the paramaters of the job, so if you try to queue 2 jobs with category_id 123, it ignores the 2nd one since the job is already queued.
class Product
after_save :queue_category_update
def queue_category_update
Resque.enqueue_at(1.minute.from_now, Jobs::UpdateCategory, category.id) if need_to_update_category?
end
end
module Jobs
module UpdateCategory
include Resque::Plugins::UniqueJob
def self.perform(category_id)
category = Category.find_by_id(category_id)
category.update_some_stuff if category
end
end
end

Answer 2

Do the dependent updates in single SQL calls. #update_all will update many records at once. For example,
In an after_update callback, update all the dependent column values:
class Category
after_update :update_dependent_products
def update_dependent_products
products.update_all(disabled: disabled?) if disabled_changed?
end
end
If that's too slow, move it into a resque job:
class Category
after_update :queue_update_dependent_products
def update_dependent_products
products.update_all(disabled: disabled?) if disabled_changed?
end
def queue_update_dependent_products
Resque.enqueue(Jobs::UpdateCategoryDependencies, self.id) if disabled_changed?
end
end
class Jobs::UpdateCategoryDependencies
def self.perform(category_id)
category = Category.find_by_id(category_id)
category.update_dependent_products if category
end
end
Do similar things for the other model callbacks.