I have a tree of active record objects, something like:
class Part < ActiveRecord::Base
has_many :sub_parts, :class_name => "Part"
def complicated_calculation
if sub_parts.size > 0
return self.sub_parts.inject(0){ |sum, current| sum + current.complicated_calculation }
else
sleep(1)
return rand(10000)
end
end
end
It is too costly to recalculate the complicated_calculation each time. So, I need a way to cache the value. However, if any part is changed, it needs to invalidate its cache and the cache of its parent, and grandparent, etc.
As a rough draft, I created a column to hold the cached calculation in the "parts" table, but this smells a little rotten. It seems like there should be a cleaner way to cache the calculated values without stuffing them along side the "real" columns.
I suggest using association callbacks.
class Part < ActiveRecord::Base
has_many :sub_parts,
:class_name => "Part",
:after_add => :count_sub_parts,
:after_remove => :count_sub_parts
private
def count_sub_parts
update_attribute(:sub_part_count, calculate_sub_part_count)
end
def calculate_sub_part_count
# perform the actual calculation here
end
end
Nice and easy =)
You can stuff the actually cached values in the Rails cache (use memcached if you require that it be distributed).
The tough bit is cache expiry, but cache expiry is uncommon, right? In that case, we can just loop over each of the parent objects in turn and zap its cache, too. I added some ActiveRecord magic to your class to make getting the parent objects simplicity itself -- and you don't even need to touch your database. Remember to call Part.sweep_complicated_cache(some_part) as appropriate in your code -- you can put this in callbacks, etc, but I can't add it for you because I don't understand when complicated_calculation is changing.
class Part < ActiveRecord::Base
has_many :sub_parts, :class_name => "Part"
belongs_to :parent_part, :class_name => "Part", :foreign_key => :part_id
##MAX_PART_NESTING = 25 #pick any sanity-saving value
def complicated_calculation (...)
if cache.contains? [id, :complicated_calculation]
cache[ [id, :complicated_calculation] ]
else
cache[ [id, :complicated_calculation] ] = complicated_calculation_helper (...)
end
end
def complicated_calculation_helper
#your implementation goes here
end
def Part.sweep_complicated_cache(start_part)
level = 1 # keep track to prevent infinite loop in event there is a cycle in parts
current_part = self
cache[ [current_part.id, :complicated_calculation] ].delete
while ( (level <= 1 < ##MAX_PART_NESTING) && (current_part.parent_part)) {
current_part = current_part.parent_part)
cache[ [current_part.id, :complicated_calculation] ].delete
end
end
end
Have a field similar to a counter cache. For example: order_items_amount and have that be a cached calculated field.
Use a after_save filter to recalculate the field on anything that can modify that value. (Including the record itself)
Edit: This is basically what you have now. I don't know of any cleaner solution unless you wanted to store cached calculated fields in another table.
Either using a before_save or an ActiveRecord Observer is the way to go to make sure the cached value is up-to-date. I would use a before_save and then check to see if the value you use in the calculation actually changed. That way you don't have to update the cache if you don't need to.
Storing the value in the db will allow you to cache the calculations over multiple requests. Another option for this is to store the value in memcache. You can make a special accessor and setter for that value that can check the memcache and update it if needed.
Another thought: Will there be cases where you will change a value in one of the models and need the calculation to be updated before you do the save? In that case you will need to dirty the cache value whenever you update any of the calculation values in the model, not with a before_save.
I've found that sometimes there is good reason to de-normalize information in your database. I have something similar in an app that I am working on and I just re-calculate that field anytime the collection changes.
It doesn't use a cache and it stores the most up to date figure in the database.
Related
Context:
Each Order has many Items & Logistics. Each Item & Logistic (as well as the Order itself) have many Revenues.
I am creating Order + Items & Logistics at once using an accepts_nested_attributes_for on Order. However, Revenues gets created using an after_create callback on each of the models Order, Item, and Logistics. Why? Because given the difference in interpretation in these models, the code reads cleaner this way. (But if this way of doing it is what's causing this question to be asked, I will obviously reconsider!)
One key attribute that I need to store in Revenues is pp_charge_id. But pp_charge_id is not something that either Order, Items, or Logistics needs to worry about. I've attached an attr_accessor :pp_charge_id to Order, so that one works fine, however, once I'm in the child Items or Logistics models, I no longer have access to pp_charge_id which again I need to save an associated Revenue. How should I do this?
Controller Code:
#order = Order.new(params) #params includes Order params, and nested params for child Item & Logistics
#order.pp_charge_id = "cash"
#order.save #I need this to not only save the Order, the children Item & Logistics, but then to also create the associated Revenue for each of the aforementioned 3 models
ORDER Model Code:
has_many :items
has_many :revenues
attr_accessor :pp_charge_id
after_create :create_revenue
def create_revenue
self.revenues.create(pp_charge_id: self.pp_charge_id)
end
#This WORKS as expected because of the attr_accessor
ITEM/ LOGISTIC model code:
has_many :revenues
belongs_to :order
after_create :create_revenue
def create_revenue
self.revenues.create(pp_charge_id: self.order.pp_charge_id)
end
#This DOES NOT work because self.order.pp_charge_id is nil
ORDER model code:
belongs_to :order
belongs_to :item
belongs_to :logistic
Again I understand the attr_accessor is not designed to persist across a request or even if the Order itself is reloaded. But it also doesn't make sense to save it redundantly in a table that has no use for it. If the only way to do this is to put the pp_charge_id into the params for the order and save everything all at once (including Revenues), then let me know because I know how to do that. (Again, would just rather avoid that because of how it's interpreted: params are coming from User, Revenue data is something I'm providing)
I think if you want the order's pp_charge_id to apply to all its items and logistics, I'd put all that into the order's after_create callback:
# order.rb
def create_revenue
revenues.create(pp_charge_id: pp_charge_id)
items.each {|i| i.revenues.create(pp_charge_id: pp_charge_id)}
logistics.each {|l| l.revenues.create(pp_charge_id: pp_charge_id)}
end
EDIT: Alternately, you could add inverse_of to your belongs_to declarations, and then I believe Item#create_revenue would see the same Order instance that you set in the controller. So if you also added an attr_accessor to the Item class, you could write its create_revenue like this:
# item.rb
def create_revenue
revenues.create(pp_charge_id: pp_charge_id || order.pp_charge_id)
end
This should cover the new requirement you've mentioned in your comment.
instead of using after_create and accessors you should consider having a proper method that does exactly what you need, ie:
Order.create_with_charge(:cash, params)
i find it disturbing to persist redundant information in the database just because the code reads cleaner that way!
I have a model:
class A < ActiveRecord::Base
has_many :B
end
And I want to reset or update A's B association, but only save it later:
a = A.find(...)
# a.bs == [B<...>, B<...>]
a.bs = []
#or
a.bs = [B.new, B.new]
# do some validation stuff on `a` and `a.bs`
So there might be some case where I will call a.save later or maybe not. In the case I don't call a.save I would like that a.bs stay to its original value, but as soon as I call a.bs = [], the old associations is destroyed and now A.find(...).bs == []. Is there any simple way to set a record association without persisting it in the database right away? I looked at Rails source and didn't find anything that could help me there.
Thanks!
Edit:
I should add that this is for an existing application and there are some architecture constraint that doesn't allow us to use the the regular ActiveRecord updating and validation tools. The way it works we have a set of Updater class that take params and assign the checkout object the value from params. There are then a set of Validater class that validate the checkout object for each given params. Fianlly, if everything is good, we save the model.
In this case, I'm looking to update the association in an Updater, validate them in the Validator and finally, persist it if everything check out.
In summary, this would look like:
def update
apply_updaters(object, params)
# do some stuff with the updated object
if(validate(object))
object.save(validate: false)
end
Since there are a lot of stuff going on between appy_updaters and object.save, Transaction are not really an option. This is why I'm really looking to update the association without persisting right away, just like we would do with any other attribute.
So far, the closest solution I've got to is rewriting the association cache (target). This look something like:
# In the updater
A.bs.target.clear
params[:bs].each{|b| A.bs.build(b)}
# A.bs now contains the parameters object without doing any update in the database
When come the time to save, we need to persist cache:
new_object = A.bs.target
A.bs(true).replace(new_object)
This work, but this feel kind of hack-ish and can easily break or have some undesired side-effect. An alternative I'm thinking about is to add a method A#new_bs= that cache the assigned object and A#bs that return the cached object if available.
Good question.
I can advice to use attributes assignment instead of collection manipulation. All validations will be performed as regular - after save or another 'persistent' method. You can write your own method (in model or in separated validator) which will validate collection.
You can delete and add elements to collection through attributes - deletion is performed by additional attribute _destroy which may be 'true' or 'false' (http://api.rubyonrails.org/classes/ActiveRecord/NestedAttributes/ClassMethods.html), addition - through setting up parent model to accept attributes.
As example set up model A:
class A < ActiveRecord::Base
has_many :b
accepts_nested_attributes_for :b, :allow_destroy => true
validates_associated :b # to validate each element
validate :b_is_correct # to validate whole collection
def b_is_correct
self.bs.each { |b| ... } # validate collection
end
end
In controller use plain attributes for model updating (e.g update!(a_aparams)). These methods will behave like flat attribute updating. And don't forget to permit attributes for nested collection.
class AController < ApplicationController
def update
#a = A.find(...)
#a.update(a_attributes) # triggers validation, if error occurs - no changes will be persisted and a.errors will be populated
end
def a_attributes
params.require(:a).permit([:attr_of_a, :b_attributes => [:attr_of_b, :_destroy]])
end
end
On form we used gem nested_form (https://github.com/ryanb/nested_form), I recommend it. But on server side this approach uses attribute _destroy as mentioned before.
I finally found out about the mark_for_destruction method. My final solution therefor look like:
a.bs.each(&:mark_for_destruction)
params[:bs].each{|b| a.bs.build(b)}
And then I can filter out the marked_for_destruction? entry in the following processing and validation.
Thanks #AlkH that made me look into how accepts_nested_attributes_for was working and handling delayed destruction of association.
I have two tables:
stores
raw_stores_data
The raw_stores_data is received from a third party daily.
I'd update certain fields of the stores model if those fields have been modified for that record in raw_stores_data.
Currently I have a bunch of conditional statements that check each of those fields. Is there any better way to code this?
new_data = raw_stores_data.all.select do |item|
item.store_id.present?
end
new_data.each do |item|
if item.field1 != item.stores.field1
...
...
...
# update record with hash of fields to update created above
end
You could add an association and special mutators to the 'raw' model that know how manipulate the 'stores' object. This serves to keep the model code in the model. Thin controller, comprehensive models, etc.
class Store < ActiveRecord::Base
has_one :raw_stores_data
end
class RawStoresData < ActiveRecord::Base
belongs_to :store
def field1=(value)
store.field1 = value
store.save!
field1 = value
end
end
I'm hand waving at some of the details, and you might want to reverse the direction of the association or make it go both directions.
EDIT:
You would use this as such:
raw_data = RawStoreData.find(param[:id]) # or new or however you get this object
raw_data.field1 = param[:field1]
The act of assigning will use the 'field1=' method, and make the change to the associated store object. If you're worried about saving unnecessarily, you could conditionalize in that method to only save if the value changed.
I hope this is clearer.
I have the following one to many associations. Document has many Sections and Section has many Items.
class Document < ActiveRecord::Base
has_many :document_sections, :dependent => :destroy, :autosave => true
has_many :document_items, :through => :document_sections
end
class DocumentSection < ActiveRecord::Base
belongs_to :document
has_many :document_items, :dependent => :destroy, :autosave => true
end
class DocumentItem < ActiveRecord::Base
belongs_to :document_section
end
Here is the params hash:
-
Parameters: {"commit"=>"Submit Document", "authenticity_token"=>"4nx2B0pJkvavDmkEQ305ABHy+h5R4bZTrmHUv1setnc=", "id"=>"10184", "document"=>{"section"=>{"10254"=>{"seqnum"=>"3", "item"=>{"10259"=>{"comments"=>"tada"}}}}, "comment"=>"blah"}}
I have the following update method...
# PUT /documents/1
# PUT /documents/1.xml
def update
#document = Document.find(params[:id])
# This is header comment
#document.comment = params[:document][:comment]
params[:document][:section].each do |k,v|
document_section = #document.document_sections.find_by_id(k)
if document_section
v[:item].each do |key, value|
document_item = document_section.feedback_items.find_by_id(key)
if document_item
# This is item comments
document_item.comments = value[:comments]
end
end
end
end
#document.save
end
When I save the document it only updates the document header comments. It does not save the document_item comments. Shouldn't the autosave option also update the associations.
In the log only the following DML is registered:
UPDATE documents SET updated_at = TO_DATE('2010-03-09 08:35:59','YYYY-MM-DD HH24:MI:SS'), comment = 'blah' WHERE id = 10184
How do I save the associations by saving the document.
I think I see what the problem is. I'm pretty sure that you cannot do the following:
# Triggers a database call
document_section = #document.document_sections.find_by_id(k)
And expect ActiveRecord to keep the association for autosaves. Instead, you should save the loaded records individually. Which of course would not be atomic.
I believe for autosave to work like you are thinking, you want to do something like this:
# untested...
#document.document_sections.collect { |s| s.id == k }.foo = "bar"
Notice that here I'm actually modifying a fake param foo in the array, instead of calling find_by_id, which will re-query the database and return a new object.
A third option you have is that you could of course, do what you had originally planned, but handle all the transactions yourself, or use nested transactions, etc, to get the atmoic saves. This would be necessary if your data was too large for array manipulation to work since autosave by it's natures triggers a load of all associated data into memory.
It all depends on your application.
Some clarifications on the underlying problem:
If you run the find_by_id method, you are asking ActiveRecord to return to you a new set of objects that match that query. The fact that you executed that method from an instance (document_sections) is really just another way of saying:
DocumentSection.find_by_id(k)
Calling it from an object instance I think is just some syntactic niceness that rails is adding on the top of things, but in my mind it doesn't make a lot of sense; I think it could be handy in some application, I'm not sure.
On the other side, collect is a Ruby Array method that offers a way to "slice" an array using a block. Basically a fancy foreach loop. :) By interacting with the document_sections array directly, you are changing the same objects already loaded into the containing object (#document), which will then be committed when you save with the special autosave flag set.
HTH! Glad you are back and running. :)
I need to cache (and expire) all the models in a table.
For example, if i have a model named Currency, i only have less than 10 possible currencies. Therefore, it would be nice to have:
class Currency < ActiveRecord::Base
cache_all(:expire_in => 10.minutes)
end
so that
Currency.all
Currency.find_by_name("USD")
should not hit the DB.
What do you think it could be a good approach?
Also, if you believe it would be better to use a model that is not backed up by a DB, please comment on this. Please notice that i would like to have a AR-style association.
Since the data set is so small, probably the best thing is to cache it in local memory. There are a couple ways to do this, one is to use Memoization like I show here. However that isn't the most efficient because it will store the all method and find_by_name method in separate caches even though they are the same object.
An alternative is to redefine the methods to cache the objects manually. Something like this.
class Currency < ActiveRecord::Base
def self.all
#all_cache ||= super.map(&:freeze) # freeze so you don't modify the cached objects
end
def self.find_by_name(name)
all.detect { |c| c.name.to_s.downcase == name.to_s.downcase }
end
def self.flush_all_cache
#all_cache = nil
end
end
There may be a plugin to handle this for you, but I haven't looked into that at all.