I have a model Foo with several belongs_to associations; I'll refer to them here as Bar and Baz. So the model would look like this:
class Foo
belongs_to :bar
belongs_to :baz
def do_stuff_with_bar_and_baz
bar.do_stuff(baz)
end
end
We noticed that do_stuff_with_bar_and_baz was unusually slow (~4 seconds), even though the underlying MySQL statements were very fast (~0.5ms). I benchmarked the bar and baz calls, and discovered that they took ~2.3s and ~221ms respectively... just to go through the Rails association code.
I then put in the following methods:
class Foo
belongs_to :bar
belongs_to :baz
def bar
Bar.find(self.bar_id)
end
def baz
Baz.find(self.baz_id)
end
def do_stuff_with_bar_and_baz
bar.do_stuff(baz)
end
end
This bypasses the ActiveRecord association code and loads the associated records directly. With this code, the time to load the Bar and Baz in do_stuff_with_bar_and_baz dropped to 754ms and 5ms respectively.
This is disheartening. The standard Rails associations appear to be horrendously inefficient, but I really don't want to have to replace all of them (that defeats a the purpose of a significant amount of ActiveRecord).
So, I'm looking for alternatives:
Is there something that I'm potentially doing wrong that's slowing things down? (The real code is obviously more complicated that this. However, the belongs_to is accurate; there's no additional options on the real code).
Have other people encountered this?
How have they dealt with it?
Seems like you have different queries and problem is not in rails, but in DB.
Maybe you have other conditions and you haven't right indexes for they.
Related
I'm able to get the behavior I'd like if I create a Foo class, some STI classes that inherit it (Foo1, Foo2, etc.) and then to add different has_many :bars associations to each class. For example:
class Foo < ApplicationRecord
has_many :bars
end
class Foo1 < Foo
has_many :bars, -> { where(user_id: 1) }
end
I want to be able to call bars on an object and get a different association behavior depending on the state of the object. This works, but is there a way to do it without setting up STI?
I tried doing everything inside foo.rb but I seem to be loading my first has_many :bars definition even if I do something like this:
has_many :bars ... if some_method_returning_boolean?
has_many :bars ... if some_other_method_returning_boolean?
And even if this did work, it seems kind of clunky.
I also considered scopes, but as far as I understand scopes then I'd have to call foo.bars.something and foo.bars.something_else instead of relying on the state of the object foo to give me foo.bars differently. Also, it looks like scopes only cover part of a has_many definition but can't modify arguments like :source, :foreign_key, but I may be wrong.
Is there another way?
STI is the correct way of doing this. It's generally understood that any particular model class should have well-defined and consistent relationships with other records.
If you have relationships that only apply to certain kinds of records that's exactly what STI is for: Create a subclass that defines these relationships.
You see this all the time. An example would be that both "Parent" and "Student" are of base type "Person", but that the students have a belongs_to: parent association and the parents have a has_many: children relationship.
Of course this presumes parents are unlikely to become students, and students parents, an assumption that may not be correct. Whatever assumptions you make, I hope there's some serious thinking about if these restrictions are relevant or just overly paranoid.
It's generally bad form to switch types on a record even though it can technically be done. You could use this to adjust how a record's relationships are defined if you think this is strictly necessary.
Generally I'd advise you to stick with a consistent relationship structure. Those records which should not be related to anything don't need those methods physically removed, they can be there while not doing anything useful. They come along for free anyway.
Maybe you will be satisfacted by overwrite your bars method in this way ?
This super just return usual result from something_foo.bars.
class Foo
def bars
super.where(something: something_value) if your_condition
super.where(something_else: something_value) if other_condition
end
end
I've got something like the following:
module Bar < ActiveRecord::Base
belongs_to :foo
...
module Foo < ActiveRecord::Base
has_many :bars, dependent: :destroy
def build_bars
1000.times do |i|
bars.build(num: i)
end
end
def create_default_bars!
build_bars
save
end
Notice that Foo#build_bars is cheap. Even though it loops 1000 times, it takes very little time. But then, once you hit save, suddenly ActiveRecord decides to perform 1000 inserts, which is incredibly slow.
How can I write a custom save_the_bars method such that it performs a single bulk INSERT query for all of the bars I have built up (I remind you, the 1000 builds are apparently very cheap), but which functions as a drop-in replacement for save in this example?
I'm expecting an answer along the lines of recommendation #3 of this blog post:
https://www.coffeepowered.net/2009/01/23/mass-inserting-data-in-rails-without-killing-your-performance/
But since my example uses build and relies on some slightly nontrivial rails magic, it isn't immediately obvious how to translate it over. Bonus points for benchmarks!
I would try the activerecord-import gem.
def save_the_bars(bars)
Bars.import bars
end
This call to import does whatever is most efficient for the underlying database adapter. Pretty slick, eh?
For bonus points: Benchmarks
Question-asker here, hijacking this answer with details on what I did following the above suggestion:
def build_bars
built_bars = []
1000.times do |i|
built_bars << bars.build(num: i)
end
built_bars
end
def create_default_bars
save
Bar.insert built_bars, validate: false
reload
end
This gave a nice speedup for relatively little effort. I still suspect that more speedup could be had (by leveraging the nuances of insert) but I'm satisfied with this for now. In my use case, it was safe to turn off validation, since the method generating all the Bars is guaranteed to generate valid ones.
I have an organizational hierarchy represented via awesome nested set on a Node model. Great, works great, the updates are expensive but the finds are super efficient.
Each Node model has_many on other models, let's call them Foo and Bar.
class Node < ActiveRecord::Base
acts_as_nested_set
has_many :foos
has_many :bars
end
class Foo < ActiveRecord::Base
belongs_to :node
end
Frequently, I want to find all of the foos or bars for a given subtree, looking down from a current node. Naively, I could do:
#foos = #node.self_and_descendants.collect(&:foos).compact
I could even use an ActiveRecord .includes(:foos) to avoid N+1 queries. What I really want is just to ask for #node.all_foos so I implement something like this:
class Node < ActiveRecord::Base
def all_foos
Foo.where(node_id: self_and_descendants.pluck(:id))
end
# More efficient?
def all_foos_alternately
Foo.where(node_id: self_and_descendants.includes(:foos).pluck(:id))
end
end
But, let's say that I want to “collect” more than just foos and bars, let's say that I have half a dozen or a dozen of this models. Now I'm littering my Node class or some lib with a bunch of all_* methods.
Should I be defining class methods on the foos and bars that accept a node as an argument and return all of the foos/bars for that subtree? But then the foos and bars need to understand/be aware of node.self_and_descendants.
Or the Foo class method could accept a collection of nodes, not needing to be aware of the nested set methods, but then I lose the easy interface via node.all_foos or the like.
What's the pattern or method I'm missing here? I've played with implementing a catch-all for node.all_* via method_missing but don't like the performance hit. What I'm trying to execute here is, at its core, a database lookup and, as such, it should be efficient and elegant.
Thank you to #adamsanderson for pointing me in the right direction with regards to joins and merge. Merge allows you to filter a joined association via another scope or relation. Brilliant!
def all_foos
Foo.joins(:node).merge(self_and_descendants)
end
I verified tests on one of my all_* methods in my application, rewrote the method using this joins.merge technique, and reran my tests. All green!
This answer does not address the design issue of how to apply this method across many different has_many relations but I will leave that as an exercise for the reader (me).
See also, with regards to merge:
http://blog.mitchcrowe.com/blog/2012/04/14/10-most-underused-activerecord-relation-methods
I am going to propose a solution that I don't think is very good in order to attempt to kick off discussion...
class Node < ActiveRecord::Base
acts_as_nested_set
has_many :foos
has_many :tree_foos, :through => :children
def all_foos
# Shoot, how do I write this so I still get a relation back?
foos + tree_foos
# Nope, how about...
Foo.where(id: foo_ids + tree_foo_ids)
end
end
Blech. I don't like that. And I'll still have to repeat this code across all the has_many model associations.
I'm trying to get associations in FactoryGirl to work, and they just ... don't. I've basically got this:
class Foo
include Mongoid::Document
belongs_to :bar
end
class Bar
include Mongoid::Document
has_many :foos
end
FactoryGirl.define do
factory :foo, class => Foo do
bar
end
factory :bar, class => Bar do
end
end
At least so the docs lead me to believe... But then in my test, I have
a_foo=FactoryGirl.create :foo
a_foo.bar # Hooray! It's an associated object
Foo.where( _id: a_foo._id ).includes( :bar ).first.bar # This is nil!
Why is the associated value nil on the last line? I need it not to be, because the actual code being tested does this same thing, and it has a right to expect it to work... What am I missing about why this doesn't work right? Something to do with eager loading, perhaps?
Your code actually works for me with FactoryGirl 4.2.0, Mongoid 3.0.9. But I've run into similar issues when I've been running mongoid with the identitymap disabled (which is default behavior). Without the identitymap, you can have two different ruby objects representing the same document in the database, getting out of sync with each other. So, if you have autosave off, for example, this could cause the problem you're seeing.
Try pasting your simplified code into the rails console yourself -- if it works, then you probably changed something significant in pairing down your real code. (Sorry to point out the obvious, but the fact that you have a syntax error in your factory code makes me think you didn't actually test your sample code.)
I am working on a very large Rails application. We initially did not use much inheritance, but we have had some eye opening experiences from a consultant and are looking to refactor some of our models.
We have the following pattern a lot in our application:
class Project < ActiveRecord::Base
has_many :graph_settings
end
class GraphType < ActiveRecord::Base
has_many :graph_settings
#graph type specific settings (units, labels, etc) stored in DB and very infrequently updated.
end
class GraphSetting < ActiveRecord::Base
belongs_to :graph_type
belongs_to :project
# Project implementation of graph type specific settings (y_min, y_max) also stored in db.
end
This also results in a ton of conditionals in views, helpers and in the GraphSetting model itself. None of this is good.
A simple refactor where we get rid of GraphType in favor of using a structure more like this:
class Graph < ActiveRecord::Base
belongs_to :project
# Generic methods and settings
end
class SpecificGraph < Graph
# Default methods and settings hard coded
# Project implementation specific details stored in db.
end
Now this makes perfect sense to me, eases testing, removes conditionals, and makes later internationalization easier. However we only have 15 to 30 graphs.
We have a very similar model (to complicated to use as an example) with close to probably 100 different 'types', and could potentially double that. They would all have relationships and methods they inheritated, some would need to override more methods then others. It seems like the perfect use, but that many just seems like a lot.
Is 200 STI classes to many? Is there another pattern we should look at?
Thanks for any wisdom and I will answer any questions.
If the differences are just in the behavior of the class, then I assume it shouldn't be a problem, and this is a good candidate for STI. (Mind you, I've never tried this with so many subclasses.)
But, if your 200 STI classes each have some unique attributes, you would need a lot of extra database columns in the master table which would be NULL, 99.5% of the time. This could be very inefficient.
To create something like "multiple table inheritance", what I've done before with success was to use a little metaprogramming to associate other tables for the details unique to each class:
class SpecificGraph < Graph
include SpecificGraphDetail::MTI
end
class SpecificGraphDetail < ActiveRecord::Base
module MTI
def self.included(base)
base.class_eval do
has_one :specific_graph_detail, :foreign_key => 'graph_id', :dependent => :destroy
delegate :extra_column, :extra_column=, :to => :specific_graph_detail
end
end
end
end
The delegation means you can access the associated detail fields as if they were directly on the model instead of going through the specific_graph_detail association, and for all intents and purposes it "looks" like these are just extra columns.
You have to trade off the situations where you need to join these extra detail tables against just having the extra columns in the master table. That will decide whether to use STI or a solution using associated tables, such as my solution above.