I have a situation like, I have a product table which has a JSON field, I have to call a method to update that. So there are two approaches
Product.all.each do |pr|
pr.update_column(:boolean_tree, pr.change_boolean_tree)
end
There is nothing wrong with this apart from performance . To improve the performance we can run raw SQL . Which will definitely works fine, but not in a rails way.
I was thinking is it possible to use update_all for this kind of situation. Any idea ?
As far as I know, you won't be able to use update_all unless you are trying to update all your records with the same content. When there are a lot of Products in the database the instruction Product.all would load all the records in memory before starting the update.
I would recommend using find_in_batches method from Activerecord (see doc) to load the products in batches.
Related
I have Rails 3 project that I updated to 5, that uses Mongoind instead of Active Record. I'm trying to implement fragment caching.
My understanding is, that with ActiveRecord, even if I have something like
#films = Film.all in a controller, but never use #films in the view, the query won't actually run. Hence, if I cache #films in the view, on the second request, it'll be read from cache, and query isn't going to run.
This is how I think ActiveRecord works.
Now to Mongoid. I cache variable in the view, but even though it's being read from cache, the query still hits db.
My question is, is there a way to avoid that with Mongoid?
Or am I missing something in terms of caching?
I tried searching online, but there isn't much on Rails Mongoid caching, not to mention, anything written after 2012.
According to the Rails docs here and here, using update_all does not do the following -
It skips validations
It does not update the updated_at field
It silently ignores the :limit and :order methods
I'm trying to go through my code base and remove instances of update_all, particularly because of the first point.
Is there a way to still have the convenience of update_all and still run validations? I understand that I can loop through each record and save it, but that's not only messier visually but also more more inefficient because it executes N SQL statements instead of 1
# before
User.where(status: "active").update_all(status: "inactive")
# after
User.where(status: "active").each { |u| u.update(status: "inactive") }
Thanks!
Edit: I'm using Rails 4.2
Unfortunately update_all is way faster because it doesn't instantiate an active record object for each record and instead deals directly with the database. In your case, since you need validations and callbacks, you'll need to instantiate the objects and so you're best bet is iterating in batches of 1000 and performing the update as originally shown. Such as:
User.where(status: "active").find_each { |u| u.update(status: "inactive") }
The find_each method only loads 1000 objects at a time thus not overloading the garbage collector. If you have bulk records in the hundreds of thousands of rows I'd consider going back to update_all or moving the updating to a background task since it can easily cause a timeout when deployed.
I'm working on an existing Rails 2 site with a large codebase that recently updated to Ruby 1.9.2 and the mysql2 gem. I've noticed that this setup allows for non-blocking database queries; you can do client.query(sql, :async => true) and then later call client.async_result, which blocks until the query completes.
It seems to me that we could get a performance boost by having all ActiveRecord queries that return a collection decline to block until a method is called on the collection. e.g.
#widgets = Widget.find(:all, :conditions=> conditions) #sends the query
do_some_stuff_that_doesn't_require_widgets
#widgets.each do #if the query hasn't completed yet, wait until it does, then populate #widgets with the result. Iterate through #widgets
...
This could be done by monkey-patching Base::find and its related methods to create a new database client, send the query asynchronously, and then immediately return a Delegator or other proxy object that will, when any method is called on it, call client.async_result, instantiate the result using ActiveRecord, and delegate the method to that. ActiveRecord association proxy objects already work similarly to implement ORM.
I can't find anybody who's done this, though, and it doesn't seem to be an option in any version of Rails. I've tried implementing it myself and it works in console (as long as I append ; 1 to the line calling everything so that to_s doesn't get called on the result). But it seems to be colliding with all sorts of other magic and creating various problems.
So, is this a bad idea for some reason I haven't thought of? If not, why isn't it the way ActiveRecord already works? Is there a clean way to make it happen?
I suspect that that .async_result method isn't available for all database drivers; if not, it's not something that could be merged into generic ActiveRecord calls.
A more portable way to help performance when looping over a large recordset would be to use find_each or find_in_batches. I think they'll work in rails 2.3 as well as rails 3.x. http://guides.rubyonrails.org/active_record_querying.html#retrieving-multiple-objects-in-batches
In a stats part of a Rails app, I have some custom SQL calls that are called with ActiveRecord::Base.execute() from the model code. They return various aggregates.
Some of these (identical) queries are run in a loop in the controller, and it seems that they aren't cached by the ActiveRecord query cache.
Is there any way to cache custom SQL queries within a single request?
Not sure if AR supports query caching for #execute, you might want to dig in the documentation.
Anyway, what you can do, is to use memoization, which means that you'll keep the results manually until the current request is over.
do something like this in your model:
def repeating_method_with_execute
#rs ||= ActiveRecord::Base.execute(...)
end
This will basically run the query only on the first time and then save the response to #rs until the entire request is over.
If I am not wrong, Rails 2.x already has a macro named memoization on ActiveRecord that does all that automatically
hope it helps
I have a number of custom find_by_sql queries in Rails. I would like to use eager loading with them but there doesn't seem to be a good way to do this.
I have seen the eager_custom.rb file floating around and it doesn't seem to work with Rails now. It appear Rails does eager loading differently now, using 2 queries (the regular query plus a query where the 'id IN' the ids from the first query), instead of the single join query used in the past.
My question is if I do a custom SQL query, then do 'id IN' query, is there a way to add back associated objects into the initial query results?
For example I have topics loaded with find_by_sql, then I find topic images where the topic id is in the topics ids, is there a way to add the images manually back to the topics?
Thanks
As you noticed, in Rails 2.1 a new kind of eager/pre-loading was introduced which uses multiple queries with id IN (...). This method is usually faster, especially when there are multiple associations being pre-loaded. You can use this functionality manually with find_by_sql by using the preload_associations class method inherited from ActiveRecord (not recommended). For example:
class Person
def self.find_a_special_group
people = find_by_sql("...")
preload_associations(people, [:jobs, :addresses])
return people
end
end
The preload_associations method is protected, so you must call it from within the class, and it takes (1) an array of objects, (2) an array, hash, or symbol of associations (same format as find's :include option), and (3) an options hash. See the documentation for the ActiveRecord::AssociationPreload::ClassMethods module for more details.
However, having said all of that, this technique is certainly undesirable as the Rails documentation discourages programmers from using preload_associations directly. Are you sure you have to use find_by_sql? Are you sure you know all of the options find takes? (:select, :from, :joins, :group, :having, etc) I'm not saying you don't need find_by_sql, but it might be worth a few minutes to make sure.