I have a Rails app that I was able to speed up significantly using ARel "includes" e.g. (contrived)
class User < ActiveRecord::Base
has_many :posts
scope :eager, includes(:posts => [:rating, :author, {:tags => [:day, {:foo => :bar}]}] )
end
Calling
#posts = current_user.posts.eager
reduces that page load hugely and reduces the number queries by a very large factor. Rails does this by first selecting the posts in one query
select * from posts where ...
and then selecting all the comments for all those posts in one query instead of one query per comment:
select * from comments where post_id in (6,7,8,9,10,...)
Is there an equivalent in grails? I am familiar with criteria and named queries where I could write a query with a lot of joins but what I want is for Grails to produce a few queries with "IN" operator.
I found some references to this problem: Eager and Lazy Fetching and fetchMode.
Related
I have the following ActiveRecord Models
class Publication < ActiveRecord::Base
attr_accessible :id, :pname
has_many :advertisements
end
class Vendor < ActiveRecord::Base
attr_accessible :id, :vname
has_many :advertisements
end
class Advertisement < ActiveRecord::Base
attr_accessible :id, :vendor_id, :publication_id, :prose, :aname
belongs_to :vendor
belongs_to :publication
end
The tables for these have the same fields as their accessible attributes.
I would like to be able to sort on the publication name, ad name, or vendor name, ascending or descending.
I also have a controller for the advertisements, where I want to display a list of ads. The list displays the name of the ad (aname), prose of the ad (prose), the name of the vendor (vname), and the name of the publication (pname).
The SQL query for ordering by publication name would look something like:
SELECT ads.aname AS aname, ads.id, ads.prose, ven.vname AS vname, pub.pname AS pname
FROM advertisements AS ads
INNER JOIN publications AS pub ON ads.publication_id = pub.id
INNER JOIN vendors AS ven ON ads.vendor_id = ven.id
ORDER BY <sort_column> <sort_order>
Where sort_column could be one of "pname", "aname", or "vname", and sort_order could be one of "ASC" or "DESC", and both would come as parameters from the web form along with the pagination page number.
The controller index code looks like this:
class AdvertisementsController < ApplicationController
def index
sort_column = params[:sort_column]
sort_order = params[:sort_order]
#ads = Advertisement.join( somehow join tables)
.where(some condition).where(some other condition)
.order("#{sort_column} #{sort_order}") ### I don't know what to do here
.paginate(page: params[:page], per_page: 10) #from will_paginate
end
# other controller methods.......
end
The index view table snippet (written in SLIM) looks like this:
tr
- #ads.each do |ad|
td = ad.id
td = ad.aname
td = ad.pname
td = ad.vname
I am aware that I could use AREL to do this, but I have been mucking around with AREL in the Rails console trying to generate and execute this query with pagination, and reading tutorials on the web and I can't figure out how to get this query in AREL, with sorting on joined fields, and with the ability to use a will_paginate Ruby query clause to paginate the query.
How does one use AREL, or even ActiveRecord to do this? I appreciate any help I can get.
You can accomplish what you want with vanilla ActiveRecord methods, without Arel. What you have is pretty close, this might help you get there.
# whitelist incoming params
sort_column = %w(pname aname vname).include?(params[:sort_column]) ? params[:sort_column] : "pname"
sort_order = %w(asc desc).include?(params[:sort_order]) ? params[:sort_order] : "desc"
#ads = Advertisement.select("advertisements.*, vendors.vname, publications.pname").
joins(:publication, :vendor).
where(some condition).
where(some other condition).
order("#{sort_column} #{sort_order}").
page(params[:page]).per_page(10)
You can have the solution work with both Arel and ActiveRecord. I would suggest you stick to ActiveRecord as much as you can unless you cant do it with AR.
Arel is great, but lately I have seen that in my code base, it reduces overall readability, esp if you mix it with AR or use too much of it.
Also couple of other suggestions:
On the same query about try using "includes" instead of using "joins". You might that its easier than having to add the select clause. I use includes for outerjoins, but for a more detailed comparison, google "includes vs joins", it is very interesting.
In complete opposite direction of my first suggestion, in case you queries are going to get complex, I highly recommend using https://github.com/activerecord-hackery/ransack or https://github.com/activerecord-hackery/squeel for your use case.
Especially if you not doing the above for just learning purposes.
I'm trying to figure out the count of certain types of articles. I have a very inefficient query:
Article.where(status: 'Finished').select{|x| x.tags & Article::EXPERT_TAGS}.size
In my quest to be a better programmer, I'm wondering how to make this a faster query. tags is an array of strings in Article, and Article::EXPERT_TAGS is another array of strings. I want to find the intersection of the arrays, and get the resulting record count.
EDIT: Article::EXPERT_TAGS and article.tags are defined as Mongo arrays. These arrays hold strings, and I believe they are serialized strings. For example: Article.first.tags = ["Guest Writer", "News Article", "Press Release"]. Unfortunately this is not set up properly as a separate table of Tags.
2nd EDIT: I'm using MongoDB, so actually it is using a MongoWrapper like MongoMapper or mongoid, not ActiveRecord. This is an error on my part, sorry! Because of this error, it screws up the analysis of this question. Thanks PinnyM for pointing out the error!
Since you are using MongoDB, you could also consider a MongoDB-specific solution (aggregation framework) for the array intersection, so that you could get the database to do all the work before fetching the final result.
See this SO thread How to check if an array field is a part of another array in MongoDB?
Assuming that the entire tags list is stored in a single database field and that you want to keep it that way, I don't see much scope of improvement, since you need to get all the data into Ruby for processing.
However, there is one problem with your database query
Article.where(status: 'Finished')
# This translates into the following query
SELECT * FROM articles WHERE status = 'Finished'
Essentially, you are fetching all the columns whereas you only need the tags column for your process. So, you can use pluck like this:
Article.where(status: 'Finished').pluck(:tags)
# This translates into the following query
SELECT tags FROM articles WHERE status = 'Finished'
I answered a question regarding general intersection like queries in ActiveRecord here.
Extracted below:
The following is a general approach I use for constructing intersection like queries in ActiveRecord:
class Service < ActiveRecord::Base
belongs_to :person
def self.with_types(*types)
where(service_type: types)
end
end
class City < ActiveRecord::Base
has_and_belongs_to_many :services
has_many :people, inverse_of: :city
end
class Person < ActiveRecord::Base
belongs_to :city, inverse_of: :people
def self.with_cities(cities)
where(city_id: cities)
end
# intersection like query
def self.with_all_service_types(*types)
types.map { |t|
joins(:services).merge(Service.with_types t).select(:id)
}.reduce(scoped) { |scope, subquery|
scope.where(id: subquery)
}
end
end
Person.with_all_service_types(1, 2)
Person.with_all_service_types(1, 2).with_cities(City.where(name: 'Gold Coast'))
It will generate SQL of the form:
SELECT "people".*
FROM "people"
WHERE "people"."id" in (SELECT "people"."id" FROM ...)
AND "people"."id" in (SELECT ...)
AND ...
You can create as many subqueries as required with the above approach based on any conditions/joins etc so long as each subquery returns the id of a matching person in its result set.
Each subquery result set will be AND'ed together thus restricting the matching set to the intersection of all of the subqueries.
I'm working on a Rails 3.2 app that uses Datamapper as its ORM. I'm looking for a way to sort a result set by an attribute of the associated model. Specifically I have the following models:
class Vehicle
include DataMapper::Resource
belongs_to :user
end
class User
include DataMapper::Resource
has n, :vehicles
end
Now I want to be able to query the vehicles and sort them by the name of the driver. I tried the following but neither seems to work with Datamapper:
> Vehicle.all( :order => 'users.name' )
ArgumentError: +options[:order]+ entry "users.name" does not map to a property in Vehicle
> Vehicle.all( :order => { :users => 'name' } )
ArgumentError: +options[:order]+ entry [:users, "name"] of an unsupported object Array
Right now I'm using Ruby to sort the result set post-query but obviously that's not helping performance any, also it stops me from further chaining on other scopes.
I spent some more time digging around and finally turned up an old blog which has a solution to this problem. It involves manually building the ordering query in DataMapper.
From: http://rhnh.net/2010/12/01/ordering-by-a-field-in-a-join-model-with-datamapper
def self.ordered_by_vehicle_name direction = :asc
order = DataMapper::Query::Direction.new(vehicle.name, direction)
query = all.query
query.instance_variable_set("#order", [order])
query.instance_variable_set("#links", [relationships['vehicle'].inverse])
all(query)
end
This will let you order by association and still chain on other scopes, e.g.:
User.ordered_by_vehicle_name(:desc).all( :name => 'foo' )
It's a bit hacky but it does what I wanted it to do at least ;)
Note: I'm not familiar with DataMapper and my answer might not be within the standards and recommendations of using DataMapper, but it should hopefully give you the result you're looking for.
I've been looking through various Google searches and the DataMapper documentation and I haven't found a way to "order by assocation attribute". The only solution I have thought of is "raw" SQL.
The query would look like this.
SELECT vehicles.* FROM vehicles
LEFT JOIN users ON vehicles.user_id = users.id
ORDER BY users.name
Unfortunately, from my understanding, when you directly query the database you won't get the Vehicle object, but the data from the database.
From the documentation: http://datamapper.org/docs/find.html. It's near the bottom titled "Talking directly to your data-store"
Note that this will not return Zoo objects, rather the raw data straight from the database
Vehicle.joins(:user).order('users.name').all
or in Rails 2.3,
Vehicle.all(:joins => "inner join users on vehicles.user_id = user.id", :order => 'users.name')
I have a very simple Rails app with a very simple relational database: Category has many Samples. I'd simply like to load the categories that have X number of samples.
In plain SQL I would do something like this:
SELECT
categories.*
FROM
categories
JOIN
(SELECT
category_id, COUNT(*) as sample_count
FROM
samples
GROUP BY
category_id
) AS subselect
ON
categories.id=subselect.category_id
WHERE
subselect.sample_count = X; -- where X is whatever
That works just fine, by the way, but it's not terribly Rails-like to use raw SQL. And obviously I'd like to get those categories as model instances, so:
How would I go about re-writing something like that to an ActiveRecord or Arel query? Is it even feasible, or should I go with the plain SQL? Is there perhaps an altogether simpler way of doing it?
A possible nice way would be to use counter_cache, as described on this page: http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html
Add a column named samples_count to your Category model:
add_column :categories, :samples_count, :integer
In your Sample model update belongs_to as follows:
belongs_to :category , :counter_cache => true
You can now use the count as a condition, for example:
Category.where(:samples_count => 7)
I am new to rails and would appreciate some help optimizing my database usage.
Is there a way to load two models associated with each other with one DB query?
I have two models Person and Image:
class Person < ActiveRecord::Base
has_many :images
end
class Image < ActiveRecord::Base
belongs_to :person
end
I would like to load a set of people and their associated images with a single trip to the DB using a join command. For instance, in SQL, I can load all the data I need with the following query:
select * from people join images on people.id = images.person_id where people.id in (2, 3) order by timestamp;
So I was hoping that this rails snippet would do what I need:
>> people_and_images = Person.find(:all, :conditions => ["people.id in (?)", "2, 3"], :joins => :images, :order => :timestamp)
This code executes the SQL statement I am expecting and loads the instances of Person I need. However, I see that accessing a a Person's images leads to an additional SQL query.
>> people_and_images[0].images
Image Load (0.004889) SELECT * FROM `images` WHERE (`images`.person_id = 2)
Using the :include option in the call to find() does load both models, however it will cost me an additional SELECT by executing it along with the JOIN.
I would like to do in Rails what I can do in SQL which is to grab all the data I need with one query.
Any help would be greatly appreciated. Thanks!
You want to use :include like
Person.find(:all, :conditions => ["people.id in (?)", "2, 3"], :include => :images, :order => :timestamp)
Check out the find documentation for more details
You can use :include for eager loading of associations and indeed it does call exactly 2 queries instead of one as with the case of :joins; the first query is to load the primary model and the second is to load the associated models. This is especially helpful in solving the infamous N+1 query problem, which you will face if you doesn't use :include, and :joins doesn't eager-load the associations.
the difference between using :joins and :include is 1 query more for :include, but the difference of not using :include will be a whole lot more.
you can check it up here: http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations