I have a rails app where I often need to generate xls files with several values.
To do so I load customers orders in my controller (valid is just a custom scope)
#orders = Order.valid.order('id DESC')
In my view, I want to count meals for each orders with different scopes (i removed all xml tags, which are not interesting here):
<% #orders.each do |order| %>
<%= order.meals.count %>
<%= order.meals.meat.count %>
<%= order.meals.fish.count %>
<%= order.meals.drink.count %>
<%= order.meals.dessert.count %>
<% end %>
My problem is that this generates a very large number of SQL requests.
I tried to preload meals like this:
#orders = Order.valid.order('id DESC').preload(:meals)
But count requests keep being generated
I also found an old gem: preload_count to do this, but it doesn't works with rails 4.
Is there a way to optimize my requests ?
EDIT
After many tries and help from Andrey Deineko, my request turned into this:
Order.includes(:meals).valid.order('id DESC').references(:meals)
Then i realized it may come from meal types which are in another table in my DB
Order.eager_load(meals: :type).valid.order('orders.id DESC').references(:meals)
But still, here are my db requests:
(0.3ms) SELECT COUNT(*) FROM "meals" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 [["order_id", 1044]]
(0.3ms) SELECT COUNT(*) FROM "meals" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 [["order_id", 1044]]
(0.4ms) SELECT COUNT(*) FROM "meals" INNER JOIN "types" ON "types"."id" = "meals"."type_id" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 AND "types"."name" = $2 [["order_id", 1044], ["name", "meat"]]
(0.4ms) SELECT COUNT(*) FROM "meals" INNER JOIN "types" ON "types"."id" = "meals"."type_id" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 AND "types"."name" = $2 [["order_id", 1044], ["name", "meat"]]
(0.4ms) SELECT COUNT(*) FROM "meals" INNER JOIN "types" ON "types"."id" = "meals"."type_id" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 AND "types"."name" = $2 [["order_id", 1044], ["name", "fish"]]
(0.4ms) SELECT COUNT(*) FROM "meals" INNER JOIN "types" ON "types"."id" = "meals"."type_id" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 AND "types"."name" = $2 [["order_id", 1044], ["name", "fish"]]
(0.3ms) SELECT COUNT(*) FROM "meals" INNER JOIN "types" ON "types"."id" = "meals"."type_id" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 AND "types"."name" = $2 [["order_id", 1044], ["name", "drink"]]
(0.3ms) SELECT COUNT(*) FROM "meals" INNER JOIN "types" ON "types"."id" = "meals"."type_id" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 AND "types"."name" = $2 [["order_id", 1044], ["name", "drink"]]
(0.3ms) SELECT COUNT(*) FROM "meals" INNER JOIN "types" ON "types"."id" = "meals"."type_id" INNER JOIN "meals_orders" ON "meals"."id" = "meals_orders"."meal_id" WHERE "meals_orders"."order_id" = $1 AND "types"."name" = $2 [["order_id", 1044], ["name", "dessert"]]
go with
#orders = Order.includes(:meals).valid.order('id DESC')
if you want to have only orders with meals (no orders, where meal is nil) go with:
#orders = Order.joins(:meals).valid.order('id DESC')
Have a read about AR querying.
If your models have a has_many belongs_to relationship you can use includes to eager load the associations. This is a good article on eager loading.
Related
I have two models: show_request and show. A show_Request belongs_to a show and a show has_many show_requests. On the show_request page in active_admin, I want to order show_requests by the show's created_at value. Here is my code so far:
ActiveAdmin.register ShowRequest do
controller do
def scoped_collection
end_of_association_chain.includes(:show)
#I also tried ShowRequest.includes(:show)
end
end
index do
column 'Show', sortable: "shows.created_at_asc" do |show_req|
link_to show_req.show.name, admin_show_path(show_req.show)
end
end
end
Here are the server logs:
Started GET "/admin/show_requests" for 127.0.0.1 at 2015-09-18 09:35:36 -0400
Processing by Admin::ShowRequestsController#index as HTML
AdminUser Load (0.3ms) SELECT "admin_users".* FROM "admin_users" WHERE "admin_users"."id" = 1 ORDER BY "admin_users"."id" ASC LIMIT 1
(1.2ms) SELECT COUNT(*) FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't')
(0.2ms) SELECT COUNT(*) FROM "show_requests"
(0.2ms) SELECT COUNT(*) FROM "show_requests" WHERE (not_going_to_show = 't' AND i_want_my_horse_to_compete = 'f')
(0.3ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') LIMIT 30 OFFSET 0) subquery_for_count
CACHE (0.0ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') LIMIT 30 OFFSET 0) subquery_for_count
CACHE (0.0ms) SELECT COUNT(*) FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't')
CACHE (0.0ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') LIMIT 30 OFFSET 0) subquery_for_count
ShowRequest Load (2.0ms) SELECT "show_requests".* FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') ORDER BY "show_requests"."id" desc LIMIT 30 OFFSET 0
Show Load (9.7ms) SELECT "shows".* FROM "shows" WHERE "shows"."id" IN (2, 1)
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 2]]
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 1]]
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 1]]
Show Load (0.2ms) SELECT "shows".* FROM "shows"
User Load (0.2ms) SELECT "users".* FROM "users"
This is not working. It is not affecting the order of the columns at all. How do I fix this?
Take a look at this part of the log:
ShowRequest Load (2.0ms) SELECT "show_requests".* FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') ORDER BY "show_requests"."id" desc LIMIT 30 OFFSET 0
Show Load (9.7ms) SELECT "shows".* FROM "shows" WHERE "shows"."id" IN (2, 1)
You can see that the ShowRequests and Shows are loaded in separate queries. sortable is not going to work here, because you can't order one table by a field of another without a join. The fix should be to tell ActiveRecord that you need to reference the shows table in your query:
controller do
def scoped_collection
super.includes(:show).references(:shows)
end
end
index do
column :show, sortable: 'shows.created_at'
end
references only works with includes, and forces Rails to perform a join when eager loading.
This is really nothing more than just looking for some explanation around why the joins clause is necessary for what I am doing. Both return everything I want properly, but the queries are different and one is less optimal:
TL;DR: What is Rails doing exactly when I combine the joins with the includes, I wouldn't have thought I needed the joins really. Thanks for anyone willing to actually read through this monster.
Note: object is a ToDoListsUser object, and this is all happening in an exporter for what it's worth. And ProgressItem is a joins table between User and ToDoItem.
With the joins clause (queries look good and as expected):
object.to_do_list.progress_items.where(user_id: object.user_id)
.joins(:to_do_item)
.includes(:to_do_item)
Server output:
ToDoListsUser Load (0.5ms) SELECT "to_do_lists_users".* FROM "to_do_lists_users" WHERE "to_do_lists_users"."user_id" = $1 ORDER BY "to_do_lists_users"."id" DESC OFFSET 0 [["user_id", 543]]
ToDoList Load (0.3ms) SELECT "to_do_lists".* FROM "to_do_lists" WHERE "to_do_lists"."id" = $1 LIMIT 1 [["id", 144]]
SQL (1.6ms) SELECT "progress_items"."id" AS t0_r0, "progress_items"."completed" AS t0_r1, "progress_items"."user_id" AS t0_r2, "progress_items"."to_do_item_id" AS t0_r3, "progress_items"."created_at" AS t0_r4, "progress_items"."updated_at" AS t0_r5, "progress_items"."hidden" AS t0_r6, "to_do_items_progress_items"."id" AS t1_r0, "to_do_items_progress_items"."to_do_list_id" AS t1_r1, "to_do_items_progress_items"."description" AS t1_r2, "to_do_items_progress_items"."date_due" AS t1_r3, "to_do_items_progress_items"."created_at" AS t1_r4, "to_do_items_progress_items"."updated_at" AS t1_r5, "to_do_items_progress_items"."name" AS t1_r6, "to_do_items_progress_items"."element_id" AS t1_r7, "to_do_items_progress_items"."linkable_id" AS t1_r8, "to_do_items_progress_items"."linkable_type" AS t1_r9, "to_do_items_progress_items"."action" AS t1_r10 FROM "progress_items" INNER JOIN "to_do_items" "to_do_items_progress_items" ON "to_do_items_progress_items"."id" = "progress_items"."to_do_item_id" INNER JOIN "to_do_items" ON "progress_items"."to_do_item_id" = "to_do_items"."id" WHERE "to_do_items"."to_do_list_id" = $1 AND "progress_items"."user_id" = $2 ORDER BY to_do_items.created_at [["to_do_list_id", 144], ["user_id", 543]]
ToDoList Load (0.4ms) SELECT "to_do_lists".* FROM "to_do_lists" WHERE "to_do_lists"."id" = $1 LIMIT 1 [["id", 133]]
SQL (3.3ms) SELECT "progress_items"."id" AS t0_r0, "progress_items"."completed" AS t0_r1, "progress_items"."user_id" AS t0_r2, "progress_items"."to_do_item_id" AS t0_r3, "progress_items"."created_at" AS t0_r4, "progress_items"."updated_at" AS t0_r5, "progress_items"."hidden" AS t0_r6, "to_do_items_progress_items"."id" AS t1_r0, "to_do_items_progress_items"."to_do_list_id" AS t1_r1, "to_do_items_progress_items"."description" AS t1_r2, "to_do_items_progress_items"."date_due" AS t1_r3, "to_do_items_progress_items"."created_at" AS t1_r4, "to_do_items_progress_items"."updated_at" AS t1_r5, "to_do_items_progress_items"."name" AS t1_r6, "to_do_items_progress_items"."element_id" AS t1_r7, "to_do_items_progress_items"."linkable_id" AS t1_r8, "to_do_items_progress_items"."linkable_type" AS t1_r9, "to_do_items_progress_items"."action" AS t1_r10 FROM "progress_items" INNER JOIN "to_do_items" "to_do_items_progress_items" ON "to_do_items_progress_items"."id" = "progress_items"."to_do_item_id" INNER JOIN "to_do_items" ON "progress_items"."to_do_item_id" = "to_do_items"."id" WHERE "to_do_items"."to_do_list_id" = $1 AND "progress_items"."user_id" = $2 ORDER BY to_do_items.created_at [["to_do_list_id", 133], ["user_id", 543]]
vs.
Without the joins clause (queries are not optimal):
object.to_do_list.progress_items.where(user_id: object.user_id)
.includes(:to_do_item)
Server output:
ToDoListsUser Load (0.3ms) SELECT "to_do_lists_users".* FROM "to_do_lists_users" WHERE "to_do_lists_users"."user_id" = $1 ORDER BY "to_do_lists_users"."id" DESC OFFSET 0 [["user_id", 543]]
ToDoList Load (0.4ms) SELECT "to_do_lists".* FROM "to_do_lists" WHERE "to_do_lists"."id" = $1 LIMIT 1 [["id", 144]]
ProgressItem Load (2.0ms) SELECT "progress_items".* FROM "progress_items" INNER JOIN "to_do_items" ON "progress_items"."to_do_item_id" = "to_do_items"."id" WHERE "to_do_items"."to_do_list_id" = $1 AND "progress_items"."user_id" = $2 ORDER BY to_do_items.created_at [["to_do_list_id", 144], ["user_id", 543]]
ToDoList Load (0.5ms) SELECT "to_do_lists".* FROM "to_do_lists" WHERE "to_do_lists"."id" = $1 LIMIT 1 [["id", 133]]
ProgressItem Load (0.5ms) SELECT "progress_items".* FROM "progress_items" INNER JOIN "to_do_items" ON "progress_items"."to_do_item_id" = "to_do_items"."id" WHERE "to_do_items"."to_do_list_id" = $1 AND "progress_items"."user_id" = $2 ORDER BY to_do_items.created_at [["to_do_list_id", 133], ["user_id", 543]]
ToDoItem Load (0.4ms) SELECT "to_do_items".* FROM "to_do_items" WHERE "to_do_items"."id" IN (193, 194, 195, 196)
From ActiveRecord documentation:
conditions
If you want to add conditions to your included models you’ll have to explicitly reference them. For example:
User.includes(:posts).where('posts.name = ?', 'example')
Will throw an error, but this will work:
User.includes(:posts).where('posts.name = ?', 'example').references(:posts)
Note that includes works with association names while references needs the actual table name.
That happens, because actually Rails with include doesn't do join, it fetches records with two requests.
For example:
class List < ActiveRecord::Base
has_many :tasks
end
List.includes(:tasks)
# List Load (1.9ms) SELECT "lists".* FROM "lists"
# Task Load (0.8ms) SELECT "tasks".* FROM "tasks" WHERE "tasks"."list_id" IN (1, 2, 3)
But with references it falls back to join:
List.includes(:tasks).references(:tasks)
# SQL (0.6ms) SELECT "lists"."id" AS t0_r0, "lists"."name" AS t0_r1,
# "tasks"."id" AS t1_r0, "tasks"."name" AS t1_r1 FROM
# "lists" LEFT OUTER JOIN "tasks" ON "tasks"."list_id" = "lists"."id"
Using Rails 4 and Ruby 1.9.3.
I have this view in usuarios#show I'm rendering. The controller demands a lot of data, but I have managed to reduce the queries with eager_load and akin from 300ms odd to a 44 ms.
My problem is that the database call is ok, but the views are taking close to 30 seconds to render.
I have made sure that I'm getting everything I use from the controller variables.
EDIT: I have added the query itself from the controller
This is the controller method, using eager_loading
#app/controllers/usuarios_controller.rb
before_action :show_usuario, only: [:show]
def show
if !#usuario.country_id.blank?
#country = #usuario.country
end
respond_to do |format
format.html
format.js
end
end
private
def show_usuario
if usuario_signed_in?
id = current_usuario.id
#usuario = Usuario.eager_load(:profile).find(id)
else
#usuario = Usuario.eager_load(:profile, textos: [:likes, :text_medals, :tags]).find(params[:id])
end
#textos = Texto.eager_load(:likes, :text_medals, :tags).where(:usuario_id => #usuario.id).order("textos.created_at DESC").paginate(:page => params[:page]).per_page(6)
end
Here is my log
Started GET "/" for 127.0.0.1 at 2014-03-25 12:55:39 +0100
Usuario Load (1.4ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = 12 ORDER BY "usuarios"."id" ASC LIMIT 1
Processing by UsuariosController#show as HTML
Notification Load (1.2ms) SELECT "notifications".* FROM "notifications" INNER JOIN "textos" ON "textos"."id" = "notifications"."text" WHERE (textos.usuario_id = 12 AND notifications.user_related != 12) ORDER BY "notifications"."id" DESC LIMIT 10
SQL (1.2ms) SELECT "usuarios"."id" AS t0_r0, "usuarios"."email" AS t0_r1, "usuarios"."encrypted_password" AS t0_r2, "usuarios"."reset_password_token" AS t0_r3, "usuarios"."reset_password_sent_at" AS t0_r4, "usuarios"."remember_created_at" AS t0_r5, "usuarios"."sign_in_count" AS t0_r6, "usuarios"."current_sign_in_at" AS t0_r7, "usuarios"."last_sign_in_at" AS t0_r8, "usuarios"."current_sign_in_ip" AS t0_r9, "usuarios"."last_sign_in_ip" AS t0_r10, "usuarios"."created_at" AS t0_r11, "usuarios"."updated_at" AS t0_r12, "usuarios"."nombre" AS t0_r13, "usuarios"."fecha_nac" AS t0_r14, "usuarios"."country_id" AS t0_r15, "usuarios"."is_admin" AS t0_r16, "usuarios"."publish_fbviews" AS t0_r17, "usuarios"."accept_terms" AS t0_r18, "profiles"."id" AS t1_r0, "profiles"."usuario_id" AS t1_r1, "profiles"."quote" AS t1_r2, "profiles"."quote_author" AS t1_r3, "profiles"."fb_account" AS t1_r4, "profiles"."twt_account" AS t1_r5, "profiles"."gpls_account" AS t1_r6, "profiles"."biografia" AS t1_r7, "profiles"."created_at" AS t1_r8, "profiles"."updated_at" AS t1_r9, "profiles"."hide_email" AS t1_r10, "profiles"."pic_file_name" AS t1_r11, "profiles"."pic_content_type" AS t1_r12, "profiles"."pic_file_size" AS t1_r13, "profiles"."pic_updated_at" AS t1_r14 FROM "usuarios" LEFT OUTER JOIN "profiles" ON "profiles"."usuario_id" = "usuarios"."id" WHERE "usuarios"."id" = $1 LIMIT 1 [["id", 12]]
USER CARGADO
Country Load (0.7ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT 1 [["id", 246]]
TEXTOS CARGADOS
Rendered usuarios/_main_frame.html.erb (12082.9ms)
(1.6ms) SELECT COUNT(DISTINCT "textos"."id") FROM "textos" LEFT OUTER JOIN "likes" ON "likes"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals_textos" ON "text_medals_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals" ON "text_medals"."id" = "text_medals_textos"."text_medal_id" LEFT OUTER JOIN "tags_textos" ON "tags_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "tags_textos"."tag_id" WHERE "textos"."usuario_id" = 12
SQL (1.9ms) SELECT DISTINCT "textos".id, textos.created_at AS alias_0 FROM "textos" LEFT OUTER JOIN "likes" ON "likes"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals_textos" ON "text_medals_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals" ON "text_medals"."id" = "text_medals_textos"."text_medal_id" LEFT OUTER JOIN "tags_textos" ON "tags_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "tags_textos"."tag_id" WHERE "textos"."usuario_id" = 12 ORDER BY textos.created_at DESC LIMIT 6 OFFSET 0
SQL (5.6ms) SELECT "textos"."id" AS t0_r0, "textos"."usuario_id" AS t0_r1, "textos"."titulo" AS t0_r2, "textos"."contenido" AS t0_r3, "textos"."idioma" AS t0_r4, "textos"."created_at" AS t0_r5, "textos"."updated_at" AS t0_r6, "textos"."is_borrador" AS t0_r7, "textos"."is_on_contest" AS t0_r8, "textos"."portada_file_name" AS t0_r9, "textos"."portada_content_type" AS t0_r10, "textos"."portada_file_size" AS t0_r11, "textos"."portada_updated_at" AS t0_r12, "textos"."contest_id" AS t0_r13, "textos"."views" AS t0_r14, "textos"."has_draft" AS t0_r15, "textos"."license_type" AS t0_r16, "likes"."id" AS t1_r0, "likes"."usuario_id" AS t1_r1, "likes"."texto_id" AS t1_r2, "likes"."created_at" AS t1_r3, "likes"."updated_at" AS t1_r4, "text_medals"."id" AS t2_r0, "text_medals"."name" AS t2_r1, "text_medals"."slogan" AS t2_r2, "text_medals"."created_at" AS t2_r3, "text_medals"."updated_at" AS t2_r4, "text_medals"."image_file_name" AS t2_r5, "text_medals"."image_content_type" AS t2_r6, "text_medals"."image_file_size" AS t2_r7, "text_medals"."image_updated_at" AS t2_r8, "tags"."id" AS t3_r0, "tags"."nombre" AS t3_r1, "tags"."created_at" AS t3_r2, "tags"."updated_at" AS t3_r3 FROM "textos" LEFT OUTER JOIN "likes" ON "likes"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals_textos" ON "text_medals_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals" ON "text_medals"."id" = "text_medals_textos"."text_medal_id" LEFT OUTER JOIN "tags_textos" ON "tags_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "tags_textos"."tag_id" WHERE "textos"."usuario_id" = 12 AND "textos"."id" IN (75, 74, 73, 70, 69, 68) ORDER BY textos.created_at DESC
CACHE (0.0ms) SELECT COUNT(DISTINCT "textos"."id") FROM "textos" LEFT OUTER JOIN "likes" ON "likes"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals_textos" ON "text_medals_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals" ON "text_medals"."id" = "text_medals_textos"."text_medal_id" LEFT OUTER JOIN "tags_textos" ON "tags_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "tags_textos"."tag_id" WHERE "textos"."usuario_id" = 12
Usuario Load (0.8ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
Profile Load (0.7ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.1ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.0ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.1ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.0ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.1ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.1ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.0ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.0ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.1ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.1ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
Rendered usuarios/_user_texts.html.erb (16760.5ms)
Rendered usuarios/show.html.erb within layouts/application (28889.5ms)
Completed 200 OK in 29246ms (Views: 28898.8ms | ActiveRecord: 44.1ms)
As you can see, the views take ages to render.
This the "_user_texts" partial.
<div class="ui grid">
<% #textos.each do |texto| %>
<div class="text row" text-pages="<%= #textos.total_pages%>">
<div class="five wide column">
<%= image_tag texto.portada.url(:medium), :class => "ui rounded medium left floated image text-cover" %>
</div>
<div class="eleven wide column">
<div class="row">
<h1 class="ui header">
<%= link_to text_name(texto), usuario_texto_path(texto.usuario.id, texto.id), :class => "title-format"%>
<div class="sub header subheader-format">
escrito por <%= text_author_image texto %><%= text_author_name texto %> el <%= humanize_creation_date(texto.created_at) %>
</div>
</h1>
</div>
<div class="tag-list row">
<div class="ui horizontal list">
<% if not texto.tags.empty? %>
<% texto.tags.each do |tag| %>
<div class="tag item">
<%= link_to tag.nombre, tag_path(tag.id), :class => "ui red large label" %>
</div>
<% end %>
<% end %>
</div>
</div>
</div>
</div>
<% end %>
</div>
<% if #textos.total_pages > #textos.current_page %>
<a id="more_texts" href="javascript:void(0)">
<span class="vertical"></span><span>Ver más</span>
</a>
<% end %>
EDIT: I am using pagination. Specifically the will_paginate gem. When I get to the bottom of the page, there's a button for fetching more results.
Can somebody give me insights on how could I make it faster? This is driving me nuts.
Thanks!
I would benchmark your query, seems pretty heavy to me. I think the focus should be on enhancing the performance of your sql query. I have no knowledge of the exact content, but I doubt you need this much intelligence in a single query.
For instance, you could add indices within your table to improve performance. You should also consider what attributes you really need and use select to define them. Less columns in your query yield faster results.
Finally, you are doing a lot of joins. It seems like you are pretty much joining all of your models. Left joins are really slow, because you go over all records. If you really need all these joins, it would be great if you could turn some of those left joins to inner joins . This would mean a massive improvement, because you have to check only relevant records.
The query is not actually run until you call .all or .each in the view even if you store the results in an instance variable in the controller. This can make it seem like the views are slow when actually it's something in the database.
Try putting some calls to benchmark different places in your view to see where the time is being used.
http://api.rubyonrails.org/classes/ActiveSupport/Benchmarkable.html#method-i-benchmark
For seeing what you're query is doing and spotting missing indexes etc. explain is handy.
http://guides.rubyonrails.org/active_record_querying.html#running-explain
Solved. With the help of the Bullet gem, I reformulate the way I was eager loading.
It turns out that everything done was, in practice, well done. The issue was that my partner was storing the images for paperclip in Dropbox.
Dropbox is most likely not made for this kind of requests, so the delay was because of this. We will change to Amazon's S3.
For the reader from the future: be advised! And check where are you storing your images.
Thanks to everybody!
I have 2 models
Class Ride
has_many :trips
#state (string: active or expired)
end
Class Trip
#date (Date attribute)
scope :active, -> (start_at = Date.today) { where("trips.date >= ?", [Date.today, start_at].max) }
end
Daily, I need update state on Rides with active state having all trips with date attribute < Date.today
how to perform this in 1 query?
i can archive such result using:
Ride.with_active_state.select{|r| r.trips.active.size ==0}
but it makes huje queries to count trips, eq:
[1] pry(main)> Ride.with_active_state.select{|r| r.trips.active.size ==0}
(7.3ms) SELECT f_geometry_column,coord_dimension,srid,type FROM geometry_columns WHERE f_table_name='rides'
Ride Load (1.6ms) SELECT "rides".* FROM "rides" WHERE (rides.workflow_state = 'active')
(2.9ms) SELECT f_geometry_column,coord_dimension,srid,type FROM geometry_columns WHERE f_table_name='trips'
(1.3ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 9]]
(0.7ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 10]]
(0.7ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 11]]
(0.7ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 12]]
(0.8ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 13]]
(0.8ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 14]]
(0.5ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 15]]
(0.5ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 16]]
(0.5ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 17]]
(0.5ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 18]]
(0.5ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 19]]
(0.5ms) SELECT COUNT(*) FROM "trips" WHERE "trips"."ride_id" = $1 AND (trips.date >= '2013-09-24') [["ride_id", 20]]
....
Add scopes on Ride with a group and having clause. It would check the count of all future trips of a ride and return the rides with 0 count.
Class Ride
scope :active_state, where(state: "active")
scope :with_nonactive_trips, -> (start_date = Date.today){ joins(:trips).
group("rides.id").
having( ["sum(trips.date > ?) = 0",start_date] ) }
end
Ride.active_state.with_nonactive_trips
# returns All the rides with state == active, alteast one trip and having no trips with date > Date.today
Using a lambda since you had it on the active scope in Trip. I am guessing you need to use a different date than Date.today for some queries.
I have a rake task that foreach through a CSV file of IDs. I then query the database to see if that id exists in my database. if it exists i use act_as_taggable to add a tag. the script runs fine until it gets to the first match and attempts to write a tag to my SQLite DB. and i get a database locked error. I am wondering if i am hitting the I/O limit of SQLite and need to switch over to a fullblown MySQL db.
if defined?(Rails) && (Rails.env == 'development')
Rails.logger = Logger.new(STDOUT)
end
require 'csv'
desc "Tag Voters that early voted from the Secretary of State Website"
task :tag_early => [:environment] do
file ="db/AllAbsentees.csv"
CSV.foreach(file, :headers=> true) do |row|
#voter_id = row[1]
puts "Working on Row" + row[1]
#voter = Voter.where(:state_id => #voter_id).first()
unless #voter.blank?
#voter.tag_list = "2012_GEN_EARLY_VOTER"
#voter.save()
puts "Voter #" + #voter_id + "tagged"
end
end
end
Voter Load (1.2ms) SELECT "voters".* FROM "voters" WHERE "voters"."state_id" = '00008030' LIMIT 1
ActsAsTaggableOn::Tag Load (0.2ms) SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 11944 AND "taggings"."taggable_type" = 'Voter' AND (taggings.context = 'tags' AND taggings.tagger_id IS NULL)
(0.1ms) begin transaction
(0.4ms) UPDATE "voters" SET "updated_at" = '2012-11-23 00:02:33.438114' WHERE "voters"."id" = 11944
ActsAsTaggableOn::Tag Load (0.1ms) SELECT "tags".* FROM "tags" WHERE (lower(name) = '2012_gen_early_voter')
ActsAsTaggableOn::Tag Load (0.2ms) SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 11944 AND "taggings"."taggable_type" = 'Voter' AND (taggings.context = 'tags' AND taggings.tagger_id IS NULL)
ActsAsTaggableOn::Tagging Exists (0.1ms) SELECT 1 AS one FROM "taggings" WHERE ("taggings"."tag_id" = 19 AND "taggings"."taggable_type" = 'Voter' AND "taggings"."taggable_id" = 11944 AND "taggings"."context" = 'tags' AND "taggings"."tagger_id" IS NULL AND "taggings"."tagger_type" IS NULL) LIMIT 1
SQL (1.4ms) INSERT INTO "taggings" ("context", "created_at", "tag_id", "taggable_id", "taggable_type", "tagger_id", "tagger_type") VALUES (?, ?, ?, ?, ?, ?, ?) [["context", "tags"], ["created_at", Fri, 23 Nov 2012 00:02:33 UTC +00:00], ["tag_id", 19], ["taggable_id", 11944], ["taggable_type", "Voter"], ["tagger_id", nil], ["tagger_type", nil]]
(5053.1ms) commit transaction
SQLite3::BusyException: database is locked: commit transaction
(99.7ms) rollback transaction
rake aborted!
SQLite3::BusyException: database is locked: commit transaction
SQLite has not much concurrency; for a transaction to write to the DB, there must be no other reading or writing connections.
Ensure that no other program is reading from or writing to the DB at the same time, and that all DB accesses in your own program use the same database connection.