I am trying to optimise the performance of a query in Active Record. Previously I would do two SQL queries and it should be possible to do it in one.
These are the tables that I am running the queries on:
# Table name: notifications
#
# id :integer not null, primary key
# content :text(65535)
# position :integer
# Table name: dismissed_notifications
#
# id :integer not null, primary key
# notification_id :integer
# user_id :integer
This is the existing query:
where.not(id: user.dismissed_notifications.pluck(:id))
which produces:
SELECT `dismissed_notifications`.`id` FROM `dismissed_notifications` WHERE `dismissed_notifications`.`user_id` = 655
SELECT `notifications`.* FROM `notifications` WHERE (`notifications`.`id` != 1)
This is the SQL I would like to get, which returns the same records:
select *
from notifications n
where not exists(
select 1
from dismissed_notifications dn
where dn.id = n.id
and dn.user_id = 655)
You can write not exists Query like below
where('NOT EXISTS (' + user.dismissed_notifications.where('dismissed_notifications.id = notifications.id').to_sql + ')')
OR
There is also another way to reduce the number of queries is use select instead of pluck, it will create sub-query instead pulling records from database. Rails ActiveRecord Subqueries
where.not(id: user.dismissed_notifications.select(:id))
Which will generate below SQL query
SELECT `notifications`.*
FROM `notifications`
WHERE (
`notifications`.`id` NOT IN
(SELECT `dismissed_notifications`.`id`
FROM `dismissed_notifications`
WHERE `dismissed_notifications`.`user_id` = 655
)
)
Related
I wanted to insert data into a related table. A 1 to many relationships. After searching the best practice I found this link and then I implemented this.
class InsertDataIntoPermissionAndPermissionGroup < ActiveRecord::Migration
def up
execute <<-SQL
-- ----------------------------------------------------------------------------
WITH a AS (
INSERT INTO spree_permission_groups (name, created_at, updated_at)
VALUES ('role_manager', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP) RETURNING id
)
INSERT INTO
spree_permissions (name, action, permission_group_id, created_at, updated_at)
SELECT 'Role', 'manage', id, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP FROM a
-- ----------------------------------------------------------------------------
-- ----------------------------------------------------------------------------
WITH b AS (
INSERT INTO spree_permission_groups (name, created_at, updated_at)
VALUES ('department_manager', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP) RETURNING id
)
INSERT INTO
spree_permissions (name, action, permission_group_id, created_at, updated_at)
SELECT 'Department', 'manage', id, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP FROM b
-- ----------------------------------------------------------------------------
-- ----------------------------------------------------------------------------
WITH c AS (
INSERT INTO spree_permission_groups (name, created_at, updated_at)
VALUES ('holiday_manager', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP) RETURNING id
)
INSERT INTO
spree_permissions (name, action, permission_group_id, created_at, updated_at)
SELECT 'Holiday', 'manage', id, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP FROM c
-- ----------------------------------------------------------------------------
SQL
end
def down
raise ActiveRecord::IrreversibleMigration
end
end
but I have more than 300 data. Is this still the right way of doing this? or I can import an excel data and do rails create method.
I also have an Error. Thought this will ok since I wrap it in sql.
Caused by:
PG::SyntaxError: ERROR: syntax error at or near "WITH"
LINE 14: WITH b AS (
^
UPDATE
fixed error
I would suggest you do insert records in rake task. And you can use Active record import gem(https://github.com/zdennis/activerecord-import) to insert bulk data to the table. You can even use active record import in migration too. Do check the gem documentation for using more available options.
For creating rake tasks, you can refer this - https://railsguides.net/how-to-generate-rake-task/
desc 'task description'
task :bulk_insert_records => :environment do |_, args|
spree_permission_groups = ['holiday_manager', 'department_manager', 'role_manager']
spree_permission_groups_list = []
spree_permission_groups.each do |spree_permission_group|
spree_permission_groups_list << SpreePermissionGroup.new(name: spree_permission_group)
end
SpreePermissionGroup.import(spree_permission_groups_list)
# Likewise use the id of the created records to set permissions
end
I have setup my databases like this :
Product :
# id :integer not null, primary key
# code :string
# name :string
# category_id :integer
...
Order items :
# id :integer not null, primary key
# order_id :integer
# product_id :integer
# color_id :integer
# qty :integer default("0")
# price :money
...
Order :
# id :integer
# state :string
# placed_on :datetime
...
Now this setup make it really hard for me to pick the best selling products in each week from each category. How can I fix this? Another database to keep track of sales? Please help.
What you basically need is to join categories, products, order_items and orders tables.
Joining can be done with the following code:
rel = Category.joins(products: [order_items: :order])
#=> SELECT "categories".* FROM "categories" INNER JOIN "products" ON "products"."category_id" = "categories"."id" INNER JOIN "order_items" ON "order_items"."product_id" = "products"."id" INNER JOIN "orders" ON "orders"."id" = "order_items"."order_id"
Based on this you can filter on dates interval.
Let's assume we already have d1 and d2 values, which define start and end of the interval we are interested in:
rel = rel.where('orders.placed_on >= ? AND orders.placed_on <= ?', d1, d2)
Now you can aggregate on fields:
result = rel.select('categories.id, categories.name, SUM(order_items.qty) as qty, SUM(order_items.qty * order_items.price) as total')
.group('categories.id, categories.name')
sample = result.first
sample.id # => 1
sample.name # => "Category 1"
sample.qty # => 100.0
sample.total # => 500.0
This looks like a simple database query to me. The following should be the simple and straight-forward steps to accomplish it.
Join the three tables
Filter it by date
Group by product_id
Aggregate the qty
And, sort by aggregated value.
I'm not confident about the method to get the date. Please fill it in yourself in the query below.
SELECT P.id, P.name, P.category_id, SUM(qty) as LastWeekSales
FROM Product as P INNER JOIN Order Items as OI
ON P.id = OI.product_id
INNER JOIN Order as O
ON O.id = OI.order_id
WHERE O.placed_on <= GetTodaysDate() AND O.placed_on > GetOneWeekBacksDate()
GROUPBY P.category_id
ORDERBY WeekSales
All you need to do is, prepare this query in ruby-on-rails. I would do it but I don't know anything about ruby-on-rails.
+1 on taking care of this in the model. Here is some working drop-in code if you need it in the meantime. I'm practicing manipulating hashes tonight in case you couldn't tell, ha.
Add to Order model:
def self.last_week
Order.where(" created_at >= ? ", 1.week.ago.utc)
end
Add to whatever controller:
#qty_hash = category = Hash.new 0;
#output_hash = Hash.new { |hash, key| hash[key] = {} }
#recently_ordered_items = OrderItem.find_all_by_order_id(Order.last_week)
#recently_ordered_items.each { |i| #qty_hash[i.product_id] += i.qty }
#recent_products=Product.find_all_by_id(#qty_hash.keys)
#qty_hash.each do |key, value|
#recent_products.each { |i| category = i.category_id if i.id == key }
#output_hash[category] = #output_hash[category].merge(#qty_hash.slice(key))
end
#output_hash is the output and is in the format:
{1=>{3=>9}, 2=>{4=>8, 6=>5, 7=>4}}
In this case the categories are 1 and 2, product ids are 3 (9 sold), 4 (8 sold), 6 (5 sold), and 7 (4 sold)
Tested and working. Good luck.
I have the following models:
# == Schema Information
#
# Table name: quotes
#
# id :integer not null, primary key
# bound_rate_id :integer
class Quote < ActiveRecord::Base
#snip
end
# == Schema Information
#
# Table name: rates
#
# id :integer not null, primary key
# quoted_premium :integer
class Rate < ActiveRecord::Base
#snip
end
I want to create a query that would calculate the same thing as this loop:
sum = 0
for quote in Quote.all
rate = Rate.find(quote.bound_rate_id)
sum += rate.quoted_premium
end
How would I do this using ActiveRecord's query interface? (I am using Rails 4.)
EDIT: I already have ActiveRecord instances from previous queries over Quote, so I would prefer to have my query start from the quotes table and join to the rates table, not the other way around. Like this:
some_quotes = Quote.where(:some_other_property, my_param);
sum_of_rates = some_quotes.?????
Try this out
sum = Rate.where(:id => Quote.pluck(:bound_rate_id).compact).sum(:quoted_premium)
After adding relations try this out
sum = Quote.joins(:rate).sum('rates.quoted_premium') # it will get sum of all query's quoted_premium
To get sum of some specific add where clause
sum = Quote.joins(:rate).where(:bound_rate_id => [list of Rate ids]).sum('rates.quoted_premium')
If you get a Mysql2::Error: Unknown column 'rates.bound_rate_id' in 'on clause' error, specify how ActiveRecord should put together the join
sum = Quote.joins('INNER JOIN rates ON quotes.bound_rate_id = rates.id').sum('rates.quoted_premium')
I've 3 tables in database dishes, price_deals and give_away_deals
# == Schema Information
#
# Table name: dishes
#
# id :integer not null, primary key
# name :string(255)
# created_at :datetime
# updated_at :datetime
# Table name: price_deals
#
# id :integer not null, primary key
# name :string(255)
# dish_id :integer
# created_at :datetime
# updated_at :datetime
# Table name: give_away_deals
#
# id :integer not null, primary key
# name :string(255)
# dish_id :integer
# created_at :datetime
# updated_at :datetime
I've to get the id and name from dishes table. Where id not in price_deals and id not in give_away_deals without duplicate id.
Suppose I've 10 records in dishes table id 1 to 10.
In price_deals table dish_id are 2,4,5.
In give_away_deals table dish_id are 1,3,6
Expected result:
Then, I've to get the id and name from dishes table where id are 7,8,9,10
I tried this query,
Dish.all(:select => "id, name", :conditions => ["id not in (select dish_id from price_deals)", "id not in (select dish_id from give_away_deals)"])
But it returns only data which is not in price_deals.
What's wrong in the above query and how to get the expected result?
This is the SQL query I got in rails server
SELECT id, name FROM "dishes" WHERE (id not in (select dish_id from price_deals))
OK time to some bigger answer.
Try this code:
first = PriceDeal.select(:dish_id).map(&:dish_id)
second = GiveAwayDeal.select(:dish_id).map(&:dish_id)
Dish.select('id, name').where("id not IN(?)", (first + second).uniq)
I think that should generate a good one SQL query
You can try this code
Dish.includes(:price_deal, :give_away_deal).where("price_deals.id is null and give_away_deals.id is null")
methods includes and where works together with one query
SQL (0.5ms) SELECT `dishes`.`id` AS t0_r0, `dishes`.`name` AS t0_r1, `dishes`.`created_at` AS t0_r2, `dishes`.`updated_at` AS t0_r3, `price_deals`.`id` AS t1_r0, `price_deals`.`dish_id` AS t1_r1, `price_deals`.`created_at` AS t1_r2, `price_deals`.`updated_at` AS t1_r3, `give_away_deals`.`id` AS t2_r0, `give_away_deals`.`dish_id` AS t2_r1, `give_away_deals`.`created_at` AS t2_r2, `give_away_deals`.`updated_at` AS t2_r3 FROM `dishes` LEFT OUTER JOIN `price_deals` ON `price_deals`.`dish_id` = `dishes`.`id` LEFT OUTER JOIN `give_away_deals` ON `give_away_deals`.`dish_id` = `dishes`.`id` WHERE (price_deals.id is null and give_away_deals.id is null)
I use Rails 3.2.1 and ruby 1.9.3
The following query works on local but not on production: (Production is heroku is running postgreSQL and locally I am running a sqllite database)
Ruby
Tutor.joins(:expertises).where(:expertises => {:subject_id => [2,4]}).group("tutors.id").having("COUNT(*) = 2")
SQL
SELECT "tutors".* FROM "tutors" INNER JOIN "expertises" ON "expertises"."tutor_id" = "tutors"."id" WHERE ("expertises"."subject_id" IN (9)) GROUP BY tutors.id HAVING COUNT(*) = 1 ORDER BY rank DESC)
I get the following error on production ActiveRecord::StatementInvalid (PGError: ERROR: column "tutors.fname" must appear in the GROUP BY clause or be used in an aggregate function
I have the following values in my table
id :integer not null, primary key
fname :string(255)
lname :string(255)
school :string(255)
major :string(255)
year :string(255)
bio :text
vid :string(255)
price :integer
phone :string(255)
skype :string(255)
address :text
When I try to adjust the query to group by all attributes, I get another error:
Ruby
>> Tutor.joins(:expertises).where(:expertises => {:subject_id => [2,4]}).group("tutors.*").having("COUNT(*) = 2")
SQL
SELECT "tutors".* FROM "tutors" INNER JOIN "expertises" ON "expertises"."tutor_id" = "tutors"."id" WHERE ("expertises"."subject_id" IN (2, 4)) GROUP BY tutors.* HAVING COUNT(*) = 2 ORDER BY rank DESC
ActiveRecord::StatementInvalid: PGError: ERROR: could not identify an ordering operator for type tutors HINT: Use an explicit ordering operator or modify the query.
Help!
PostgreSQL does not support unaggregated and ungrouped expressions in GROUP BY queries.
Use this:
SELECT t.*
FROM (
SELECT tutor_id
FROM expertises e
WHERE e.subject_id IN (2, 4)
GROUP BY
tutor_id
HAVING COUNT(*) = 2
) e
JOIN tutor t
ON t.id = e.tutor_id
This is cross-platform.
Hmm, nice try, but asterisk expansion doesn't work that way in the GROUP BY clause. You need to actually list all the columns. Or upgrade to PostgreSQL 9.1 in a few months.
Try this:
subjects = [1,2]
sub_query_sql = Tutor.select("tutors.id").joins(:expertises).
where(:expertises => {:subject_id => subjects}).
group("tutors.id").having("COUNT(*) = #{subjects.size}").to_sql
Tutor.joins("JOIN (#{sub_query_sql}) A ON A.id = tutors.id")