I'm using the validates_overlap gem (https://github.com/robinbortlik/validates_overlap) in a Rails app. Here is the Model code:
validates :start_time, :end_time, overlap: { scope: "device_id", exclude_edges: ["start_time", "end_time"] }
And here is the SQL it triggers:
SELECT 1 AS one FROM "bookings" WHERE
((bookings.end_time IS NULL OR bookings.end_time > '2014-04-11 13:00:00.000000') AND
(bookings.start_time IS NULL OR bookings.start_time < '2014-04-11 16:00:00.000000') AND
bookings.device_id = 20) LIMIT 1
I just want to know if I should be adding an index in my postgres database that covers start_time, end_time and device_id, or something similar? e.g. something like this:
add_index :bookings, [:device_id, :start_time, :end_time], unique: true
Adding the above index to ensure database consistency would make no sense. After all you are validating the Range AND excluding the actual edges (the unique index would check exactly the edges!).
Adding a non unique index to speed up the validation is a good idea. If so you should analyze your data and app queries.
The easiest approach is to simply add a single index for each column. Postgres can still use these for the multicolumn query (see heroku devcenter ).
Only if it really matters (or you do not query the columns in other combinations) a multicolumn index is necessary. If so the device_id should be first in index Rule of thumb: index for equality first—then for ranges.
Related
I am having some performance problems with this two queries:
any_impression = Impression.exists?(user_id: user_id, created_at: range)
any_visit = Visit.exists?(user_id: user_id, created_at: range)
They have about 500k of records for each user and are taking more than 15s to run.
Based on this I would like to create two indexes, one for each search.
My question is, the indexes I should create are:
add_index :visits, [:user_id, :created_at]
add_index :impressions, [:user_id, :created_at]
Or need more some specific information to queries above use the indexes created ?
Thanks much.
Those indexes should be fine. In Postgres an index doesn't always know how to use a given operator---it depends on the index type. This page from the manual explains the details.
Your proposed indexes would be btree indexes. In my experiments, telling ActiveRecord to query a timestamp column based on a range produces BETWEEN ... AND ... SQL:
User.where(created_at: (Date.parse('2015-01-01') ..
Date.parse('2016-01-01'))).to_sql
gives:
SELECT "users".*
FROM "users"
WHERE ("users"."created_at" BETWEEN '2015-01-01' AND '2016-01-01')
Is that what you're seeing also? Then Postgres should use your index, because BETWEEN is just <= and >=.
You could also run the query by hand with EXPLAIN or EXPLAIN ANALYZE to see if the index is used as you expect.
I have to run a query where a single column is called twice .
Transaction.where("datetime >= ? && datetime <= ?", params[:date_one], :params[:date_two])
Now while indexing this is the basic we do
add_index :transactions, :datetime
Now my question is can I do something like.....
add_index :transactions, [:datetime, :datetime]
Will it really speedup the search or benefit performance wise. Thanks in advance
You don't have to do that. adding index to column speeds up queries to that column. It does not matter how many times you use this column in your query, only the presence or absence of index matters. Also, you can rewrite your query like this:
Transaction.where("datetime BETWEEN ? AND ?", params[:date_one], :params[:date_two])
Let's say I do Image.column_names and that shows all the columns such as post_id but how do I check if post_id has an index on it?
There is an index_exists? method on one of the ActiveRecord "connection adapters" classes.
You can use it like on of the following methods:
ActiveRecord::Migration.connection.index_exists? :images, :post_id
ActiveRecord::Base.connection.index_exists? :images, :post_id
If you know the name of the index, instead you'll need to use index_name_exists?
ActiveRecord::Base.connection.index_name_exists? :images, :index_images_on_join_key
As others have mentioned, you can use the following to check if an index on the column exists:
ActiveRecord::Base.connection.index_exists?(:table_name, :column_name)
It's worth noting, however, that this only returns true if an index exists that indexes that column and only that column. It won't return true if you're using compound indices that include your column. You can see all of the indexes for a table with
ActiveRecord::Base.connection.indexes(:table_name)
If you look at the source code for index_exists?, you'll see that internally it's using indexes to figure out whether or not your index exists. So if, like me, their logic doesn't fit your use case, you can loop through these indexes and see if one of them will work. In my case, the logic was thus:
ActiveRecord::Base.connection.indexes(:table_name).select { |i| i.columns.first == column_name.to_s}.any?
It's also important to note, indexes does not return the index that rails automatically generates for ids, which explains why some people above were having problems with calls to index_exists?(:table_name, :id)
The following worked for me:
ActiveRecord::Base.connection.index_exists?(:table_name, :column_name)
For an updated answer, as of Rails 3+ multi-column, uniqueness and custom name are all supported within the #index_exists? method.
# Check an index exists
index_exists?(:suppliers, :company_id)
# Check an index on multiple columns exists
index_exists?(:suppliers, [:company_id, :company_type])
# Check a unique index exists
index_exists?(:suppliers, :company_id, unique: true)
# Check an index with a custom name exists
index_exists?(:suppliers, :company_id, name: "idx_company_id")
Source: Rails 6 Docs
I've seen other SO questions like - How do you validate uniqueness of a pair of ids in Ruby on Rails? - which describes adding a scoped parameter to enforce uniqueness of a key pair, i.e. (from the answer)
validates_uniqueness_of :user_id, :scope => [:question_id]
My question is how do you do this kind of validation for an entire row of data?
In my case, I have five columns and the data should only be rejected if all five are the same. This data is not user entered and the table is essentially a join table (no id or timestamps).
My current thought is to search for a record with all of the column values and only create if the query returns nil but this seems like a bad work around. Is there an easier 'rails way' to do this?
You'll need to create a custom validator (http://guides.rubyonrails.org/active_record_validations.html#performing-custom-validations):
class TotallyUniqueValidator < ActiveModel::Validator
def validate(record)
if record.attributes_for_uniqueness.values.uniq.size == 1
record.errors[:base] << 'All fields must be unique!'
end
end
end
class User
validates_with TotallyUniqueValidator
def attributes_for_uniqueness
attributes.except :created_at, :updated_at, :id
end
end
The important line here is:
if record.attributes_for_uniqueness.values.uniq.size == 1
This will grab a hash of all the attributes you want to check for uniqueness (in this case everything except id and timestamps) and converts it to an array of just the values, then calls uniq on it which returns only uniq values and if the size is 1 then they were all the same value.
Update based on your comment that your table doesn't have an id or timestamps:
You can then simply do:
if record.attributes.except(:id).values.uniq.size == 1
...because I'm pretty sure it still has an id unless you're sure it doesn't then just remove the except part.
You can add a unique index to the table in a migration:
add_index :widgets, [:column1, :column2, :column3, :column4, :column5], unique: true
The resulting index will require that each combination of the 5 columns must be unique.
My User table has about a million records.
My Region table has maybe 200 records.
Should I add_index :users, :region_id ?
And if it was the other way round where a region has a user_id, should I index the user_id in the region table?
Yes you should add. If you think to you are going to have lot of query that got condition for those fields.
In your case I think you might have lot queries like people living in one region, so I would create a index on users table for (id(user_id) and region_id fields together in one single index)
add_index :users, [:id, :user_id]
"Premature optimization is the root of all evil".
Add an index only if profiling of your application indicates that you need one.