What's the best way to implement an atomic insert/update for a model with a counter in Rails? A good analogy for the problem I'm trying to solve is a "like" counter with two fields:
url : string
count : integer
Upon insert, if there is not currently a record with a matching url, a new record should be created with count 1; else the existing record's count field should be incremented.
Initially I tried code like the following:
Like.find_or_create_by_url("http://example.com").increment!(:count)
But unsurprisingly, the resulting SQL shows that the SELECT happens outside the UPDATE transaction:
Like Load (0.4ms) SELECT `likes`.* FROM `likes` WHERE `likes`.`url` = 'http://example.com' LIMIT 1
(0.1ms) BEGIN
(0.2ms) UPDATE `likes` SET `count` = 4, `updated_at` = '2013-01-17 19:41:22' WHERE `likes`.`id` = 2
(1.6ms) COMMIT
Is there a Rails idiom for dealing with this, or do I need to implement this at the SQL level (and thus lose some portability)?
You can use pessimistic locking,
like = Like.find_or_create_by_url("http://example.com")
like.with_lock do
like.increment!(:count)
end
I am not aware of any ActiveRecord method that implemts atomic increments in a single query. Your own answer is a far as you can get.
So your problem may not be solvable using ActiveRecord. Remember: ActiveRecord is just a mapper to simplify some things, while complicating others. Some problems just solvable by plain SQL queries to the database. You will loose some portability. The example below will work in MySQL, but as far as I know, not on SQLlite and others.
quoted_url = ActiveRecord::Base.connection.quote(#your_url_here)
::ActiveRecord::Base.connection.execute("INSERT INTO likes SET `count` = 1, url = \"#{quoted_url}\" ON DUPLICATE KEY UPDATE count = count+1;");
Here is what I have come up with so far. This assumes a unique index on the url column.
begin
Like.create(url: url, count: 1)
puts "inserted"
rescue ActiveRecord::RecordNotUnique
id = Like.where("url = ?", url).first.id
Like.increment_counter(:count, id)
puts "incremented"
end
Rails' increment_counter method is atomic, and translates to SQL like the following:
UPDATE 'likes' SET 'count' = COALESCE('count', 0) + 1 WHERE 'likes'.'id' = 1
Catching an exception to implement normal business logic seems rather ugly, so I would appreciate suggestions for improvement.
Rails supports Transactions, if that's what you're looking for, so:
Like.transaction do
Like.find_or_create_by_url("http://example.com").increment!(:count)
end
My favorite solution thus far:
Like.increment_counter(
:count,
Like.where(url: url).first_or_create!.id
)
For this use case and others, I think update_all approach is cleanest.
num_updated =
ModelClass.where(:id => my_active_record_model.id,
:service_status => "queued").
update_all(:service_status => "in_progress")
if num_updated > 0
# we updated
else
# something else updated in the interim
end
Not exactly you example, just pasted from the linked page. But you control the where condition and what is to be updated. Very nifty
Related
I currently am pulling some info out of
model_hash = Model.where(id: some_id, name: some_name).pluck(:name, :id).to_h
I'd like to update another attribute on every model that it sees. I know I can write:
Model.where(id: some_id, name: some_name).update_all(last_import_date: DateTime.now)
But I'm wondering if there's a way that I can combine the two statements
ActiveRecord pluck generates a SQL SELECT query while update_all generates a SQL UPDATE query. You can retrieve data during an UPDATE for certain SQL dialects, e.g. in PostgreSQL you can use RETURNING to specify which values you want returned from the updated rows. In your case, e.g.:
UPDATE models
SET last_import_date = '2017-04-04 20:49:07'
WHERE name = 'Some Name'
RETURNING id;
ActiveRecord doesn't support this, likely because its highly dialect-dependent, and ActiveRecord intends to abstract away from those differences.
I don't think there is any such thing is required. Your code is generally used. But if you really want to do this in one query you can try this.
model_hash = Map.where(id: some_id, name: some_name).map { |x| x.update_attributes(last_import_date: DateTime.now) ? [x.id, x.some_name] : ["value","nil"] }.to_h
Its a complex one though.
Hope this Helps...
I'm trying to utilize .exists?() to return true or false, pretty simple. It would be nice to do the one-liner scope like so:
scope :any_alternates, lambda{|apikey| Track.exists?(:track_id => apikey)}
Or even using this scope syntax:
scope :any_alternates, ->(apikey) {Track.exists?(:track_id => apikey)}
But for some reason, the above scopes will return all rows in my db table when there's not a match. It works how it should when it finds a match however, but breaks if none...
I'm forced to create a method, which (to my knowledge) should be doing the same thing in the above scope:
def self.any_alternates(apikey)
return Track.exists?(:track_id => apikey)
end
Any idea why .exists?() isn't working inside of my scope?
After some testing...
If there is no match, then the scope will return all rows in the DB... (I updated above to mention that). I checked the generated query on both the scope and method to see if there's a difference, but they're the same:
SELECT 1 AS one FROM `tracks` WHERE `tracks`.`track_id` = '_btbd_uUmQT8hYUK3SrJ9Q' LIMIT 1
Update:
even though I'm searching on a column called track_id, this column is not setup as a relationship to another model. I know this is confusing, but that's how this table got setup (for good reason, beyond this issue so not worth touching on here)
Are you passing in nil? That would cause all records to be returned. You can drop the Track in the scope, like this:
scope :any_alternates, lambda{|apikey| exists?(:track_id => apikey)}
Here is what happens when you pass in nil:
> Track.any_alternates(nil).count
Track Exists (1.7ms) SELECT 1 AS one FROM "track" WHERE "tracks"."track_id" IS NULL LIMIT 1
instead of passing in a value:
> Track.any_alternates('X4DBA36gbtqgWl4F1')
Track Exists (0.4ms) SELECT 1 AS one FROM "tracks" WHERE "tracks"."track_id" = $1 LIMIT 1 [["track_id", "X4DBA36gbtqgWl4F1"]]
Are you sure that you have the right search? A Track having a track_id implies that there is a relation between Track and another track model which would be interesting.
Also the scope syntax is off. A scope is just a query of the table, so traditionally it is only things associated with the current model.
scope :any_alternates, ->(api_key) { |api_key| where(track_id: api_key }
I'm a rails newbie and am building an app. My current problem is trying to find the average time since the last purchase for each customer of an online store using the app, where we have data about their orders and their customers. The problem is that I'm getting an error that says "undefined method `src_created_at' for nil:NilClass." Right now I'm trying to do this only for customers that have purchased once, and leaving aside those that purchased multiple times.
Here's my code:
#customers = Customer.where(:identity_id => #identity.id)
#single_order_customers = #customers.where("orders_count = ?", 1)
days_since_array = []
#single_order_customers.each do |s|
one_order = Order.where(:identity_id => #identity.id, :src_customer_id => s.src_id)
the_date = one_order[0].src_created_at
purchase_date = the_date.to_date
days_between = (Date.today - purchase_date)
days_since_array << days_between
end
days = days_since_array.inject(:+)
#adslp = days / days_since_array.count
Thanks in advance. I can provide what customer and order data looks like if necessary. Any advice would help, even though I know this question is somewhat vague. I've tried some kind if and unless statements validating presence or nil values and they're not working.
Edit:
Here's what's run in the console:
Order Load (123.0ms) SELECT "orders".* FROM "orders" WHERE "orders"."identity_id" = 2 AND "orders"."src_customer_id" = '114863554'
NoMethodError: undefined method `src_created_at' for nil:NilClass
(The above is several orders successfully run and then breaking on the last I've shown.)
Last point: when I try, specifically, to find nil values for this purpose I don't find any.
Order.where(:identity_id => 2, :src_created_at => nil)
Order Load (209.6ms) SELECT "orders".* FROM "orders" WHERE "orders"."identity_id" = 2 AND "orders"."src_created_at" IS NULL
=> []
For your last point where you tried to find nil values, you are getting an empty array [] because your query returned an empty set (no records matched your query). Trying to access any index on an empty array in rails won't throw an IndexOutOfBoundsException, but will instead just return nil, which is where you are getting your nil from. You are expecting one_order[0] to be an order item, but it is instead nil.
The solution to your problem would be to make sure the Order you are searching for already exists in the database. You might want to check how your orders are being created, e.g. if you use Order.create(params[:order]), do you have any validations that are failing, etc. You can check the orders you have through the rails console; just run Order.all. There should be an Order that has an identity_id of 2 and src_created_at of nil for the last query you wrote to return an actual order in the set.
A quick fix for now would be to remove the extra query to set one_order and just get it from the customer:
...
#single_order_customers.each do |s|
one_order = s.orders.first
the_date = one_order.src_created_at
...
The extra query does not seem to be necessary. You have already filtered #customers by identity_id, and src_customer_id will definitely match s.src_id if you get orders by calling s.orders. Relationships should be set up correctly in the models so that a customer has_many orders. You mentioned that you are just starting with rails; I would highly recommend reading a tutorial for rails first to save yourself headaches like these in the future :)
What's the best way to implement an atomic insert/update for a model with a counter in Rails? A good analogy for the problem I'm trying to solve is a "like" counter with two fields:
url : string
count : integer
Upon insert, if there is not currently a record with a matching url, a new record should be created with count 1; else the existing record's count field should be incremented.
Initially I tried code like the following:
Like.find_or_create_by_url("http://example.com").increment!(:count)
But unsurprisingly, the resulting SQL shows that the SELECT happens outside the UPDATE transaction:
Like Load (0.4ms) SELECT `likes`.* FROM `likes` WHERE `likes`.`url` = 'http://example.com' LIMIT 1
(0.1ms) BEGIN
(0.2ms) UPDATE `likes` SET `count` = 4, `updated_at` = '2013-01-17 19:41:22' WHERE `likes`.`id` = 2
(1.6ms) COMMIT
Is there a Rails idiom for dealing with this, or do I need to implement this at the SQL level (and thus lose some portability)?
You can use pessimistic locking,
like = Like.find_or_create_by_url("http://example.com")
like.with_lock do
like.increment!(:count)
end
I am not aware of any ActiveRecord method that implemts atomic increments in a single query. Your own answer is a far as you can get.
So your problem may not be solvable using ActiveRecord. Remember: ActiveRecord is just a mapper to simplify some things, while complicating others. Some problems just solvable by plain SQL queries to the database. You will loose some portability. The example below will work in MySQL, but as far as I know, not on SQLlite and others.
quoted_url = ActiveRecord::Base.connection.quote(#your_url_here)
::ActiveRecord::Base.connection.execute("INSERT INTO likes SET `count` = 1, url = \"#{quoted_url}\" ON DUPLICATE KEY UPDATE count = count+1;");
Here is what I have come up with so far. This assumes a unique index on the url column.
begin
Like.create(url: url, count: 1)
puts "inserted"
rescue ActiveRecord::RecordNotUnique
id = Like.where("url = ?", url).first.id
Like.increment_counter(:count, id)
puts "incremented"
end
Rails' increment_counter method is atomic, and translates to SQL like the following:
UPDATE 'likes' SET 'count' = COALESCE('count', 0) + 1 WHERE 'likes'.'id' = 1
Catching an exception to implement normal business logic seems rather ugly, so I would appreciate suggestions for improvement.
Rails supports Transactions, if that's what you're looking for, so:
Like.transaction do
Like.find_or_create_by_url("http://example.com").increment!(:count)
end
My favorite solution thus far:
Like.increment_counter(
:count,
Like.where(url: url).first_or_create!.id
)
For this use case and others, I think update_all approach is cleanest.
num_updated =
ModelClass.where(:id => my_active_record_model.id,
:service_status => "queued").
update_all(:service_status => "in_progress")
if num_updated > 0
# we updated
else
# something else updated in the interim
end
Not exactly you example, just pasted from the linked page. But you control the where condition and what is to be updated. Very nifty
I want to execute one update raw sql like below:
update table set f1=? where f2=? and f3=?
This SQL will be executed by ActiveRecord::Base.connection.execute, but I don't know how to pass the dynamic parameter values into the method.
Could someone give me any help on it?
It doesn't look like the Rails API exposes methods to do this generically. You could try accessing the underlying connection and using it's methods, e.g. for MySQL:
st = ActiveRecord::Base.connection.raw_connection.prepare("update table set f1=? where f2=? and f3=?")
st.execute(f1, f2, f3)
st.close
I'm not sure if there are other ramifications to doing this (connections left open, etc). I would trace the Rails code for a normal update to see what it's doing aside from the actual query.
Using prepared queries can save you a small amount of time in the database, but unless you're doing this a million times in a row, you'd probably be better off just building the update with normal Ruby substitution, e.g.
ActiveRecord::Base.connection.execute("update table set f1=#{ActiveRecord::Base.sanitize(f1)}")
or using ActiveRecord like the commenters said.
ActiveRecord::Base.connection has a quote method that takes a string value (and optionally the column object). So you can say this:
ActiveRecord::Base.connection.execute(<<-EOQ)
UPDATE foo
SET bar = #{ActiveRecord::Base.connection.quote(baz)}
EOQ
Note if you're in a Rails migration or an ActiveRecord object you can shorten that to:
connection.execute(<<-EOQ)
UPDATE foo
SET bar = #{connection.quote(baz)}
EOQ
UPDATE: As #kolen points out, you should use exec_update instead. This will handle the quoting for you and also avoid leaking memory. The signature works a bit differently though:
connection.exec_update(<<-EOQ, "SQL", [[nil, baz]])
UPDATE foo
SET bar = $1
EOQ
Here the last param is a array of tuples representing bind parameters. In each tuple, the first entry is the column type and the second is the value. You can give nil for the column type and Rails will usually do the right thing though.
There are also exec_query, exec_insert, and exec_delete, depending on what you need.
None of the other answers showed me how to use named parameters, so I ended up combining exec_update with sanitize_sql:
User.connection.exec_update(
User.sanitize_sql(
[
"update users set name = :name where id = :id and name <> :name",
{
id: 123,
name: 'My Name'
}
]
)
)
This works for me on Rails 5, and it executes this SQL:
update users set name = 'My Name' where id = 123 and name <> 'My Name'
You need to use an existing Rails model instead of User if you don't have that.
I wanted to use named parameters to avoid issues with the ordering when I use ? or $1/$2,etc. Positional ordering is kind of frustrating when I have more than a handful of parameters, but named parameters allow me to refactor the SQL command without having to update the parameters.
You should just use something like:
YourModel.update_all(
ActiveRecord::Base.send(:sanitize_sql_for_assignment, {:value => "'wow'"})
)
That would do the trick. Using the ActiveRecord::Base#send method to invoke the sanitize_sql_for_assignment makes the Ruby (at least the 1.8.7 version) skip the fact that the sanitize_sql_for_assignment is actually a protected method.
Sometime would be better use name of parent class instead name of table:
# Refers to the current class
self.class.unscoped.where(self.class.primary_key => id).update_all(created _at: timestamp)
For example "Person" base class, subclasses (and database tables) "Client" and "Seller"
Instead using:
Client.where(self.class.primary_key => id).update_all(created _at: timestamp)
Seller.where(self.class.primary_key => id).update_all(created _at: timestamp)
You can use object of base class by this way:
person.class.unscoped.where(self.class.primary_key => id).update_all(created _at: timestamp)
Here's a trick I recently worked out for executing raw sql with binds:
binds = SomeRecord.bind(a_string_field: value1, a_date_field: value2) +
SomeOtherRecord.bind(a_numeric_field: value3)
SomeRecord.connection.exec_query <<~SQL, nil, binds
SELECT *
FROM some_records
JOIN some_other_records ON some_other_records.record_id = some_records.id
WHERE some_records.a_string_field = $1
AND some_records.a_date_field < $2
AND some_other_records.a_numeric_field > $3
SQL
where ApplicationRecord defines this:
# Convenient way of building custom sql binds
def self.bind(column_values)
column_values.map do |column_name, value|
[column_for_attribute(column_name), value]
end
end
and that is similar to how AR binds its own queries.
I needed to use raw sql because I failed at getting composite_primary_keys to function with activerecord 2.3.8. So in order to access the sqlserver 2000 table with a composite primary key, raw sql was required.
sql = "update [db].[dbo].[#{Contacts.table_name}] " +
"set [COLUMN] = 0 " +
"where [CLIENT_ID] = '#{contact.CLIENT_ID}' and CONTACT_ID = '#{contact.CONTACT_ID}'"
st = ActiveRecord::Base.connection.raw_connection.prepare(sql)
st.execute
If a better solution is available, please share.
In Rails 3.1, you should use the query interface:
new(attributes)
create(attributes)
create!(attributes)
find(id_or_array)
destroy(id_or_array)
destroy_all
delete(id_or_array)
delete_all
update(ids, updates)
update_all(updates)
exists?
update and update_all are the operation you need.
See details here: http://m.onkey.org/active-record-query-interface