This is my class:
class Plan < ActiveRecord::Base
def testing
self.with_lock do
update_columns(lock: true)
byebug
end
end
def testing2
self.lock!
byebug
end
end
I opened two rails consoles.
In first console:
p = Plan.create
=> (basically success)
p.id
=> 12
p.testing2
(byebug) # simulation of halting the execution,
(BYEBUG) # I just leave the rails console open and wait at here. I expect others won't be able to update p because I still got the lock.
On second console:
p = Plan.find(12)
=> (basically said found)
p.name = 'should not be able to be stored in database'
=> "should not be able to be stored in database"
p.save!
=> true # what????? Why can it update my object? It's lock in the other console!
lock! in testing2 doesn't lock while with_lock in testing does work. Can anybody explain why lock! doesn't work?
#lock! uses SELECT … FOR UPDATE to acquire a lock.
According to PostgreSQL doc.
FOR UPDATE causes the rows retrieved by the SELECT statement to be locked as though for update. This prevents them from being locked, modified or deleted by other transactions until the current transaction ends.
You need a transaction to keep holding a lock of a certain row.
Try
console1:
Plan.transaction{Plan.find(12).lock!; sleep 100.days}
console2:
p = Plan.find(12)
p.name = 'should not be able to be stored in database'
p.save
#with_lock acquire a transaction for you, so you don't need explicit transaction.
(This is PostgreSQL document. But I think other databases implement similar logic. )
Related
In Rails 6.1, ActiveStorage creates database records for all variants when they're loaded for the first time: https://github.com/rails/rails/pull/37901
I'd like to enable this, but since I have tens of thousands of files in my production Rails app, it'd be problematic (and presumably slow) to have users creating so many database records as they browse the site. Is there a way to write a Rake task that'll iterate through every attachment in my database, and generate the variants and save them in the database?
I'd run that once, after enabling the new active_storage.track_variants config, and then any newly-uploaded files would be saved when they're loaded for the first time.
Thanks for the help!
This is the Rake task I ended up creating for this. The Parallel stuff can be removed if you have a smaller dataset, but I found that with 70k+ variants it was intolerably slow when doing it without any parallelization. You can also ignore the progress bar-related code :)
Essentially, I just take all the models that have an attachment (I do this manually, you could do it in a more dynamic way if you have a ton of attachments), and then filter the ones that are not variable. Then I go through each attachment and generate a variant for each size I've defined, and then call process on it to force it to be saved to the database.
Make sure to catch MiniMagick (or vips, if you prefer) errors in the task so that a bad image file doesn't break everything.
# Rails 6.1 changes the way ActiveStorage works so that variants are
# tracked in the database. The intent of this task is to create the
# necessary variants for all game covers and user avatars in our database.
# This way, the user isn't creating dozens of variant records as they
# browse the site. We want to create them ahead-of-time, when we deploy
# the change to track variants.
namespace 'active_storage:vglist:variants' do
require 'ruby-progressbar'
require 'parallel'
desc "Create all variants for covers and avatars in the database."
task create: :environment do
games = Game.joins(:cover_attachment)
# Only attempt to create variants if the cover is able to have variants.
games = games.filter { |game| game.cover.variable? }
puts 'Creating game cover variants...'
# Use the configured max number of threads, with 2 leftover for web requests.
# Clamp it to 1 if the configured max threads is 2 or less for whatever reason.
thread_count = [(ENV.fetch('RAILS_MAX_THREADS', 5).to_i - 2), 1].max
games_progress_bar = ProgressBar.create(
total: games.count,
format: "\e[0;32m%c/%C |%b>%i| %e\e[0m"
)
# Disable logging in production to prevent log spam.
Rails.logger.level = 2 if Rails.env.production?
Parallel.each(games, in_threads: thread_count) do |game|
ActiveRecord::Base.connection_pool.with_connection do
begin
[:small, :medium, :large].each do |size|
game.sized_cover(size).process
end
# Rescue MiniMagick errors if they occur so that they don't block the
# task from continuing.
rescue MiniMagick::Error => e
games_progress_bar.log "ERROR: #{e.message}"
games_progress_bar.log "Failed on game ID: #{game.id}"
end
games_progress_bar.increment
end
end
games_progress_bar.finish unless games_progress_bar.finished?
users = User.joins(:avatar_attachment)
# Only attempt to create variants if the avatar is able to have variants.
users = users.filter { |user| user.avatar.variable? }
puts 'Creating user avatar variants...'
users_progress_bar = ProgressBar.create(
total: users.count,
format: "\e[0;32m%c/%C |%b>%i| %e\e[0m"
)
Parallel.each(users, in_threads: thread_count) do |user|
ActiveRecord::Base.connection_pool.with_connection do
begin
[:small, :medium, :large].each do |size|
user.sized_avatar(size).process
end
# Rescue MiniMagick errors if they occur so that they don't block the
# task from continuing.
rescue MiniMagick::Error => e
users_progress_bar.log "ERROR: #{e.message}"
users_progress_bar.log "Failed on user ID: #{user.id}"
end
users_progress_bar.increment
end
end
users_progress_bar.finish unless users_progress_bar.finished?
end
end
This is what the sized_cover looks like in game.rb:
def sized_cover(size)
width, height = COVER_SIZES[size]
cover&.variant(
resize_to_limit: [width, height]
)
end
sized_avatar is pretty much the same thing.
I'm using raw sql bulk updates (for performance reasons) in the context of a rake task. Something like the following:
update_sql = Book.connection.execute("UPDATE books AS b SET
stock = vs.stock,
promotion = vs.promotion,
sales = vs.sales
FROM (values #{values_string}) AS vs
(stock, promotion, sales) WHERE b.id = vs.id;")
While everything is "transparent" in local development, if this SQL fails in production during the execution of the rails task (for example because the promotion column is nil and the statement becomes invalid), no error is logged.
I can manually log this with catching the exception, like below, however some option that would allow for automatic logging would be better.
begin
...
rescue ActiveRecord::StatementInvalid => e
Rails.logger.fatal "Books update: ActiveRecord::StatementInvalid: "+ e.to_s
end
You can make your own custom class in your model folder:
app/models/custom_sql_logger.rb :
class CustomSqlLogger
def self.debug(msg=nil)
#custom_log ||= Logger.new("#{Rails.root}/log/custom_sql.log")
#custom_log.debug(msg) unless msg.nil?
end
end
Then go to the rake task where you would like to debug updated fields for example lib/task/calculate_avarages.rake and call your custom debugger:
CustomSqlLogger.debug "The field was successfully updated into DB"
Example from my project:
require 'rake'
task :calculate_averages => :environment do
products = Product.all
products.each do |product|
puts "Calculating average rating for #{product.name}..."
product.update_attribute(:average_rating, product.reviews.average("rating"))
CustomSqlLogger.debug "#{product.name} was susscefully updated into DB"
end
end
Custom debugger will create the new file custom_sql.log into log folder: log/custom_sql.log and saved all information there. Beware of a log file size after a while.
Is there a way to get changes applied on an ActiveRecord from the beginning until the end of the transaction? I tried previous_changes, but it just returns changes applied on the last operation on the ActiveRecord. For example:
Account.transaction do
account = Account.create(name: 'test')
account.update(last_name: 'testing')
end
I would like to know that 'name' and 'last_name' has changed in the transaction.
You can inspect ActiveModel::Dirty and just:
Account.transaction do
account = Account.create(name: 'test')
account.set_attributes(last_name: 'testing')
account.previous_changes[:last_name].present?
account.save!
end
I've got some STI in my data model. There are two types of Task records: PrimaryTask and SecondaryTask. So my ActiveRecord models look like this:
class Task < ActiveRecord::Base
end
class PrimaryTask < Task
has_many :secondary_tasks
end
class SecondaryTask < Task
belongs_to :primary_task
end
I want to provide a way to "promote" a SecondaryTask to a PrimaryTask permanently (as in, persisted in the database). From perusing the docs, looks like the #becomes! method is what I want, but I can't get it to save the changes in the database.
id = 1
secondary_task = SecondaryTask.find(id)
primary_task = secondary_task.becomes!(PrimaryTask)
primary_task.id # => 1
primary_task.class # => PrimaryTask
primary_task.type # => "PrimaryTask"
primary_task.new_record? # => false
primary_task.changes # => { "type"=>[nil,"PrimaryTask"] }
primary_task.save! # => true
primary_task.reload # => raises ActiveRecord::RecordNotFound: Couldn't find PrimaryTask with id=1 [WHERE "tasks"."type" IN ('PrimaryTask')]
# Note: secondary_task.reload works fine, because the task's type did not change in the DB
Any idea what's up? I tried the following things, to no avail. Am I misunderstanding becomes!?
Force the record to be 'dirty' in case the save! call was a no-op because none of the attributes were marked dirty (primary_task.update_attributes(updated_at: Time.current) -- didn't help)
Destroy secondary_task in case the fact that they both have the same id was a problem. Didn't help. The SecondaryTask record was deleted but no PrimaryTask was created (despite the call to save! returning true)
UPDATE 1
The logs show the probable issue:
UPDATE "tasks" SET "type" = $1 WHERE "tasks"."type" IN ('PrimaryTask') AND "tasks"."id" = 2 [["type", "PrimaryTask"]]
So the update is failing because the WHERE clause causes the record not to be found.
Figured it out. Turns out there was a bug in ActiveRecord version 4.0.0. It has since been patched. The key change this patch introduced was to set the changes correctly in both instances. So now you can call save on the original instance (in my example secondary_task) and it will change the type in the database. Note that calling save on the new instance (for me primary_task) will NOT save the changes, because of the behavior described in the question: it will include a WHERE clause in the SQL UPDATE call that will cause the record not to be found and thus the call to do nothing.
Here's what works with ActiveRecord > 4.1.0:
id = 1
secondary_task = SecondaryTask.find(id)
primary_task = secondary_task.becomes!(PrimaryTask)
secondary_task.changes # => { "type"=>["SecondaryTask","PrimaryTask"] }
primary_task.changes # => { "type"=>["SecondaryTask","PrimaryTask"] }
secondary_task.save! # => true
primary_task.reload # => works because the record was updated as expected
secondary_task.reload # => raises ActiveRecord::RecordNotFound, as expected
I have a loop like this:
# Iterate a list of items
req_wf_list.each do |req_wf|
# Begin a transaction
ReqWf.transaction do # ReqWf is an ActiveRecord model class
# Do some things
# ...
# 1. I want to be able to continue processing with the
# next iteration of the loop if there is an error here
# 2. I also want to rollback the transaction associated with
# this particular iteration if I encounter an error
begin
# Do something that might return an error
rescue
# Do some error processing
puts "Caught such and such error"
# Don't complete transaction (rollback),
# don't "do some more things",
# proceed to next item in req_wf_list
next
end
# Do some more things
# Shouldn't make it here if there is an error but I do indeed make it here
# ...
# End transaction
end
# End loop
end
Now, I would expect that calling "next" within the rescue block would cause the transaction associated with that particular iteration of the loop to rollback and for execution to resume at the top of the next iteration of the loop. Instead, execution appears to resume at the "Do some more things" line. It is as if the "next" statement is completely ignored. What am I missing?
Most likely that in this case next applies to transaction so you are in a nested loop situation.
This is an example of what can be done to solve the issue
req_wf_list.each do |req_wf|
catch :go_here do #:missingyear acts as a label
ReqWf.transaction do
throw :go_here unless something #break out of two loops
end
end #You end up here if :go_here is thrown
end
But in general, it is not a good practice to use next. You should be able to put a global begin .. rescue and have all the conditions inside of it, so that nothing else gets executed once you catch an error.
Update
I did some a small test and the behavior is as you expect it.
loop = [1,2,3]
loop.each do |value|
puts "value => #{value}"
ActiveRecord::Base.transaction do
puts "Start transaction"
begin
raise
rescue
puts "ActiveRecord::StatementInvalid"
next
end
puts "Should not get here!"
end
end
The output is the following:
value => 1
Start transaction
ActiveRecord::StatementInvalid
value => 2
Start transaction
ActiveRecord::StatementInvalid
value => 3
Start transaction
ActiveRecord::StatementInvalid
Is it possible that you had another error in your code before the next was being called ?
In any case, using the next statement is not the best option as I said before.