ActiveJob could not find record in `test` environment - ruby-on-rails

This issue only exists in test environment. Everything runs fine in development environment.
I am facing a strange issue after recently upgrading to Rails 5.0.0.1 from Rails 4.2.7.1. Everything was working fine before this upgrade.
In one of my models, I use ActiveJob to perform a task.
# webhook_invocation.rb
def schedule_invocation
WebhookRequestJob.perform_later(id)
end
def init
remember_webhook # No DB changes
init_errors_context # No DB changes
flow_step_invocation.implementation = self
flow_step_invocation.save!
return unless calculate_expressions # No DB changes
calculated! # An AASM event, with no callbacks
schedule_invocation
end
and in WebhookRequestJob#perform, I retrieve the object using the ID supplied
# webhook_request_job.rb
def perform(webhook_invocation_id)
invocation = WebhookInvocation.find_by(id: webhook_invocation_id)
invocation.run_request
end
The problem is that in the #perform, it cannot find the record (invocation becomes nil). I even tried putting p WebhookInvocation.all as the first line, but all it prints is an empty collection. On the other hand, if I try p WebhookInvocation.all in #schedule_invocation method, it properly prints out all the objects of WebhookInvocation.
There is no exception being raised, no lines of warnings either.
Edit 1:
I even tried passing the object directly to #perform_later i.e. WebhookRequestJob.perform_later(self), but the received object at #perform is nil.
Edit 2:
I noticed that there are some messages like Creating scope :fail. Overwriting existing method FlowStepInvocation.fail, caused by using AASM. I eliminated them by using create_scopes: false. But that still didn't solve the problem.

My guess from the info you supplied is that you have are calling the schedule_invocation method in a after_save or after_create callback. Since the callback is called, ActiveJob might start processing the job even before the object is actually persisted (before COMMIT is done). In this case your record will not show up in the database when job is processed and you will get an empty collection or nil.
To fix this change your callback to after_commit to make sure that the COMMIT action has happened before you queue the job.

It turns out that config.active_job.queue_adapter was set to :inline as default before Rails 5, but it is set to :async in Rails 5.
This made the specs to fail (don't know why). To resolve this, I put the following line in my config/environments/test.rb:
config.active_job.queue_adapter = :inline

Related

How can I prevent any ActiveRecord::PreparedStatementCacheExpired errors immediately after running `rake db:migrate`?

I am working on a Rails 5.x application, and I use Postgres as my database.
I often run rake db:migrate on my production servers. Sometimes the migration will add a new column to the database, and this causes some controller actions to crash with the following error:
ActiveRecord::PreparedStatementCacheExpired: ERROR: cached plan must not change result type
This is happening in a critical controller action that needs to have zero downtime, so I need to find a way to prevent this crash from ever happening.
Should I catch the ActiveRecord::PreparedStatementCacheExpired error and retry the save? Or should I add some locking to this particular controller action, so that I don't start serving any new requests while a database migration is running?
What would be the best way to prevent this crash from ever happening again?
I was able to fix this issue in some places by using this retry_on_expired_cache helper:
class ApplicationRecord < ActiveRecord::Base
self.abstract_class = true
class << self
# Retry automatically on ActiveRecord::PreparedStatementCacheExpired.
# (Do not use this for transactions with side-effects unless it is acceptable
# for these side-effects to occasionally happen twice.)
def retry_on_expired_cache(*_args)
retried ||= false
yield
rescue ActiveRecord::PreparedStatementCacheExpired
raise if retried
retried = true
retry
end
end
end
I would use it like this:
MyModel.retry_on_expired_cache do
#my_model.save
end
Unfortunately this was like playing "whack-a-mole", because this crash just kept happening all over my application during my rolling deploys (I'm not able to restart all the Rails processes at the same time.)
I finally learned that I can turn off prepared_statements to completely avoid this issue. (See this other question and answers on StackOverflow.)
I was worried about the performance penalty, but I found many reports from people who had set prepared_statements: false, and they hadn't noticed any problems. e.g. https://news.ycombinator.com/item?id=7264171
I created a file at config/initializers/disable_prepared_statements.rb:
db_configuration = ActiveRecord::Base.configurations[Rails.env]
db_configuration.merge!('prepared_statements' => false)
ActiveRecord::Base.establish_connection(db_configuration)
This allows me to continue setting the database configuration from the DATABASE_URL env variable, and 'prepared_statements' => false will be injected into the configuration.
This completely solves the ActiveRecord::PreparedStatementCacheExpired errors and makes it much easier to achieve high-availability for my service while still being able to modify the database.

Rails 6 & deliver_later doesn't affect ActionMailer::Base.deliveries

After upgrading to Rails 6 I am noticing that default mailer's .deliver_later is not working the same as in Rails 5.
Configuration:
config.active_job.queue_adapter = :inline
When running Mailer.register_email(...).deliver_later - nothing is stored in ActionMailer::Base.deliveries. This array gets filled if I run perform_enqueued_jobs - it seams like queue_adapter = :inline doesn't work the way I expect it to work.
If I run Mailer.send(...).deliver_now then ActionMailer::Base.deliveries has proper value in it.
Any idea why this is happening and how to solve this?
I had same problem in my tests. A search on the Internet yielded nothing, so I started experimenting.
I tried wrapping the call method of sending mail in
assert_emails 1 do
Mailer.register_email(...).deliver_later
end
After that, ActionMailer::Base.deliveries populated correctly.
If the exact number of emails could easily change, this is another option:
assert_changes 'enqueued_jobs.size' do
# Some code that sends email with deliver_later
end
This allows you to test that emails were sent but it disregards the exact number (which is a limitation of the asserts_emails method - other than this, the asserts_emails method is great).
I found that the enqueued_jobs method is very helpful in testing anything background jobs, including deliver_later
NOTE: the above example only checks that the enqueued jobs list was changed. If you want to be more specific and check that the queue was changed with emails, you should do this:
assert_changes 'enqueued_jobs.select {|job| job["job_class"] == "ActionMailer::MailDeliveryJob"}.size' do
# Some code that sends email with deliver_later
end
The issue
The issue lies in two new lines of code added to Rails 6 (line 1 & line 2),
where basically, the callback before_setup defined here (in RSpec) and here (in Minitest) gets overridden (by this), thus forcing the queue_adapter to be the test adapter instead of the one defined by config.active_job.queue_adapter.
Workaround
So in order to use the queue_adapter defined by config.active_job.queue_adapter and therefore restore the Rails 5 behaviour we can do something like the below.
# spec/support/active_job/test_helper.rb
module ActiveJob
module TestHelper
def before_setup
super
end
end
end

Rails spring wisper listener method caching

It turns out that Spring caches my wisper listener method (I'm writing quite simple Engine).
Example:
app/models/myengine/my_class.rb
class Myengine::MyClass
include Wisper::Publisher
def something
# some logic
publish(:after_something, self)
end
end
config/initializers/wisper.rb
Wisper.subscribe(Myengine::MyObserver.new)
app/observers/myengine/my_observer.rb
class Myengine::MyObserver
def after_something my_class_instance
# any changes here requires Spring manual restart in order to be reflected in tests
another_method
end
def another_method
# all changes here or in any other class methods works ok with Spring and are instantly visible in tests
return true
end
end
By Spring restart I mean manual execution of spring stop command which is really annoying.
What is more mysterious I may change another_method return value to false and then tests are failing which is OK, but when I change after_something method body to let say return false it doesn't have any effect on tests (like the body of the after_something is somehow cached).
It is not high priority problem because this strange behaviour is only visible inside listener method body and easy to overcome by moving all logic to another method in the class. Anyway it might be confusing (especially at the beginning when I didn't know the exact problem).
The problem is properly caused because when you subscribe a listener globally, even if its class is reloaded, the object remains in memory pointing to the class it was originally constructed from, even if the class has been reloaded in the meantime.
Try this in config/initializers/wisper.rb:
Rails.application.config.to_prepare do
Wisper.clear if Rails.env.development?
Wisper.subscribe(Myengine::MyObserver.new)
end
to_prepare will run the block before every request for development environment, but once, as normal for production environment. Therefore provided your listener does not maintain any state it should work as expected.
The Wisper.clear is needed to remove the existing listeners subscribed before we re-subscribe a new instance from the reloaded class. Be aware that #clear will clear all subscribers, so if you have similar code as the above in more than one engine only the last engine to be loaded will have its listeners subscribed.

Rails 3 - Delayed_Job

I'm working to learn how to user delayed_job on my rails 3 + heroku app.
I currently have the following which emails on a request (not delayed job) but it works!
UserMailer.conversation_notification(record.commentable, participant, record, #comments).deliver
I updated that to this to start using delayed_job:
Delayed::Job.enqueue UserMailer.conversation_notification(record.commentable, participant, record, #comments).deliver
But that error'd with: "ArgumentError (Cannot enqueue items which do not respond to perform):"
I also tried:
UserMailer.delay.conversation_notification(record.commentable, participant, record, #comments)
But that error'd with:
NoMethodError (undefined method `delay' for UserMailer:Class):
Any delayed_job guru's out there? Thanks
From the docs https://github.com/collectiveidea/delayed_job
Your second method was correct which removes the .deliver method:
UserMailer.delay.conversation_notification(record.commentable, participant, record, #comments)
If you are getting an undefined method delay did you add DelayedJob to the Gemfile?
gem "delayed_job"
Since including the delayed_job will add the "delay" method to everything.
I have mixed results with using delay, and I've found it very challenging to debug. So you are not alone! But when you get it working, its worth it.
I've learned to save my object before calling delay on it. Typically I will trigger my job from an after_save call back.
As an experiment, for awhile I was using a different pattern. I'd create a job object for each job that I have. For example, I would call
Delayed::Job.enqueue(PersonJob.new(#person.id))
Elsewhere in my project I would create the job object. In Rails 2, I put these in lib/ if you do that with rails 3, you need to alter the application.rb config.autload_path
class PersonJob < Struct.new(:person_id)
def perform
person = Person.find(person_id)
#do work
end
end
config.autoload_paths += Dir["#{config.root}/lib/**/"]
I just had a look at the documentation, it's been a while since I actually used delayed_job...
Jobs are Ruby objects with a method called perform, so you'd need enqueue an object which does
UserMailer.conversation_notification(record.commentable, participant, record, #comments).deliver
in its perform method.
Alternatively, you can use send_later:
UserMailer.conversation_notification(record.commentable, participant, record, #comments).send_later(:deliver)

Rails test db doesn't persist record changes

I've been trying to solve a problem for a few weeks now. I am running rspec tests for my Rails app, and they are working fine except for one error that I can't seem get my head around.
I am using MySQL with the InnoDB engine.
I have set config.use_transactional_fixtures = true in spec_helper.rb
I load my test fixtures manually with the command rake spec:db:fixtures:load.
The rspec test is being written for a BackgrounDRb worker, and it is testing that a record can have its state updated (through the state_machine gem).
Here is my problem:
I have a model called Listings. The rspec test calls the update_sold_items method within a file called listing_worker.rb.
This method calls listing.sell for a particular record, which sets the listing record's 'state' column to 'sold'.
So far, this is all working fine, but when the update_sold_items method finishes, my rspec test fails here:
listing = Listing.find_by_listing_id(listing_id)
listing.state.should == "sold"
expected: "sold",
got: "current" (using ==)
I've been trying to track down why the state change is not persisting, but am pretty much lost. Here is the result of some debugging code that I placed in the update_sold_items method during the test:
pp listing.state # => "current"
listing.sell!
listing.save!
pp listing.state # => "sold"
listing.reload
pp listing.state # => "current"
I cannot understand why it saves perfectly fine, but then reverts back to the original record whenever I call reload, or Listing.find etc.
Thanks for reading this, and please ask any questions if I haven't given enough information.
Thanks for your help,
Nathan B
P.S. I don't have a problem creating new records for other classes, and testing those records. It only seems to be a problem when I am updating records that already exist in the database.
I suspect, like nathan, transaction issues. Try putting a Listing.connection.execute("COMMIT") right before your first save call to break the transaction and see what changes. That will break you out of the transaction so any additional rollback calls will be non-effectual.
Additionally, by running a "COMMIT" command, you could pause the test with a debugger and inspect the database from another client to see what's going on.
The other hypothesis, if the transaction experimentation doesn't yield any results, is that perhaps your model really isn't saving to the database. Check your query logs. (Specifically find the update query).
These kind of issues really stink! Good luck!
If you want to investigate what you have in DB while running tests you might find this helpful...
I have a rspec test where I save #user.save and it works like a charm, but then I wanted to see if it's really saved in the DB.
I opened rails console for test environment
rails c test
ran
User.all
and as expected got nothing
I ran my spec that contains:
user_attr_hash = FactoryGirl.attributes_for(:user)
#user = User.new user_attr_hash
#user.save
binding.pry
I thought that stopping the test after save would mean that it's persisted, but that's not the case. It seems that COMMIT on the connection is fired later (I have no idea when:\ )
So, as #Tim Harper suggests, you have to fire that commit yourself in the pry console:
pry(#<RSpec::Core::ExampleGroup::Nested_1>)> User.connection.execute("COMMIT")
Now, if you run User.all in your rails console you should see it ;)

Resources