I am using Redis + Resque in production and want to test that jobs are getting queued and run properly. I am looking for something like this
Resque.jobs(:queue_name).size.should == 0
post :some_action # This action causes a Resque job to be enqueued
# Test Enqueuing
Resque.jobs(:queue_name).size.should == 1
Resque.jobs(:queue_name).last.klass.should == "MyJob"
Resque.jobs(:queue_name).last.args.should == [1, "Arg_2"]
# Test performing
Resque.jobs(:queue_name).perform_all
# test the effect of running the job
How do I start Redis + Resque in test environment? I don't people to manually run a redis server all the time. I have tried the solution where you try and run the redis server in config.before(:suite) but the redis-server never starts up in time and the Resque complains that it can't connect to Redis.
I have tried using Resque.inline but 1) It doesnt let me test that the Job was enqueued 2) It always enqueues the job inside the :inline queue (I want to test that the job ends up in the correct queue).
Personally, I rely on gems which I include in my project, including Resque and Redis, to be tested by the developers who write them. As a result, I do not include testing them in my test suite. For example, when choosing a gem for my application, I look at the gem's documentation to see if TravisCI / Code Climate / etc. statistics are included and if the project is "green." If it is, I use it. If it's not, I look for an earlier (e.g. more stable) version, or look for alternatives. In the case of Resque and Redis for Rails, both of these are well maintained and popular, thus extremely stable.
For my apps, I simply write tests where I present expectations of messages being called to Resque / Redis. For example:
it "should make a call to Resque for #my_job" do
expect(Resque).to_receive(:enqueue).with(SomeJob, args)
my_method_which_calls_resque
end
Then, assuming that you have a method which you are testing called my_method_which_calls_resque that looks something like:
def my_method_which_calls_resque
...
Resque.enqueue(SomeJob, args)
...
end
This test should be successful.
For additional documentation on messages and setting RSpec expectations, see RelishApp's docs on message expectations.
Then, if you wish to test your code within the Resque job itself, you can create a RSpec test for your job. Example:
# spec/lib/jobs/some_job_spec.rb
describe Jobs::SomeJob do
describe "#perform" do
it "should update someone's account" do
...
end
end
end
Related
For context, this question arose because we are migration from Rails 5 to Rails 6, and introducing reader / writer database connections via the new replication features.
Our specific problem is with request specs, with an eye towards using transactional fixtures. When we run our request specs files in isolation, they pass. When run as part of a multiple-file pass (such as a full bundle exec parallel_rspec pass used on circle CI) they fail. If we turn off transactional fixtures, the tests take far too long to run, but pass.
Using byebug, we've poked in and determined that the problem is that our test data has been written to / is accessible by the writer DB connection, but the route is attempting to use the reader DB connection to read it. I. E. ActiveRecord::Base.connected_to(role: :reading) { puts Foo.count } is 0, while the same code connecting to writing role is non-zero.
The problem from there seems fairly obvious: because we're using transactional tests / fixtures, the code is never committed to the DB. It's only available on the connection it was made on. The request spec is reading from the 'right' db for the call (a GET request should use the reader db), but in the use-case of tests that's producing errors.
It seems like this is a fairly obvious use case that either Rails or rspec should have a tool for handling, we just don't seem to be able to find the relevant documentation.
You need to tell the test environment that it should be using a single connection for both. There are multiple ways of doing this:
You can configure your test environment not to use replicas at all. See Setting up your application for examples of using a replica and not using a replica then reproduce the non-replica version in your database.yml for the test environment only.
You can use connected_to within your specs themselves so that those tests are forced to use the specific connection you want them to use. One way to do this is with around hooks:
describe "around filter" do
around(:each) do |example|
puts "around each before"
ActiveRecord::Base.connected_to(role: :writing) { example.run }
puts "around each after"
end
it "gets run in order" do
puts "in the example"
end
end
You can monkey patch your ActiveRecord configuration in rails_helper so that it doesn't use replicas (but I'd really recommend #1 over this option)
In Rspec, I'm creating records, e.g. let!(:user) { create(:user) }.
I can access the new user in Rspec and in the main thread of the subject. Both of these return the user:
puts User.all.to_a
puts ActiveRecord::Base.connection.exec_query('select * from users')
However, I can now longer access the user in a new thread:
Thread.new do
puts User.all.to_a
puts ActiveRecord::Base.connection.exec_query('select * from users')
end
How do I fix this? Is this just an Rspec issue? I don't think I can reproduce it in Rails console.
You have probably configured RSpec to run its test within a database transaction
Quote from the RSpec Docs:
>
When you run rails generate rspec:install, the spec/rails_helper.rb file
includes the following configuration:
RSpec.configure do |config|
config.use_transactional_fixtures = true
end
The name of this setting is a bit misleading. What it really means in Rails
is "run every test method within a transaction." In the context of rspec-rails,
it means "run every example within a transaction."
The idea is to start each example with a clean database, create whatever data
is necessary for that example, and then remove that data by simply rolling back
the transaction at the end of the example.
You might want to disable this configuration when your application tests have the requirement to support multiple threads at the same time.
But that means that your test database will not be cleared from test data automatically anymore when a test is done. Instead, you will need to delete the test data manually or use another gem (like database_cleaner) to reset the database after running tests.
Like a lot of non-greenfeild projects, I arrived at my new job to find that most tests are performed using feature specs and not controller/model specs. Naturally this means that the tests take a really long time to execute.
Is there a way I can setup RSpec so that my feature tests run last - and they will only run if all other tests first succeed. I am migrating the application over to controller tests to speed execution, but there is no need to wait the 15mins for the feature tests to complete if there is an issue with model validation or one of my controllers is busted.
Is what I want to do possible?
(I am running the tests in RubyMine at the moment; but I can move to rake spec if need be).
Although I haven't actually tried this myself, reading through the documentation it appears it is possible to do what you wish.
Add the following to your spec_helper.rb to override the global ordering and force rspec to fail on first failure.
RSpec.configure do |config|
config.register_ordering :global do |examples|
feature, other = examples.partition do |example|
example.metadata[:type] == :feature
end
other + feature
end
config.fail_fast = true
end
When running minutest tests, is it possible to peek at the information about the errors that has happened?
For example, this test suite takes ten minutes to complete. But I would like some more info about the letter E appearing in the tests result.
I don't want to wait ten minutes.
*** Running FRONTEND component engine specs
Run options: --seed 29704
# Running:
......................................................................................................................................................................................E...........
That's E for "error", so one of your tests is failing. Normally you get output that explains more. Once you identify which test is failing you can run that test in a more focused capacity, like:
ruby test/unit/broken_test.rb --name=test_that_is_broken
Where that is the path to your test script and the name of the testing method that's failing.
You may need to make your tests self-contained, able to be run this way, by using:
require_relative '../test_helper'
Or whatever the helper stub is that kicks off the testing framework. Some skeleton files contain things like require 'test_helper' which won't be found in your current $LOAD_PATH.
I would like to test some resque workers, and the actions that enqueue jobs with these workers. I am using rspec and rails.
Currently I have a model, let's call it Article.rb, that has a before_save method called updates_related_categories that checks if a job with CategoriesSorter needs to be enqueued. If so, it enqueues a job with that worker, with the argument of the category id that the article is related to.
In test, however, these jobs are sent to the same queue as the development server sends jobs to. (i check using the resque server that you can tie into your server at root/redis/overview)
I want to know:
1) How can I send test jobs to a different queue than the development jobs?
If this is possible, any other advice on testing resque is also welcome.
I have seen some related questions that suggest resque-unit and resque-spec but these are pretty undeveloped, and I couldn't get them to a useful working state. Also, I have heard of using Resque.inline but I don't know if that is relevant in this case, as resque isn't called in test specs, it's called from the article model on save of objects created in test.
Sample Code:
Article.rb:
before_save :update_related_categories
def update_related_categories
#some if statements/checks to see if a related category needs updating
Resque.enqueue(CategoriesWorker, [category_id])
end
end
CategoriesSorter:
class CategoriesSorter
#queue=:sorting_queue
def self.perform(ids)
ids.each do |id|
#some code
end
end
end
Specs:
it "should do something" do
#article = #set something to enqueue a job
#article.save
#can i check if a job is enqueued? can i send this job NOT to the development queue but a different one?
end
As I see it, you don't want to test whether enqueueing the job results in perform being called - we trust Resque does what it should.
However, you can test that 1. calling update_related_categories enqueues the job. Then you can test, separately, whether 2. a worker calling perform results in the desired behavior.
For testing Resque in general, a combination of resque-spec and simulating a worker can accomplish the above two goals.
For 1, with resque-spec, you can simulate a worker calling the perform method on your class, and then check that it has been enqueued correctly:
describe "Calling update_related_categories " do
before(:each) do
ResqueSpec.reset!
end
it "should enqueue the job" do
Article.update_related_categories
CategoriesSorter.should have_queue_size_of(1)
end
end
For 2, you can create a Job (and specify a separate queue name than your development queue), a Resque::Worker and then assign the Worker to the Job:
def run_worker_simulation
# see Resque::Job api for setting the args you want
Resque::Job.create('test_queue_name', 'class_name', 'type', 'id')
worker = Resque::Worker.new('test_queue_name')
worker.very_verbose = true
job = worker.reserve
worker.perform(job)
end
Hope that gives you some ideas.
You shouldn't test resque, that's the resque development team job, your application should only test that your perfom method does what it have to do. You can also test that your model Article.rb enqueues the job as you want. Sending real jobs to resque is useless, your test will end and the reque queue will be full of useless jobs.
Do something like:
describe 'Article' do
before(:each) do
Resque.stub!(:enqueue)
end
it 'enqueues the job' do
cat_id = 1
Resque.should_receive(:enqueue).once.with(CategoriesWorker, [cat_id])
Article.create(:category_id => cat_id)
end
end
describe 'CategoriesSorter' do
it 'sorts the categories' do
result = CategoriesSorter.perform([1,4,6,3,2])
result.should == [1,2,3,4,6]
end
end
stub or mock unneeded methods/classes
EDIT: also, as you test Article, it's good that you set a before(:each) filter to stub resque so your spec never send jobs to the real queue, i've edited my answer
I have
Resque.inline = ENV['RAILS_ENV'] == "test"
This makes all the resque tasks as inline in test environment.
For testing each Job class , I have separate specs to test perform method of each job separately.