I've got an interesting problem that's causing myself and my team a lot of headaches when it comes to running our spec suite.
Our spec suite is broken up into the following sub-folders, and next to them is their total execution time to completion:
rspec spec/acceptance 311.67s
rspec spec/controllers 18.97s
rspec spec/decorators 4.39s
rspec spec/helpers 9.45s
rspec spec/lib 16.88s
rspec spec/mailers 5.27s
rspec spec/models 121.05s
rspec spec/presenters 0.03s
rspec spec/workers 19.3s
Total run time: 8m 27s
Which certainly could be improved, but all in all is pretty managable.
However, if, I run rspec spec and run the entire suite at once, the total time to complete is 27m 11s!
So, obviously something we are doing is dramatically affecting the performance of the entire suite when run at once. I'm hoping that I can get some pointers as to where I can begin to try to troubleshoot this problem.
If it helps, I've posted my spec_helper.rb file here
Thanks in advance,
Joe
At a guess, I'd say your integration specs are setting DatabaseCleaner to :truncation and it's not getting switched back to :transaction for the other specs. I have a sample spec_helper that navigates the situation here. There are also a couple methods provided that help you dig in and figure out which strategy is being used at any given time. Here it is for your convenience:
def cleaner_strategy
active_record_cleaner.instance_variable_get(:#strategy).class
end
def active_record_cleaner
DatabaseCleaner.instance_variable_get(:#cleaners)[[:active_record, {}]]
end
Side note: we use a filter for our integration specs since they are so slow. We disable them by default in our local env, and then let CI run them.
config.filter_run_excluding :slow unless ENV['SLOW']
Then you can run them individually with:
$ SLOW=true rspec spec/features/some_awesome_feature_spec.rb
What does profiling tell you?
Run rspec spec --profile or add --profile to your ~/.rspec file.
On completion of your tests, it will report the 10 slowest tests.
Related
I am using the out of the box, vanilla test suite for Rails 5 beta. I am wondering if anyone has figured out how to run a global setup, and tear down.
The reason this is required to I am spinning up a in memory Elasticsearch cluster before any test begins and stop the cluster once the tests are done.
Rspec is not an option.
Under Minitest (which is the default testing environment in Rails 4+), to get the "global setup" behavior, simply run anything in your test_helper.rb (outside the tests themselves or any setup methods), i.e. in the file where you load your testing environment from. The test helper is usually required in the tests, so its code is run once before any tests.
For a "global teardown", Minitest provides the Minitest.after_run method. Anything inside its block will be run once after all tests are finished (it uses the program exit hook). Place it e.g. in the test_helper again. For this to work you need to require 'minitest/autorun' at the beginning of the test helper file.
test/test_helper.rb
class ActiveSupport::TestCase
# Some pre-generated stuff here
setup do
do_something
end
teardown do
do_something
end
end
When testing lib code there is rarely any need to require spec_helper and load all of rails. This is why I have been removing require "spec_helper" in favour of require_relative "../../lib/my_lib.rb".
These tests pass when called directly (rspec spec/lib/my_lib.rb) and are blazingly fast. Winner.
My issue comes when I try and run these tests as a group.
When I call rspec spec/lib it runs any lib specs that have a require "spec_helper" line but not any tests that don't.
I have played with spec_helper.rb to load in these tests, and that kind of works only it means that when I run rspec spec/models/blah.rb it will also run these lib tests, which obviously isn't what I want.
Is there a different way I should be calling my tests? Or is there a way I can get them added to the test run?
Notes
My spec_helper is configured to run the tests in a random order, I wonder if this has anything to do with it?
You should add _spec to your files:
spec/lib/my_lib_spec.rb
Rspec rake task will look only for files which end in _spec.
I have some slow RSpec tests that I've tagged with :slow. I've set RSpec up to skip them default by adding the line config.filter_run_excluding slow: true to my RSpec configuration. And I can run the slow tests by running rspec --tag slow.
But how can I run all the tests using one command, including slow and non-slow tests? I can't figure it out from the docs.
You can find a similar question here : Command line to run all examples in RSpec, including ones that are filtered out?
In few words, this feature doesn't exist on rspec but you can use env variable :
RSpec.configure do |c|
c.filter_run_excluding slow: true unless ENV['ALL']
end
Call ALL=1 rspec will run all the specs including the slow tag.
For a couple of months now I have the problem that running rake spec cucumber (or each one separately) runs the specs and/or features always twice. It's kind of annoying, since it makes me wait longer.
Now if I run rspec or cucumber then the specs/or features only run a single time.
I'm not sure what details I could provide here, so I'll just link the repo here https://github.com/deiga/new-Roydon/tree/feature/allow-orders-without-registeration
I have a small test project that I'm using to test the waters for a much larger project. I am using rspec on rails for testing, but recently looked into Cucumber. It looks very nice, but I'm wondering if there's a way for cucumber to run my spec tests, or for rspec (autospec) to run my cucumber features. I've looked around extensively, but have yet to find a solid conclusion.
Thanks,
Mike
I've been experimenting with Cucumber as well. It supports autotest:
AUTOFEATURE=true autospec
That runs both the rspec & cucumber test suite continuously.
An easy way to do this would be to create a Rake task that invokes both tools, such as this minimal example:
desc 'Run rspec + cucumber'
task :build => [:spec, :features]
Then you can build both with:
rake build
Both RSpec and Cucumber come with some default tasks which work with Rails, but you can customize the tasks to suit your needs. There's more info on writing rake tasks here.
Depends on what you are looking for, but the best way to run cucumber tests with rspec is to use turnip, which uses exactly the same feature syntax as cucumber (the syntax is called gherkin), but allows you to use most of the functionality of rspec. https://github.com/jnicklas/turnip.
If you want to take things one step further, you can run your features from inside rspec _spec.rb files using a gem we created called rutabaga. https://github.com/simplybusiness/rutabaga
In both cases, all the tests can be run by executing rspec or bundle exec rspec as needed.