I would like to run some specs that are marked with :first in the beginning of an RSpec session. So, if I have:
describe 'one' { ... }
describe 'two', :first { ... }
describe 'three' { ... }
the order in which I want specs to be run is two, one, three.
Can this be done with RSpec?
Note: I know it's counterproductive to run tests in a certain order, but I need a quick workaround for our project so that all specs pass (unfortunately some specs are poorly written and fail depending on the order in which they are run).
RSpec has an --order flag you can pass to define a specific order. By default, this flag understands only a couple of strategies, such as random or seed, where the seed is a specific instance of a random order.
However, if you dig into the Ordering module, you will see you can define new ordering strategies.
There is a Custom strategy which apparently orders by a specific block. These strategies are managed by an ordering register, and dependent on the :order config.
You can investigate the internals, define a custom order strategy based on your rules, and make it the default.
To run your tests in a specific order you need to arrange the tests in the order you would like them to be run then run the rspec command with the --order default flag:
rspec --order default
If you run rspec without this flag the tests will be run in a random order (as I'm sure you've noticed).
Source:
The release notes from 2012 when Rspec added this flag - http://blog.davidchelimsky.net/blog/2012/01/04/rspec-28-is-released/
Related
I use before(:all) in my tests for database initialization, which causes no problems for my test cases... except one. I'd like to run this special case before or after others, as I want to initialize database for it separately. Is there some way to do this? I'd like to avoid replacing before(:all) with before(:each), because I'd like to keep my tests fast. Is it possible?
before(:all) except one - is it possible?
Probally not. before(:all) runs before all the examples in that scope and there is no way to hoist examples above it.
I think a better idea is to work around the problem and just create different contexts:
RSpec.describe Thing do
context "with :all" do
before(:all) do
# ...
end
# ...
end
context "without :all" do
# ...
end
end
Use shared contexts or the outer scope if you need to share setup steps or variables. Use shared examples if you want to run the same examples in different contexts.
I'd like to run this special case before or after others, as I want to
initialize database for it separately. Is there some way to do this?
You can use the --order defined option when running rspec to run the tests sequentially. However this sets you up for test ordering issues than can mask critical bugs in the application. Fast test are worthless if they don't catch bugs.
You can also use :order => :defined metadata to set the order per context. And for the reason in the beginning of this question this is a fools errand.
I want to run feature specs written in rspec/capybara in a fixed sequence as follows:
signup_spec.rb
login_spec.rb
project_creation_spec.rb
project_migration_spec.rb
The --order feature of given here says that
Use the --order option to tell RSpec how to order the files, groups, and
examples
How would I use .rspec file to mention pass my requirements ?
I have a shell script with test cases running in a sequence like:
rspec spec/features/signup_spec.rb
rspec spec/features/login_spec.rb
rspec spec/features/project_creation_spec.rb
rspec spec/features/project_migration_spec.rb
The order option does not allow this. It allows switching between the default ordering (which is essentially that they are run in the order they are defined which in turns depends on file system ordering ) or random ordering (optionally with a seed)
I would consider any order dependence to be a bug - the random ordering option is there to flush out such bugs.
Folks,
I am having some trouble working with the Afterhook. I have organized my tests in folders like this:
features/Accounts/accounts_api.feature
features/Accounts/step_definition/account_steps.rb
features/labs/create_lab.feature
features/labs/step_definition/labs_steps.rb
Now I have an After hook present in the step definition of the Accounts feature, I want that hook to run after every scenario of the "Accounts" feature, but I do not want it to run after every scenario of the "labs" feature. I tried this:
cucumber --tags #newlabs
the above should run all the scenarios present in the labs feature tagged as newlabs but what I am seeing is that once the scenario tagged as#newlabs runs the #after hook present in the step definition of Accounts starts to run. I am thinking why is this happening, am I using the hook in the wrong way or is my overall understanding of hooks wrong?
Thanks a lot for taking the time to respond, this helps a lot.
Hooks don't care what step definition script they're located in and will run for every scenario. Or, more specifically, your after hook will run after every scenario that runs, for every feature, regardless of the tags you pass in to Cucumber.
If you want a little more control over that, check out the Cucumber wiki page on hooks and look in the section called 'Tagged hooks'.
Possibly you define After hook in wrong place. Note that After hook (as well as other hooks) must be defined in the .rb, not in the .feature file. Common place for hooks is features/support/hooks.rb. You will define your hook this way:
# features/support/hooks.rb
After('#newlabs') do # will run after each scenario tagged with #newlabs
# your teardown ruby code
end
# features/Accounts/accounts_api.feature
#newlabs # tag all scenarious of this feature with #newlabs tag
Feature: your feature
Scenario: your scenario
Given: ...
When: ...
Then: ...
In cucumber output you won't see that After hook is executed (unless you output something to STDOUT from the hook definition) - hooks will run implicitly.
I have a scenario outline with multiple scenarios. I'd like my Before hook to run only once so I can bootstrap the ActiveRecord objects I need to run against all of the scenarios. The problem is if I use
Before do
# my code here
end
This will execute before each Scenario. Is there anyway to run it once for the entire Outline?
I think if you simply create the objects in a file in features/support they will be persisted:
ImportantThing.create(:name => "USEFUL THING")
This is because before every Scenario Cucumber will start a database transaction and then rollback to its prior status, which should contain the objects you've loaded.
I had the same problem, where I needed to create a subscriber manager once for all of my event logging tests. If I just used a before hook or a regular step (e.g. a Given), the manager would be created before each scenario.
My solution was ultimately to use a tagged before hook on my first scenario.
Before('#first_logging_scenario') do
# do something useful
end
To shutdown my manager, I used a tagged After hook with my last scenario
After('#last_logging_scenario') do
# do something useful
end
In one of my projects I need to collaborate with several backend systems. Some of them somewhat lacks in documentation, and partly therefore I have some test code that interact with some test servers just to see everything works as expected. However, accessing these servers is quite slow, and therefore I do not want to run these tests every time I run my test suite.
My question is how to deal with a situation where you want to skip certain tests. Currently I use an environment variable 'BACKEND_TEST' and a conditional statement which checks if the variable is set for each test I would like to skip. But sometimes I would like to skip all tests in a test file without having to add an extra row to the beginning of each test.
The tests which have to interact with the test servers are not many, as I use flexmock in other situations. However, you can't mock yourself away from reality.
As you can see from this question's title, I'm using Test::Unit. Additionally, if it makes any difference, the project is a Rails project.
The features referred to in the previous answer include the omit() method and omit_if()
def test_omission
omit('Reason')
# Not reached here
end
And
def test_omission
omit_if("".empty?)
# Not reached here
end
From: http://test-unit.rubyforge.org/test-unit/en/Test/Unit/TestCaseOmissionSupport.html#omit-instance_method
New Features Of Test Unit 2.x suggests that test-unit 2.x (the gem version, not the ruby 1.8 standard library) allows you to omit tests.
I was confused by the following, which still raises an error to the console:
def test_omission
omit('Reason')
# Not reached here
end
You can avoid that by wrapping the code to skip in a block passed to omit:
def test_omission
omit 'Reason' do
# Not reached here
end
end
That actually skips the test as expected, and outputs "Omission: Test Reason" to the console. It's unfortunate that you have to indent existing code to make this work, and I'd be happy to learn of a better way to do it, but this works.