Is there a way to turn off outputting pending tests in Rspec (rspec 3, specifically)? Rather than having to tag everyone and skipping them.
Perhaps a --pending type command for .rspec?
If you want a spec not to be run at all in rspec 3 you should use skip rather than pending
Related
Say I have a user_spec.rb for my User model, and I want to run that test inside the rails console.
My first thought is to execute the usual shell command:
exec("./spec/user_spec.rb")
But is there a simpler way to run the spec? I'm trying to automate some of the tests (and reinvent the wheel a little, yes), so being able to trigger an rspec test inside of another Ruby class seems ideal.
Edit:
output = `./spec/user_spec.rb`
This will provide the rspec output and $?.success? will then provide a pass fail value. Is this the best solution here? Or is there a way to call an RSpec class itself?
As pointed out by Anthony in his comment, you can use RSpec::Core::Runner to basically invoke the command line behavior from code or an interactive console. However, if you use something like Rails, consider that your environment is likely going to be set to development (or even production, if this is where you'll execute the code). So make sure that whatever you do doesn't have any unwanted side-effects.
Another thing to consider is that RSpec globally stores its configuration including all example groups that were registerd with it before. That's why you'll need to reset RSpec between subsequent runs. This can be done via RSpec.reset.
So putting it all together, you'll get:
require 'rspec/core'
RSpec::Core::Runner.run(['spec/path/to_spec_1.rb', 'spec/path/to_spec_2.rb'])
RSpec.reset
The call to RSpec::Core::Runner.run will output to standard out and return the exit code as a result (0 meaning no errors, a non-zero exit code means a test failed).
..
Finished in 0.01791 seconds (files took 17.25 seconds to load)
2 example, 0 failures
=> 0
You can pass other IO objects to RSpec::Core::Runner.run to specify where it should output to. And you can also pass other command line parameters to the first array of RSpec::Core::Runner.run, e.g. '--format=json' to output the results in JSON format.
So if you, for example, want to capture the output in JSON format to then further do something with it, you could do the following:
require 'rspec/core'
error_stream = StringIO.new
output_stream = StringIO.new
RSpec::Core::Runner.run(
[
'spec/path/to_spec_1.rb',
'spec/path/to_spec_2.rb',
'--format=json'
],
error_stream,
output_stream
)
RSpec.reset
errors =
if error_stream.string
JSON.parse(error_stream.string)
end
results =
if output_stream.string
JSON.parse(output_stream.string)
end
Run bundle exec rspec to run all tests or bundle exec rspec ./spec/user_spec.rb to run the specific test
When running minutest tests, is it possible to peek at the information about the errors that has happened?
For example, this test suite takes ten minutes to complete. But I would like some more info about the letter E appearing in the tests result.
I don't want to wait ten minutes.
*** Running FRONTEND component engine specs
Run options: --seed 29704
# Running:
......................................................................................................................................................................................E...........
That's E for "error", so one of your tests is failing. Normally you get output that explains more. Once you identify which test is failing you can run that test in a more focused capacity, like:
ruby test/unit/broken_test.rb --name=test_that_is_broken
Where that is the path to your test script and the name of the testing method that's failing.
You may need to make your tests self-contained, able to be run this way, by using:
require_relative '../test_helper'
Or whatever the helper stub is that kicks off the testing framework. Some skeleton files contain things like require 'test_helper' which won't be found in your current $LOAD_PATH.
I want to run feature specs written in rspec/capybara in a fixed sequence as follows:
signup_spec.rb
login_spec.rb
project_creation_spec.rb
project_migration_spec.rb
The --order feature of given here says that
Use the --order option to tell RSpec how to order the files, groups, and
examples
How would I use .rspec file to mention pass my requirements ?
I have a shell script with test cases running in a sequence like:
rspec spec/features/signup_spec.rb
rspec spec/features/login_spec.rb
rspec spec/features/project_creation_spec.rb
rspec spec/features/project_migration_spec.rb
The order option does not allow this. It allows switching between the default ordering (which is essentially that they are run in the order they are defined which in turns depends on file system ordering ) or random ordering (optionally with a seed)
I would consider any order dependence to be a bug - the random ordering option is there to flush out such bugs.
I would like to run some specs that are marked with :first in the beginning of an RSpec session. So, if I have:
describe 'one' { ... }
describe 'two', :first { ... }
describe 'three' { ... }
the order in which I want specs to be run is two, one, three.
Can this be done with RSpec?
Note: I know it's counterproductive to run tests in a certain order, but I need a quick workaround for our project so that all specs pass (unfortunately some specs are poorly written and fail depending on the order in which they are run).
RSpec has an --order flag you can pass to define a specific order. By default, this flag understands only a couple of strategies, such as random or seed, where the seed is a specific instance of a random order.
However, if you dig into the Ordering module, you will see you can define new ordering strategies.
There is a Custom strategy which apparently orders by a specific block. These strategies are managed by an ordering register, and dependent on the :order config.
You can investigate the internals, define a custom order strategy based on your rules, and make it the default.
To run your tests in a specific order you need to arrange the tests in the order you would like them to be run then run the rspec command with the --order default flag:
rspec --order default
If you run rspec without this flag the tests will be run in a random order (as I'm sure you've noticed).
Source:
The release notes from 2012 when Rspec added this flag - http://blog.davidchelimsky.net/blog/2012/01/04/rspec-28-is-released/
I'm struggling with this for quite a while now: I'm trying to upgrade an app from Rails 3.2 to Rails 4. While on Rails 3.2 all specs are passing, they fail under certain conditions in Rails 4.
Some specs are passing in isolation while failing when run together with other specs.
Example Video
https://www.wingolf.org/ak-internet-files/Spec_Behaviour.mp4 (4 mins)
This example video shows:
Running 3 specs using :focus–––green.
Running them together with another spec–––two specs passing before now fail.
Running the 3 specs, but inserting two empty lines–––one spec fails.
Undo does not help when using guard.
focus/unfocus does not help.
Restarting guard does not help.
Running all specs and then running the 3 specs again does help and make them green again. But adding the other task makes two specs fail, again.
As one can see, some specs are red when run together with other specs. Even entering blank lines can make a difference.
More Observations
For some specs, passing or failing occurs randomly when run several times.
The behavior is not specific to one development machine but can be reproduced on travis.
To delete the database completely between the specs using database_cleaner does not help.
To Rails.cache.clear between the specs does not help.
Wrapping each spec in an ActiveRecord::Base.transaction does not help.
This does occur in Rails 4.0.0 as well as in Rails 4.1.1.
Using this minimal spec_helper.rb without spring or anything does not help.
Using guard vs. using bundle exec rspec some_spec.rb:123 directly doesn't make a difference.
This behavior goes for model specs, thus doesn't have to do anything with parallel database connections for features specs.
I've already tried to keep as many gems at the same version as in the (green) Rails-3.2 branch, including guard, rspec, factory_girl, etc.–––does not help.
Update: Observations Based on Comments & Answers
Thanks to engineerDave, I've inserted require 'pry'; binding.pry; into one of the concerning specs. Using the cd and show-source of pry, it was ingeniously easy and fun to narrow down the problem: Apparently, this has_many :through relation does not return objects when run together with other specs, even when called with (true).
has_many(:groups,
-> { where('dag_links.descendant_type' => 'User').uniq },
through: :memberships,
source: :ancestor, source_type: 'Group'
)
If I call groups directly, I get an empty result. But if I go through the memberships, the correct groups are returned:
#user.groups # => []
#user.groups(true) # => []
#user.memberships.collect { |m| m.group } # returns the correct groups
Has Rails changed the has many through behavior in Rails 4 in a way that could be responsible? (Remember: The spec works in isolation.)
Any help, insights and experiences are appreciated. Thanks very much in advance!
Code
Current master branch on Rails 3.2––all green.
Rails-4 branch––strange behavior.
The file/commit seen in the video––strange behavior.
All specs passing on travis for Rails 3.2.
Diff of the Gemfile.lock (or use git diff master..sf/rails4-minimal-update Gemfile.lock |grep rspec)
How to Reproduce
This is how one can check if the issue still exists:
Preparation
git clone git#github.com:fiedl/wingolfsplattform.git
cd wingolfsplattform
git checkout sf/rails4-minimal-update
bundle install
# please create `config/database.yml` to your needs.
bundle exec rake db:create db:migrate db:test:prepare
Run the specs
bundle exec rspec ./vendor/engines/your_platform/spec/models/user_group_membership_spec.rb
bundle exec rspec ./vendor/engines/your_platform/spec/models/user_group_membership_spec.rb:213
The problem still exists, if the spec :213 is green in the second call but is red when run together with the other specs in the first call.
Based on that you're using:
the should syntax
that you indicate you've upgraded recently (perhaps a bundle update?)
that your failure messages indicate a NilObject error.
Is something like this perhaps what is causing it?
https://stackoverflow.com/a/16427072/793330
Are you are calling an object in your test which hasn't been instantiated yet?
I think this might be an rspec 3 upgrade issue where should is deprecated.
Have you ruled out an rspec gem upgrade to the new rspec 3 syntax (2.99 or 3.0.0+) as the culprit?
"Remove support for deprecated expect(...).should. (Myron Marston)"
IMO this behavior would not be caused by a Rails 4 update as its centered around your test suite.
Update (with pry debug):
Also you could use the pry gem to get a window into what is going on in your specs.
Essentially you can put a big "stop" sign (similar to a debug break) right before the spec executes to get a handle on the environment at that point.
it {
require 'pry'; binding.pry
should == something
}
Although beaware sometimes these pry calls wreck havoc on guard's threading and you have to kill it with CTRL+Z and then kill -9 PID that shows.
Update #2: Looking at updated answer.
You might be running up against FactoryGirl issues based on your has_many issue
You may need to trigger a before action in your Factory to pre-populate the associated record. Although this could get messy, i.e. here be monsters, you can trigger after and before callbacks in your factory that will bring these objects into being.
after(:create) do |instance|
do stuff here
end