Is there any way to retest previously broken tests?
So, say, I run rspec and several tests in different files and directories fail.
I fix something and now I have to manually specify all files and folders I want to retest or run tests for whole project again(It takes considerable amount of time for big projets).
What I was looking for is something like a flag
rspec --prev-failed-only
I realize that such flag would require considerable amount of additional actions from rspec, like storing results of previous tests and so on. But I think it would be super convenient to me.
Is there any such(or similar) tool/gem?
rspec-rerun gem does what you want : https://github.com/dblock/rspec-rerun
https://github.com/rspec/rspec-core/issues/456 has a good discussion on the topic of making rspec itself be able to rerun failed tests.
On the Giant Robots podcast, Sam Phippen of the core team mentioned this feature is due to be added to RSpec soon.
In case anyone else finds this, a month after the question was asked (open source <3 ) rspec 3.3 introduced the --only-failures option, also the delightfully handy --next-failure (-n) option. rspec --help for more info.
Related
I have one rails spec file.
When I run: rspec my_spec.rb
There were 3 failures. These failures are in the same context.
But when I run them seperately (Example: rspec my_spec.rb:231),
they passed.
Rarely individually test also failed, if that, I restarted docker then it passed again.
I am not sure what was wrong?
I tried adding DatabaseCleaner.clean and redis.flushdb in spec_helper but no lucky.
Any idea can help?
Is very difficult to help you without the specs.But based on my own experience, usually this is a problem with hardcoded expects. Like:
expect(response.id).to eq(1)
rather then:
expect(response.id).to eq(object.id)
When you run your test only it works because of there one entity only. Check this and if it don't help, please provide us more information. Glad to help.
Is there a recommended way of testing Rails generators with RSpec?
The only solution I've found is the Generator Spec gem, that hasn't been updated in over two years.
There's the gem
https://github.com/petergoldstein/generator_spec, which does a decent job, although it is not very actively maintained
I would write a file by hand that acts as a test fixture. I would then as part of my test generate the file with the generator. At that point I would diff the two files. Looks like the diffy gem could help you there. If there is no diff then pass the test. Fail if otherwise.
https://github.com/samg/diffy
Don't forget to clean up your temp files after the tests. You don't want them hanging around in your repos.
I didn't found official recommendations how to test Rails generators too. So, I am just run generator directly on dummy application, compare generated files and delete them at the end of test.
Here is my spec which implement described approach. I had used minispec instead of rspec here but they a very similar.
I'm using Test::Unit on a large app with a large number of gem dependencies (>75). I'm trying to develop using BDD, but it takes minutes for the app to load it's dependencies before it can run the tests. Is there a way to preload the dependencies and just auto-run the test on changes, or a similar solution?
I would look into Spork. It works wonders.
https://github.com/sporkrb/spork
https://github.com/sporkrb/spork-testunit
I am using RSpec and there's a great tool for it, called Spork. It basically loads your app once and then just reloads modified parts. If you combine it with Guard, you get "continuous testing". That is, you hit 'Save' in your editor and tests start executing, giving you instant feedback. This still amazes me after some months :)
Edit
As #THEM points out, there's a plugin for Spork to support TestUnit. You should look into it.
There was also an interesting article about test speed on the 37Signals blog a while back. Might be of interest even if you end up going with Spork or another solution.
We've recently upgraded our Rails application.
To be extra sure everything works, I've tried to get the tests and specs of the various used plugins (26 at current count) to work, thinking then to add those to our continuous integration, which only runs the main application's specs.
I've run into a lot of problems even getting the specs/tests to run at all, not even getting to any individual test failures. For example, I've run across this problem: http://rails_security.lighthouseapp.com/projects/15332/tickets/7-rake-spec-plugin-fails-on-rails-2-1 (thanks by the way for that ticket, even though the issue wasn't fixed).
So the question is: Are we unusual in that we've ever cared about running plugin tests ? It doesn't seem to feature much here on SO. My nagging feeling is that they should be run as much as the main specs, but you could also argue that since the main specs work, the plugins must also work.
Alot of it depends on the plugin/gem being used.
If I know the author/community of the gem is competant I will skip the tests and simply use the latest stable release and freeze that gem. I will then track the progress of the development using github.
If the plugin/gem is written by an unknown party I will run the tests and freeze the gem/plugin and again monitor the development.
Sometimes however I will write my own contributions to the gem and fork the code. I will clone the repo in github and base my installations from that. At which point any and all changes result in a complete test run.
With all things in the open source world there is an element of trust between the creator and the users of those pieces of code. The tests themselves don't tell me much about the codebase, it shows there are tests and thats it. Do they test everything ? Are there edge cases ? . Its this element of trust I have with certain developers in the community that means I forgo worrying over running tests for those gems.
Its a slippery slope testing everything, where does it stop ? Would you test rails every release ? No, you assume the community has done this for you already.
Today I am going to get as far as I can setting up my testing environment and workflow. I'm looking for practical advice on how to setup the test environment from you guys who are very passionate and versed in Ruby Testing.
By the end of the day (6am PST?) I would like to be able to:
Type one 1-command to run test suites for ANY project I find on Github.
Run autotest for ANY Github project so I can fork and make TESTABLE contributions.
Build gems from the ground up with Autotest and Shoulda.
For one reason or another, I hardly ever run tests for projects I clone from Github. The major reason is because unless they're using RSpec and have a Rake task to run the tests, I don't see the common pattern behind it all.
I have built 3 or 4 gems writing tests with RSpec, and while I find the DSL fun, it's less than ideal because it just adds another layer/language of methods I have to learn and remember. So I'm going with Shoulda. But this isn't a question about which testing framework to choose.
So the questions are:
What is your, the SO reader and Github project committer, test environment setup using autotest so that whenever you git clone a gem, you can run the tests and autotest-develop them if desired?
What are the guys who are writing the Paperclip Tests and Authlogic Tests doing? What is their setup?
Thanks for the insight. There are tons of resources describing how to use the different testing frameworks, but almost nothing on the actual setup and workflow. Looking for answers that will make me a more effective tester.
The most common convention probably is rake test, rake spec, or maybe even just rake.
Of course, there is no question that this will fail with many projects, in particular the ones without tests or specs.
It might be possible to parse the output of rake -T if a Rakefile is there, and act on that, but there really is no way you will cover ALL projects on GitHub.