I'm using Minitest with Rails.
How do I get it to exit on the first failure if I'm running a bunch of tests? I want this to happen when I'm writing the tests because it's a waste of time for later tests to run after the failed one.
I've just found this answer while searching for the solution, but at least in Rails 5.1 you have one option:
rails test -h Usage: bin/rails test [options] [files or directories]
You can run a single test by appending a line number to a filename:
bin/rails test test/models/user_test.rb:27
You can run multiple files and directories at the same time:
bin/rails test test/controllers test/integration/login_test.rb
By default test failures and errors are reported inline during a run.
Rails options:
-w, --warnings Run with Ruby warnings enabled
-e, --environment Run tests in the ENV environment
-b, --backtrace Show the complete backtrace
-d, --defer-output Output test failures and errors after the test run
-f, --fail-fast Abort test run on first failure or error
-c, --[no-]color Enable color in the output
So you just just need to run rails test -f
You can probably use the minitest-fail-fast gem. You may also modify the Minitest reporters engine to exit after reporting a failure.
Related
I am testing using Docker to run my ruby on rails Rspec tests. This will allow me greater flexibility to test against different databases etc.
In our Bamboo pipeline, its all working - except I assume the rspec.xml file is being placed inside the docker container, and not into the Bamboo working directory. If any tests fail - the bamboo job fails and number of tests is now not reported in bamboo and I assume its because of the 'missing' rspec.xml file.
We have a JUnit XML parser task which now also fails since it cannot find the XML output, and since the docker container is deleted at the end of the tests I assume the file will be deleted also.
Is there anyway to output this file to the Bamboo working directory?
Running the specs like this:
docker run --volume /home/bamboo/bamboo-agent-home/xml-data/build-dir/DIR-ABS2711-UTJGB:/usr/src/app --rm --env RAILS_ENV=test bond:latest bundle exec rake db:migrate rspec_tests:model_tests:run
Note that I am using a rake task to run the specs.
Thanks
So in order to have rspec output results in xml you need something like this:
rspec -r rspec_junit_formatter --format RspecJunitFormatter -o rspec.xml
But since I am using a rake task I cannot use that, I instead need to modify my rake task to out to xml:
require 'rspec/core/rake_task'
RSpec::Core::RakeTask.new(:spec) do |t|
t.fail_on_error = false
t.rspec_opts = "--no-drb -r rspec_junit_formatter --format RspecJunitFormatter -o rspec.xml"
end
So I have some script/runners set up in a cron job, but according to the logs, I'm getting this error below. First, I'm not sure why Test::Unit automatic runner is happening in production to begin with. I don't have autospec or autotest going on. Secondly, I'm not sure how to resolve this pesky invalid option error. I'm using the javan-whenever gem to handle the cron schedule. Any help out there?
0 tests, 0 assertions, 0 failures, 0 errors
invalid option: -e
Test::Unit automatic runner.
Usage: /apps/ion/releases/20091210210633/script/runner [options] [-- untouched arguments]
-r, --runner=RUNNER Use the given RUNNER.
(c[onsole], f[ox], g[tk], g[tk]2, t[k])
-n, --name=NAME Runs tests matching NAME.
(patterns may be used).
-t, --testcase=TESTCASE Runs tests in TestCases matching TESTCASE.
(patterns may be used).
-I, --load-path=DIR[:DIR...] Appends directory list to $LOAD_PATH.
-v, --verbose=[LEVEL] Set the output level (default is verbose).
(s[ilent], p[rogress], n[ormal], v[erbose])
-- Stop processing options so that the
remaining options will be passed to the
test.
-h, --help Display this help.
Deprecated options:
--console Console runner (use --runner).
--gtk GTK runner (use --runner).
--fox Fox runner (use --runner).
This is because something in your environment is requiring Test::Unit. I was having the same problem running rails runner in a Rails 3.1.0 application, and it was because we had gem test-unit in our Gemfile for all groups, rather than just for the test group where it was actually needed. Once I moved it into the test group, my runners ran as expected.
This code seems to be the culprit:
# test_unit/lib/test/unit.rb
at_exit do
unless $! || Test::Unit.run?
exit Test::Unit::AutoRunner.run
end
end
If you can't remove the requirement on Test::Unit, you can add this hack somewhere in your environment to prevent AutoRunner from auto-running:
Test::Unit.run = true
Recently I switched from fixtures to factory_girl to test my Ruby on Rails application. If I run rake test:units, to run the tests in my /units directory, they all run perfectly. The same is true if I run my functional tests (in my /functional directory) with rake test:functionals.
However, if I simply run rake test, to run both my unit and functional tests together, my validation fails on the second group of tests (functional, in this case), with the message "Validation failed: Name has already been taken."
I believe this is caused by the functional tests creating objects with the same parameters as the objects that were created in the unit tests -- leading me to believe that the test database isn't cleared in between the unit and functional tests.
I use factory_girl's sequencing to have unique attributes for objects, which means that factory_girl is being reset between tests, while the database is not. What can I do to solve this problem? Is there a way to clear the database between my two test packages?
A command line solution to clear (reconstruct) test database:
rake db:test:prepare
Try writing this in your test/test_helper.rb
eval IO.read(File.expand_path(File.dirname(__FILE__) + "/../Rakefile"))
class Test::Unit::TestCase
....
#db:test:prepare won't work, don't know why,
#as DROP DATABASE won't execute (me on PostgreSQL).
#even if I write,
#ActiveRecord::Base.connection.disconnect!
Rake::Task["db:reset"].invoke
end
It's not a recommended solution. It makes tests slower, but it works.
A rails plugin called "override_rake_task" could be used to override Rake task "test" which is defined inside if Rails gem. This is a very simple task that executes 3 other tasks one after another: test:units, test:functionals and test:integration. You could include the execution of "db:test:purge" task to clear the test DB before executing test:functionals.
Apparently if you are not using this plugin and if you define a task in your rails application with the same name, rake would execute both tasks: the default and yours.
The above solutions didn't work for me. If you are trying to reach out to an external database running unit tests can give some weird errors. For some reason they do not get cleared after running the test so you have to run rake db:test:purge after running the unit tests. Put this in your Rakefile and it should fix it.
Rake::Task["db:test:prepare"].enhance do
Rake::Task["db:test:purge"].invoke
end
I ran into this problem on my machine. I was getting test failures, from validation problems because the database wasn't properly being reset between tests. Some back story about my situation:
-I had a linux box, and was running code, that I knew should pass the tests.
-I bought a Mac with Lion installed and attempted to get my code running on that machine.
-I installed mysql from source
Everything installed fine. My database worked, and rails could access it. When I ran tests however, I ran into the same problem. I came across this post, and tried both the proposed solutions (even though it didn't seem like a code issue, it seemed like a configuration problem since rake ran fine on my linux box). None of the solutions works.
I removed mysql:
sudo rm /usr/local/mysql
sudo rm -rf /usr/local/mysql*
sudo rm -rf /Library/StartupItems/MySQLCOM
sudo rm -rf /Library/PreferencePanes/My*
sudo rm -rf /Library/Receipts/mysql*
sudo rm -rf /Library/Receipts/MySQL*
sudo rm /etc/my.cnf
sudo rm /usr/local/bin/mysql*
I reinstalled mysql with homebrew instead of manually doing it from source (this step was courtesy of a co-worker's advice):
export PATH="/usr/local/bin:/usr/local/sbin:/usr/local/mysql/bin:$PATH"
brew install https://github.com/adamv/homebrew-alt/raw/master/versions/mysql51.rb
unset TMPDIR
mysql_install_db
I then re-ran rake, and all the tests passed. If anyone is on Lion, built mysql from source, and ran into this problem, this might be a solution.
DB cleaner is a pretty nice gem specifically for cleaning between tests. It gives a few options including wrapping every test in a transaction and rolling back, truncating the table, and deleting.
It also supports multiple ORMS in case you're not using/ using more than active record.
The documentation is pretty good and includes examples of using it with MiniTest, Rspec, and Cucumber.
https://github.com/bmabey/database_cleaner
How can I run a single test from a Rails test suite?
rake test ANYTHING seems to not help.
NOTE: This doesn't run the test via rake. So any code you have in Rakefile will NOT get executed.
To run a single test, use the following command from your rails project's main directory:
ruby -I test test/unit/my_model_test.rb -n test_name
This runs a single test named "name", defined in the MyModelTest class in the specified file. The test_name is formed by taking the test name, prepending it with the word "test", then separating the words with underscores. For example:
class MyModelTest < ActiveSupport::TestCase
test 'valid with good attributes' do
# do whatever you do
end
test 'invalid with bad attributes' do
# do whatever you do
end
end
You can run both tests via:
ruby -I test test/unit/my_model_test.rb
and just the second test via
ruby -I test test/unit/my_model_test.rb -n test_invalid_with_bad_attributes
Run a test file:
rake test TEST=tests/functional/accounts_test.rb
Run a single test in a test file:
rake test TEST=tests/functional/accounts_test.rb TESTOPTS="-n /paid accounts/"
(From #Puhlze 's comment.)
For rails 5:
rails test test/models/my_model.rb
Thanks to #James, the answer seems to be:
rails test test/models/my_model.rb:22
Assuming 22 is the line number of the given test. According to rails help:
$ rails test --help
You can run a single test by appending a line number to a filename:
bin/rails test test/models/user_test.rb:27
Also, please note that your test should inherit from ActionDispatch::IntegrationTest for this to work (That was my mistake):
class NexApiTest < ActionDispatch::IntegrationTest
.
.
.
Rails 5
I used this way to run single test file (all the tests in one file)
rails test -n /TopicsControllerTest/ -v
Another option is to use the line number (which is printed below a failing test):
rails test test/model/my_model.rb:15
In my situation for rake only works TESTOPTS="-n='/your_test_name/'":
bundle exec rake test TEST=test/system/example_test.rb TESTOPTS="-n='/your_test_name/'"
To run a single test in the actual Rails suite:
bundle exec ruby -I"railties/test" actionpack/test/template/form_options_helper_test.rb
That was a silly midnight question of mine. Rails kindly prints the command it is executing upon rake test. The rest is a cut and paste exercise.
~/projects/rails/actionpack (my2.3.4)$ ruby -I"lib:test" "/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake/rake_test_loader.rb" "test/controller/base_test.rb"
The best way is to look directly into the guides: http://guides.rubyonrails.org/contributing_to_ruby_on_rails.html#running-tests
cd actionmailer
bundle exec ruby -w -Itest test/mail_layout_test.rb -n test_explicit_class_layout
If you want to run a single test, you can just run them as a regular Ruby script
ruby actionmailer/test/mail_layout_test.rb
You can also run a whole suite (eg. ActiveRecord or ActionMailer) by cd-ing into the directory and running rake test inside there.
To re-run a test that just failed, copy-n-paste the failed test name into
rails test -n [test-name]
EXAMPLE
When your test suite reports this:
> rails test
...
Error:
PlayersControllerTest#test_should_show_player:
ActionView::Template::Error: no implicit conversion from nil to integer
you rerun the failing test with this:
rails test -n PlayersControllerTest#test_should_show_player
If rake is running MiniTest, the option is --name instead of -n.
rake test TEST=test/unit/progress_test.rb TESTOPTS="--name=testCreate"
First, access the folder of the lib you want to test(this is important) and then run:
~/Projects/rails/actionview (master)$ ruby -I test test/template/number_helper_test.rb
Rails folder
bundle install
bundle exec ruby -I"activerecord/test" activerecord/test/cases/relation/where_test.rb
Note you need to load appropriate folder: "activerecord/test" (where you have test)
As title.
ruby test/functionals/whatevertest.rb doesn't work, that requires me to replace all require 'test_helper' to require File.dirname(__FILE__) + '/../test_helper'. For some reason most of those test templates have such issue, so I rather to see if there is a hack I could get around it.
The following answer is based on: How to run single test from rails test suite? (stackoverflow)
But very briefly, here's the answer:
ruby -I test test/functional/whatevertest.rb
For a specific functional test, run:
ruby -I test test/functional/whatevertest.rb -n test_should_get_index
Just put underscores in places of spaces in test names (as above), or quote the title as follows:
ruby -I test test/functional/whatevertest.rb -n 'test should get index'
Note that for unit tests just replace functional with unit in the examples above. And if you're using bundler to manage your application's gem dependencies, you'll have to execute the tests with bundle exec as follows:
bundle exec ruby -I test test/unit/specific_model_test.rb
bundle exec ruby -I test test/unit/specific_model_test.rb -n test_divide_by_zero
bundle exec ruby -I test test/unit/specific_model_test.rb -n 'test divide by zero'
Most importantly, note that the argument to the -n switch is the name of the test, and the word "test" prepended to it, with spaces or underscores depending on whether you're quoting the name or not. The reason is that test is a convenience method. The following two methods are equivalent:
test "should get high" do
assert true
end
def test_should_get_high
assert true
end
...and can be executed as either of the following (they are equivalent):
bundle exec ruby -I test test/integration/misc_test.rb -n 'test should get high'
bundle exec ruby -I test test/integration/misc_test.rb -n test_should_get_high
Try this:
ruby -Ilib:test test/functionals/whatevertest.rb
On Linux? why not try (cd test && ruby functionals/whatevertest.rb). Note, the parentheses are important as otherwise your current directory will change to the subdirectory. What it does is fork another shell, change to the subdirectory in it, and run the test.
If you are on Rails 4, then rake supports file / directory arguments. Example:
rake test test/unit/user_test.rb
rake test test/unit
The answer for the title question would be:
ruby unit/post_test.rb -n selected_test # use to run only one selected test
but for the body of the question tvanfosson gave a good answer.
After spending endless hours on this, I finally found the solution (in Rails 3.0.7) :
ruby -I test test/functional/users_controller_test.rb -n "/the_test_name/"
Note, this only works with underbars (_) in the command. It does not work with spaces!
Define the test with spaces as:
test "the test name" do
This solution uses pattern matching, so you can use part of the test name. If the test is "should do foo", then either of the following will work as well.
ruby -I test test/functional/alerts_controller_test.rb -n "/foo/"
ruby -I test test/functional/alerts_controller_test.rb -n "/do_foo/"
The following (with spaces) will not work:
ruby -I test test/functional/alerts_controller_test.rb -n "/do foo/"
most conventional method for 2 and 3 is:
ruby -I test test/functional/your_test_file.rb