I am testing using Docker to run my ruby on rails Rspec tests. This will allow me greater flexibility to test against different databases etc.
In our Bamboo pipeline, its all working - except I assume the rspec.xml file is being placed inside the docker container, and not into the Bamboo working directory. If any tests fail - the bamboo job fails and number of tests is now not reported in bamboo and I assume its because of the 'missing' rspec.xml file.
We have a JUnit XML parser task which now also fails since it cannot find the XML output, and since the docker container is deleted at the end of the tests I assume the file will be deleted also.
Is there anyway to output this file to the Bamboo working directory?
Running the specs like this:
docker run --volume /home/bamboo/bamboo-agent-home/xml-data/build-dir/DIR-ABS2711-UTJGB:/usr/src/app --rm --env RAILS_ENV=test bond:latest bundle exec rake db:migrate rspec_tests:model_tests:run
Note that I am using a rake task to run the specs.
Thanks
So in order to have rspec output results in xml you need something like this:
rspec -r rspec_junit_formatter --format RspecJunitFormatter -o rspec.xml
But since I am using a rake task I cannot use that, I instead need to modify my rake task to out to xml:
require 'rspec/core/rake_task'
RSpec::Core::RakeTask.new(:spec) do |t|
t.fail_on_error = false
t.rspec_opts = "--no-drb -r rspec_junit_formatter --format RspecJunitFormatter -o rspec.xml"
end
Related
Say I have a file containing name of test files to run tests from and It can contain specific test names too. If test file contains that specific test, run only that test from the file containing the test and run all tests from other test files.
I use Codebuild to run tests for our application but Codebuild does not provide a way to run only specific tests. So I am replacing bin/rails test command on codebuild with our custom rake task. That rake task will check for a file in our system containing list of tests to run and If it finds the file it run only those tests other normal bin/rails test
You could copy how rails is already defining their tasks like test:system here:
namespace :test do
desc "Run tests from file"
task from_file: "test:prepare" do
$: << "test"
Rails::TestUnit::Runner.rake_run(
File.read(Rails.root.join("tests_to_run.txt")).lines.map(&:chomp)
)
end
end
$ bundle exec rails test:from_file
I'm using Minitest with Rails.
How do I get it to exit on the first failure if I'm running a bunch of tests? I want this to happen when I'm writing the tests because it's a waste of time for later tests to run after the failed one.
I've just found this answer while searching for the solution, but at least in Rails 5.1 you have one option:
rails test -h Usage: bin/rails test [options] [files or directories]
You can run a single test by appending a line number to a filename:
bin/rails test test/models/user_test.rb:27
You can run multiple files and directories at the same time:
bin/rails test test/controllers test/integration/login_test.rb
By default test failures and errors are reported inline during a run.
Rails options:
-w, --warnings Run with Ruby warnings enabled
-e, --environment Run tests in the ENV environment
-b, --backtrace Show the complete backtrace
-d, --defer-output Output test failures and errors after the test run
-f, --fail-fast Abort test run on first failure or error
-c, --[no-]color Enable color in the output
So you just just need to run rails test -f
You can probably use the minitest-fail-fast gem. You may also modify the Minitest reporters engine to exit after reporting a failure.
I'm trying to output text to console during tests, to know what happens and have an history of the tests, but nothing seems to work, not printf, neither $stdout.write.
Should I just use a text log file and be done with it or it's possible to output to jenkins console?
As explained in https://content.pivotal.io/blog/what-happened-to-stdout-on-ci and https://github.com/ci-reporter/ci_reporter#environment-variables, you need to set CI_CAPTURE=off
Here is a copy of my jenkins config for running RSpec tests of a Rails app in Jenkins:
[ -d jenkins ] && rm -rf jenkins
mkdir jenkins
cp ~/configs/yourApp/default-db-config config/database.yml
rake db:migrate
rake db:test:prepare
export RAILS_ENV=test
export SPEC_OPTS="--no-drb --format documentation --format html --out jenkins/rspec.html"
rake spec
First it deletes any previous test history from the workspace if it exists.
Next it creates a jenkins directory in the workspace for storing the test output.
Then it sets up the app for testing with a working DB config (I don't store DB config files in my git repo).
Finally it migrates the dev DB if required, prepares the test DB, sets the RAILS_ENV to test and runs the tests with the specified SPEC_OPTS.
The important bits are as follows:
--format documentation ... this sends sensible output of test progress to your console log. Write your tests properly and this will be infinitely more useful than any puts commands you might have considered using.
--format html ... this outputs HTML files of the test results to the jenkins directory created earlier and specified in the --out attribute. Add the following to your job description to show those results on the main page of this job:
<iframe src='http://jenkins.your-domain.com/job/your-tests/ws/jenkins/rspec.html' width="100%" height="600" frameborder="0"/>
Hopefully that should get you up and running with a more useful jenkins test job for RSpec.
Not sure if Jenkins will change this (I don't think so), but in Rspec you can write
puts response.body
or
puts "My mommy made me mash my M&M's"
or whatever else you want and it will be put in the console/results
Every time I run "rake spec" on my Rails 3 / RSpec 2 project, the first thing it does is print out the "bundle exec spec ...." command it runs. The part I omitted, however, is a list of all the spec files in the project, which is a big chunk of text that gets in the way of reading test results. How do I turn that off?
You can change the output of your specs by placing flags in a spec/spec.opts file in your rails app.
Example from this blog post:
--colour
--format progress
--format specdoc:spec/spec_full_report.txt
--format failing_examples:spec/spec_failing_examples.txt
--format html:spec/spec_report.html
--loadby mtime
--reverse
You can see all the available options here: https://github.com/dchelimsky/rspec/blob/master/lib/spec/runner/option_parser.rb
I don't use rake spec and instead I run my specs via rspec runner. So instead of "rake spec" I just do "rspec spec/". You can pass various options to this command as described in the first answer to your question.
Just add this to your Rakefile
require 'rspec/core/rake_task'
task(:spec).clear
RSpec::Core::RakeTask.new(:spec) do |t|
t.verbose = false
end
How can I run a single test from a Rails test suite?
rake test ANYTHING seems to not help.
NOTE: This doesn't run the test via rake. So any code you have in Rakefile will NOT get executed.
To run a single test, use the following command from your rails project's main directory:
ruby -I test test/unit/my_model_test.rb -n test_name
This runs a single test named "name", defined in the MyModelTest class in the specified file. The test_name is formed by taking the test name, prepending it with the word "test", then separating the words with underscores. For example:
class MyModelTest < ActiveSupport::TestCase
test 'valid with good attributes' do
# do whatever you do
end
test 'invalid with bad attributes' do
# do whatever you do
end
end
You can run both tests via:
ruby -I test test/unit/my_model_test.rb
and just the second test via
ruby -I test test/unit/my_model_test.rb -n test_invalid_with_bad_attributes
Run a test file:
rake test TEST=tests/functional/accounts_test.rb
Run a single test in a test file:
rake test TEST=tests/functional/accounts_test.rb TESTOPTS="-n /paid accounts/"
(From #Puhlze 's comment.)
For rails 5:
rails test test/models/my_model.rb
Thanks to #James, the answer seems to be:
rails test test/models/my_model.rb:22
Assuming 22 is the line number of the given test. According to rails help:
$ rails test --help
You can run a single test by appending a line number to a filename:
bin/rails test test/models/user_test.rb:27
Also, please note that your test should inherit from ActionDispatch::IntegrationTest for this to work (That was my mistake):
class NexApiTest < ActionDispatch::IntegrationTest
.
.
.
Rails 5
I used this way to run single test file (all the tests in one file)
rails test -n /TopicsControllerTest/ -v
Another option is to use the line number (which is printed below a failing test):
rails test test/model/my_model.rb:15
In my situation for rake only works TESTOPTS="-n='/your_test_name/'":
bundle exec rake test TEST=test/system/example_test.rb TESTOPTS="-n='/your_test_name/'"
To run a single test in the actual Rails suite:
bundle exec ruby -I"railties/test" actionpack/test/template/form_options_helper_test.rb
That was a silly midnight question of mine. Rails kindly prints the command it is executing upon rake test. The rest is a cut and paste exercise.
~/projects/rails/actionpack (my2.3.4)$ ruby -I"lib:test" "/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake/rake_test_loader.rb" "test/controller/base_test.rb"
The best way is to look directly into the guides: http://guides.rubyonrails.org/contributing_to_ruby_on_rails.html#running-tests
cd actionmailer
bundle exec ruby -w -Itest test/mail_layout_test.rb -n test_explicit_class_layout
If you want to run a single test, you can just run them as a regular Ruby script
ruby actionmailer/test/mail_layout_test.rb
You can also run a whole suite (eg. ActiveRecord or ActionMailer) by cd-ing into the directory and running rake test inside there.
To re-run a test that just failed, copy-n-paste the failed test name into
rails test -n [test-name]
EXAMPLE
When your test suite reports this:
> rails test
...
Error:
PlayersControllerTest#test_should_show_player:
ActionView::Template::Error: no implicit conversion from nil to integer
you rerun the failing test with this:
rails test -n PlayersControllerTest#test_should_show_player
If rake is running MiniTest, the option is --name instead of -n.
rake test TEST=test/unit/progress_test.rb TESTOPTS="--name=testCreate"
First, access the folder of the lib you want to test(this is important) and then run:
~/Projects/rails/actionview (master)$ ruby -I test test/template/number_helper_test.rb
Rails folder
bundle install
bundle exec ruby -I"activerecord/test" activerecord/test/cases/relation/where_test.rb
Note you need to load appropriate folder: "activerecord/test" (where you have test)