I have a Ruby on Rails 6.1 application I am testing with rspec 3.10, all tests passing and it exits with code 0. When I add Simplecov (using 0.21.2) the exit code is always 1.
My simplecov configuration:
# spec/rails_helper.rb
require 'simplecov'
if !ENV["NO_COVERAGE"]
SimpleCov.start :rails do
minimum_coverage 0
minimum_coverage_by_file 0
filesize = Proc.new { |source_file| source_file.lines.count < 10 }
add_group "Workers", "/app/workers"
add_group "Builders", "/app/builders"
add_group "Queries", "/app/queries"
add_group "Decorators", "/app/decorators"
end
end
Example output (from a Github action)
Finished in 4 minutes 18.1 seconds (files took 11.15 seconds to load)
2415 examples, 0 failures, 154 pending
Randomized with seed 29170
Coverage report generated for RSpec to /app/coverage. 8665 / 13736 LOC (63.08%) covered.
1
Error: Process completed with exit code 1.
If I remove or disable simplecov everything works fine. If I turn it back on, exit code is always 1. This happens on GH Actions, my dev machine and inside Docker containers.
Am I doing something wrong? How do I get simplecov to always return 0 since I don't care about minimum coverage anyway?
Related
As the simpleCov gem documentation indicates, the test_helper.rb file was amended in a Rails 7 app to include
require 'simplecov'
SimpleCov.start 'rails'
puts "SimpleCov started successfully!"
# Previous content of test helper now starts here
ENV["RAILS_ENV"] ||= "test"
require_relative "../config/environment"
require "rails/test_help"
require 'webmock/minitest'
and a model test file
class CategoryTest < ActiveSupport::TestCase
puts "MyCode is being loaded!"
runs as expected
> rails test test/models
SimpleCov started successfully!
MyCode is being loaded!
However the net coverage is blatantly wrong:
93 runs, 192 assertions, 0 failures, 0 errors, 0 skips
51 / 3095 LOC (1.65%) covered.
as this was a copy of another application that had full coverage and was renamed.
I noticed at some point that the coverage percentages were dropping, but that was while running individual tests (and thus ignorable given SimpleCov re-writes all); now the result remains invariably at that figure.
The only thing I can think of is that some lines are covered and thus something might be interrupting SimpleCov's analysis somehwere down the line.
Any thoughts on how to debug this? [now moot question]
update
I had the itch about this some time ago and came across a solution.
Commenting out parallelize(workers: :number_of_processors) allows coverage to be measured.
So why is this default setting clashing with SimpleCov ?
Is there a way to customize test reports. Currently I am getting this line at the end of each test:
3 tests, 0 assertions, 0 failures, 0 errors, 0 skips
I want to include test file name to this line.
Minitest has a verbose option that will show the test name and execution time for each test that's run. How you pass the -v flag to Minitest will vary depending on how you're running your tests. See http://chriskottom.com/blog/2014/12/command-line-flags-for-minitest-in-the-raw/ for a sample of verbose output.
Another option would be using minitest-reporters with the SpecReporter by adding it to your Gemfile and putting this in your test helper:
require "minitest/reporters"
Minitest::Reporters.use! Minitest::Reporters::SpecReporter.new
See https://github.com/kern/minitest-reporters for more info.
I've got a bunch of files in test/policies, and I've tried to enhance rake test like so:
# lib/tasks/test.rake
namespace :test do
desc "Test Pundit policies"
Rake::TestTask.new(:policies) do |t|
t.libs << 'test'
t.pattern = 'test/policies/*_test.rb'
end
end
Rake::Task["test"].enhance do
Rake::Task["test:policies"].invoke
end
It works great if I run bin/rake test:all, but bin/rake test now only runs the policy tests, and none of my others.
Can anyone advise what I am doing wrong here? In case it's not clear, I want rake test to run all my tests, just like it used to.
UPDATE
Actually, it's kind of working now, but I've noticed that my functional, unit and integration tests do run now, but only if the policy tests all pass. If any of the policy tests fail, then the rest of my test suite fails to run.
And I don't like the output, notice how there are two blocks of test output below:
[vagrant#vagrant-centos-6-4 vagrant]$ bin/rake test
Run options: --seed 54880
# Running:
.........................
Finished in 0.584530s, 42.7694 runs/s, 56.4556 assertions/s.
25 runs, 33 assertions, 0 failures, 0 errors, 0 skips
Run options: --seed 19900
# Running:
.........................................................
Finished in 4.132299s, 51.0612 runs/s, 128.9839 assertions/s.
I'd prefer if the policy tests output was merged into the same output block as that from my other tests. Any ideas, or is this as good as it's going to get?
Someone else asked the same question and I answered it for them. The solution in my case was to change my code from:
Rake::TestTask.new(:policies) do |t|
to:
Rails::TestTask.new(:policies) do |t|
It just works, and fixes all the issues I was having.
I am writing tests for my sidekiq workers and I want them to run when I type "rake" in the terminal. I have that working - I added the following to my Rakefile:
namespace :test do
Rake::TestTask.new(:workers) do |t|
t.libs << "test"
t.pattern = 'test/workers/**/*_test.rb'
end
end
Rake::Task[:test].enhance ["test:workers"]
When I run rake I get something like this as my output:
Run options: --seed 51172
# Running tests:
SS
Finished tests in 0.005594s, 357.5259 tests/s, 0.0000 assertions/s.
2 tests, 0 assertions, 0 failures, 0 errors, 2 skips
Run options: --seed 17561
# Running tests:
S............................................SSSS..SSSSS......
Finished tests in 2.037526s, 30.4291 tests/s, 45.6436 assertions/s.
62 tests, 93 assertions, 0 failures, 0 errors, 10 skips
The S characters are skips - I haven't finished all of my tests yet. So my question is basically - is there a way to merge the two sets of tests after enhancing? Should I be doing something different than an enhance?
If I'm doing anything blatantly wrong please let me know, and thanks for reading this. And just in case it is needed: Rails 4 w/ Ruby 2.0
Try this, change the following:
Rake::TestTask.new(:workers) do |t|
to this:
Rails::TestTask.new(:workers) do |t|
It's a small change, but fixes the problem for me. It means your entire test suite will run with merged output.
there is a simple workaround for this:
bundle exec rake test:all
if you want to see how rails tasks are created look here
I want to do a performance test for my rails 3 app and I did a try according to the rails online guide
rake test:profile
and it gave some output as:
Specify ruby-prof as application's dependency in Gemfile to run benchmarks.
Loaded suite /usr/local/lib/ruby/gems/1.9.1/gems/rake-0.9.2/lib/rake/rake_test_loader
Started
.
Finished in 0.326233 seconds.
1 tests, 0 assertions, 0 failures, 0 errors, 0 skips
But according to the guide, there should be something like:
BrowsingTest#test_homepage (31 ms warmup)
wall_time: 6 ms
memory: 437.27 KB
objects: 5,514
gc_runs: 0
gc_time: 19 ms
and also some log files produced in app's tmp/performance dir, which doesn't exist in my case.
The performance test is the generated sample test, browsing_test.rb, in my app's test\performance dir:
require 'test_helper'
require 'rails/performance_test_help'
# Profiling results for each test method are written to tmp/performance.
class BrowsingTest < ActionDispatch::PerformanceTest
def test_homepage
get '/'
end
end
And my rails version is 3.0.10. Can anyone give me some tips or hints?
Had the exact same issue, and solved it by running
rvm install 1.9.2-p290 --patch gcdata # Make sure you have the same patch version
(note: not sure how much that step helped since I had an error, but since it's part of what I did...)
Adding to my gemfile
gem 'ruby-prof', :git => 'git://github.com/wycats/ruby-prof.git'
gem 'test-unit'
And of course running
bundle install
Some reading that helped me
http://rubylearning.com/blog/2011/08/14/performance-testing-rails-applications-how-to/
https://rails.lighthouseapp.com/projects/8994/tickets/4171-fresh-rails-3-app-cannot-run-performance-tests
http://guides.rubyonrails.org/performance_testing.html#installing-gc-patched-mri