Minitest: rake test:all fails while explicit testing does not - ruby-on-rails

I have a weird problem which is bugging me for days now... and I don't have a single clue.
I have a rails 4 app which i test using Minitest and guard. So far everything went fine until at some point (which unfortunately I'm not able to reconstruct although I was using guard, maybe my collaborators introduced the issues...) some of our tests fail. The weird thing is: bundle exec rake test:all fails:
DEPRECATION WARNING: rake test:all is deprecated and will be removed in Rails 5. Use rake test to run all tests in test directory. (called from mon_synchronize at /usr/lib/ruby/2.3.0/monitor.rb:214)
# Running tests with run options --seed 53809:
WARNING: The next major version of capybara-webkit will require at least version 5.0 of Qt. You're using version 4.8.7.
............EE....EE.............F..
Finished tests in 26.018707s, 1.3836 tests/s, 1.6142 assertions/s.
Error:
PlaceTest#test_return_''_if_too_many_characters_to_translate:
NoMethodError: undefined method `failsafe_translate' for nil:NilClass
test/models/place_auto_translation_test.rb:29:in `block in <class:PlaceTest>'
Error:
PlaceTest#test_test_places_are_valid:
NoMethodError: undefined method `geocode' for nil:NilClass
test/models/place_geocode_test.rb:21:in `block in <class:PlaceTest>'
Error:
PlaceTest#test_should_translate_text_below_character_limit:
NoMethodError: undefined method `failsafe_translate' for nil:NilClass
test/models/place_auto_translation_test.rb:34:in `block in <class:PlaceTest>'
Error:
PlaceTest#test_can_translate_if_valid_credentials_given:
NoMethodError: undefined method `bing_translator' for nil:NilClass
test/models/place_auto_translation_test.rb:16:in `block in <class:PlaceTest>'
Failure:
Index page Feature Test#test_0002_language is switched to german [/home/blubber/software_projects/rails/workspace/lberg-map/test/features/index_page_test.rb:16]
Minitest::Assertion: Expected to include "Neuer Ort".
36 tests, 42 assertions, 1 failures, 4 errors, 0 skips
Now, while trying to understand the issues raised I was testing the problematic files individually (within my editor and guard or on the command-line, same thing of course) it turns out, that there is actually no problem at all, all tests pass oO. Hence bundle exec rake test test/models/place_auto_translation_test.rb returns
# Running tests with run options --seed 55164:
.....
Finished tests in 25.472545s, 0.1963 tests/s, 0.2355 assertions/s.
5 tests, 6 assertions, 0 failures, 0 errors, 0 skips
Same goes for place_geocode_test.rb as well as the "failing" feature test. Any idea on what's happening here? The test file test/models/place_auto_translation_test.rb looks like this:
require_relative '../test_helper'
class PlaceTest < ActiveSupport::TestCase
def setup
#valid_translator = BingTranslatorWrapper.new(ENV['bing_id'], ENV['bing_secret'], ENV['microsoft_account_key'])
#place = Place.new(name: 'Kiezspinne',
street: 'Schulze-Boysen-Straße',
house_number: '13',
postal_code: '10365',
city: 'Berlin',
description_en: 'This is a test')
end
# AUTO TRANSLATION WRAPPER TESTS
test 'can translate if valid credentials given' do
assert_not_nil #valid_translator.bing_translator
end
test 'cannot translate if API id and key invalid or no microsoft account key' do
invalid_translator = BingTranslatorWrapper.new(ENV['wrong_id'], ENV['wrong_secret'], ENV['microsoft_account_key'])
assert_nil invalid_translator.bing_translator
invalid_translator = BingTranslatorWrapper.new(ENV['bing_id'], ENV['bing_secret'], ENV['wrong_microsoft_account_key'])
assert_equal '', invalid_translator.failsafe_translate('Dies ist ein Test', 'en', 'de')
end
test "return '' if too many characters to translate" do
text = '13 characters' * 1000
assert_equal '', #valid_translator.failsafe_translate(text, 'en', 'de')
end
test 'should translate text below character limit' do
text = 'This is a test'
assert_equal 'Automatische Übersetzung: Dies ist ein Test', #valid_translator.failsafe_translate(text, 'en', 'de')
end
test 'should autotranslate after_create' do
#place.save
#place.reload
assert_equal "Automatische Übersetzung: Dies ist ein Test", #place.description_de
end
end
and test_helper.rb looks like this
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require 'minitest/rails'
require 'minitest/reporters'
require 'minitest/rails/capybara'
reporter_options = { color: true }
Minitest::Reporters.use!(
Minitest::Reporters::DefaultReporter.new(reporter_options),
ENV,
Minitest.backtrace_filter
)
class ActiveSupport::TestCase
# Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
fixtures :all
end
Geocoder.configure(:lookup => :test)
Geocoder::Lookup::Test.set_default_stub(
[
{
'latitude' => 52,
'longitude' => 12,
'address' => {
'road' => 'Magdalenenstr.',
'house_number' => '19',
'postcode' => '10365',
'town' => 'Berlin',
},
'type' => 'house',
}
]
)
# allow tile loading from foreign server
Capybara::Webkit.configure do |config|
%w[a b c].each { |x| config.allow_url("tile-#{x}.openstreetmap.fr") }
end
# While testing with Javascript flag, test runs in another thread,
# thus created fixtures are not available without the following setup
class Capybara::Rails::TestCase
self.use_transactional_fixtures = false
before do
if metadata[:js]
Capybara.javascript_driver = :webkit
Capybara.current_driver = Capybara.javascript_driver
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.start
end
end
after do
if metadata[:js]
DatabaseCleaner.clean
end
Capybara.reset_sessions!
Capybara.current_driver = Capybara.default_driver
end
end
Thanks a thousand, best
Andi

Related

Is there a way to cleanup database tests in Rails 6.0.3.1 System Tests. I thought Rails does the cleanup?

Using
Ruby "2.7.1"
Rails "6.0.3.1"
Mac OS Catalina 10.15.5
Using the default Rails Minitest and FactoryBot for test data.
Issue: I have test data being shared between system test files and hence failing tests. When I run the system test files individually the tests pass but this is inconvenient.
I use the the system test config out of the box as in the application_system_test_case.rb
This is my system test scenario.
I have 3 Customers in customers_system_test.rb file and 1 Customer (so far) in job_description_system_test.rb Customer 1 appears in the customers_system_test.rb failing test screenshots that Rails conveniently provide.
Note: The 3 failing tests all concern record counts such as assert_equal Customer.count, 3
The 3 failing tests
test "visit create customer and navigate back" do
...
assert_equal Customer.count, 3
end
test "cancel destroying a customer" do
assert_equal Customer.count, 3
...
end
test "destroy a customer" do
assert_equal Customer.count, 3
...
end
Interestingly I had the same issue with test data not being cleaned up when running rails test for controller, integration tests etc: The way I get around that is adding the following snippet to the test_helper.rb :
(ActiveRecord::Base.connection.tables - %w{schema_migrations}).each do |table_name|
ActiveRecord::Base.connection.execute "TRUNCATE TABLE #{table_name} CASCADE;"
end Doesn't work for system tests
I feel like I am missing the obvious?
test_helper.rb
require "simplecov"
SimpleCov.start "rails"
ENV["RAILS_ENV"] ||= "test"
require_relative "../config/environment"
require "rails/test_help"
class ActiveSupport::TestCase
include FactoryBot::Syntax::Methods
include Warden::Test::Helpers
# Run tests in parallel with specified workers
# parallelize(workers: :number_of_processors)
parallelize(workers: 1)
# Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
# fixtures :all
# Add more helper methods to be used by all tests here...
include Devise::Test::IntegrationHelpers
end
(ActiveRecord::Base.connection.tables - %w{schema_migrations}).each do |table_name|
ActiveRecord::Base.connection.execute "TRUNCATE TABLE #{table_name} CASCADE;"
end
Ok I got all tests running and passing including system tests by doing the following (wasn't aware this was necessary in Rails 6)
...
setup do
sign_in user
#customer_1 = FactoryBot.create(:customer_with_job_descriptions)
#job_description_2 = FactoryBot.create(:job_description, customer_id: #customer_1.id)
end
teardown do
#customer_1.destroy!
#job_description_2.destroy!
end
...
Ref: 8 Setup and Teardown
https://guides.rubyonrails.org/v3.2/testing.html#setup-and-teardown
........................................................................................
Finished in 22.971069s, 4.3968 runs/s, 21.4182 assertions/s.
101 runs, 492 assertions, 0 failures, 0 errors, 0 skips

Why are the test results listed twice in the console output?

This is how my test results are currently being output to the console. Notice how "Run options... Running: ..." and "Finished in..." are both listed twice.
Running via Spring preloader in process 8515
Run options: --seed 3074
# Running:
Run options: --seed 3074
# Running:
2018-02-14T14:46:14-06:00 - Changing from waiting_to_be_circulated to offered_to_user (event: offer)
..........
Finished in 1.059601s, 4.7188 runs/s, 4.7188 assertions/s.
5 runs, 5 assertions, 0 failures, 0 errors, 0 skips
Finished in 1.059780s, 4.7180 runs/s, 4.7180 assertions/s.
5 runs, 5 assertions, 0 failures, 0 errors, 0 skips
Any idea where I can check to see what may be causing the output to duplicate? I'm running Rails 5.1.5 and Ruby 2.4.1. Please let me know if you'd like to see any more information, I'm happy to provide it!
Here is my test_helper.rb
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require 'minitest/autorun'
require 'minitest/reporters'
# Minitest::Reporters.use!(
# Minitest::Reporters::ProgressReporter.new(:color => true),
# ENV,
# Minitest.backtrace_filter
# )
# Integrate AASM gem's custom testing methods into minitest
require 'aasm/minitest'
require 'fileutils' # Used below in setting up CarrierWave
... <various CarrierWave settings also here> ...
class ActiveSupport::TestCase
include ApplicationHelper
include ActionDispatch::TestProcess
# Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
fixtures :all
class ActionDispatch::IntegrationTest
# Log in as a particular user
def log_in_as(user, password: 'password', remember_me: '1')
post login_path, params: { session: { email: user.email,
password: password,
remember_me: remember_me } }
end
end

Minitest - NoMethodError: undefined method `get'

I was stuck with error when i run very simple test with minitest-rails gem.
I have rails 4.1.5 and minitest 5.4.0
rake test:controllers
1) Error:
DashboardController::index action#test_0001_anonymous:
NoMethodError: undefined method get' for #<#<Class:0x00000008e28170>:0x00000008eeb9b8>
test/controllers/dashboard_controller_test.rb:6:inblock (3 levels) in '
Test:
require "test_helper"
describe DashboardController do
context "index action" do
before do
get :index
end
it { must_respond_with :success }
it "must render index view" do
must_render_template :index
end
end
end
My test_helper:
ENV["RAILS_ENV"] = "test"
require File.expand_path("../../config/environment", __FILE__)
require "rails/test_help"
require "minitest/rails"
require "minitest/rails/capybara"
class MiniTest::Spec
class << self
alias :context :describe
end
end
class RequestTest < MiniTest::Spec
include Rails.application.routes.url_helpers
register_spec_type(/request$/, self)
end
class ActionDispatch::IntegrationTest
# Register "request" tests to be handled by IntegrationTest
register_spec_type(/Request( ?Test)?\z/i, self)
end
class ActiveSupport::TestCase
ActiveRecord::Migration.check_pending!
# Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
#
# Note: You'll currently still have to declare fixtures explicitly in integration tests
# -- they do not yet inherit this setting
fixtures :all
# Add more helper methods to be used by all tests here...
extend MiniTest::Spec::DSL
end
There are many things wrong with what you are doing. As I understand it you want to use Minitest's spec DSL in your Rails tests, correct? It looks like you are doing things to accomplish this that you don't need to do. I don't understand why half that code in your test_helper.rb file is there. I also suspect that you have other code doing things that are not being shown.
Here is what I did to reproduce your setup:
$ echo "Creating a new Rails app"
☣ [rails41:rails41] $ rails new undefined_get
☣ [rails41:rails41] $ cd undefined_get/
$ echo "Generate a Dashboard controller"
$ rails g controller dashboard index
$ echo "Add minitest-rails dependencies"
$ echo 'gem "minitest-rails"' >> Gemfile
$ echo 'gem "minitest-rails-capybara"' >> Gemfile
$ bundle install
$ echo "The test runs fine now:"
$ rake test
Run options: --seed 47210
# Running:
.
Finished in 0.457972s, 2.1835 runs/s, 2.1835 assertions/s.
1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
$ echo "Update to your test code and test_helper code"
$ echo "Use whatever editor you want. Not shown here."
$ echo "Now rerun the tests:"
$ rake test
rake aborted!
NoMethodError: undefined method `context' for #<Class:0x007f860258ae50>
The error I get is different than yours. You aliased the method context to describe in your test_helper.rb file, but unfortunately the object you aliased is not in the inheritance chain for the rails test objects. The rails test objects extend Minitest::Spec::DSL, but they do not inherit from Minitest::Spec. So, I am strongly suspicious that the code you provided is indeed producing the results you have presented. That said, here is the code in my test_helper.rb that will run your test:
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require "minitest/rails"
require "minitest/rails/capybara"
class ActiveSupport::TestCase
# Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
fixtures :all
# Allow context to be used like describe
class << self
alias :context :describe
end
# Add more helper methods to be used by all tests here...
end
This is the standard test_helper.rb with two changes. First, it has the requires for minitest-rails and minitest-rails-capybara. That is all you need to do in order to enable the Minitest spec DSL in your rails tests. Second, it adds the alias for context to describe on ActiveSupport::TestCase, which is the basis for all the rails tests. If you want to add tests that do not inherit from ActiveSupport::TestCase then you can also alias it on Minitest::Spec, but that will not help you use context within your controller tests.
Still here? Okay. So why did your code give you a different error than mine? Likely the test object used for your controller tests isn't ActionController::TestCase. I say that because your error was undefined method get. The get method is something that ActionController::TestCase defines, and is not on Minitest::Spec. So, you somehow messed up your Minitest configuration. A simple way to make sure that your tests are using the correct test objects is to add an additional assertion to your test. Like this:
require "test_helper"
describe DashboardController do
context "index action" do
before do
# Make sure we are using the correct test class
self.class.ancestors.must_include ActionController::TestCase
# Continue with setup
get :index
end
it { must_respond_with :success }
it "must render index view" do
must_render_template :index
end
end
end
If that first assertion fails then you know you have done something wrong in your configuration.

How to Test Factories First with Minitest but without the spec framework?

I found a blog post about Testing Factories First (by BigBinary - which happens to be a Minitest/spec version of Thoughtbot's RSpec original).
Could you please show me the equivalent without the spec framework - just with Minitest (Rails)?
The Thoughtbot approach (RSpec)
spec/factories_spec.rb
FactoryGirl.factories.map(&:name).each do |factory_name|
describe "The #{factory_name} factory" do
it 'is valid' do
build(factory_name).should be_valid
end
end
end
Rakefile
if defined?(RSpec)
desc 'Run factory specs.'
RSpec::Core::RakeTask.new(:factory_specs) do |t|
t.pattern = './spec/factories_spec.rb'
end
end
task spec: :factory_specs
The BigBinary approach (Minitest, spec)
spec/factories_spec.rb
require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
describe FactoryGirl do
EXCEPTIONS = %w(base_address base_batch bad_shipping_address)
FactoryGirl.factories.each do |factory|
next if EXCEPTIONS.include?(factory.name.to_s)
describe "The #{factory.name} factory" do
it 'is valid' do
instance = build(factory.name)
instance.must_be :valid?
end
end
end
end
lib/tasks/factory.rake
desc 'Run factory specs.'
Rake::TestTask.new(:factory_specs) do |t|
t.pattern = './spec/factories_spec.rb'
end
task test: :factory_specs
What is the Minitest equivalent (without spec)?
The approach I am presenting below is slightly different than the two original solutions - in the sense that my approach creates only one test, within which I cycle through the factories and run an assertion against each. I was not able to create a solution that mimics the original solutions any closer - which is (I believe) a separate test method for each factory. If someone could show such an implementation, that would be cool.
test/aaa_factories_tests/factories_test.rb
require File.expand_path(File.dirname(__FILE__) + '/../test_helper.rb')
class FactoriesTest < Minitest::Unit::TestCase
puts "\n*** Factories Test ***\n\n"
EXCEPTIONS = %w(name_of_a_factory_to_skip another_one_to_skip)
def test_factories
FactoryGirl.factories.each do |factory|
next if EXCEPTIONS.include?(factory.name.to_s)
instance = FactoryGirl.build(factory.name)
assert instance.valid?, "invalid factory: #{factory.name}, error messages: #{instance.errors.messages.inspect}"
instance = factory = nil
end
end
end
Thanks to the way Minitest works out of the box -- add any directories under test/ and minitest-rails will automatically create the associated rake task for it. So let's say you add a test/api/ directory, rake minitest:api will automagically be available. -- I see the task when I run bundle exec rake -T with no other configurations:
rake minitest:aaa_factories_tests # Runs tests under test/aaa_factories_tests
And I am able to run this task successfully:
-bash> bundle exec rake minitest:aaa_factories_tests
*** Factories Test ***
Run options: --seed 19208
# Running tests:
.
Finished tests in 0.312244s, 3.2026 tests/s, 9.6079 assertions/s.
1 tests, 3 assertions, 0 failures, 0 errors, 0 skips
Despite the ugliness of prepending the directory with aaa, I am able to have the factories tested first with:
bundle exec rake minitest:all
The reason for the aaa prepend solution is MiniTest does a Dir glob and on Mac OS X (and other Unix variants) the results are sorted alphabetically (though the results differ across different platforms).
As well, I prepended the default_tasks array with aaa_factories_tests to have the factories tested first in the default Minitest task (i.e. when running bundle exec rake minitest).
lib/tasks/factories_first.rake
MiniTest::Rails::Testing.default_tasks.unshift('aaa_factories_tests') if Rails.env =~ /^(development|test)\z/
Note that the above condition avoids erroneously referencing Minitest in environments where it is unavailable (I have confined minitest-rails to :test and :development groups in Gemfile). Without this if-condition, pushing to Heroku (for example to staging or production) will result in uninitialized constant MiniTest.
Of course I am also able to run the factories test directly:
bundle exec ruby -I test test/aaa_factories_tests/factories_test.rb
Here is a solution for MiniTest without the spec framework:
test/factories_test.rb
require File.expand_path(File.dirname(__FILE__) + '/test_helper')
class FactoriesTest < ActiveSupport::TestCase
EXCEPTIONS = %w(griddler_email)
FactoryBot.factories.map(&:name).each do |factory_name|
next if factory_name.to_s.in?(EXCEPTIONS)
context "The #{factory_name} factory" do
should 'be valid' do
factory = build(factory_name)
assert_equal true, factory.valid?, factory.errors.full_messages
end
end
end
end
lib/tasks/factory.rake
namespace :test do
desc 'Test factories'
Rake::TestTask.new(:factories) do |t|
t.pattern = './test/factories_test.rb'
end
end
task minitest: 'test:factories'
The most important thing is to use taks minitest instead of task test if you want the factories tests to be run before other tests.

How to get the minitest capture_io method found in my specs?

This is my spec/helper.rb file:
require 'minitest/autorun'
require 'minitest/spec'
require_relative '../lib/launcher'
class MiniTest::Spec
include MiniTest::Assertions
end
And this is the spec file:
require 'helper'
describe Launcher do
it "should get the stdout" do
out, err = capture_io do
warn "You did a bad thing"
end
err.should =~ /bad/
end
end
But when I run the spec:
± rake spec
/Users/millisami/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/launcher_spec.rb -Ispec:lib
F
Failures:
1) Launcher should be a launcher for cloud
Failure/Error: out, err = capture_io do
NoMethodError:
undefined method `capture_io' for #<RSpec::Core::ExampleGroup::Nested_1:0x000001012a4840>
# ./spec/launcher_spec.rb:5:in `block (2 levels) in <top (required)>'
Finished in 0.00044 seconds
1 example, 1 failure
Failed examples:
rspec ./spec/launcher_spec.rb:4 # launcher should be a launcher for cloud
rake aborted!
/Users/millisami/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/launcher_spec.rb -Ispec:lib failed
Tasks: TOP => spec
But when I do this with the default MiniTest::Unit way, it works?
And why not the spec way?
Your spec_helper.rb should look more like this:
require 'minitest/unit'
RSpec.configure do |c|
c.include MiniTest::Assertions
end

Resources