I have a test more or less like this:
class FormDefinitionTest < ActiveSupport::TestCase
context "a form_definition" do
setup do
#definition = SeedData.form_definition
# ...
I've purposely added a
raise "blah"
somewhere down the road and I get this error:
RuntimeError: blah
test/unit/form_definition_test.rb:79:in `__bind_1290079321_362430'
when I should be getting something along:
/Users/pupeno/projectx/db/seed/sheet_definitions.rb:17:in `sheet_definition': blah (RuntimeError)
from /Users/pupeno/projectx/db/seed/form_definitions.rb:4:in `form_definition'
from /Users/pupeno/projectx/test/unit/form_definition_test.rb:79
Any ideas what is sanitizing/destroying my backtraces? My suspicious is shoulda because the when the exception happens inside a setup or should is whet it happens.
This is a Rails 3 project, in case that's important.
That is because the shoulda method #context is generating code for you. for each #should block it generates a completely separate test for you so e.g.
class FormDefinitionTest < ActiveSupport::TestCase
context "a form_definition" do
setup do
#definition = SeedData.form_definition
end
should "verify some condition" do
assert something
end
should "verify some other condition" do
assert something_else
end
end
end
Then #should will generate two completely independent tests (for the two invocations of #should), one that executes
#definition = SeedData.form_definition
assert something
and another one that executes
#definition = SeedData.form_definition
assert something_else
It is worth noting that it does not generate one single test executing all three steps in a sequence.
These generated blocks of codes have method names like _bind_ something and the generated test have name that is a concatenation of all names of the contexts traversed to the should block plus the string provided by the should block (prefixed with "should "). There is another example in the documentation for shoulda-context.
I think this will give you the backtrace that you want. I haven't tested it, but it should work:
def exclude_backtrace_from_location(location)
begin
yeild
rescue => e
puts "Error of type #{e.class} with message: #{e.to_s}.\nBacktrace:"
back=e.backtrace
back.delete_if {|b| b~=/\A#{location}.+/}
puts back
end
end
exclude_backrace_from_location("test/unit") do
#some shoulda code that raises...
end
Have you checked config/initializers/backtrace_silencers.rb? That is the entry point to customize that behavior. With Rails.backtrace_cleaner.remove_silencers! you can cleanup the silencers stack.
More informations about ActiveSupport::BacktraceCleaner can be found here.
Related
Background
I have a rails model that contains an ActiveRecord::Enum. I have a view helper that takes a value of this enum, and returns one of several possible responses. Suppose the cases were called enum_cases, for example:
enum_cases = [:a, :b, :c]
def foo(input)
case input
when :a then 1
when :b then 2
when :c then 3
else raise NotImplementedError, "Unhandled new case: #{input}"
end
end
I want to unit-test this code. Checking the happy paths is trivial:
class FooHelperTests < ActionView::TestCase
test "foo handles all enum cases" do
assert_equal foo(:a), 1
assert_equal foo(:b), 2
assert_equal foo(:c), 3
assert_raises NotImplementedError do
foo(:d)
end
end
end
However, this has a flaw. If new cases are added (e.g. :z), foo will raise an error to bring our attention to it, and add it as a new case. But nothing stops you from forgetting to update the test to test for the new behaviour for :z. Now I know that's mainly the job of code coverage tools, and we do use one, but just not to such a strict level that single-line gaps will blow up. Plus this is kind of a learning exercise, anyway.
So I amended my test:
test "foo handles all enum cases" do
remaining_cases = enum_cases.to_set
tester = -> (arg) do
remaining_cases.delete(arg)
foo(arg)
end
assert_equal tester.call(:a), 1
assert_equal tester.call(:b), 2
assert_equal tester.call(:c), 3
assert_raises NotImplementedError do
tester.call(:d)
end
assert_empty remaining_cases, "Not all cases were tested! Remaining: #{remaining_cases}"
end
This works great, however it's got 2 responsibilities, and it's a pattern I end up copy/pasting (I have multiple functions to test like this):
Perform the actual testing of foo
Do book keeping to ensure all params were exhausitvely checked.
I would like to make this test more focused by removing as much boiler plate as possible, and extracting it out to a place where it can easily be reused.
Attempted solution
In another language, I would just extract a simple test helper:
class ExhaustivityChecker
def initialize(all_values, proc)
#remaining_values = all_values.to_set
#proc = proc
end
def run(arg, allow_invalid_args: false)
assert #remaining_values.include?(arg) unless allow_invalid_args
#remaining_values.delete(arg)
#proc.call(arg)
end
def assert_all_values_checked
assert_empty #remaining_values, "Not all values were tested! Remaining: #{#remaining_values}"
end
end
Which I could easily use like:
test "foo handles all enum cases" do
tester = ExhaustivityChecker.new(enum_cases, -> (arg) { foo(arg) })
assert_equal tester.run(:a), 1
assert_equal tester.run(:b), 2
assert_equal tester.run(:c), 3
assert_raises NotImplementedError do
tester.run(:d, allow_invalid_args: true)
end
tester.assert_all_values_checked
end
I could then reuse this class in other tests, just by passing it different all_values and proc arguments, and remembering to call assert_all_values_checked.
Issue
However, this breaks because I can't call assert and assert_empty from a class that isn't a subclass of ActionView::TestCase. Is it possible to subclass/include some class/module to gain access to these methods?
enum_cases must be kept up to date when the production logic changes violating the DRY principle. This makes it more likely for there to be a mistake. Furthermore it is test code living in production, another red flag.
We can solve this by refactoring the case into a Hash lookup making it data driven. And also giving it a name describing what it's associated with and what it does, these are "handlers". I've also turned it into a method call making it easier to access and which will bear fruit later.
def foo_handlers
{
a: 1,
b: 2,
c: 3
}.freeze
end
def foo(input)
foo_handlers.fetch(input)
rescue KeyError
raise NotImplementedError, "Unhandled new case: #{input}"
end
Hash#fetch is used to raise a KeyError if the input is not found.
Then we can write a data driven test by looping through, not foo_handlers, but a seemingly redundant expected Hash defined in the tests.
class FooHelperTests < ActionView::TestCase
test "foo handles all expected inputs" do
expected = {
a: 1,
b: 2,
c: 3
}.freeze
# Verify expect has all the cases.
assert_equal expect.keys.sort, foo_handlers.keys.sort
# Drive the test with the expected results, not with the production data.
expected.keys do |key|
# Again, using `fetch` to get a clear KeyError rather than nil.
assert_equal foo(key), expected.fetch(value)
end
end
# Simplify the tests by separating happy path from error path.
test "foo raises NotImplementedError if the input is not handled" do
assert_raises NotImplementedError do
# Use something that obviously does not exist to future proof the test.
foo(:does_not_exist)
end
end
end
The redundancy between expected and foo_handlers is by design. You still need to change the pairs in both places, there's no way around that, but now you'll always get a failure when foo_handlers changes but the tests do not.
When a new key/value pair is added to foo_handlers the test will fail.
If a key is missing from expected the test will fail.
If someone accidentally wipes out foo_handlers the test will fail.
If the values in foo_handlers are wrong, the test will fail.
If the logic of foo is broken, the test will fail.
Initially you're just going to copy foo_handlers into expected. After that it becomes a regression test testing that the code still works even after refactoring. Future changes will incrementally change foo_handlers and expected.
But wait, there's more! Code which is hard to test is probably hard to use. Conversely, code which is easy to test is easy to use. With a few more tweaks we can use this data-driven approach to make production code more flexible.
If we make foo_handlers an accessor with a default that comes from a method, not a constant, now we can change how foo behaves for individual objects. This may or may not be desirable for your particular implementation, but its in your toolbox.
class Thing
attr_accessor :foo_handlers
# This can use a constant, as long as the method call is canonical.
def default_foo_handlers
{
a: 1,
b: 2,
c: 3
}.freeze
end
def initialize
#foo_handlers = default_foo_handlers
end
def foo(input)
foo_handlers.fetch(input)
rescue KeyError
raise NotImplementedError, "Unhandled new case: #{input}"
end
end
Now individual objects can define their own handlers or change the values.
thing = Thing.new
puts thing.foo(:a) # 1
puts thing.foo(:b) # 2
thing.foo_handlers = { a: 23 }
puts thing.foo(:a) # 23
puts thing.foo(:b) # NotImplementedError
And, more importantly, a subclass can change their handlers. Here we add to the handlers using Hash#merge.
class Thing::More < Thing
def default_foo_handlers
super.merge(
d: 4,
e: 5
)
end
end
thing = Thing.new
more = Thing::More.new
puts more.foo(:d) # 4
puts thing.foo(:d) # NotImplementedError
If a key requires more than a simple value, use method names and call them with Object#public_send. Those methods can then be unit tested.
def foo_handlers
{
a: :handle_a,
b: :handle_b,
c: :handle_c
}.freeze
end
def foo(input)
public_send(foo_handlers.fetch(input), input)
rescue KeyError
raise NotImplementedError, "Unhandled new case: #{input}"
end
def handle_a(input)
...
end
def handle_b(input)
...
end
def handle_c(input)
...
end
This question tells me how to test logger statements from RSpec model and controller specs. Unfortunately, it doesn't seem to work from a feature spec. With this code:
# controller.rb
def action
logger.info 'foobar'
end
# spec.rb
scenario 'logging should work' do
expect(Rails.logger).to receive(:info).with('foobar')
visit action_path
end
I get the error:
Failure/Error: visit action_path
#<ActiveSupport::Logger:0x007ff45b6e5ad0> received :info with unexpected arguments
expected: ("foobar")
got: (no args)
The test.log file does not contain foobar, so it seems the test is failing immediately, before the controller action has a chance to complete.
Is there some way to use this expect(Rails.logger) syntax in a feature spec?
The Rails.logger.info method can take a string or a block. If you're invoking the block form then it will give this "got: (no args)" output.
For example
logger.info 'foobar'
...all on one line will call .info with a string, but if you do
logger.info
"foobar foobar longer message so I'll put it on its own line"
split across two lines without brackets, then you're actually passing a block. Add some brackets...
logger.info(
"foobar foobar longer message so I'll put it on its own line"
)
...and you're back to a string.
He says knowingly after bashing his head on this problem for a few hours :-)
Before realising that, I started figuring out how to mock the Rails.logger class. That might be a useful approach for you or others. Maybe you're calling with a block for some other reason (something to do with feature vs controller specs?), or maybe you can't change the calling code, in which case... something like this might be a useful starting point:
class LoggerMock < Object
def initialize; end
def info(progname = nil, &block)
mock_info(block.call)
end
end
and
logger_mock = LoggerMock.new
allow(Rails).to receive(:logger).and_return(logger_mock)
expect(logger_mock).to receive(:mock_info).with('foobar')
Scenario
Have a race case where concurrency can cause a duplicate key error. Take for example:
def before_create_customer_by_external_id
end
def create_customer_from_external_id(external_id = nil)
#customer = current_account.customers.create!(external_id: external_id || #external_id)
end
def set_new_or_old_customer_by_external_id
if #customer.blank?
before_create_customer_by_external_id
create_customer_from_external_id
end
rescue ActiveRecord::RecordInvalid => e
raise e unless Customer.external_id_exception?(e)
#customer = current_account.customers.find_by_external_id(#external_id)
end
The Test
Now, to test the race case (based on the answer to Simulating race conditions in RSpec unit tests) we just need to monkey patch before_create_customer_by_external_id to call create_customer_from_external_id.
The Question
How can you do this without overriding the whole class and getting a "method not found" error?
After some digging, I came up with the following solution:
context 'with race condition' do
it 'should hit race case and do what is expected' do
ControllerToOverride.class_eval do
def before_create_new_customer_by_external_id
create_customer_from_external_id
end
end
# ...expect...
ControllerToOverride.class_eval do
undef before_create_new_customer_by_external_id
end
end
end
I verified that it was hitting the race case by using a code coverage tool and debug statements.
Happy to know if there's a cleaner way here.
Edit 2020-04-24
Per the comment, we should undef this method so it doesn't affect subsequent tests. Ref: https://medium.com/#scottradcliff/undefining-methods-in-ruby-eb7fba21f63f
I did not verify this, as I no longer have this test suite. Please let me know if it does/does not work.
A step on from monkey patching the class is to create an anonymous subclass:
context "with race condition" do
controller(ControllerToOverride) do
def before_create_customer_by_external_id
end
end
it "should deal with it " do
routes.draw { # define routes here }
...
end
end
This is not so very different to your solution but keeps the monkeypatch isolated to that context block.
You may not need the custom routes block - rspec sets up some dummy routes for the rest methods (edit, show, index etc)
If this context is inside a describe ControllerToOverride block then the argument to controller is optional, unless you have turned off config.infer_base_class_for_anonymous_controllers
I have a bunch of codes with repeating structures in a feature test in Rails. I would like to dry up my spec by reusing the structure. Any suggestions?
An example is:
feature "Search page"
subject { page }
it "should display results"
# do something
within "#A" do
should have_content("James")
end
within "#B" do
should have_content("October 2014")
end
# do something else
# code with similar structure
within "#A" do
should have_content("Amy")
end
within "#B" do
should have_content("May 2011")
end
end
At first, I tried to define a custom matcher in RSpec, but when I add within block, it did not seem to work. My guess is within is specific to Capybara, and cannot be used in custom matcher in RSpec.
Why not factor the common code into helper methods in a module. Then you can include that module in your spec_helper.rb file
I usually put common code like user_login in such a module in a file in the spec/support folder
spec_helper.rb
#Load all files in spec/support
Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}
RSpec.configure do |config|
#some config
config.include LoginHelper
#more config
end
spec/support/login_helper.rb
module LoginHelper
def do_login(username, password)
visit root_path
within("#LoginForm") do
fill_in('username', :with => username)
fill_in('password', :with => password)
click_button('submit')
end
end
end
I don't think you're using within as a matcher, since a matcher would be used after a should, should_not, etc. You can load custom, non-matcher methods into your specs by writing a module and including it in your spec_helper.rb config block, e.g.:
spec/support/my_macros.rb
module MyMacros
def within(tag, &block)
# your code here
end
end
spec/spec_helper.rb
require 'spec/support/my_macros'
...
RSpec.configure do |config|
config.include MyMacros
...
end
I'm using Capybara + Cucumber for end-to-end testing. In the end, I think I've pretty much done what both #hraynaud and #eirikir suggest (directionally speaking) - although the details are different since I'm in the Cucumber context. So, consider this not a whole different idea - but maybe a slightly more complete description and discussion. Also, note that my examples focus on testing results - not navigation and form filling. Since it looked like you were in a testing mindset (given your use of should have_content), I thought this might be of interest.
In general, my approach is:
Wrap Capybara tests in validation helper methods within a module. The motivation for wrapping is (a) to save me from having to remember Capybara syntax and (b) to avoid having to type all those repetitive test statements. Also, it ends up making my tests cleaner and more readable (at least for me).
Create a generic validate method that receives (i) a validation helper method name (as a symbol) and (ii) an array of items each of which is to be passed to the validation helper. The validate method simply iterates over the array of items and calls the validation helper method (using the send method), passing each item along with each call.
Attach the helpers and generic validate method to World (read more about World here) so that they are available throughout my Cucumber tests.
Enjoy testing happiness!
Steps 1-3 happen in a file called form_validation_helpers.rb.
features/support/form_validation_helpers.rb
module FormValidationHelpers
...more methods before
# ============================================================================
# Tests that an element is on the page
# ============================================================================
def is_present(element)
expect(find(element)).to be_truthy
end
# ============================================================================
# Tests for the number of times an element appears on a page
# ============================================================================
def number_of(options={})
page.should have_css(options[:element], count: options[:count])
end
# ============================================================================
# Checks that a page has content
# ============================================================================
def page_has_content(content)
page.should have_content(content)
end
...more methods after
# ============================================================================
# The generic validation method
# ============================================================================
def validate(validation, *items)
items.each do |item|
send(validation, item)
end
end
end
World(FormValidationHelpers)
Step 4 (from above) happens in my step files.
features/step_definitions/sample_steps.rb
Then(/^she sees the organization events content$/) do
validate :number_of,
{element: 'ul#organization-tabs li.active', is: 1}
validate :is_present,
"ul#organization-tabs li#events-tab.active"
validate :page_has_content,
"A Sample Organization that Does Something Good",
"We do all sorts of amazing things that you're sure to love."
end
As you can see from the validate :page_has_content example, I can run the test multiple times by adding the appropriate arguments onto the validate call (since the validate method receives everything after the first argument into an array).
I like having very specific selectors in my tests - so I can be sure I'm testing the right element. But, when I start changing my view files, I start breaking my tests (bad) and I have to go back and fix all the selectors in my tests - wherever they may be. So, I made a bunch of selector helpers and attached them to World the same as above.
features/support/form_selectors_helpers.rb
module FormSelectorsHelper
...more _selector methods before
def event_summary_selector
return 'input#event_summary[type="text"]'
end
...more _selector methods after
end
World(FormSelectorsHelper)
So now, I have only one place where I need to keep my selectors up to date and accurate. Usage is as follows (note that I can pass whatever the validation helper method needs - strings, methods, hashes, arrays, etc.)...
features/step_definitions/more_sample_steps.rb
Then(/^she sees new event form$/) do
validate :is_present,
event_summary_selector,
start_date_input_selector,
start_time_input_selector,
end_time_input_selector
validate :is_absent,
end_date_input_selector
validate :is_unchecked,
all_day_event_checkbox_selector,
use_recur_rule_checkbox_selector
validate :is_disabled,
submit_button_selector
validate :has_text,
{ element: modal_title_bar_selector, text: "Okay, let's create a new event!" }
end
Turning back to your question, I imagine you could end up with something like:
feature "Search page"
subject { page }
it "should display results"
# do something
validate :has_content_within,
[a_selector, "James"],
[b_selector, "October 2014"]
# do something else
validate :has_content_within,
[a_selector, "Amy"],
[b_selector, "May 2011"]
end
Capybara Test Helpers provides a nice way to encapsulate test code when using Capybara + RSpec.
RSpec.feature "Search page", test_helpers: [:search] do
before do
visit search_path
end
it "should display results"
search.filter_by(name: 'James')
search.should.have_result(name: 'James', date: 'October 2014')
search.filter_by(name: 'Amy')
search.should.have_result(name: 'Amy', date: 'May 2011')
end
end
You can then implement your own actions and assertions as needed:
class SearchTestHelper < Capybara::TestHelper
aliases(
name_container: '#A',
date_container: '#B',
)
def filter_by(attrs)
attrs.each { |key, name| ... }
click_link('Search')
end
def have_result(name:, date:)
have(:name_container, text: name)
within(:date_container) { have_content(date) } # equivalent
end
end
You can read the guide here.
So based on my understanding, I beleive when you do
Resque.inline = Rails.env.test?
Your resque tasks will run synchronously. I am writing a test on resque task that gets enqueue during an after_commit callback.
after_commit :enqueue_several_jobs
#class PingsEvent < ActiveRecord::Base
...
def enqueue_several_jobs
Resque.enqueue(PingFacebook, self.id)
Resque.enqueue(PingTwitter, self.id)
Resque.enqueue(PingPinterest, self.id)
end
In the .perform methd of my Resque task class, I am doing a Rails.logger.info and in my test, I am doing something like
..
Rails.logger.should_receive(:info).with("PingFacebook sent with id #{dummy_event.id}")
PingsEvent.create(params)
And I have the same test for PingTwitter and PingPinterest.
I am getting failure on the 2nd and third expectation because it seems like the tests actually finish before all the resque jobs get run. Only the first test actually passes. RSpec then throws a MockExpectationError telling me that Rails.logger did not receive .info for the other two tests. Anyone has had experience with this before?
EDIT
Someone mentioned that should_receive acts like a mock and that I should do .exactly(n).times instead. Sorry for not making this clear earlier, but I have my expectations in different it blocks and I don't think a should_receive in one it block will mock it for the next it block? Let me know if i'm wrong about this.
class A
def bar(arg)
end
def foo
bar("baz")
bar("quux")
end
end
describe "A" do
let(:a) { A.new }
it "Example 1" do
a.should_receive(:bar).with("baz")
a.foo # fails 'undefined method bar'
end
it "Example 2" do
a.should_receive(:bar).with("quux")
a.foo # fails 'received :bar with unexpected arguments
end
it "Example 3" do
a.should_receive(:bar).with("baz")
a.should_receive(:bar).with("quux")
a.foo # passes
end
it "Example 4" do
a.should_receive(:bar).with(any_args()).once
a.should_receive(:bar).with("quux")
a.foo # passes
end
end
Like a stub, a message expectation replaces the implementation of the method. After the expectation is fulfilled, the object will not respond to the method call again -- this results in 'undefined method' (as in Example 1).
Example 2 shows what happens when the expectation fails because the argument is incorrect.
Example 3 shows how to stub multiple invocations of the same method -- stub out each call with the correct arguments in the order they are received.
Example 4 shows that you can reduce this coupling somewhat with the any_args() helper.
Using should_receive behaves like a mock. Having multiple expectations on the same object with different arguments won't work. If you change the expectation to Rails.logger.should_receive(:info).exactly(3).times your spec will probably past.
All that said, you may want to assert something more pertinent than what is being logged for these specs, and then you could have multiple targeted expectations.
The Rails.logger does not get torn down between specs, so it doesn't matter if the expectations are in different examples. Spitting out the logger's object id for two separate examples illustrates this:
it 'does not tear down rails logger' do
puts Rails.logger.object_id # 70362221063740
end
it 'really does not' do
puts Rails.logger.object_id # 70362221063740
end