I have feature specs for my ActiveAdmin view. It works on my local machine. But when ran in CircleCi it fails with
undefined method `text' for nil:NilClass
spec
it 'uses the update_at date when prepaid_invoice' do
travel_to(5.days.ago) { create_prepayment }
travel_to(3.days.ago) do
visit '/admin/payments'
expect(page).not_to have_css('.col.col-created > div')
amount = all('.col-amount').last
expect(amount.text).to eq('$1,000.00') # failing here
all behaves different depending on whether you're on a current release of Capybara or the older 2.x version. In current versions all will wait for up to Capybara.default_max_wait_time seconds for any matching elements to appear and if none do it will return an empty array (actually Array like Result object, but close enough). In 2.x all (by default) did no waiting for matching elements, it would just return the empty array if no elements matched. Either way - you're not finding any matching elements, and then calling last on an empty array - giving you nil.
There are a couple of ways around this. You could tell all that you expect at least one matching element ( which will then force waiting for matching elements Capybara 2.x )
amount = all('.col-amount', minimum: 1).last
or depending on exactly what you're checking you could just combine it into one
expect(page).to have_css('.col-amount', exact_text:'$1,000.00')
which gets away from calling methods on Capybara elements and using the generic RSpec matchers on them (which is something you don't want to do, for test stability reasons, unless you have no other options). Those two solution don't test exactly the same thing, but may test enough for what you want. Depending on exactly how your HTML is structured there may be more efficient solutions too.
If you're already using a recent version of Capybara then your error would indicate that what you expect to be on the page isn't (maybe you're on an error page, etc) or you don't have Capybara.default_max_wait_time set high enough for the system you're testing on.
Related
I have an existing application (Rails 6) with a set of tests (minitest). I've just converted my tests to use factory_bot instead of fixtures but I'm having a strange problem with records created and confirmed in the test and controller being unavailable in a PORO that does the actual work. This problem occurs inconsistently and never seems to happen when I run an individual test, only when tests are run in bulk (e.g. a whole file or directory). The more tests are run, the more likely the failure, it seems.
(NB I've never seen this code fail in actual use - it only happens during tests.)
Summary
Previously, when using fixtures, every test ran successfully both individually and when run all together with rails t. Now, with factory_bot, a few of my tests often (but not always) fail, all related to the use of the same object that is defined as a PORO.
Drilling down, I have found that there's an issue with records sometimes mysteriously going missing or being unavailable within the PORO during the test, even though they're confirmed as present in the test and in the controller that calls the PORO!
Details
In my application, I have a RichText object that receives some text and processes it, highlighting words in the text that match those stored in a Dictionary table. In my tests, I create several test Dictionary records, and expect the RichText object to perform appropriately when passed test data. And it does, when the individual test files are run (and always did when I used fixtures).
However, now, the records are created and available in the test and in the controller it calls, but then are not available within the RichText object created by the controller. With no Dictionary records available in the RichText object, the test naturally fails because no words are highlighted in the text. And, again, this only seems to happen when I run the tests as a group rather than running just a single test file (e.g. rails t test/objects/rich_text.rb passes, but rails t test/objects will fail within the same rich_text.rb test file).
It doesn't seem to matter whether I create the records with factory_bot#create or directly with Dictionary#create, which suggests it's nothing to do with factory_bot - but then why has this just started happening?
I do have parallelisation enabled in minitest but disabling it makes no difference - the tests still fail the same way.
Code
Example test code that runs and passes, up to the last assertion here, which sometimes fails as described above:
test 'can create new content' do
create(:dictionary, word_root: 'word_1')
create(:dictionary, word_root: 'word_2')
create(:dictionary, word_root: 'word_3')
assert_equal 3, Dictionary.all.count
...
# This next line is the one that calls the relevant controller code below
post '/api/v0/content', headers: #auth_headers, params: #new_content_params
...
# This assertion passes, as it did above, even though the error's already happened after the post above
assert_equal 3, Dictionary.all.count
# This assertion checks the response from the above post and fails under certain circumstances, as described above
assert_equal #new_content_output, response.body
...
end
I've added checks to the controller as below and, again, everything's fine through this code, which is called by the post line in the test above (i.e. the database records are present and correct just before the RichText object is called):
def create
...
byebug unless Dictionary.all.count == 3
rich_text = RichText::Basic.new(#organisation, new_version[:content])
...
end
However, the RichText object's initialize method immediately fails the same check for these records - but only if the test is being run in bulk rather than individually:
class RichText::Basic
def initialize(organisation, text)
byebug unless Dictionary.all.count == 3
...
end
end
Rails 6.1.4, ruby 2.7.1
Having tried various things (like disabling transactions in the affected tests), I found that the root cause was a constant defined in the RichText class (a line I didn't include in the question!). It looks like there was a race condition or similar that meant that the RichText class sometimes ran before the database was populated, leaving it with an empty constant.
Replacing the constant with a direct database call resolved the problem. It does mean slightly more database calls but, on the flip side, does mean it's slightly easier to update the Dictionary table. (This happens rarely - on the order of once a month - which is why I'd put it into a constant.)
From:
class RichTest::Basic
WORDS = Dictionary.standard
def some_method
WORDS.each do...
end
end
to
class RichTest::Basic
def some_method
Dictionary.standard.each do...
end
end
I've just started to take on my first model spec task at work. After writing a lot of feature specs, I find it hard to get into the different perspective of writing model specs (not taking the context into consideration). I'll take a method of the Order model as an example, to explain which difficulties I am experiencing:
def update_order_prices
self.shipping_price_cents = SHIPPING_PRICE_CENTS unless shipping_price_cents
return if order_lines.empty?
self.total_price_cents = calculate_order_price
self.total_line_items_price_cents = calculate_total_order_line_price
self.total_tax_cents = calculate_tax_amount
end
EDIT TL;DR
I am totally happy with an answer that simply writes me a spec for this method. The rest of the post just shows what I tried so far but is not necessary to answer this question.
First approach:
At first I didn't know what to test for. I tried to find out when and where the method was called and to find a scenario where I would know what the attributes that are touched in this method should be equal to. Put short, I spent a lot of time trying to understand the context. Then a coworker said that I should test methods in model specs self-contained, independent from the context. I should just make sure I identify all cases. So for this method that would be:
it sets shipping price cents to default (if not done already)
it returns early if order_lines is empty
it sets values if order_line is set
Current approach:
I tried writing the tests for these points but still questions arise:
Test 1
it 'sets shipping price cents to default (if not done already)' do
order.shipping_price_cents = nil
order.update_order_prices
expect(order.shipping_price_cents).to eq(Order::SHIPPING_PRICE_CENTS)
end
I am confident I got this one right, but feel free to prove me wrong. I set shipping_price_cents to nil to trigger the code that sets it, call the tested method on the cents to be equal to the default value as defined in the model.
Test 2
it 'returns early if order_lines is empty' do
expect(order.update_order_prices).to eq(nil)
end
So here I want to test that the method returns early when there is no object in the order_lines association. I didn't have a clue how to do that so I went into the console, took an order, removed the order_lines associated with it, and called the method to see what would be returned.
2.3.1 :011 > o.order_lines
=> #<ActiveRecord::Associations::CollectionProxy []>
2.3.1 :012 > o.update_order_prices
=> nil
Then did the same for an order with associated order_line:
2.3.1 :017 > o.update_order_prices
=> 1661
So I tested for 'nil' to be returned. But it doesn't feel like I am testing the right thing.
Test 3
it 'sets (the correct?) values if order_line is set' do
order_line = create(:order_line, product: product)
order = create(:order, order_lines: [order_line])
order.update_order_prices
expect(order.total_price_cents).to eq(order.calculate_order_price)
expect(order.total_line_items_price_cents).to eq(order.calculate_order_line_price)
expect(order.total_tax_cents).to eq(order.calculate_tax_amount)
end
I simply test that the attributes equal what they are set to, without using actual values, as I shouldn't look outside. If I wanted to test for an absolute value, I would have to investigate outside of this function which then wouldn't test the method but also status of the Order object etc.?
Running the tests
Failures:
1) Order Methods: #update_order_prices sets (the correct?) values if order_line is set
Failure/Error: expect(order.total_price_cents).to eq(order.calculate_order_price)
NoMethodError:
private method `calculate_order_price' called for #<Order:0x007ff9ee643df0>
Did you mean? update_order_prices
So, the first two tests passed, the third one didn't. At this point I feel a bit lost and would love hear how some experienced developers would write this seemingly simple test.
Thanks
I guess you have to spec against the exact values you are expecting after update_order_prices.
Let's say you set up your order and order lines to have a total price of 10 euros then I'd add the following expectation
expect(order.total_price_cents).to eq(1000)
Same for the other methods. Generally I try to test against specific values. Also as you are relying on the result of a private method you only care about the result.
It said here https://www.relishapp.com/rspec/rspec-core/v/3-5/docs/helper-methods/let-and-let what variable defined by let is changing across examples.
I've made the same simple test as in the docs but with the AR model:
RSpec.describe Contact, type: :model do
let(:contact) { FactoryGirl.create(:contact) }
it "cached in the same example" do
a = contact
b = contact
expect(a).to eq(b)
expect(Contact.count).to eq(1)
end
it "not cached across examples" do
a = contact
expect(Contact.count).to eq(2)
end
end
First example passed, but second failed (expected 2, got 1). So contacts table is empty again before second example, inspite of docs.
I was using let and was sure it have the same value in each it block, and my test prove it. So suppose I misunderstand docs. Please explain.
P.S. I use DatabaseCleaner
P.P.S I turn it off. Nothing changed.
EDIT
I turned off DatabaseCleaner and transational fixtures and test pass.
As I can understand (new to programming), let is evaluated once for each it block. If I have three examples each calling on contact variable, my test db will grow to three records at the end (I've tested and so it does).
And for right test behevior I should use DatabaseCleaner.
P.S. I use DatabaseCleaner
That's why your database is empty in the second example. Has nothing to do with let.
The behaviour you have shown is the correct behaviour. No example should be dependant on another example in setting up the correct environment! If you did rely on caching then you are just asking for trouble later down the line.
The example in that document is just trying to prove a point about caching using global variables - it's a completely different scenario to unit testing a Rails application - it is not good practice to be reliant on previous examples to having set something up.
Lets, for example, assume you then write 10 other tests that follow on from this, all of which rely on the fact that the previous examples have created objects. Then at some point in the future you delete one of those examples ... BOOM! every test after that will suddenly fail.
Each test should be able to be tested in isolation from any other test!
I have a RSpec test with this:
within all('tr')[1] do
expect(page).to have_content 'Title'
expect(page).to have_content 'Sub Title'
end
And it's failing at expect(page).to have_content 'Title' with the following error message:
Element at 54 no longer present in the DOM
I have not been able to find the exact meaning of what this error message means and this test is flakey, sometimes it passes, sometimes not.
Unlike other capybara finders, all doesn't really wait for elements to appear in the DOM. So if your table is not fully loaded, it simply goes straight to the expectation within that block and could potentially fail. That could easily explain why sometimes it fails and other times it succeeds.
I suggest using another expectation before this block and ensure that the table is fully loaded first. I don't know what your DOM looks like, but you can try something like:
expect(page).to have_css('tr td', :count => 15)
So at this point, you've waited for 15 rows to show up in the DOM prior to moving onto your next steps. Hope that helps.
As the other answer details, #all by default doesn't wait for elements to appear so it's possible you're not actually getting the elements you think you are. Rather than the solution in the other answer, it is possible to make #all wait for elements to appear by specifying one of the :count, :minimum, :maximum, or :between options.
within all('tr', minimum: 10)[1] do
...
end
for instance. This is all assuming that the action before the #within in your test isn't triggering an ajax action that is replacing existing rows in a table with other rows. If row replacement is happening then you may be running into one of the biggest downsides of using #all -- when using #all the returned elements cannot automatically be re-queried if they leave the page and are replaced, since their entire query can't be stored with them (no index into the results). In that case you're better off changing the code to
within find(:xpath, './/tr[2]') do
...
end
This way the element you're searching within can be reloaded automatically if needed
Which Rspec convention is more up to date and should be used in new projects ?
subject { [] }
it { should == [] }
or
subject { [] }
it { expect(subject).to eq([]) }
I haven't found way of composing shorter version using subject implicitly with expect method.
Using expect (your second example) is the more up to date version. Rspec is moving away from methods being added onto existing objects (such as should) because they can occasionally cause odd behavior.
As stated in http://myronmars.to/n/dev-blog/2012/06/rspecs-new-expectation-syntax
"In the future, we plan to change the defaults so that only expect is available unless you explicitly enable should. We may do this as soon as RSpec 3.0, but we want to give users plenty of time to get acquianted with it"
I copy from this site: "The underlying problem is RSpec’s should syntax: for should to work properly, it must be defined on every object in the system… but RSpec does not own every object in the system and cannot ensure that it always works consistently. As we’ve seen, it doesn’t work as RSpec expects on proxy objects. Note that this isn’t just a problem with RSpec; it’s a problem with minitest/spec’s must_xxx syntax as well."
Personally, I use a mix of both still, as all "shoulda" matchers use the old syntax anyway, and are so easy to use...