Testing Rails model validations with RSpec, without testing AR itself
Lets as setup we have model User:
class User < ActiveRecord::Base
validate :name, presence: true, uniqueness: { case_sensitive: false }, on: :create
validate :password, presence: true, format: { with: /\A[a-zA-z]*\z/ }
end
A see several ways to test this:
it { expect(user).to validate_presence_of(:name).on(:create) }
or
it do
user = User.create(name: '')
expect(user.errors[:name]).to be_present
end
My main question is which of the approaches is better and why? Can suggest me different approach?
Additional questions:
How much should I test? As an example, I can write so many tests for the regex, but it will be hell for maintenance.
How much you think will be full test coverage in this example?
The functionalities of:
Rails being able to validate the presence of an arbitrary value on your model
errors being added to an object for an attribute that is missing when a validation for it is configured
are covered in the tests for Rails itself (specifically, in the ActiveModel tests).
That leaves needing to write the tests for the config that covers the business logic of your app eg validating the presence of the specific name attribute on your specific User class etc. In my opinion, the matchers from the shoulda-matchers gem should have you covered:
RSpec.describe User, type: :model do
subject(:user) { build(:user) } # assuming you're using FactoryGirl
describe 'validations' do
specify 'for name' do
expect(user).to validate_presence_of(:name).on(:create)
# NOTE: saving here is needed to test uniqueness amongst users in
# the database
user.save
expect(user).to validate_uniqueness_of(:name)
end
specify 'for password' do
expect(user).to validate_presence_of(:password)
expect(user).to allow_value('abcd').for(:password)
expect(user).to_not allow_value('1234').for(:password)
end
end
end
I think that unless you have specific custom error messages for your errors that you want to test for (ie you've overridden the default Rails ones), then tests like expect(user.errors[:name]).to be_present can be removed (even if you have custom errors, I still think they're of dubious value since those messages will become locale-dependent if you internationalise your app, so I'd test for the display of some kind of error on the page in a feature spec instead).
I can write so many tests for the regex, but it will be hell for maintenance.
I don't think you can really get around this when testing validations for format, so I'd suggest just write some representative test cases and then add/remove those cases as you discover any issues you may have missed, for example:
# use a `let` or extract out into a test helper method
let(:valid_passwords) do
['abcd', 'ABCD', 'AbCd'] # etc etc
end
describe 'validations' do
specify 'for password' do
valid_passwords.each do |password|
expect(user).to allow_value(password).for(:password)
end
end
end
How much you think will be full test coverage in this example?
I've gotten 100% code coverage from reports like SimpleCov when writing unit specs as described above.
These 2 of them should be used, because:
it { expect(user).to validate_presence_of(:name).on(:create) }
=> You are expecting the validate_presence_of should be run on create, this should be the test case for model
it do
user = User.create(name: '')
expect(user.errors[:name]).to be_present
end
=> You are expecting a side effect when creating user with your input, so this should be the test case for controller
Why you shouldn't remove 1 of them:
Remove the 1st test case: what happens if you do database validation level instead, you expect an active record level validation
Remove the 2nd test case: what happens on controller actually creates a new User, how do you expect the error returning!
Related
In Rails models we usually have attributes and relations tests, like:
describe 'attributes' do
it { is_expected.to have_db_column(:identifier).of_type(:uuid) }
it { is_expected.to have_db_column(:content).of_type(:jsonb) }
it { is_expected.to have_db_column(:created_at).of_type(:datetime) }
end
describe 'relations' do
it { is_expected.to belong_to(:user).class_name('User') }
end
And using a TDD style it seems to be some useful tests, however I have been dwelling if these are really necessary tests, and I would like to know if there is some common knowledge about it, is it good practice to create these tests? or are we just testing rails?
Amongst the purposes of a unit test are...
Does it work?
Does it still work?
If it's a promise, if other things rely on it, you should test it to ensure you keep that promise. This is regression testing.
But don't test more than you promise. You'll be stuck with it, or your code will break when you make an internal change.
For example...
it { is_expected.to have_db_column(:identifier).of_type(:uuid) }
This promises that it has a column called identifier which is a UUID. Usually you don't promise all that detail; it is glass-box testing and it makes your test brittle.
Instead, promise as little as you can. Its ID is a UUID. This is black-box testing.
require "rspec/uuid"
describe '#id' do
subject { thing.id }
let(:thing) { create(:thing) }
it 'has a uuid ID' do
expect(thing.id).to be_a_uuid
end
end
It's possible there is an even higher level way to express this without holding yourself specifically to a UUID.
it { is_expected.to have_db_column(:content).of_type(:jsonb) }
Similarly, don't promise it has a jsonb column. That is blackbox testing. Promise that you can store complex data structures.
describe '#content' do
subject { create(:thing) }
it 'can round trip complex data' do
data = [1, { two: 3, four: [5] }]
thing.update!(content: data)
# Force it to re-load content from the database.
thing.reload
expect(thing.content).to eq data
end
end
it { is_expected.to belong_to(:user).class_name('User') }
Instead of promising what it belongs to, promise the relationship.
describe '#user' do
let(:thing) { create(:thing) }
let(:user) { create(:user) }
before {
user.things << thing
}
it 'belongs to a user' do
expect(thing.user).to eq user
expect(user.things).to contain(thing)
end
end
I have answered a nearly identical question here: https://stackoverflow.com/a/74195850/14837782
In summary: If it is end-developer code, I believe it should be tested. If it can be fat-fingered, I believe it should be tested. If you're going to remove it deliberately, I also believe you should have to remove a test deliberately as well. If it can fail, there should be a specific test for that failure mode.
This is not to be confused with testing the Rails framework. You obviously want to design your tests so that you're not testing Rails itself or Rails implementation, only your own code.
Attributes should be tested. Here is how I do it in minitest:
test/models/car_test.rb
class CarTest < ActiveSupport::TestCase
###################################################################
#
# Attributes
#
###################################################################
test 'describe some attr_reader fields' do
expected = [:year, :make, :model, :vin]
assert_has_attr_readers(Car, expected)
end
###############################################
test 'describe some attr_writer fields' do
expected = [:infotainment_fimrware_version]
assert_has_attr_writers(Car, expected)
end
###############################################
test 'describe some attr_accessor fields' do
expected = [:owner, :color, :mileage]
assert_has_attr_readers(Car, expected)
assert_has_attr_writers(Car, expected)
end
end
test/test_helpers/attributes_helper.rb
# frozen_string_literal: true
module AttributesHelper
###################################################################
#
# Assertions
#
###################################################################
#
# Performs an assertion that the given class contains reader/getter methods for the given attribute names.
# This helper checks for the existence of `attribute_name` methods on the class, and does not concern itself
# with how those methods are declared: directly defined, attr_reader, attr_accessor, etc.
#
def assert_has_attr_readers(klass, attribute_names)
# Get public and protected method names, passing `false` to exclude methods from super classes.
actual_method_names = klass.instance_methods(false).map(&:to_s)
attribute_names.each do |attribute|
message = "Expected class #{klass.name} to contain a reader for attribute #{attribute}"
assert_includes(actual_method_names, attribute.to_s, message)
end
end
#
# Performs an assertion that the given class contains writer/setter methods for the given attribute names.
# This helper checks for the existence of `attribute_name=` methods on the class, and does not concern itself
# with how those methods are declared: directly defined, attr_writer, attr_accessor, etc.
#
def assert_has_attr_writers(klass, attribute_names)
# Get public and protected method names, passing `false` to exclude methods from super classes.
actual_method_names = klass.instance_methods(false).map(&:to_s)
attribute_names.each do |attribute|
message = "Expected class #{klass.name} to contain a writer for attribute #{attribute}"
assert_includes(actual_method_names, "#{attribute}=", message)
end
end
#
# Performs an assertion that the given class implements attr_encrypted for the given attribute names.
# This helper is tied to the implementation details of the attr_encrypted gem. Changes to how attributes
# are encrypted will need to be accounted for here.
#
def assert_has_encrypted_attrs(klass, attribute_names)
message = "Expected class #{klass.name} to encrypt specific attributes"
actual_attributes = klass.encrypted_attributes.keys
assert_equal(attribute_names.map(&:to_s).sort, actual_attributes.map(&:to_s).sort, message)
end
end
Your example tests seem to be testing the existence of DB fields, not getter/setter model attributes. Database fields are impossible to fat-finger (they require a migration to modify) so if that's what you're talking about, I do not believe it makes sense to test them. (And I personally believe it is useful to test nearly everything.)
Although I guess in the case where the DB is accessible by other applications and could potentially be modified outside of a single application then it could make sense to test for the existence of those fields as well, as pointed out by Dave Newton in a comment below.
Ultimately it is up to you, and if your one application is the only one with access to the DB but you still want to test DB field existence and settings, maybe a 3rd option is some sort of migration test that you're looking for to make sure the migration is written properly. I've not written anything like that yet, but it might be feasible. I would hate to try to write one, and it does seem to go too far, but it's an idea...
Sorry for the vague title, there are a lot of moving parts to this problem so I think it will only be clear after seeing my code. I'm fairly sure I know what's going on here and am looking for feedback on how to do it differently:
I have a User model that sets a uuid attr via an ActiveRecord callback (this is actually in a "SetsUuid" concern, but the effect is this):
class User < ActiveRecord::Base
before_validation :set_uuid, on: :create
validates :uuid, presence: true, uniqueness: true
private
def set_uuid
self.uuid = SecureRandom.uuid
end
end
I am writing a functional rspec controller test for a "foo/add_user" endpoint. The controller code looks like this (there's some other stuff like error-handling and #foo and #params being set by filters, but you get the point. I know this is all working.)
class FoosController < ApplicationController
def add_user
#foo.users << User.find_by_uuid!(#params[:user_id])
render json: {
status: 'awesome controller great job'
}
end
end
I am writing a functional rspec controller test for the case "foo/add_user adds user to foo". My test looks roughly this (again, leaving stuff out here, but the point should be obvious, and I know it's all working as intended. Also, just to preempt the comments: no, I'm not actually 'hardcoding' the "user-uuid" string value in the test, this is just for the example)
RSpec.describe FoosController, type: :controller do
describe '#add_user' do
it_behaves_like 'has #foo' do
it_behaves_like 'has #params', {user_id: 'user-uuid'} do
context 'user with uuid exists' do
let(:user) { create(:user_with_uuid, uuid: params[:user_id]) } # params is set by the 'has #params' shared_context
it 'adds user with uuid to #foo' do
route.call() # route is defined by a previous let that I truncated from this example code
expect(foo.users).to include(user) # foo is set by the 'has #foo' shared_context
end
end
end
end
end
end
And here is my user factory (I've tried setting the uuid in several different ways, but my problem (that I go into below) is always the same. I think this way (with traits) is the most elegant, so that's what I'm putting here):
FactoryGirl.define do
factory :user do
email { |n| "user-#{n}#example.com" }
first_name 'john'
last_name 'naglick'
phone '718-555-1234'
trait :with_uuid do
after(:create) do |user, eval|
user.update!(uuid: eval.uuid)
end
end
factory :user_with_uuid, traits: [:with_uuid]
end
end
Finally, The problem:
This only works if I reference user.uuid before route.call() in the spec.
As in, if I simply add the line "user.uuid" before route.call(), everything works as intended.
If I don't have that line, the spec fails because the user's uuid doesn't actually get updated by the after(:create) callback in the trait in the factory, and thus the User.find_by_uuid! line in the controller does not find the user.
And just to preempt another comment: I'm NOT asking how to re-write this spec so that it works like I want. I already know a myriad of ways to do this (the easiest and most obvious being to manually update user.uuid in the spec itself and forget about setting the uuid in the factory altogether). The thing I'm asking here is why is factorygirl behaving like this?
I know it has something to do with lazy-attributes (obvious by the fact it magically works if I have a line evaluating user.uuid), but why? And, even better: is there some way I can do what I want here (setting the uuid in the factory) and have everything work like I intend? I think it's a rather elegant looking use of rspec/factorygirl, so I'd really like it to work like this.
Thanks for reading my long question! Very much appreciate any insight
Your issue has less to do with FactoryGirl and more to do with let being lazily evaluated.
From the docs:
Use let to define a memoized helper method. The value will be cached across
multiple calls in the same example but not across examples.
Note that let is lazy-evaluated: it is not evaluated until the first time
the method it defines is invoked. You can use let! to force the method's
invocation before each example.
Since your test doesn't invoke the user object until the expectation there is nothing created. To force rspec to load object, you can use let!.
Instead of using the before_validation callback you should be using after_initialize. That way the callback is fired even before .valid? is called in the model lifecycle.
class User < ActiveRecord::Base
before_initialization :set_uuid!, on: :create, if: :set_uuid?
validates :uuid, presence: true, uniqueness: true
private
def set_uuid!
# we should also check that the UUID
# does not actually previously exist in the DB
begin
self.uuid = SecureRandom.uuid
end while User.where(uuid: self.uuid).any?
end
def set_uuid?
self.uuid.nil?
end
end
Although the chance of generating the same hash twice with SecureRandom.uuid is extremely slim it is possible due to the pigeonhole principle. If you maxed out in the bad luck lottery this would simply generate a new UUID.
Since the callback fires before validation occurs the actual logic here should be completely self contained in the model. Therefore there is no need to setup a callback in FactoryGirl.
Instead you would setup your spec like so:
let!(:user) { create(:user) }
it 'adds user with uuid to #foo' do
post :add_user, user_id: user.uuid, { baz: 3 }
end
I'm currently developing a RoR 4 application.
In my application, I'm trying to implement Users tests similar to Mickael Hartl's tutorial. I'd like to understand a few things about the eMail uniqueness test :
1 - Uniqueness is tested when actually writing to the database, right ?
When writing my test the same way as in the tutorial, it is not efficient.
The #user = User.new(...) actually instantiates the user in memory only. The following test:
describe "when email address is already used" do
before do
user_with_same_email = #user.dup
user_with_same_email.save
end
it { should_not be_valid }
end
works only, in my case, when the #user = user.new(...) instruction is followed by a user.save instruction. Am I right ?
Additional note: when email uniqueness is implemented in the user model, it always fails !?!
Can you tell me what is wrong with this test ?
VALID_EMAIL_REGEX = /\A[\w+\-.]+#[a-z\d\-.]+\.[a-z]+\z/i
validates :email, presence: true, uniqueness: true, length: { maximum: 100 }, format: { with: VALID_EMAIL_REGEX }
2 - Does RSPEC really write to the TEST database ?
After running the above test (user.new(...) - user.save - user.dup - user.save), I check the 'modified date' of the TEST SQLite3 database file. It has not been touched when running the test. The spec_hepler.rb files starts with:
ENV["RAILS_ENV"] ||= 'test'
Did I miss something ?
Thank you for your help,
Best regards,
Fred
About your first point, the validation for uniqueness indeed occurs when you call save. However, you can also invoke all the validations by calling valid? - which is what be_valid does. So it doesn't actually need to save the #user model for this test to work.
About your second point, RSpec absolutely writes to the database. However, it runs each test within a transaction scope (by default). At the end of each test, instead of committing the transaction, it rolls it back. This leaves the database in the same state it was before the test started, effectively allowing each test to be isolated from the others while making use of the database just as your code would do in production (almost). Regarding your point about the file modification time, SQLite doesn't necessarily write out the data to the file as you make calls until they are actually committed.
I'm writing integration tests using Rspec and Capybara. I've noticed that quite often I have to execute the same bits of code when it comes to testing the creation of activerecord options.
For instance:
it "should create a new instance" do
# I create an instance here
end
it "should do something based on a new instance" do
# I create an instance here
# I click into the record and add a sub record, or something else
end
The problem seems to be that ActiveRecord objects aren't persisted across tests, however Capybara by default maintains the same session in a spec (weirdness).
I could mock these records, but since this is an integration test and some of these records are pretty complicated (they have image attachments and whatnot) it's much simpler to use Capybara and fill out the user-facing forms.
I've tried defining a function that creates a new record, but that doesn't feel right for some reason. What's the best practice for this?
There are a couple different ways to go here. First of all, in both cases, you can group your example blocks under either a describe or context block, like this:
describe "your instance" do
it "..." do
# do stuff here
end
it "..." do
# do other stuff here
end
end
Then, within the describe or context block, you can set up state that can be used in all the examples, like this:
describe "your instance" do
# run before each example block under the describe block
before(:each) do
# I create an instance here
end
it "creates a new instance" do
# do stuff here
end
it "do something based on a new instance" do
# do other stuff here
end
end
As an alternative to the before(:each) block, you can also use let helper, which I find a little more readable. You can see more about it here.
The very best practice for your requirements is to use Factory Girl for creating records from a blueprint which define common attributes and database_cleaner to clean database across different tests/specs.
And never keep state (such as created records) across different specs, it will lead to dependent specs. You could spot this kind of dependencies using the --order rand option of rspec. If your specs fails randomly you have this kind of issue.
Given the title (...reusing code in Rspec) I suggest the reading of RSpec custom matchers in the "Ruby on Rails Tutorial".
Michael Hartl suggests two solutions to duplication in specs:
Define helper methods for common operations (e.g. log in a user)
Define custom matchers
Use these stuff help decoupling the tests from the implementation.
In addition to these I suggest (as Fabio said) to use FactoryGirl.
You could check my sample rails project. You could find there: https://github.com/lucassus/locomotive
how to use factory_girl
some examples of custom matchers and macros (in spec/support)
how to use shared_examples
and finally how to use very nice shoulda-macros
I would use a combination of factory_girl and Rspec's let method:
describe User do
let(:user) { create :user } # 'create' is a factory_girl method, that will save a new user in the test database
it "should be able to run" do
user.run.should be_true
end
it "should not be able to walk" do
user.walk.should be_false
end
end
# spec/factories/users.rb
FactoryGirl.define do
factory :user do
email { Faker::Internet.email }
username { Faker::Internet.user_name }
end
end
This allows you to do great stuff like this:
describe User do
let(:user) { create :user, attributes }
let(:attributes) { Hash.new }
it "should be able to run" do
user.run.should be_true
end
it "should not be able to walk" do
user.walk.should be_false
end
context "when user is admin" do
let(:attributes) { { admin: true } }
it "should be able to walk" do
user.walk.should be_true
end
end
end
I'm using FactoryGirl for my fixtures and am finding that it's not really producing useful validation errors.
I always get the message for activerecord.errors.models.messages.record_invalid.
Not sure what further details are needed to help diagnose this. This makes it an excruciatingly slow process to track each error down.
Example factory:
Factory.define :partner do |partner|
partner.sequence(:username){ |n| "amcconnon#{n}" }
partner.first_name "Bobby Joe"
partner.last_name "Smiley"
partner.sequence(:email){ |n| "bob{n}#partners.com" }
partner.phone_number "5557 5554"
partner.country_id 75
partner.password "password"
partner.password_confirmation "password"
end
Then Factory(:partner) => "ActiveRecord::RecordInvalid Exception: Looks like something went wrong with these changes"
The actual problem is of course the email sequence doesn't use n properly and there is a unique validation on email. But that's for illustrative purposes.
rails => 3.2.2
factory_girl 2.6.1
Any other deets needed to help diagnose this?
(Note: edited this just to add an easier to read factory)
EDIT:
As per bijan's comment: "What exactly am I trying to do."
Trying to run "rspec spec". I would like when I use a factory like Factory(:partner) in this case for the error message when that fails to contain the same error I would get from Partner.new({blah...}).valid? then looked at the validation failures.
I'm not sure if this is exactly the same problem you were having, but it's a problem I was having with validation error messages not being very useful and I thought it could be useful to others searching for this problem. Below is what I came up with. This could be modified to give different info too.
include FactoryGirl::Syntax::Methods
# Right after you include Factory Girl's syntax methods, do this:
def create_with_info(*args, &block)
create_without_info(*args, &block)
rescue => e
raise unless e.is_a? ActiveRecord::RecordInvalid
raise $!, "#{e.message} (Class #{e.record.class.name})", $!.backtrace
end
alias_method_chain :create, :info
Then, when you use create :model_name, it will always include the model name in the error message. We had some rather deep dependencies (yes, another problem), and had a validation error like, "name is invalid"... and name was an attribute on a few different models. With this method added, it saves significant debugging time.
I think the key point here is that when you're setting up a test you need to make sure that the code you're testing fails at the point that you set the expectation (i.e. when you say "should" in Rspec). The specific problem with testing validations is of course that the validation fails as soon as you try to save the ActiveRecord object; thus the test setup mustn't invoke save.
Specific suggesions:
1) IMO Factory definitions should contain the minimum information required to create a valid object and should change as little as possible. When you want to test validations you can override the specific attribute being tested when you instantiate the new test object.
2) When testing validations, use Factory.build. Factory.build creates the ActiveRecord instance with the specified attributes but doesn't try to save it; this means you get to hold off triggering the validation until you set the expectation in the test.
How about something like this?
it "should fail to validate a crap password" do
partner_with_crap_password = Factory.build(:partner, :password => "crap password")
partner_with_crap_password.should_not be_valid
end