In a Rails app, the column names, types, and default values are inferred directly from the database. Is there any way of inferring validations from database constraints on initialization or while attempting to save?
This would allow more DRYness, and ensure that all data could be validated softly before hitting the DB and getting an exception, because the validations would cover all the database constraints. The database's constraints are the authoritative source of information on data invalidity when they are used.
Alternatively, is it possible to make ActiveRecord rescue from hitting a database constraint, and act as though a weak validation has failed? That would mean that database constraints could be manipulated externally without restarting or editing the Rails app, the performance would be improved because uniqueness validations would not require a separate query, and also that uniqueness validations would be immune to race conditions.
You can use the Enforce Schema Rules gem:
https://github.com/twinge/enforce_schema_rules
It validates your model against database rules you’ve already created in your schema.
Example:
class Person < ActiveRecord::Base
enforce_schema_rules :except => :dhh
end
Related
I have designed a very simple web application which associates authors, books and ratings.
In each of the respective models
Author
has_many :books
Book
belongs_to :author
has_many :reviews
Review
belongs_to :book
Model attributes
Author : title, fname, lname, DOB
Book : ISBN, title, publish_date, pages
Review : rating(1-5), description
I am wondering if I completely validate all of these attributes to my liking in the models, 1 attribute for example
validates :ISBN, :only_integer => true, length: { is: 13 }
do I need to worry about validations for data elsewhere?
I know that validations for the model run on the server side so there may need to be some validation on the client side (in JS). I am trying to ensure that there are no flaws when it comes to asserting data correctness.
As is so often the case: it depends.
In a simple Rails application, all models will be updated through a view request to a controller which in turn fills in the params into the models, then tries to save the model and any validation errors that occur are rendered back to the server.
In that scenario, all your code will have to do is to react to failed calls to #save but you can sleep soundly knowing that everything in your database is how it is supposed to be.
In more complex applications, putting all the validation logic into your model might not work as well anymore: every call to #save will have to run through all the validation logic, slowing things down, and different parts of your application might have different requirements to the input parameters.
In that scenario there are many ways to go about it. Form objects with validations specific to the forms they represent are a very common solution. These form models then distribute their input among one or more underlying ActiveRecord models.
But the Rails way is to take these one step at a time and avoid premature optimization. For the foreseeable future, putting your validation into your model will be enough to guarantee consistency.
do I need to worry about validations for data elsewhere?
Yes you do.
Application level validations are still prone to race conditions.
For things that should be unique like for example ISBN numbers database constraints are vital if uniqueness is to be guarenteed. Other areas where this can cause issues are when you have a limit on the count of an association.
While validations prevent most errors they are not a replacement for database constraints to ensure the correctness of data. Both are needed if correctness is important.
I am not sure if I understand totally active record validation role.
Of course, if a user inputs data (like an email or a country), I can and should validate its existence, its uniqueness or its inclusion in a list of countries
But for example, if I have methods in the backend that change an attribute page_clicked or click_date or even the column update_at, that I "control" i.e 'is not generated by a user's input', should I use active record validations ?
I'm asking this because on a very 'hot database' (need speed for millions of frequent updates), I wonder if checking on each update that updated_at is a datetime, and that if a clicked column is true/false and nothing esle is really necessary as the user is not the one inputting/controlling these data but I am through Rails custom methods I wrote
Thanks
I don't think there is a general satisfying answer to your question. It's up to you to enforce validation or not.
Remember that you don't have to use ActiveRecord for validation, you can also use your DBMS to ensure that:
a value will never be NULL (one of the most annoying errors)
a value has the correct TYPE
a FOREIGN KEY always points to an existing row in another table
and depending on your DBMS, a lot more is possible
If you need high INSERT speed and want to go with raw SQL INSERTS, putting some validation in your database can prevent nasty application errors later.
Validations should guard your database and its job should be to stop saving the records that are considered invalid by your application.
There is no hard rule on what is valid record you have to decide it your self by adding the validations. If the record wont pass the validation step it is simply not going to be saved to the database.
From Active Record Callbacks:
3.1 Creating an Object
before_validation
after_validation
before_save
around_save
before_create
around_create
after_create
after_save
after_commit/after_rollback
3.2 Updating an Object
before_validation
after_validation
before_save
around_save
before_update
around_update
after_update
after_save
after_commit/after_rollback
You can see that validation hooks run at the beginning of the object life cycle.
So in your case instead of asking your self a question:
Should I use active record validations if the record is not generated by a user's input.
You should ask your self:
Is this record invalid without page_clicked or click_date(aka them being nil)
UPDATE
If you consider record to be invalid but worrying about speed problems with running validations I would do the validations to make sure that all the records in the database are valid and try to find the way to optimise the speed somewhere else. Plus not 100% sure but time spend on saving invalid records and filtering them later on will be probably much longer then validating in the first place.
When performance is really a priority and that I am sure that we developers / the server are the only ones who can manipulate specific attributes of a Model, I will
Make sure that I create a separate method / wrapper method for this specific action.
In this specific method, I call .save (validate: false) instead of the usual .save
I still write validations for the said attributes for developers' reference to prevent future development errors, and in case a new developer comes in and accidentally save an invalid record, precisely just because there's no validation to safeguard it.
Or, I will use .update_column instead of .save (validate: false) to perform a direct DB call, skipping Model validations and callbacks (If you also do not want callbacks to be called).
Note that .update_column is different from .update.
This question already has answers here:
Ruby on Rails: Is it better to validate in the model or the database?
(6 answers)
Closed 7 years ago.
By SQL level validations, I meant the validations that we put in migrations
like this:
add_column :leads, :count, null: false
By model-level validations, I meant the validations we put in models like this:
validates_presence_of :count
How are these two different? How is having both helpful? Is it enough that I only put model-level validations?
SQL validations are performed on database level and not having them implemented in the model might lead into some exceptions raised from db layer (basically from mysql/pg/put-your-adapter gem).
Model level validations are performed before the query is run against database.
The best option is to have both. Even if something goes wrong with your model validation - it will be catched on database level.
Good example here is unique constraint. If you have a huge load on the database you can encounter race condition problem and model validation is not enough here. Thanks to having unique constraint you can be sure that your data integrity is protected.
Model level validation helps you by providing user error messages when there is a validation error. and other helpers like valid? methods to check the object is ready to insert.
Mean while db level validations make sure your database sanity at DB layer.
That said you can always have more complex business logic based validations in model level may be by involving and transforming data in few other tables.
Using uniqueness validations in Rails is not safe when there are multiple processes unless the constraint is also enforced on the database (in my case a PostgreSQL database, so see this blog post).
In my case, the uniqueness validation is conditional: it should only be enforced if another attribute in the model becomes true. So I have
class Model < ActiveRecord::Base
validates_uniqueness_of :text, if: :is_published?
def is_published?
self.is_published
end
end
So the model has two attributes: is_published (a boolean) and text (a text attribute). text should be unique across all models of type Model if is_published is true.
Using a unique index (as suggested in the linked blog post) is too constraining because it would enforce the constraint regardless of the value of is_published.
Is anyone aware of a "conditional" index on a PostgreSQL database? Or another way to fix this?
Yes, use a partial UNIQUE index.
CREATE UNIQUE INDEX tbl_txt_is_published_idx ON tbl (text) WHERE is_published;
Example:
How to add a conditional unique index on PostgreSQL
I think that - given speed is not your main concern - you can achieve proper uniqueness validation without creating additional db indexes. Goal can be achieved at the application level. This is especially valuable if you want conditional uniqueness, as some dbs (e.g. versions of MySQL < 8) does not support partial indexes (or so called filtered indexes).
My solution is based on following assumption:
uniqueness check (validator) is run by Rails in the same transaction as save/destroy action that relies on it.
This assumption seems to be true: https://api.rubyonrails.org/classes/ActiveRecord/Transactions/ClassMethods.html
Both #save and #destroy come wrapped in a transaction that ensures that whatever you do in validations or callbacks will happen under its protected cover.
transaction calls can be nested. By default, this makes all database statements in the nested transaction block become part of the parent transaction.
Having that you can use pessimistic locking (https://api.rubyonrails.org/classes/ActiveRecord/Locking/Pessimistic.html) to exclusively lock records that you want to evaluate for uniqueness in validator. That will prevent another, simultaneously running validator - and actually anything that happens after it - from executing until lock is released at the end of transaction. That ensures atomicity of validate-save pair and proper uniqueness enforcement.
In your code it would look like that:
class Model < ActiveRecord::Base
validates :text, uniqueness: {
conditions: ->{ lock.where(is_published: true) }
}
end
The only downside I can see is having db records locked for the whole validate-save process. That won't work well under heavy load, but then many applications don't work under such conditions anyway.
I have a model, Clients and a corresponding database with lastname and firstname columns. Originally there were no constraints on the uniqueness of [lastname, firstname], and the database currently contains duplicates. I would like to clean up the database and impose constraints on the model, such as: validates_uniqueness_of :lastname, scope: :firstname.
Idea that comes to my mind is to back up the data in some fashion, impose constraints on an empty model database and then pull the data back in with duplicates that I can now process separately rescueing from exception.
I feel, however, that I am doing something off-the-wall here.
Is there a better, "rails way" to do this?
The only really pure-Rails way to discover the problems is to run through every model and ensure that it is still valid. For instance, roughly:
Client.all.each do |client|
unless (client.valid?)
puts "Client #{client.id} invalid: #{client.errors.full_messages}"
end
end
Loading all records might be a bad idea if it requires too much memory. ActiveRecord 3.0 is supposed to be smarter about this, loading it in chunks, but I can't prove this is the case at the moment.
As for what you do with the duplicate data:
Always back up your tables before starting using the appropriate database snapshotting tool.
Always test your modifications on a copy of the data before running this on a production database.
Always document your changes by writing a Rails migration that performs the operation in a reliable and predicable manner. Test it repeatedly before deploying.
I would presume that your production database is subjected to regular snapshots as it is, in which case you can grab test data from there. If this is not the case, your first priority should be to ensure it is.