Rails unique column has duplicates - ruby-on-rails

This is the stripped down version of my model .
model Paper
PAPER_STARTING_NUMBER = 1
validate_uniqueness_of :number, :allow_blank => true
before_create :alocate_paper_number
def alocate_paper_number
return true if self.number.present?
p_number = Paper.maximum('number') || Paper::PAPER_STARTING_NUMBER
self.number = p_number >= Paper::PAPER_STARTING_NUMBER ? p_number+1 : Paper::PAPER_STARTING_NUMBER
return true
end
end
the problem is I have duplicates in the number column .
Any ideas why and how I can fix this without changing the callback .
I know I could add a uniqueness validation on the database or make a sequence on that column , any other ideas ?

First you have to understand the order of callbacks :
(-) save
(-) valid
(1) before_validation
(-) validate
(2) after_validation
(3) before_save
(4) before_create
(-) create
(5) after_create
(6) after_save
(7) after_commit
So as you can see , it validates the uniquity of your number attribute, and then before_create can at its own disposal go against what your validation wants to accomplish.
In regards to a more cleaner architecture, I would put both of these ideas together in your custom model, as it doesn't seem that the number can be choosen by the User. It's just an incrementer, right?
def alocate_paper_number
p_number = Paper.maximum('number') || Paper::PAPER_STARTING_NUMBER
self.number = p_number + 1
end
That snippet alone, would prevent duplicates, as it always increments upwards ( unless, there's the possibility of the number going the other way that I'm not aware of ), and also there's no reason to return all those trues. Its true enough!

It is in de docs. validate_uniqueness_of TRIES to make it unique. But if two processes add one record at the same time, they both can contain the same number.
If you want to guarantee uniqueness, let the database do it. But because that is different for each DB, Rails does not support it by design.
It's explained here: http://guides.rubyonrails.org/active_record_validations_callbacks.html#uniqueness
With the solution: "To avoid that, you must create a unique index in your database."

How I fixed it ( bare in mind that I couldn't return a validation error )
I've added a uniquness index on the number column ( as mu and Hugo suggested )
and because I couldn't return a validation error in the controller
class PaperController < ApplicationController
def create
begin
#paper.save
rescue ActiveRecord::RecordNotUnique
#paper.number = nil
create
end
end
end

Related

Rails validation: Only N records can be true

How would you write a validation that checks if a certain amount of records already have an attribute that is true?
class BlogPost
scope :onStartpage, -> { where(onStartpage: true) }
validate :number_on_startpage
def number_on_startpage
return unless onStartpage?
errors.add(:foobar, 'foo') if Blog::Post.where(onStartpage: true).count > 3
end
end
Like in the example above - I want to make sure that there are no more than three BlogPosts with the attribute onStartpage set to true.
But it's not working because when I want to save the record the count is still 2. And all later updates get rejected because the count is too high.
The problem is that your validation will be checked for previous blog posts, even if you just wanted to change the title for instance.
You could check your validation only if the attribute onStartPage has changed.
Also, you could change the condition to >=, as if you already have 3 articles with that attribute, 3 is not strictly superior to 3...so it will only return false after you have 4 articles with onStartpage set to true.
class BlogPost
scope :onStartpage, -> { where(onStartpage: true) }
validate :number_on_startpage
def number_on_startpage
return unless onStartpage_changed?
errors.add(:foobar, 'foo') if Blog::Post.where(onStartpage: true).count >= 3
end
end

ActiveRecord validations

I'm wondering if its possible to have active record hold off on validating records for a short time? For example, in my webapp I have a table called Sizes that contain an attribute called sort_order, users are able to reorganize the order in which sizes are displayed by modifying the sort_order. When they do that I have this function which will go through the list of sizes and recalculate the sort order, updating the column accordingly.
def update_size_order
#size_ids = params[:categories]
n = 0
ActiveRecord::Base.transaction do
#size_ids.each do |temp|
temp = temp.split('_')
id = temp[1]
size = Size.find(id)
size.sort_order = n
n += 1
size.save!
end
end
render :json => {}
end
The problem arises because in my model I have
validates :sort_order, presence: true, uniqueness: true
but when the model tries to save size I get an error because size isn't necessarily unique. Is there a way I can have ActiveRecord not execute validations until this function has finished? My googling skills have met their match so it seems, but I feel like there is a simple workaround for this problem. Thank you for your help!
Use size.update_attribute :sort_order, n instead of save!. It will simply update the database without running any form of validation. Since you're running this in a transaction, you should be OK, but generally you should avoid update_attribute for the very reason that it bypasses validation and callbacks.
Also, each_with_index is your friend:
def update_size_order
#size_ids = params[:categories]
ActiveRecord::Base.transaction do
#size_ids.each_with_index do |temp,n|
temp = temp.split('_')
id = temp[1]
Size.find(id).update_attribute :sort_order, n
end
end
render :json => {}
end
Since you're only interested in updating the attribute, you can also skip the entire process of instantiating an AR object, and drop your Size.find(id) call in favor of using update_all. This will silently ignore any IDs that aren't found, but it will drastically improve the performance of your loop as you're halving the number of queries being run. Instead of a select query, followed by fully instantiating an AR model, and an update for every single record, it will generate only the update query:
#size_ids.each_with_index do |temp,n|
temp = temp.split('_')
id = temp[1]
Size.where(id: tmp[1]).update_all(sort_order: n)
end
Let's say you put this :
validates: :sort_order,
presence: true,
uniqueness: true,
on: :create
Is that what you wanted ?
In this way, your validation execute only on creating new object .

Rails: How can I validate my records so that no two records are the same?

I've been trying to get the validates_uniqueness_of to work for my database. I pull in records from a CSV file, and I want to make sure that I record them all but when I check the next time I don't want to save them all again if they're just duplicates.
Example Object
PlayerStats {session_date, uniform_number, last_name, first_name, throws, throws_caught, throws_dropped, intercepted_throws, defended_throws }
Example Records
2013-01-01, 11A, Jacobsen, Mike, 11, 4, 7, 0 0
2013-01-01, 11A, Jacobsen, Mike, 0, 0, 0, 2, 1
I want to keep both of these records, but when I try to validate like so...
validates_uniqueness_of :uniform_number, :scope => [:session_date, :last_name]
this will only keep for instance the first record and consider the second a duplicate.
I would like to have it where when the second record goes through a save attempt it will save the second record also.
The problem is that you are only validating uniqueness of three fields and not all fields. You should add all your fields to :scope. But that will not be very good performance wise if you have a table with too many rows. I would suggest you generate a token out of all the fields you want to validate uniqueness of and add uniqueness validation on token. Going this route you have to add one more column to your table to store the calculated token. Don't forget to add unique index on the column to get the optimum performace. After that following should do the trick:
before_validate :generate_unique_token
# assuming you named your slug column `unique_token`
validates :unique_token, uniqueness: true
private
# add all fields you want to validate uniqueness on
UNIQUE_FIELDS = [:uniform_number, :session_date, :more_fields]
def generate_unique_token
return if self.unique_token.present?
token_string = ''
# additional comma is to ensure that [1, 10] and [11, 0] don't get treated as same input
UNIQUE_FIELDS.each {|field| token_string << self.send(field).to_s << ','}
self.unique_token = token_string
end
This string can get big, but you will not get any false collisions. If you want to control the size of the generated token you can do it like following:
def generate_unique_token
token_string = ''
UNIQUE_FIELDS.each {|field| token_string << self.send(field).to_s << ','}
self.unique_token = compress_token(token_string)
end
def compress_token(token_string)
# you can further compress the token by encoding it in base 62/64
::Digest::SHA256.hexdigest(token_string)
end
But beware, the later solution can have rare false collisions.

Rails Specifying the order of validations

I have a validator class that i am writing that has three validations, that are run when calling MyVariableName.valid?
validates_length_of :id_number, :is => 13, :message => "A SA ID has to be 13 digits long"
validates_format_of :id_number, :with => /^[0-9]+$/, :message => "A SA ID cannot have any symbols or letters"
validate :sa_id_validator
The third one is a custom validator. The thing is that my validator sa_id_validator requires that the data that is passed in is a 13 digit number, or I will get errors. How can I make sure that the validate :sa_id_validator is only considered after the first two have run?
Sorry if this is a very simple question I have tried figuring this out all of yesterday afternoon.
Note: this validator has to run over a couple thousand entries and is also run on a spreadsheet upload so I need it to be fast..
I saw a way of doing this but it potentially runs the validations twice, which in my case would be bad.
EDIT:
my custom validator looks like this
def sa_id_validator
#note this is specific to South African id's
id_makeup = /(\d{6})(\d{4})(\d{1})(\d{1})(\d{1})/.match(#id_number)
birthdate = /(\d{2})(\d{2})(\d{2})/.match(id_makeup[1])
citizenship = id_makeup[3]
variable = id_makeup[4]
validator_key = id_makeup[5]
birthdate_validator(birthdate) && citizenship_validator(citizenship) && variable_validator(variable) && id_algorithm(id_makeup[0], validator_key)
end
private
def birthdate_validator(birthdate)
Date.valid_date?(birthdate[1].to_i,birthdate[2].to_i,birthdate[3].to_i)
end
def citizenship_validator(citizenship)
/[0]|[1]/.match(citizenship)
end
def variable_validator(variable)
/[8]|[9]/.match(variable)
end
def id_algorithm(id_num, validator_key)
odd_numbers = digits_at_odd_positions
even_numbers = digits_at_even_positions
# step1: the sum off all the digits in odd positions excluding the last digit.
odd_numbers.pop
a = odd_numbers.inject {|sum, x| sum + x}
# step2: concate all the digits in the even positions.
b = even_numbers.join.to_i
# step3: multiply step2 by 2 then add all the numbers in the result together
b_multiplied = (b*2)
b_multiplied_array = b_multiplied.to_s.split('')
int_array = b_multiplied_array.collect{|i| i.to_i}
c = int_array.inject {|sum, x| sum + x}
# step4: add the result from step 1 and 3 together
d = a + c
# step5: the last digit of the id must equal the result of step 4 mod 10, subtracted from 10
return false unless
validator_key == 10 - (d % 10)
end
def digits_at_odd_positions
id_num_as_array.values_at(*id_num_as_array.each_index.select(&:even?))
end
def digits_at_even_positions
id_num_as_array.values_at(*id_num_as_array.each_index.select(&:odd?))
end
def id_num_as_array
id_number.split('').map(&:to_i)
end
end
if i add the :calculations_ok => true attribute to my validation, and then pass in a 12 digit number instead i get this error:
i.valid?
NoMethodError: undefined method `[]' for nil:NilClass
from /home/ruberto/work/toolkit_3/toolkit/lib/id_validator.rb:17:in `sa_id_validator'
so you can see its getting to the custom validation even though it should have failed the validates_length_of :id_number??
I am not quite sure but i have read at some blog that Rails always runs all validations even if the first one is invalid.
What you can do is to make your custom method in such a way that it would become flexible or bouncy in such a way that i would handle all the cases.
This answer would definitely help you.
Hope it would answer your question

Rails / ActiveRecord: field normalization

I'm trying to remove the commas from a field in a model. I want the user to type a number, i.e. 10,000 and that number should be stored in the database as 10000. I was hoping that I could do some model-side normalization to remove the comma. I don't want to depend on the view or controller to properly format my data.
I tried:
before_validation :normalize
def normalize
self['thenumber'] = self['thenumber'].to_s.gsub(',','')
end
no worky.
http://github.com/mdeering/attribute_normalizer looks like a promising solution to this common problem. Here are a few examples from the home page:
# By default it will strip leading and trailing whitespace
# and set to nil if blank.
normalize_attributes :author, :publisher
# Using one of our predefined normalizers.
normalize_attribute :price, :with => :currency
# You can also define your normalization block inline.
normalize_attribute :title do |value|
value.is_a?(String) ? value.titleize.strip : value
end
So in your case you might do something like this:
normalize_attribute :title do |value|
value.to_s.gsub(',', '')
end
I think you're doing it right. This test passes:
test "should remove commas from thenumber" do
f = Foo.new(:thenumber => "10,000")
f.save
f = Foo.find(f.id)
assert f.thenumber == "10000"
end
And I used your code.
class Foo < ActiveRecord::Base
before_validation :normalize
def normalize
self['thenumber'] = self['thenumber'].to_s.gsub(',','')
end
end
Now, my schema is set up for thenumber to be a string though, not an integer.
Started
.
Finished in 0.049666 seconds.
1 tests, 1 assertions, 0 failures, 0 errors
If you wanted to store this in the db as an integer, then you definitely need to override the setter:
def thenumber=(value)
self['thenumber'] = value.to_s.gsub(',','').to_i
end
If you do it your way, with an integer column, it gets truncated by AR....
>> f.thenumber = "10,000"
=> "10,000"
>> f.thenumber
=> 10
That's a little-known thing with Ruby and integers... it auto-casts by truncating anything that's no longer an integer.
irb(main):004:0> i = "155-brian-hogan".to_i
=> 155
Can be cool for things like
/users/155-brian-hogan
#user = User.find_by_id(params[:id])
But not so cool for what you're doing.
So either change the col to a string and use the filter, or change the setter :)
Good luck!
The problem with doing it that way is that for a while, the non-normalized stuff will exist in the object; if you have code that works on the attributes before stuff gets normalised, then that will be a problem.
You could define a setter:
def thenumber=(value)
# normalise stuff here, call write_attribute
end
Unfortunately I think a lot of the Rails form stuff writes the attributes directly, which is one of the reasons I don't tend to use it.
Or you could normalise the params in the controller before you pass them through.
Does ruby let you interchange between a . and [''] ?
I don't know, I'll try later, but I think you are supposed to use .
self.thenumber = self.thenumber.to_s.gsub(',','')
You should return true from your before_validation method, otherwise if the expression being assigned to self['thenumber'] ends up being nil or false, the data will not be saved, per the Rails documention:
If a before_* callback returns false,
all the later callbacks and the
associated action are cancelled.
Ostensibly, you are trying to normalize here then check the result of the normalization with your Rails validations, which will decide if nil/false/blank are okay or not.
before_validation :normalize
def normalize
self['thenumber'] = self['thenumber'].to_s.gsub(',','')
return true
end

Resources