I'm having a bizzarre glitch, where Rails does not validate the uniqueness of an attribute on a model, despite the attribute being saved perfectly, and despite the validation being written correctly.
I added a validation to ensure the uniqueness of a value on one of my Rails models, Spark, with this code:
validates :content_hash, :presence => true, :uniqueness => true
The content_hash is an attribute created from the model's other attributes in a method called using a before_validation callback. Using the Rails console, I've confirmed that this hash is actually being created before the validation, so that is not the issue.
When I call in the Rails console spark.valid? on a spark for which I know a collision exists on its content_hash, the console tells me that it has run this query:
Spark Exists (0.2ms) SELECT 1 AS one FROM "sparks" WHERE "sparks"."content_hash" = '443524b1c8e14d627a3fadfbdca50118c6dd7a7f' LIMIT 1
And the method returns that the object is valid. It seems that the validator is working perfectly fine, and is running the correct query to check the uniqueness of the content_hash, the problem is instead on the database end (I'm using sqlite3). I know this because I decided to check on the database myself to see if a collision really exists using this query:
SELECT "sparks".* FROM "sparks" WHERE "sparks"."content_hash" = '443524b1c8e14d627a3fadfbdca50118c6dd7a7f'
Bizarrely, this query returns nothing from the database, despite the fact that I can see with my own eyes that other records with this content_hash exist on the table.
For some reason, this is an issue that exists exclusively with the content_hash attribute of the sparks table, because when I run similar queries for the other attributes of the table, the output is correct.
The content_hash column is no different from the others which work as expected, as seen in this relevant part of my schema.rb file:
create_table "sparks", :force => true do |t|
t.string "spark_type"
t.string "content_type"
t.text "content"
t.text "content_hash"
t.datetime "created_at", :null => false
t.datetime "updated_at", :null => false
end
Any help on this problem would be much appreciated; I'm about ready to tear my hair out over this thing.
Okay, I managed to fix the problem. I think it was an sqlite3 issue, because everything worked perfectly once I changed the type of content_hash from a text column to a string column. Weird.
Related
I'm struggling to wrap my mind around an ActiveRecord query.
I'm trying to search my database for GolfRetailer objects with ID's 1..100, that have something (not nil) in their :website field, and that don't have true in their duplicate_domain field.
Here's the query I expected to work:
GolfRetailer.where.not(website: nil, duplicate_domain: true).where(id: 1..100)
I also tried this variant of essentially the same query: GolfRetailer.where.not(website: nil).where(id: 1..100, duplicate_domain: !true)
But both return an empty array, despite there definitely being records that meet those requirements.
When I run GolfRetailer.where.not(website: nil).where(id: 1..100) I get an array, and when I run GolfRetailer.where.not(website: nil, duplicate_domain: nil).where(id: 1..100) I also get an array, but with all records that do have the true duplicate_domain flag, which isn't what I'm looking for.
I'd rather not search for records that have duplicate_domain: nil as that's not always correct (I may not have processed their domain yet).
For clarity, here is the Schema for the Model.
create_table "golf_retailers", force: :cascade do |t|
t.string "name"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.string "place_id"
t.string "website"
t.string "formatted_address"
t.string "google_places_name"
t.string "email"
t.boolean "duplicate_domain"
t.index ["duplicate_domain"], name: "index_golf_retailers_on_duplicate_domain"
end
What am I missing to make this query work?
This is happening because in SQL when you do a != TRUE, any NULL values will not be included in the result. This is because the NULL value represents an unknown value, so the DB does not know how to do any comparison operations on an unknown value and therefore they're excluded.
One way to get around this is to use IS DISTINCT FROM:
GolfRetailer
.where(id: 1..100)
.where.not(website: nil)
.where("duplicate_domain IS DISTINCT FROM ?", true)
As others have mentioned, you should also ask yourself if it's really the case that it's ever unknown to you if a GolfRetailer has a duplicate_domain.
If, all GolfRetailers with a duplicate_domain of NULL actually mean they don't have one (false) than you should consider preventing a NULL value for that column entirely.
You can do this by adding a NOT NULL constraint on the column with a change_column database migration.
In order to add the NOT NULL constraint you will first need to make sure all of the data in the column has non-null values.
def change
GolfRetailer.in_batches.update_all(duplicate_domain: false)
change_column_null :golf_retailers, :duplicate_domain
end
If your application is under load, you should also be careful about the potential performance any migration like this might have - notably if you add a NOT NULL constraint with a default value.
Consider using something like the Strong Migrations gem to help find DB migrations that might cause downtime before production.
I'm sure this is something simple I'm overlooking but since I've been dealing with this strange issue for a few days now I'm asking for help.
Here is my apps setup and the issue:
A Rails 3.2.13 app with the pg gem and the following db scheme:
create_table "clients", :force => true do |t|
t.string "uuid"
t.string "private_key"
t.datetime "created_at", :null => false
t.datetime "updated_at", :null => false
end
add_index "clients", ["private_key"], :name => "index_clients_on_private_key"
add_index "clients", ["uuid"], :name => "index_clients_on_uuid"
I currently have two rows in my "clients" database. They pictured below:
The issue
When I perform a simple "SELECT * FROM clients;" using the RubyMine database console I get the rows pictured in the screen shot above. When I perform a "SELECT * FROM clients WHERE id = 11;" I get that row, all is well there. However, the issue is that when I try to perform a "SELECT * FROM clients WHERE private_key = "MkRBNUZBQzUtMzVDRi00NzQ3LThFNjEtNjI4OThERUQzQkRF";" it fails and shows the following in the console output...
sql> SELECT * FROM clients WHERE private_key = "MkRBNUZBQzUtMzVDRi00NzQ3LThFNjEtNjI4OThERUQzQkRF"
[2013-04-17 20:09:51] [42703] ERROR: column "MkRBNUZBQzUtMzVDRi00NzQ3LThFNjEtNjI4OThERUQzQkRF" does not exist
Position: 43
I've also tried pulling this row out of the db by utilizing the active record helper methods...
Client.exists?(:private_key => token)
Client.find_by_private_key(token)
However, nothing works, it never finds the record that does exist in the db. Does anyone have any idea what is going on here? All I need to be able to do is utilize the active record helpers, specifically the "Client.exists?(:private_key => token)" to be able to check and see if the supplied token exists in the db.
Thanks for your help!
Update 4/18/13 # 9:30am
I just tried Client.find(12) just to see if it would find the record by id, and this works. This doesn't help me understand why I still can't use Client.exists?(:private_key => token) or Client.find_by_private_key(token).
Update 4/19/13 # 9:15am
The answer to this problem was that the code generating my private_key value was adding a return/white space to the end of the value. So when trying to query the db for row based on a private_key it was always failing to find anything.
To fix the problem I added a gsub to the end of the code that generates the private_key, like this...
private_key = SecurityHelper.encrypt_private_key(client_uuid).gsub(/\s+/, "")
This strips all white space from the generated private key and solved the problem.
You're using the wrong quoting style. ANSI SQL quoting as used in PostgreSQL is:
"SELECT * FROM clients WHERE private_key = 'MkRBNUZBQzUtMzVDRi00NzQ3LThFNjEtNjI4OThERUQzQkRF';"
Note the single quotes.
It isn't clear how the first statement even got parsed by Ruby, since the inner double quotes would've ended the quoted string, leading to:
"SELECT * FROM clients WHERE private_key = "MkRBNUZBQzUtMzVDRi00NzQ3LThFNjEtNjI4OThERUQzQkRF";"
^^^^
should be Ruby syntax error here
I don't really do Rails so I can't speak for the correct Rails syntax to do this check via ActiveRecord, though.
Have you tried
Client.exists?(:private_key => "MkRBNUZBQzUtMzVDRi00NzQ3LThFNjEtNjI4OThERUQzQkRF")
I have an application that has tens of thousands of snapshot records. A very small number of these 'snapshots' (say 1 in 1000) will have one or more 'positions' through a :has_many association.
How can I efficiently discover if a snapshot has a position without firing an active record query for each snapshot? My current solution is to add a boolean field to snapshots - if a snapshot has a position, 'has_position' is set to true. This seems a little messy since it means I have to modify the associated snapshot every time I create a position. Is there a cleaner way to handle this scenario?
create_table "snapshots", :force => true do |t|
t.datetime "created_at",
t.datetime "updated_at",
t.boolean "has_position",
end
create_table "positions", :force => true do |t|
t.integer "snapshot_id"
t.datetime "created_at",
t.datetime "updated_at",
end
What will happen if you generate the migration for positions with the reference to snapshots, the migration file will be generated with a
add_index :positions, :snapshot_id
appended to the end of it.
With an index on snapshot_id the DB will take log(n) queries to figure out whether or not a position has at least one associated record. Not as good as constant time with the boolean, but with mere tens of thousands of records it shouldn't take noticeably longer (unless you're doing this very, very frequently).
Additionally, a simple has_position boolean might be harder than you think to maintain without an index. You can set it to true on creation of an associated position, but you can't set it to false on the deletion because there might exist another one, and you'd have to do a table scan again.
If for some reason using an index is undesirable (or you really need constant time lookup), then I'd recommend using a :counter_cache column.
Newbie question.
I am trying to use ActiveRecord::Base.connection.execute(..) in a ruby 3.1 app and docs I have read seem to be straight forward but for the life in me I can seem to understand why I cant get the code below to work. The error message I am getting suggests that the execute function is looking for a column with the name of one of the values I am trying to save but I dont understand why.
Firstly, my db table structure is as follows:
create_table "countries", :force => true do |t|
t.string "iso3"
t.string "iso2"
t.string "name"
t.datetime "created_at"
t.datetime "updated_at"
end
And the code Im playing with is as follows:
code = 'ZA'
name = 'South Africa'
ActiveRecord::Base.connection.execute("INSERT INTO countries ('iso3', 'iso2', 'name')
VALUES ('Null', #{code}, #{name})")
The error message I am getting is as follows:
SQLite3::SQLException: no such column: ZA: INSERT INTO countries ('iso3', 'iso2', 'name')
VALUES ('Null', ZA, SouthAfrica)
Where did you get the basis for this? Code of this variety is a sterling example of what not to do.
If you have ActiveRecord, then you have ActiveRecord::Model, and with that you're on the right track and pretty much done. You don't need to write raw SQL for routine things of this variety. It's not necessary, and more, it's extremely dangerous for the reasons you've just discovered. You can't just shove random things in to your query or you will end up with nothing but trouble.
What you should be doing is declaring a model and then using it:
# Typically app/models/country.rb
class Country < ActiveRecord::Base
end
To insert once you have a model is made seriously easy:
Country.create(
:code => 'ZA',
:name => 'South Africa'
)
A good ActiveRecord reference is invaluable as this facility will make your life significantly easier if you make use of it.
Within Rails you usually go about generating these automatically so that you have something rough to start with:
rails generate model country
This will take care of creating the migration file, the model file, and some unit test stubs you can fill in later.
The error is just because if the missing quotes. it should be like:
INSERT INTO countries ('iso3', 'iso2', 'name') VALUES ('Null', 'ZA', 'SouthAfrica')
I want to have a "Customer" Model with a normal primary key and another column to store a custom "Customer Number". In addition, I want the db to handle default Customer Numbers. I think, defining a sequence is the best way to do that. I use PostgreSQL. Have a look at my migration:
class CreateAccountsCustomers < ActiveRecord::Migration
def up
say "Creating sequenze for customer number starting at 1002"
execute 'CREATE SEQUENCE customer_no_seq START 1002;'
create_table :accounts_customers do |t|
t.string :type
t.integer :customer_no, :unique => true
t.integer :salutation, :limit => 1
t.string :cp_name_1
t.string :cp_name_2
t.string :cp_name_3
t.string :cp_name_4
t.string :name_first, :limit => 55
t.string :name_last, :limit => 55
t.timestamps
end
say "Adding NEXTVAL('customer_no_seq') to column cust_id"
execute "ALTER TABLE accounts_customers ALTER COLUMN customer_no SET DEFAULT NEXTVAL('customer_no_seq');"
end
def down
drop_table :accounts_customers
execute 'DROP SEQUENCE IF EXISTS customer_no_seq;'
end
end
If you know a better "rails-like" approach to add sequences, would be awesome to let me know.
Now, if I do something like
cust = Accounts::Customer.new
cust.save
the field customer_no is not pre filled with the next value of the sequence (should be 1002).
Do you know a good way to integrate sequences? Or is there a good plugin?
Cheers to all answers!
I have no suggestions for a more 'rails way' of handling custom sequences, but I can tell you why the customer_no field appears not to be being populated after a save.
When ActiveRecord saves a new record, the SQL statement will only return the ID of the new record, not all of its fields, you can see where this happens in the current rails source here https://github.com/rails/rails/blob/cf013a62686b5156336d57d57cb12e9e17b5d462/activerecord/lib/active_record/persistence.rb#L313
In order to see the value you will need to reload the object...
cust = Accounts::Customer.new
cust.save
cust.reload
If you always want to do this, consider adding an after_create hook in to your model class...
class Accounts::Customer < ActiveRecord::Base
after_create :reload
end
I believe that roboles answer is not correct.
I tried to implement this on my application (exactly the same env: RoR+PostgreSQL), and I found out that when save is issued on RoR with the object having empty attributes, it tries to perform an INSERT on the database mentioning that all VALUES shall be set to NULL. The problem is the way PostgreSQL handles NULLs: in this case, the new row will be created but with all values empty, i.e. the DEFAULT will be ignored. If save only wrote on the INSERT statement attributes filled on RoR, this would work fine.
In other words, and focusing only on the type and customer_no attribute mentioned above, this is the way PostgreSQL behaves:
SITUATION 1:
INSERT INTO accounts_customers (type, customer_no) VALUES (NULL, NULL);
(this is how Rails' save works)
Result: a new row with empty type and empty customer_no
SITUATION 2:
INSERT INTO accounts_customers (type) VALUES (NULL);
Result: a new row with empty type and customer_no filled with the sequence's NEXTVAL
I have a thread going on about this, check it out at:
Ruby on Rails+PostgreSQL: usage of custom sequences
I faced a similar problem, but I also put :null => false on the field hopping that it will be auto-populated with nextval.
Well, in my case AR was still trying to insert NULL if no attribute was supplied in the request, and this resulted in an exception for not-null constraint violation.
Here's my workaround. I just deleted this attribute key from #attributes and #changed_attributes and in this case postgres correctly put the expected sequence nextval.
I've put this in the model:
before_save do
if (#attributes["customer_no"].nil? || #attributes["customer_no"].to_i == 0)
#attributes.delete("customer_no")
#changed_attributes.delete("customer_no")
end
end
Rails 3.2 / Postgres 9.1
If you're using PostgreSQL, check out the gem I wrote, pg_sequencer:
https://github.com/code42/pg_sequencer
It provides a DSL for creating, dropping and altering sequences in ActiveRecord migrations.