I'm currently in the process of writing code to convert our databases default ID columns to a UUID format.
One example of the code that I have for our migration is
def up
remove_reference :examples, :user, null: false, foreign_key: true
add_column :users, :uuid, :uuid, default: "gen_random_uuid()", null: false
change_table :users do |t|
t.remove :id, type: :id
t.rename :uuid, :id
end
execute "ALTER TABLE users ADD PRIMARY KEY (id);"
add_reference :examples, :user, null: false, foreign_key: true, type: :uuid
end
Essentially this allowed me to convert my ID column to a UUID format.
I created a down function so I would be able to rollback but it fails due to this error ERROR: column "user_id" of relation "examples" contains null values
I realize that there would be an issue because once there is data in the database it would be unable to rollback and create the correct references again. Does anyone have any ideas on how I should work on my down function?
def down
remove_reference :examples, :user, null: false, foreign_key: true, type: :uuid
execute 'ALTER TABLE users DROP CONSTRAINT users_pkey'
add_column :users, :new_id, :primary_key
change_table :users do |t|
t.remove :id, type: :uuid
t.rename :new_id, :id
end
add_reference :examples, :user, null: false, foreign_key: true
end
Does anyone have any suggestions on how I should proceed with this? The original migration was in one change function, but it would be unable to rollback due to the execute block.
Be careful doing this kind of database change in one-shot. I would suggest you to break into steps.
First step (new column uuid)
Create the new column for the uuid
def up
add_column :users, :uuid, :uuid, default: "gen_random_uuid()", null: false
add_column :examples, :user_uuid, :uuid
end
Adapt your code to populate the examples.user_uuid column with the recent created column; You can easily achieve this by creating a model callback, feeling the user_uuid automatically.
If your database has a GB of data, consider adding the uuid as nullable, and populate the column using queues or in batches. The new records will be already filled.
We have now two new columns, with new data comming and all synced
Second step (the renaming)
Once populated and working with new columns, is time to rename the columns and associate the new keys.
def up
change_table :examples do |t|
t.rename :user_id, type: :old_user_id
t.rename :user_uuid, :user_id
end
change_table :users do |t|
t.rename :id, type: :old_id
t.rename :uuid, :id
end
execute "ALTER TABLE users ADD PRIMARY KEY (id);"
add_reference :examples, :user, null: false, foreign_key: true, type: :uuid
end
def down
# ...
end
Rember to remove, or review the model code changed before, to support this new columns or ignore.
Now we have changed the columns, without loosing the old reference.
Be careful here, if your database is big, you may lock your operation. Perhaps you may need a maintenance window here.
Third step (removing the old columns)
Now we can remove the old columns and everything should work fine
note: Always be careful when making this kind of change into your database. It is very risk to perform something like this. If you want to go on, simulate several times the step of renaming. Make snapshot of your database before performing and inform your clients that might have a downtime in your service.
I don't know why you want to change your primary keys to be an uuid, this costs a lot to the database to query and join data. It's more complicated to compare an UUID than an integer. Consider just create a new indexed column uuid into your tables and let the database to join and based on this field.
Related
So I read this question, answer and the comments, but it doesn't answer my case, which is what to do when of the columns is a foreign key?
Here is my original migration to create the table in question:
class CreateTemplates < ActiveRecord::Migration[5.1]
def change
create_table :templates, id: :uuid do |t|
t.references :account, type: :uuid, foreign_key: true
t.string :name
t.text :info
t.string :title
t.timestamps
end
end
end
Since account_id is a foreign_key (and identifies the customer) it will appear in almost all (99%) of queries on this table.
Now it has been decided that name should be unique to account, so the model has been updated:
validates_uniqueness_of :name, scope: [:account]
So once I add the joint index:
add_index :templates, [:name, :account_id], unique: true
should I delete the index on account_id?
I ask because in SQLLite (see this), it seems the answer would be that I don't need the single index on account_id and to create my new index with account_id in the first position:
add_index :templates, [:account_id, :name], unique: true
I'm using postgres, so does the same idea apply?
You have to add extra index if it's not the first index.
So if you have this:
add_index :templates, [:name, :account_id], unique: true
then you should not delete the original :account_id foreign key index, since it is second index.
I recommend you to read about index implementations. It's pretty interesting and you can learn a lot from it.
Here is a migration to create a table:
class CreateTemplates < ActiveRecord::Migration[5.1]
def change
create_table :templates, id: :uuid do |t|
t.references :account, type: :uuid, foreign_key: true
t.string :name
t.text :info
t.string :title
t.timestamps
end
end
end
Since account_id is a foreign_key (and identifies the customer) it will appear in almost all (99%) of queries on this table - there is not much point in retrieving a template that belongs to another customer.
So should I drop the index the above migration created for the account_id foreign_key and create this one instead?
add_index :templates, [:id, :account_id], unique: false
Or should I keep the original and also add this?
EDIT
To clarify the 99% use case - I think I was mistaken there. When creating a template, the account_id is always inserted so that the index method of the tempaltes_controller will always return all templates using the account_id, so that a user only sees a list of templates belonging to their account. For edits, updates, deletes, those actions only need the template_id. So my 99% guess is wrong! Most queries won't actually need a composite key it seems to me.
If most of your queries are going to filter on a combination of [:id, :account_id](which is unlikely) then creating a composite index will improve the performance of your queries.
However, it sounds like that most of your queries will only require :account_id, If that is the case then you do not need to add a composite index.
Trying to add Timestamps to existing table.
According to Api documenation add_timestamps
Here is my code in migration:
def change
add_timestamps(:products, null: false)
end
Getting error:
*-- add_timestamps(:products, {:null=>false})
rails aborted!
StandardError: An error has occurred, this and all later migrations canceled:
SQLite3::SQLException: Cannot add a NOT NULL column with default value NULL: ALTER TABLE "products" ADD "created_at" datetime NOT NULL*
I've also tried all solution in this thread
Same error...
Rails 5.1.4
Ruby 2.4.0
You cannot add columns with not-null constraint to a non-empty table because the existing lines in the table would have empty values right away and therefore the condition fails.
Instead, introduce the columns in three steps:
def change
# add new column but allow null values
add_timestamps :products, null: true
# backfill existing records with created_at and updated_at
# values that make clear that the records are faked
long_ago = DateTime.new(2000, 1, 1)
Product.update_all(created_at: long_ago, updated_at: long_ago)
# change to not null constraints
change_column_null :products, :created_at, false
change_column_null :products, :updated_at, false
end
In my opinion, it is wrong to manipulate existing data with activerecord queries or even SQL in migrations.
The correct rails 5.2+ way to do this is :
class AddTimestampsToCars < ActiveRecord::Migration[5.2]
def change
add_timestamps :cars, null: false, default: -> { 'NOW()' }
end
end
It's a proc so you should be able to set a date in the past if you want to.
Source: https://github.com/rails/rails/pull/20005
I like #spickermann's approach since it takes into account the existing records and probably your migration already went all the way to production, his method ensures data perseverance.
Nevertheless, many of you guys might find yourselves in that situation, but still in development, meaning that there's no real sensitive data you might be afraid of losing... That gives you a bit more freedom on how you can perform the change in the table.
If your code and records only exist locally (if you still have no records created, just skip step 1.) and that table was created in the last migration , my suggestion is:
1.- Delete all the records from that table.
2.- Go to your migration file and edit it by adding t.timestamps so that it looks something like this:
class CreateInstitutionalLegals < ActiveRecord::Migration[5.0]
def change
create_table :institutional_legals do |t|
# Your original migration content goes here
.
.
t.timestamps # This is your addition
end
end
end
3.- Then go to your console and enter rails:db:redo. As explained here, that command is a shortcut for doing a rollback and then migrating back up again.
Now you will see that your schema is updated with the corresponding created_atand updated_at columns.
The concrete benefit of this is that it is super easy to do, you don't create an extra migration file and you learn to use a very handy command ;)
I'm on rails 5.0 and none of these options worked. The rails:db:redo will work but isn't a feasible solution for most.
The only thing that worked was
def change
add_column :products, :created_at, :timestamp
add_column :products, :updated_at, :timestamp
end
I had the same issue. I wanted the end result to be strictly equivalent to add_timestamps :products on an fresh database.
Instead of running a query to backfill, I ended up doing a 3-steps process.
add column with null allowed and default to current time to backfill
change constraint to not null
remove default
And it is reversible.
add_column :products, :created_at, :datetime, precision: 6, null: true, default: -> { "CURRENT_TIMESTAMP" }
add_column :products, :updated_at, :datetime, precision: 6, null: true, default: -> { "CURRENT_TIMESTAMP" }
change_column_null :products, :created_at, false
change_column_null :products, :updated_at, false
change_column_default :products, :created_at, from: -> { "CURRENT_TIMESTAMP" }, to: nil
change_column_default :products, :updated_at, from: -> { "CURRENT_TIMESTAMP" }, to: nil
NB: This is with Rails 6.1 and PostgreSQL
What I need is a migration to apply unique constraint to a combination of columns. i.e. for a people table, a combination of first_name, last_Name and Dob should be unique.
add_index :people, [:firstname, :lastname, :dob], unique: true
According to howmanyofme.com, "There are 46,427 people named John Smith" in the United States alone. That's about 127 years of days. As this is well over the average lifespan of a human being, this means that a DOB clash is mathematically certain.
All I'm saying is that that particular combination of unique fields could lead to extreme user/customer frustration in future.
Consider something that's actually unique, like a national identification number, if appropriate.
(I realise I'm very late to the party with this one, but it could help future readers.)
You may want to add a constraint without an index. This will depend on what database you're using. Below is sample migration code for Postgres. (tracking_number, carrier) is a list of the columns you want to use for the constraint.
class AddUniqeConstraintToShipments < ActiveRecord::Migration
def up
execute <<-SQL
alter table shipments
add constraint shipment_tracking_number unique (tracking_number, carrier);
SQL
end
def down
execute <<-SQL
alter table shipments
drop constraint if exists shipment_tracking_number;
SQL
end
end
There are different constraints you can add. Read the docs
For completeness sake, and to avoid confusion here are 3 ways of doing the same thing:
Adding a named unique constraint to a combination of columns in Rails 5.2+
Let's say we have Locations table that belongs to an advertiser and has column reference_code and you only want 1 reference code per advertiser. so you want to add a unique constraint to a combination of columns and name it.
Do:
rails g migration AddUniquenessConstraintToLocations
And make your migration look either something like this one liner:
class AddUniquenessConstraintToLocations < ActiveRecord::Migration[5.2]
def change
add_index :locations, [:reference_code, :advertiser_id], unique: true, name: 'uniq_reference_code_per_advertiser'
end
end
OR this block version.
class AddUniquenessConstraintToLocations < ActiveRecord::Migration[5.2]
def change
change_table :locations do |t|
t.index ['reference_code', 'advertiser_id'], name: 'uniq_reference_code_per_advertiser', unique: true
end
end
end
OR this raw SQL version
class AddUniquenessConstraintToLocations < ActiveRecord::Migration[5.2]
def change
execute <<-SQL
ALTER TABLE locations
ADD CONSTRAINT uniq_reference_code_per_advertiser UNIQUE (reference_code, advertiser_id);
SQL
end
end
Any of these will have the same result, check your schema.rb
Hi You may add unique index in your migration to the columns for example
add_index(:accounts, [:branch_id, :party_id], :unique => true)
or separate unique indexes for each column
In the typical example of a join table between users and posts:
create_table :users
create_table :posts
create_table :ownerships do |t|
t.belongs_to :user, foreign_key: true, null: false
t.belongs_to :post, foreign_key: true, null: false
end
add_index :ownerships, [:user_id, :post_id], unique: true
Trying to create two similar records will throw a database error (Postgres in my case):
ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_ownerships_on_user_id_and_post_id"
DETAIL: Key (user_id, post_id)=(1, 1) already exists.
: INSERT INTO "ownerships" ("user_id", "post_id") VALUES ($1, $2) RETURNING "id"
e.g. doing that:
Ownership.create!(user_id: user_id, post_id: post_id)
Ownership.create!(user_id: user_id, post_id: post_id)
Fully runnable example: https://gist.github.com/Dorian/9d641ca78dad8eb64736173614d97ced
db/schema.rb generated: https://gist.github.com/Dorian/a8449287fa62b88463f48da986c1744a
If you are creating a new table just add unique: true
class CreatePosts < ActiveRecord::Migration[6.0]
def change
create_table :posts do |t|
t.string :title, unique: true
t.text :body
t.references :user, foreign_key: true
t.timestamps
end
add_index :posts, :user_id, unique: true
end
end
Took me a while to track down this error but I finally found out why. I am modeling a card game using the Rails framework. Currently my database looks (mostly) like this:
cards cards_games games
----- ----------- -----
id id id
c_type card_id ...
value game_id other_stuff
And the Rails ActiveRecord card.rb and game.rb currently look like this
#card.rb
class Card < ActiveRecord::Base
has_and_belongs_to_many :player
has_and_belongs_to_many :game
has_and_belongs_to_many :cardsInPlay, :class_name => "Rule"
end
#game.rb
class Game < ActiveRecord::Base
has_and_belongs_to_many :cards
has_many :players
has_one :rules, :class_name => Rule
end
When I attempt to run a game and there are multiple games (more than 1), I get the error
ActiveRecord::StatementInvalid in GameController#start_game
# example
Mysql::Error: Duplicate entry '31' for key 1: INSERT INTO `cards_games` (`card_id`, `id`, `game_id`) VALUES (31, 31, 7)
Every time the action fails, cardid == id. This, I assume, has something with how Rails inserts the data into the database. Since there is no cardsgames object, I think it is just pulling card_id into id and inserting it into the database. This works fine until you have two games with the same card, which violates the primary key constraint on cardsgames. Being affluent with databases, my first solution to this problem was to try to force rails to follow a "real" definition of this relationship by dropping id and making cardid and gameid a primary key. It didn't work because the migration couldn't seem to handle having two primary keys (despite the Rails API saying that its okay to do it.. weird). Another solution for this is to omit the 'id' column in the INSERT INTO statement and let the database handle the auto increment. Unfortunately, I don't know how to do this either.
So, is there another work-around for this? Is there some nifty Rails trick that I just don't know? Or is this sort of structure not possible in Rails? This is really frustrating because I know what is wrong and I know several ways to fix it but due to the constraints of the Rail framework, I just cannot do it.
has_and_belongs_to_many implies a join table, which must not have an id primary key column. Change your migration to
create_table :cards_games, :id => false do ...
as pointed out by Matt. If you will only sleep better if you make a key from the two columns, create a unique index on them:
add_index :cards_games, [ :card_id, :game_id ], :unique => true
Additionally, your naming deviates from Rails convention and will make your code a little harder to read.
has_and_belongs_to_many defines a 1:M relationship when looking at an instance of a class. So in Card, you should be using:
has_and_belongs_to_many :players
has_and_belongs_to_many :games
Note plural "players" and "games". Similarly in Game:
has_one :rule
This will let you drop the unnecessary :class_name => Rule, too.
To drop the ID column, simply don't create it to begin with.
create_table :cards_rules, :id => false do ...
See Dr. Nics composite primary keys
http://compositekeys.rubyforge.org/
I found the solution after hacking my way through. I found out that you can use the "execute" function inside of a migration. This is infinitely useful and allowed me to put together an non-elegant solution to this problem. If anyone has a more elegant, more Rails-like solution, please let me know. Here's the solution in the form of a migration:
class Make < ActiveRecord::Migration
def self.up
drop_table :cards_games
create_table :cards_games do |t|
t.column :card_id, :integer, :null => false
t.column :game_id, :integer, :null => false
end
execute "ALTER TABLE cards_games DROP COLUMN id"
execute "ALTER TABLE cards_games ADD PRIMARY KEY (card_id, game_id)"
drop_table :cards_players
create_table :cards_players do |t|
t.column :card_id, :integer, :null => false
t.column :player_id, :integer, :null => false
end
execute "ALTER TABLE cards_players DROP COLUMN id"
execute "ALTER TABLE cards_players ADD PRIMARY KEY (card_id, player_id)"
drop_table :cards_rules
create_table :cards_rules do |t|
t.column :card_id, :integer, :null => false
t.column :rule_id, :integer, :null => false
end
execute "ALTER TABLE cards_rules DROP COLUMN id"
execute "ALTER TABLE cards_rules ADD PRIMARY KEY (card_id, rule_id)"
end
def self.down
drop_table :cards_games
create_table :cards_games do |t|
t.column :card_id, :integer
t.column :game_id, :integer
end
drop_table :cards_players
create_table :cards_players do |t|
t.column :card_id, :integer
t.column :player_id, :integer
end
drop_table :cards_rules
create_table :cards_rules do |t|
t.column :card_id, :integer
t.column :rule_id, :integer
end
end
end
You might want to check out this foreign_key_migrations plugin