I have this migration in a Rails 7 app using Postgresql 13:
class AddContractsDateRangeConstraint < ActiveRecord::Migration[7.0]
def up
execute "CREATE EXTENSION IF NOT EXISTS btree_gist"
execute <<~SQL
ALTER TABLE contracts
ADD CONSTRAINT date_overlap_exclude
EXCLUDE USING GIST (
employee_id WITH =,
daterange(starts_on, ends_on) WITH &&
)
SQL
end
def down
execute "ALTER TABLE contracts DROP CONSTRAINT date_overlap_exclude"
end
end
When executing via rails db:migrate the constraint gets added to the DB like this and everything works:
app_development=# \d+ contracts
...
Indexes:
...
"date_overlap_exclude" EXCLUDE USING gist (employee_id WITH =, daterange(starts_on, ends_on) WITH &&)
...
The generated schema.rb from Rails looks like this:
create_table "contracts", force: :cascade do |t|
# ...
t.index "employee_id, daterange(starts_on, ends_on)", name: "date_overlap_exclude", using: :gist
# ...
end
This looks suspicious, as it lacks the whole EXCLUDE part from my constraint. And indeed, when creating the database from the generated schema using rails db:schema:load the constraint is broken and is generated like this:
app_development=# \d+ contracts
...
Indexes:
"contracts_pkey" PRIMARY KEY, btree (id)
"date_overlap_exclude" gist (employee_id, daterange(starts_on, ends_on))
And of course, the whole constraint doesn't work anymore. Any idea how to solve this?
The Rails Ruby schema dumper works very well for simple applications and use cases but it only actually understands a very limited subset of SQL. As you have experienced here any parts of the database schema that it doesn't understand are simply lost in translation.
Once you get the point where you need to use functions, constraints, materialized views or any other database specific features you need to use SQL schema dumps which are much more high fidelity since its just using the databases own dumping tools to output the schema as SQL. While you're losing the simplicity and database-agnostic characteristics of schema.rb this is an inevitable change as the complexity of the application grows.
You can switch the schema format with a simple line in your configuration:
module YourApp
class Application < Rails::Application
# Add this line:
config.active_record.schema_format = :sql
end
end
When you run rails db:schema:dump it will now output a structure.sql file instead. At this point you should delete your schema.rb and check structure.sql into version control.
See:
Pros and Cons of Using structure.sql in Your Ruby on Rails Application
Rails Guides : Schema Dumping and You
Related
In Rails 7:
Migration:
class AddSavedSearchAPIKeyIndex < ActiveRecord::Migration[7.0]
def change
execute <<~SQL
ALTER TABLE saved_searches
ADD INDEX ecomm_analyze_api_key ((
CAST(configuration->>"$.ecomm_analyze_api_key" as CHAR(255)) COLLATE utf8mb4_bin
)) USING BTREE;
SQL
end
end
My schema ends up looking like this:
t.index "(cast(json_unquote(json_extract(`configuration`,_utf8mb4\\'$.ecomm_analyze_api_key\\')) as char(255) charset utf8mb4) collate utf8mb4_bin)", name: "ecomm_analyze_api_key"
I can't load this schema.rb without getting an error when I use:
RAILS_ENV=test rake db:test:prepare
Error message
ActiveRecord::StatementInvalid: Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use
I am able to fix this by manually editing the schema.rb but this will just break again on the next migration.
The Ruby schema dumper only actually understands a limited subset of SQL that can be created by the migrations DSL and is only actually useful until your app reaches the degree of complexity where you want to use database specific features.
Anything else that the Ruby schema dumper doesn't properly understand like this index will either be mangled or just lost in translation when it tries to parse the resulting SQL from dumping the database schema.
The solution is to use SQL schema dumps instead:
module YourApp
class Application < Rails::Application
# Add this line:
config.active_record.schema_format = :sql
end
end
This will dump the file structure.sql which should serve as the source of truth for the database schema. Check this file into version control and delete schema.rb to avoid confusion.
See:
Rails Guides : Schema Dumping and You
Pros and Cons of Using structure.sql in Your Ruby on Rails Application
The app in question was originally created as a Rails 4 app, and later upgraded to Rails 5.
I will create a rails migration that might look like this:
class AddPubliclyVisibleToGcodeMacros < ActiveRecord::Migration[5.0]
def change
add_column :gcode_macros, :publicly_visible, :boolean, default: false
end
end
And when I run it, I expect the schema to have a few lines updated, specifically adding t.boolean "publicly_visible", default: false
to the gcode_macros table.
However, running the migration creates a LOT of changes to my schema, mostly just moving indexes from outside a create_table block, into it.
Im quite confused over whats going on here. This isn't something that happened all of a sudden, I've just been working around it for a while now.
Any help would be greatly appreciated!
The answer is that this is just how the schema dumper works in Rails. It takes the schema from the database, absolutely irrelevant from how you created the structure in the first place, whether with migrations or direct sql statements.
So when you create a new migration or change anything in the db a new schema is dumped based on the database.
Edit
I should add that schema.rb is not updated if the db is changed directly with sql statements, that is not through a migration. Only when either
rake db:migrate
or ...
rake db:schema:dump
are run is the schema.rb file updated.
How could I create an index like this
CREATE INDEX index_companies_on_addresses_on_zipcode ON companies USING gin ((addresses -> 'zipcode'));
using the syntax of migrations?
I've created using the syntax below and it was created on database
execute "CREATE INDEX index_companies_on_addresses_on_zipcode ON companies USING gin ((addresses -> 'zipcode'));"
but when I saw schema.rb index wasn't there.
I've tried
add_index :companies, :addresses, using: :gin
but it create an index only on column addresses(is a jsonb) not on key zipcode.
AFAIK there's no way to make add_index understand that sort of indexing. Fear not, there are ways around this. You're using jsonb so there's no need to worry about database portability so you can switch from db/schema.rb to db/structure.sql for managing your schema. The db/structure.sql file is pretty much a native SQL dump of your database structure so it will contain everything that the database understands (CHECK constraints, advanced indexes, ...).
The process is quite simple:
Edit config/application.rb to set the schema format:
class Application < Rails::Application
#...
config.active_record.schema_format = :sql
#...
end
Run your migration that uses execute as normal. You'll want to use separate up and down methods in your migrations or reversible inside the usual change method because ActiveRecord won't know how to reverse execute some_raw_sql on its own.
This should leave you with a db/structure.sql file so add that to your revision control and delete db/schema.rb.
Use different rake tasks for working with structure.sql instead of schema.rb:
db:structure:dump instead of db:schema:dump
db:structure:load instead of db:schema:load
As you will see, I'm pretty new to Rails.
I am trying to follow section 3.3.2 in the RailsGuides on Active Record Associations and it says there that when defining a many to many relationship, besides adding the has_and_belongs_to_many directive to the model, you "need to explicitly create the joining table".
It then gives an example of the content of the migration file:
class CreateAssembliesPartsJoinTable < ActiveRecord::Migration
def change
create_table :assemblies_parts, id: false do |t|
t.integer :assembly_id
t.integer :part_id
end
end
end
My question is: what name should I give to that file? I see that the files generated by the rails g ... command are all in the ..db\migrate folder and have a kind of timestamp at the beginning of the file. Can I use any name? I'm afraid to test and mess up the whole thing. I'm used to MS-SQL and being able to see the tables, add/modify columns, etc.
Side question: there are a few files already there, from previous migrations. How does rails know which ones were already run? And what about running them afterwards when deploying to, say, Heroku?
You can give any name to your migrations. Preferably something self explanatory like create_table_something. You can generate a migration by doing
rails generate migration create_assemblies_parts_joins_table
This will generate a file like below in db/migrate folder
<timestamp>_create_assemblies_parts_joins_table
Rails keeps track of already run migrations in scheme_migrations table. It stores the timestamp of all the migrations that are already run
UPDATE:
You can change the table name to whatever you want in the migration file. Table will be created with the name you give in the following line
create_table :assemblies_parts, id: false do |t|
assemblies_parts will be the table name.
You should not build the file from scratch yourself. As #Vimsha has already said - you can run a rails migration to create a join table migration for you.
The rails standard naming for a join table is to take the two names of the models you are joining, and write them in alphabetical order and pluralised.
eg if your models are "user" and "post", it would be "posts_users", but if it were "post" and "comment" it would be "comments_posts"
I'm currently running a Rails migration where I am adding a datatype specific to Postgres, the tsvector. It holds search information in the form that Postgres expects for its built-in text searching capabilities.
This is the line from my migration:
t.column "search_vectors", :tsvector
Everything seems to be working fine, and the search works with it. However, when I opened up schema.rb, this is what I got:
Could not dump table "users" because of following StandardError
Unknown type 'tsvector' for column 'search_vectors'
This is preventing me from running unit tests on the user table, and also strikes me as really dangerous looking given that the schema.rb is supposed to be the authoritative definition of my database.
I notice there are a number of Rails plugins that seem to use the same approach of storing the tsvector like I would expect, such as tsearchable. Am I really just stuck without testing and without an authoritative definition of my database?
FYI for anyone who happens across this page, I fixed this by adding this (actually uncommenting it) to my Rails config:
config.active_record.schema_format = :sql
Have you tried specifying the type as a string instead of a symbol?
t.column "search_vectors", "tsvector"
If that doesn't work then you might need to drop down to database-specific SQL:
def self.up
execute "--Put your PostgreSQL specific SQL statements here"
end