How could I create an index like this
CREATE INDEX index_companies_on_addresses_on_zipcode ON companies USING gin ((addresses -> 'zipcode'));
using the syntax of migrations?
I've created using the syntax below and it was created on database
execute "CREATE INDEX index_companies_on_addresses_on_zipcode ON companies USING gin ((addresses -> 'zipcode'));"
but when I saw schema.rb index wasn't there.
I've tried
add_index :companies, :addresses, using: :gin
but it create an index only on column addresses(is a jsonb) not on key zipcode.
AFAIK there's no way to make add_index understand that sort of indexing. Fear not, there are ways around this. You're using jsonb so there's no need to worry about database portability so you can switch from db/schema.rb to db/structure.sql for managing your schema. The db/structure.sql file is pretty much a native SQL dump of your database structure so it will contain everything that the database understands (CHECK constraints, advanced indexes, ...).
The process is quite simple:
Edit config/application.rb to set the schema format:
class Application < Rails::Application
#...
config.active_record.schema_format = :sql
#...
end
Run your migration that uses execute as normal. You'll want to use separate up and down methods in your migrations or reversible inside the usual change method because ActiveRecord won't know how to reverse execute some_raw_sql on its own.
This should leave you with a db/structure.sql file so add that to your revision control and delete db/schema.rb.
Use different rake tasks for working with structure.sql instead of schema.rb:
db:structure:dump instead of db:schema:dump
db:structure:load instead of db:schema:load
Related
In Rails 7:
Migration:
class AddSavedSearchAPIKeyIndex < ActiveRecord::Migration[7.0]
def change
execute <<~SQL
ALTER TABLE saved_searches
ADD INDEX ecomm_analyze_api_key ((
CAST(configuration->>"$.ecomm_analyze_api_key" as CHAR(255)) COLLATE utf8mb4_bin
)) USING BTREE;
SQL
end
end
My schema ends up looking like this:
t.index "(cast(json_unquote(json_extract(`configuration`,_utf8mb4\\'$.ecomm_analyze_api_key\\')) as char(255) charset utf8mb4) collate utf8mb4_bin)", name: "ecomm_analyze_api_key"
I can't load this schema.rb without getting an error when I use:
RAILS_ENV=test rake db:test:prepare
Error message
ActiveRecord::StatementInvalid: Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use
I am able to fix this by manually editing the schema.rb but this will just break again on the next migration.
The Ruby schema dumper only actually understands a limited subset of SQL that can be created by the migrations DSL and is only actually useful until your app reaches the degree of complexity where you want to use database specific features.
Anything else that the Ruby schema dumper doesn't properly understand like this index will either be mangled or just lost in translation when it tries to parse the resulting SQL from dumping the database schema.
The solution is to use SQL schema dumps instead:
module YourApp
class Application < Rails::Application
# Add this line:
config.active_record.schema_format = :sql
end
end
This will dump the file structure.sql which should serve as the source of truth for the database schema. Check this file into version control and delete schema.rb to avoid confusion.
See:
Rails Guides : Schema Dumping and You
Pros and Cons of Using structure.sql in Your Ruby on Rails Application
I have this migration in a Rails 7 app using Postgresql 13:
class AddContractsDateRangeConstraint < ActiveRecord::Migration[7.0]
def up
execute "CREATE EXTENSION IF NOT EXISTS btree_gist"
execute <<~SQL
ALTER TABLE contracts
ADD CONSTRAINT date_overlap_exclude
EXCLUDE USING GIST (
employee_id WITH =,
daterange(starts_on, ends_on) WITH &&
)
SQL
end
def down
execute "ALTER TABLE contracts DROP CONSTRAINT date_overlap_exclude"
end
end
When executing via rails db:migrate the constraint gets added to the DB like this and everything works:
app_development=# \d+ contracts
...
Indexes:
...
"date_overlap_exclude" EXCLUDE USING gist (employee_id WITH =, daterange(starts_on, ends_on) WITH &&)
...
The generated schema.rb from Rails looks like this:
create_table "contracts", force: :cascade do |t|
# ...
t.index "employee_id, daterange(starts_on, ends_on)", name: "date_overlap_exclude", using: :gist
# ...
end
This looks suspicious, as it lacks the whole EXCLUDE part from my constraint. And indeed, when creating the database from the generated schema using rails db:schema:load the constraint is broken and is generated like this:
app_development=# \d+ contracts
...
Indexes:
"contracts_pkey" PRIMARY KEY, btree (id)
"date_overlap_exclude" gist (employee_id, daterange(starts_on, ends_on))
And of course, the whole constraint doesn't work anymore. Any idea how to solve this?
The Rails Ruby schema dumper works very well for simple applications and use cases but it only actually understands a very limited subset of SQL. As you have experienced here any parts of the database schema that it doesn't understand are simply lost in translation.
Once you get the point where you need to use functions, constraints, materialized views or any other database specific features you need to use SQL schema dumps which are much more high fidelity since its just using the databases own dumping tools to output the schema as SQL. While you're losing the simplicity and database-agnostic characteristics of schema.rb this is an inevitable change as the complexity of the application grows.
You can switch the schema format with a simple line in your configuration:
module YourApp
class Application < Rails::Application
# Add this line:
config.active_record.schema_format = :sql
end
end
When you run rails db:schema:dump it will now output a structure.sql file instead. At this point you should delete your schema.rb and check structure.sql into version control.
See:
Pros and Cons of Using structure.sql in Your Ruby on Rails Application
Rails Guides : Schema Dumping and You
We are starting a project based on Ruby on Rails. We used to work with Perl and PostgreSQL functions, and with Rails and Active Record I've not seen how we are supposed to create functions in PostgreSQL and keep the record with Active Record and models.
I know we can create it manually in PostgreSQL, but the "magic" with Active Record is that the database can be recreated with all the models.
Is there any way to create the PostgreSQL function using Rails and keep it in the models?
This part of your question:
I know we can create it manually in PostgreSQL, but the "magic" with Active Record is that the database can be recreated with all the models.
tells me that you're really looking for a way to integrate PostgreSQL functions with the normal Rails migration process and Rake tasks such as db:schema:load.
Adding and removing functions in migrations is easy:
def up
connection.execute(%q(
create or replace function ...
))
end
def down
connection.execute(%q(
drop function ...
))
end
You need to use separate up and down methods instead of a single change method because ActiveRecord will have no idea how to apply let alone reverse a function creation. And you use connection.execute to feed the raw function definition to PostgreSQL. You can also do this with a reversible inside change:
def change
reversible do |dir|
dir.up do
connection.execute(%q(
create or replace function ...
))
end
dir.down do
connection.execute(%q(
drop function ...
))
end
end
end
but I find that noisier than up and down.
However, schema.rb and the usual Rake tasks that work with schema.rb (such as db:schema:load and db:schema:dump) won't know what to do with PostgreSQL functions and other things that ActiveRecord doesn't understand. There is a way around this though, you can choose to use a structure.sql file instead of schema.rb by setting:
config.active_record.schema_format = :sql
in your config/application.rb file. After that, db:migrate will write a db/structure.sql file (which is just a raw SQL dump of your PostgreSQL database without your data) instead of db/schema.rb. You'll also use different Rake tasks for working with structure.sql:
db:structure:dump instead of db:schema:dump
db:structure:load instead of db:schema:load
Everything else should work the same.
This approach also lets you use other things in your database that ActiveRecord won't understand: CHECK constraints, triggers, non-simple-minded column defaults, ...
If your only requirement is creating them somewhere in your Rails app, this is possible through ActiveRecord::Base.connection.execute, which you can use to execute raw SQL queries.
stmt = 'CREATE FUNCTION...'
ActiveRecord::Base.connection.execute stmt
You would then call the function using ActiveRecord::Base.connection.execute as well (I'd imagine you'd have methods in your model to handle this).
Among my migrations, I have 3 that pertain to this question. In order, there is CreateEvents < ActiveRecord::Migration. The next down the line is CreateYears < ActiveRecord::Migration. Then the last is AddYearIdToEvents < ActiveRecord::Migration. The last one looks like this...
class AddYearIdToEvents < ActiveRecord::Migration
def change
add_column :events, :year_id, :integer, null: false, index: true
end
end
Now the problem is, whenever I try to drop the entire database (not near deploying to production), I get an obvious error of
ERROR: cannot drop table years because other objects depend on it
DETAIL: constraint events_year_id_fk on table events depends on table seasons
HINT: Use DROP ... CASCADE to drop the dependent objects too.
Now, I'm not a rails expert, but I believe I need to define a def down in that last migration. Rather than def change, do I need a def up and def down? The def down to drop this particular column.
If so, how can I do this. This migration is 20 migrations old. You can't just edit a migration like that can you? Do I add a new migration and just specify a def down? Or is the answer something completely different?
Have you tried rolling back your migrations? This undoes your migrations in reverse order, i.e. starting with the most recent migration.
$ rake db:rollback
If you have 20 migrations and you'd like to rollback all of them, you can use STEP=20
$ rake db:rollback STEP=20
Once you rollback past a migration that you'd like to change, you can change it.
In addition, you should be able to use rake db:drop, which drops the entire database rather than going through each migration in reverse order. If you'd like to drop the database, then recreate it and re-run all the migrations, you can run rake db:reset. Keep in mind this also runs rake db:seed if you have a db/seed.rb file.
And in response to your question of "Rather than def change, do I need a def up and def down?", the answer is no. Newer versions of Rails use def change since (among other reasons) it makes it easier to edit migrations once they are generated, i.e. if you spell a column name incorrectly when adding a column to a table.
I have an application that requires a sequence to be present in the database. I have a migration that does the following:
class CreateSequence < ActiveRecord::Migration
def self.up
execute "CREATE SEQUENCE sequence"
end
def self.down
execute "DROP SEQUENCE sequence"
end
end
This does not modify the schema.rb and thus breaks rake db:setup. How can I force the schema to include the sequence?
Note: The sequence exists after running rake db:migrate.
Rails Migrations because they aim toward a schema of tables and fields, instead of a complete database representation including stored procedures, functions, seed data.
When you run rake db:setup, this will create the db, load the schema and then load the seed data.
A few solutions for you to consider:
Choice 1: create your own rake task that does these migrations independent of the Rails Migration up/down. Rails Migrations are just normal classes, and you can make use of them however you like. For example:
rake db:create_sequence
Choice 2: run your specific migration after you load the schema like this:
rake db:setup
rake db:migrate:up VERSION=20080906120000
Choice 3: create your sequence as seed data, because it's essentially providing data (rather than altering the schema).
db/seeds.rb
Choice 4 and my personal preference: run the migrations up to a known good point, including your sequence, and save that blank database. Change rake db:setup to clone that blank database. This is a bit trickier and it sacrifices some capabilities - having all migrations be reversible, having migrations work on top of multiple database vendors, etc. In my experience these are fine tradeoffs. For example:
rake db:fresh #=> clones the blank database, which you store in version control
All the above suggestions are good. however, I think I found a better solution. basically in your development.rb put
config.active_record.schema_format = :sql
For more info see my answer to this issue -
rake test not copying development postgres db with sequences
Check out the pg_sequencer gem. It manages Pg sequences for you as you wish. The one flaw that I can see right now is that it doesn't play nicely with db/schema.rb -- Rails will generate a CREATE SEQUENCE for your tables with a serial field, and pg_sequencer will also generate a sequence itself. (Working to fix that.)