I have created a migration with custom postgres function creation:
class CreatePopularityPgFunctions < ActiveRecord::Migration[5.2]
def up
execute %{
CREATE OR REPLACE FUNCTION popularity(count integer, weight integer default 3) RETURNS integer AS $$
SELECT count * weight
$$ LANGUAGE SQL IMMUTABLE;
}
end
def down
execute 'drop function popularity(integer, integer) cascade'
end
end
and running rake db:migrate properly adds it to the schema. However, running rake db:reset seems to not create this function in PG for some reason. The function is not in the schema and trying to use it in SQL query results in error about missing function.
db:reset runs a db:drop db:setup. In db:setup creates a database schema. But schema.rb doesn't handle a custom functions or views.
From documentation:
db/schema.rb cannot express database specific items such as foreign key constraints, triggers, or stored procedures. While in a migration you can execute custom SQL statements, the schema dumper cannot reconstitute those statements from the database. If you are using features like this, then you should set the schema format to :sql.
You can use structure.sql instead. To use structure.sql:
This is set in config/application.rb by the config.active_record.schema_format setting, which may be either :sql or :ruby.
More details here.
Related
In Rails 7:
Migration:
class AddSavedSearchAPIKeyIndex < ActiveRecord::Migration[7.0]
def change
execute <<~SQL
ALTER TABLE saved_searches
ADD INDEX ecomm_analyze_api_key ((
CAST(configuration->>"$.ecomm_analyze_api_key" as CHAR(255)) COLLATE utf8mb4_bin
)) USING BTREE;
SQL
end
end
My schema ends up looking like this:
t.index "(cast(json_unquote(json_extract(`configuration`,_utf8mb4\\'$.ecomm_analyze_api_key\\')) as char(255) charset utf8mb4) collate utf8mb4_bin)", name: "ecomm_analyze_api_key"
I can't load this schema.rb without getting an error when I use:
RAILS_ENV=test rake db:test:prepare
Error message
ActiveRecord::StatementInvalid: Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use
I am able to fix this by manually editing the schema.rb but this will just break again on the next migration.
The Ruby schema dumper only actually understands a limited subset of SQL that can be created by the migrations DSL and is only actually useful until your app reaches the degree of complexity where you want to use database specific features.
Anything else that the Ruby schema dumper doesn't properly understand like this index will either be mangled or just lost in translation when it tries to parse the resulting SQL from dumping the database schema.
The solution is to use SQL schema dumps instead:
module YourApp
class Application < Rails::Application
# Add this line:
config.active_record.schema_format = :sql
end
end
This will dump the file structure.sql which should serve as the source of truth for the database schema. Check this file into version control and delete schema.rb to avoid confusion.
See:
Rails Guides : Schema Dumping and You
Pros and Cons of Using structure.sql in Your Ruby on Rails Application
SO...
I am adding history tables populated by triggers for auditing in my project via something like...
execute <<-SQL
CREATE OR REPLACE FUNCTION process_history_table() RETURNS TRIGGER AS $history_table$
BEGIN
IF (TG_OP = 'DELETE') THEN
INSERT INTO history_table VALUES (DEFAULT, 'D', now(), OLD.*);
RETURN OLD;
ELSIF (TG_OP = 'UPDATE') THEN
INSERT INTO history_table VALUES (DEFAULT, 'U', now(), NEW.*);
RETURN NEW;
ELSIF (TG_OP = 'INSERT') THEN
INSERT INTO history_table VALUES (DEFAULT, 'I', now(), NEW.*);
RETURN NEW;
END IF;
RETURN NULL; -- result is ignored since this is an AFTER trigger
END;
$history_table$ LANGUAGE plpgsql;
CREATE TRIGGER history_table
AFTER INSERT OR UPDATE OR DELETE ON table
FOR EACH ROW EXECUTE PROCEDURE process_history_table();
SQL
...and this will work for production and other environments. The problem is when someone runs bundle exec rake db:drop db:create db:schema:load db:migrate RAILS_ENV=test or something similar (most important is the db:schema:load portion), this will bypass trigger creation as triggers are not saved in the db/schema.rb file.
Perhaps the correct solution is to say that when using rails, developers should never run db:schema:load and always run db:migrate instead to ensure all migrations can continuously be re-run. However, we have not been operating that way for a long time and I believe it would be quite painful to do so as we may need to update several dozen or more migrations. Any thoughts on how I could incorporate triggers into my application incrementally and have the developer / test environments continue to be built / re-created the same way as today would be very helpful.
Thanks!
If you need or want database-specific features that ActiveRecord doesn't understand then you should switch to db/structure.sql for keeping track of your schema. db/structure.sql is pretty much a raw dump of your schema made using the database's native tools so it will contain triggers, CHECK constraints, indexes on function results, and everything else.
Switching is easy:
Update your config/application.rb to contain config.active_record.schema_format = :sql.
Do a rake db:structure:dump to get an initial db/structure.sql.
Delete db/schema.rb from your directory tree and revision control.
Add db/structure.sql to revision control.
Adjust your rake habits:
Use db:structure:dump instead of db:schema:dump
Use db:structure:load instead of db:schema:load
Everything else should work as usual (assuming, of course, that you're sane and using PostgreSQL for development, testing, and production).
With this change made, your triggers will be tracked in db/structure.sql and recreating the database won't lose them.
You can use gem 'paper_trail' which Tracks changes to your models, for auditing or versioning. Here is the Link
Isn't https://github.com/jenseng/hair_trigger what you need ?
It allows you to register your trigger in your project and then recreate, update them with a command.
NOTE: I always wanted to use it, but could never had the chance to do it in the end for various reasons so I can't vouch for the quality of the gem.
EDIT: No, they should always use rake db:schema:load for an existing DB
How could I create an index like this
CREATE INDEX index_companies_on_addresses_on_zipcode ON companies USING gin ((addresses -> 'zipcode'));
using the syntax of migrations?
I've created using the syntax below and it was created on database
execute "CREATE INDEX index_companies_on_addresses_on_zipcode ON companies USING gin ((addresses -> 'zipcode'));"
but when I saw schema.rb index wasn't there.
I've tried
add_index :companies, :addresses, using: :gin
but it create an index only on column addresses(is a jsonb) not on key zipcode.
AFAIK there's no way to make add_index understand that sort of indexing. Fear not, there are ways around this. You're using jsonb so there's no need to worry about database portability so you can switch from db/schema.rb to db/structure.sql for managing your schema. The db/structure.sql file is pretty much a native SQL dump of your database structure so it will contain everything that the database understands (CHECK constraints, advanced indexes, ...).
The process is quite simple:
Edit config/application.rb to set the schema format:
class Application < Rails::Application
#...
config.active_record.schema_format = :sql
#...
end
Run your migration that uses execute as normal. You'll want to use separate up and down methods in your migrations or reversible inside the usual change method because ActiveRecord won't know how to reverse execute some_raw_sql on its own.
This should leave you with a db/structure.sql file so add that to your revision control and delete db/schema.rb.
Use different rake tasks for working with structure.sql instead of schema.rb:
db:structure:dump instead of db:schema:dump
db:structure:load instead of db:schema:load
We are starting a project based on Ruby on Rails. We used to work with Perl and PostgreSQL functions, and with Rails and Active Record I've not seen how we are supposed to create functions in PostgreSQL and keep the record with Active Record and models.
I know we can create it manually in PostgreSQL, but the "magic" with Active Record is that the database can be recreated with all the models.
Is there any way to create the PostgreSQL function using Rails and keep it in the models?
This part of your question:
I know we can create it manually in PostgreSQL, but the "magic" with Active Record is that the database can be recreated with all the models.
tells me that you're really looking for a way to integrate PostgreSQL functions with the normal Rails migration process and Rake tasks such as db:schema:load.
Adding and removing functions in migrations is easy:
def up
connection.execute(%q(
create or replace function ...
))
end
def down
connection.execute(%q(
drop function ...
))
end
You need to use separate up and down methods instead of a single change method because ActiveRecord will have no idea how to apply let alone reverse a function creation. And you use connection.execute to feed the raw function definition to PostgreSQL. You can also do this with a reversible inside change:
def change
reversible do |dir|
dir.up do
connection.execute(%q(
create or replace function ...
))
end
dir.down do
connection.execute(%q(
drop function ...
))
end
end
end
but I find that noisier than up and down.
However, schema.rb and the usual Rake tasks that work with schema.rb (such as db:schema:load and db:schema:dump) won't know what to do with PostgreSQL functions and other things that ActiveRecord doesn't understand. There is a way around this though, you can choose to use a structure.sql file instead of schema.rb by setting:
config.active_record.schema_format = :sql
in your config/application.rb file. After that, db:migrate will write a db/structure.sql file (which is just a raw SQL dump of your PostgreSQL database without your data) instead of db/schema.rb. You'll also use different Rake tasks for working with structure.sql:
db:structure:dump instead of db:schema:dump
db:structure:load instead of db:schema:load
Everything else should work the same.
This approach also lets you use other things in your database that ActiveRecord won't understand: CHECK constraints, triggers, non-simple-minded column defaults, ...
If your only requirement is creating them somewhere in your Rails app, this is possible through ActiveRecord::Base.connection.execute, which you can use to execute raw SQL queries.
stmt = 'CREATE FUNCTION...'
ActiveRecord::Base.connection.execute stmt
You would then call the function using ActiveRecord::Base.connection.execute as well (I'd imagine you'd have methods in your model to handle this).
I have an application that requires a sequence to be present in the database. I have a migration that does the following:
class CreateSequence < ActiveRecord::Migration
def self.up
execute "CREATE SEQUENCE sequence"
end
def self.down
execute "DROP SEQUENCE sequence"
end
end
This does not modify the schema.rb and thus breaks rake db:setup. How can I force the schema to include the sequence?
Note: The sequence exists after running rake db:migrate.
Rails Migrations because they aim toward a schema of tables and fields, instead of a complete database representation including stored procedures, functions, seed data.
When you run rake db:setup, this will create the db, load the schema and then load the seed data.
A few solutions for you to consider:
Choice 1: create your own rake task that does these migrations independent of the Rails Migration up/down. Rails Migrations are just normal classes, and you can make use of them however you like. For example:
rake db:create_sequence
Choice 2: run your specific migration after you load the schema like this:
rake db:setup
rake db:migrate:up VERSION=20080906120000
Choice 3: create your sequence as seed data, because it's essentially providing data (rather than altering the schema).
db/seeds.rb
Choice 4 and my personal preference: run the migrations up to a known good point, including your sequence, and save that blank database. Change rake db:setup to clone that blank database. This is a bit trickier and it sacrifices some capabilities - having all migrations be reversible, having migrations work on top of multiple database vendors, etc. In my experience these are fine tradeoffs. For example:
rake db:fresh #=> clones the blank database, which you store in version control
All the above suggestions are good. however, I think I found a better solution. basically in your development.rb put
config.active_record.schema_format = :sql
For more info see my answer to this issue -
rake test not copying development postgres db with sequences
Check out the pg_sequencer gem. It manages Pg sequences for you as you wish. The one flaw that I can see right now is that it doesn't play nicely with db/schema.rb -- Rails will generate a CREATE SEQUENCE for your tables with a serial field, and pg_sequencer will also generate a sequence itself. (Working to fix that.)