ERROR: must be owner of language plpgsql - ruby-on-rails

I'm using PostgreSQL v9.0.1 with Rails (and it's deps) # v2.3.8, owing to the use of the fulltext capability of postgres, I have a table which is defined as:
CREATE TABLE affiliate_products (
id integer NOT NULL,
name character varying(255),
model character varying(255),
description text,
price numeric(9,2),
created_at timestamp without time zone,
updated_at timestamp without time zone,
textsearch_vector tsvector,
);
Note the last line, this ensures that active record isn't able to process it with the standard schema dumper, so I have to set config.active_record.schema_format = :sql in ./config/environment.rb; and use rake db:test:clone_structure instead of rake db:test:clone.
None of this is too remarkable, only inconvenient - however rake db:test:clone_structure fails with the error:
ERROR: must be owner of language plpgsql
Because of line #16 in my resulting ./db/development_schema.sql:
CREATE OR REPLACE PROCEDURAL LANGUAGE plpgsql;
Under PostgreSQL v9.0+ the language plpsql is installed by the superuser, to the initial template, which is then available to the newly created schema.
I cannot run tests on this project without resolving this, and even editing ./db/development_schema.sql manually is futile as it is regenerated every time I run rake db:test:clone_structure (and ignored by rake db:test:clone).
I hope someone can shed some light on this?
Note: I have used both the pg 0.9.0 adapter gem, and the postgres gem at version 0.7.9.2008.01.28 - both display identical behaviour.
My teammates run PostgreSQL v8.4 where the language installation is a manual step.

I had the same problem. I fixed my template with the commands below
psql template1
template1=# alter role my_user_name with superuser;
read more at http://gilesbowkett.blogspot.com/2011/07/error-must-be-owner-of-language-plpgsql.html

For new readers, I read this older post after having run into this error in one of my own projects. I strongly feel that giving the app's PostgreSQL a superuser role is a terrible idea and changing the template is not ideal either. Since the referenced PSQL commands that are added by db:structure:dump are not needed by the Rails app's database, I have written a custom rake task that comments out the problematic lines in structure.sql. I have shared that code publicly on Github as a Gist at https://gist.github.com/rietta/7898366.

The solution was as follows:
On my installation, there are standard templates template0 and template1 - at least as I understand it postgres will look for the highest numbered templateN when creating a new database, unless the template is specified.
In this instance, as template0 included plpgsql, so did template1… the idea being that you will customise template1 to suite your site specific default needs, and in the case that you blow everything up, you would restore template1 from template0.
As my site specific requirement was to install plpgsql as part of the automated build of my web application (a step we had to keep to maintain 8.4 compatibility) - the solution was easy: remove plpgsql from template1 and the warning/error went away.
In the case that the site-specific defaults would change, and we should need to go back to the default behaviour, we would simply remove template1 and recreate it (which would use template0)

I encountered this error while attempting to do RAILS_ENV=development bundle exec rake db:reset. I was able to accomplish the same thing (for my purposes) by doing RAILS_ENV=development bundle exec rake db:drop db:create db:migrate instead.

I just filter the plpgsql extension statements from the structure.sql file post-dump:
# lib/tasks/db.rake
namespace :db do
desc "Fix 'ERROR: must be owner of extension plpgsql' complaints from Postgresql"
task :fix_psql_dump do |task|
filename = ENV['DB_STRUCTURE'] || File.join(ActiveRecord::Tasks::DatabaseTasks.db_dir, "structure.sql")
sql = File.read(filename)
sql.sub!(/(CREATE EXTENSION IF NOT EXISTS plpgsql)/, '-- \1')
sql.sub!(/(COMMENT ON EXTENSION plpgsql)/, '-- \1')
File.open(filename, 'w') do |f|
f.write(sql)
end
task.reenable
end
end
Rake::Task["db:structure:dump"].enhance do
Rake::Task["db:fix_psql_dump"].invoke
end

Related

ERROR: extension "btree_gist" must be installed in schema "heroku_ext"

Heroku made a change to the way postgressql extensions gets installed
This is screwing up new rails review apps in heroku with the following error.
ERROR: extension "btree_gist" must be installed in schema "heroku_ext"
This is screwing up things as I need to drop existing extensions and re-enable with heroku_ext schema. I use bin/rails db:structure:load which is before running a migration.
Also the structure.sql is going to diverge as heroku add the schema in review app and we need to run the creation manually in local dev machine.
Does anybody came across this issue?
I have developed a monkey-patch solution that is, well, definitely a hack, but better than pre-dating migrations, or deleting lines from schema.rb.
Put this in config/initializers. I called mine _enable_extension_hack.rb to make sure it gets loaded early.
module EnableExtensionHerokuMonkeypatch
# Earl was here
def self.apply_patch!
adapter_const = begin
Kernel.const_get('ActiveRecord::ConnectionAdapters::PostgreSQLAdapter')
rescue NameError => ne
puts "#{self.name} -- #{ne}"
end
if adapter_const
patched_method = adapter_const.instance_method(:enable_extension)
# Only patch this method if it's method signature matches what we're expecting
if 1 == patched_method&.arity
adapter_const.prepend InstanceMethods
end
end
end
module InstanceMethods
def enable_extension(name)
name_override = name
if schema_exists?('heroku_ext')
puts "enable_extension -- Adding SCHEMA heroku_ext"
name_override = "#{name}\" SCHEMA heroku_ext -- Ignore trailing double quote"
end
super name_override
end
end
end
EnableExtensionHerokuMonkeypatch.apply_patch!
What does it do? It monkey-patches the enable_extension call that's causing the problem in db/schema.rb. Specifically, if it finds that there is a schema called "heroku_ext", it will decorate the parameter given to enable_extension so that the SQL that gets executed specifies the "heroku_ext" schema. Like this:
CREATE EXTENSION IF NOT EXISTS "pg_stat_statements" SCHEMA heroku_ext -- Ignore trailing double quote"
Without this hack, the generated SQL looks like this:
CREATE EXTENSION IF NOT EXISTS "pg_stat_statements"
That's cleaner way of patching schema.rb
# config/initializers/patch_enable_extension.rb
require 'active_record/connection_adapters/postgresql_adapter'
# NOTE: patch for https://devcenter.heroku.com/changelog-items/2446
module EnableExtensionHerokuPatch
def enable_extension(name, **)
return super unless schema_exists?("heroku_ext")
exec_query("CREATE EXTENSION IF NOT EXISTS \"#{name}\" SCHEMA heroku_ext").tap { reload_type_map }
end
end
module ActiveRecord
module ConnectionAdapters
class PostgreSQLAdapter
prepend EnableExtensionHerokuPatch
end
end
end
EDIT: Here’s the official changelog as a reference https://devcenter.heroku.com/changelog-items/2446
You'll need to do the unthinkable because of the recent heroku changes and modify past migration files for your review apps to work with the new Heroku system for extensions.
Add a predated, or at the top of the first migration file connection.execute 'CREATE SCHEMA IF NOT EXISTS heroku_ext'
Potentially also add to the database.yml a schema_search_path that includes heroku_ext (or set it to public,heroku_ext if you hadn't customized it)
Grep all enable_extension('extension_name')
Replace them all by a connection.execute('CREATE EXTENSION IF NOT EXISTS "extension_name" WITH SCHEMA "heroku_ext")
Pray that is enough to fix the problem
After making those changes we still had to contact heroku support because of, in order:
a pgaudit stack is not empty error
The fix here was to run a maintenance twice (because the postgres add-on which was scheduled for maintenance pre-dated the schema/extension changes change)
a ERROR: function pg_stat_statements_reset(oid, oid, bigint) does not exist error
The fix here was a manual intervention from heroku on the databases. It was caused by heroku trying to run a pg_stat_statements_reset each time a schema is created.
This hack seems to be working for me. The following script runs in the postdeploy step in app.json.
#!/bin/bash -xue
# Create extensions in the schema where Heroku requires them to be created
# The plpgsql extension has already been created before this script is run
heroku pg:psql -a $HEROKU_APP_NAME -c 'create extension if not exists citext schema heroku_ext'
heroku pg:psql -a $HEROKU_APP_NAME -c 'create extension if not exists pg_stat_statements schema heroku_ext'
# Remove enable_extension statements from schema.rb before loading it, since
# even 'create extension if not exists' fails when the schema is not heroku_ext
mv db/schema.rb{,.orig}
grep -v enable_extension db/schema.rb.orig > db/schema.rb
rails db:schema:load
As of Tuesday 16th August 2022, we've found that a patch no longer seems to be needed and enable_extension appears to be working seamlessly as normal without the need to explicitly specify a schema.
When I mentioned this in our ongoing thread with Heroku Support, they said that some of their fixes around the heroku_ext change may have been released at the time of resolving https://status.heroku.com/incidents/2450, but that there are still further fixes in development and yet to be released - so my guess is that this particular issue is one on those that has been fixed!
The answer supplied by dzirtusss got me most of the way there, the last thing I had to do to get this working for my Rails application was to edit the search paths in config/database.yml to include heroku_ext. This is what I appended to my default configuration.
# config/database.yml
default:
schema_search_path: public,heroku_ext
After adding the initialization script and the database.yml edit, I was able to successfully run a heroku pg:copy from one application to another with no errors and no unexpected behavior.
Heroku has introduced an --extensions flag recently, which helps as a workaround. See this FAQ.
For example, if you're using the btree_gist extension, you can use heroku run pg:reset --extensions 'btree_gist' --app YOUR_APP_NAME before running your migrations, they should then work without any changes in your codebase.
Use the Heroku datastore durability tool to create a backup on the
source database or heroku pg:backups:capture -a <SOURCE_APP>.
Determine which pg extensions the database uses (can check from psql
with \dx)
create a comma-separated string of the extensions (e.g.:fuzzystrmatch,pg_stat_statements,pg_trgm,pgcrypto,plpgsql,unaccent,uuid-ossp')
Make sure your Heroku CLI is updated to at least version 7.63.0 (use
heroku update to update)
Run this:
heroku pg:backups:restore $(heroku pg:backups public-url -a <SOURCE_APP>) DATABASE_URL --extensions '<EXTENSIONS>' -a <TARGET_APP>
Reset dynos on the TARGET_APP

Can Rails schema table be outside the database?

We have a legacy PostgreSQL database that is perfectly accessible from a Rails app except that when the vendor provides an update one of the consistency checks they perform against the database fails because Rails has built a "new" table there, the schema migrations table. Is it possible to direct creation of this table elsewhere? Or is it possible to use the schema cache in Rails 4 to effect this? The release notes section 3.3 on General things says "Schema cache dump (commit) - To improve Rails boot time, instead of loading the schema directly from the database, load the schema from a dump file."
I found an old blog post about this last time I tried it here. Copying the relevant parts:
To make a dump of your schema, execute the following rake task:
RAILS_ENV=production bundle exec rake db:schema:cache:dump
This will generate a file db/schema_cache.dump, that Rails will use to load the internal state of the SchemaCache instance.
To disable the schema cache dump, add the following to your config/production.rb file:
config.active_record.use_schema_cache_dump = false
If you would like to clear the schema cache, execute:
RAILS_ENV=production bundle exec rake db:schema:cache:clear

Ruby on Rails: how to export a DB to an sql-file?

I would like to export the database into a single sql-file. I have taken on a project built by a developer who didn't seem to back it up before leaving the project. I need to take a copy of the database structure so that i can work on it in my local test-environment basically.
I have noticed the rake db:structure:dump -command, wich in the terminal gives me the following respons: rake aborted!
Don't know how to build task 'dataset:export'
Is there a gem I need to install or something, or how do you simply export the database into a single file?
You can do the following:
a) make a rails version of the schema with rake db:schema.dump
The generated file is db/schema.rb
you can load then it locally with db:schema:load
b) use the mysql tools to get real sql dump. See for example
http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html
c) try to debug why the rake task fails. a problem may be that you need to set RAILS_ENV first, for example on the Unix-bash with
RAILS_ENV=production; export RAILS_ENV
But if you do not find a solution fast, try a) or b), because having the database structure is important before changing code, updating gems, etc.

rake db:schema:dump show no fields with rails 3.2.3 and SQL Server 2008

I am migrating an app from rails 2.3.8 to 3.2.3. The 'rake db:schema:dump' works fine in rails 2.3
but generates only the tables names with no column names in rails 3.2.
Even for the app to connect successfully through the console, I had to change config/application.rb to include
ActiveRecord::Base.table_name_prefix = 'dbo.'
Do I need to do something different for the rake task to pick up these prefixes? Or is something else causing the missing column names problem?
Further clarification:
I am looking for rake db:schema:dump because the programmers on site stopped using migrations and started making changes to the db directly. Now I am trying to restart using migrations. The first step recommended in the process is to use the schema dump as the starting point. Also, (and I am not sure) it is needed for the tests to rebuild the test db from the development db.
Short answer:
db:schema:dump isn't the right thing to use, but you can add a few lines of code to your Rakefile to get the outcome you want.
Longer answer:
The scuttlebutt is that the task db:schema:dump is actually not supposed to dump anything more than the structure. (I know, it's a misnomer.) It's analagous to db:structure:dump, except that the one gives you an .rb file, and the other gives you a .sql file.
You can create your own dumping rake task by appending the following code to your Rakefile:
For SQL 2008
task :mydump do
ActiveRecord::Base.connection.execute(
"dbcc traceon(2544, -1) \n go \n dbcc traceon(2546, -1) \n go \n dbcc stackdump"
)
end
Using the SQL server itself to create the dump (which is what the forgoing code does) limits you because the dump will always go to your log directory; you cannot specify otherwise.
If you use SqlDumper or some other utility, you will have more freedom. You can call such a utility from your rake task by executing it as through from the command line, using the system method. (See the example for MySQL below, which uses the mysqldump utility.)
(I have not tested the forgoing code, not having an installation of SQL 2008, myself, but the raw SQL code for creating a dump from with the SQL server is explained on this blog.)
Running the rake task
Then on your command line, call rake mydump or rake mydump RAILS_ENV=production.
For MySQL
You could do something similar for MySQL with the following:
task :mydump do
config = Rails.configuration.database_configuration[Rails.env]
system "mysqldump -h #{config["host"]} -u #{config["username"]} -p#{config["password"]} #{config["database"]} > db/dump.sql"
end

How do I configure a heroku app's database that uses postgresql using the schema?

I've been using mysql forever. never really needed anything fancier. But I'm using heroku a lot and while I'm working, I like free search, so I'm using the acts_as_tsearch plugin. If you go to the git repository, it tells you:
* Preparing your PostgreSQL database
Add a text search configuration 'default':
CREATE TEXT SEARCH CONFIGURATION public.default ( COPY = pg_catalog.english )
So guess what? I
changed from mysql to postgresql in my rails config
ran that "CREATE TEXT" code in the sql pain of pgAdmin (a gui for postgres)
noticed that now my development DB has something called an "FTS configuration"
tried the search functionality and it works GREAT
But I'm having trouble getting that configuration to show up in the schema. When I did rake db:dump it doesn't make it in there. I know I can add this line to the schema.rb:
execute 'CREATE TEXT SEARCH CONFIGURATION public.default ( COPY = pg_catalog.english )'
and that works, but how can I get that configuration into the schema without my having to hand-add it? Can I create a file that is also loaded after schema.rb when someone types rake db:load?
And for the postgres people, a question: What does that CREATE TEXT SEARCH CONFIGURATION... do?
Why don't you try adding it to a migration file and rake that against the heroku db?

Resources