I am completely new to Postgresql, Postgis, and SQL and have a little bit of Rails knowledge. I am trying to get an existing app going on my local and am currently receiving the following messages when running a rake task to set up my database.
Blane-Cordess-MacBook:ajungo blanecordes$ rake postgres:create_postgis_template --trace
(in /Users/blanecordes/ajungo)
** Invoke postgres:create_postgis_template (first_time)
** Execute postgres:create_postgis_template
Enter your postgres username: blane
Password for user blane:
psql (9.2.1, server 9.1.4)
WARNING: psql version 9.2, server version 9.1.
Some psql features might not work.
You are now connected to database "template1" as user "blane".
psql:assets/sql/postgis_template_osx.sql:2: ERROR: permission denied to create database
psql:assets/sql/postgis_template_osx.sql:6: ERROR: permission denied for relation pg_database
psql:assets/sql/postgis_template_osx.sql:7: \connect: FATAL: database "template_postgis" does not exist
My postgis_template_osx.sql file is the following:
\c template1
CREATE DATABASE template_postgis WITH template = template1;
-- set the 'datistemplate' record in the 'pg_database' table for
-- 'template_postgis' to TRUE indicating its a template
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
\c template_postgis
CREATE LANGUAGE plpgsql ;
\i /usr/local/Cellar/postgis/2.0.1/share/postgis/postgis.sql;
\i /usr/local/Cellar/postgis/2.0.1/share/postgis/spatial_ref_sys.sql;
-- 1.5.2
-- in a production environment you may want to
-- give role based permissions, but granting all for now
GRANT ALL ON geometry_columns TO PUBLIC;
GRANT ALL ON spatial_ref_sys TO PUBLIC;
-- vacuum freeze: it will guarantee that all rows in the database are
-- "frozen" and will not be subject to transaction ID wraparound
-- problems.
VACUUM FREEZE;
The PostgreSQL ROLE blane does not have permission to create databases. Either ALTER blane and add the CREATEDB option or use an existing administrator role that has that capability.
postgres=# ALTER ROLE blane WITH CREATEDB;
Related
I have a rails app with a PostGIS database running in docker containners. I'm using apartment for multitenancy and activerecord-postgis-adapter to access the PostGIS geospatial database features from ActiveRecord. I have PostGIS installed in shared_extentions schema as the apartment docs suggest. When I try to configure a datastore in geoserver I get the following error:
Error creating data store, check the parameters. Error message: Unable to obtain connection: ERROR: function postgis_lib_version() does not exist Hint: No function matches the given name and argument types. You might need to add explicit type casts. Position: 8
Geoserver datastore connection parameters:
host: app-db
port: 5432
database: app_development
schema: shared_extensions
user:
passwd:
database.yml:
default: &default
adapter: postgis
postgis_schema: shared_extensions
schema_search_path: public, shared_extensions
encoding: unicode
database: app_development
host: app-db
username:
password:
lib/tasks/db_enhancements.rake:
namespace :db do
desc 'Also create shared_extensions Schema'
task :extensions => :environment do
# Create Schema
ActiveRecord::Base.connection.execute 'CREATE SCHEMA IF NOT EXISTS shared_extensions;'
# Enable Hstore
ActiveRecord::Base.connection.execute 'CREATE EXTENSION IF NOT EXISTS HSTORE SCHEMA shared_extensions;'
# Enable Postgis
ActiveRecord::Base.connection.execute 'CREATE EXTENSION IF NOT EXISTS postgis SCHEMA shared_extensions;'
# Enable UUID-OSSP
ActiveRecord::Base.connection.execute 'CREATE EXTENSION IF NOT EXISTS "uuid-ossp" SCHEMA shared_extensions;'
# Grant usage to public
ActiveRecord::Base.connection.execute 'GRANT usage ON SCHEMA shared_extensions to public;'
end
end
I've also tried it on pgAdmin with SELECT shared_extensions.postgis_version() and it works fine:
2.3 USE_GEOS=1 USE_PROJ=1 USE_STATS=1
But of course when I run SELECT postgis_version() I get:
ERROR: function postgis_version() does not exist
LÍNEA 1: SELECT postgis_version()
^
SUGERENCIA: No function matches the given name and argument types. You might need to add explicit type casts.
SQL state: 42883
Hint: No function matches the given name and argument types. You might need to add explicit type casts.
Character: 8
GeoServer expects PostGIS to be installed in the public schema does not prepend shared_extensions to it's postgis functions. If you want to do this then you need to add shared_extensions to the search path so that GeoServer can find the functions it will need.
ALTER DATABASE mydb SET 'search_path' = public,shared_extension;
See this note for details on how to move postgis to a different schema.
I am trying to set a statement_timeout. I tried both setting in database.yml file like this
variables:
statement_timeout: 1000
And this
ActiveRecord::Base.connection.execute("SET statement_timeout = 1000")
Tested with
ActiveRecord::Base.connection.execute("select pg_sleep(4)")
And they both don't have any effect.
I am running postgres 10 in my local and the statement_timeouts works just expected. But on my server that is running postgres 9.4.4, it simply doesn't do anything.
I've check Postgres' doc for 9.4 and statement_timeout is available. Anyone can shed some light?
I wasn't able to replicate this issue locally using: Postgresql 9.4.26. But it might be useful to share what I've tried and some thoughts around the server issue.
Here is what I've tried (a useful bit might be a query to verify the PG version from rails):
# Confirming I am executing against 9.4.x PG:
irb(main):002:0> ActiveRecord::Base.connection.execute("select version()")
(10.8ms) select version()
=> #<PG::Result:0x00007ff74782e060 status=PGRES_TUPLES_OK ntuples=1 nfields=1 cmd_tuples=1>
irb(main):003:0> _.first
=> {"version"=>"PostgreSQL 9.4.26 on x86_64-apple-darwin18.7.0, compiled by Apple clang version 11.0.0 (clang-1100.0.33.17), 64-bit"}
# Set timeout:
irb(main):004:0> ActiveRecord::Base.connection.execute("SET statement_timeout = 1000")
(0.4ms) SET statement_timeout = 1000
=> #<PG::Result:0x00007ff7720a3d88 status=PGRES_COMMAND_OK ntuples=0 nfields=0 cmd_tuples=0>
# Confirm it works - it is ~1s and also stacktrace is pretty explicit about it:
irb(main):005:0> ActiveRecord::Base.connection.execute("select pg_sleep(4)")
(1071.2ms) select pg_sleep(4)
.... (stacktrace hidden)
ActiveRecord::StatementInvalid (PG::QueryCanceled: ERROR: canceling statement due to statement timeout)
: select pg_sleep(4)
Here is what to try
Since the issue occurs on server only and since statement_timeout works on other minor version and locally, one thing that comes to mind is the lack of privileges to update statement_timeout from where it is attempted. Perhaps rails pg login used to make db connection is not allowed to update that setting.
The best would be to verify that either via rails console on a server:
irb(main):004:0> ActiveRecord::Base.connection.execute("SET statement_timeout = 1000")
irb(main):004:0> irb(main):003:0> ActiveRecord::Base.connection.execute("show statement_timeout").first
(0.2ms) show statement_timeout
=> {"statement_timeout"=>"1s"}
Or, it can be checked directly via psql console (some deployments allow this too):
psql myserveruser # if this was heroku's pg: heroku pg:psql
postgres=# set statement_timeout = 1000;
SET
postgres=# select pg_sleep(4);
ERROR: canceling statement due to statement timeout
Time: 1068.067 ms (00:01.068)
Other thing to keep in mind (taken from https://dba.stackexchange.com/a/83035/90903):
The way statement_timeout works, the time starts counting when the
server receives a new command from the client...
And if a function does SET statement_timeout = 100; it will have an effect only starting at
the next command from the client.
I would like to perform a regular backup of a PostgreSQL database, my current intention is to use the Backup and Whenever gems. I am relatively new to Rails and Postgres, so there is every chance I am making a very simple mistake...
I am currently trying to setup the process on my development machine (MAC), but keep getting an error when trying to connect to the database.
In the terminal window, I have performed the following to check the details of my database and connection:
psql -d my_db_name
my_db_name=# \conninfo
You are connected to database "my_db_name" as user "my_MAC_username" via socket in "/tmp" at port "5432".
\q
I have also manually created a backup of the database:
pg_dump -U my_MAC_username -p 5432 my_db_name > name_of_backup_file
However, when I try to repeat this within db_backup.rb (created by the Backup gem) I get the following error:
[2018/10/03 19:59:00][error] Model::Error: Backup for Description for db_backup (db_backup) Failed!
--- Wrapped Exception ---
Database::PostgreSQL::Error: Dump Failed!
Pipeline STDERR Messages:
(Note: may be interleaved if multiple commands returned error messages)
pg_dump: [archiver (db)] connection to database "my_db_name" failed: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/pg.sock/.s.PGSQL.5432"?
The following system errors were returned:
Errno::EPERM: Operation not permitted - 'pg_dump' returned exit code: 1
The contents of my db_backup.rb:
Model.new(:db_backup, 'Description for db_backup') do
##
# PostgreSQL [Database]
#
database PostgreSQL do |db|
# To dump all databases, set `db.name = :all` (or leave blank)
db.name = "my_db_name"
db.username = "my_MAC_username"
#db.password = ""
db.host = "localhost"
db.port = 5432
db.socket = "/tmp/pg.sock"
# When dumping all databases, `skip_tables` and `only_tables` are ignored.
# db.skip_tables = ["skip", "these", "tables"]
# db.only_tables = ["only", "these", "tables"]
# db.additional_options = ["-xc", "-E=utf8"]
end
end
Please could you suggest what I need to do to resolve this issue and perform the same backup through the db_backup.rb code
In case someone else gets stuck in a similar situation, the key to unlocking this problem was the lines:
psql -d my_db_name
my_db_name=# \conninfo
I realised that I needed to change db.socket = "/tmp/pg.sock" to db.socket = "/tmp", which seems to have resolved the issue.
However, I don't understand why the path on my computer differs to the default as I didn't do anything to customise the installation of any gems or the Postgres App
My project uses a few custom PostgreSQL stored functions for some features that would be a pain in raw SQL or ActiveRecord. Every now and then I will run the RSpec test suite, and find that all my stored functions have been blown away. Re-running the migrations to create them fixes the problem, but "rake db:structure:load" does NOT.
I am deeply confused. I never drop either the dev or test database unless this happens, but my functions are like Schrodinger's PL/pgSQL. I am REALLY hoping this never happens in production.
Here is an example of a failing test and my attempts to fix it:
ActiveRecord::StatementInvalid:
PG::UndefinedFunction: ERROR: function round_half_down(numeric) does not exist
# Damn. We have to drop the database so we can reload structure.sql:
$ RAILS_ENV=test rake db:drop
$ RAILS_ENV=test rake db:create
# load structure.sql instead of schema.rb:
$ RAILS_ENV=test rake db:structure:load
# Not fixed:
ActiveRecord::StatementInvalid:
PG::UndefinedFunction: ERROR: function round_half_down(numeric) does not exist
$ RAILS_ENV=test rake db:migrate:redo VERSION=20160421184708
== 20171002190107 CreateRoundHalfDownFunction: reverting ======================
-- execute("DROP FUNCTION IF EXISTS round_half_down(numeric)")
-> 0.0004s
== 20171002190107 CreateRoundHalfDownFunction: reverted (0.0005s) =============
== 20171002190107 CreateRoundHalfDownFunction: migrating ======================
-- execute("CREATE OR REPLACE FUNCTION ROUND_HALF_DOWN(NUMERIC)\n RETURNS NUMERIC LANGUAGE SQL AS\n$FUNC$\n SELECT CASE WHEN ($1%1) < 0.6 THEN FLOOR($1) ELSE CEIL($1) END;\n$FUNC$\n")
-> 0.0014s
== 20171002190107 CreateRoundHalfDownFunction: migrated (0.0014s) =============
Now it is fixed!
Yes, I verified that the function is present in structure.sql:
--
-- Name: round_half_down(numeric); Type: FUNCTION; Schema: public; Owner: -
--
CREATE FUNCTION round_half_down(numeric) RETURNS numeric
LANGUAGE sql
AS $_$
SELECT CASE WHEN ($1%1) < 0.6 THEN FLOOR($1) ELSE CEIL($1) END;
$_$;
For the record, this stopped happening to me with newer versions of the pg gem and PostgreSQL itself. I also added config.active_record.schema_format = :sql to application.rb, because my application makes heavy use of Postgres-specific features, and a number of stored functions.
I am getting the following uuid error while running a rails app with postgres as backend. Can someone help me out with which dependency is needed.
[root#localhost webapp]# rake db:migrate
(in /root/mysite/webapp)
== CreateContributors: migrating =============================================
-- create_table(:contributors, {:id=>false})
-> 0.0121s
-- execute("alter table contributors add primary key (id)")
NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "contributors_pkey" for table "contributors"
-> 0.0797s
-- execute("alter table contributors alter column id set default uuid_generate_v1()::varchar")
rake aborted!
An error has occurred, this and all later migrations canceled:
PGError: ERROR: function uuid_generate_v1() does not exist
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
: alter table contributors alter column id set default uuid_generate_v1()::varchar
The uuid_generate_v1() function is part of the uuid-ossp package and you have to install that into PostgreSQL before you can use it. You should have a file called uuid-ossp.sql in your PostgreSQL contrib directory. You can install the package with:
$ psql -d your_database < /the/path/to/uuid-ossp.sql
You'll probably want to run that as the database super user.