I'm trying to migrate users from one system to another. Each system has its own database and different classes.
My plan is to connect to one database, read some info from one database via SQL commands:
ActiveRecord::Base.connection.execute(sql_command)
do something with the data and then write some results on the new database using normal models.
I plan on doing that inside Sidekiq job but I'm trying to do some testing using a Rails console on Heroku.
This should be pretty straightforward, but this proves ridiculously difficult.
When I launched a Rails console on Heroku. I'm connecting to DATABASE_URL, which is ok, but when I try to connect to the old database and execute a command, like this:
ActiveRecord::Base.establish_connection(adapter: "postgresql", encoding: "unicode", pool: 10, url: "postgres://...info...")
ActiveRecord::Base.connection.execute("select count(*) from users")
I end up with:
PG::ConnectionBad (could not connect to server: No such file or directory)
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I can, however, connect to this old database by launching my rails console on heroku using DATABASE_URL as the env variable:
$ heroku run 'DATABASE_URL=postgres://...info... rails console' -a console' -a app
BUT I don't know how to switch back to my new database so I can update things.
How does one switch databases at run time when using rails console on heroku?
Why try to runtime switch the database? Why not have both connected at the same time and specify at the model level which database they read/write from? Rails supports connecting multiple databases and specifying in individual models what database connection to use: https://guides.rubyonrails.org/active_record_multiple_databases.html
The problem was that using url: didn't work and all parameters needed to be specified.
config = {"adapter"=>"postgresql", "encoding"=>"unicode", "pool"=>10, "username"=>"u...", "password"=>"p...", "port"=>5432, "database"=>"d...", "host"=>"ec2..."}
If you go for a 3tier database yml, you can use this:
config = ActiveRecord::Base.configurations["production"]["seconddb"]
Then, you can use establish_connection
ActiveRecord::Base.establish_connection(config)
ActiveRecord::Base.connection.execute("select count(*) from users")
Once I started specifying username, password, port, database and host, it all worked like a charm.
To work with both databases at the same time, a good way is to create a class
class OtherDB < ActiveRecord::Base
establish_connection(ActiveRecord::Base.configurations["production"]["seconddb"])
end
Then you can call things that way
OtherDB.table_name = "table_name"
OtherDB.first
ref (Establish a connection to another database only in a block?)
And to run SQL commands:
OtherDB.connection.execute("select count(*) from users")
Used the
neo4j-admin import -mode database -database dbname -from location
Got the message :Imported data from <> to <>
I could see the databse created into the neo4j-enterprise-3.0.3\data\databases directory
Then i start the neo4j server. I dont see error log in debug.log or neo4j.log
But when i try to see the contents of the in the browseron localhost 7474 i see empty db with no labels or anything..
Database Information
Node labels
No labels in database Relationship types
No relationships in database Property keys No property keys in
database Database Version:
3.0.3 Name: neo4j-hpp-db-default_CTDCA_HKGDT Size:
88.24 KiB
Why dont I see the data?
Stop the 2.2.9 instance gracefully (aka make sure there's no unclean shutdown).
Copy over your datastore from neo4j-2.2.9/data/graph.db to neo4j-3.0.3/data/databases/graph.db (not the additional databases part).
Fire up the database on 3.0.3 and keep an eye on the files in logs.
I'm trying to create the database in Rails. In Postgres I see the development and test database, however, I'm getting a permissions error. I've tried to follow this link, didn't work for me.
Error: PG::InsufficientPrivilege: ERROR: permission denied for relation schema_migrations : SELECT "schema_migrations".* FROM "schema_migrations"
Rails: permission denied for relation schema_migrations
default: &default
adapter: postgresql
encoding: unicode
pool: 5
host: localhost
username: root
password:
development:
<<: *default
database: svp-chicago_development
I log into postgres and did these commands.
psql postgres
CREATE USER root
CREATE DATABASE svp-chicago_development
GRANT ALL PRIVILEGES ON DATABASE svp-chicago_development to root
ALTER DATABASE svp-chicago_development OWNER TO root
When I do \list I see the database is there.
I had same issue and I solved by adding "Superuser" to the role.
First, list users and their privileges. If you followed above commands, root user does not have "Superuser" Attributes.
postgres=# \du
List of roles
Role name | Attributes | Member of
-----------+------------------------------------------------------------+-----------
other | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
root | | {}
Next, upgrade root to be a "Superuser".
postgres=# ALTER USER root WITH SUPERUSER;
ALTER ROLE
Again, list users and their privileges. Now root has "Superuser".
postgres=# \du
List of roles
Role name | Attributes | Member of
-----------+------------------------------------------------------------+-----------
other | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
root | Superuser | {}
Hope it helps.
I guess you missed create password for your user. Try to create password as following:
CREATE USER root WITH PASSWORD 'your_new_password';
CREATE DATABASE svp-chicago_development;
GRANT ALL PRIVILEGES ON DATABASE svp-chicago_development to root;
ALTER DATABASE svp-chicago_development OWNER TO root;
I had this issue when working on a Rails 6 application with PostgreSQL.
The first check is to ensure that you've granted all privileges for a database to the particular user that you want using:
GRANT ALL PRIVILEGES ON DATABASE mydatabase TO myusername;
But in my own case, the cause of the issue was that I created a Database and then granted all privileges to a particular user. After some time, I granted another user the privileges using GRANT ALL PRIVILEGES ON DATABASE mydatabase TO myusername;
So the second user even though I granted all privileges to it, didn't have permissions to perform actions on the database tables.
Here's how I fixed it:
Log into the PostgreSQL console where the database is stored:
sudo -u postgres psql
List all databases in that PostgreSQL database server:
\l
OR
\list
Connect to the database that you want to fix it's permissions:
\c mydatabase
OR
\connect mydatabase
List all tables in the current database using your search_path:
\dt
OR
List all tables in the current database regardless of your search_path:
\dt *.
You will notice that the tables still reference the initial user or role as the owner.
Now you will have to modify the tables to reference the new user or role as the owner.
You can modify each table individually using this:
ALTER TABLE table_name OWNER TO new_owner;
This does not require specifing the old_owner. It is essential when the user is postgres (the default database user) and you want to modify the owner to a new user.
OR modify all the tables simultaneously using this:
REASSIGN OWNED BY old_owner TO new_owner;
This requires specifing the old_owner. It is essential when you have already modified the user from postgres (the default database user) to another user and you want to modify the owner to a new user.
Note: Ensure that you are connected to the database that you want to modify privileges/permissions for, else you might run into errors.
Also, if you're using a service like Heroku, it's worth checking to see if you have overrun your row limit and write access has been revoked in the database.
You can do that by going to the dashboard, click on the app, click on the postgres service icon, then check the row limit.
Just in case someone else comes here with the same issue, I did try many other solutions and the one that worked for me the best was the following: Modify OWNER on all tables simultaneously in PostgreSQL
This worked since my user (e.g. root or postgres) had Superuser privileges so trying REASSIGN OWNED gives error when trying to assign system objects
ALTER DATABASE didn't work since the issue is on a table object ownership and not in the DB ownership. Altering the owner on the DB doesn't propagate to the other object on that DB schema
Try listing your tables to see who the owner is. In my case, I had imported data via psql < dump.sql and all the imported tables were owned by postgres instead of my user.
To check this, start psql and enter the command \dt within your database. Look for the following line:
public | schema_migrations | table | postgres
Not good! It's owned by postgres and you'll need to fix that:
You can use #ddreliv's solution here to reassign the owner of every table, or
if you have a dump, simply drop the database and re-import it with the proper user. For example, use sudo -u my_user psql < dump.sql instead of sudo -u postgres psql < dump.sql.
Here's how I fixed it:
Log into the PostgreSQL console where the database is stored:
sudo -u postgres psql
List all databases in that PostgreSQL database server:
\l
OR
\list
Connect to the database that you want to fix it's permissions:
\c mydatabase
OR
\connect mydatabase
List all tables in the current database using your search_path:
\dt
OR
List all tables in the current database regardless of your search_path:
\dt *.
Then giving privilleages to the table :-
One Table
GRANT ALL PRIVILEGES ON TABLE side_adzone TO jerry;
All Tables of schema
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO jerry;
for our case (with docker-compose)
remove postgres data
$ docker-compose down # CAUTION: may cause data loss
$ docker volume rm XXX_DB_VOLUME # CAUTION: will cause data loss
then downgrade postgres from 11.11 to 11.8 in docker-compose.yml
xxx_db:
image: "postgres:11.8"
finally start postgres docker again
$ docker-compose up -d
Problem solved.
Adding "Superuser" to the role is a security issue, superusers can drop databases not only the svp-chicago_development pointed out by the OP but any other database existent.
I believe the issue that OP have is related to tables owner as pointed out by #kevin-cooper.
Even if the OP did the commands that he said granting all privileges and owner to the database, if the OP still runs any table creation or restore dump with the postgres credential the tables on the database svp-chicago_development it will have postgres as owner and if the application in running with other user/role it will throw the error mentioned by the OP.
You can check if that's the case in psql by:
\c svp-chicago_development
then listing the tables:
\dt
It will show something like:
Schema | Name | Type | Owner
--------+------------+-------+----------
public | table_name | table | postgres
If you have a few rows you can manually change the owner of each row by running:
ALTER TABLE table_name OWNER TO new_owner;
If you are running a pg_restore make sure to pass --role so objects will be owned by the role specified.
I'm very new with ruby and coding in general, I'm working on a ruby on rails project and PostgreSQL is giving me an error: FATAL:role "myRailsApp" does not exist. googled for posible solutions and couldn't find anything. thanks
You don't have a role with that name created in the Postgres DBMS. If you are new, you probably should download Pgadmin (http://www.pgadmin.org/), there you can configure your postgres database.
When you open it you will see your servers and stuff, loof for the 'Login roles' part and create a new role that can create databases (That role needs to have the same user and password you chose on the configuration file)
https://www.youtube.com/watch?v=1wvDVBjNDys
When I execute heroku db:pull it finds my local dev db at:
postgres://127.0.0.1/myapp_development?encoding=utf8
once I confirm though, it fails with:
Sequel::DatabaseConnectionError -> PGError: fe_sendauth: no password supplied
I tried running the pull with the local db specified, e.g.
heroku db:pull postgres://root:#localhost/db_name
which gives the same no password supplied error.
I thought I may need to change root: to myname: because thats the user I granted superuser rights to when I setup postgres but neither root: or myname: works
My database.yml has username: and password: blank for all databases specified.
From the command line as myname#ubuntu I can type psql myapp_development and connect fine and run selects.
What am I missing here?
Is it related to my pg_hba.conf settings? I had a look inside that and it says:
# Database administrative login by UNIX sockets
local all postgres ident
# TYPE DATABASE USER CIDR-ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all ident
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
Should 'trusted' be used there? And if so if I edit that file do I need to restart postgres? And if 'trust' is necessary then how come rails and the psql command line tools work without passwords when logged in as my myname user?
Thank you!
Authentication method trust might do the trick, but as you are undoubtedly aware, this is not secure.
After editing pg_hba.conf, you don't have to restart. A reload is enough (quoting the manual):
The pg_hba.conf file is read on start-up and when the main server
process receives a SIGHUP signal. If you edit the file on an active
system, you will need to signal the postmaster (using pg_ctl reload or
kill -HUP) to make it re-read the file.
pg_ctl reload
See the fine manual. You might need the manual for version for 8.3. Shared db on heroku currently runs on PostgreSQL 8.3. (Also, I doubt you have access to pg_ctl on heroku.)
Be aware of this:
If no password has been set up for a user, the stored password is null
and password authentication will always fail for that user.
Emphasis mine. You might be able to log in locally, because the auth-methods ident or peer allow for that. But for your purpose you may need a password!