Postgres Auth failed, no pg_hba.conf - ruby-on-rails

I've got an app that seems to be running just fine for the most part on Postgres (for Heroku), but now that I'm trying to do some fancier stuff like starting a delayed_job worker with
RAILS_ENV=production script/delayed_job
I get this error:
FATAL: password authentication failed for user "<myusername>" (PG::Error)
This is troublesome, because
Installing postgres was a huge, confusing mess for me, as a Rails newbie, and I never remember setting a password. (I've got a password for PGAdminII, but I know that one, and this isn't it). When I go into my database.yml file and try to change the password to everything I can think of it being, it doesn't work.
Fishing around on the internet, it looks like I should do something to a pg_hba.conf file, but I don't have one anywhere, apparently.
I've been working on this app for weeks, and I really don't want to erase what I've got going on, so I'm wary of initdb'ing in another directory.
Database stuff makes no sense to me. I've tried to figure it out, but I think I'm just too new to this stuff. And I never know where to start to fix things.
This question is sort of vague cause I don't know enough to know what specific question to ask -- but can can anybody help me with this? Like: How do I figure out my password? What do I do about pg_hba.conf? Will I have to start a new database?
EDIT -- Per the below suggestions (Thanks!), I ran both "ps -A | grep postgres" and "ps -A | grep pg_ctl". The output of each, respectively, is
85 ?? 0:06.94 postgres: logger process
101 ?? 0:32.04 postgres: writer process
102 ?? 0:23.98 postgres: wal writer process
103 ?? 0:06.70 postgres: autovacuum launcher process
104 ?? 0:07.60 postgres: stats collector process
6337 ttys002 0:00.01 grep postgres
and
6340 ttys002 0:00.00 grep pg_ctl
neither of which, unfortunately, appears to have anything preceded by -D.

I can find my pg_hba.conf file in the path:
/etc/postgresql/9.1/main/

Related

Postgres server error on mac ["root" execution of the PostgreSQL server is not permitted. The server must be started under an unprivileged user ID]

Anyone to help me solve this issue that has been on my case for the last couple of hours? I have so far tried all suggested solutions but can't seem to find anything working.
Error:
"root" execution of the PostgreSQL server is not permitted.
The server must be started under an unprivileged user ID to prevent
possible system security compromise. See the documentation for
more information on how to properly start the server.
I will forever be grateful to anyone who helps me solve this.
Try, change you user in postgres
Switching Over to the postgres Account
Switch over to the postgres account on your server by typing:
sudo -i -u postgres
You can now access a Postgres prompt immediately by typing:
psql
This will log you into the PostgreSQL prompt, and from here you are free to interact with the database management system right away.
Exit out of the PostgreSQL prompt by typing:
\q
This will bring you back to the postgres Linux command prompt.
Accessing a Postgres Prompt Without Switching Accounts
You can also run the command you’d like with the postgres account directly with sudo.
For instance, in the last example, you were instructed to get to the Postgres prompt by first switching to the postgres user and then running psql to open the Postgres prompt. You could do this in one step by running the single command psql as the postgres user with sudo, like this:
sudo -u postgres psql
This will log you directly into Postgres without the intermediary bash shell in between.
Again, you can exit the interactive Postgres session by typing:
\q
Many use cases require more than one Postgres role. Read on to learn how to configure these.
Steps to properly install Postgres via Homebrew on Mac M1:
brew install postgres [Maybe you need to use the alternative install method, as described here: https://docs.brew.sh/Installation#alternative-installs]
initdb /Users/<username>/db -E utf8
[This initializes postgres to use the given directory as the database directory. Normally it is not adviced to use the user directory for database storage.]
pg_ctl -D /Users/<username>/db -l logfile start
[After getting success with step 2 it will prompt to run step 3. This command manually starts the server.]

Postgresql, Rails - could not fork autovacuum worker process: Resource temporarily unavailable

This is happening to me while in my local environment, Mac OSX, every time I start my server - puma - and workers - resque.
The logs don't say anything helpful, just a repeated, "could not fork autovacuum worker process: Resource temporarily unavailable."
Until I turn ctr-c out of the server, it locks up my entire computer. When I try to visit a site in the browser it just hangs, and when I open a new tab in the terminal it says, 'pipe broken' and closes it. The MAC console isn't spitting out anything helpful, at least from what I can tell.
Anyone have any thoughts to why this is?
I've restarted Postgres multiple times to no avail.
EDIT:
Log just started spitting out, 'LOG: could not fork new process for connection: Resource temporarily unavailable'
Puma thread count:
threads_count = ENV.fetch("RAILS_MAX_THREADS") { 10 }.to_i
DB: pool: 100
EDIT2:
Tried to increase max_connection count from 100 to 200, still nothing. Ran into the duplicate postmaster.id error. Removed it, restarted Postgres, but still the same issue remains - cleared the postmaster.id error though.
I followed these two articles, and so far this seems to work. Will update if something changes.
https://github.com/sociam/indx/wiki/Increasing-max-connections-under-os-x
http://big-elephants.com/2012-12/tuning-postgres-on-macos/
Edit: From what I have experienced, this hasn't helped me. Removing the postmaster.pid doesn't seem to do much either, but seems to do more than said above. If any one stumbles upon this and figures it out, if you could post about it that would be great. I'll update if anything else changes. FWIW, when this happens, sometimes Reddis takes a shit and says that it no longer can save to disk.
I was having the same issue and tried the fixes that #jack-rothrock had proposed on his answer, to no avail.
I noticed that when I tried to start postgres from the command line using homebrew services, I would get a message that postgres was already running, which reminded me that I had installed the "Postgres App" (the postgres application you can download from http://postgresapp.com/ . I relaunched that application and not everything works.
Nice way to start the year!

Why does my Rails server exit right away when I try to start it?

It was working just a couple of days ago and then I let my laptop's battery go dead over the weekend while the server was running.
Now when I try to start my rails server with the command
bundle exec rails s
It does a few things and then I get the message
Exiting
/Users/me/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/postgresql_adapter.rb:1216:in `initialize': FATAL: no pg_hba.conf entry for host "fe80::1%lo0", user "city72", database "city72_development", SSL off (PG::ConnectionBad)
+ lots more stuff
... then tons more information about .rbenv and railties and active-record and stuff that I don't usually have to deal with.
Any ideas what I might have messed up when my server got shutdown by letting my battery die? Maybe it's just a coincidence that my battery died and something is all of a sudden wrong with my gems?
you probably got a new ip-adress after booting again and the security line in the pg_hba.conf line of the postgresql server no longer matches your address.
You can find this file in the main data folder of your postgresql server. (Well usually at least, the location is configurable)

Sunspot Solr environment difficult

I'm using the amazing sunspot gem(github.com/outoftime/sunspot) on a rails application but I'm having such a huge problem. I confess that I still don't know how to configure it correctly according to my environment, but everything is set up and running well on my local and stage servers.
Well, to sum up, my problem is that in production I have a model that is currently updated - every list that envolves this model, an attribute is incremented. And the main problem is that when I try to perform a complex search on this model(not contextual) the Connection Refused error appears to me but solr is running up and performing all other searches.
My solrconfig.xml is just like sunspot installation, I didn't change anything. Is the autoCommit section the solution for this, or is nothing to do with it ?
Sorry for the last update, I wasn't made a newbie mistake, the result of "ps aux | grep java" on the server:
ubuntu 4039 0.0 1.8 2278060 144084 ? Ssl Jan21 8:10 java -Djetty.port=8983 -Dsolr.data.dir=/home/ubuntu/mallguide/mallguide-rails/solr/data/production -Dsolr.solr.home=/home/ubuntu/mallguide/mallguide-rails/solr -Djava.util.logging.config.file=/tmp/logging.properties20120121-4039-co662r-0 -jar start.jar
ubuntu 23125 0.0 0.0 7628 1004 pts/1 S+ 10:47 0:00 grep --color=auto java
And my sunspot.yml file:
production:
solr:
hostname: localhost
port: 8983
log_level: WARNING
development:
solr:
hostname: localhost
port: 8982
log_level: INFO
test:
solr:
hostname: localhost
port: 8981
log_level: WARNING
auto_commit_after_request: false
Sorry for the poor english, hope that someone could help me.
I still don't know what to do to correct this problem, the point is that I have only one model that is updated(not the indexed fields on searchable) all the time and solr just fail for this model, and not for the others.
Any help ?
I fixed a similar error before. It may be related if your error symptoms match:
The kind of error messages I have encountered is like this:
Connection refused - connect(2)
with the backtrace:
/home/john/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/net/http.rb:762:in `initialize'
...
rest-client (1.6.7) lib/restclient/net_http_ext.rb:51:in `request'
rsolr (1.0.8) lib/rsolr/connection.rb:15:in `execute'
Once I restarted sunspot solr server, errors will stop. During this time I can read indexes but not write to them.
Solution
The reason behind my error is that: I manually called RAILS_ENV=production rake sunspot:solr:start on the production server, and I also use capistrano to deploy so it has the current directory and the shared directory.
When I run that start command in the current directory, for some reason the index file is still referenced by the release path (e.g. release/2012xxxxxxxx/...). Capistrano will delete old releases, so every now and then solr will not be able to reference the folder if it gets deleted.
The solution is to explicitly specify the index file path with the symlinked current directory:
RAILS_ENV=production rake sunspot:solr:start --port=8983 --data-directory=#{current_path}/solr/data/#{rails_env} --pid-dir=#{current_path}/solr/pids/#{rails_env}"

Rails + Postgres drop error: database is being accessed by other users

I have a rails application running over Postgres.
I have two servers: one for testing and the other for production.
Very often I need to clone the production DB on the test server.
The command I'm runnig via Vlad is:
rake RAILS_ENV='test_server' db:drop db:create
The problem I'm having is that I receive the following error:
ActiveRecord::StatementInvalid: PGError: ERROR: database <database_name> is being accessed by other users DROP DATABASE IF EXISTS <database_name>
This happens if someone has accessed the application via web recently (postgres keeps a "session" opened)
Is there any way that I can terminate the sessions on the postgres DB?
Thank you.
Edit
I can delete the database using phppgadmin's interface but not with the rake task.
How can I replicate phppgadmin's drop with a rake task?
If you kill the running postgresql connections for your application, you can then run db:drop just fine. So how to kill those connections? I use the following rake task:
# lib/tasks/kill_postgres_connections.rake
task :kill_postgres_connections => :environment do
db_name = "#{File.basename(Rails.root)}_#{Rails.env}"
sh = <<EOF
ps xa \
| grep postgres: \
| grep #{db_name} \
| grep -v grep \
| awk '{print $1}' \
| xargs kill
EOF
puts `#{sh}`
end
task "db:drop" => :kill_postgres_connections
Killing the connections out from under rails will sometimes cause it to barf the next time you try to load a page, but reloading it again re-establishes the connection.
Easier and more updated way is:
1. Use ps -ef | grep postgres to find the connection #
2. sudo kill -9 "# of the connection
Note: There may be identical PID. Killing one kills all.
Here's a quick way to kill all the connections to your postgres database.
sudo kill -9 `ps -u postgres -o pid`
Warning: this will kill any running processes that the postgres user has open, so make sure you want to do this first.
I use the following rake task to override the Rails drop_database method.
lib/database.rake
require 'active_record/connection_adapters/postgresql_adapter'
module ActiveRecord
module ConnectionAdapters
class PostgreSQLAdapter < AbstractAdapter
def drop_database(name)
raise "Nah, I won't drop the production database" if Rails.env.production?
execute <<-SQL
UPDATE pg_catalog.pg_database
SET datallowconn=false WHERE datname='#{name}'
SQL
execute <<-SQL
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = '#{name}';
SQL
execute "DROP DATABASE IF EXISTS #{quote_table_name(name)}"
end
end
end
end
When we used the "kill processes" method from above, the db:drop was failing (if :kill_postgres_connections was prerequisite). I believe it was because the connection which that rake command was using was being killed. Instead, we are using a sql command to drop the connection. This works as a prerequisite for db:drop, avoids the risk of killing processes via a rather complex command, and it should work on any OS (gentoo required different syntax for kill).
cmd = %(psql -c "SELECT pg_terminate_backend(procpid) FROM pg_stat_activity WHERE procpid <> pg_backend_pid();" -d '#{db_name}')
Here is a rake task that reads the database name from database.yml and runs an improved (IMHO) command. It also adds db:kill_postgres_connections as a prerequisite to db:drop. It includes a warning that yells after you upgrade rails, indicating that this patch may no longer be needed.
see: https://gist.github.com/4455341, references included
Step 1
Get a list of all postgres connections with ps -ef | grep postgres
The list will look like this:
502 560 553 0 Thu08am ?? 0:00.69 postgres: checkpointer process
502 565 553 0 Thu08am ?? 0:00.06 postgres: bgworker: logical replication launcher
502 45605 553 0 2:23am ?? 0:00.01 postgres: st myapp_development [local] idle
Step 2
Stop whatever connection you want with sudo kill -9 <pid>, where pid is the value in the second column. In my case, I wanted to stop the last row, with pid 45605, so I use:
sudo kill -9 45605
Please check if your rails console or server is running in another tab and then
stop the rails server and console.
then run
rake db:drop
Let your application close the connection when it's done. PostgreSQL doesn't keep connections open , it's the application keeping the connection.
I wrote a gem called pgreset that will automatically kill connections to the database in question when you run rake db:drop (or db:reset, etc). All you have to do is add it to your Gemfile and this issue should go away. At the time of this writing it works with Rails 4 and up and has been tested on Postgres 9.x. Source code is available on github for anyone interested.
gem 'pgreset'
Rails is likely connecting to the database to drop it but when you log in via phppgadmin it is logging in via the template1 or postgres database, thus you are not affected by it.
This worked for me (rails 6):
rake db:drop:_unsafe
I think we had something in our codebase that initiated a db connection before the rake task attempted to drop it.
After restarting the server or computer, please try again.
It could be the simple solution.
You can simply monkeypatch the ActiveRecord code that does the dropping.
For Rails 3.x:
# lib/tasks/databases.rake
def drop_database(config)
raise 'Only for Postgres...' unless config['adapter'] == 'postgresql'
Rake::Task['environment'].invoke
ActiveRecord::Base.connection.select_all "select pg_terminate_backend(pg_stat_activity.pid) from pg_stat_activity where datname='#{config['database']}' AND state='idle';"
ActiveRecord::Base.establish_connection config.merge('database' => 'postgres', 'schema_search_path' => 'public')
ActiveRecord::Base.connection.drop_database config['database']
end
For Rails 4.x:
# config/initializers/postgresql_database_tasks.rb
module ActiveRecord
module Tasks
class PostgreSQLDatabaseTasks
def drop
establish_master_connection
connection.select_all "select pg_terminate_backend(pg_stat_activity.pid) from pg_stat_activity where datname='#{configuration['database']}' AND state='idle';"
connection.drop_database configuration['database']
end
end
end
end
(from: http://www.krautcomputing.com/blog/2014/01/10/how-to-drop-your-postgres-database-with-rails-4/)
I had this same issue when working with a Rails 5.2 application and PostgreSQL database in production.
Here's how I solved it:
First, log out every connection to the database server on the PGAdmin Client if any.
Stop every session using the database from the terminal.
sudo kill -9 `ps -u postgres -o pid=`
Start the PostgreSQL server, since the kill operation above stopped the PostgreSQL server.
sudo systemctl start postgresql
Drop the database in the production environment appending the production arguments.
rails db:drop RAILS_ENV=production DISABLE_DATABASE_ENVIRONMENT_CHECK=1
That's all.
I hope this helps
[OSX][Development/Test] Sometimes it is hard to determine the proper PID when you have a lot of PostgreSQL processes like checkpointer, autovacuum launcher, etc. In this case, you can simply run:
brew services restart postgresql#12
If you dockerized your app, then restart your db service
sudo docker-compose restart db
Just make sure that the you have exited the rails console on any open terminal window and exited the rails server...this is one of the most common mistake made by people
I had a similar error saying 1 user was using the database, I realized it was ME! I shut down my rails server and then did the rake:drop command and it worked!
Solution
Bash script
ENV=development
# restart postgresql
brew services restart postgresql
# get name of the db from rails app
RAILS_CONSOLE_COMMAND="bundle exec rails c -e $ENV"
DB_NAME=$(echo 'ActiveRecord::Base.connection_config[:database]' | $RAILS_CONSOLE_COMMAND | tail -2 | tr -d '\"')
# delete all connections to $DB_NAME
for pid in $(ps -ef | grep $DB_NAME | awk {'print$2'})
do
kill -9 $pid
done
# drop db
DISABLE_DATABASE_ENVIRONMENT_CHECK=1 RAILS_ENV=$ENV bundle exec rails db:drop:_unsafe
ActiveRecord::StatementInvalid: PG::ObjectInUse: ERROR: database "database_enviroment" is being accessed by other users
DETAIL: There is 1 other session using the database.
For rails 7: you can use -> rails db:purge

Resources