I have the setup of my Rails app as following:
Rails: 3.0.5 (under Apache proxy) running on RHEL 5.6
Postgres: 8.4, running on Windows Server 2008
The 2 servers are on the same LAN.
The problem is, after some idle time, when I make a new request to the Rails app, it gives me the following error:
ActiveRecord::StatementInvalid (PGError: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
From what I have researched, it seems that the database connections are dropped after some timeout by Postgres. During this time, from the Rails side,
if I make a request to Rails (1st request), it will display the connection error as above
if I make another request to Rails (2nd request), Rails seems to reconnect to Postgres and functions correctly.
It means that I will always experience the first connection error then will have all normal operation again, which is very serious in my case since I'd like to deliver a non-error response to my client.
I have looked in following questions and answers, but they do not seem to be appropriate for my case:
"PGError: no connection to the server" after idle (Ubuntu for DB server)
How to use tcp_keepalives settings in Postgresql? (keepalives in MacOS)
Do you have any advice in order to make my app free from db connection errors? Thank you.
We had this problem on Heroku, a lot. As a hackish solution, here's what we did. Put the following in your ApplicationController:
prepend_before_filter :confirm_connection
def confirm_connection
c = ActiveRecord::Base.connection
begin
c.select_all "SELECT 1"
rescue ActiveRecord::StatementInvalid
ActiveRecord::Base.logger.warn "Reconnecting to database"
c.reconnect!
end
end
Basically, tests the connection on each controller hit. Scalable? Not really. But it fixed the problem for us.
In database.yml, do you have the reconnect: true option set for the connection? ex:
production:
adapter: postgresql
database: myapp
username: deploy
password: password
reconnect: true
I'm not sure about the specific error, but without this option you need to handle a class of expected db errors yourself.
I faced this error while going live on production.I just opened my production console and tried to connect the PG db and it worked.
RAILS_ENV=production rails c
User.new
and it worked after this.
hope it helps someone ;)
Related
It was working just a couple of days ago and then I let my laptop's battery go dead over the weekend while the server was running.
Now when I try to start my rails server with the command
bundle exec rails s
It does a few things and then I get the message
Exiting
/Users/me/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/postgresql_adapter.rb:1216:in `initialize': FATAL: no pg_hba.conf entry for host "fe80::1%lo0", user "city72", database "city72_development", SSL off (PG::ConnectionBad)
+ lots more stuff
... then tons more information about .rbenv and railties and active-record and stuff that I don't usually have to deal with.
Any ideas what I might have messed up when my server got shutdown by letting my battery die? Maybe it's just a coincidence that my battery died and something is all of a sudden wrong with my gems?
you probably got a new ip-adress after booting again and the security line in the pg_hba.conf line of the postgresql server no longer matches your address.
You can find this file in the main data folder of your postgresql server. (Well usually at least, the location is configurable)
We have a Ruby v.2.0.0-p247 on Rails v4.0.1 application using pg gem v0.17.0.
The application runs smoothly under Mac OS X Mavericks v10.9 with PostgreSQL Server v9.2.4 installed using HomeBrew but it throws the following exception under Ubuntu v13.04 using PostgreSQL Server 9.1:
PG::UnableToSend: server closed the connection unexpectedly
This probably means the server terminated abnormally before or while processing the request.
The exception occurs after transactional queries (form submission).
I tried the following with database.yml:
Adding reconnect: true
Adding port: 5432
Adding socket: /var/run/postgresq/SOCKET_FILE
And tried the following with PostgreSQL configuration under Ubuntu:
Disabling SSL.
Changing TCP keepalives parameters to pump timeout.
Changing log level to DEBUG and search for possible errors on PostgreSQL Server.
Also tried:
Downgrade to pg gem v0.16.0.
Update all Ubuntu 13.04 packages to latest versions.
What could possible be wrong?
UPDATES:
12/03/2013: Some suggested checking firewall settings. ufw status said that ufw is disabled.
12/08/2013: After trying out with a vanilla Rails app and a lot of mangling with the current application, the problem is originating from rails4/activerecord-session_store gem. Line 47 in lib/active_record/session_store/session.rb is the culprit.
This basically happens when you use an old version of launchy , and as per this issue on launchy's git repo quoting #infertux
In the rare case when exec fails to run the command - typically when the file cannot be opened raising Errno::ENOENT - Launchy would raise an exception but not showing any output
You can check your Gemfile.lock to see if you're using a version of launchy below 2.4.1, and I doubt that you're using letter_opener Gem in your development environment which depends on launchy so updating letter_opener to 1.2.0 will update launchy to a version above 2.4.0 mostly 2.4.2 which has this issue fixed
All credit goes to #infertux
It would really help if you provided your database.yml file
The default connection method is a unix domain socket, not to be confused with a TCP/IP socket. The unix domain socket connection method is used by default.
You should check that the unix user that you are trying to run rails under has sufficient permissions to access the domain socket (typically at /var/run/postgresql/.s.PGSQL.5432)
Try typing the following as your rails user:
psql
If you get a database connection error then its likely a permissions problem if postgres is actually running.
You can check your /etc/postgresql.conf file and have postgres configure the group and permissions on the socket when it starts:
unix_socket_directory = '/var/run/postgresql' # dont worry if yours is different
#unix_socket_group = '' # default is usually ok
#unix_socket_permissions = 0777 # uncomment this if needed
Another option is to add the user to the group that has write access to the socket, vs allowing all users on the machine access with the 0777 permissions setting above. You may want to create a postres_users group for this purpose if the default Ubuntu groups provide insufficient granularity for your needs.
It looks to me like there's confusion about the connection method. PostgreSQL supports the two methods:
Socket
TCP/IP
These are completely, utterly different. :-)
In your question, you show a socket setting, but mention TCP. I suggest to focus on each of these two ways of connecting in isolation and see which produces results. I.e., create two version branches pg-socket and pg-tcpip just to make things clear. Then, clean up your config file and attempt to connect via both methods.
Often, a socket connection is easier because you just need to know the pathname of the socket "file". No fireware settings are necessary (because no TCP/IP networking is involved.)
Update your question after doing this and tell us how each of the two methods did for you.
Have you set keepalives as specified here http://www.postgresql.org/docs/9.2/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS similar issue was solved with it at: https://bitbucket.org/ged/ruby-pg/issue/170/pg-unabletosend-connection-timed-out
We're not sure if this is something not configured properly in our Rails setting, but we're seeing this error in our nginx error logs get repeated: Exception PhusionPassenger::UnknownError in PhusionPassenger::Rack::ApplicationSpawner (Failed to connect to a master node at x.objectrocket.com:10804 ...
Here's a gist with more from the error log: https://gist.github.com/panabee/a291526f4dcf4cd434d9
How do we stop these errors?
We're on Rails 3.2.12.
This error means the ObjectRocket mongo server is refusing or timing out your connections. Contact them with the specific logs to work out why they refuse the connection.
Also, try setting your timeouts lower to avoid hanging connections for your clients:
MongoMapper.connection = Mongo::Connection.new(host, 27017, :connect_timeout => 1, :op_timeout => 1)
This sets the connect and read timeouts. There's a :timeout option as well: it's related to connection pooling, not connecting to the server.
Neither :op_timeout nor :connect_timeout have a default value. Verify their settings by reading MongoMapper.connection.connect_timeout.
If you're using the 1.8.0 mongo gem version or later, you may want to replace the deprecated Mongo::Connection class with Mongo::MongoClient.
Try to add
# encoding: UTF-8
In the first line of environment.rb.
this is a follow up to my other question regarding unicorn. following problem:
i'm using ruby 1.8.7, rails 3.0, pg gem (0.13.0).
when i start the rails console in production mode, i can query records etc, so the
database connection and the pgsql adapter works. i can disconnect the AR connection via ActiveRecord::Base.connection.disconnect!, but as soon as i try to reconnect using ActiveRecord::Base.establish_connection, ruby just dies. no error, no log output, no seg fault, it just quits (with status 1). i tried setting the host in my database.yml to the postgres unix socket, as well as to 127.0.0.1, doesn't change a thing. what could be the problem, or what can i do to get to the root of this? i'm not sure how to debug this, with absolutely no output or error message.
i solved the problem (more or less) by downgrading pg to version 0.11.0. this works for me for now. cost me 1 and a 1/2 days ...
Why are you closing explicitely the connection to the database ? Rails/AR handles the connection pool for you. The rack middleware is supposed to release connections at the end of the request processing.
ive a peculiar consistent problem in production. im running rails3 + nginx with latest datamapper and ruby-enterprise
Everytime a deploy a new version (touch restart.txt) i get a bunch of errors (happen during different requests) just after the deploy has happened. The errors are not always the same:
DataObjects::SQLError: Lost connection to MySQL server during query
ArgumentError: Field-count mismatch. Expected 1 fields, but the query yielded 10
ArgumentError: Field-count mismatch. Expected 10 fields, but the query yielded 1
DataObjects::SQLError: Lost connection to MySQL server during query
im running an other rails app (2.3+apache+ruby-ent) with active record and i NEVER EVER had any problem during restarts
Does anyone have some advise on why this happens and how to get rid of it?
thanks
Anders
Do you get the same errors when you do a sudo kill -HUP nginx_pid? (do a sudo ps aux|grep nginx to get the pid).
It is indeed a very strange set of errors you're getting. Perchance you still have a session open to your db while you're restarting, causing problems with your db pool? Rails db access is usually intermittent, but I can imagine issues happening if you have a long running db query going and you attempt to restart Rails.
The fact that the errors keep changing would lead me to believe that the errors are related to resource access, rather than problems with your config.