random request timeouts - rails, postgres, passenger, nginx - ruby-on-rails

Question

Have a rails API app, hosted on cloud with phusion passenger and nginx. Requests to the API are getting timed out if the API is not accessed for some time.
Checking the logs, it seems the requests successfully hits rails and this is where it times out.
Here is the rails log
PG::UnableToSend (server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
activerecord (4.2.5.1) lib/active_record/connection_adapters/postgresql_adapter.rb:215:in `exec'
activerecord (4.2.5.1) lib/active_record/connection_adapters/postgresql_adapter.rb:215:in `dealloc'
activerecord (4.2.5.1) lib/active_record/connection_adapters/postgresql_adapter.rb:198:in `block in clear'
activerecord (4.2.5.1) lib/active_record/connection_adapters/postgresql_adapter.rb:197:in `each_value'
activerecord (4.2.5.1) lib/active_record/connection_adapters/postgresql_adapter.rb:197:in `clear'
activerecord (4.2.5.1) lib/active_record/connection_adapters/postgresql_adapter.rb:258:in `clear_cache!'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract_adapter.rb:291:in `reconnect!'
activerecord (4.2.5.1) lib/active_record/connection_adapters/postgresql_adapter.rb:275:in `reconnect!'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract_adapter.rb:328:in `verify!'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:456:in `block in checkout_and_verify'
activesupport (4.2.5.1) lib/active_support/callbacks.rb:88:in `__run_callbacks__'
activesupport (4.2.5.1) lib/active_support/callbacks.rb:778:in `_run_checkout_callbacks'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:455:in `checkout_and_verify'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:351:in `block in checkout'
/usr/local/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:348:in `checkout'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:263:in `block in connection'
/usr/local/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:262:in `connection'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:571:in `retrieve_connection'
activerecord (4.2.5.1) lib/active_record/connection_handling.rb:113:in `retrieve_connection'
activerecord (4.2.5.1) lib/active_record/connection_handling.rb:87:in `connection'
activerecord (4.2.5.1) lib/active_record/query_cache.rb:31:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
activerecord (4.2.5.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
actionpack (4.2.5.1) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
activesupport (4.2.5.1) lib/active_support/callbacks.rb:88:in `__run_callbacks__'
activesupport (4.2.5.1) lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
activesupport (4.2.5.1) lib/active_support/callbacks.rb:81:in `run_callbacks'
actionpack (4.2.5.1) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
actionpack (4.2.5.1) lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
airbrake (5.2.3) lib/airbrake/rack/middleware.rb:22:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
actionpack (4.2.5.1) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
actionpack (4.2.5.1) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
railties (4.2.5.1) lib/rails/rack/logger.rb:38:in `call_app'
railties (4.2.5.1) lib/rails/rack/logger.rb:20:in `block in call'
activesupport (4.2.5.1) lib/active_support/tagged_logging.rb:68:in `block in tagged'
activesupport (4.2.5.1) lib/active_support/tagged_logging.rb:26:in `tagged'
activesupport (4.2.5.1) lib/active_support/tagged_logging.rb:68:in `tagged'
railties (4.2.5.1) lib/rails/rack/logger.rb:20:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
actionpack (4.2.5.1) lib/action_dispatch/middleware/request_id.rb:21:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
rack (1.6.4) lib/rack/runtime.rb:18:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
activesupport (4.2.5.1) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
railties (4.2.5.1) lib/rails/engine.rb:518:in `call'
railties (4.2.5.1) lib/rails/application.rb:165:in `call'
newrelic_rpm (3.15.1.316) lib/new_relic/agent/instrumentation/middleware_tracing.rb:96:in `call'
/usr/share/ruby/vendor_ruby/phusion_passenger/rack/thread_handler_extension.rb:97:in `process_request'
/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:152:in `accept_and_process_next_request'
/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:113:in `main_loop'
/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler.rb:416:in `block (3 levels) in start_threads'
/usr/share/ruby/vendor_ruby/phusion_passenger/utils.rb:113:in `block in create_thread_and_abort_on_exception'
database.yml configuration
database: xxx
username: xxx
password: xxx
host: xxx
port: 5432
pool: 10
Passenger configuration
server {
listen 80 default_server;
server_name {{server_name}};
passenger_enabled on;
rails_env {{env}};
root {{rails_app_root}}/{{rails_app_package_name}}-{{node_env}}/public;
passenger_min_instances 3;
}
passenger_max_request_queue_size 500;
passenger_pre_start http://{{server_name}}{{pre_start_url}};
passenger_spawn_method direct;
passenger_pool_idle_time 0;
passenger_max_preloader_idle_time 0;
Any pointers on why this could be happening?
This does not happen always, only if you access the service after it has been idle for sometime.