Puma server hangs after capistrano "puma:restart" signal is sent - ruby-on-rails

Question

When I deploy my application for the first time using cap [env] deploy, everything works as expected. The code is deployed and the puma server is started successfully with puma:start.
capistrano output of puma starting
* 2015-01-09 12:19:37 executing `puma:start'
* executing "cd /path/to/app/current && bundle exec puma -q -d -e production -C ./config/puma/production.rb"
servers: ["example.com"]
[example.com] executing command
** [out :: example.com] Puma starting in single mode...
** [out :: example.com] * Version 2.9.2 (ruby 2.1.5-p273), codename: Team High Five
** [out :: example.com] * Min threads: 0, max threads: 16
** [out :: example.com] * Environment: production
** [out :: example.com] * Listening on unix:///path/to/app/shared/sockets/puma.sock
** [out :: example.com] * Daemonizing...
If I make a code change and attempt to re-deploy, instead of calling puma:start, capistrano calls puma:restart, which acts as if successful, but puma wasn't actually restarted.
* 2015-01-09 12:27:56 executing `puma:restart'
* executing "cd /path/to/app/current && bundle exec pumactl -S /path/to/app/shared/sockets/puma.state restart"
servers: ["example.com"]
[example.com] executing command
** [out :: example.com] Command restart sent success
At this point, if I refresh the web page, I get a 504 Gateway Time-out error. It's very similar to this issue..
As the person suggested, if I added workers 1 to my puma configuration file, restart would be working, but start/stop would not.
At my current state (without workers), if I do cap [env] puma:stop, it does not stop puma. It also does NOT delete any of these files:
/path/to/app/shared/pids/puma.pid
/path/to/app/shared/sockets/puma.sock
/path/to/app/shared/sockets/puma.state
Important Note
In my Rails server, I am using ActionController::Live with Rails 4.1. I am connected to an event stream with Javascript / Redis. I noticed this in my nginx error log:
nginx error log
2015/01/09 12:29:32 [error] 8992#0: *371 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client: [ip], server: example.com, request: "GET /build_configurations/refresh_workers HTTP/1.1",
upstream: "http://unix:///path/to/app/shared/sockets/puma.sock/build_configurations
/refresh_workers", host: "example.com", referrer: "https://example.com/"
All in all, how can I successfully use cap [env] deploy to deploy updates to my code while having puma start and restart okay?
Update
I found this issue that talks about restarting puma with ActionController::Live, but no solution seems present.

Answer 1

Using the information I found in this other Stack Overflow answer, I was able to implement a heartbeat that fixes this situation it seems. I haven't fully understood all of the mechanics myself, but I started with an initializer to start the heartbeat:
config/initializers/redis.rb
REDIS = Redis.new(url: "redist://localhost:6379")
heartbeat_thread = Thread.new do
while true
REDIS.publish('heartbeat','thump')
sleep 2.seconds
end
end
at_exit do
# not sure this is needed, but just in case
heartbeat_thread.kill
REDIS.quit
end
My controller has:
def build_status_events
response.headers["Content-Type"] = "text/event-stream"
redis = Redis.new(url: "redist://localhost:6379")
# blocks the current thread
redis.psubscribe(['my_event', 'heartbeat']) do |on|
on.pmessage do |pattern, event, data|
response.stream.write("event: #{event}\n")
if event == 'heartbeat'
response.stream.write("data: heartbeat\n\n")
else
response.stream.write("data: #{data.to_json}\n\n")
end
end
end
rescue IOError
logger.info 'Events stream closed'
ensure
logger.info 'Stopping Events streaming thread'
redis.quit
response.stream.close
end
I believe what happens is that the heartbeat gets posted, and if there's an error, the ensure block will be called and the subscription is closed. Then puma reboots as expected. If anyone has a better solution, or more information, please comment or add another answer.