I have a Rails 3.1.4 app hosted on heroku. I have added the logentries add-on.
I didn't touch the default configuration for alerts, and I now receive every few hours an alert: 'Exit timeout':
Exit timeout: Heroku/myappname
2012-03-23 11:01:41.723
168 <40>1 2012-03-23T11:01:41+00:00 d. heroku web.1 - - Error R12 (Exit timeout) -> Process failed to exit within 10 seconds of SIGTERM
You are receiving this email because your Logentries alarm "Exit timeout"
has been triggered.
In context:
2012-03-23 11:01:28.878 95 <40>1 2012-03-23T11:01:28+00:00 d. heroku web.1 - - Idling
2012-03-23 11:01:31.740 118 <40>1 2012-03-23T11:01:31+00:00 d. heroku web.1 - - Stopping process with SIGTERM
2012-03-23 11:01:41.723 168 <40>1 2012-03-23T11:01:41+00:00 d. heroku web.1 - - Error R12 (Exit timeout) -> Process failed to exit within 10 seconds of SIGTERM
The logs nor this email alert do not give me anything useful to start looking for this issue.
I have a very low traffic (application still in private beta). I suspect it's heroku shutting down the application, and my application does not respond as expected to the SIGTERM signal.
I do not have anything fancy in the application (such as a rake task launched with Heroku Scheduler).
The only thing slightly out of the ordinary is the use of the gem apn_on_rails, which open a connection to Apple's APN servers.
Do you have any idea what is causing this? Should I just ignore this and turn the alert off?
If you're using webrick to run your application on Heroku, you should switch to using 'thin': See https://devcenter.heroku.com/articles/rails3#webserver
This removed all occurrences of the R12 error from my logs.
Related
I'm making an expensive call to my Heroku Rails server. After 13-15 seconds the console in my browser reports a Service Unavailable 503 error. However when I check my heroku logs, it reports:
Completed 200 OK in 45592ms (Views: 220.3ms | ActiveRecord: 33457.5ms)
Other times the heroku logs report a memory exceeded quota. Here is an example of that below.
2015-06-11T15:17:20.238285+00:00 app[web.1]: Completed 200 OK in 81881ms (Views: 201.6ms | ActiveRecord: 18021.2ms)
2015-06-11T15:17:33.482930+00:00 heroku[web.1]: Process running mem=841M(164.4%)
2015-06-11T15:17:33.482930+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2015-06-11T15:17:53.147570+00:00 heroku[web.1]: Process running mem=841M(164.4%)
2015-06-11T15:17:53.147679+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2015-06-11T15:17:59.751540+00:00 app[web.1]: E, [2015-06-11T15:17:59.695813 #3] ERROR -- : worker=2 PID:13 timeout (121s > 120s), killing
2015-06-11T15:17:59.916750+00:00 app[web.1]: E, [2015-06-11T15:17:59.906435 #3] ERROR -- : reaped #<Process::Status: pid 13 SIGKILL (signal 9)> worker=2
2015-06-11T15:18:02.487428+00:00 app[web.1]: I, [2015-06-11T15:18:02.427293 #16] INFO -- : worker=2 ready
Why is it reporting a Completed 200 when the console is reporting a 503?
there are 2 different things here:
your app
the heroku load balancer
In this case the load balancer see that the request is taking too long and sends you the 503. In the back the service processes the request and returns completes with a 200.
See:
https://devcenter.heroku.com/articles/limits
https://devcenter.heroku.com/articles/request-timeout
Heroku does not gives us much information for error-code = H10. Simple put, something is going wrong with your application code / configuration. To see what's going wrong, run heroku run rails console and you will be able to see error details which will be help you in resolving the error. No need to use logs. heroku run rails console is a big help
This happened with me when I used Node. I put a timeout for 30 seconds and I used to get HTTP:503. I then came to know that it's the timeout that caused the issue. I changed the timeout to <30 seconds and it is working fine. Maybe it's because of the loadbalancer that Heroku uses.
Thanks.
We get errors like this one daily (sometimes more than once):
Apr 13 12:16:31 app1 unicorn.log: E, [2014-04-13T12:16:31.302011 #17269] ERROR -- : reaped #<Process::Status: pid 17300 SIGABRT (signal 6)> worker=2
We use Unicorn 4.8.2 with a Ruby on Rails app.
It doesn't seem to happen at any obvious time, like during or just after deploys.
We're not sure how to debug this – any suggestions on either what the problem could be, or how to debug it?
Turns out it was simply that our monit did a "kill -6" (SIBABRT) when the process
used too much memory, so we bumped that limit for now.
We've yet to research why it used that much memory.
I've tried heroku run rake db:migrate and get Connection timed out error.
Adn heroku logs shows this error.
2013-10-18T10:07:20.765963+00:00 heroku[run.5173]: Awaiting client
2013-10-18T10:07:30.324619+00:00 heroku[run.5173]: Error R99 (Platform
error) -> Failed to launch the dyno within 10 seconds
2013-10-18T10:07:30.324809+00:00 heroku[run.5173]: Stopping process
with SIGKILL 2013-10-18T10:07:50.767590+00:00 heroku[run.5173]: Error
R13 (Attach error) -> Failed to attach to process
2013-10-18T10:07:51.840739+00:00 heroku[run.5173]: Process exited with
status 128 2013-10-18T10:07:30.363887+00:00 heroku[run.5173]: State
changed from starting to complete
According to the document, there is nothing that I can do with R99 error, except for waiting.
But I've been trying the command from yesterday, and still getting errors.
What should I do to solve this problem?
I manage a rails app for a client of mine and recently it went down. The site was down for 9 hours before I noticed. I checked the logs and every request for the past 9 hours is prepended with the following code:
at=error code=H10 desc="App crashed"
Before that, I see the following logs:
2012-11-16T00:55:46+00:00 heroku[web.1]: Idling
2012-11-16T00:55:50+00:00 heroku[web.1]: Stopping all processes with SIGTERM
2012-11-16T00:55:51+00:00 app[web.1]: [2012-11-16 00:55:51] ERROR SignalException: SIGTERM
2012-11-16T00:55:51+00:00 app[web.1]: /usr/local/lib/ruby/1.9.1/webrick/server.rb:90:in `select'
2012-11-16T00:56:00+00:00 heroku[web.1]: Error R12 (Exit timeout) -> At least one process failed to exit within 10 seconds of SIGTERM
2012-11-16T00:56:00+00:00 heroku[web.1]: Stopping remaining processes with SIGKILL
2012-11-16T00:56:02+00:00 heroku[web.1]: State changed from up to down
2012-11-16T00:56:02+00:00 heroku[web.1]: Process exited with status 137
2012-11-16T01:03:55+00:00 heroku[web.1]: Unidling
2012-11-16T01:03:55+00:00 heroku[web.1]: State changed from down to starting
2012-11-16T01:03:59+00:00 heroku[web.1]: Starting process with command `bundle exec rails server -p 4303`
2012-11-16T01:04:00+00:00 heroku[nginx]: 98.139.241.251 - - [16/Nov/2012:01:04:00 +0000] "GET / HTTP/1.1" 499 0 "-" "YahooCacheSystem" domain.com
2012-11-16T01:04:22+00:00 app[web.1]: => Ctrl-C to shutdown server
2012-11-16T01:04:22+00:00 app[web.1]: ** [NewRelic][11/16/12 01:04:21 +0000 b8af98a1-2246-4b34-9dfe-61b9d4b747bc (2)] INFO : Dispatcher: webrick
2012-11-16T01:04:22+00:00 app[web.1]: ** [NewRelic][11/16/12 01:04:21 +0000 b8af98a1-2246-4b34-9dfe-61b9d4b747bc (2)] INFO : Application: acsolar
2012-11-16T01:04:22+00:00 app[web.1]: ** [NewRelic][11/16/12 01:04:21 +0000 b8af98a1-2246-4b34-9dfe-61b9d4b747bc (2)] INFO : New Relic Ruby Agent 3.4.0.1 Initialized: pid = 2
2012-11-16T01:04:22+00:00 app[web.1]: => Booting WEBrick
2012-11-16T01:04:22+00:00 app[web.1]: => Rails 3.1.1 application starting in production on http://0.0.0.0:4303
2012-11-16T01:04:22+00:00 app[web.1]: => Call with -d to detach
2012-11-16T01:04:25+00:00 app[web.1]: [DEPRECATION] Your applications public directory contains an assets/products and/or assets/taxons subdirectory.
2012-11-16T01:04:25+00:00 app[web.1]: Run `rake spree:assets:relocate_images` to relocate the images.
2012-11-16T01:04:34+00:00 app[web.1]: ** [NewRelic][11/16/12 01:04:32 +0000 b8af98a1-2246-4b34-9dfe-61b9d4b747bc (2)] INFO : Reporting performance data every 60 seconds.
2012-11-16T01:04:34+00:00 app[web.1]: Connected to NewRelic Service at collector-5.newrelic.com
2012-11-16T01:05:00+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
2012-11-16T01:05:00+00:00 heroku[web.1]: Stopping process with SIGKILL
2012-11-16T01:05:02+00:00 heroku[web.1]: Process exited with status 137
2012-11-16T01:05:02+00:00 heroku[web.1]: State changed from crashed to down
2012-11-16T01:05:02+00:00 heroku[web.1]: State changed from starting to crashed
I'm guessing that it may have spun down and had an error booting back up, but how come it stayed in the crashed state without restarting itself? Is there anything I can do to have it automatically restart if this happens again in the future?
I've got NewRelic running on this too and it didn't notify me at all, but that's another problem I'll have to investigate.
Heroku's support answer suggests restarting your app manually with heroku restart. They're fixing the issue right now.
Hi, A process management error on our side caused some crashed apps
only running 1 web dyno to be reported as "idle" even though they were
actually crashed. This means that the crashed dyno was never
restarted, causing subsequent requests to fail. We've identified this
problem and are implementing a fix. If your app is still unresponsive,
please try restarting it with the heroku restart command. Please let
us know if you need more help. Thanks, Heroku Support
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
I'm suddenly getting an "Application Error" when I deploy some changes to my rails app on heroku. And not really sure why and I can't find a proper solution.
Here are the logs:
2012-10-22T20:05:26+00:00 heroku[slugc]: Slug compilation started
2012-10-22T20:06:54+00:00 heroku[web.1]: State changed from up to starting
2012-10-22T20:06:54+00:00 heroku[slugc]: Slug compilation finished
2012-10-22T20:06:56+00:00 heroku[web.1]: Starting process with command `bundle e
xec rails server -p 28612`
2012-10-22T20:06:56+00:00 heroku[web.1]: Stopping all processes with SIGTERM
2012-10-22T20:06:57+00:00 app[web.1]: /usr/local/lib/ruby/1.9.1/webrick/server
.rb:90:in `select'
2012-10-22T20:06:57+00:00 app[web.1]: [2012-10-22 20:06:57] ERROR SignalExceptio
n: SIGTERM
2012-10-22T20:07:06+00:00 heroku[web.1]: Stopping remaining processes with SIGKI
LL
2012-10-22T20:07:06+00:00 heroku[web.1]: Error R12 (Exit timeout) -> At least on
e process failed to exit within 10 seconds of SIGTERM
2012-10-22T20:07:08+00:00 heroku[web.1]: Process exited with status 137
2012-10-22T20:07:57+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process
failed to bind to $PORT within 60 seconds of launch
2012-10-22T20:07:57+00:00 heroku[web.1]: Stopping process with SIGKILL
2012-10-22T20:07:58+00:00 heroku[web.1]: State changed from starting to crashed
2012-10-22T20:07:58+00:00 heroku[web.1]: State changed from crashed to starting
2012-10-22T20:07:58+00:00 heroku[web.1]: Process exited with status 137
2012-10-22T20:08:01+00:00 heroku[web.1]: Starting process with command `bundle e
xec rails server -p 26859`
2012-10-22T20:09:01+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process
failed to bind to $PORT within 60 seconds of launch
2012-10-22T20:09:01+00:00 heroku[web.1]: Stopping process with SIGKILL
2012-10-22T20:09:02+00:00 heroku[web.1]: Process exited with status 137
2012-10-22T20:09:02+00:00 heroku[web.1]: State changed from starting to crashed
2012-10-22T20:09:27+00:00 heroku[router]: Error H10 (App crashed) -> GET www.thr
illengine.com/ dyno= queue= wait= service= status=503 bytes=`
In development it works fine.
Update:
40 min later everything is back to normal and working. Strange, considering that I didn't really do nothing.
Anyone else experiencing something similar?
Does the gem file has pg gem?
did u migrate your db on production DB?
does the deploy give any errors?