Heroku devise/facebook oauth glitch with identical source code - ruby-on-rails

I try to solve problems myself, but this one has got the best of me and I'm on the verge of head-butting my desk.
Here's what I do:
I test the production oAuth, all works perfectly as it has for months
I use heroku git:clone -a myapp to get the source code locally
I make no changes whatsoever, I simply re-commit and push back up to Heroku
I test the production oAuth again, it doesn't work
I keep having to use heroku rollback, so that I can go back to the working version. I thought that since the source code is identical, maybe a gem I'm using has updated and broken something. But I checked the oAuth gems and none of them have changed for a while.
Here's my log when it doesn't work:
2015-10-25T15:20:47.802222+00:00 app[web.1]: Started GET "/humans/auth/facebook?ip=123&loc=8&mac=123&var=11" for 162.158.38.218 at 2015-10-25 15:20:47 +0000
2015-10-25T15:20:47.804207+00:00 app[web.1]: I, [2015-10-25T15:20:47.804128 #3] INFO -- omniauth: (facebook) Request phase initiated.
2015-10-25T15:20:47.871762+00:00 heroku[router]: at=info method=GET path="/humans/auth/facebook?ip=123&loc=8&mac=123&var=11" host=yourhotspot.net request_id=9804b0a9-a190-4881-a64f-ff222aa216e3 fwd="82.34.163.154,162.158.38.218" dyno=web.1 connect=0ms service=75ms status=302 bytes=1916
2015-10-25T15:20:48.181538+00:00 app[web.1]: Started GET "/humans/auth/facebook/callback?ip=123&loc=8&mac=123&var=11&code=AQCqfCPHQUM_YzgXPJSUniVKumx6ii2fsQtgEw9Bc3B9MFwNyKmnRtdqZxCRWSl1bDxdZKIAgU_M_-DQGVedt7FRYsm8DxA7gPA8bky86H-jlaCaFQa1Gc2BRLxtuzEUPKtu3UPRmKjUpXBuJI0nHOmG4IF3WWjwRk1jtfvWvgHmEH5Q-h-JjoEgiTC5uqK6XN2E48hKqxYcqBVs-pDMLMp74WRmRCXN_R_R_RuAjyyj_9-GdNHhZ6uajuri_41AN5V8W9UqWPmtPHznL7ztNLzS9dDj7S9e25bINXUMneHVVNHwGpp5PWZULsN_hwJueUU4SFZlNgzrnupacHAXF9kU&state=1e8f5be4b10b96baedd6aab3cd2eb125b150e100c5a2c7bb" for 162.158.38.218 at 2015-10-25 15:20:48 +0000
2015-10-25T15:20:48.183305+00:00 app[web.1]: I, [2015-10-25T15:20:48.183218 #3] INFO -- omniauth: (facebook) Callback phase initiated.
2015-10-25T15:20:48.733053+00:00 app[web.1]: {"error":{"message":"Error validating verification code. Please make sure your redirect_uri is identical to the one you used in the OAuth dialog request","type":"OAuthException","code":100,"fbtrace_id":"B8bmqcG7i6M"}}
2015-10-25T15:20:48.733049+00:00 app[web.1]: E, [2015-10-25T15:20:48.732948 #3] ERROR -- omniauth: (facebook) Authentication failure! invalid_credentials: OAuth2::Error, :
2015-10-25T15:20:48.735557+00:00 app[web.1]: Parameters: {"ip"=>"123", "loc"=>"8", "mac"=>"123", "var"=>"11", "code"=>"AQCqfCPHQUM_YzgXPJSUniVKumx6ii2fsQtgEw9Bc3B9MFwNyKmnRtdqZxCRWSl1bDxdZKIAgU_M_-DQGVedt7FRYsm8DxA7gPA8bky86H-jlaCaFQa1Gc2BRLxtuzEUPKtu3UPRmKjUpXBuJI0nHOmG4IF3WWjwRk1jtfvWvgHmEH5Q-h-JjoEgiTC5uqK6XN2E48hKqxYcqBVs-pDMLMp74WRmRCXN_R_R_RuAjyyj_9-GdNHhZ6uajuri_41AN5V8W9UqWPmtPHznL7ztNLzS9dDj7S9e25bINXUMneHVVNHwGpp5PWZULsN_hwJueUU4SFZlNgzrnupacHAXF9kU", "state"=>"1e8f5be4b10b96baedd6aab3cd2eb125b150e100c5a2c7bb"}
2015-10-25T15:20:48.735493+00:00 app[web.1]: Processing by Humans::OmniauthCallbacksController#failure as HTML
And when it does:
2015-10-25T15:56:56.726873+00:00 heroku[router]: at=info method=GET path="/humans/auth/facebook?ip=123&loc=8&mac=123&var=11" host=yourhotspot.net request_id=1c01da5f-1bea-4dbb-9a1b-a80dcc442539 fwd="82.34.163.154,162.158.38.211" dyno=web.1 connect=1ms service=18ms status=302 bytes=1828
2015-10-25T15:56:56.714754+00:00 app[web.1]: I, [2015-10-25T15:56:56.714674 #3] INFO -- omniauth: (facebook) Request phase initiated.
2015-10-25T15:56:56.920771+00:00 app[web.1]: I, [2015-10-25T15:56:56.920692 #3] INFO -- omniauth: (facebook) Callback phase initiated.
2015-10-25T15:56:56.712897+00:00 app[web.1]: Started GET "/humans/auth/facebook?ip=123&loc=8&mac=123&var=11" for 162.158.38.211 at 2015-10-25 15:56:56 +0000
2015-10-25T15:56:56.919315+00:00 app[web.1]: Started GET "/humans/auth/facebook/callback?code=AQC7jHL3hk8TSP5F9lsDjzkyXlAIbk-Kf3027g3bU-9x0etBoOxUvHd6ofEIZjEolh-UajkUCS4WZ3z40q7tgrIYi8ZIy6fuhjRJwyqLhXBPfOnmL1LqCDlPFVIdizrHPUomZ5WZ-VLmFc7A1UsEUVpyW907LgoaL8S90Gukh9oGvykIcpc1tmHqpc-WaayJNCkTUPxaRtlEmuD5FTVUDoRAzFnVDYCURhNAZMs9eTk-O40_leapHbTX0Uz6sFC6RHn-qgien_diOJst2DlRvtKWCcfxXuoBZPFPa5BtRYg2_eP4K8zUxRtDeazwYsxJt7o1i9V93np56EABMMrfBmIK&state=952e4d3cbe773056b12f56d76b0a54c5ea841f51e61d5041" for 162.158.38.211 at 2015-10-25 15:56:56 +0000
2015-10-25T15:56:57.897756+00:00 app[web.1]: Processing by Humans::OmniauthCallbacksController#facebook as HTML
I just cannot wrap my head around what's going on here. The fact that my code, environments, config vars and app settings are identical is hurting my head.
If anyone can shed some light on what might be going wrong, I'd be very appreciative.
EDIT: (Commands used)
heroku git:clone -a myapp (At this point of cloning, I had already rolled back to v410, so this clone should be of that working copy)
(cd into new app directory, make a line
change so that I can re-commit)
git add .
git commit . -m "msg"
git push heroku master
(realise it's not working as expected)
heroku
rollback v410
Command logs gave no errors and the commands worked as expected. Yet something is changing oauth.

What definitely could change on deployment is a physical server ('dyno'). I would check following things: 1) does your app cache any authentication tokens issued by facebook, e.g. in configuration? 2) how possible is that such tokens are bound to the server's IP address?

Related

How to use goaccess with heroku rails app

I want to generate goaccess dashboard result from heroku logs of rails application, As I checked there is no production.rb file in heroku
Format of logs in heroku is
2020-07-16T17:18:54.972063+00:00 app[web.1]: I, [2020-07-16T17:18:54.971939 #10] INFO -- : [2712edd9-bca1-4d23-9cfd-399e6dedfbdc] Started GET "/api/v1/articles" for 18.225.35.172 at 2020-07-16 17:18:54 +0000
2020-07-16T17:18:54.973433+00:00 app[web.1]: I, [2020-07-16T17:18:54.973345 #10] INFO -- : [2712edd9-bca1-4d23-9cfd-399e6dedfbdc] Processing by Api::V1::ArticlesController#index as JSON
2020-07-16T17:18:54.973525+00:00 app[web.1]: I, [2020-07-16T17:18:54.973450 #10] INFO -- : [2712edd9-bca1-4d23-9cfd-399e6dedfbdc] Parameters: {"article"=>{}}
2020-07-16T17:18:55.082811+00:00 heroku[router]: at=info method=GET path="/api/v1/article" host=abc-1234.herokuapp.com request_id=2779d556-67fc-4602-a58b-60dbcb209833 fwd="18.225.35.172" dyno=web.1 connect=1ms service=41ms status=200 bytes=539 protocol=https
I tried installing goaccess on my heroku server , but I am not able to do that, and I am not able to get log file also.
I want response time for each request in goaccess dashboard.
Anyone has idea about this ???How I can use goaccess + rails + heroku together
Try this:
cat ./production.log | grep 'Started GET' | goaccess --log-format='I, "%^" "%r" %^ %h %^ %d %t %^' --date-format=%F --time-format=%T
Note: it will work only for line like:
2020-07-16T17:18:54.972063+00:00 app[web.1]: I, [2020-07-16T17:18:54.971939 #10] INFO -- : [2712edd9-bca1-4d23-9cfd-399e6dedfbdc] Started GET "/api/v1/articles" for 18.225.35.172 at 2020-07-16 17:18:54 +0000
other lines were ignored by grep, which were:
2020-07-16T17:18:54.973433+00:00 app[web.1]: I, [2020-07-16T17:18:54.973345 #10] INFO -- : [2712edd9-bca1-4d23-9cfd-399e6dedfbdc] Processing by Api::V1::ArticlesController#index as JSON
2020-07-16T17:18:54.973525+00:00 app[web.1]: I, [2020-07-16T17:18:54.973450 #10] INFO -- : [2712edd9-bca1-4d23-9cfd-399e6dedfbdc] Parameters: {"article"=>{}}
2020-07-16T17:18:55.082811+00:00 heroku[router]: at=info method=GET path="/api/v1/article" host=abc-1234.herokuapp.com request_id=2779d556-67fc-4602-a58b-60dbcb209833 fwd="18.225.35.172" dyno=web.1 connect=1ms service=41ms status=200 bytes=539 protocol=https
My original answer was at goaccess issue, Good luck!

rails-latex gem: ActionView::Template::Error (rails-latex failed ...) with Heroku

Rails Version: Rails 6.0.2.1
Ruby Version: ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
When I try to generate a PDF with the gem rails-latex in Heroku, I get the following error: ActionView::Template::Error rails-latex failed (Full error message below).
This error does not arise locally.
What I tried
There is already a stackoverflow-question with the same problem: Debugging rails-latex gem on Heroku | ActionView::Template::Error rails-latex failed
#blackbrokkoli writes:
If you have a similar problem, I recommend deleting all tex
dependencies, get a MWE and then slowly add them back in to identify
the evildoers.
But I don't know what he means with "MWE" and I doubt that I am able to "identify the evildoers" by myself.
There is a thread called "Deploying on Heroku" suggesting that one needs to install a buildpack.
I tried to follow the instructions for the "Multipacks"-option.
The command cat .buildpacks returns
cat: .buildpacks: No such file or directory
The command heroku config:add BUILDPACK_URL=git://github.com/ddollar/heroku-buildpack-multi.git returns:
Setting BUILDPACK_URL and restarting
BUILDPACK_URL: git://github.com/ddollar/heroku-buildpack-multi.git
But after that, I don't know how to proceed further.
I tried to figure out how to add buildpacks with Heroku with the heroku buildpacks:add command. But I couldn't figure out, how I can add a third-party buildpack with this command.
Does anyone have any suggestions on how to fix this error?
Error message in heroku logs
2020-02-07T05:50:55.909360+00:00 app[web.1]: I, [2020-02-07T05:50:55.909264 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Started GET "/articles/2.pdf" for 87.245.109.31 at 2020-
02-07 05:50:55 +0000
2020-02-07T05:50:55.910207+00:00 app[web.1]: I, [2020-02-07T05:50:55.910130 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Processing by ArticlesController#show as PDF
2020-02-07T05:50:55.910257+00:00 app[web.1]: I, [2020-02-07T05:50:55.910202 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Parameters: {"id"=>"2"}
2020-02-07T05:50:55.913042+00:00 app[web.1]: D, [2020-02-07T05:50:55.912964 #4] DEBUG -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Article Load (1.1ms) SELECT "articles".* FROM "articl
es" WHERE "articles"."id" = $1 LIMIT $2 [["id", 2], ["LIMIT", 1]]
2020-02-07T05:50:55.914866+00:00 app[web.1]: I, [2020-02-07T05:50:55.914789 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Rendering articles/show.pdf.erb within layouts/applica
tion
2020-02-07T05:50:55.915357+00:00 app[web.1]: I, [2020-02-07T05:50:55.915290 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Rendered articles/show.pdf.erb within layouts/applicat
ion (Duration: 0.4ms | Allocations: 101)
2020-02-07T05:50:55.916214+00:00 app[web.1]: I, [2020-02-07T05:50:55.916141 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Running 'pdflatex -halt-on-error -shell-escape -interact
ion=batchmode input' in /app/tmp/rails-latex/4-521112460259582216 1 times...
2020-02-07T05:50:55.923855+00:00 app[web.1]: I, [2020-02-07T05:50:55.923739 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Running 'bibtex input' in /app/tmp/rails-latex/4-5211124
60259582216 1 times...
2020-02-07T05:50:55.929983+00:00 app[web.1]: I, [2020-02-07T05:50:55.929846 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Running 'pdflatex -halt-on-error -shell-escape -interact
ion=batchmode input' in /app/tmp/rails-latex/4-521112460259582216 2 times...
2020-02-07T05:50:55.936216+00:00 app[web.1]: I, [2020-02-07T05:50:55.936104 #4] INFO -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] Completed 500 Internal Server Error in 26ms (ActiveRecor
d: 1.1ms | Allocations: 1610)
2020-02-07T05:50:55.937843+00:00 app[web.1]: F, [2020-02-07T05:50:55.937756 #4] FATAL -- : [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3]
2020-02-07T05:50:55.937845+00:00 app[web.1]: [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] ActionView::Template::Error (rails-latex failed: See /app/tmp/rails-latex/4-521112460259582216/input.l
og for details):
2020-02-07T05:50:55.937845+00:00 app[web.1]: [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] 15:
2020-02-07T05:50:55.937846+00:00 app[web.1]: [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] 16: \end{document}
2020-02-07T05:50:55.937846+00:00 app[web.1]: [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3]
2020-02-07T05:50:55.937847+00:00 app[web.1]: [1c4e2355-293c-4cb1-b591-b97e4ba4ecf3] app/views/layouts/application.pdf.erbtex:18
#Syed Asim Hashmi suggested to use this command to install a third-party buildpack:
heroku buildpacks:add git://github.com/Thermondo/heroku-buildpack-tex.git#VERSION_NUMBER
This is what I get:
heroku buildpacks:add git://github.com/Thermondo/heroku-buildpack-te
x.git#VERSION_NUMBER
› Error: Could not find the buildpack:
› git://github.com/Thermondo/heroku-buildpack-tex.git#VERSION_NUMB
› ER. To specify a buildpack, please format it like the following:
› namespace/name (e.g. heroku/ruby). Also names can only contain
› letters, numbers, '_', and '-'.
I tried to add the buildpack via the settings on the heroku website:
I restarted heroku with heroku restart
but I receive the same error message.
rails-latex uses third-party buildpack
https://github.com/Thermondo/heroku-buildpack-tex
You can install it by running this command:
heroku buildpacks:add git://github.com/Thermondo/heroku-buildpack-tex.git#VERSION_NUMBER
Alternatively you can also add the above git repository into your buildpacks by going to your dyno settings.
One thing to keep in mind is that The Heroku filesystem is ephemeral -
that means that any changes to the filesystem whilst the dyno is
running only last until that dyno is shut down or restarted
You can find more information regarding third party buildpacks here
More information regarding heroku filesystem here
Hope it helps.

Rails/net-ssh - ssh to remote server with private key in Heroku

I'm trying to ssh to a remote server using a private key through Heroku.
I'm using this gem https://github.com/net-ssh/net-ssh to complete the request and have everything working fine locally.
The reason why everything works fine locally is because I have the private key saved with the passphrase entered. When I deploy to Heroku, I'm unable to use the agent like I have it on my PC and so the connection fails.
I have changed the settings on the connection using net-ssh to not use the agent but still can't get it functioning.
Worst case I will have to create a separate API call in php hosted elsewhere to complete this connection but I'm hoping I can make it work through my rails app in Heroku.
This is what I'm currently doing:
options = {}
options[:key_data] = ENV['LANDING_PAGE_SSH_KEY']
options[:keys_only] = true
options[:use_agent] = false
options[:verbose] = :debug
options[:port] = '18765'
options[:passphrase] = ENV['SSH_KEY_PASSPHRASE']
Net::SSH.start(ENV['SERVER_HOSTNAME'], ENV['SERVER_USERNAME'], options) do |ssh|
end
I've entered the private key in a global variable and everything else that I would need. This also works locally without using the agent but fails on Heroku with this message:
[2017-07-21T19:20:35.506794 #4] DEBUG -- socket[383b2bc]: read 52 bytes
2017-07-21T19:20:35.507070+00:00 app[web.1]: D, [2017-07-21T19:20:35.507016 #4] DEBUG -- socket[383b2bc]: received packet nr 3 type 6 len 28
2017-07-21T19:20:35.507262+00:00 app[web.1]: D, [2017-07-21T19:20:35.507210 #4] DEBUG -- net.ssh.authentication.session[3fd3f610f914]: trying none
2017-07-21T19:20:35.507477+00:00 app[web.1]: D, [2017-07-21T19:20:35.507412 #4] DEBUG -- socket[383b2bc]: queueing packet nr 4 type 50 len 44
2017-07-21T19:20:35.507603+00:00 app[web.1]: D, [2017-07-21T19:20:35.507553 #4] DEBUG -- socket[383b2bc]: sent 68 bytes
2017-07-21T19:20:35.536336+00:00 app[web.1]: D, [2017-07-21T19:20:35.536194 #4] DEBUG -- socket[383b2bc]: read 52 bytes
2017-07-21T19:20:35.536472+00:00 app[web.1]: D, [2017-07-21T19:20:35.536422 #4] DEBUG -- socket[383b2bc]: received packet nr 4 type 51 len 28
2017-07-21T19:20:35.536573+00:00 app[web.1]: D, [2017-07-21T19:20:35.536526 #4] DEBUG -- net.ssh.authentication.session[3fd3f610f914]: allowed methods: publickey
2017-07-21T19:20:35.536657+00:00 app[web.1]: D, [2017-07-21T19:20:35.536612 #4] DEBUG -- net.ssh.authentication.methods.none[3fd3f6103e84]: none failed
2017-07-21T19:20:35.536749+00:00 app[web.1]: D, [2017-07-21T19:20:35.536703 #4] DEBUG -- net.ssh.authentication.session[3fd3f610f914]: trying publickey
2017-07-21T19:20:35.537566+00:00 app[web.1]: Enter passphrase for <key in memory>:Completed 500 Internal Server Error in 337ms (ActiveRecord: 1.4ms)
2017-07-21T19:20:35.538404+00:00 app[web.1]:
2017-07-21T19:20:35.538406+00:00 app[web.1]: Errno::ENOTTY (Inappropriate ioctl for device):
Any suggestions?
In my case I needed to add the private key to the server's ssh setup
ssh-add ~/.ssh/path-to-private-key
and it started working.

heroku[router] gets old and wrong url

I have a rails app deployed on heroku, and there is an annoying issue: my router heroku gets an url that doesn't exist anymore (I have deleted everything on my WebApp regarding the notifications).
Do you have any idea where it could come from?
Here are what my logs :
2015-06-25T12:22:30.357205+00:00 heroku[router]: at=info method=GET path="/notifications" host=www.krawd.com request_id=6b03fbec-88ee-48bd-8afd-aa6d59a9bf53 fwd="82.237.217.103" dyno=web.1 connect=1ms service=12ms status=500 bytes=377
2015-06-25T12:22:30.334224+00:00 app[web.3]: source=rack-timeout id=8b3b9046-8613-44fb-8274-c0dc976d3472 wait=16ms timeout=25000ms state=ready
2015-06-25T12:22:30.362821+00:00 app[web.3]: source=rack-timeout id=8b3b9046-8613-44fb-8274-c0dc976d3472 wait=16ms timeout=25000ms service=29ms state=active
2015-06-25T12:22:30.406680+00:00 app[web.3]: Completed 500 Internal Server Error in 25ms
2015-06-25T12:22:30.341720+00:00 app[web.1]: source=rack-timeout id=6b03fbec-88ee-48bd-8afd-aa6d59a9bf53 wait=0ms timeout=25000ms service=0ms state=active
2015-06-25T12:22:30.344956+00:00 app[web.1]: ActionController::RoutingError (No route matches [GET] "/notifications"):
Make sure you search your web app folder for "/notifications" and see if you forgot some reference somewhere. Does the request happen when you run on localhost? You could trace it more easily that way.

Intermittent timeouts from an app that should never timeout (NOT an "R12 Request Timeout" error)

I've got a simple business Rails 3.2.13 app with a base of only one user, with very small Postgres DB tables, and very simple Active Record DB queries. I'm running it on two dynos, zero workers, on the Unicorn web server. The root page of the app typically loads in about 150 ms in development, and in under 1 s in production at Heroku.
EXCEPT, when it randomly times out. I'm seeing timeouts in the log, but NOT the typical R12 Request Timeout. The timeouts look like this:
2013-05-28T16:04:28.004290+00:00 app[web.2]: Started GET "/" for 72.28.209.197 at 2013-05-28 16:04:28 +0000
2013-05-28T16:04:44.225643+00:00 app[web.2]: E, [2013-05-28T16:04:44.225444 #2] ERROR -- : worker=2 PID:87 timeout (16s > 15s), killing
2013-05-28T16:04:44.238422+00:00 heroku[router]: at=error code=H13 desc="Connection closed without response" method=GET path=/ host=[REDACTED].herokuapp.com fwd="72.28.209.197" dyno=web.2 connect=7ms service=16240ms status=503 bytes=0
2013-05-28T16:04:44.255813+00:00 app[web.2]: E, [2013-05-28T16:04:44.255554 #2] ERROR -- : reaped #<Process::Status: pid 87 SIGKILL (signal 9)> worker=2
2013-05-28T16:04:44.298023+00:00 app[web.2]: I, [2013-05-28T16:04:44.297643 #103] INFO -- : worker=2 ready
When that request works, it looks like this:
2013-05-28T16:03:36.965507+00:00 app[web.1]: sequenceId="102642"] Started GET "/" for 72.28.209.197 at 2013-05-28 16:03:36 +0000
2013-05-28T16:03:37.388261+00:00 app[web.1]: sequenceId="102690"] Processing by OverviewController#index as HTML
2013-05-28T16:03:38.238877+00:00 app[web.1]: sequenceId="102707"] Rendered overview/index.html.erb within layouts/application (315.8ms)
2013-05-28T16:03:38.311974+00:00 app[web.1]: sequenceId="102710"] Completed 200 OK in 924ms (Views: 432.4ms | ActiveRecord: 92.1ms)
2013-05-28T16:03:38.322786+00:00 heroku[router]: at=info method=GET path=/ host=[REDACTED].herokuapp.com fwd="72.28.209.197" dyno=web.1 connect=2ms service=1375ms status=200 bytes=136795
2013-05-28T16:03:42.625787+00:00 app[web.1]: sequenceId="102940"] Started GET "/assets/favicon.ico" for 72.28.209.197 at 2013-05-28 16:03:42 +0000
2013-05-28T16:03:58.753408+00:00 app[web.1]: sequenceId="104359"] E, [2013-05-28T16:03:58.753173 #2] ERROR -- : worker=2 PID:66 timeout (16s > 15s), killing
2013-05-28T16:03:58.801152+00:00 app[web.1]: sequenceId="104362"] E, [2013-05-28T16:03:58.800993 #2] ERROR -- : reaped #<Process::Status: pid 66 SIGKILL (signal 9)> worker=2
2013-05-28T16:03:58.798722+00:00 heroku[router]: at=error code=H13 desc="Connection closed without response" method=GET path=/assets/favicon.ico host=[REDACTED].herokuapp.com fwd="72.28.209.197" dyno=web.1 connect=3ms service=16175ms status=503 bytes=0
2013-05-28T16:03:59.127150+00:00 app[web.1]: sequenceId="104397"] I, [2013-05-28T16:03:59.126795 #84] INFO -- : worker=2 ready
Things to note: The request response time was 924 ms. But then some kind of strange error happened after that.
This is what that request looks like in terms of DB queries. There is nothing exciting happening there, and those DB tables only contain a few hundred rows of data. There is no reason why the response should take 1 second sometimes, and 15+ seconds on other times. Here's the code for the extremely simple controller action:
class OverviewController < ApplicationController
def index
#current_event = Event.recent
#areas = Area.order(:position)
#reservations = Reservation.find(:all,
:conditions => { :event_id => #current_event.id },
:order => 'created_at DESC',
:limit => 5)
end
end
Here's the Event#recent method, just so that you can see that it's not doing anything that could potentially take 15+ seconds:
def self.recent
Event.last || Event.create
end
I'm at a loss for how to get this app stabilized, other than moving it off of Heroku. I don't know of any way to troubleshoot the problem, and I guess Heroku won't help. Am I simply stuck? Do I have any options for troubleshooting the problem other than moving the app to Engine Yard?
Nothing here helped, and this site is Heroku's technical support. Paying $35/mo doesn't buy me any support, so I'm stuck and there is no way forward with Heroku.
So my answer to this problem is to move the app to Engine Yard. I don't see that I have any other options. Heroku is awesome when it works and it's a complete mystery with no technical support when it doesn't work.

Resources