I have a Rails app running on Heroku that serves as the API for a front-end application.
I noticed that for a specific, dynamic URL, /bands/:band_id/members it consistently throws net::ERR_CONNECTION_CLOSED errors which breaks the app.
That specific URL doesn't throw an error when I run the Rails app locally and other URLs work fine on Heroku so I suspect this is a Heroku error but I'm not sure.
I couldn't get deeper in analyzing the problem as the request doesn't even appear in the Heroku logs.
Setup error monitoring on heorku. There are many addons listed under "Errors and Exceptions" category here - https://elements.heroku.com/addons
Eg. You can try Airbrake or Bugsnag. Most likely error is coming from you application. It's best practise to setup error monitoring but even before that you can check your server logs to debugs the issue - https://devcenter.heroku.com/articles/heroku-cli-commands#heroku-logs
Without more details I'm afraid I can only try to help you troubleshoot. Post as much code as you can. The route, the controller action, the view it's rendering, and any relevant logs from localhost and heroku would be a great start.
I've had Heroku requests timeout on my rails apps many times - in development there is often no time limit but if your request is taking too long that definitely could be the issue. How long does the request take on development? It could be as simple as shaving off a few seconds.
Otherwise I would say to check this out:
Heroku websocket connection
Also be sure to clear everything you can on your browser, try other browsers, incognito mode, all of that. Try to isolate the problem to one area - even though Heroku is throwing the error it is almost certainly not causing the error.
Check your routes. Look at everything that is happening with that request in your dev and prod logs and try and find something different about this request. Compare it to others.
It is also a good idea to understand your logs and increase their verbosity -
https://devcenter.heroku.com/articles/logging
What levels of logging are available for Heroku?
Good luck!
We created a multi-nested form in Rails 5, the forms having image objects, many fields.
after having 200+ items in a single form and trying to save the form, we started getting those errors.
An unhandled lowlevel error occurred. The application logs may have details.
there is nothing logged in the error logs, nginx access logs seems like it got the request, but Puma really didnt like the huge request.
any idea how can we fix this? we need the huge request.
so I ended up taking a copy of the live server database and installing it locally, then I took a look at the console which was logging:
Too many open files - Maximum file multiparts in content reached
googling around for a bit knowing that the multi nested form of mine is the cause of the problem, I decided to increase the rack multi part limit to 512 like so in my config.rb
Rack::Utils.multipart_part_limit = 512
saved the file, uploaded to the server and restarted it. everything works perfectly now, thanks everyone.
I have a Rails 3.2.14 app in production that as designed when an exception is raised, I get the 500 page "Something went wrong". When this happens, an employee will usually call me right away and tell me that an error occurred but this is not always the case. Sometimes I won't know about the error in production until later that day or longer.
What I'd like to do is set up my production environment to where when an exception/error is raised it displays and error page, but sends me an email in the background.
I've done some research and I see there's AirBrake, but I'm really looking to build/implement my own solution instead of paying for a 3rd-party service.
Can anyone suggest a gem or snippet of code that emails when an error/exception is raised and/or gives the end-user the option to submit the error from the 500 page?
The exception notification does exactly this.
You might also be interested in errbit. It implements the same api as airbrake - in a nutshell you'd be hosting your own version of airbrake.
I'm working on a Rails application that works with data via the Shopify API, however it has just started giving me 500 errors when certain resources are requested via a proxy (as set in the app settings in Shopify).
The request is along the lines of:
/app/my-application/customers/1234.json
however the error log on Heroku is showing a GET request to:
/app/my-application/cusotmers,1234.json
I'm using the Shopify/shopify_api gem which was recently updated, otherwise nothing else in the config/routes has changed since this error began occurring.
Any help or pointers greatly appreciated!
I'll happily provide more information if anything relevant is missing above.
This was a bug in Shopify's service to proxy requests to applications.
I have just deployed a fix for the issue. I take full responsibility for the issue, and will try to improve our tests to avoid similar issues in the future.
We're using EngineYard Cloud to deploy our Ruby on Rails application. We are running Rails v2.3.3.
EngineYard Cloud deploys to AWS instances in a manner similar to Capistrano. After each deploy, we're running into Invalid Authenticity Token errors. Specifically, any user that has previously visited our application and then visits after the deploy and then tries to submit a form gets an invalid authenticity token error. This error persists until they reset their cookies for the site. After they reset their cookies, the site works as expected with no errors.
We are using ActiveRecord's session store and sessions are being saved to the database.
This is the error we are seeing:
ActionController::InvalidAuthenticityToken
/usr/lib/ruby/gems/1.8/gems/actionpack-2.3.3/lib/action_controller/request_forgery_protection.rb:79:in `verify_authenticity_token'
The session object is nil after the deploy, however, the session data still persists in the database and the session ID cookie still exists:
Session:
session id: nil
data: nil
We haven't been able to explain this one. Any thoughts on what could be the root cause?
Thanks for any suggestions!
EDIT: Just to update on this, we've been able to isolate an example of the error.
1) User loads form
2) Code is updated on server
3) User submits form
** Invalid Authenticity Token error occurs
It seems that when the environment changes, Rails is unable to handle this with the authenticity token.
We've tried several steps to resolve:
Resetting the session
Deleting the session cookie (both in JavaScript and Rails)
Wiping the session table in the database after deploying code
Nothing works. The only thing that works is having the user clear their cookies client-side.
(We've been Googling (even tried Binging!) for answers, but no dice. This seems to be a similar related issue: http://railsforum.com/viewtopic.php?id=21479)
Also: initially we thought this was isolated to our deployment to EngineYard, but we've also been able to reproduce it on our development server that we deploy to via Capistrano.
Any thoughts would be gratefully accepted.
Thanks!
ANSWER: After extensive work by EngineYard (they're awesome!) they were able to diagnose the issue. The root cause of this issue is a bug with mongrel clusters. Mongrel doesn't seem to see the first post request after being started. EngineYard did extensive work to diagnose this:
There doesn't appear to be anything in your code causing the issue and I have found people outside of our environment that have experienced the bug as well (http://www.thought-scope.com/2009/07/mongrelcluster-rails-23x-bad-post.html). I suppose a lot of people don't see it because the first request to a site generally isn't a post or they chalk it up to flukes.
[There is a potential workaround using CURL.] The curl work around would do a simple GET request to each of your mongrels on the server to prime them so to speak. You could do this with capistrano, but that won't work if you deploy via the dashboard. You can find a short section on deploy hooks we have built into the infrastructure here:
https://cloud-support.engineyard.com/faqs/overview/getting-started-with-engine-yard-cloud
Adding a simple run curl http://localhost:500x > /dev/null should work (where x is the port you have 5000-50005 on your current setup).
We have addressed the issue by switching our stack from Mongrel to Passenger, but apparently, a fix for Mongrel is in the works. Hopefully, this helps someone who sees this same strange issue.
The authenticity token is a hidden field on the form that rails checks when the form is submitted to ensure that the post data is coming from a live session.
It is there as a security measure to prevent malicious people from using a form submit on their site to say a delete action on someones account.
You can turn it off on your whole app by adding this to config/environment.rb
config.action_controller.allow_forgery_protection = false
You can turn it off a single controller using
skip_before_filter :verify_authenticity_token
or turn it on
protect_from_forgery :except => :index
check out the ActionController::RequestForgeryProtection::ClassMethods docs for more details
It sounds like the secret key used for authentication is changing when you redeploy, invalidating all existing sessions.
Do you have the configuration parameter config.action_controller.session set anywhere, and if you do, is there anything which would cause it to change when you redeploy?
One of my apps has it configured in config/environment.rb, and a more recent one (generated with Rails 2.3) has it set in config/initializers/session_store.rb. The setting looks like:
config.action_controller.session = {
:secret => 'long-string-of-hex-digits'
}
If you don't have this configured for some reason, rake secret will generate a key for you, which can then be inserted into your configuration.
(If it is — and it's not being changed by your deployment processes — then I have no idea what's going on.)
If it would only be there for mongrels! I'm getting the exact same error on passenger as well (user loads form, deploy, submit -> invalid authenticity token). It'd be interesting to know how you solved the issue by switching to passenger? Any further hints are highly welcome. I'll have a closer look as well...
Cheers!
Have encountered this same problem with Rails 2.3 and a Mongrel cluster where the session secret is definitely set in the session initializer. The problem occured even after clearing the client cookies on the client.
However the suggestion of doing a curl get request across all the mongrels after they restart appears to work - thank goodness someone figured this out because it appears to be pretty darned obscure.
The only added info I can supply we are using Apache mod_proxy_balancer along with https in front of our Mongrels, however this problem was occuring before we turned on SSL. Is anyone seeing this with haproxy as the balancer instead of Apache?
This solved this issue for me :-) :-) :-)
https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/4690-mongrel-doesnt-work-with-rails-238#ticket-4690-37 Posted by Mike Bethany
August 30th, 2010 # 06:43 PM.
I've never gone to any length to figure out the details, but for me, this is a client-side data rot issue. If I've been messing around with the way I store my sessions (and therefore, my authorization details,) I get this error from time to time. Clearing out the private browser data; cookies, authenticated sessions, the works, has always solved it for me.
Hope this helps.