I'm running the Exception Notification gem on Rails 5. I have it setup the default way in config/environments/production.rb:
Rails.application.config.middleware.use ExceptionNotification::Rack,
email: {
# deliver_with: :deliver, # Rails >= 4.2.1 do not need this option since it defaults to :deliver_now
email_prefix: '[MFTA Error Notification] ',
sender_address: %{"notifier" <almost#got.me>},
exception_recipients: %w{butnotquite#gmail.com}
}
This works fine for standard errors when the site is up...
But shouldn't it send me a report on 500 server errors a well? Very randomly... about once a month or so... the rails app will crash on me and I'll need to redeploy it to get it to work again. But I won't even know that the site's down without a notification.
So is there some separate config... or even another Gem... to let me know when this happens?
Since your app is hosted in aws, you can setup a healthcheck endpoint in your app and use a lambda function to ping it periodically. If there’s no 200 response, its very likely your app is down as serving a healthcheck is a dead simple thing which shouldnt fail.
Normally people would set a threshold say X consecutive health check failure within Y duration to verify that the app is down. But this would require your lambda function to be stateful. If you dont mind getting false alarm due to say deployment or server restart, you can forget about this.
Also, if you want the health check to be more performant, you can just implement your rack middleware to intercept this healthcheck request and return a 200 response. In that sense teh request doesnt has to go through all the stacks until it reach Rails
Related
I have a Rails app running on Heroku that serves as the API for a front-end application.
I noticed that for a specific, dynamic URL, /bands/:band_id/members it consistently throws net::ERR_CONNECTION_CLOSED errors which breaks the app.
That specific URL doesn't throw an error when I run the Rails app locally and other URLs work fine on Heroku so I suspect this is a Heroku error but I'm not sure.
I couldn't get deeper in analyzing the problem as the request doesn't even appear in the Heroku logs.
Setup error monitoring on heorku. There are many addons listed under "Errors and Exceptions" category here - https://elements.heroku.com/addons
Eg. You can try Airbrake or Bugsnag. Most likely error is coming from you application. It's best practise to setup error monitoring but even before that you can check your server logs to debugs the issue - https://devcenter.heroku.com/articles/heroku-cli-commands#heroku-logs
Without more details I'm afraid I can only try to help you troubleshoot. Post as much code as you can. The route, the controller action, the view it's rendering, and any relevant logs from localhost and heroku would be a great start.
I've had Heroku requests timeout on my rails apps many times - in development there is often no time limit but if your request is taking too long that definitely could be the issue. How long does the request take on development? It could be as simple as shaving off a few seconds.
Otherwise I would say to check this out:
Heroku websocket connection
Also be sure to clear everything you can on your browser, try other browsers, incognito mode, all of that. Try to isolate the problem to one area - even though Heroku is throwing the error it is almost certainly not causing the error.
Check your routes. Look at everything that is happening with that request in your dev and prod logs and try and find something different about this request. Compare it to others.
It is also a good idea to understand your logs and increase their verbosity -
https://devcenter.heroku.com/articles/logging
What levels of logging are available for Heroku?
Good luck!
I understand that sometimes a client will show an HTTP request as returning a 0 status code when the request fails to connect, timesout, etc, but I have never seen a server logging 0 as what it is sending back to the client.
I am running a Rails 4.2, ruby 2.2.x API. I was analyzing our logs the other day and noticed that a non-trivial number of requests were being responded with an HTTP status code of 0 by our Rails API. I have been unable to figure out why.
In some cases, it appears the request never makes it to the rails app. I only see the log message which I believe is logged by rack as to the request path and status returned. In other cases, I can see one of the early log messages we log from our application controller.
Has anyone seen such a behavior? I am not sure how to debug further without beginning to modify the standard rack middlewares that rails provides. I am not able to reproduce the situation myself; I only see this sporadically in our logs.
A bit more about our stack:
Rails 4.2.5
Ruby 2.2.3
Puma 3.4.0
We are reverse proxying with nginx, but I dont think is effecting it since the request is received by rack at least.
I realize it would probably be impossible to answer what exactly is happening here, so I am hoping instead for suggestions on how best to troubleshoot this.
Found that this is an issue with our Rails logging setup. By correlating these logs to our nginx access logs, I can see that a 302 is actually being returned to the client. This was commonly happening when the CSRF protection failed.
It looks like this is caused by a lograge (gem) bug: https://github.com/roidrage/lograge/issues/67
I have seen this behavior when requests were sent from different subdomains that were not allowed based on Access-Control-Allow-Origin. So maybe that's one possibility
I am using Nginx, passenger, and Rails. The rails app is caching responses for HTTP GET requests. Is there a way to disable this? If so, where and how do I do this?
The GET request returns the same response every time. When I run rails locally, without nginx or passenger, on development mode, I don't get this issue. It's only in the production environment that it returns the same response every time the call is made. If the server is reset, it returns a different value.
Generally it's a bad idea to disable caching, but if you want to so.
in rails app in production.rb file, set config.action_controller.perform_caching to false
My situation is like this:
1. User uploads 150MB zip file, with 600 files inside. It takes 4 minutes or so to upload the file to the server.
2. Server processes the file contents, takes 70 seconds or so.
3. The server responds with Service Unavailable, with a log like, "could not forward the response to the client... stop button was clicked"
4. The Rails application log says, 200 OK response was returned.
So, I am guessing it must be a problem within one of Nginx or Passenger that is causing it return with the error even thought it is going fine inside the Rails app. My suspect is a timeout setting, because I could reproduce it by just putting a sleep of 180 seconds inside the long running method and doing nothing.
I will appreciate if you guys know what specific nginx/passenger config may fix it.
If you're using S3 as your storage you may consider using something like carrierwave_direct to skip passing the file through the web server and instead upload directly to S3.
Like noted above you could incorporate a queueing process like delayed_job.
https://github.com/dwilkie/carrierwave_direct
I presume that nginx is the public-facing server and it proxies requests through to another server running your RoR application for you. If this assumption is correct, you may need to increase the value of your nginx proxy_read_timeout setting for the specific locations that are causing you trouble.
For long run request, I think you should return an 'please wait' page immediately and make the processing background. After the processing is completed, set the task in the database as 'completed'. Within the period, whenever user refresh the page, return 'please wait' immediately. After completed, return the result. You can set an autorefresh timeout in the page to refresh the page after an estimated period.
I'd instantly store the upload somewhere and redirect to a "please wait" page which asks for the status of the background processing and could even display some progress bar then, e.g. using ajax.
For the actual background processing I'd recommend DelayedJob which worked great for us and supports easy job deployment and implementation.
How can I disable the console output on a Rails 3 application?
More specifically, I want to disable at least the Mailer output, that outputs the entire email content, including the pictures, making the action processing much slower (it takes almost 10sec to send an email).
ps: I think the slowdown is because of the output, if it can be from another source, such as slow smtp server (it's gmail atm, so no.) or something else like that please let me know.
By this you mean you want to hide the output shown in the console run you run rails s (or script/server in rails 2)?
Are you on Linux or OSX?
If so, then just do the following
$ rails server 1> /dev/null
this sends all output from stdout into a blackhole.
So right now you are trying to send emails from your dev machine?
I try to avoid this,
as accidents are going to happen
and you'll send clients test data.
Try Mailcatcher
http://mailcatcher.me/
It lets you catch all the emails your app would be sending
shows them off in a nice web interface
and importantly avoids the risk of accidentally sending real emails to customers with random test data.
SMTP server's (even Gmail) response can really take some time. You'd rather use a mail queue that stores all emails in a database and then they are sent by an independent process.
E.g. https://github.com/beam/action-mailer-queue
Concerning logger - make sure that your logging level is :error or :fatal. If not, run:
config.log_level = :error
config.logger = Logger.new('log/development.log')