I have a piece of Rack middleware that loads a tenant, via subdomain, and applies some default settings. The middleware, while not pretty, does it's job fine enough. However, when an exception is thrown within the app the middleware "traps" the full stack trace. When I say trap I mean it hides the expected stack trace.
Here is an example.
I am throwing an exception in an a controller action like so:
def index
throw "Exception in a Rails controller action"
#taxonomies = Spree::Taxonomy.all
end
You would expect that the stack trace would reference this location but it does not. Instead it reference a line in the middleware.
Completed 500 Internal Server Error in 139ms
UncaughtThrowError (uncaught throw "Exception in a Rails controller action"):
lib/tenant_manager/middleware/loader.rb:42:in `call'
Why does this happen? Have you seen anything like this before?
Here is the middleware:
# lib/tenant_manager/middleware/loader.rb
module TenantManager
module Middleware
class Loader
# Middleware to detect an tenant via subdomain early in
# the request process
#
# Usage:
# # config/application.rb
# config.middleware.use TenantManager::Middleware::Loader
#
# A scaled down version of https://github.com/radar/houser
def initialize(app)
#app = app
end
def call(env)
domain_parts = env['HTTP_HOST'].split('.')
if domain_parts.length > 2
subdomain = domain_parts.first
tenant = Leafer::Tenant.find_by_database(subdomain)
if tenant
ENV['CURRENT_TENANT_ID'] = tenant.id.to_s
ENV['RAILS_CACHE_ID'] = tenant.database
Spree::Image.change_paths tenant.database
Apartment::Tenant.process(tenant.database) do
country = Spree::Country.find_by_name('United States')
Spree.config do |config|
config.default_country_id = country.id if country.present?
config.track_inventory_levels = false
end
Spree::Auth::Config.set(:registration_step => false)
end
end
else
ENV['CURRENT_TENANT_ID'] = nil
ENV['RAILS_CACHE_ID'] = ""
end
#app.call(env)
end
end
end
end
I am running ruby 2.2.0p0 and rails 4.1.8.
I have searched the webs for this but could not find anything, probably because I'm not serching for the right thing.
Any thoughts on why this is happening and what I am doing wrong?
Cheers!
I finally found the solution to this. It turns out that the last line in what is considered my application is in the middleware. I was running the rest of the code in a local rails engine located in a components directory. All we needed to do was create a new silencer for BacktraceCleaner. Notice components dir is now included.
# config/initializers/backtrace_silencers.rb
Rails.backtrace_cleaner.remove_silencers!
Rails.backtrace_cleaner.add_silencer { |line| line !~ /^\/(app|config|lib|test|components)/}
If you are interested here is an issue I posted on the rails project about how to replicate this in detail. https://github.com/rails/rails/issues/22265
Your middleware seems good. I think you have an issue with your backtrace_cleaner setting. Perhaps the cleaner gets overridden by a 3rd party gem. Try put a breakpoint (debugger) in the controller action method before the error raising, and print:
puts env['action_dispatch.backtrace_cleaner'].instance_variable_get(:#silencers).map(&:source_location).map{|l| l.join(':')}
to see the source locations of all the silencers which strip off non-app traces. By default it should only use Rails::BacktraceCleaner which locates at railties-4.1.8/lib/rails/backtrace_cleaner.rb
To directly see the silencer source code:
puts env['action_dispatch.backtrace_cleaner'].instance_variable_get(:#silencers).map{|s| RubyVM::InstructionSequence.disasm s }
See more from https://github.com/rails/rails/blob/master/railties/lib/rails/backtrace_cleaner.rb
https://github.com/rails/rails/blob/master/activesupport/lib/active_support/backtrace_cleaner.rb
You're not doing anything wrong. But a lot of middleware traps exceptions in order to do cleanup, including middleware that Rack inserts automatically in development mode. There is a specific Rack middleware inserted in development that will catch uncaught exceptions and give a reasonable HTML page instead of a raw stack dump (which you often won't see at all with common app servers.)
You can catch the exception yourself by putting a begin/rescue/end around your top level. Remember to catch "Exception", not just the default "rescue" with no arg, if you want to get everything. And you may want to re-throw the exception if you're going to leave this code in.
You can run in production mode -- that might stop the middleware from being inserted automatically by Rack.
You can find out what middleware is inserted (in Rails: "rake middleware") and then remove the middleware manually (in Rails "config.middleware.delete" or "config.middleware.disable").
There are probably other methods.
Related
I'm trying to add some middleware to a Rails project I'm working on, and when I try to do so, it seems to cause an endless loop.
Specifically, I have the following middleware shell file:
# app/middleware/log_data.rb
class LogData
def initialize(app)
#app = app
end
def call(env)
# To-do: Write code here.
end
end
I then created a new middleware directory under the app directory and put the file in that directory.
After that, I added the following towards the bottom of config/application.rb:
config.middleware.use("LogData")
After restarting the Puma server running on Vagrant with sudo service puma restart, if I run rake middleware, I can see the middleware correctly show up in the list towards the bottom.
However, when I try to refresh the website, it fails with an endless loop, displaying the following in Chrome:
If I comment out the config.middleware.use("LogData") line in config/application.rb, then the middleware disappears from the rake middleware command list, and the website stops crashing and loads properly.
What am I doing wrong? What am I missing? I imagine it's something simple, but I'm not sure why a simple (and empty) shell middleware file would cause the whole site to crash. Thank you.
I should note that I'm using Rails 4.2.11, which I know is old, but upgrading right now is not an option.
Your middleware does nothing, returns nil (which translates to an Incomplete Server Response), and basically the request ends there. It needs to return something (an array of [status, headers, response], or call the env) to allow the request to pass through the middleware chain.
# app/middleware/log_data.rb
class LogData
def initialize(app)
#app = app
end
def call(env)
# To-do: Write code here.
# this should be at the very end of the method
#app.call(env)
end
end
Here is more info about middlewares.
When using custom exceptions_app and rescue_responses, an application has more control of uncaught exceptions and an excessive logging from DebugExceptions middleware becomes noise.
As an example, the application knows how to process ActionPolicy::Unauthorized, renders proper page in exceptions_app and thus the following log is redundant:
FATAL -- :
FATAL -- : ActionPolicy::Unauthorized (Not Authorized):
FATAL -- :
FATAL -- : app/controllers/topics_suggest_controller.rb:47:in `topic_load'
What would be the most idiomatic way to skip logging only those exceptions listed in rescue_responses?
Some historical notes are below. As of June 25, 2021, my PR to Rails https://github.com/rails/rails/pull/42592 has been accepted and this functionality will be available in Rails 6.1.5 7.0.0
Approach 1
It's tempting to delete DebugExceptions middleware from the application's Rails stack.
config/application.rb:
config.middleware.delete ActionDispatch::DebugExceptions
The problem is that it is exactly the middleware that determines Rails cannot find a route and throws ActionController::RoutingError in such cases. Hence, if you would like to react to this exception in your exceptions_app, this approach is a no-go for you.
Go ahead and use this approach if it's fine for you to see HTTP status 404 and plain text response Not found when no route was found.
Approach 2
Monkey patch or change ActionDispatch::DebugExceptions in some way. In fact, there was a proposal in 2013 to change DebugExceptions behavior so that it skips logging exceptions registered in rescue_responses: 9343, there is an example code in there.
I think it's too much to overwrite the whole class, so I chose to override its method log_error responsible for logging errors.
lib/ext/suppress_exceptions.rb:
module Ext
module SuppressExceptions
private
def log_error(_request, wrapper)
exception = wrapper.exception
return if ActionDispatch::ExceptionWrapper.rescue_responses.key? exception.class.name
super
end
end
end
config/initializers/error_handling.rb:
require_dependency 'ext/suppress_exceptions'
ActiveSupport.on_load(:action_controller) do
ActionDispatch::DebugExceptions.prepend Ext::SuppressExceptions
end
I am using the Apartment gem for a multi tenant Rails 5.2 app. I'm not sure that this even matters for my question but just giving some context.
Is there a way to override the Rails logger and redirect every single log entry to a file based on the tenant (database) that is being used?
Thinking... is there a method I can monkeypatch in Logger that will change the file written to dynamically?
Example: I want every error message directed to a file for that day. So at the end of a week there will be 7 dynamically generated files for errors that occurred on each specific day.
Another example: Before you write any server log message check if it is before 1pm. If it is before 1pm write it to /log/before_1.log ... if it is after 1pm write it to /log/after_1.log
Silly examples... but I want that kind of dynamic control before any line of log is written.
Thank you!
Usually the logger is usually configured per server (or per environment really) while apartment sets tenants per request - which means that in practice its not really going to work that well.
You can set the logger at any point by assigning Rails.logger to a logger instance.
Rails.logger = Logger.new(Rails.root.join('log/foo.log'), File::APPEND)
# or for multiple loggers
Rails.logger.extend(Logger.new(Rails.root.join('log/foo.log'), File::APPEND))
However its not that simple - you can't just throw that into ApplicationController and think that everything is hunky-dory - it will be called way to late and most of the entries with important stuff like the request or any errors that pop up before the controller will end up in the default log.
What you could do is write a custom piece of middleware that switches out the log:
# app/middleware/tenant_logger.rb
class TenantLogger
def initialize app
#app = app
end
def call(env)
file_name = "#{Appartment::Tenant.current}.log"
Rails.logger = Logger.new(Rails.root.join('log', file_name), File::APPEND)
#app.call(env)
end
end
And mount it after the "elevator" in the middleware stack:
Rails.application.config.middleware.insert_after Apartment::Elevators::Subdomain, TenantLogger
However as this is pretty far down in the middleware stack you will still miss quite a lot of important info logged by the middleware such as Rails::Rack::Logger.
Using the tagged logger as suggested by the Rails guides with a single file is a much better solution.
I would like to add a custom Tab to bugsnag for all notifications generated by my rails app.
I cannot use :before_bugsnag_notify because the errors are sometimes generated by models which are being called from resque background jobs.
So, I settled on using middleware and I am running to some issues here:
so I went through this guide here:
https://bugsnag.com/docs/notifiers/ruby#bugsnag-middleware
and added my middleware at app/middleware as follows:
class CustomMiddleware
def initialize(bugsnag)
#bugsnag = bugsnag
end
def call(notification)
puts "doing something"
#bugsnag.call(notification)
end
end
The config file for bugsnag is as follows:
in config/initializers/bugsnag.rb:
Bugsnag.configure do |config|
config.api_key = "#{ENV['BUGSNAG_API_KEY']}"
config.middleware.use "CustomMiddleWare"
end
Eventually, I would like to add a tab using add_tab() method before #bugsnag.call(), but now I keep running into the error that I cannot fix:
** [Bugsnag] Bugsnag middleware error: undefined method `new' for "CustomMiddleWare":String
Any ideas?
Edit: I had to put the method name is strings because of this:
Where do you put your Rack middleware files and requires?
I have rails 3 app that generates a lot of requests for analytics. Unfortunately this drowns the logs and I lose the main page requests that I actually care about. I want to separate these requests in to a separate log file. Is there a way to specify certain actions to go to a certain log file? Or possibly a way to reduce the logging level of these actions, and then only show certain level logs when reading back the log file?
I found this site, which talked about using a middleware for silencing log actions. I used the same sort of idea and ended up writing a middleware that would swap the logger depending on which action was being called. Here is the middleware, which i put in lib/noisy_logger.rb
class NoisyLogger < Rails::Rack::Logger
def initialize app, opts = {}
#default_log = Rails.logger
# Put the noisy log in the same directory as the default log.
#noisy_log = Logger.new Rails.root.join('log', 'noisy.log')
#app = app
#opts = opts
#opts[:noisy] = Array #opts[:noisy]
super app
end
def call env
if #opts[:noisy].include? env['PATH_INFO']
logfile = #noisy_log
else
logfile = #default_log
end
# What?! Why are these all separate?
ActiveRecord::Base.logger = logfile
ActionController::Base.logger = logfile
Rails.logger = logfile
# The Rails::Rack::Logger class is responsible for logging the
# 'starting GET blah blah' log line. We need to call super here (as opposed
# to #app.call) to make sure that line gets output. However, the
# ActiveSupport::LogSubscriber class (which Rails::Rack::Logger inherits
# from) caches the logger, so we have to override that too
#logger = logfile
super
end
end
And then this goes in config/initializers/noisy_log.rb
MyApp::Application.config.middleware.swap(
Rails::Rack::Logger, NoisyLogger, :noisy => "/analytics/track"
)
Hope that helps someone!
One option could be using a service like New Relic which would give you the required scoping (per action).