I have rails 3 app that generates a lot of requests for analytics. Unfortunately this drowns the logs and I lose the main page requests that I actually care about. I want to separate these requests in to a separate log file. Is there a way to specify certain actions to go to a certain log file? Or possibly a way to reduce the logging level of these actions, and then only show certain level logs when reading back the log file?
I found this site, which talked about using a middleware for silencing log actions. I used the same sort of idea and ended up writing a middleware that would swap the logger depending on which action was being called. Here is the middleware, which i put in lib/noisy_logger.rb
class NoisyLogger < Rails::Rack::Logger
def initialize app, opts = {}
#default_log = Rails.logger
# Put the noisy log in the same directory as the default log.
#noisy_log = Logger.new Rails.root.join('log', 'noisy.log')
#app = app
#opts = opts
#opts[:noisy] = Array #opts[:noisy]
super app
end
def call env
if #opts[:noisy].include? env['PATH_INFO']
logfile = #noisy_log
else
logfile = #default_log
end
# What?! Why are these all separate?
ActiveRecord::Base.logger = logfile
ActionController::Base.logger = logfile
Rails.logger = logfile
# The Rails::Rack::Logger class is responsible for logging the
# 'starting GET blah blah' log line. We need to call super here (as opposed
# to #app.call) to make sure that line gets output. However, the
# ActiveSupport::LogSubscriber class (which Rails::Rack::Logger inherits
# from) caches the logger, so we have to override that too
#logger = logfile
super
end
end
And then this goes in config/initializers/noisy_log.rb
MyApp::Application.config.middleware.swap(
Rails::Rack::Logger, NoisyLogger, :noisy => "/analytics/track"
)
Hope that helps someone!
One option could be using a service like New Relic which would give you the required scoping (per action).
Related
My Rails application runs in Heroku; recently, we have changed the Heroku LOG_LEVEL to WARN as the system logs flooded with so many unwanted information. But still, in some of the areas, I wanted to use Rails.logger.info;
Currently, in Heroku we have this:
LOG_LEVEL = WARN
And in production.rb, still that is
config.log_level = :info
config.log_formatter = ::Logger::Formatter.new
The above configuration we didn't change it, as the precedence is for LOG_LEVEL if we set that. So with the above configuration, if we put Rails.logger.info "Hello world," that will not work because the logger will only handle the logs equal or higher to warn in importance.
So we have tried one other way.
Created a new initializer called custom_logger.rb; we put
$INFO_LOGGER = Rails.logger.dup
$INFO_LOGGER.level = :info
then wherever we wanted to use info, we just called $INFO_LOGGER.info "Hello World," this prints
Is this a correct approach, like using the global variable?
Is this a correct approach, like using the global variable?
This question could be considered opinion based, so in my opinion I would not recommend it. Additionally the question posed in the title is regarding a custom logger and while we could implement one to facilitate the request I would propose a simpler solution that will still log exactly what you want, to the current log file, and without the need for a custom logger, a secondary logger, or any kind of global variable.
My Suggestion:
Rails uses ActiveSupport::Logger by default which is essentially just a ruby Logger.
If there are messages you always want logged regardless of the level you can use Logger#unknown.
Per the Documents for the Logger class:
Levels:
UNKNOWN - An unknown message that should always be logged.
Logger#unknown
Log an UNKNOWN message. This will be printed no matter what the logger's level is.
So You can use this to your advantage for the messages that you always want to show while still avoiding the noise of the standard info messages:
For Example:
Rails.logger.info "Noisy message" # won't show when LOG_LEVEL > INFO
Rails.logger.unknown "VERY IMPORTANT MESSAGE" # will show no matter what the level is set to
You don't want to use global variables. Instead create a custom class for example
class MyLogger
class << self
def info(*args)
new.info(*args)
end
end
delegate :info, to: :logger
attr_reader :logger
def initialize
#logger = ActiveSupport::Logger.new(Rails.root.join("log/my_logger.#{Rails.env}.log"))
#logger.formatter = ::Logger::Formatter.new
end
end
Now you can call MyLogger.info("this is a test message") and it will output the message in the log file regardless of the LOG_LEVEL or config.log_level = :info
I am using the Apartment gem for a multi tenant Rails 5.2 app. I'm not sure that this even matters for my question but just giving some context.
Is there a way to override the Rails logger and redirect every single log entry to a file based on the tenant (database) that is being used?
Thinking... is there a method I can monkeypatch in Logger that will change the file written to dynamically?
Example: I want every error message directed to a file for that day. So at the end of a week there will be 7 dynamically generated files for errors that occurred on each specific day.
Another example: Before you write any server log message check if it is before 1pm. If it is before 1pm write it to /log/before_1.log ... if it is after 1pm write it to /log/after_1.log
Silly examples... but I want that kind of dynamic control before any line of log is written.
Thank you!
Usually the logger is usually configured per server (or per environment really) while apartment sets tenants per request - which means that in practice its not really going to work that well.
You can set the logger at any point by assigning Rails.logger to a logger instance.
Rails.logger = Logger.new(Rails.root.join('log/foo.log'), File::APPEND)
# or for multiple loggers
Rails.logger.extend(Logger.new(Rails.root.join('log/foo.log'), File::APPEND))
However its not that simple - you can't just throw that into ApplicationController and think that everything is hunky-dory - it will be called way to late and most of the entries with important stuff like the request or any errors that pop up before the controller will end up in the default log.
What you could do is write a custom piece of middleware that switches out the log:
# app/middleware/tenant_logger.rb
class TenantLogger
def initialize app
#app = app
end
def call(env)
file_name = "#{Appartment::Tenant.current}.log"
Rails.logger = Logger.new(Rails.root.join('log', file_name), File::APPEND)
#app.call(env)
end
end
And mount it after the "elevator" in the middleware stack:
Rails.application.config.middleware.insert_after Apartment::Elevators::Subdomain, TenantLogger
However as this is pretty far down in the middleware stack you will still miss quite a lot of important info logged by the middleware such as Rails::Rack::Logger.
Using the tagged logger as suggested by the Rails guides with a single file is a much better solution.
I have a piece of Rack middleware that loads a tenant, via subdomain, and applies some default settings. The middleware, while not pretty, does it's job fine enough. However, when an exception is thrown within the app the middleware "traps" the full stack trace. When I say trap I mean it hides the expected stack trace.
Here is an example.
I am throwing an exception in an a controller action like so:
def index
throw "Exception in a Rails controller action"
#taxonomies = Spree::Taxonomy.all
end
You would expect that the stack trace would reference this location but it does not. Instead it reference a line in the middleware.
Completed 500 Internal Server Error in 139ms
UncaughtThrowError (uncaught throw "Exception in a Rails controller action"):
lib/tenant_manager/middleware/loader.rb:42:in `call'
Why does this happen? Have you seen anything like this before?
Here is the middleware:
# lib/tenant_manager/middleware/loader.rb
module TenantManager
module Middleware
class Loader
# Middleware to detect an tenant via subdomain early in
# the request process
#
# Usage:
# # config/application.rb
# config.middleware.use TenantManager::Middleware::Loader
#
# A scaled down version of https://github.com/radar/houser
def initialize(app)
#app = app
end
def call(env)
domain_parts = env['HTTP_HOST'].split('.')
if domain_parts.length > 2
subdomain = domain_parts.first
tenant = Leafer::Tenant.find_by_database(subdomain)
if tenant
ENV['CURRENT_TENANT_ID'] = tenant.id.to_s
ENV['RAILS_CACHE_ID'] = tenant.database
Spree::Image.change_paths tenant.database
Apartment::Tenant.process(tenant.database) do
country = Spree::Country.find_by_name('United States')
Spree.config do |config|
config.default_country_id = country.id if country.present?
config.track_inventory_levels = false
end
Spree::Auth::Config.set(:registration_step => false)
end
end
else
ENV['CURRENT_TENANT_ID'] = nil
ENV['RAILS_CACHE_ID'] = ""
end
#app.call(env)
end
end
end
end
I am running ruby 2.2.0p0 and rails 4.1.8.
I have searched the webs for this but could not find anything, probably because I'm not serching for the right thing.
Any thoughts on why this is happening and what I am doing wrong?
Cheers!
I finally found the solution to this. It turns out that the last line in what is considered my application is in the middleware. I was running the rest of the code in a local rails engine located in a components directory. All we needed to do was create a new silencer for BacktraceCleaner. Notice components dir is now included.
# config/initializers/backtrace_silencers.rb
Rails.backtrace_cleaner.remove_silencers!
Rails.backtrace_cleaner.add_silencer { |line| line !~ /^\/(app|config|lib|test|components)/}
If you are interested here is an issue I posted on the rails project about how to replicate this in detail. https://github.com/rails/rails/issues/22265
Your middleware seems good. I think you have an issue with your backtrace_cleaner setting. Perhaps the cleaner gets overridden by a 3rd party gem. Try put a breakpoint (debugger) in the controller action method before the error raising, and print:
puts env['action_dispatch.backtrace_cleaner'].instance_variable_get(:#silencers).map(&:source_location).map{|l| l.join(':')}
to see the source locations of all the silencers which strip off non-app traces. By default it should only use Rails::BacktraceCleaner which locates at railties-4.1.8/lib/rails/backtrace_cleaner.rb
To directly see the silencer source code:
puts env['action_dispatch.backtrace_cleaner'].instance_variable_get(:#silencers).map{|s| RubyVM::InstructionSequence.disasm s }
See more from https://github.com/rails/rails/blob/master/railties/lib/rails/backtrace_cleaner.rb
https://github.com/rails/rails/blob/master/activesupport/lib/active_support/backtrace_cleaner.rb
You're not doing anything wrong. But a lot of middleware traps exceptions in order to do cleanup, including middleware that Rack inserts automatically in development mode. There is a specific Rack middleware inserted in development that will catch uncaught exceptions and give a reasonable HTML page instead of a raw stack dump (which you often won't see at all with common app servers.)
You can catch the exception yourself by putting a begin/rescue/end around your top level. Remember to catch "Exception", not just the default "rescue" with no arg, if you want to get everything. And you may want to re-throw the exception if you're going to leave this code in.
You can run in production mode -- that might stop the middleware from being inserted automatically by Rack.
You can find out what middleware is inserted (in Rails: "rake middleware") and then remove the middleware manually (in Rails "config.middleware.delete" or "config.middleware.disable").
There are probably other methods.
I wanted to use this api: https://github.com/coinbase/coinbase-ruby and the first step is to initialize the API, like this:
coinbase = Coinbase::Client.new(ENV['COINBASE_API_KEY'], ENV['COINBASE_API_SECRET'])
I was wondering what the best place to put this code is, and how would I access it if I put it "there"? I want this variable (coinbase) to be accessible ANYWHERE in the application.
Thanks!
The answer to this question really depends on your use case and your approach. My geral recommendation, however, is to create a Service Object (in the DDD sense) (see the section named "Domain Objects Should Not Know Anything About Infrastructure Underneath" in that link), that handles all communication with the Coinbase API. And then, within this service object, you can simply initialize the Coinbase::Client object once for however many times you call into it. Here's an example:
# app/services/coinbase_service.rb
class CoinbaseService
cattr_reader :coinbase_client, instance_accessor: false do
Coinbase::Client.new(ENV['COINBASE_API_KEY'], ENV['COINBASE_API_SECRET'])
end
def self.do_something
coinbase_client.do_something_in_their_api
end
def self.do_something_else
coinbase_client.do_something_else_in_their_api
end
end
So then you might do, e.g.:
# From MyController#action_1
if CoinbaseService.do_something
# ...
else
# ...
end
Or:
# From MyModel
def do_something
CoinbaseService.do_something_else
end
To get the service object working, you may need to add app/services to your autoload paths in application.rb file. I normally just add this:
# config/application.rb
config.autoload_paths += %W(#{config.root}/app)
I find this Service Object approach to be very beneficial organizationally, more efficient (only 1 invocation of the new Coinbase client needed), easier to test (easy to mock-out calls to Coinbase::Client), and simply joyful :).
One way to go about having a global variable can be done as similar as initializing redis in a Rails application by creating an initializer in config/initializers/coinbase.rb with:
$coinbase = Coinbase::Client.new(ENV['COINBASE_API_KEY'], ENV['COINBASE_API_SECRET'])
Now, you can access $coinbase anywhere in the application!
In the file config/initializers/coinbase.rb
Rails.application.config.after_initialize do
CoinbaseClient = Coinbase::Client.new(
Rails.application.credentials.coinbase[:api_key],
Rails.application.credentials.coinbase[:api_secret])
end
In place of the encrypted credentials, you could also use environment variables: ENV['COINBASE_API_KEY'], ENV['COINBASE_API_SECRET']
The above will make the constant CoinbaseClient available everywhere in your app. It will also ensure all your gems are loaded before the client is initialized.
Note: I am using Rails 6.1.4.4, and Ruby 2.7.5
#Here is how I have delayed job set up.
Delayed::Worker.backend = :active_record
#Delayed::Worker.logger = Rails.logger
Delayed::Worker.logger = ActiveSupport::BufferedLogger.new("log/
##{Rails.env}_delayed_jobs.log", Rails.logger.level)
Delayed::Worker.logger.auto_flushing = 1
class Delayed::Job
def logger
Delayed::Worker.logger
end
end
if JobsCommon::check_job_exists("PeriodicJob").blank?
Delayed::Job.enqueue PeriodicJob.new(), 0, 30.seconds.from_now
end
#end
#Here is my simple job.
class PeriodicJob
def perform
Rails.logger.info "Periodic job writing #{Time.now}"
Delayed::Job.enqueue PeriodicJob.new(), 0,
30.seconds.from_now
end
end
I don't see any log messages from delayed job in my rails logs or delayed job log file, the only messages I see are jobs starting/success/failure in the delayed_jobs.log file.
this is causing big problems, including detecting bugs and memory leaks in workers almost impossible! Please help!
We've gotten it to work on Rails 3/Delayed Job 2.0.3 by hacking Rails.logger itself to use a different log file (the one we want for delayed_job entries) and also setting the delayed job logger to use the exact same object:
file_handle = File.open("log/#{Rails.env}_delayed_jobs.log", (File::WRONLY | File::APPEND | File::CREAT))
# Be paranoid about syncing, part #1
file_handle.sync = true
# Be paranoid about syncing, part #2
Rails.logger.auto_flushing = true
# Hack the existing Rails.logger object to use our new file handle
Rails.logger.instance_variable_set :#log, file_handle
# Calls to Rails.logger go to the same object as Delayed::Worker.logger
Delayed::Worker.logger = Rails.logger
If the above code doesn't work, try replacing Rails.logger with RAILS_DEFAULT_LOGGER.
This may be a simple workaround but it works well enough for me:
system("echo #{your message here} >> logfile.log")
Simple but works
I have it working with the following setup in initializer:
require 'delayed/worker'
Delayed::Worker.logger = Rails.logger
module Delayed
class Worker
def say_with_flushing(text, level = Logger::INFO)
if logger
say_without_flushing(text, level)
logger.flush
end
end
alias_method_chain :say, :flushing
end
end
i simply did this:
/config/environments/development.rb
MyApp::Application.configure do
[...]
[...]
[...]
Delayed::Worker.logger = Rails.logger
end
In every next request you do the mail will be appear on the log.
NOTE: Sometimes you have to refresh the page to the mail be logged on the log. Don't forget to restart the server ;)
DelayedJob does not seem to output if there is something wrong:
1- Non-active record classes need to be required and initialized:
How:
Create a file config/initializers/load_classes_for_dj.rb
Add to it the lines:
require 'lib/libtest/delayed_test.rb'
DelayedTest
Note that if you have '#{config.root}/lib/libtest' in config.autoload_paths in config/application.rb, you do not need to do the require.
Source:
Rails Delayed Job & Library Class
2- Classes that implement the Singleton module won't work by calling:
SingletonClass.instance.delay.sayhello
In order to fix that, do the following:
class SingletonClass
include Singleton
# create a class method that does the call for you
def self.delayed_sayhello
SingletonClass.instance.sayhello
end
def sayhello
# this is how you can actually write to delayed_job.log
# https://stackoverflow.com/questions/9507765/delayed-job-not-logging
Delayed::Worker.logger.add(Logger::INFO, "hello there")
end
end
In order to call the delayed class method, do the following:
SingletonClass.delay.delayed_sayhello
Yes, I am calling .delay on a class here. Since classes in Ruby are also objects, the call is valid here and allows me to access the class method "delayed_sayhello"
3- Do not pass ActiveRecord objects or some complex objects to your call but rather pass ids, look up the objects in the database in your delayed method, and then do your work:
DO NOT DO:
DelayedClass.new.delay.do_some_processing_on_album Album.first
DO THIS INSTEAD:
DelayedClass.new.delay.do_some_processing_on_album Album.first.id
and inside DelayedClass do_some_processing_on_album , do
a = Album.find_by_id id
There was a stackoverflow post about this that I saw a while ago -- not sure which :-)
4- For completion, this is how to do mailers (do not call the deliver method):
Notifier.delay.signup(user_id)
As per 3, do not pass the user's object but rather their id, do the lookup inside the signup method.
Now once you've ensured you have followed the guidelines above, you can use:
Delayed::Worker.logger.add(Logger::INFO, "hello")
If you are still facing issues, I suggest you break down your problem:
a- Create a new class
b- Make sure you have it included and initialized as per step 1
c- Add logging from step 4 and try calling MyNewClass.new.delay to see if it works
I hope this helps you guys :-)
Don't forget to change the ActiveRecord::Base.logger
Delayed::Worker.logger = Logger.new("log/delayed_job.log", 5, 104857600)
if caller.last =~ /script\/delayed_job/ or (File.basename($0) == "rake" and ARGV[0] =~ /jobs\:work/)
ActiveRecord::Base.logger = Delayed::Worker.logger
end
More detailed answer: Have delayed_job log "puts", sql queries and jobs status