How can I configure the rails logger to output its log strings in another format? I would like to get something that is more informative like:
[Log Level] [Time] [Message]
Debug : 01-20-2008 13:11:03.00 : Method Called
This would really help me when I want to tail my development.log for messages that only come from a certain log level, like debug.
Did some digging and found this post in the RubyOnRails Talk google group.
So I modified it a little bit and put it at the end of my environment.rb:
module ActiveSupport
class BufferedLogger
def add(severity, message = nil, progname = nil, &block)
return if #level > severity
message = (message || (block && block.call) || progname).to_s
level = {
0 => "DEBUG",
1 => "INFO",
2 => "WARN",
3 => "ERROR",
4 => "FATAL"
}[severity] || "U"
message = "[%s: %s #%d] %s" % [level,
Time.now.strftime("%m%d %H:%M:%S"),
$$,
message]
message = "#{message}\n" unless message[-1] == ?\n
buffer << message
auto_flush
message
end
end
end
This results in a format string like this:
[DEBUG: 0121 10:35:26 #57078] Rendered layouts/_header (0.00089)
For rails 4 apps, I've put together a simple gem that not only adds support for basic tagging like time stamp and log level, but even adds color to the log messages themselves.
https://github.com/phallguy/shog
The problem with tags is that they clutter your logs to the point where they are unreadable.
I'd recommend something like timber. It automatically augments your logs with context (level, time, session id, etc) without sacrificing readability.
# config/initializers/rack_logger.rb
module Rails
module Rack
class Logger < ActiveSupport::LogSubscriber
# Add UserAgent
def started_request_message(request)
'Started %s "%s" for %s at %s by %s' % [
request.request_method,
request.filtered_path,
request.ip,
Time.now.to_default_s,
request.env['HTTP_USER_AGENT'] ]
end
end
end
end
source link
Related
I use the gem “Lograge” 0.3.6, and Rails 4.2. I have this configured in my config/environments/development.rb file
config.lograge.enabled = true
config.lograge.formatter = CustomLogstash.new
However, I notice the output in my log/development.log file doesn’t contain date/times in front of each line. How do I configure lograge (or maybe just my Rails logger?) to prefix each line in that file with a date and time?
As per the document, the lograge gem provides below log formatters.
Lograge::Formatters::Lines.new # need to install "lines" gem
Lograge::Formatters::Cee.new
Lograge::Formatters::Graylog2.new
Lograge::Formatters::KeyValue.new # default lograge format
Lograge::Formatters::Json.new
Lograge::Formatters::Logstash.new # need to install "logstash-event" gem
Lograge::Formatters::LTSV.new
Lograge::Formatters::Raw.new # Returns a ruby hash object
By default the lograge gem uses Lograge::Formatters::KeyValue.new format for log.
You can customize this and make it universal by using your CustomLogStash class with some changes.
class CustomLogStash < Lograge::Formatters::KeyValue
def call(data)
# I'm using timestamp key here, you can choose whatever you want.
data_hash = { timestamp: Time.now.utc.strftime("%Y-%m-%dT%H:%M:%S.%3N")}.merge!(data)
super(data_hash)
end
end
Same way you can use any Lograge::Formatters class and apply the custom format to the log.
Now add below code to your config/initializers/lograge.rb file.
Rails.application.configure do
config.lograge.enabled = true
config.lograge.formatter = CustomLogStash.new
end
Now restart your server and load a page in your browser. You will see the logs something like below:
timestamp=2021-11-21T17:14:10.726 method=GET path=/categories format=html controller=categories action=index status=200 duration=259.11 view=244.91 db=2.60
EDITED
If you are looking for logs something like
then you don't need any gem for this. You can achieve this by adding below lines to your preferred environment development/production/test
config.log_level = :debug
config.log_formatter = ::Logger::Formatter.new
If you want to apply this across all environments then add above lines to the config/application.rb file.
Let me know if it helps.
Can you add the following in config if it helps.
config.lograge.formatter = ->(data) { data.reverse_merge({time: Time.now}) }# data is a ruby hash.
It will give you output like following
{:time=>2021-11-16 12:26:24.65362 +0000, :method=>"GET", :path=>"/", :format=>:html, :controller=>"Controller", :action=>"index", :status=>200, :duration=>393.41, :view=>85.55, :db=>1.38}
Can you paste contents of CustomLogstash class? From docs, this class should respond to call method and return Hash.
This works for me:
class CustomLogstash
def call(data)
{ time: Time.now, controller: data[:controller] } # this can be anything as long it is Hash, eg. data.merge(time: Time.now)
end
end
Sample output from above:
{:time=>"2021-11-18T20:31:41.486+01:00", :controller=>"calendar_events"}
As per official documentation for lograge, you can make use of custom_options
EDIT 1 : custom_options using time: Time.now or time:event.time
Rails.application.configure do
config.lograge.enabled = true
config.lograge.formatter = Lograge::Formatters::Logstash.new
# add time to lograge
config.lograge.custom_options = lambda do |event|
{ time: Time.now } #or use time:event.time
end
end
Note: When using the logstash output, you need to add the additional gem logstash-event. You can simply add it to your Gemfile like this
gem "logstash-event"
EDIT 2: Update based on comments custom_options using :time => event.time
#config/environments/production.rb
MyApp::Application.configure do
config.lograge.enabled = true
# add time to lograge
config.lograge.custom_options = lambda do |event|
{:time => event.time}
end
end
OR the below custom options which was a fix in lograge issue to ensure both date and time logged using time: event.time.to_s(:db)
config.lograge.custom_options = lambda do |event|
unwanted_keys = %w[format action controller utf8]
params = event.payload[:params].reject { |key,_| unwanted_keys.include? key }
{time: event.time.to_s(:db), user: event.payload[:user], params: params}
end
ALTERNATIVELY you can use this Custom logger
# Define a setter to pass in a custom log formatter
class ActiveSupport::BufferedLogger
def formatter=(formatter)
#log.formatter = formatter
end
end
# Defines a custom log format (time, severity, message, PID, backtrace)... all with color!
class Formatter
SEVERITY_TO_TAG = {'DEBUG'=>'meh', 'INFO'=>'fyi', 'WARN'=>'hmm', 'ERROR'=>'wtf', 'FATAL'=>'omg', 'UNKNOWN'=>'???'}
SEVERITY_TO_COLOR = {'DEBUG'=>'37', 'INFO'=>'32', 'WARN'=>'33', 'ERROR'=>'31', 'FATAL'=>'31', 'UNKNOWN'=>'37'}
HUMOR_FOR_ENV = {development: true, test: true, production: false}
DEPTH_FOR_ENV = {development: 3, test: 3, production: 1}
EXCLUSION_REGEX = /log|active_support|active_record/
def humorous?
return #is_humorous if defined? #is_humorous
#is_humorous = HUMOR_FOR_ENV[ Rails.env.to_sym ]
end
def depth
#depth ||= DEPTH_FOR_ENV[ Rails.env.to_sym ]
end
def call(severity, time, progname, msg)
t = time.strftime("%Y-%m-%d %H:%M:%S.") << time.usec.to_s[0..2].rjust(3)
color = SEVERITY_TO_COLOR[severity]
sev = humorous? ? "%-3s" % SEVERITY_TO_TAG[severity] # pad to at least 3 characters
: "%-5s" % severity # pad to at least 5 characters
# 2013-05-01 19:16:00.785 [omg] oh noes! (pid:30976) (admin/user.rb:45:in `block (4 levels) in <top (required)>') <- `call' <- `content_for' <- `block (2 levels) in row' <- `block in build_tag'
"\033[0;37m#{t}\033[0m [\033[#{color}m#{sev}\033[0m] #{msg.strip} (pid:#{$$}) #{whodunit}\033[0m\n"
end
def whodunit
latest, *others = caller.select{ |a| a !~ EXCLUSION_REGEX }[0, depth]
latest = latest[/(lib|app)\/(.*)/,-1] || latest
string = ""
string << "\033[36m(#{latest})"
string << "\033[35m <- " + others.map{ |s| s[/`.*/] }.join(' <- ') if others.any?
string
end
end
Rails.logger.formatter = Formatter.new
For Rails 4.2 don’t forget to add ActiveSupport::TaggedLogging to be able to call custom logger like a default rails logger
ActiveSupport::TaggedLogging is used to wrap any standard logger instance to add "tags" to a log statement. A "tag" in this case usually describes a subdomain, and is used by the default Rails.logger to allow you to tag log statements with subdomains, request ids, etc. in your multi-user, multi-instance production applications.
include
ActiveSupport::TaggedLogging::Formatter
to the Formatter class.
In my Rails app, I have set up the following backtrace silencer, as suggested by Michael Hartl in his Rails tutorial:
Rails.backtrace_cleaner.add_silencer { |line| line =~ /rvm/ }
But still I get all the noise I intended to filter out:
7:13:55 - INFO - Running: test/controllers/tags_controller_test.rb
Started
ERROR["test_should_get_index", TagsControllerTest, 0.45206]
test_should_get_index#TagsControllerTest (0.45s)
ActionController::UrlGenerationError:
ActionController::UrlGenerationError: No route matches {:action=>"index", :controller=>"tags"}
/Users/chris/.rvm/gems/ruby-2.0.0-p353/gems/actionpack-4.1.6/lib/action_dispatch/journey/formatter.rb:39:in `generate'
/Users/chris/.rvm/gems/ruby-2.0.0-p353/gems/actionpack-4.1.6/lib/action_dispatch/routing/route_set.rb:599:in `generate'
Clearly the string "rvm" is present in the last two lines. But still they show up. Changing the string to ".rvm" didn't make any difference.
This is because of https://github.com/vipulnsward/rails/blob/ecc8f283cfc1b002b5141c527a827e74b770f2f0/actionpack/lib/action_dispatch/middleware/debug_exceptions.rb#L155-L156
Since application_trace is empty(This is because error is not from user code but route error), we are falling back to framework_trace, which does not filter it (it filters only noise).
You can solve it with creating your own log_formatter. In your development.rb and/or test.rb add
config.log_formatter = SilentLogger.new
config.log_formatter.add_silencer { |line| line =~ /rvm/ }
And create simple class in models with only method call required. There you can modify your backtrace as you wish.
class SilentLogger
def initialize
#silencers = []
end
def add_silencer(&block)
#silencers << block
end
def call(severity, timestamp, progname, msg)
backtrace = (String === msg) ? "#{msg}\n" : "#{msg.inspect}\n"
return backtrace if #silencers.empty?
#silencers.each do |s|
backtrace = backtrace.split("\n").delete_if { |line| s.call(line) }
end
backtrace.join("\n")
end
end
Your logs show /.rvm/ your setting /rvm/ - can this be the cause?
Give it a try and dont forget to stop / restart the server.
I have a logger to print to a file:
logger = Logger.new Rails.root.join 'log/my.log'
How can I print several dots in one line in different logger calls?
i.e., if I do
some_array.each do |el|
logger.info '.'
el.some_method
end
These dots will be printed in different lines.
The << method lets you write messages without any formatting the log file.
For example:
require 'logger'
log = Logger.new "test.log"
log.info "Start"
5.times { log << "." }
log << "\n"
log.info "End"
Produces:
I, [2014-03-10T15:11:04.320495 #3410] INFO -- : Start
.....
I, [2014-03-10T15:11:42.157131 #3410] INFO -- : End
Unfortunately, this doesn't let you write to the same line as previous calls to log.info
You can use Logger#<<(msg)
Dump given message to the log device without any formatting. If no log
device exists, return nil.
some_array.each do |el|
logger << '.'
el.some_method
end
And you can still keep it restricted by the logger.level by using .info?
some_array.each do |el|
logger << '.' if logger.info?
el.some_method
end
I receive undefined method scan for User:Class when scan method
(line 13: last = resp.headers['Link'].scan(/\d+/).last)
is called by Rails User model (inside class method self.create_user(auth)) :
class User
include Mongoid::Document
field :email
field :nickname
embeds_many :posts
def self.create_user(auth)
conn = FaradayStack.build 'https://api.example.com'
resp = conn.get "/users/#{auth.nickname}/list"
last = resp.headers['Link'].scan(/\d+/).last # <== error occurs here
n = 0
create! do |user|
user.email = auth["user_info"]["email"]
user.nickname = auth['user_info']['nickname']
while n <= last.to_i do
resp = conn.get "/users/#{auth.nickname}/list?page=#{n=n+1}"
resp.body.each do |repo|
user.posts.build( html_url: "#{repo['html_url']}",
description: "#{repo['description']}",
created_at: "#{repo['created_at']}",
pushed_at: "#{repo['pushed_at']}",
avatar_url: "#{repo['owner']['avatar_url']}" )
end
end
end
end
end
If I call the same code by Rails console it works fine :
ruby-1.9.2-p136 :158 > last = resp.headers['Link'].scan(/\d+/).last
=> "16"
ruby-1.9.2-p136 :159 > last = resp.headers['Link'].scan(/\d+/).last.to_i
=> 16
ruby-1.9.2-p136 :160 >
Something related to istance and class method concept I guess,
but I can't fix it.
UPDATE:
On suggestion of jimworm, I put rails logger instead of "offending" line :
Rails.logger.info "\r\n" + "#{Time.now} " + "resp.headers['Link']: #{resp.headers['Link']}" + "\r\n"
what I get is :
2011-09-30 16:40:03 +0200 resp.headers['Link']:
Redirected to http://localhost:3001/
Completed 302 Found in 3151ms
MONGODB blumb_dev['users'].find({:_id=>BSON::ObjectId('4e85d4c41d41c8103f000006')})
while in Rails console it is :
ruby-1.9.2-p136 :181 > resp.headers['Link']
=> "<https://api.example.com/users/lgs/list?page=18>; rel=\"next\", <https://api.example.com/users/lgs/list?page=51>; rel=\"last\""
ruby-1.9.2-p136 :182 >
Any idea ?
Our hero was perplexed... it worked in the console, surely it'll work from the controller? The village will be doomed if the bug wasn't caught. He decided to take a snapshot of the bug, catch it in the act.
Rails.logger.info...
The trap was set, then he waited. The screen flashed. A hit! The bug has left its traces in our trap. Our hero looked and...
to his surprise, the trap was empty!
2011-09-30 16:40:03 +0200 resp.headers['Link']: #nothing here#
"Now how could that be" he wondered, "a bug that left no traces? What kind of bug leaves no traces?"
Then it came to him. "Eureka, a nil!" he smacked the table triumphantly, "a nil appears as nothing in the log! That's why I couldn't scan it!"
The identity of the bug was revealed, and it wasn't long before our hero traced it back to its home at api.example.com and mashed it once and for all. Turns out it was the messages that the village was sending to api.example.com that lured the bug out of hiding. Now with the bug gone, the village was saved and everyone lived in peace, happily ever after.
I use tail -f to display the log file when developing my Rails app.
It shows the log messages (in color! :), which is great.
But with so much information in the 80-width console, it becomes difficult to track where a certain "set" of log messages started when, say, I clicked on a button to GET a resource.
It would be easier if there was a line number or even a time stamp at the start of each log message/line. This way I could remember that I need to start looking at the log "after line number 2365" or "after 2010/10/10 23:33:23:45".
Is this possible to do? Is there some Rails internal option for this ?
why don't you just edit your desired environment's log tags
development.rb
config.log_tags [ lambda {|r| DateTime.now } ]
If you wanted to get a time stamp:
class ApplicationController < ActionController::Base
# ...
before_filter :log_tracker
def log_tracker
Rails.logger.add(1, "Log Date: #{DateTime.now}")
end
end
And format the date however you see fit....
That would work for Rails 2.1 +, prior you could access the ActiveSupport::Buffered log object with the constant: RAILS_DEFAULT_LOGGER
Get access to the actual log file with Rails.logger.instance_values["log"]
Getting the number of lines is difficult because the logger only opens the file for writing, probably for economy. I get an IOError: not opened for reading when I try.
`
Thanks #scaney.
I found a solution here.
I modified that code to add my own coloring highlights (for development only of course!) and now I can see things like 'parameters' in yellow in the console and I'm very pleased now!
In case someone is interested, here is the code I put at the end of environment.rb.
Here is my current (dirty) implementation. Will probably fix this up later (maybe make a gem, but for now this serves me fine)
WARNING
DIRTY CODE FOLLOWS! Use at your own risk!
module ActiveSupport
class BufferedLogger
#define the ANSI escape codes for normal and bright colors
$my_my_ansi_colors = {
:normal => "\x1B[0m",
:black => "\x1B[30m",
:red => "\x1B[31m", #red
:green => "\x1B[32m",
:yellow => "\x1B[33m",
:blue => "\x1B[34m",
:magenta => "\x1B[35m",
:cyan => "\x1B[36m",
:white => "\x1B[37m",
:bred => "\x1B[1m\x1B[31m", #bright red
:bgreen => "\x1B[1m\x1B[32m",
:byellow => "\x1B[1m\x1B[33m",
:bblue => "\x1B[1m\x1B[34m",
:bmagenta => "\x1B[1m\x1B[35m",
:bcyan => "\x1B[1m\x1B[36m",
:bwhite => "\x1B[1m\x1B[37m",
}
#take a string and using the keys in the hash, replace the keys in the
#string but surround the keys with ANSI color codes
#No idea how to retain the case of the key!(TODO someday)
def my_highlight msgx,hash
return msgx if msgx.blank?
return msgx if hash.empty?
hash.each_pair do |k,v|
if not k.nil?
msgx.gsub! Regexp.new(k, Regexp::IGNORECASE), $my_my_ansi_colors[:normal]+$my_my_ansi_colors[v]+k.upcase+$my_my_ansi_colors[:normal]
end
end
msgx
end
def add(severity, message = nil, progname = nil, &block)
return if #level > severity
message = (message || (block && block.call) || progname).to_s
#INSERT BEGINS
if not $myownglobalnumbercounter.nil?
$myownglobalnumbercounter += 1
else
$myownglobalnumbercounter = 1
end
level = {
0 => "DEBUG",
1 => "INFO",
2 => "WARN",
3 => "ERROR",
4 => "FATAL"
}[severity] || "U"
message = "\x1B[0m[%d %s] : %s" % [$myownglobalnumbercounter,level,message]
message = my_highlight message, {
"debug" => :white,
"error" => :bred,
"info" => :bwhite,
"warning" => :byellow,
"warn" => :byellow ,
"parameters" => :byellow,
"#" => :bgreen,
"ms " => :bmagenta,
"GET " => :bmagenta,
"PUT " => :bmagenta,
"POST " => :bmagenta,
"DELETE " => :bmagenta
}
#INSERT ENDS
message = "#{message}\n" unless message[-1] == ?\n
buffer << message
auto_flush
message
end
end
end