Duplicated request info log in Rails 5 server only app - ruby-on-rails

I have a Rails 5.2.3 application with Ruby 2.4.5.
I found a weird issue that the request info are logged twice in stdout.
Here is the log config in config/environments/product.rb
config.logger = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
config.log_tags = [ lambda {|req| "#{req.cookie_jar["_session_id"]}" }, :remote_ip, :uuid ]
Supposedly it will tag every log with the remote ip and request uuid, and it does for most of logs, except there is a weird additional log for the request. In following example, the last line is a duplicated log for the request without the tag:
[INFO] [2019-10-28 06:11:45 UTC] [127.0.0.1] [f6de1900-a7e5-4486-8b73-7095d0cacb35] Started GET "/api/v1/nodes?pageSize=20&pageNumber=1" for 127.0.0.1 at 2019-10-28 06:11:45 +0000
[INFO] [2019-10-28 06:11:45 UTC] [127.0.0.1] [f6de1900-a7e5-4486-8b73-7095d0cacb35] Processing by Api::V1::NodesController#index as XML
...
...
[INFO] [2019-10-28 06:20:02 UTC] [127.0.0.1] [993e0db4-3995-41ef-851a-bfea1bc25781] Completed 200 OK in 1084ms (Views: 341.9ms | ActiveRecord: 150.6ms)
127.0.0.1 - - [28/Oct/2019:06:20:02 +0000] "GET /api/v1/nodes?pageSize=20&pageNumber=1 HTTP/1.1" 200 - 1.1347
I checked configurations, there is no any other logger configured.
The consequence is that there is a timer in client side to check notification status by every 5 seconds. I added a log silencer to avoid log such request:
# config/application.rb
config.middleware.insert_before Rails::Rack::Logger, LogSilencer, silenced: /notification_messages/
# lib/log_silencer.rb
class LogSilencer
def initialize(app, opts = {})
#app = app
#silenced = opts.delete(:silenced)
end
def call(env)
if #silenced.match(env['PATH_INFO'])
Rails.logger.silence do
#app.call(env)
end
else
#app.call(env)
end
end
end
It does avoid logging from the tagged logger, but the duplicated request logs are still there, then the stdout is full of this request
127.0.0.1 - - [28/Oct/2019:07:04:21 +0000] "GET /api/v1/notification_messages/to_notify HTTP/1.1" 200 - 0.0118
127.0.0.1 - - [28/Oct/2019:07:04:26 +0000] "GET /api/v1/notification_messages/to_notify HTTP/1.1" 200 - 0.0140
127.0.0.1 - - [28/Oct/2019:07:04:31 +0000] "GET /api/v1/notification_messages/to_notify HTTP/1.1" 200 - 0.0137
127.0.0.1 - - [28/Oct/2019:07:04:36 +0000] "GET /api/v1/notification_messages/to_notify HTTP/1.1" 200 - 0.0149
...
Spent a whole day try to find out who generated this log, but no any clue...
Would like to ask for help on how to turn off this log so make the log data clear...
Thanks!

Seems the logs come from the web server used when up the Rails app.
When use WEBrick, the log is like
=> Booting WEBrick
=> Rails 5.2.3 application starting in development on http://localhost:4000
=> Run `rails server -h` for more startup options
[2019-10-30 07:01:13] INFO WEBrick 1.3.1
[2019-10-30 07:01:13] INFO ruby 2.4.5 (2018-10-18) [x86_64-linux]
[2019-10-30 07:01:13] INFO WEBrick::HTTPServer#start: pid=5812 port=4000
127.0.0.1 - - [30/Oct/2019:07:01:17 UTC] "GET /test HTTP/1.1" 304 0
- -> /test
When use unicorn, the log is like:
I, [2019-10-30T07:02:47.626962 #5956] INFO -- : Refreshing Gem list
I, [2019-10-30T07:02:48.289373 #5956] INFO -- : listening on addr=0.0.0.0:4000 fd=17
I, [2019-10-30T07:02:48.393598 #5956] INFO -- : master process ready
I, [2019-10-30T07:02:48.394853 #5965] INFO -- : worker=0 ready
I, [2019-10-30T07:02:48.399878 #5968] INFO -- : worker=1 ready
127.0.0.1 - - [30/Oct/2019:07:02:52 +0000] "GET /test HTTP/1.1" 304 - 0.0724
I can find that the request log format is a bit different (the time zone info, the time consumed etc).
I use Unicorn, its logger by default uses stderr, but seems the logger configuration does not work
logger Logger.new("#{rails_root}/log/unicorn.log")
so I have to set the stderr path
stderr_path "#{rails_root}/log/unicorn.stderr.log
Then the request logs are in the unicorn.stderr.log file, and STDOUT are the rails app logs.
But still dont know how to turn it off since it is a kind of duplicated and useless log...

Related

Rails/Nginx: Routes work on dev not on prod

My application works perfectly in my development environment, but routes don't work in my production server running nginx and Passenger, where I get:
404 Not Found
However, the index page does work as well as CSS and JavaScript.
I thought it was related to locale or i18n issues, but by changing routes.rb to remove :locale I got the same output.
Routes:
$ rake routes
Prefix Verb URI Pattern Controller#Action
GET /:locale(.:format) welcome#index
services GET /:locale/services(.:format) welcome#services
projects GET /:locale/projects(.:format) welcome#projects
contact GET /:locale/contact(.:format) welcome#contact
GET /:locale(.:format) welcome#index
root GET / welcome#index
rails_service_blob GET /rails/active_storage/blobs/:signed_id/*filename(.:format) active_storage/blobs#show
rails_blob_representation GET /rails/active_storage/representations/:signed_blob_id/:variation_key/*filename(.:format) active_storage/representations#show
rails_disk_service GET /rails/active_storage/disk/:encoded_key/*filename(.:format) active_storage/disk#show
update_rails_disk_service PUT /rails/active_storage/disk/:encoded_token(.:format) active_storage/disk#update
rails_direct_uploads POST /rails/active_storage/direct_uploads(.:format) active_storage/direct_uploads#create
Nginx/site-enable:
server {
listen 80;
listen [::]:80;
root /var/www/test.com/public;
index index.html index.htm index.nginx-debian.html;
server_name test.com www.test.com;
# Turn on Passenger
passenger_enabled on;
passenger_ruby /home/ubuntu/.rbenv/versions/2.5.1/bin/ruby;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
}
I'm using:
- Ruby 2.5.1
- gem 'rails', '~> 5.2.0'
production.log shows the rendering of the Welcome page but not /en/services for example when trying to go there:
I, [2020-02-23T17:08:28.146331 #19532] INFO -- : [4cd74925-0637-4b92-92aa-12f693691ecb] Processing by WelcomeController#index as HTML
I, [2020-02-23T17:08:28.146998 #19532] INFO -- : [4cd74925-0637-4b92-92aa-12f693691ecb] Rendering welcome/index.html.erb within layouts/application
I, [2020-02-23T17:08:28.147227 #19532] INFO -- : [4cd74925-0637-4b92-92aa-12f693691ecb] Rendered welcome/index.html.erb within layouts/application (0.1ms)
I, [2020-02-23T17:08:28.148286 #19532] INFO -- : [4cd74925-0637-4b92-92aa-12f693691ecb] Completed 200 OK in 2ms (Views: 1.4ms)
nginx/error.log:
[ N 2020-02-23 17:13:07.1716 25594/T1 age/Wat/WatchdogMain.cpp:1373 ]: Starting Passenger watchdog...
[ N 2020-02-23 17:13:07.1994 25602/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core...
[ N 2020-02-23 17:13:07.1995 25602/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode.
[ N 2020-02-23 17:13:07.2058 25602/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 25602
nginx/access.log:
[23/Feb/2020:17:14:23 +0000] "GET /favicon.ico HTTP/1.1" 200 0 "http://www.XXX.fr/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.106 Safari/537.36"
[23/Feb/2020:17:14:28 +0000] "GET /en/services HTTP/1.1" 404 209 "http://www.XXX.fr/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.106 Safari/537.36"
How do I fix this?
I would start by looking into log file of your application, 404 response could be caused by many elements of your stack.
When you open your application root page you should see a request logged to log/production.log file (assuming this is in production environment). If you don't see it, look up the stack at nginx error log (normally Passenger is using this one for stderr).
I resolved by commenting the "location" part of the nginx conf:
# location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
# try_files $uri $uri/ =404;
# }

Rails 5.1.5 - 500 Errors Without Info In Logs

I am running a Rails 5.1.5 app with webpacker 3.2.2, postgres 9.4, apache2, and passenger. I am not sure what other info is relevant here but can edit as necessary.
My app works locally in development. After I deploy to the server, it will work but eventually gives a 500 error on certain pages from 2 different model/controller actions.
Here is the access log that does not give much info:
71.143.443.562 - - [28/Feb/2018:19:15:11 -0700] "GET /posts/jewelry-appraisals-got-even-easier HTTP/1.1" 500 497 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML,$
I am not sure why and do not know how to reproduce the error (or where to find it). Any ideas? Thank you!
Update -
Here is the apache2 error.log
[ N 2018-02-28 15:56:07.0526 1243/T3 age/Cor/CoreMain.cpp:917 ]: Checking whether to disconnect long-running connections for process 1416, application /var/www/sellmyjewelry/current (productio$
here is my rails production.log:
I, [2018-03-01T02:59:04.227988 #6241] INFO -- : [a45553e8-7b89-45ad-ba05-dade585c17d4] Completed 401 Unauthorized in 2ms (ActiveRecord: 0.4ms)
F, [2018-03-01T02:59:04.228965 #6241] FATAL -- : [a45553e8-7b89-45ad-ba05-dade585c17d4]
F, [2018-03-01T02:59:04.229058 #6241] FATAL -- : [a45553e8-7b89-45ad-ba05-dade585c17d4] NoMethodError (To respond to a custom format, register it as a MIME type first: http://guides.rubyonrails.org$
F, [2018-03-01T02:59:04.229093 #6241] FATAL -- : [a45553e8-7b89-45ad-ba05-dade585c17d4]
F, [2018-03-01T02:59:04.229149 #6241] FATAL -- : [a45553e8-7b89-45ad-ba05-dade585c17d4] actionpack (5.1.5) lib/abstract_controller/collector.rb:26:in `method_missing
'
Update #2
I made it so all the errors are local. I get a "Mime-Type" error on pages that use the method .html_safe on longer pieces of content. I added mime types in the initialize file but still get the error.
Here is a screenshot of the error I get:
Thanks for the help.

Dropwizard not logging to file

I have logging configured in my dropwizard yml file to log to a file and not to the console, however some logs are still being logged to the console.
service.yml
logging:
level: INFO
appenders:
- type: file
threshold: DEBUG
logFormat: "%-6level [%d{HH:mm:ss.SSS}] [%t] %logger{5} - %X{code} %msg %n"
currentLogFilename: /tmp/application.log
archivedLogFilenamePattern: /tmp/application-%d{yyyy-MM-dd}.log
archivedFileCount: 7
timeZone: UTC
When I execute I get my service logs in my log file but i get request logs on the console and not in the log file
127.0.0.1 - - [11/Nov/2015:22:31:52 +0000] "GET /api/v1/hello HTTP/1.1" 200 - "-" "curl/7.15.3 (x86_64-unknown-linux-gnu) libcurl/7.15.3 OpenSSL/0.9.8w" 1
Im using Dropwizard 0.7.1
You will need to configure an appender for request logs. The default appender is console.
Reference: https://dropwizard.github.io/dropwizard/0.7.1/docs/manual/configuration.html#request-log

Log output too verbose - how to change log level?

I'm trying to debug my simple Rails app and find navigating and printing with puts works well but the log in Bash is way too verbose. I've looked through lots of questions/responses here and have tried some of them but without success. I tried importing the quiet_assets Gem with
gem 'quiet_assets', group: :development
and have tried changing log levels to :error in production.rb
config.log_level = Logger::error
But I still get all the GET requests of Assets when navigating making review of logs difficult.
Worth noting I've tried restarting the server and ran bundle just in case. I did hard refresh even though browser cache shouldn't have anything to do with this.
Here's example navigation:
20:08:55 web.1 | 127.0.0.1 - - [22/Oct/2014 20:08:55] "GET /application.js HTTP/1.1" 304 - 0.0185
20:08:55 web.1 | 127.0.0.1 - - [22/Oct/2014 20:08:55] "GET /application.css HTTP/1.1" 304 - 0.0790
20:08:56 web.1 | 127.0.0.1 - - [22/Oct/2014 20:08:56] "GET /brands/46 HTTP/1.1" 304 - 0.0111
20:08:56 web.1 | 127.0.0.1 - - [22/Oct/2014 20:08:56] "GET /application.js HTTP/1.1" 304 - 0.0062
20:08:56 web.1 | 127.0.0.1 - - [22/Oct/2014 20:08:56] "GET /application.css HTTP/1.1" 304 - 0.0066
20:08:57 web.1 | 127.0.0.1 - - [22/Oct/2014 20:08:57] "GET /brands/46 HTTP/1.1" 304 - 0.0105
20:08:57 web.1 | 127.0.0.1 - - [22/Oct/2014 20:08:57] "GET /application.js HTTP/1.1" 304 - 0.0056
20:08:57 web.1 | 127.0.0.1 - - [22/Oct/2014 20:08:57] "GET /application.css HTTP/1.1" 304 - 0.0063
20:09:13 web.1 | E, [2014-10-22T20:09:13.695412 #5880] ERROR -- : worker=0 PID:5916 timeout (16s > 15s), killing
20:09:13 web.1 | E, [2014-10-22T20:09:13.704183 #5880] ERROR -- : reaped #<Process::Status: pid 5916 SIGKILL (signal 9)> worker=0
20:09:13 web.1 | I, [2014-10-22T20:09:13.710805 #5923] INFO -- : worker=0 ready
It seems you are seeing production log, are you? if that is the case quiet_assets should be included in production too.
gem 'quiet_assets'

ruby on rails use openid auth Exception

I use ruby-openid to allow auth with openid
when i use ruby 1.8 with rails as below:
ii rails 2.3.5-1.2+squeeze1 MVC ruby based framework geared for web application development
ii rails-ruby1.8 2.3.5-1.2+squeeze1 MVC ruby based framework geared for web application development
ii rake 0.8.7-2 a ruby build program
ii ruby 4.5 An interpreter of object-oriented scripting language Ruby
ii ruby1.8 1.8.7.302-2squeeze1 Interpreter of object-oriented scripting language Ruby 1.8
ii ruby1.8-dev 1.8.7.302-2squeeze1 Header files for compiling extension modules for the Ruby 1.8
ii rubygems 1.3.7-3 package management framework for Ruby libraries/applications
ii rubygems1.8 1.3.7-3 package management framework for Ruby libraries/applications
everything work fine.
but when i update to:
ruby 1.9.3p125 (2012-02-16) [x86_64-linux]
Rails 3.2.2
1.8.11
after auth succ from the the OpenID Provider and return to my page, exception raised:
Verification failed: Unexpected parameter (not on return_to): 'controller'=nil)
i try to read idres.rb and found:
#message.get_args(BARE_NS).each_pair do |bare_key, bare_val|
rt_val = return_to_parsed_query[bare_key]
if not return_to_parsed_query.has_key? bare_key
# This may be caused by your web framework throwing extra
# entries in to your parameters hash that were not GET or
# POST parameters. For example, Rails has been known to
# add "controller" and "action" keys; another server adds
# at least a "format" key.
raise ProtocolError, ("Unexpected parameter (not on return_to): "\
"'#{bare_key}'=#{rt_val.inspect})")
end
if rt_val != bare_val
raise ProtocolError, ("Parameter '#{bare_key}' value "\
"#{bare_val.inspect} does not match "\
"return_to's value #{rt_val.inspect}")
i am a newbie to ruby. how to solve this ? thanks very much
log below:
Started GET "/consumer/start" for 218.107.55.254 at 2012-03-06 18:12:26 +0800 Processing by ConsumerController#start as HTML Redirected to http://pip.verisignlabs.com/server?openid.assoc_handle=e25c42b0-6774-11e1-9fdb-8f540be9bc3e&openid.claimed_id=http%3A%2F%2Fstutiredboy.pip.verisignlabs.com%2F&openid.identity=http%3A%2F%2Fstutiredboy.pip.verisignlabs.com%2F&openid.mode=checkid_setup&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.ns.sreg=http%3A%2F%2Fopenid.net%2Fextensions%2Fsreg%2F1.1&openid.realm=http%3A%2F%2F114.113.197.25%3A3000%2Fconsumer&openid.return_to=http%3A%2F%2F114.113.197.25%3A3000%2Fconsumer%2Fcomplete%3Fdid_sreg%3Dy&openid.sreg.required=email%2Cnickname%2Cfullname Completed 302 Found in 4914ms (ActiveRecord: 0.0ms)
Started GET "/consumer/complete?did_sreg=y&openid.sreg.fullname=tiredboy&openid.assoc_handle=e25c42b0-6774-11e1-9fdb-8f540be9bc3e&openid.response_nonce=2012-03-06T10%3A12%3A32ZzI16RQ%3D%3D&openid.sreg.email=stumyreg%40gmail.com&openid.sreg.nickname=tiredboy&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.mode=id_res&openid.op_endpoint=http%3A%2F%2Fpip.verisignlabs.com%2Fserver&openid.pape.auth_policies=http%3A%2F%2Fschemas.openid.net%2Fpape%2Fpolicies%2F2007%2F06%2Fnone&openid.claimed_id=http%3A%2F%2Fstutiredboy.pip.verisignlabs.com%2F&openid.sig=yNsdpak%2FyNi%2BcFp0oxmjtL3DmoY%3D&openid.identity=http%3A%2F%2Fstutiredboy.pip.verisignlabs.com%2F&openid.ns.pape=http%3A%2F%2Fspecs.openid.net%2Fextensions%2Fpape%2F1.0&openid.pape.auth_time=2012-03-06T07%3A16%3A09Z&openid.signed=assoc_handle%2Cidentity%2Cresponse_nonce%2Creturn_to%2Cclaimed_id%2Cop_endpoint%2Cns.sreg%2Csreg.nickname%2Csreg.email%2Csreg.fullname%2Cns.pape%2Cpape.auth_policies%2Cpape.auth_time&openid.ns.sreg=http%3A%2F%2Fopenid.net%2Fextensions%2Fsreg%2F1.1&openid.return_to=http%3A%2F%2F114.113.197.25%3A3000%2Fconsumer%2Fcomplete%3Fdid_sreg%3Dy" for 218.107.55.254 at 2012-03-06 18:12:32 +0800 Processing by ConsumerController#complete as HTML Parameters: {"did_sreg"=>"y", "openid.sreg.fullname"=>"tiredboy", "openid.assoc_handle"=>"e25c42b0-6774-11e1-9fdb-8f540be9bc3e", "openid.response_nonce"=>"2012-03-06T10:12:32ZzI16RQ==", "openid.sreg.email"=>"stumyreg#gmail.com", "openid.sreg.nickname"=>"tiredboy", "openid.ns"=>"http://specs.openid.net/auth/2.0", "openid.mode"=>"id_res", "openid.op_endpoint"=>"http://pip.verisignlabs.com/server", "openid.pape.auth_policies"=>"http://schemas.openid.net/pape/policies/2007/06/none", "openid.claimed_id"=>"http://stutiredboy.pip.verisignlabs.com/", "openid.sig"=>"yNsdpak/yNi+cFp0oxmjtL3DmoY=", "openid.identity"=>"http://stutiredboy.pip.verisignlabs.com/", "openid.ns.pape"=>"http://specs.openid.net/extensions/pape/1.0", "openid.pape.auth_time"=>"2012-03-06T07:16:09Z", "openid.signed"=>"assoc_handle,identity,response_nonce,return_to,claimed_id,op_endpoint,ns.sreg,sreg.nickname,sreg.email,sreg.fullname,ns.pape,pape.auth_policies,pape.auth_time", "openid.ns.sreg"=>"http://openid.net/extensions/sreg/1.1", "openid.return_to"=>"http://114.113.197.25:3000/consumer/complete?did_sreg=y"} Redirected to http://114.113.197.25:3000/consumer Completed 302 Found in 3ms (ActiveRecord: 0.0ms)
Started GET "/consumer" for 218.107.55.254 at 2012-03-06 18:12:32 +0800 Processing by ConsumerController#index as HTML Rendered consumer/index.html within layouts/application (0.3ms) Completed 200 OK in 3ms (Views: 3.0ms | ActiveRecord: 0.0ms)
Started GET "/assets/application.css?body=1" for 218.107.55.254 at 2012-03-06 18:12:32 +0800 Served asset /application.css - 304 Not Modified (0ms) [2012-03-06 18:12:32] WARN Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
Started GET "/assets/jquery_ujs.js?body=1" for 218.107.55.254 at 2012-03-06 18:12:32 +0800 Served asset /jquery_ujs.js - 304 Not Modified (0ms) [2012-03-06 18:12:32] WARN Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
Started GET "/assets/application.js?body=1" for 218.107.55.254 at 2012-03-06 18:12:32 +0800 Served asset /application.js - 304 Not Modified (0ms) [2012-03-06 18:12:32] WARN Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
Started GET "/assets/jquery.js?body=1" for 218.107.55.254 at 2012-03-06 18:12:32 +0800 Served asset /jquery.js - 304 Not Modified (0ms) [2012-03-06 18:12:32] WARN Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
*** /usr/local/ruby/lib/ruby/gems/1.9.1/gems/ruby-openid-2.1.8/lib/openid/consumer/idres.rb 2012-03-06 15:13:54.000000000 +0800
--- /home/tiredboy/login/gem/ruby/1.9.1/gems/ruby-openid-2.1.8/lib/openid/consumer/idres.rb 2012-03-07 13:25:58.000000000 +0800
***************
*** 170,175 ****
--- 170,178 ----
end
#message.get_args(BARE_NS).each_pair do |bare_key, bare_val|
rt_val = return_to_parsed_query[bare_key]
+ if bare_key == "controller" or bare_key == "action"
+ next
+ end
if not return_to_parsed_query.has_key? bare_key
# This may be caused by your web framework throwing extra
# entries in to your parameters hash that were not GET or
i ignore controller and action as temporary solution
It's problem created by Rails, it sometimes adds some parameters like "controller" and "action" in the url.
Instead of modifying the source of idres.rb, you can modify your Rails controller in which you manage the return call in this way:
parameters = params.reject{|k,v|request.path_parameters[k]}.reject{|k,v| k == 'action' || k == 'controller'}
like I found in an issue on the ruby-openid gem on GitHub
This solved the problem for me.

Resources