Formatting of Rails logs when using ActiveRecord - ruby-on-rails

So I have an application that earlier did not need ActiveRecord - thus we removed ActiveRecord from the application and formatted our logging as such:
In application.rb:
class DreamLogFormatter < Logger::Formatter
SEVERITY_TO_COLOR_MAP = {'DEBUG'=>'32', 'INFO'=>'0;37', 'WARN'=>'35', 'ERROR'=>'31', 'FATAL'=>'31', 'UNKNOWN'=>'37'}
def call(severity, time, progname, msg)
color = SEVERITY_TO_COLOR_MAP[severity]
"\033[0;37m[%s] \033[#{color}m%5s - %s\033[0m\n" % [time.to_s(:short), severity, msg2str(msg)]
end
end
class ActiveSupport::BufferedLogger
def formatter=(formatter)
#log.formatter = formatter
end
end
In development.rb
config.logger = Logger.new("log/#{Rails.env}.log")
config.logger.formatter = DreamLogFormatter.new
ActiveResource::Base.logger = Logger.new("log/#{Rails.env}.log")
ActiveResource::Base.logger.formatter = DreamLogFormatter.new
Note: The ActiveResource logger configuration was added because we want the URL output of our ActiveResource calls like so:
GET http://localhost:2524/users/
--> 200 OK 239 (13.0ms)
Logging with this configuration gave us a nice output combination of ActiveResource calls and our own logging using Rails.logger.
However, we have needed to add ActiveRecord back into our application as we needed to change our session storage from cookie store to ActiveRecord store. And since adding ActiveRecord back in, the logging is no longer working nicely.
Previous Log Output:
Started GET "/audit/?key1=value1&key2=value2" for 0:0:0:0:0:0:0:1%0 at 2012-08-15 15:39:58 -0400
[15 Aug 15:39] INFO - Processing by AuditController#index as HTML
[15 Aug 15:39] INFO - Parameters: {"utf8"=>"✓", "key1"=>"value1", "key2"=>"value2"}
[15 Aug 15:39] INFO - GET http://localhost:2524/api/users/jeff/prefs/?label=language
[15 Aug 15:39] INFO - --> 200 OK 151 (55.0ms)
[15 Aug 15:39] WARN - There is no user currently logged in - retrieving default theme.
[15 Aug 15:39] INFO - GET http://localhost:2524/api/users/jeff/prefs/?label=theme
[15 Aug 15:39] INFO - --> 200 OK 151 (35.0ms)
Note: What I really enjoy about this format is each request is logged beginning with a Started GET <URL> or POST or PUT, etc - followed by which controller and function is doing the processing and what the parameters sent are. This is extremely helpful for debugging. As well, this format allowed us to print out our own Rails.logger logging information.
Current Log Output (With ActiveRecord):
[20 Aug 11:40] INFO - GET http://localhost:2524/api/users/jeff
[20 Aug 11:40] INFO - --> 200 OK 199 (144.0ms)
[20 Aug 11:40] INFO - GET http://localhost:2524/api/users/jeff/prefs/?label=language
[20 Aug 11:40] INFO - --> 200 OK 148 (12.0ms)
[20 Aug 11:40] INFO - GET http://localhost:2524/api/users/jeff/prefs/?label=theme
[20 Aug 11:40] INFO - --> 200 OK 155 (15.0ms)
Essentially all we get now is a constant stream of URL calls - it doesn't log anything coming from Rails.logger and also there is no separation between different requests - it is literally just one constant stream of URL's and responses.
I have tried setting ActiveResource::Base.logger = ActiveRecord::Base.logger as a lot of the blogs recommend but that just made things worse - it logged a couple URLs, and then just stopped logging completely unless it was at ERROR level (and nowhere do I set the logging level, so it should still be at default)
Any help or suggestions would be greatly appreciated!! Thanks

Maybe the implementation of lograge will help:
https://github.com/roidrage/lograge/blob/master/lib/lograge.rb#L57

Related

Rails 5 - Retrieving MiniMagick-ed version of image (CarrierWave/Google Cloud Storage)

I'm currently using Google Cloud Storage and CarrierWave to handle my file uploading.
Just to be able to explain the "problem" clearly, lets assume I have a model called User that has the column avatar which holds the image string.
After the user uploads their file, this is done:
version :foo1 do
process :auto_orient
process resize_to_fill: [150, 150]
end
version :foo2 do
process :auto_orient
process resize_to_fill: [250, 250]
end
version :foo3 do
process :auto_orient
process resize_to_fill: [350, 350]
end
Which successfully creates the various versions of the image in my bucket.
So the problem is upon looking at my logs, I realised that when i do
<%= image_tag(User.avatar.foo1.url) %>
The query is actually being performed for every single variation of the image, so it calls the original one, along with foo1-3, before rendering foo1.
So the question is, how do i retrieve the specific variation needed only? (I'm not sure if my uploader or carrierwave code is essential here. If it is, let me know and i will include it in)
For reference, this is what I see in my logs when I perform User.avatar.foo1.url, which leads me to think that I'm making an "excessive" call to Google Cloud Storage. (I replaced certain details which i thought might be sensitive with foo)
Sending HTTP get https://www.googleapis.com/storage/v1/b/foo?
200
#<Hurley::Response GET https://www.googleapis.com/storage/v1/b/foo == 200 (374 bytes) 69ms>
Success - #<Google::Apis::StorageV1::Bucket:foo
#etag="CAI=",
#id="foo",
#kind="storage#bucket",
#location="ASIA-SOUTHEAST1",
#metageneration="2",
#name="foo",
#project_number="foo",
#self_link="https://www.googleapis.com/storage/v1/b/foo",
#storage_class="NEARLINE",
#time_created=Wed, 20 Dec 2017 05:50:41 +0000,
#updated=Wed, 20 Dec 2017 07:08:46 +0000>
Sending HTTP get https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2FWeChat_Image_20180112102430.jpg?
200
#<Hurley::Response GET https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2FWeChat_Image_20180112102430.jpg == 200 (926 bytes) 62ms>
Success - #<Google::Apis::StorageV1::Object:0x007f98b3332220
#bucket="foo",
#content_type="image/jpeg",
#crc32c="5SNA/w==",
#etag="CNrr8om4h9sCEAE=",
#generation="1526377710663130",
#id=
"foo/uploads/profile/avatar/328/WeChat_Image_20180112102430.jpg/1526377710663130",
#kind="storage#object",
#md5_hash="g6cUQBBiGC31uRJUtM1dng==",
#media_link=
"https://www.googleapis.com/download/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2FWeChat_Image_20180112102430.jpg?generation=1526377710663130&alt=media",
#metageneration="1",
#name="uploads/profile/avatar/328/WeChat_Image_20180112102430.jpg",
#self_link=
"https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2FWeChat_Image_20180112102430.jpg",
#size="5499",
#storage_class="NEARLINE",
#time_created=Tue, 15 May 2018 09:48:30 +0000,
#time_storage_class_updated=Tue, 15 May 2018 09:48:30 +0000,
#updated=Tue, 15 May 2018 09:48:30 +0000>
Sending HTTP get https://www.googleapis.com/storage/v1/b/foo?
200
#<Hurley::Response GET https://www.googleapis.com/storage/v1/b/foo == 200 (374 bytes) 53ms>
Success - #<Google::Apis::StorageV1::Bucket:foo
#etag="CAI=",
#id="foo",
#kind="storage#bucket",
#location="ASIA-SOUTHEAST1",
#metageneration="2",
#name="foo",
#project_number="foo",
#self_link="https://www.googleapis.com/storage/v1/b/foo",
#storage_class="NEARLINE",
#time_created=Wed, 20 Dec 2017 05:50:41 +0000,
#updated=Wed, 20 Dec 2017 07:08:46 +0000>
Sending HTTP get https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2Fthumb_WeChat_Image_20180112102430.jpg?
200
#<Hurley::Response GET https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2Fthumb_WeChat_Image_20180112102430.jpg == 200 (950 bytes) 58ms>
Success - #<Google::Apis::StorageV1::Object:foo
#bucket="foo",
#content_type="image/jpeg",
#crc32c="XDUTLQ==",
#etag="CN/7+oq4h9sCEAE=",
#generation="1526377712893407",
#id=
"foo/uploads/profile/avatar/328/thumb_WeChat_Image_20180112102430.jpg/1526377712893407",
#kind="storage#object",
#md5_hash="CRdX68ncBsEqJFY/uRg03A==",
#media_link=
"https://www.googleapis.com/download/storage/v1/b/deanslist-9322/o/uploads%2Fprofile%2Favatar%2F328%2Fthumb_WeChat_Image_20180112102430.jpg?generation=1526377712893407&alt=media",
#metageneration="1",
#name="uploads/profile/avatar/328/thumb_WeChat_Image_20180112102430.jpg",
#self_link=
"https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2Fthumb_WeChat_Image_20180112102430.jpg",
#size="5923",
#storage_class="NEARLINE",
#time_created=Tue, 15 May 2018 09:48:32 +0000,
#time_storage_class_updated=Tue, 15 May 2018 09:48:32 +0000,
#updated=Tue, 15 May 2018 09:48:32 +0000>
Sending HTTP get https://www.googleapis.com/storage/v1/b/foo?
200
#<Hurley::Response GET https://www.googleapis.com/storage/v1/b/foo == 200 (374 bytes) 59ms>
Success - #<Google::Apis::StorageV1::Bucket:0x007f98b3252328
#etag="CAI=",
#id="foo",
#kind="storage#bucket",
#location="ASIA-SOUTHEAST1",
#metageneration="2",
#name="foo",
#project_number="1028403006209",
#self_link="https://www.googleapis.com/storage/v1/b/foo",
#storage_class="NEARLINE",
#time_created=Wed, 20 Dec 2017 05:50:41 +0000,
#updated=Wed, 20 Dec 2017 07:08:46 +0000>
Sending HTTP get https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2Ftestimonial_WeChat_Image_20180112102430.jpg?
200
#<Hurley::Response GET https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2Ftestimonial_WeChat_Image_20180112102430.jpg == 200 (974 bytes) 59ms>
Success - #<Google::Apis::StorageV1::Object:0x007f98b6a82a40
#bucket="foo",
#content_type="image/jpeg",
#crc32c="VFIp7Q==",
#etag="CJbO2Yu4h9sCEAE=",
#generation="1526377714444054",
#id=
"foo/uploads/profile/avatar/328/testimonial_WeChat_Image_20180112102430.jpg/1526377714444054",
#kind="storage#object",
#md5_hash="QGE8XArsE5C9QD2A6ybpVg==",
#media_link=
"https://www.googleapis.com/download/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2Ftestimonial_WeChat_Image_20180112102430.jpg?generation=1526377714444054&alt=media",
#metageneration="1",
#name=
"uploads/profile/avatar/328/testimonial_WeChat_Image_20180112102430.jpg",
#self_link=
"https://www.googleapis.com/storage/v1/b/foo/o/uploads%2Fprofile%2Favatar%2F328%2Ftestimonial_WeChat_Image_20180112102430.jpg",
#size="2481",
#storage_class="NEARLINE",
#time_created=Tue, 15 May 2018 09:48:34 +0000,
#time_storage_class_updated=Tue, 15 May 2018 09:48:34 +0000,
#updated=Tue, 15 May 2018 09:48:34 +0000>

Sending mail in Redmine

I have updated my Remine on servers from 2.3.0 to 2.4.1 and one (and only one of them) stops sending mail. If I was switched it back to 2.3.0 version, all works fine.
I am going to try to debug the code and just wondering which file in source code will be a good starting point to it ?
I have found notified_users(), recipients(), each_notification() in Issue model, but where is a code line which send email ?
Updated:
When I edit issue I get next message on console:
Rendered mailer/_issue.html.erb (2.2ms)
Rendered mailer/issue_edit.html.erb within layouts/mailer (4.8ms)
Sent email "[Redmine - ÐапÑÐ¾Ñ #13757] test" (26ms)
to: mymail#gmcs.ru
Date: Thu, 05 Dec 2013 17:34:07 +0400
....
cG9ydC5nbWNzLnJ1L215L2FjY291bnQ8L2E+PC9wPjwvc3Bhbj4KPC9ib2R5
Pgo8L2h0bWw+Cg==
----==_mimepart_52a080cfa4564_af93f8d53ef7714733eb--
Email delivery error: wrong argument (NilClass)! (Expected kind of OpenSSL::SSL::SSLContext)
(5.9ms) COMMIT
Redirected to http://vm-mecomstracker:90/issues/13757
Completed 302 Found in 574.3ms (ActiveRecord: 33.5ms)
I think you should look at app/models/mailer.
For example notified_users is used here
If update breaks sending emails I suggest to review config files, for example config/configuration.yml.example and check fresh issues on redmine.org

Slow Rails website problem

I have a Rails website which, for the last 24 hours has been running very slowly. The odd thing is that (see logs below) every request is taking 45 or 46 seconds, as though is is waiting for some kind of timeout. Static content is loading quickly and the server is otherwise performing normally.
The site is at http://plantality.com and is hosted by rail playground. It uses Apache + Passenger, Rails 2.1.1, attachment_fu, thinking sphinx, rmagick, bbruby.
Server log samples:
Processing WikiController#finder (for 66.249.67.103 at 2010-12-05 02:58:59) [GET]
Session ID: e9e947f58fb1f0f60eeff2fc62a2de72
Parameters: {"group"=>"1", "upright"=>"1", "houseplant"=>"1", "wi_6to12"=>"1", "bushy"=>"1", "action"=>"finder", "controller"=>"wiki", "pots_tubs"=>"1"}
Cookie set: referer=; path=/
Cookie set: geo_country=; path=/
Rendering template within layouts/global
Rendering wiki/finder
Completed in 45.06793 (0 reqs/sec) | Rendering: 0.03833 (0%) | DB: 0.00878 (0%) | 200 OK [http://plantality.com/finder?bushy=1&group=1&houseplant=1&pots_tubs=1&upright=1&wi_6to12=1]
Processing MainController#index (for 67.225.164.12 at 2010-12-05 02:59:44) [GET]
Session ID: 0389102261c509523911c65b28c7661b
Parameters: {"action"=>"index", "controller"=>"main"}
Cookie set: referer=; path=/
Cookie set: geo_country=; path=/
Rendering template within layouts/global
Rendering main/index
Completed in 45.42652 (0 reqs/sec) | Rendering: 0.23435 (0%) | DB: 0.04589 (0%) | 200 OK [http://plantality.com/]
Solution: I was using an IP to country geo-coding service. I coded it so that the result was cached in a cookie, but my site was getting about 100,000 daily hits from google. Google (and other spiders) weren't returning the cookie to me, so the IP service was throttling my usage of it.
Yah boo to all those who said this was a hosting issue and not a programming one.

Rails and ip-spoofing

I have small application on Rails 2.3.2, served with nginx+thin(127).
OS FreeBSD 7.1, DB - Posgresql.
Twise at this week my app has fall.
In log I get something like (~2-50 request per second):
/!\ FAILSAFE /!\ Mon Oct 04 20:13:55 +0300 2010
Status: 500 Internal Server Error
bad content body
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/vendor/rack-1.0/rack/utils.rb:311:in `parse_multipart'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/vendor/rack-1.0/rack/request.rb:125:in `POST'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/request.rb:428:in `request_parameters'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/request.rb:381:in `parameters'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/base.rb:1279:in `assign_shortcuts'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/base.rb:518:in `process_without_filters'
Or:
/!\ FAILSAFE /!\ Tue Nov 09 09:24:39 +0200 2010
Status: 500 Internal Server Error
IP spoofing attack?!
HTTP_CLIENT_IP="XX.XX.XX.XX"
HTTP_X_FORWARDED_FOR="192.168.XX.XX, YY.YY.YY.YY"
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/request.rb:229:in `remote_ip'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/base.rb:1372:in `request_origin'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/base.rb:1304:in `log_processing_for_request_id'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/base.rb:1296:in `log_processing'
/usr/home/../../history/vendor/rails/actionpack/lib/action_controller/base.rb:522:in `process_without_filters'
After that system rests into max open file limit (I guess it open by postgesql sessions), postgresql can't establish new connection and app fall.
Any suggestion, how I can protect my asspp in this situation?
Quoted from rails 2.3 release notes :
The fact that Rails checks for IP spoofing can be a nuisance for sites that do heavy traffic with cell phones, because their proxies don’t generally set things up right. If that’s you, you can now set ActionController::Base.ip_spoofing_check = false to disable the check entirely.

In a Rails log file, what does "Processing by ProductsController#index as */*" mean?

In my Rails log file, I see lots of
Started GET "/" for 63.148.78.244 at Fri Sep 24 19:03:39 +0000 2010
Processing by ProductsController#index as HTML
I understand this means Rails is serving up an HTML page. However, what does this mean?
Started GET "/" for 63.148.78.244 at Fri Sep 24 18:05:51 +0000 2010
Processing by ProductsController#index as */*
Completed in 0ms
Why the */*?
It depends on the HTTP_ACCEPT header that is sent by the browser. The common scenario is that the browser sends list of all MIME types that can process and server returns the result in one of them - typically HTML.
But in some cases it's not this way. For example if you use wget without any other parameters.
Try
wget http://yourserver
and you will see in your log file * / * which means that the "browser" accepts anything you will send back (it's quite obvious that wget can accept anything as it is just storing it into the file).

Resources