Rails 2.3.5 parses date inconsistently - ruby-on-rails

To start, I am using an old rails version because of Redmine.
I am trying to write a script for updating Redmine's database dump timestamps' to a specific timezone, therefore I need ruby to parse them and change the timezone, assuming rails can handle dst boundaries.
Nevertheless, I found an inconsistency in (or misunderstood) how Time.parse works.
Here's the script I executed on my Rails 2.3.5 raw application:
impulse#ImpulseServer:~/pgtz_converter$ script/console
Loading development environment (Rails 2.3.5)
>> Time.zone
=> #<ActiveSupport::TimeZone:0x7fa94a57b9a8 #tzinfo=nil, #name="UTC", #utc_offset=0>
>> Time.parse('2011-02-19 23:00:00')
=> Sat Feb 19 23:00:00 -0200 2011
>> Time.zone
=> #<ActiveSupport::TimeZone:0x7fa94a57b9a8 #tzinfo=nil, #name="UTC", #utc_offset=0>
>> Time.parse('2011-02-20 00:00:00')
=> Sun Feb 20 00:00:00 -0300 2011
>> Time.zone
=> #<ActiveSupport::TimeZone:0x7fa94a57b9a8 #tzinfo=nil, #name="UTC", #utc_offset=0>
>> Time.parse('2011-02-19 23:00:00')
=> Sat Feb 19 23:00:00 -0300 2011
>> Time.zone
=> #<ActiveSupport::TimeZone:0x7fa94a57b9a8 #tzinfo=nil, #name="UTC", #utc_offset=0>
>> exit
Here's what is going on above:
The server local datetime zone is BRT.
impulse#ImpulseServer:~/pgtz_converter$ date
Fri Apr 8 19:17:34 BRT 2011
First I check Time.zone (which is UTC), then try to parse a DST date.
The output is fine, DST is on (I presume Time.zone = UTC means Ruby checked the server zone configuration and used it).
Then I try another date, in which DST was not active anymore (DST ended on 02/20/2011 00:00 here).
This date is correctly parsed (GMT-3).
Now the problem begins.
I try to parse again the old DST date, and the result is wrong! It wasn't read as a DST date!
I thought perhaps the parsing changed the Time.zone, but as it is pasted it is UTC all the time.
What is causing this ?
Thank you for reading.
Nilo

You might want to change to use the ActiveSupport::TimeWithZone class in rails.

Related

Rsyslog collect logs from different timezones

Im using rsyslog on server to collect logs from remote hosts.
Collect server config:
# timedatectl
Local time: Wed 2022-04-27 16:02:43 MSK
Universal time: Wed 2022-04-27 13:02:43 UTC
RTC time: n/a
Time zone: Europe/Moscow (MSK, +0300)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
# cat /etc/rsyslog.d/20_external.conf
$CreateDirs on
$PreserveFQDN on
# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")
# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="514")
template(
name="external"
type="string"
string="/var/log/external/%HOSTNAME%/%syslogfacility-text%.%programname%.%syslogseverity-text%.log"
)
action(
type="omfile"
dirCreateMode="0775"
FileCreateMode="0644"
dynaFile="external"
)
On remote host
# timedatectl
Local time: Wed 2022-04-27 13:04:03 UTC
Universal time: Wed 2022-04-27 13:04:03 UTC
RTC time: n/a
Time zone: UTC (UTC, +0000)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
# cat /etc/rsyslog.d/10-external.conf
*.* #rserver
# logger "hello, local time $(date)"
And get on rsyslogserver:
cat /var/log/external/ruser.home.xmu/user.root.notice.log
2022-04-27T13:07:06+03:00 ruser.home.xmu root: hello, local time 2022-04-27T13:07:06 UTC
# date
2022-04-27T16:08:56 MSK
What i can do for change time zone settings for some remote hosts on collect-server?
When i reserch incedents from all servers the time does not match in logs. I want the time on the collector in the logs to be in his time zone.
2022-04-27T16:07:06+03:00 ruser.home.xmu root: hello, local time 2022-04-27T13:07:06 UTC
You can define the timezone in rsyslog on the client - which in my opinion is the cleaner solution.
In /etc/rsyslog.conf do the following:
Comment/remove the current template
# Use default timestamp format
#$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
Then add the timezone, as well as a custom log template:
timezone(id="MSK" offset="+03:00")
# Custom time format
$template myTemplate,"%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"
$ActionFileDefaultTemplate myTemplate
However, if you can't access the remote client which is sending the logs, it's possible to use the timestamp when the log was received on the server.
$template myTemplate,"%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"
ruleset(name="myRuleset"){
$ActionFileDefaultTemplate myTemplate
# Do some other stuff
}
module(load="imtcp")
input(type="imtcp" port="5000" ruleset="myRuleset")
module(load="imudp")
input(type="imudp" port="5000" ruleset="myRuleset")
NOTE: Don't forget to restart the rsyslog service after applying the changes.
sudo service rsyslog restart
EDIT:
Creating a template using the advanced syntax would look like the following:
template (name="myTemplate" type="string"
string="%timegenerated% %HOSTNAME% %syslogtag%%msg%\n")
The string is the actual template of the messages that should be logged, not the destination to which the messages should be logged.
I ran into a similar problem recently. I think the simplest way to synchronize data coming from multiple sources is to use UTC timestamps everywhere, this way you avoid any timezone and summer time/winter time problems.
I think the problem here (and the problem I had) is that the syslog ouput uses the rfc3164 format which does not include the timezone information:
<6>Feb 28 12:00:00 192.168.0.1 fluentd[11111]: [error] Hello!
There's no way of knowing at what time that message actually happened without the timezone information. Since rsyslog version 8.18.0 what you can do though is define custom rsyslog templates that print the timestamps in UTC. If you use UTC everywhere you don't need that timestamp information anymore.
I'm sending my logs to fluentd, I use the following configuration in my rsyslog.conf file on my hosts:
template(
name="UTCTraditionalForwardFormat"
type="string"
string="<%PRI%>%TIMESTAMP:::date-utc% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
)
*.* action(
type="omfwd"
Target="10.5.0.9"
Port="5145"
Protocol="udp"
Template="UTCTraditionalForwardFormat"
)
The UTCTraditionalForwardFormat here is the same as the default template RSYSLOG_TraditionalForwardFormat (rsyslog templates) but with an added :::date-utc to the TIMESTAMP property.
On your rsyslog that collects logs, looking at the input module's doc there an experimental parameter DefaultTZ which should let you define the source timezone, something like this should work (I haven't tested):
input(
type="imudp"
port="514"
DefaultTZ="+00:00"
)
Assuming this DefaultTZ parameter works, this should work regardless of your hosts timezone.
Little word about fluentd, the fluentd syslog module also accepts the rfc5424 message format which does encode timezone information. I decided not to go this way though because rsyslog default templates does not seem to have a real rfc5424 version (the RSYSLOG_SyslogProtocol23Format template is very close to the rfc5424, but I don't know how close).

Why does strptime behave differently in Rails' test environment?

I'm trying to parse the date 2/31/2019, which should raise an error.
In the Rails console this works as expected in development mode, but my tests are failing in test mode because it returns March 3, 2019.
Date.strptime("2/31/2019", "%m/%d/%Y") #=> dev env: error
Date.strptime("2/31/2019", "%m/%d/%Y") #=> test env: “Sun, 03 Mar 2019”
The commands I'm using are:
rails c
RAILS_ENV=test rails c
rspec <filename>
I'm using Ruby 2.3.1 and Rails 5.0.6.
Timecop overwrites the Date.strptime definition:
[4] pry(main)> puts Date.method(:strptime).source
def strptime_with_mock_date(str = '-4712-01-01', fmt = '%F', start = Date::ITALY)
unless start == Date::ITALY
raise ArgumentError, "Timecop's #{self}::#{__method__} only " +
"supports Date::ITALY for the start argument."
end
Time.strptime(str, fmt).to_date
end
That's why the output is different between environments, since Timecop is only required in test.

Print load time per file when start a Rails 2 project

How can I print for each file how long is it taking.
I am thinking about monkey patching load or some load method in Rails, but I don't know a lot about Rails 2 inner things.
I would expect this kind of output:
config/environment.rb ---> 60 s
config/development.rb ---> 30 s
config/initializers/some.rb ---> 20 s
Any starting point is really appreciated.
Update
I am measuring time with:
from = Time.now; p from.to_s + '----- env.rb'
to = Time.now - from; p "env.rb ---" + to.to_s
Changing env.rb to boot.rb in boot.rb file, these are my results:
"Thu Feb 13 18:54:17 +0000 2014----- boot.rb"
"boot.rb ---0.001493"
"boot.rb ---0.941835"
Loading development environment (Rails 2.3.18)
"Thu Feb 13 18:54:19 +0000 2014----- env.rb"
"env.rb ---0.001378"
"Thu Feb 13 18:54:19 +0000 2014----- boot.rb"
"boot.rb ---0.001712"
"boot.rb ---0.02536"
"env.rb ---67.272745"
"env.rb ---70.847089"
I am not sure why I am getting that config/boot is being called twice. From what I see most of the time is spent in environment.rb, but I removed code and time remains the same, so it should be related with other thing.
This other thing is not bundling gems, since I have:
from = Time.now; p from.to_s + '----- env.rb'
# Be sure to restart your web server when you modify this file.
#
#
require 'rubygems'
require "bundler"
Bundler.setup
to = Time.now - from; p "env.rb ---" + to.to_s
And this is printing:
"env.rb ---0.001378"
Turned out that most of the time was spent in loading files, I started a new Rails 2.3 project with part of the models of the bigger app, and time reduced from more than a minute to about 5 seconds.

How can I access rails methods in a ruby script?

I have a ruby script in my rails app in the script directory. How can I access the in_time_zone method in the script?
If you only need that one method, you could just whack require 'active_support/all' at the top of your script.
It's in active_support/core_ext.
> require 'active_support/core_ext'
=> true
> d = DateTime.new(2000)
=> Sat, 01 Jan 2000 00:00:00 +0000
> d.in_time_zone("Alaska")
=> Fri, 31 Dec 1999 15:00:00 AKST -09:00
>
You can create a custom rake task, and then access everything in your application for instance. You can check this great screencast

hung ruby process (ruby 1.8)

Using: ruby 1.8.7 (2009-06-12 patchlevel 174) [x86_64-linux]
I have a ruby batch job that the last 2 nights in a row has hung up at about the same time.
The weird thing is when I do a kill -QUIT on the process it frees it up and continues processing.
Here is the stack when I send the SIGQUIT:
Wed Mar 23 2011 11:07:55 SignalException: SIGQUIT: SELECT * FROM `influencers` WHERE (`influencers`.`external_id` = 199884972) LIMIT 1
Wed Mar 23 2011 11:07:55 /usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record /connection_adapters/abstract_adapter.rb:219:in `log'/usr/lib/ruby/gems/1.8/gems/activerecord- 2.3.5/lib/active_record/connection_adapters/mysql_adapter.rb:323:in `execute'/usr/lib/ruby/gems/1.8 /gems/activerecord-2.3.5/lib/active_record/connection_adapters/mysql_adapter.rb:608:in `select'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all_without_query_cache'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/connection_adapters/abstract/query_cache.rb:62:in `select_all'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/base.rb:661:in `find_by_sql'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/base.rb:1548:in `find_every'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/base.rb:1505:in `find_initial'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/base.rb:613:in `find'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/base.rb:1900:in `find_by_twitter_id'/u/apps/myapp/releases/20110323011051/app/models/influencer.rb:148:in `add_follower'/u/apps/myapp/releases/20110323011051/app/models/influencer.rb:93:in `sync_follower_list'/u/apps/myapp/releases/20110323011051/app/models/influencer.rb:91:in `each'/u/apps/myapp/releases/20110323011051/app/models/influencer.rb:91:in `sync_follower_list'/u/apps/myapp/releases/20110323011051/lib/twitter_helper.rb:379:in `retrieve_followers_of_competitors'/u/apps/myapp/releases/20110323011051/lib/twitter_helper.rb:372:in `each'/u/apps/myapp/releases/20110323011051/lib/twitter_helper.rb:372:in `retrieve_followers_of_competitors'/u/apps/myapp/releases/20110323011051/vendor/gems/will_paginate-2.3.11/lib/will_paginate/finder.rb:168:in `method_missing'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/associations/association_collection.rb:369:in `method_missing_without_paginate'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/associations/association_proxy.rb:215:in `method_missing'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/associations/association_proxy.rb:215:in `each'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/associations/association_proxy.rb:215:in `send'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/associations/association_proxy.rb:215:in `method_missing'/usr/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/associations/association_collection.rb:369:in `method_missing_without_paginate'/u/apps/myapp/releases/20110323011051/vendor/gems/will_paginate-2.3.11/lib/will_paginate/finder.rb:168:in `method_missing'/u/apps/myapp/releases/20110323011051/lib/twitter_helper.rb:370:in `retrieve_followers_of_competitors'/u/apps/myapp/releases/20110323011051/lib/twitter_helper.rb:42:in `retrieve_twitter_data'/u/apps/myapp/releases/20110323011051/lib/tasks/fetch_data.rake:19:in `fetch_data'/u/apps/myapp/releases/20110323011051/lib/tasks/fetch_data.rake:11:in `each'/u/apps/myapp/releases/20110323011051/lib/tasks/fetch_data.rake:11:in `fetch_data'/u/apps/myapp/releases/20110323011051/lib/tasks/fetch_data.rake:128/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'/usr/lib/ruby/1.8/monitor.rb:242:in `synchronize'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake:31/usr/bin/rake:19:in `load'/usr/bin/rake:19
I suspect that from looking at the code im getting some kind of deadlock on the rails logger. Any suggestions on how to troubleshoot. Maybe something to do with rails logger folding. Not sure why this would start in last couple days....
I'm not sure if you use logrotate on your logs, but it could be that the logs get rotated and Rails.logger can't write anything any more. Doing a kill -QUIT would re-open the log file and continue processing.

Resources