Parsing JSON datetime? - ruby-on-rails
I am consuming a JSON API that returns datetime values as a string in the following format:
/Date(1370651105153)/
How can I parse a value like this into a datetime variable in rails?
That appears to be a UNIX timestamp (seconds since epoch). Additionally it appears to be milliseconds since epoch.
So you can convert it like so - given that value is a String that looks like:
value = "/Date(1370651105153)/"
if value =~ /\/Date\((\d+)\)\//
timestamp = $1.to_i
time = Time.at(timestamp / 1000)
# time is now a Time object
end
You need to divide by 1000 since Time#at expects its argument to be seconds and not milliseconds since the epoch.
There are a few errors in andyisnowskynet's answer (https://stackoverflow.com/a/25817039/5633460): There's a minor RegEx error which means it won't detect positive timezones; and it won't support timezones that aren't aligned to full hours such as -0430 (VET). Also, the Time object it returns could be honouring proper Ruby timezone handling instead of adding a time offset, which would be the more-conventional way to do this.
This is my improved version of that answer...
First, for datetime values expressed in this format, the time serial part (e.g. 1370651105153 in OP's example) is ALWAYS the number of milliseconds since 1970-01-01 00:00:00 GMT, and that is not influenced by the presence or absence of a timezone suffix. Hence, any timezone included as part of the string does not change the point in history that this represents. It only serves to state which timezone the "observer" was in at the time.
Ruby Time objects are able to handle these two pieces of information (i.e. the actual datetime "value" and the timezone "metadata"). To demonstrate:
a = Time.at(-1).utc
# => 1969-12-31 23:59:59 UTC
b = Time.at(-1).getlocal('+09:30')
# => 1970-01-01 09:29:59 +0930
a == b
# => true
As you can see, even though both look like different values (owing to their different timezones in which they are expressed), the == equality operator shows they actually reference the same moment in time.
Knowing this, we could improve on parse_json_datetime as follows (correcting the other errors too):
def parse_json_datetime(datetime)
# "/Date(-62135575200000-0600)/" seems to be the default date returned
# if the value is null:
if datetime == "/Date(-62135575200000-0600)/"
# Return nil because it is probably a null date that is defaulting to 0.
# To be more technically correct you could just return 0 here if you wanted:
return nil
elsif datetime =~ %r{/Date\(([-+]?\d+)([-+]\d+)?\)/}
# We've now parsed the string into:
# - $1: Number of milliseconds since the 1/1/1970 epoch.
# - $2: [Optional] timezone offset.
# Divide $1 by 1000 because it is in milliseconds and Time uses seconds:
seconds_since_epoch = $1.to_i / 1000.0
time = Time.at(seconds_since_epoch.to_i).utc
# We now have the exact moment in history that this represents,
# stored as a UTC-based "Time" object.
if $2
# We have a timezone, so convert its format (adding a colon)...
timezone = $2.gsub(/(...)(..)/, '\1:\2')
# ...then apply it to the Time object:
time = time.getlocal(timezone)
end
time
else
raise "Unrecognized date format."
end
end
Then we can test:
# See: http://momentjs.com/docs/#/parsing/asp-net-json-date/
parse_json_datetime("/Date(1198908717056-0700)/")
# => 2007-12-28 23:11:57 -0700
# 1 minute before the epoch, but in ACST:
parse_json_datetime("/Date(-60000+0930)/")
# => 1970-01-01 09:29:00 +0930
# Same as above, but converted naturally to its UTC equivalent:
parse_json_datetime("/Date(-60000+0930)/").utc
# => 1969-12-31 23:59:00 UTC
# Same as above, with timezone unspecified (implying UTC):
parse_json_datetime("/Date(-60000)/")
# => 1969-12-31 23:59:00 UTC
# OP's example:
parse_json_datetime("/Date(1370651105153)/")
# => 2013-06-08 00:25:05 UTC
# Same as above, but stated in two different timezones:
aaa = parse_json_datetime("/Date(1370651105153-0200)/")
# => 2013-06-07 22:25:05 -0200
bbb = parse_json_datetime("/Date(1370651105153-0800)/")
# => 2013-06-07 16:25:05 -0800
# As "rendered" strings they're not the same:
aaa.to_s == bbb.to_s
# => false
# But as moments in time, they are equivalent:
aaa == bbb
# => true
# And, for the sake of the argument, if they're both expressed in the
# same timezone (arbitrary '-04:00' in this case) then they also render
# as equal strings:
aaa.getlocal('-04:00').to_s == bbb.getlocal('-04:00').to_s
The integer seems to be a unixtime (in milliseconds). Just cut the last three digits off and feed the rest to Time.at:
Time.at(1370651105) # => 2013-06-08 04:25:05 +0400
For anyone who needed a bit more robust solution that included a timezone, this is the parser I came up with:
def parse_json_datetime(datetime)
# "/Date(-62135575200000-0600)/" seems to be the default date returned if the value is null
if datetime == "/Date(-62135575200000-0600)/"
# return nil because it is probably a null date that is defaulting to 0.
# to be more technically correct you could just return 0 here if you wanted.
return nil
elsif datetime =~ /\/Date\(([-+]?\d+)(-+\d+)?\)\// # parse the seconds and timezone (if present)
milliseconds_since_epoch = $1
time = Time.at(milliseconds_since_epoch.to_i/1000.0).utc # divide by 1000 because it is in milliseconds and Time uses seconds
if timezone_hourly_offset = $2
timezone_hourly_offset = timezone_hourly_offset.gsub("0","").to_i
time = time+(timezone_hourly_offset*60*60) # convert hourly timezone offset into seconds and add onto time
end
time
else
raise "Unrecognized date format."
end
end
Active Support makes it easy to do this:
#!/usr/bin/env ruby
require 'active_support/json'
require 'active_support/time'
Time.zone = 'Eastern Time (US & Canada)'
ActiveSupport.parse_json_times = true
puts ActiveSupport::JSON.decode('{"hi":"2009-08-10T19:01:02Z"}')
Which will output:
{"hi"=>Mon, 10 Aug 2009 15:01:02 EDT -04:00}
Once you turn the ActiveSupport.parse_json_times flag on, then any JSON payload you decode will get any times automatically convered into ActiveSupport::TimeWithZone objects.
Active Support uses a regex to detect times. You can read the tests to get a good idea of the time formats that it supports. Looks like it supports ISO 8601, but does not support Unix/POSIX/Epoch timestamps.
I was informed of this feature in Active Support when I created this issue against Rails.
Related
Rails - Convert time to string
This is the time format I want to convert time = Time.parse('2020-07-02 03:59:59.999 UTC') #=> 2020-07-02 03:59:59 UTC I want to convert to string in this format. "2020-07-02T03:59:59.999Z" I have tried. time.strftime("%Y-%m-%dT%H:%M:%S.999Z") Is this correct? Any better way?
You can just use Time#iso8601 with the desired number of fraction digits as an argument: time = Time.current.end_of_hour time.iso8601(3) #=> "2020-07-01T10:59:59.999Z"
If you want to handle the output format explicitly via strftime, there are some things to keep in mind: Instead of hard-coding 999, you should use %L to get the actual milliseconds: time = Time.parse('2020-07-02 03:59:59.999 UTC') #=> 2020-07-02 03:59:59 UTC time.strftime('%Y-%m-%dT%H:%M:%S.%LZ') #=> "2020-07-02T03:59:59.999Z" Use combinations for common formats, e.g. %F for %Y-%m-%d and %T for %H:%M:%S: time.strftime('%FT%T.%LZ') #=> "2020-07-02T03:59:59.999Z" If you are dealing with time zones other than UTC (maybe your machine's local time zone), make sure to convert your time instance to utc first: time = Time.parse('2020-07-02 05:59:59.999+02:00') #=> 2020-07-02 05:59:59 +0200 time.utc #=> 2020-07-02 03:59:59 UTC time.strftime('%FT%T.%LZ') #=> "2020-07-02T03:59:59.999Z" or to use %z / %:z to append the actual time zone offset: time = Time.parse('2020-07-02 05:59:59.999+02:00') time.strftime('%FT%T.%L%:z') #=> "2020-07-02T05:59:59.999+02:00"
For APIs you should use utc.iso8601: > timestamp = Time.now.utc.iso8601 => "2015-07-04T21:53:23Z" See: https://thoughtbot.com/blog/its-about-time-zones#working-with-apis
Rails absolute time from reference + french "ago" string
I need to reimport some data that was exported using the "ago" stringification helper, in French. I have a reference Time/DateTime date at which the import was done, and from there I need to substract this "time ago" difference to find the absolute time. I need to code the parse_relative_time method below Some sample input/output of what I'm trying to achieve IMPORT_DATE = Time.parse('Sat, 11 Jun 2016 15:15:19 CEST +02:00') sample_ago_day = 'Il y a 5j' # Note : 'Il y a 5j" = "5d ago" parse_relative_time(from: IMPORT_DATE, ago: sample_ago_day) # => Should output sthing like Sat, 6 Jun 2016 (DateTime object) sample_ago_month = 'Il y a 1 mois' # Note : 'Il y a 5j" = "1 month ago" parse_relative_time(from: IMPORT_DATE, ago: sample_ago_month) # => 11 May 2016 (it's not big deal if it's 10 or 11 or 12 because of months with odd numbers, just need something approximate) EDIT Range of values "il y a xj" -> x belongs to (1..31) "il y a y mois" -> y belongs to (2..10) and "un" (for one)
Let's divide the problem into 2 sub-tasks: Parse the 'ago' string Since there is no reversible way in ruby to parse an 'ago' string, lets use regular expressions to extract the data as seconds: def parse_ago(value) # If the current value matches 'il y a Xj' if match = /^il y a (.*?)j$/i.match(value) # Convert the matched data to an integer value = match[1].to_i # Validate the numeric value (between 1 and 31) raise 'Invalid days value!' unless (1..31).include? value # Convert to seconds with `days` rails helper value.days # If the current value matches 'il y a X mois' elsif match = /^il y a (.*?) mois$/i.match(value) # If the matched value is 'un', then use 1. Otherwise, use the matched value value = match[1] == 'un' ? 1 : match[1].to_i # Validate the numeric value (between 1 and 10) raise 'Invalid months value!' unless (1..10).include? value # Convert to seconds with `months` rails helper value.months # Otherwise, something is wrong (or not implemented) else raise "Invalid 'ago' value!" end end Substract from current time This is pretty straightforward; once we have the seconds from the 'ago' string; just call the ago method on the seconds extracted from the 'ago' string. An example of usage of this method for Ruby on Rails: 5.months.ago # "Tue, 12 Jan 2016 15:21:59 UTC +00:00" The thing is, you are substracting it from IMPORT_DATE, and not from current time. For your code, you need to specify the current time to IMPORT_DATE: parse_ago('Il y a 5j').ago(IMPORT_DATE) Hope this helps!
Rails: What is the correct way of saving the time of a video?
There is a time span for every video. For example, for every youtube video, the time it last may be 4minutes or 30s. The problem is how shold I save it into database? Just store it as a string like 4:30? Or change it to seconds and store it as an integer? Then how can I change it back to minute:second format? An additonal note: Now I'm making a video congrugator, so the time actually come in as seconds: 30, or just in plain format like 4:30, then how can I turn the latter format into seconds integer?
The way of keeping time depends of you business logic. I would prefer to keep length in seconds, and then convert to minutes, hours and so on. You can easily do it by simple devision: minutes = (seconds / 60) seconds_left = (seconds % 40) human_time = "#{minutes}:#{seconds_left}"
You can use the Time class to parse your time string. require 'date' # Only needed outside of rails! require 'active_support/all' # Only needed outside of rails! video_length = "04:30" # This ensures the format of video_length is HH:MM:SS def hours_minutes_seconds(str) parts = str.split(':').reverse len = parts.length if ! parts.length || parts.length > 3 raise("bad input (#{str}) for hours_minutes_seconds") and return end (0..2).map do |i| part = parts[len -i] part || "00" end.join(':') end #Remember how unix time starts at 1 Jan 1970? time = DateTime.iso8601( "1970-01-01T#{hours_minutes_seconds(video_length)}Z" ) # Lets print it out! puts time.utc.strftime("%H:%M:%S") # 00:04:30 You could then store the value as the number of seconds [integer]. Getting a time object back from the length in seconds is easy. puts Time.at(seconds).utc.strftime("%H:%M:%S") # 00:04:30
How to convert a long to date?
I have a long value in Rails, 134740800, which is the number of milliseconds since the epoch. How do I convert that to a date in mm-dd-yyyy format? I figure the formatting would be done with something like strftime but I can't seem to find the right method to convert the long into a valid date.
secs = 134740800/1000 # millisecs / 1000 t = Time.at(secs) t.strftime("%m-%d-%Y") Output "01-02-1970"
Try this: require 'date' DateTime.strptime("1318996912",'%s')
I assume you mean seconds since the epoch. Time.at seconds_since_epoch You can also pass a float. If you have milliseconds, divide by 1000.0 first. You can then call strftime on the returned Time object.
Use Time.at: irb(main):003:0> Time.at(134740800) => Tue Apr 09 08:00:00 -0400 1974
This is an advisory... It's often a good idea to look at how fast some answers run. Here's a simple benchmark: require 'benchmark' require 'date' SECS = 134740800 LOOPS = 1_000_000 puts Time.at(SECS).strftime('%m-%d-%Y') puts Date.strptime(SECS.to_s, '%s').strftime('%m-%d-%Y') Benchmark.bm(14) do |x| x.report('Time.at:') { LOOPS.times { Time.at(SECS) }} x.report('Date.strptime:') { LOOPS.times { Date.strptime(SECS.to_s, '%s') }} end And the output is: 04-09-1974 04-09-1974 user system total real Time.at: 0.370000 0.020000 0.390000 ( 0.392761) Date.strptime: 6.320000 0.050000 6.370000 ( 6.373248)
Lua ISO 8601 datetime parsing pattern
I'm trying to parse a full ISO8601 datetime from JSON data in Lua. I'm having trouble with the match pattern. So far, this is what I have: -- Example datetime string 2011-10-25T00:29:55.503-04:00 local datetime = "2011-10-25T00:29:55.503-04:00" local pattern = "(%d+)%-(%d+)%-(%d+)T(%d+):(%d+):(%d+)%.(%d+)" local xyear, xmonth, xday, xhour, xminute, xseconds, xmillies, xoffset = datetime:match(pattern) local convertedTimestamp = os.time({year = xyear, month = xmonth, day = xday, hour = xhour, min = xminute, sec = xseconds}) I'm stuck at how to deal with the timezone on the pattern because there is no logical or that will handle the - or + or none. Although I know lua doesn't support the timezone in the os.time function, at least I would know how it needed to be adjusted. I've considered stripping off everything after the "." (milliseconds and timezone), but then i really wouldn't have a valid datetime. Milliseconds is not all that important and i wouldn't mind losing it, but the timezone changes things. Note: Somebody may have some much better code for doing this and I'm not married to it, I just need to get something useful out of the datetime string :)
The full ISO 8601 format can't be done with a single pattern match. There is too much variation. Some examples from the wikipedia page: There is a "compressed" format that doesn't separate numbers: YYYYMMDD vs YYYY-MM-DD The day can be omited: YYYY-MM-DD and YYYY-MM are both valid dates The ordinal date is also valid: YYYY-DDD, where DDD is the day of the year (1-365/6) When representing the time, the minutes and seconds can be ommited: hh:mm:ss, hh:mm and hh are all valid times Moreover, time also has a compressed version: hhmmss, hhmm And on top of that, time accepts fractions, using both the dot or the comma to denote fractions of the lower time element in the time section. 14:30,5, 1430,5, 14:30.5, or 1430.5 all represent 14 hours, 30 seconds and a half. Finally, the timezone section is optional. When present, it can be either the letter Z, ±hh:mm, ±hh or ±hhmm. So, there are lots of possible exceptions to take into account, if you are going to parse according to the full spec. In that case, your initial code might look like this: function parseDateTime(str) local Y,M,D = parseDate(str) local h,m,s = parseTime(str) local oh,om = parseOffset(str) return os.time({year=Y, month=M, day=D, hour=(h+oh), min=(m+om), sec=s}) end And then you would have to create parseDate, parseTime and parseOffset. The later should return the time offsets from UTC, while the first two would have to take into account things like compressed formats, time fractions, comma or dot separators, and the like. parseDate will likely use the "^" character at the beginning of its pattern matches, since the date has to be at the beginning of the string. parseTime's patterns will likely start with "T". And parseOffset's will end with "$", since the time offsets, when they exist, are at the end. A "full ISO" parseOffset function might look similar to this: function parseOffset(str) if str:sub(-1)=="Z" then return 0,0 end -- ends with Z, Zulu time -- matches ±hh:mm, ±hhmm or ±hh; else returns nils local sign, oh, om = str:match("([-+])(%d%d):?(%d?%d?)$") sign, oh, om = sign or "+", oh or "00", om or "00" return tonumber(sign .. oh), tonumber(sign .. om) end By the way, I'm assuming that your computer is working in UTC time. If that's not the case, you will have to include an additional offset on your hours/minutes to account for that. function parseDateTime(str) local Y,M,D = parseDate(str) local h,m,s = parseTime(str) local oh,om = parseOffset(str) local loh,lom = getLocalUTCOffset() return os.time({year=Y, month=M, day=D, hour=(h+oh-loh), min=(m+om-lom), sec=s}) end To get your local offset you might want to look at http://lua-users.org/wiki/TimeZone . I hope this helps. Regards!
There is also the luadate package, which supports iso8601. (You probably want the patched version)
Here is a simple parseDate function for ISO dates. Note that I'm using "now" as a fallback. This may or may not work for you. YMMV 😉. --[[ Parse date given in any of supported forms. Note! For unrecognised format will return now. #param str ISO date. Formats: Y-m-d Y-m -- this will assume January Y -- this will assume 1st January ]] function parseDate(str) local y, m, d = str:match("(%d%d%d%d)-?(%d?%d?)-?(%d?%d?)$") -- fallback to now if y == nil then return os.time() end -- defaults if m == '' then m = 1 end if d == '' then d = 1 end -- create time return os.time{year=y, month=m, day=d, hour=0} end --[[ --Tests: print( os.date( "%Y-%m-%d", parseDate("2019-12-28") ) ) print( os.date( "%Y-%m-%d", parseDate("2019-12") ) ) print( os.date( "%Y-%m-%d", parseDate("2019") ) ) ]]