How to properly parse timezone codes - parsing

In the example bellow the result is always "[date] 05:00:00 +0000 UTC" regardless the timezone you choose for the parseAndPrint function. What is wrong with this code? The time should change depending on the timezone you choose. (Go Playground servers are apparently configured in UTC timezone).
http://play.golang.org/p/wP207BWYEd
package main
import (
"fmt"
"time"
)
func main() {
now := time.Now()
parseAndPrint(now, "BRT")
parseAndPrint(now, "EDT")
parseAndPrint(now, "UTC")
}
func parseAndPrint(now time.Time, timezone string) {
test, err := time.Parse("15:04:05 MST", fmt.Sprintf("05:00:00 %s", timezone))
if err != nil {
fmt.Println(err)
return
}
test = time.Date(
now.Year(),
now.Month(),
now.Day(),
test.Hour(),
test.Minute(),
test.Second(),
test.Nanosecond(),
test.Location(),
)
fmt.Println(test.UTC())
}

When you Parse a time, you are parsing it in your current location, which is OK as long as that's what you're expecting, and the timezone abbreviation is known from within your location.
If you can forgo timezones, it's far easier to normalize all the times you're dealing with into UTC.
The next easiest is handling everything with explicit offsets, like -05:00.
If you want to deal with times originating in other timezones, you need to use time.Location. You can load Locations from the local timezone db with time.LoadLocation, and parse times there with time.ParseInLocation.

Question: How to properly parse time with abbreviated timezone names like UTC, CET, BRT, etc.?
Answer: You better should not. As JimB and others in this question Why doesn't Go's time.Parse() parse the timezone identifier? carefully suggest, you can expect that Go correctly parses only two timezones: UTC and the local one.
What they don't make quite explicit is that you can't expect Go to correctly parse time with any other timezone. At least that is so in my personal experience (go1.16.1, Ubuntu 20.04).
Also, abbreviated timezones are ambiguous. IST could mean India Standard Time, Irish Standard Time or Israel Standard Time. There's no way to disambiguate unless you know zone location, and, if you know location, you should use time.ParseInLocation.
If this is user input and you have control, you should change format requirements for users to input time with explicit offsets as JimB is also suggesting in their answer. Make sure you don't forget about minutes, i.e. use -0700, -07:00, Z0700 or Z07:00 but not -07 or Z07 in layout. Not all offsets are whole hours. For instance, Inidia Standard Time is UTC+5:30.
If you have no other choice and forced to parse such times, you can do something like that:
func parseTimeWithTimezone(layout, value string) (time.Time, error) {
tt, err := time.Parse(layout, value)
if err != nil {
return time.Time{}, err
}
loc := tt.Location()
zone, offset := tt.Zone()
// Offset will be 0 if timezone is not recognized (or UTC, but that's ok).
// Read carefully https://pkg.go.dev/time#Parse
// In this case we'll try to load location from zone name.
// Timezones that are recognized: local, UTC, GMT, GMT-1, GMT-2, ..., GMT+1, GMT+2, ...
if offset == 0 {
// Make sure you have timezone database available in your system for
// time.LoadLocation to work. Read https://pkg.go.dev/time#LoadLocation
// about where Go looks for timezone database.
// Perhaps the simplest solution is to `import _ "time/tzdata"`, but
// note that it increases binary size by few hundred kilobytes.
// See https://golang.org/doc/go1.15#time/tzdata
loc, err = time.LoadLocation(zone)
if err != nil {
return time.Time{}, err // or `return tt, nil` if you more prefer
// the original Go semantics of returning time with named zone
// but zero offset when timezone is not recognized.
}
}
return time.ParseInLocation(layout, value, loc)
}
Note that zone names that aren't present as files in timezone database will fail parsing. These are quite many. You can see what is present by checking
contents of /usr/share/zoneinfo, /usr/share/lib/zoneinfo on your system,
contents of this file https://github.com/golang/go/blob/master/lib/time/zoneinfo.zip.

Related

Why is the gettimeofday timezone wrong?

Both ftime and gettimeofday are returning 0 for the current timezone in Ubuntu 16. The timezone is set correctly in the date and time settings provided by Ubuntu. There is no TZ env variable set.
I don't want to just "fix" it because this is production software used in many different contexts. So I just want a reliable way of programmatically getting the timezone (and preferably the current DST offset as well).
My attempts so far:
#if 0
timeb tbTime;
ftime(&tbTime);
int CurTz = -tbTime.timezone;
#else
struct timeval tv;
struct timezone tz;
int r = gettimeofday(&tv, &tz);
if (r)
return NO_ZONE;
int CurTz = tz.tz_minuteswest;
#endif
The 'date' command is working:
matthew#mallen-ubuntu:~$ date +%Z
AEDT
matthew#mallen-ubuntu:~$ date +%z
+1100
I could just spawn a process to call "date", but that seems very heavy handed when some API calls are available.
On GNU/Linux, the second argument of gettimeofday is quite useless and should always be NULL. The manual page for gettimeofday says this:
The use of the timezone structure is obsolete; the tz argument should normally be specified as NULL.
Even on non-Linux systems, the tz_dsttime has useless semantics (e.g., it reports that India uses DST because it did so for a brief period about seventy years ago).
If you need to obtain the time zone for the current time, you need to use localtime or localtime_r and examine the broken-down time structure it produces (and not global variables such as daylight). The struct tm members you are probably interested are tm_isdst, tm_gmtoff, and perhaps tm_zone. The latter two are glibc extensions.

tzfile format : handling of istd and isgmt

I'm trying to parse the tzfile (Olson) format on a Unix system. In the tzfile(5) man page it states the following:
Then there are tzh_ttisstdcnt standard/wall indicators, each stored
as a one-byte value; they tell whether the transition times
associated with local time types were specified as standard time or
wall clock time, and are used when a timezone file is used in
handling POSIX-style timezone environment variables.
Finally, there are tzh_ttisgmtcnt UTC/local indicators, each stored
as a one-byte value; they tell whether the transition times
associated with local time types were specified as UTC or local time,
and are used when a timezone file is used in handling POSIX-style
timezone environment variables.
Does this mean I can ignore isstd and isgmt and still get the correct times? In spot checking, this seems to be the case but in digging around in the C source files, I see unix makes some adjustments dependant on these values.
As requested above, I asked on the mailing list. The answer was to look in the code. So the relevant code is in zic.c (zone compiler) and glib's tzfile.c. Source code for both can be found on github at the time of this wring. The relevant code for zic.c is
switch (lowerit(*ep)) {
case 's': /* Standard */
rp->r_todisstd = true;
rp->r_todisgmt = false;
*ep = '\0';
break;
case 'w': /* Wall */
rp->r_todisstd = false;
rp->r_todisgmt = false;
*ep = '\0';
break;
case 'g': /* Greenwich */
case 'u': /* Universal */
case 'z': /* Zulu */
rp->r_todisstd = true;
rp->r_todisgmt = true;
*ep = '\0';
break;
Which says there are 3 possible cases: isstd = true && isgmt = false, both false and both true. So to see what is done with these flags, the relevant code in tzfile.c is
if (trans_type->isgmt)
/* The transition time is in GMT. No correction to apply. */ ;
else if (isdst && !trans_type->isstd)
/* The type says this transition is in "local wall clock time", and
wall clock time as of the previous transition was DST. Correct
for the difference between the rule's DST offset and the user's
DST offset. */
transitions[i] += dstoff - rule_dstoff;
else
/* This transition is in "local wall clock time", and wall clock
time as of this iteration is non-DST. Correct for the
difference between the rule's standard offset and the user's
standard offset. */
transitions[i] += stdoff - rule_stdoff;
So this seems to say that if isgmt is true we can ignore everything else. If it is false then if the previous transition was DST and the current one is not standard (i.e. case 'w' above, as it's also not gmt) apply the dst offset (the last one found in the file). Otherwise apply the standard offset.
Which seems to mean that glibc ignores the offsets in the individual tt_types and case 's'. I looked in localtime.c in the tz package and it worked the same way.
I can only conclude from all this that most of the information in the tzfile isn't actually used anywhere. Some of this may be due to POSIX requirements. If anyone can expand on the details below, please do. It would be nice to have this behaviour documented somewhere on the internet besides in C source code.

How to parse Date in IST format in Golang?

time.Date(t.Year(), t.Month(), time.Now().Day(), 10, 0, 0, 0, time.UTC)
I want to set dateTime of 10:00:00 AM in IST format in golang.
It depends on the format of the time you have at hand. Go has some standard time formats ready as consts in the time package, but you can specify your own standard if it's custom. Regarding the time zone, you can parse or output a time in a specific time zone. Here is an example of parsing a time string in IST, and outputting it as UTC. It's not clear from your question what is your precise problem but I hope this helps:
// First, we create an instance of a timezone location object
loc, _ := time.LoadLocation("Asia/Kolkata")
// this is our custom format. Note that the format must point to this exact time
format := "Jan _2 2006 3:04:05 PM"
// this is your timestamp
timestamp := "Jun 25 2015 10:00:00 AM"
// now we parse it, considering it's in IST
t, err := time.ParseInLocation(format, timestamp, loc)
// printing it prints it in IST, but you can set the timezone to UTC if you want
fmt.Println(t, err)
// example - getting the UTC timestamp
fmt.Println(t.UTC())

Jodatime get milliseconds with offset

New to JodaTime library, i would like to get a DateTime's milliseconds field with the specified TimeZone's offset.
So far my attempt is:
private DateTimeZone timeZone = DateTimeZone.forID("Europe/Amsterdam");
private long now=new DateTime().withZone(timeZone).getMillis();
But i always get the UTC millis, the timezone offset isnt applied,
Is there any way to apply the timezone's offset to the DateTime object?
Thx!
First: What do you intend to do with these "local" millis? What do you really try to achieve? Normally only UTC-millis are needed.
Anyway, remember the general timezone offset definition which is:
UTC + Offset = Local Time
Then the solution is simple:
DateTimeZone tz = DateTimeZone.forID("Europe/Amsterdam");
long nowUTC = new DateTime().withZone(tz).getMillis();
long nowLocal = nowUTC + tz.getOffset(nowUTC);
But once again: What is your use-case for "local" millis? They are not even related to UNIX epoch any longer because the UTC-link is cut off.
About your last question ("Is there any way to apply the timezone's offset to the DateTime object?"):
Your DateTime-object already has got a timezone, namely "Europe/Amsterdam". It is internally used to compute the field tuple representation once you have a global UTC-timestamp expressed as millis since UNIX epoch. No need to apply an extra offset on DateTime. It is already there.
JodaTime is using machine time inside. So to find miliseconds, you can use a constant storing LocalDateTime referring to Jan 1, 1970(Because of UNIX Time).
Unix time, or POSIX time, is a system for describing points in time,
defined as the number of seconds elapsed since midnight proleptic
Coordinated Universal Time (UTC) of January 1, 1970, not counting leap
seconds.
Then calculate the difference between your DateTime.
I tried like this;
public static void main(String[] args) {
final LocalDateTime JAN_1_1970 = new LocalDateTime(1970, 1, 1, 0, 0);
DateTime local = new DateTime().withZone(DateTimeZone.forID("Europe/Amsterdam"));
DateTime utc = new DateTime(DateTimeZone.UTC);
System.out.println("Europe/Amsterdam milis :" + new Duration(JAN_1_1970.toDateTime(DateTimeZone.forID("Europe/Amsterdam")), local).getMillis());
System.out.println("UTC milis :" + new Duration(JAN_1_1970.toDateTime(DateTimeZone.UTC), utc).getMillis());
}
And the result is;
Europe/Amsterdam milis :1429695646528
UTC milis :1429692046534
And #leonbloy write here a good comment.
Your local and utc represent the same instants of time, (only with
different timezones attached). Hence, getMillis() (which gives the
"physical" time interval elapsed from the "instant" corresponding to
the unix epoch), must return the same value.
I will also look for better solution with no constant.

Lua ISO 8601 datetime parsing pattern

I'm trying to parse a full ISO8601 datetime from JSON data in Lua.
I'm having trouble with the match pattern.
So far, this is what I have:
-- Example datetime string 2011-10-25T00:29:55.503-04:00
local datetime = "2011-10-25T00:29:55.503-04:00"
local pattern = "(%d+)%-(%d+)%-(%d+)T(%d+):(%d+):(%d+)%.(%d+)"
local xyear, xmonth, xday, xhour, xminute,
xseconds, xmillies, xoffset = datetime:match(pattern)
local convertedTimestamp = os.time({year = xyear, month = xmonth,
day = xday, hour = xhour, min = xminute, sec = xseconds})
I'm stuck at how to deal with the timezone on the pattern because there is no logical or that will handle the - or + or none.
Although I know lua doesn't support the timezone in the os.time function, at least I would know how it needed to be adjusted.
I've considered stripping off everything after the "." (milliseconds and timezone), but then i really wouldn't have a valid datetime. Milliseconds is not all that important and i wouldn't mind losing it, but the timezone changes things.
Note: Somebody may have some much better code for doing this and I'm not married to it, I just need to get something useful out of the datetime string :)
The full ISO 8601 format can't be done with a single pattern match. There is too much variation.
Some examples from the wikipedia page:
There is a "compressed" format that doesn't separate numbers: YYYYMMDD vs YYYY-MM-DD
The day can be omited: YYYY-MM-DD and YYYY-MM are both valid dates
The ordinal date is also valid: YYYY-DDD, where DDD is the day of the year (1-365/6)
When representing the time, the minutes and seconds can be ommited: hh:mm:ss, hh:mm and hh are all valid times
Moreover, time also has a compressed version: hhmmss, hhmm
And on top of that, time accepts fractions, using both the dot or the comma to denote fractions of the lower time element in the time section. 14:30,5, 1430,5, 14:30.5, or 1430.5 all represent 14 hours, 30 seconds and a half.
Finally, the timezone section is optional. When present, it can be either the letter Z, ±hh:mm, ±hh or ±hhmm.
So, there are lots of possible exceptions to take into account, if you are going to parse according to the full spec. In that case, your initial code might look like this:
function parseDateTime(str)
local Y,M,D = parseDate(str)
local h,m,s = parseTime(str)
local oh,om = parseOffset(str)
return os.time({year=Y, month=M, day=D, hour=(h+oh), min=(m+om), sec=s})
end
And then you would have to create parseDate, parseTime and parseOffset. The later should return the time offsets from UTC, while the first two would have to take into account things like compressed formats, time fractions, comma or dot separators, and the like.
parseDate will likely use the "^" character at the beginning of its pattern matches, since the date has to be at the beginning of the string. parseTime's patterns will likely start with "T". And parseOffset's will end with "$", since the time offsets, when they exist, are at the end.
A "full ISO" parseOffset function might look similar to this:
function parseOffset(str)
if str:sub(-1)=="Z" then return 0,0 end -- ends with Z, Zulu time
-- matches ±hh:mm, ±hhmm or ±hh; else returns nils
local sign, oh, om = str:match("([-+])(%d%d):?(%d?%d?)$")
sign, oh, om = sign or "+", oh or "00", om or "00"
return tonumber(sign .. oh), tonumber(sign .. om)
end
By the way, I'm assuming that your computer is working in UTC time. If that's not the case, you will have to include an additional offset on your hours/minutes to account for that.
function parseDateTime(str)
local Y,M,D = parseDate(str)
local h,m,s = parseTime(str)
local oh,om = parseOffset(str)
local loh,lom = getLocalUTCOffset()
return os.time({year=Y, month=M, day=D, hour=(h+oh-loh), min=(m+om-lom), sec=s})
end
To get your local offset you might want to look at http://lua-users.org/wiki/TimeZone .
I hope this helps. Regards!
There is also the luadate package, which supports iso8601. (You probably want the patched version)
Here is a simple parseDate function for ISO dates. Note that I'm using "now" as a fallback. This may or may not work for you. YMMV 😉.
--[[
Parse date given in any of supported forms.
Note! For unrecognised format will return now.
#param str ISO date. Formats:
Y-m-d
Y-m -- this will assume January
Y -- this will assume 1st January
]]
function parseDate(str)
local y, m, d = str:match("(%d%d%d%d)-?(%d?%d?)-?(%d?%d?)$")
-- fallback to now
if y == nil then
return os.time()
end
-- defaults
if m == '' then
m = 1
end
if d == '' then
d = 1
end
-- create time
return os.time{year=y, month=m, day=d, hour=0}
end
--[[
--Tests:
print( os.date( "%Y-%m-%d", parseDate("2019-12-28") ) )
print( os.date( "%Y-%m-%d", parseDate("2019-12") ) )
print( os.date( "%Y-%m-%d", parseDate("2019") ) )
]]

Resources