One can use bjobs to get the start time of an LSF job as such:
bjobs -noheader -o start_time $JOB
However, this returns low-fidelity (e.g., seconds are not necessarily included) human readable output. Higher (but not necessarily "full", I believe) fidelity output can be parsed out of:
bjobs -l $JOB
...but that's rather messy. Also, as I alluded to, I think the output can still be ambiguous; the year doesn't always seem to be included here and I don't recall seeing any time zone information.
How can I get an LSF job's start time deterministically and unambiguously (say, as a Unix epoch)?
IMHO the seconds are always included in start_time and the timestamp is exactly the same as the one reported by bjobs -l $JOBID.
To show the year in the date, you have to set LSB_DISPLAY_YEAR=Y in lsf.conf. This is not set by default in LSF. Don't forget to run lsadmin reconfig;badmin mbdrestart after having modified lsf.conf.
You can convert dates with date:
date --date "`bjobs -noheader -o start_time $JOBID`" +"%s"
or
date --date "$(bjobs -noheader -o start_time $JOBD)" +"%s"
if you prefer.
Related
In Ruby or Rails, given a timezone identifier like America/Los_Angeles, I see how I can get the specific timezone offset abbreviation like PDT:
Time.current.in_time_zone('America/Los_Angeles').zone
=> "PDT"
Time.now.in_time_zone("America/Los_Angeles").strftime('%Z')
=> "PDT"
Time.now.in_time_zone("America/Los_Angeles").strftime('%z')
=> "-0700"
And even a friendly name for the timezone itself:
ActiveSupport::TimeZone::MAPPING.key('America/Los_Angeles')
=> "Pacific Time (US & Canada)"
But how can I get a string like Pacific Daylight Time instead of just PDT to represent the full timezone offset name for a specific date?
Input: America/Los_Angeles and a specific date or datetime
Desired Output: Pacific Daylight Time
Short story: ~for some reason,~ (as #bata mentioned in his answer the abbreviations are not unique so this might be a reason for no official abbreviation-human friendly name mapping) The standard seems not to support it. You might need to implement this mapping yourself. But read on if you want to get some pointers why it might not be that easy.
So... this was an interesting rabbit hole to fall into...
Which gems are used and where do they get the info from
ActiveSupport uses tzinfo gem for that which in turn uses tz-info gem. tz-info says it uses one of two data sources:
zoneinfo directory in the Unix like system
TZInfo::Data
Rails' Gemfile template confirms that TZInfo data is used on window
At first I thought there might be a PR opportunity to contribute to a popular gem. Quick search suggests that there's no mapping of PDT to any "human friendly form".
Looking at unix's zoneinfo database
$ cat /usr/share/zoneinfo/America/Los_Angeles
TZif2H*ˉ#pa&t\Ðݩ߉iip~KIR^-)4GJQ',3qo_O?/v( fe HG *)
x
q(a'Q
A 0C ΐ
#j$5 %J& '*'))6 "S 54+ !"V
eGYe턠g';gfiiHjke lmvG noV) pq6: XX YZ [\ ]^d _y`Mޠab-cgwdE G-Gӵ I
ros tOt v8vxx͠yz{f|~}H~^s*&
LMTPDTPSTPWTPPTTZif2^H*ˉ#pa&t\Ðݩ߉iip~KIR^-)4GJQ',3qo_O?/v( fe HG *)
x
q(a'Q
A 0C ΐ
#j$5 %J& '*'))6 "S 54+ !"V
eGYe턠g';gfiiHjke lmvG noV) pq6: XX YZ [\ ]^d _y`Mޠab-cgwdE G-Gӵ I
ros tOt v8vxx͠yz{f|~}H~^s*&
LMTPDTPSTPWTPPT
PST8PDT,M3.2.0,M11.1.0
Looks like a binary file...
$ strings /usr/share/zoneinfo/America/Los_Angeles
TZif2
v+ !
2s$
3Gt 4S
5'V 62
mvG n
oV) p
TZif2
v+
3Gt
5'V
mvG
oV)
PST8PDT,M3.2.0,M11.1.0
Hmm. Whatever each of those means, there seems to be no "human friendly" name for this TZ.
More digging for unix tzinfo format, and there's apparently a zic and zdump tool
$ zdump /usr/share/zoneinfo/America/Los_Angeles
/usr/share/zoneinfo/America/Los_Angeles Thu Sep 10 03:07:49 2020 PDT
Not very useful...
$ zic /usr/share/zoneinfo/America/Los_Angeles
"/usr/share/zoneinfo/America/Los_Angeles", line 1: line too long
Ugh.
But reading the manpages for one of those tools I've found out that timezoneinfo is an official RFC 8536. I'm not great at reading those, but in the format there's header (with version only) and
data block (check it out, apparently it stores all historical changes for the TZ)
footer which seems to contain the PDT strings and alike, and nowhere is mentioned a possibility to provide "friendly" name.
If you have hard times reading an RFC, it looks like the tzinfo-data gem tries to reproduce the same data for system that does not include tzinfo databases (i.e. Windows):
https://github.com/tzinfo/tzinfo-data/blob/4ab39f022f5537b97eed133c1169f9ace3a82e2b/lib/tzinfo/data/definitions/PST8PDT.rb
timezone 'America/Los_Angeles' do |tz|
tz.offset :o0, -28378, 0, :LMT
tz.offset :o1, -28800, 0, :PST
tz.offset :o2, -28800, 3600, :PDT
tz.offset :o3, -28800, 3600, :PWT
tz.offset :o4, -28800, 3600, :PPT
tz.transition 1883, 11, :o1, -2717640000, 7227400, 3
tz.transition 1918, 3, :o2, -1633269600, 29060207, 12
tz.transition 1918, 10, :o1, -1615129200, 19375151, 8
tz.transition 1919, 3, :o2, -1601820000, 29064575, 12
tz.transition 1919, 10, :o1, -1583679600, 19378063, 8
tz.transition 1942, 2, :o3, -880207200, 29164799, 12
# ... [cut!]
Here's offset method definition: https://github.com/tzinfo/tzinfo/blob/f361d7d0b859ba5b91d30ffd6b66c3c59f90e969/lib/tzinfo/format2/timezone_definer.rb#L37:L59 (at least I think this is it, you never know until you run the code in ruby)
As you can see, no daylight saving time has :LMT and :PST abbreviations.
And here is the transition method https://github.com/tzinfo/tzinfo/blob/8c549373736a7873e18bb787b818d8786591e5e5/lib/tzinfo/format1/timezone_definer.rb#L30:L61
Feel free to explore more on your own. But it looks like each and every one transition from DST and back is supposed to be present in the tzinfo. And LA seems to have stuff planned until 2060. That's actually interesting if you'll get TZ bugs related to DST if you try to calculate times after 2060?
Last thing: Abbreviations
No idea if this is the case for unix's tzinfo db, but in the gem they are defined locally!
It seems to be the case that :PDT means the same if all definitions (https://github.com/tzinfo/tzinfo-data/search?q=%3APDT&unscoped_q=%3APDT)
But notice that
lib/tzinfo/data/definitions/Asia/Manila.rb defines positive offset tz.offset :o3, 28800, 3600, :PDT, but
lib/tzinfo/data/definitions/America/Dawson.rb defines negative offset tz.offset :o7, -28800, 3600, :PDT with the same abbreviation.
What does that mean - no idea. But that might be the reason we don't have "global friendly timezone names" anywhere.
There might be errors in the database. The official DB README says:
This database of historical local time information has several goals:
Provide a compendium of data about the history of civil time that
is useful even if not 100% accurate.
Give an idea of the variety of local time rules that have existed
in the past and thus may be expected in the future.
Test the generality of the local time rule description system.
The information in the time zone data files is by no means authoritative;
fixes and enhancements are welcome. Please see the file CONTRIBUTING
for details.
I don’t think there is a standard for abbreviations. Also, they are not unique as you see here: https://www.timeanddate.com/time/zones/
I've a new_git_repository containing:
new_git_repository(
name = "hyperscan",
build_file = "//external-deps/hyperscan:BUILD",
commit = "[COMMIT_HASH]",
remote = "https://github.com/intel/hyperscan.git",
shallow_since = "2018-07-09",
)
When building it says:
DEBUG: Rule 'hyperscan' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1531154744 -0400"
According to this, shouldn't the shallow_since format be of YYYY-MM-DD?
And next, what does shallow_since = "1531154744 -0400" mean?!
Bazel does not process the string specified as shallow_since attribute and passes it directly to git as --shallow-since parameter. It can be seen in Bazel source code here.
The value you see is Git internal date format which is <unix timestamp> <time zone offset>, where <unix timestamp> is the number of seconds since the UNIX epoch. <time zone offset> is a positive or negative offset from UTC. For example CET (which is 1 hour ahead of UTC) is +0100.
Here is the tool for unix timestamp conversion to the human-readable date/time and back.
Bazel uses git log --date=raw to get the timestamp of the commit, and then does a string comparison with the value of shallow_since. In my opinion, it is a bug in Bazel - it should do a date comparison instead.
As specified in the comments, you can use git log --date=raw to get the commit sha and the time (shallow_since) of the desired commit.
In the Hadoop infrastructure (Java-based) I am getting timestamps as string values in this format:
2015-10-01T04:22:38:208Z
2015-10-01T04:23:35:471Z
2015-10-01T04:24:33:422Z
I tried different patters following examples for SimpleDateFormat Java class without any success.
Replaced 'T' with ' ' and 'Z' with '', then
"yyyy-MM-dd HH:mm:ss:ZZZ"
"yyyy-MM-dd HH:mm:ss:zzz"
"yyyy-MM-dd HH:mm:ss:Z"
"yyyy-MM-dd HH:mm:ss:z"
Without replacement,
"yyyy-MM-dd'T'HH:mm:ss:zzz'Z'"
In fact, this format is not listed among examples. What should I do with it?
Maybe those 3 digits are milliseconds, and time is in UTC, like this: "yyyy-MM-dd'T'HH:mm:ss.SSSZ"? But it still should look like "2015-11-27T10:50:44.000-08:00" as standardized format ISO-8601.
Maybe, this format is not parsed correctly in the first place?
I use Ruby, Python, Pig, Hive to work with it (but not Java directly), so any example helps. Thanks!
I very strongly suspect the final three digits are nothing to do with time zones, but are instead milliseconds, and yes, the Z means UTC. It's a little odd that they're using : instead of . as the separator between seconds and milliseconds, but that can happen sometimes.
In that case you want
"yyyy-MM-dd'T'HH:mm:ss:SSSX"
... or use
"yyyy-MM-dd'T'HH:mm:ss:SSS'Z'"
and set your SimpleDateFormat's time zone to UTC explicitly.
I would like to return the local time as string but with leading zeros. I tried this:
{{Year, Month, Day}, {Hour, Minute, Second}} = erlang:localtime().
DateAsString = io_lib:format("~2.10.0B~2.10.0B~4.10.0B~2.10.0B~2.10.0B~2.10.0B",
[Month, Day, Year, Hour, Minute, Second]).
But if some of the components is one digit, the returned string is:
[["0",57],"29","2011","17","33","34"]
The current month 9 is printed as ["0",57].
Please, help.
Thank you.
Try:
1> lists:flatten([["0",57],"29","2011","17","33","34"]).
"09292011173334"
io_lib:format/2 (and it's companion io:format/2) actually returns a deep IO list. Such a list is printable and can be sent on a socket or written to a file just as a flat string, but is more efficient to produce. Flattening is often useless, because in all cases where the string will be printed or output to a file/socket it will automatically be flattened by Erlang.
You want to be using something like this:
DateAsString = io_lib:format("~2..0w~2..0w~4..0w~2..0w~2..0w~2..0w",
[Month, Day, Year, Hour, Minute, Second]).
The more common w format modifier does the right thing here, what with base and such, so there's no need to use the more complex B modifier. 2..0 says "2 characters wide, zero padded, no precision specified." We don't need precision here, since we're dealing with integers.
With a "normal" (i mean "full") linux distro, it works just fine:
sleep $(echo "$[ ($RANDOM % 9 ) ]")
ok, it waits for about 0-9 sec
but under OpenWRT [not using bash, rather "ash"]:
$ sleep $(echo "$[ ($RANDOM % 9 ) ]") sleep: invalid number '$[' $
and why:
$ echo "$[ ($RANDOM % 9 ) ]" $[ ( % 9 ) ] $
So does anyone has a way to generate random numbers under OpenWRT, so i can put it in the "sleep"?
Thank you
You might try something like this:
sleep `head /dev/urandom | tr -dc "0123456789" | head -c1`
Which works on my WhiteRussian OpenWRT router.
I actually don't know if this will always return a number, but when it does, it will always return 0-9, and only 1 digit (you could make it go up to 99 if you made the second head -c2).
Good luck!
you could also use awk
sleep $(awk 'BEGIN{srand();print int(rand()*9)}')
For some scenarios, this might not yield a sufficient diversity of answers. Another approach is to use /dev/urandom directly (eg https://www.2uo.de/myths-about-urandom/):
echo $(hexdump -n 4 -e '"%u"' </dev/urandom)
When using awk, note that awk uses the time of day as the seed (https://linux.die.net/man/1/awk). This might be relevant for scenarios where the time of day is reset (eg no battery backed time of day clock), or synchronised across a fleet (eg group restart).
srand([expr])
Uses expr as a new seed for the random number generator. If no expr is provided, the time of day is used. The return value is the previous seed for the random number generator.
This is confirmed by looking at the source in busybox (https://github.com/mirror/busybox/blob/master/editors/awk.c):
seed = op1 ? (unsigned)L_d : (unsigned)time(NULL);
At least for some versions of Openwrt, it seems an explicit call to srand() is required to avoid obtaining the same answers repeatedly:
# awk 'BEGIN{print rand(), rand()}'
0 0.345001
# awk 'BEGIN{print rand(), rand()}'
0 0.345001