**I have an input like as follows with many, many rows, and I need to parse all this file into a better format, could be a CSV file or JSON (maybe in the future).
So I need to produce an outcome with columns delimited by a comma, thinking of being able to export the content in CSV file for now.
Get the name files
awk '{ if($2 ~ /A/ ) print $1 }' dir_out
Get all the paths
awt ' /[\\]/ {print}'
Get the size of the files
awk '{ if($3 ~ /^[0-9]/) print $3}'
Right now I have the individual commands to generate the desired result, however I have to find a way to put them in the same line of awk commands, or in a script.
One of the critical points that I have not been able to solve is to make column 1 of the outcome the path that delimits each block, for all the files in the block.
So I starting from this input:
**
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl
R0097A+05.00B-00-QingL.JPG A 6958377 Fri Jun 8 12:53:30 2018
R0097A+05.00B-00-QingLI.JPG A 2794933 Fri Jun 8 12:53:30 2018
R0097A+05.00B-00-QingLO.JPG A 1350397 Fri Jun 8 12:53:30 2018
R0097A+11.00B-00-QingL.JPG A 6997803 Fri Jun 8 12:53:30 2018
R0097A+11.00B-00-QingLI.JPG A 2783151 Fri Jun 8 12:53:30 2018
R0097A+11.00B-00-QingLO.JPG A 1338662 Fri Jun 8 12:53:30 2018
R0097A-00.00B-00-QingL.JPG A 7069740 Fri Jun 8 12:53:30 2018
R0097A-00.00B-00-QingLI.JPG A 2825705 Fri Jun 8 12:53:30 2018
R0097A-00.00B-00-QingLO.JPG A 1369520 Fri Jun 8 12:53:30 2018
Jhumbs.db A 20480 Fri Jun 8 13:14:41 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl
R0098A+05.00B-00-QingL.JPG A 6958377 Fri Jun 8 12:54:30 2018
R0098A+05.00B-00-QingLI.JPG A 2794933 Fri Jun 8 12:54:30 2018
R0098A+05.00B-00-QingLO.JPG A 1350398 Fri Jun 8 12:54:30 2018
R0098A+11.00B-00-QingL.JPG A 6998803 Fri Jun 8 12:54:30 2018
R0098A+11.00B-00-QingLI.JPG A 2783151 Fri Jun 8 12:54:30 2018
R0098A+11.00B-00-QingLO.JPG A 1338662 Fri Jun 8 12:54:30 2018
R0098A-00.00B-00-QingL.JPG A 7069840 Fri Jun 8 12:54:30 2018
R0098A-00.00B-00-QingLI.JPG A 2825705 Fri Jun 8 12:54:30 2018
R0098A-00.00B-00-QingLO.JPG A 1369520 Fri Jun 8 12:54:30 2018
Jhumbs.db A 20480 Fri Jun 8 13:14:41 2018`
ljkhlj
PATH, FILENAME, SIZE, TIMESTAMP
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+05.00B-00-QingL.JPG, 6958377, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+05.00B-00-QingLI.JPG, 2794933, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+05.00B-00-QingLI.JPG, 1350397, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+11.00B-00-QingL.JPG, 6997803, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+05.00B-00-QingL.JPG, 6958377, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+05.00B-00-QingLI.JPG, 6958377, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+05.00B-00-QingLO.JPG, 6958377, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+11.00B-00-QingL.JPG, 6958377, Fri Jun 8 12:54:30 2018
Here's a way of combining your awk command into a single script:
#!/bin/bash
awk '
$2 ~ /A/ {print $1; }
/[\\]/ {print}
$3 ~ /^[0-9]/ {print $3}
' "$#"
In general, awk takes multiple /search/ {command} pairs. If /search/ is missing, it defaults to all lines and if {command} is missing, it defaults to print.
Here's the additional logic you need to get your expected results:
#!/bin/bash
awk -v OFS=, '
BEGIN { print "PATH, FILENAME, SIZE, TIMESTAMP" }
/[\\]/ { path=$0 }
$2 ~ /A/ {print path,$1,$3,$4 " " $5 " " $6 " " $7 }
' "$#"
$ cat tst.awk
BEGIN {
OFS = ", "
print "PATH", "FILENAME", "SIZE", "TIMESTAMP"
}
/^ / {
file = $1
size = $3
sub(/^ ([^[:space:]]+[[:space:]]+){3}/,"")
print path, file, size, $0
next
}
{ path = $0 }
$ awk -f tst.awk file
PATH, FILENAME, SIZE, TIMESTAMP
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+05.00B-00-QingL.JPG, 6958377, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+05.00B-00-QingLI.JPG, 2794933, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+05.00B-00-QingLO.JPG, 1350397, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+11.00B-00-QingL.JPG, 6997803, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+11.00B-00-QingLI.JPG, 2783151, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A+11.00B-00-QingLO.JPG, 1338662, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A-00.00B-00-QingL.JPG, 7069740, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A-00.00B-00-QingLI.JPG, 2825705, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, R0097A-00.00B-00-QingLO.JPG, 1369520, Fri Jun 8 12:53:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0097\Qingl, Jhumbs.db, 20480, Fri Jun 8 13:14:41 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+05.00B-00-QingL.JPG, 6958377, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+05.00B-00-QingLI.JPG, 2794933, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+05.00B-00-QingLO.JPG, 1350398, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+11.00B-00-QingL.JPG, 6998803, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+11.00B-00-QingLI.JPG, 2783151, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A+11.00B-00-QingLO.JPG, 1338662, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A-00.00B-00-QingL.JPG, 7069840, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A-00.00B-00-QingLI.JPG, 2825705, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, R0098A-00.00B-00-QingLO.JPG, 1369520, Fri Jun 8 12:54:30 2018
\QJ DaJabase EltraJo\DR0151-populated\DaJa\ASAA Images\k0098\Qingl, Jhumbs.db, 20480, Fri Jun 8 13:14:41 2018
Related
I am working on parsing a file as follows. I see a duplicate line for "ver."=3 and 5 because there is only 1 second difference (Mon Jan 15 08:24:02 vs Mon Jan 15 08:24:03) software prints it twice.
Also, of you see line for "Ver."5 "Complete Time" has one second difference. I would like delete lines which has rest of the fields matching in the line except "Time" or "Complete Time" column has difference of 1 or 2 seconds.
Loc ID Img Name Ver. Time Complete Time
------------------------------------------------------------------------------------------
ssfad_fs TINT_PAP_1516048511 0 Mon Jan 15 20:35:13 2018 NA
ssfad_fs sfad_jpg 1 Mon Jan 15 18:24:02 2018 Wed Jan 17 18:24:02 2018
ssfad_fs sfad_jpg 1 Mon Jan 15 16:24:02 2018 Wed Jan 17 16:24:02 2018
ssfad_fs sfad_jpg 2 Mon Jan 15 12:24:03 2018 Wed Jan 17 12:24:02 2018
ssfad_fs sfad_jpg 3 Mon Jan 15 08:24:02 2018 Wed Jan 17 08:24:02 2018
ssfad_fs sfad_jpg 3 Mon Jan 15 08:24:03 2018 Wed Jan 17 08:24:02 2018
ssfad_fs sfad_jpg 4 Mon Jan 15 04:24:02 2018 Wed Jan 17 04:24:02 2018
ssfad_fs sfad_jpg 5 Mon Jan 15 00:24:03 2018 Wed Jan 17 00:24:59 2018
ssfad_fs sfad_jpg 5 Mon Jan 15 00:24:03 2018 Wed Jan 17 00:25:00 2018
ssfad_fs sfad_jpg 6 Sun Jan 14 20:24:03 2018 Tue Jan 16 20:24:02 2018
Expected O/P
Loc ID Img Name Ver. Time Complete Time
------------------------------------------------------------------------------------------
ssfad_fs TINT_PAP_1516048511 0 Mon Jan 15 20:35:13 2018 NA
ssfad_fs sfad_jpg 1 Mon Jan 15 18:24:02 2018 Wed Jan 17 18:24:02 2018
ssfad_fs sfad_jpg 1 Mon Jan 15 16:24:02 2018 Wed Jan 17 16:24:02 2018
ssfad_fs sfad_jpg 2 Mon Jan 15 12:24:03 2018 Wed Jan 17 12:24:02 2018
ssfad_fs sfad_jpg 3 Mon Jan 15 08:24:02 2018 Wed Jan 17 08:24:02 2018
ssfad_fs sfad_jpg 4 Mon Jan 15 04:24:02 2018 Wed Jan 17 04:24:02 2018
ssfad_fs sfad_jpg 5 Mon Jan 15 00:24:03 2018 Wed Jan 17 00:24:59 2018
ssfad_fs sfad_jpg 6 Sun Jan 14 20:24:03 2018 Tue Jan 16 20:24:02 2018
When I try #cat junk1.jnk |sort -uk 3,3 command it deletes third line as well which has same Ver.1 number but different times. I want to keep that line. Please help.
Event model which has start and end datetime attributes in the database. I want to seed some random events but the event time should be proper.
For example:
6.times { date_range << DateTime.now + (rand * 21) }
generates
[Thu, 03 Aug 2017 21:22:48 +0530,
Tue, 08 Aug 2017 17:36:29 +0530,
Sat, 29 Jul 2017 06:19:51 +0530,
Sat, 29 Jul 2017 13:36:21 +0530,
Thu, 27 Jul 2017 15:08:55 +0530,
Fri, 04 Aug 2017 13:53:03 +0530]
which is the correct behaviour.
But how to generate random datetime like this:
[Thu, 03 Aug 2017 21:00:00 +0530,
Tue, 08 Aug 2017 17:30:00 +0530,
Sat, 29 Jul 2017 06:00:00 +0530,
Sat, 29 Jul 2017 13:00:00 +0530,
Thu, 27 Jul 2017 15:30:00 +0530,
Fri, 04 Aug 2017 13:00:00 +0530]
So in order to display these events properly on a calendar.
Could try separating out each segment and adding them onto the date individually
date_range = 6.times.collect do
DateTime.now.beginning_of_day + # starting from today
rand(21).days + # pick a random day, no further than 3 weeks out
rand(24).hours + # move forward to a random hour on that day
(rand(2) * 30).minutes # and then decide whether to add 30 minutes
end
or, could combine the hours + minutes
date_range = 6.times.collect do
DateTime.now.beginning_of_day + # starting from today
rand(21).days + # pick a random day, no further than 3 weeks out
(rand(48) * 30).minutes # pick a random interval of 30 minutes to add in
end
Found the working solution but not complete:
6.times { date_range << DateTime.parse((DateTime.now + (rand * 21)).beginning_of_hour.to_s) }
[Mon, 31 Jul 2017 06:00:00 +0530,
Thu, 03 Aug 2017 15:00:00 +0530,
Fri, 11 Aug 2017 14:00:00 +0530,
Mon, 31 Jul 2017 09:00:00 +0530,
Wed, 09 Aug 2017 16:00:00 +0530,
Sat, 12 Aug 2017 13:00:00 +0530]
This can work for now but need some datetime with 30 minutes as well.
The user inputs the starting date and the end date. I want to get the dates between these two input dates.
I tried this:
(datestart..dateend).to_a
but it returns the whole month, and when I choose from the previous year, it gives an error ArgumentError (invalid date):.
This is the example return when I choose Jan. 1 2017 and Jan. 2 2017 - the result was whole month and an inaccurate dates.
[Tue, 12 Jan 2016, Wed, 13 Jan 2016, Thu, 14 Jan 2016, Fri, 15 Jan 2016, Sat, 16 Jan 2016, Sun, 17 Jan 2016, Mon, 18 Jan 2016, Tue, 19 Jan 2016, Wed, 20 Jan 2016, Thu, 21 Jan 2016, Fri, 22 Jan 2016, Sat, 23 Jan 2016, Sun, 24 Jan 2016, Mon, 25 Jan 2016, Tue, 26 Jan 2016, Wed, 27 Jan 2016, Thu, 28 Jan 2016, Fri, 29 Jan 2016, Sat, 30 Jan 2016, Sun, 31 Jan 2016, Mon, 01 Feb 2016, Tue, 02 Feb 2016, Wed, 03 Feb 2016, Thu, 04 Feb 2016, Fri, 05 Feb 2016, Sat, 06 Feb 2016, Sun, 07 Feb 2016, Mon, 08 Feb 2016, Tue, 09 Feb 2016, Wed, 10 Feb 2016, Thu, 11 Feb 2016, Fri, 12 Feb 2016]
Update! I Fix it by using
Date.strptime(params[:datestart_stat], '%m/%d/%Y')
start_date = Date.parse('date start')
end_date = Date.parse('date end')
(start..endd).to_a
Update! I was able to fix my mistake now. I just change the
Date.parse(params[:datestart_stat])
# to
Date.strptime(params[:datestart_stat], '%m/%d/%Y')
thank you guys.
I'm using moment-timezone 0.5.1 on node 6.3.0
I'm primarily dealing with the Hong Kong timezone, which has been using GMT+0800 since 1904.
Before that, it was using GMT+0736 since 1885
Yet for some reason, moment-timezone formats some dates near the epoch to display GMT+0900, which doesn't seem to have any basis in history.
I can't seem to find the pattern nor can I reproduce this issue in more recent timestamps.
After epoch
moment.tz(123456780, 'Asia/Hong_Kong').toString() // 'Fri Jan 02 1970 18:17:36 GMT+0800'
moment.tz(1234567800, 'Asia/Hong_Kong').toString() // 'Thu Jan 15 1970 14:56:07 GMT+0800'
moment.tz(5999999999, 'Asia/Hong_Kong').toString() // 'Wed Mar 11 1970 18:39:59 GMT+0800'
moment.tz(9000000000, 'Asia/Hong_Kong').toString() // 'Wed Apr 15 1970 12:00:00 GMT+0800'
moment.tz(9300000000, 'Asia/Hong_Kong').toString() // 'Sat Apr 18 1970 23:20:00 GMT+0800'
moment.tz(12345678000, 'Asia/Hong_Kong').toString() // 'Sun May 24 1970 06:21:18 GMT+0900'
moment.tz(9999999999, 'Asia/Hong_Kong').toString() // 'Mon Apr 27 1970 02:46:39 GMT+0900'
moment.tz(9900000000, 'Asia/Hong_Kong').toString() // 'Sat Apr 25 1970 23:00:00 GMT+0900'
moment.tz(9500000000, 'Asia/Hong_Kong').toString() // 'Tue Apr 21 1970 07:53:20 GMT+0900'
moment.tz(9400000000, 'Asia/Hong_Kong').toString() // 'Mon Apr 20 1970 04:06:40 GMT+0900'
moment.tz(9400000000, 'Asia/Hong_Kong').toString() // 'Mon Apr 20 1970 04:06:40 GMT+0900'
Before epoch
moment.tz(-9000000000000, 'Asia/Hong_Kong').toString() // 'Thu Oct 19 1684 15:36:42 GMT+0736'
moment.tz(-90000000000000, 'Asia/Hong_Kong').toString() // 'Sun Jan 06 -0882 15:36:42 GMT+0736'
moment.tz(-500000000000, 'Asia/Hong_Kong').toString() // 'Sat Feb 27 1954 07:06:40 GMT+0800'
moment.tz(-100000000000, 'Asia/Hong_Kong').toString() // 'Mon Oct 31 1966 22:13:20 GMT+0800'
moment.tz(-900000000000, 'Asia/Hong_Kong').toString() // 'Wed Jun 25 1941 17:00:00 GMT+0900'
moment.tz(-200000000000, 'Asia/Hong_Kong').toString() // 'Sat Aug 31 1963 13:26:40 GMT+0900'
moment.tz(-800000000000, 'Asia/Hong_Kong').toString() // 'Sat Aug 26 1944 02:46:40 GMT+0900'
moment.tz(-900000000000, 'Asia/Hong_Kong').toString() // 'Wed Jun 25 1941 17:00:00 GMT+0900'
It seems like it's also a historical answer, based on Hong Kong's adoption of Daylight Savings Time:
Hong Kong adopted daylight saving measures in 1941. However, in the 1970s, the government found these measures unnecessary as Hong Kong is at a relatively low latitude. The practice was eliminated in 1979.
Taking a quick look at the difference between 1941 and 1942, that seems like where you see the switch between GMT+8 and GMT+9:
moment.tz(new Date('1/1/1941'), 'Asia/Hong_Kong').toString()
// 'Wed Jan 01 1941 16:00:00 GMT+0800'
moment.tz(new Date('1/1/1942'), 'Asia/Hong_Kong').toString()
// 'Thu Jan 01 1942 17:00:00 GMT+0900'
We're in the timezone Bern, which is +0100. But since we're now in summertime (we have daylight saving time), the current offset is +0200. In my rails app, I set the timezone using a wrapper in the application controller since I need to have user-based timezones:
around_filter :user_timezone
def user_timezone(&block)
Time.use_zone(current_timezone, &block)
end
Now the strange part:
Time.zone.now # 2013-04-10 10:32:56 +0200
# (correct offset)
SomeArModel.first.created_at # 2013-03-28 17:49:59 +0100
# (incorrect offset, no DST)
Is there any explanation for this?
Thats normal behavior, the DST change happened on Sun Mar 31 01:00:00 UTC 2013.
t = Time.mktime(2013, 03, 31, 1, 15, 0)
6.times do
t += 900
u = Time.at(t.to_i).utc
puts t.to_s + " " + u.to_s
end
output:
Sun Mar 31 01:30:00 +0100 2013 Sun Mar 31 00:30:00 UTC 2013
Sun Mar 31 01:45:00 +0100 2013 Sun Mar 31 00:45:00 UTC 2013
Sun Mar 31 03:00:00 +0200 2013 Sun Mar 31 01:00:00 UTC 2013
Sun Mar 31 03:15:00 +0200 2013 Sun Mar 31 01:15:00 UTC 2013
Sun Mar 31 03:30:00 +0200 2013 Sun Mar 31 01:30:00 UTC 2013
Sun Mar 31 03:45:00 +0200 2013 Sun Mar 31 01:45:00 UTC 2013