I want to calculate the daily average temperature by doing:
Take first value each hour -
query1 = SELECT first("value") FROM "externaltemp" WHERE $timeFilter GROUP BY time(1h) fill(none)
And with those values I want to take the average value per day.
SELECT mean(query1) ... ?
My question is how to combine one query and use the data from the first to the next. Is that possible?
EDIT:
Added some data:
Time externaltemp.distinct
2018-04-04 07:07:00 3.80
2018-04-04 07:05:00 3.80
2018-04-04 07:04:00 3.80
2018-04-04 07:03:00 3.80
2018-04-04 07:02:00 3.80
2018-04-04 07:01:00 3.80
2018-04-04 07:00:00 3.80
2018-04-04 06:59:00 3.80
2018-04-04 06:58:00 3.80
2018-04-04 06:57:00 3.80
2018-04-04 06:56:00 3.80
2018-04-04 06:55:00 3.80
2018-04-04 06:54:00 3.80
2018-04-04 06:53:00 3.70
2018-04-04 06:52:00 3.80
2018-04-04 06:51:00 3.70
2018-04-04 06:50:00 3.70
2018-04-04 06:49:00 3.70
2018-04-04 06:48:00 3.70
2018-04-04 06:47:00 3.80
2018-04-04 06:46:00 3.70
2018-04-04 06:45:00 3.70
2018-04-04 06:44:00 3.70
2018-04-04 06:43:00 3.70
And with query:
SELECT first("value") FROM "externaltemp" WHERE $timeFilter GROUP BY time(1h)
Time externaltemp.first
2018-04-04 07:00:00
3.80
2018-04-04 06:00:00 3.70
2018-04-04 05:00:00 2.90
2018-04-04 04:00:00 2.60
2018-04-04 03:00:00 2.20
2018-04-04 02:00:00 1.90
2018-04-04 01:00:00 2.30
2018-04-04 00:00:00 3.60
2018-04-03 23:00:00 5.50
2018-04-03 22:00:00 6.50
2018-04-03 21:00:00 7.50
USE below code
SELECT MEAN("first") FROM (SELECT first("value") FROM "externaltemp" WHERE $timeFilter GROUP BY time(1h) fill(none))
Related
I'm having trouble creating a dask time series dataframe that calculates the mean per hour over multiple columns.
This is an example of my input csv file:
name,date_time,num
dan,2019-01-02 00:00:00,3
ben,2019-01-02 00:00:00,7
dan,2019-01-02 02:00:00,13
dan,2019-01-02 10:00:00,9
dan,2019-01-02 10:01:00,3
ben,2019-01-02 14:22:00,66
ben,2019-01-02 14:37:00,37
I can produce the desired output using pandas
import pandas as pd
from matplotlib import pyplot
df = pd.read_csv('my_file.csv')
df['timestamp'] = pd.to_datetime(df.date_time)
df = df.set_index(df.timestamp) # set a datetime index
df = df.groupby('name').resample('H')['num'].mean().unstack('name')
df.fillna(0).plot()
Desired output
name ben dan
timestamp
2019-01-02 00:00:00 7.0 3.0
2019-01-02 01:00:00 NaN NaN
2019-01-02 02:00:00 NaN 13.0
2019-01-02 03:00:00 NaN NaN
2019-01-02 04:00:00 NaN NaN
2019-01-02 05:00:00 NaN NaN
2019-01-02 06:00:00 NaN NaN
2019-01-02 07:00:00 NaN NaN
2019-01-02 08:00:00 NaN NaN
2019-01-02 09:00:00 NaN NaN
2019-01-02 10:00:00 NaN 6.0
2019-01-02 11:00:00 NaN NaN
2019-01-02 12:00:00 NaN NaN
2019-01-02 13:00:00 NaN NaN
2019-01-02 14:00:00 51.5 NaN
My attempt to produce the same dataframe with dask
from dask import dataframe as dd
from matplotlib import pyplot
ddf = dd.read_csv('my_file.csv')
# setting an index
ddf['timestamp'] = dd.to_datetime(ddf.date_time)
ddf = ddf.set_index(ddf.timestamp)
ddf.repartition(freq='MS')
ddf.groupby('name').resample('H')['num'].mean()
When I run the code above I get this error:
AttributeError: 'Column not found: resample'
This has me really stumped and any help would be appreciated.
It looks like dask dataframe does not implement a groupby-resample operation. It sounds like you have a feature request. I recommend raising an issue at https://github.com/dask/dask/issues/new
See https://docs.dask.org/en/latest/support.html#asking-for-help for requests on where to ask for help.
I'm scratching my head trying to work with time functions within Cognos 10.2.1 (Report Studio), using an Informix db as a data source.
My time field is stored as a smallint, 4 digits, representing the 24 hour clock. I am trying to get the time to display as 6:00pm, 11:30am, 3:00pm, etc. I have a separate data expression that calculates the string 'AM' or 'PM' depending on the hour value, but I'm running into some errors when doing the overall concat/substring function.
case when char_length([Query1].[beg_tm]) = 4
then (substring(cast([StartTime], char(5)), 1, 2)) || ':' || (substring (cast ([StartTime], char(5)), 3, 2)) || ([beg_AMPMcalc])
when char_length([Query1].[beg_tm]) = 3
then (substring(cast([StartTime], char(5)), 1, 1)) || ':' || (substring(cast ([StartTime], char(5)), 3, 2)) || ([beg_AMPMcalc])
else '--'
end
Why not use DATETIME HOUR TO MINUTE; at least you then only have to deal with converting 24 hour clock to 12 hour clock. Is midnight stored as 0 and noon as 1200, and the minute before midnight as 2359? Cognos uses a fairly modern version of Informix, I believe, so you should be able to use the TO_CHAR function:
DROP TABLE IF EXISTS times;
CREATE TEMP TABLE times(p_time SMALLINT);
INSERT INTO times VALUES(0);
INSERT INTO times VALUES(59);
INSERT INTO times VALUES(100);
INSERT INTO times VALUES(845);
INSERT INTO times VALUES(1159);
INSERT INTO times VALUES(1200);
INSERT INTO times VALUES(1259);
INSERT INTO times VALUES(1300);
INSERT INTO times VALUES(1815);
INSERT INTO times VALUES(2359);
SELECT TO_CHAR(CURRENT HOUR TO MINUTE, "%I:%M %p"),
p_time,
DATETIME(00:00) HOUR TO MINUTE + MOD(p_time, 100) UNITS MINUTE + (p_time/100) UNITS HOUR,
TO_CHAR(DATETIME(00:00) HOUR TO MINUTE + MOD(p_time, 100) UNITS MINUTE + (p_time/100) UNITS HOUR, "%I:%M %p")
FROM times;
Output:
03:49 AM 0 00:00 12:00 AM
03:49 AM 59 00:59 12:59 AM
03:49 AM 100 01:00 01:00 AM
03:49 AM 845 08:45 08:45 AM
03:49 AM 1159 11:59 11:59 AM
03:49 AM 1200 12:00 12:00 PM
03:49 AM 1259 12:59 12:59 PM
03:49 AM 1300 13:00 01:00 PM
03:49 AM 1815 18:15 06:15 PM
03:49 AM 2359 23:59 11:59 PM
I'm using a database server that has its local time set to UTC, and I'm in time zone -07:00 (US/Pacific); the current time isn't the middle of the night where I am.
I have this datetime, or something that looks like it.
2014-11-17 23:02:03 +0000 UTC
I want to convert this to a time object and I've been unable to produce any output from time.Parse apart from:
0001-01-01 00:00:00 +0000 UTC
I've tried these layouts:
time.RFC3339
0001-01-01 00:00:00 0000 UTC
2016-10-10
time.UnixDate
And a few more - none have worked.
This is how I'm calling parse :
updatedAt, err := time.Parse(time.UnixDate, updatedAtVar)
How do I create a time object from a string?
Most likely you used a wrong layout, and you didn't check the returned error.
The layout must be this date/time, in the format your input time is:
Mon Jan 2 15:04:05 -0700 MST 2006
See this working code:
layout := "2006-01-02 15:04:05 -0700 MST"
t, err := time.Parse(layout, "2014-11-17 23:02:03 +0000 UTC")
fmt.Println(t, err)
Output (try it on the Go Playground):
2014-11-17 23:02:03 +0000 UTC <nil>
EDIT:
In your question you included a + sign in your input time (as part of the zone offset), but you have error with times of other formats.
Time.String() uses the following format string:
"2006-01-02 15:04:05.999999999 -0700 MST"
So either use this to parse the times, or use Time.Format() to produce your string representations where you can specify the layout, so you can use the same layout to parse the time strings.
2nd round:
You're including your time strings in URLs. The + sign is a special character in URL encoding: it denotes the space. So the + gets converted to space (and so it vanishes from your time string). Use proper URL encoding! Check out the net/url package, and this example.
Didn't see this yet but for those that don't know the formats, time has the formats builtin as constants. so you can reference them when parsing or formating.
time.Parse(time.RFC3339, <your time.Time object here>)
<time.Time object>.Format(time.RFC3339) //or other type of formats
Here they are for reference
ANSIC = "Mon Jan _2 15:04:05 2006"
UnixDate = "Mon Jan _2 15:04:05 MST 2006"
RubyDate = "Mon Jan 02 15:04:05 -0700 2006"
RFC822 = "02 Jan 06 15:04 MST"
RFC822Z = "02 Jan 06 15:04 -0700" // RFC822 with numeric zone
RFC850 = "Monday, 02-Jan-06 15:04:05 MST"
RFC1123 = "Mon, 02 Jan 2006 15:04:05 MST"
RFC1123Z = "Mon, 02 Jan 2006 15:04:05 -0700" // RFC1123 with numeric zone
RFC3339 = "2006-01-02T15:04:05Z07:00"
RFC3339Nano = "2006-01-02T15:04:05.999999999Z07:00"
Kitchen = "3:04PM"
You are likely using the wrong layout. As explained in time.Parse, you need to specify a layout that helps Go to understand how the date passed as input is formatted.
There are predefined layouts (like the ones you were using), but none matches your input. Hence you need to define a custom layout.
A layout uses the following date as reference:
Mon Jan 2 15:04:05 MST 2006
The layout is nothing else that a representation of that date, that matches the representation of your input:
t, err := time.Parse("2006-01-02 15:04:05 -0700 MST", "2014-11-17 23:02:03 +0000 UTC")
Also remember to check err for errors. It's likely your attempts returned an error, but you didn't check it.
package main
import (
"fmt"
"log"
"time"
)
func main() {
t, err := time.Parse("2006-01-02 15:04:05 -0700 UTC", "2014-11-17 23:02:03 +0000 UTC")
if err != nil {
log.Fatalln(err)
}
fmt.Println(t)
}
I am trying to convert a Time object from the PST timezone to UTC:
time = "2016-08-15 11:51:27 America/Los_Angeles"
I am trying to use:
utc = Time.parse(time).utc
but it doesn't return the correct time. It returns:
# 2016-08-15 11:51:27 UTC
What's wrong here?
Try:
require 'tzinfo'
tz = TZInfo::Timezone.get('America/Los_Angeles')
utc_time = tz.local_to_utc(Time.parse("2016-08-15 11:51:27"))
which will return:
=> 2016-08-15 18:51:27 UTC
From firebug:
>>> jQuery.fullCalendar.parseISO8601("2011-04-18T17:00:00Z").getUTCHours()
22
Shouldn't the result be 17?
parseISO8601 returns this date object:
>>> jQuery.fullCalendar.parseISO8601("2011-04-18T17:00:00Z")
Date {Mon Apr 18 2011 17:00:00 GMT-0500 (CDT)}
I think the date object should be "12:00:00 GMT-0500" to be the same time. Am I misunderstanding it?
From FF 4's Date object:
>>> new Date("2011-04-18T17:00:00Z")
Date {Mon Apr 18 2011 12:00:00 GMT-0500 (CDT)}
>>> new Date("2011-04-18T17:00:00Z").getUTCHours()
17
you are experiencing this issue:
http://code.google.com/p/fullcalendar/issues/detail?id=750
will get it fixed soon