I'm trying to use the Esper EPL Online console and I don't know how to use Dates in the Time And Event Sequence. For example, I have a schema defined like this:
create schema EventCreated(
source String,
type String,
time Date
);
And I'm trying to add an event in the Time And Event Sequence, for example like this:
EventCreated = {
source = 'tracker1',
type = 'c8y_ObdConnectionReport',
time = '2016-10-07T10:00:00.000'
}
But of course this doesn't work. I tried using "new Date()" or [com.espertech.esper.client.util.]DateTime.parseDefaultDate() but I can't make it work.
There is an "eval" for evaluating EPL expressions.
StockTick={time=eval('com.espertech.esper.client.util.DateTime.parseDefaultDate("2016-10-07T10:00:00.000")')}
It would be nice though if the tool would just take the string and make it a date.
Related
I'm working in F# with a CSV that looks like this:
When,Where,HowMuch
11/24/2019,Germany,100 EUR
11/25/2019,France,100 EUR
11/26/2019,Switzerland,50 CHF
11/27/2019,USA,75 USD
I'm using the CSV Type Provider in the FSharp.Data package to parse this data.
type CurrencyDetector = CsvProvider<"Currencies.csv">
Obviously the first column is a date, and the second is a string.
For the third, I'd like to use this type:
type Money (amountAndDenomination : string) =
let parts = amountAndDenomination.Split ' '
member __.Amount = Decimal.Parse parts.[0]
member __.Denomination = parts.[1]
I've tried a few permutations of the Schema argument in my CsvProvider line, but nothing has worked so far. For example:
type CurrencyDetector = CsvProvider<"Currencies.csv",Schema="When (date),Where (string),HowMuch (Money)">
When comes out as DateTime and Where as string, but HowMuch becomes a string property named HowMuch (Money):
Is there a way to use my own classes with the CsvProvider, or is this just not possible?
According to the documentation for the CsvProvider, I don't think it is possible:
Schema parameter: "Optional column types, in a comma separated list. Valid types are int, int64, bool, float, decimal, date, guid, string, int?, int64?, bool?, float?, decimal?, date?, guid?, int option, int64 option, bool option, float option, decimal option, date option, guid option and string option. You can also specify a unit and the name of the column like this: Name (type<\unit>), or you can override only the name. If you don't want to specify all the columns, you can reference the columns by name like this: ColumnName=type."
Note in the above, however, that there is the possibility of using units of measure. So you might explore creating units of measure for the currency denominations. The type provider might require a different format for that column though.
In the documentation for the CSV Type Provider, you can find further information about units of measure and also how to "transform the columns themselves by using Map", which should enable you to map the string type to a custom type. Looking at the source code, suggests that the Rows collection is a sequence of Row objects, each of which is a tuple. So you should be able to use Seq.map and/or any other function in the Seq module to post-process the generated collection.
I get a daily email that lists upcoming appointments, and their length. The number of appointments vary from day to day.
The emails go like this:
================
Today's Schedule
9:30 AM
3h
Brazilian Blowout
[Client #1 name]
12:30 PM
1h
Women's Cut
[Client 2 name]
6:00 PM
45m
Men's Cut
[Client #3 name]
Projected Revenue
===================
I want to create an event in a Google Calendar for each appointment, and it seems like zapier MIGHT be able to do this, but all the help resources I can find are very general in nature.
Is this do-able on Zapier? If so, any nudges in the right direction would be awesome.
Any thoughts greatly appreciated.
I had some time to kill and enjoy the odd challenge. So I have put together a solution that should do what you are looking for. I will break it down by steps.
TEMPLATE
Zapier Trigger - Step 1
Type: Trigger
Module: Gmail
Criteria: User Dependent
Comments: For the trigger zap you will want to use a Gmail specific trigger, something to the effect of "execute trigger on emails titled 'xyz'", or "emails labeled 'xyz'" if you setup a filter in your inbox.
Input screenshot:
Output Screenshot:
Zapier Action - Step 2
Type: Action
Module: Code (Python 3)
Comments: The Code offered by Zapier executes whatever (properly written) code you place in its container. It is especially handy as it allows you to incorporate data from previous steps in it through the use of a dictionary variable titled 'input_data'. Zapier offers the Code module in two languages: Javascript and Python. As I am most familiar with Python my solution for this step was written in Python. I will append the code to the end of this answer. Using the data held in the body of the email (retrieved in step 1) we can execute some string manipulations and datetime conversions to break apart the email into its component parts and pass those on to the following Action Step: Create Calendar Event.
Input Screenshot:
Output Screenshot:
Zapier Action - Step 3
Type: Action
Module: Google Calendar - Create Event
Comments: Using the data outputted from the previous code step we can fill out the required fields for creating a new appointment.
Input Screenshot:
Output Screenshot:
PYTHON CODE
from datetime import timedelta, date, datetime
'''
Goal: Extract individual appointment details from variable length email
Steps:
Remove all extraneous and new line characters.
Isolate each individual appointment and group its relevant details.
Derive appointment start and end times using appointment time and duration.
Return all appointments in a list.
'''
def format_appt_times(appt_dict):
appt_start_str = appt_dict.get("appt_start")
appt_dur_str = appt_dict.get("appt_length")
# isolate hour and minutes from appointment time
appt_s_hour = int(appt_start_str[:appt_start_str.find(":")])
if ("pm" in appt_start_str.lower()):
appt_s_hour = 12 if appt_s_hour + 12 >= 24 else appt_s_hour + 12
appt_s_min = int(appt_start_str[appt_start_str.find(":") + 1 :
appt_start_str.find(":") + 3])
# isolate hour and minutes from duration time
appt_d_hour = 0
appt_d_min = 0
if ("h" in appt_dur_str):
appt_d_hour = int(appt_dur_str[:appt_dur_str.find("h")])
if ("m" in appt_dur_str):
appt_d_min = int(appt_dur_str[appt_dur_str.find("m") - 2 : appt_dur_str.find("m")])
# NOTE: adjust timedelta hours depending on your relation to UTC
# create datetime objects for appointment start and end times
time_zone = timedelta(hours=0)
tdy = date.today() - time_zone
duration = timedelta(hours=appt_d_hour, minutes=appt_d_min)
appt_start_dto = datetime(year=tdy.year,
month=tdy.month,
day=tdy.day,
hour=appt_s_hour,
minute=appt_s_min)
appt_end_dto = appt_start_dto + duration
# return properly formatted datetime as string for use in next step.
return (appt_start_dto.strftime("%Y-%m-%dT%H:%M"),
appt_end_dto.strftime("%Y-%m-%dT%H:%M"))
def partition_list(target, part_size):
for data in range(0, len(target), part_size):
yield target[data : data + part_size]
def main():
# Remove all extraneous and new line characters.
email_body = input_data.get("email_body")
head,delin,*email_body,delin,foot = [text for text in email_body.splitlines() if text != ""]
appointment_list = []
# Isolate each individual appointment and group its relevant details.
for text in partition_list(email_body, 4):
template = {
"appt_start" : text[0],
"appt_end" : None,
"appt_length" : text[1],
"appt_title" : text[2],
"appt_client" : text[3]
}
appointment_list.append(template)
for appt in appointment_list:
appt["appt_start"], appt["appt_end"] = format_appt_times(appt)
return appointment_list
return main()
I am not sure of your familiarity with Python, or programming more generally, but the comments in the code explain what each section is doing. If you have any specific questions regarding aspects of the code let me know. Assuming your email template does not change this setup should work exactly as needed. Let me know if anything is unclear.
UPDATE
I thought it best to address your question in the original answer should anyone else have similar questions.
explaining how this code is removing the extra characters:
There is actually a fair bit going on in the first line, so I will do my best to break it down, and provide resources where necessary.
The code in question:
head,delin,*email_body,delin,foot = [text for text in email_body.splitlines() if text != ""]
First step here was to break the text into manageable chunks. I did so with the line email_body.splitlines() which, by default, breaks strings into a list at each newline character found (you can specify your own delimiter).
If we were to inspect the list at this moment its contents would be something of the following:
["================", "", "Today's Schedule", "", "9:30 AM", "", "3h", ..., "[Client #3 name]", "", "Projected Revenue", "", "==================="]
You will notice there is a fair amount of information in there that we really don't want.
First lets look at the "" elements. These are left over as a result of the blank lines between each line of text, which even though they are blank do still have newline characters at the end of them. There a number of ways you could address this within python. We could simply write a for-loop to go through and copy all elements that are not "" to a new list.
To me this felt like additional work, and besides, Python offers list comprehension for just such a scenario. I won't go too deep into list comprehension as there is a lot that can be said about it, and in more insightful ways than I could muster, but it essentially allows you to provide logic against a set of 'data' to form a list. In this case, I specifically wanted to filter out the "" elements returned from the call to splitlines().
And so you will see I address this with the following line
[text for text in email_body.splitlines() if text != ""]
With that we have a list as above less the "" elements. Now we must turn our attention towards the more 'dynamic' garbage strings. Again there are a number of ways to do this. A, not particularly flexible, option could be to simply store the strings we want to remove in variables something to the effect of:
garb_1 = "==================="
garb_2 = "Projected Revenue"
garb_3 = ...
and once again filter the list with yet another for-loop. I instead chose to leverage Python's list unpacking idiom. Which allows us to 'unpack' list objects (and I believe tuples) into variables. As an example:
one, two, three = ["a", "b", "c"]
I'm sure you can guess what is happening above, as long as we provide the same number of variables as are in the list we can 'unpack' it in this fashion. But wait! In our case we don't know how long the list is going to be as it is entirely dependent on the number of appointments you have for any given day. Well this is where star unpacking enters to elevate the functionality. Using my code as the example:
head,delin,*email_body,delin,foot = [text for text in email_body.splitlines() if text != ""]
The *, in plain-English, is saying "I don't know how many elements to expect just give me all of them in a list". As we know that there will always be two lines of garbage at the beginning and end of the email we can assign them to throw away variables and capture everything in between using our variable length *email_body container.
With all of this complete we now have a list with only the data we are looking to capture. If, as you say, there are additional lines of garbage before or after the email_body, you can simply add additional throw away variables to account for them.
Once again feel free to ask any follow up questions.
Michael
Resources
List Comprehension
Star Unpacking
I am really newbie in F#. Below is my attempt in creating a function that calculates the difference between two dates.
let test date =
let today = DateTime.Now
let fromdate = DateTime.Parse(date)
let count = Convert.ToInt32(today - fromdate)
date
The above code prompts the error
System.InvalidCastException: Unable to cast object of type
'System.TimeSpan' to type 'System.IConvertible'.
You don't need to convert the time span resulting from applying the "minus" operator to two dates.
The screenshot below is my attempt at fixing your function and an example of value it returns.
The difference between two DateTimes will give you a TimeSpan object, on which you can further operate. This should be the last statement in your function, as it will be used as the return value. You can check the MS docs in the link for the various properties and methods. Strictly speaking, this is part of the BCL, so it's rather .NET than just F# (you would do the same in VB or C#). If you want to further refine your function you should examine DateTime.TryParse as well and handle the possibility of not receiving a valid date.
open System
let test date =
let today = DateTime.Now
let fromdate = DateTime.Parse(date)
(today - fromdate).Days
test "2017/12/31" // val it : int = -325
I'm looking for a way to determine the time difference between two consecutive events that have a time event property with values like: 2016-08-25T13:05:06.953391Z. What would be the best approach here and what type do I need to use for the event schema. When using Esper EPL online I have tried to use Long or Date but then I get the following error:
Please check the EPL Module Text
Deployment failed in expression 'create schema StockTick(symbol string, price doubl...(63 chars)' : Error starting statement: Nestable type configuration encountered an unexpected property type name 'date' for property 'time', expected java.lang.Class or java.util.Map or the name of a previously-declared Map or ObjectArray type [create schema StockTick(symbol string, price double, time date)]
OR
Please check the Advance Time and Send Events text
Instruction at line 1 invalid event: Property by name 'time' cannot accept the assigned value: Invalid assignment of column 'time' of type 'java.lang.String' to event property 'time' typed as 'java.lang.Long', column and parameter types mismatch
For the EPL online tool web application, I don't think there is support for parsing custom date format. You can go with "long" and use the millisecond since 00:00:00 UTC on 1 January 1970.
Let's say that I have a simple object:
public class StockTick{
private String symbol;
private decimal price;
private Date date;
private int unixTimestamp
}
How should I modify following query to use StockTick.date or StockTick.unixTimestamp to agregate within .win:time() window?
select avg(price) from StockTick.win:time(30 sec) where symbol='IBM'
If events are already ordered by unix timestamp, you don't need to modify the query. Just do this for each event:
runtime.sendEvent(new CurrentTimeEvent(unixTimestamp));
runtime.sendEvent(stockTickEvent);
This above code uses the external time so disable the default internal system time. For completely unordered or unstaged events, don't use a time window at all and instead think about how a group-by would look like.