I have a situation where I would like to detect when a source has not sent event into the systems for more than 24 hours. When this pattern is recognized, I would like to be able to retrieve the last know event (which may be days) the source did send something. I would like this check to be done every 24 hours. I've followed the 'ATM' type example and came up with the following, pretty simple.
select a.value from pattern[every (time:interval(24 hours) and not a=Event)
This notify my update listener when 24 hour has elapsed and no event a. But how do I get previous know? I thought about using the prev or std:lastevent functions but I need a data window to select from, wasn't sure where to put that.
thanks
You could use "prior", or join in the last event like the example below shows:
select a.value, * from pattern[every (time:interval(24 hours) and not a=Event)], Event.std:lastevent()
Related
I have a Google Dataflow job that reads data from PubSub, aggregates de data and in the end, sends the data to an InflluxDB. What I want to achieve is to aggregate the data in windows of 1 minute but to have only an entry in the DB for each minute. The problem is that I want to allow lateness data so I need to accumulate the data during a period of 5 minutes and then to send to the DB a unique entry.
Is it possible? I tried to do that with the below code, but I don't get what I want:
input.apply(Window
.<KV<String, String>>into(FixedWindows.of(Duration.standardMinutes(1)))
.triggering(
AfterProcessingTime
.pastFirstElementInPane()
.plusDelayOf(Duration.standardMinutes(5)))
.withAllowedLateness(Duration.standardMinutes(5))
.discardingFiredPanes()
I already collaborated on a similar question. You can use .triggering(Never.ever()) to omit sending the ON TIME panes. Then, as you are already doing, set the allowed lateness to 5 minutes for late records.
It's also important to change the Window.ClosingBehavior to FIRE_ALWAYS. This way we account for the case where there is no late data but we haven't emitted the on-time records. Once the window is closed it will always emit a final pane with PaneInfo.isLast set to true.
So, for your case, the code would be something like:
input.apply(Window
.<KV<String, String>>into(FixedWindows.of(Duration.standardMinutes(1)))
.triggering(Never.ever())
.withAllowedLateness(Duration.standardMinutes(5), Window.ClosingBehavior.FIRE_ALWAYS)
.discardingFiredPanes()
I'm currently using the Microsoft Graph API to sync calendar events to my local application. On my end, I don't care to save each individual occurrence in a series, but prefer instead to just save the series master and then extrapolate out the instances of the series myself. For this reason, I am using the /me/events call rather than the /me/calendarView call.
My problem is when editing a single occurrence in a series. After editing the single occurrence, I make the /me/events call and I can see the newly added "Exception" type -- which is great. However, I don't see how to relate that new event back to which occurrence was changed to cause the exception.
For example, if I have a weekly meeting on Monday at noon, and I change today's meeting from noon to 2:00, it's pretty easy to tell that today's meeting is the one that changed. But if I change today's meeting to Friday, how can I tell that it was today's meeting that changed and not next week's? Keep in mind that I am only storing the master, and not every single calendarView occurrence.
Another example is if I delete an occurrence. In this case, the /me/calendarView call will simply not return that occurrence anymore. No exception type is generated. And the series master returned from the /me/events call doesn't change at all to indicate that a date is missing.
The format that I'm used to is something like the iCal/vCal format, where there is a start date, end date, and then a list of exception dates. Using that format, I can easily tell from the series master which dates to skip, without needing to "render" the entire occurrence and skip the exceptions. And if an occurrence is deleted, it is added to the EXDATE list and then it is never considered on rendering. Does the Microsoft Graph API not have an easy way to see these changed/deleted occurrences?
I was having a similar issue, but I think I've now realized that Microsoft does not allow recurring events to move later than the next instance, or earlier than the preceding one (at least while using Outlook calendar in the browser). So you can always assume that the 3rd event is 3rd in the series, the 4th is 4th, etc.
So as long as you know the series number, you can locate it by getting all of the instances with /me/events/[event_id]/instances?startDateTime=[start_date_time]&endDateTime=[end_date_time].
The error in Outlook Calendar when I do this isn't very clear, so maybe something else is up, but I am able to move the exception events otherwise. Unfortunately, I'm not sure if there's a definite way to know what end_date_time to use, as events can be moved indefinitely later.
Based on the response object from the event marked as an exception, you can use the seriesMasterId to relate the exception to its parent recurrence.
is there a way to query for next 5 instances of given series? I am querying using time-frame:
1. ask for all the meetings for next 7 days for all user events
2. Go over each event fetched and check if event has masterSerieId
3. Return matching instances.
This feels (and is) a bit painful.
Can i request next X instances of master series right away? I can't just simply or 'simply' get them based on recurrence rule, as some might have expired.
I could image I could ask for a year ahead and pass as a query parameter masterSeriesId and limit output with $top. Is that right approach?
Based on this document, you could call /events//instances – given a start time, returns all instances in the requested time frame of the meeting specified by the provided series master ID
My goal is to add +1 every day to a global variable in Firebase to track how many days have passed. I'm building an app that give new facts every day, and at the 19:00 UTC time marker, I want the case statement number (the day global day variable) to increment by +1.
Some have suggested that I compare two dates and get the days that have passed that way. If I were to do that, I could hard code the initial time when I first want the app to start at 19:00 some day. Then when the function reached1900UTC() is called everyday thereafter, compare it to a Firebase timestamp of that current time which should be 19:00. In theory, it should show that 1 day or more day has passed.
This is the best solution so far, thanks to #DavidSeek and #Jay, but I would still like to figure it out with concurrent writes if anyone has a solution in that front. Until then, I'm marking David's answer as the correct one.
How would I make it so it can't increase more than +1 if multiple people call this? Because my fear is that, when say, 100 people calls this function, it increases by + 1 for every person that has called it.
My app works on a global time, and this function is called every day at 19:00 UTC. So when that function is called I want the day count to increase by one.
You should use transactions to handle concurrent writes:
https://firebase.google.com/docs/database/ios/read-and-write#save_data_as_transactions
You may know this but Firebase doesn't have a way to auto-increment a counter as there's no server side logic, so having a counter increment at 19:00 UTC isn't going to be possible without interaction from a client that happens to be logged on at that time.
That being said, it's fairly straightforward to have the first user that logs in increment that counter - then any other clients logging in after that would not increment it and would have access to that day's new content.
Take a look at Zapier.com - that's a service that can fire time based triggers for your app which may do the trick.
As of this writing, Zapier and Firebase don't play nice together, however, there are a number of other trigger options that Zapier can do with your app while continuing to use Firebase for storage.
One other thought...
Instead of dealing with counters and counting days, why not just have each day's content stored within a node for each day and when each user logs on, the app get's that days content:
2016-10-10
fact: "The Earth is an Oblate Spheroid"
2016-10-11
fact: "Milli Vanilli is neither a Milli or a Vanilli. Discuss."
2016-10-12
fact: "George Washington did not have a middle name"
This would eliminate a number of issues such as counters, updates, concurrent writing to Firebase, triggers etc.
It's also dynamic and expandable and a user could easily see that day's facts or the fact for any prior day(s)
I'm trying to split your question into different sections.
1) If you want to use a global variable to count the days from, let's say, today. Then I would set a timestamp hardcoded into the App that sets the NSDate.
Then In my App, when I need to know the days that have been passed by, I would call a function counting the days from the timestamp to NSDate().
2) If you have a function in your App that counts a +1 into a Firebase, then your fear is correct. It would count +1 for every person that uses the App.
3) If you want every User to have a variable count since when they use their App, then I would handle User registration. So I have a "UserID" and then I would set a Firebase tree like that:
UserID
------->
FirstOpen
-------> Date
That way you could handle each User's first open.
Then you are able to set a timestamp AND call +1 for every user independently. Because then you set the +1 for every user into their UserID .child
I need some help concerning the CEP engine Esper:
I wrote the following statement:
SELECT clientID FROM MovementEvent.win:time_batch(5 sec) GROUP BY clientID
And then I sent a MovementEvent into the engine. after 5 seconds, the subscriber is triggered - ok. But after 5 more seconds, it is triggered again - why?
No matter how many events I send to the engine, the subscriber always is triggered twice - after 5 and 10 seconds
I hope you could help me!
Thanx ;)
What you need is this:
SELECT clientID FROM MovementEvent.std:groupwin(clientID).win:time_batch(5 sec);
In general, using GROUP BY without an aggregation function applied to the properties you are selecting will almost always catch you off-guard :). For the sake of it, this statement would also produce what you expected:
SELECT distinct(clientID) FROM MovementEvent.win:time_batch(5 sec);
This link will probably explain the differences better: http://www.espertech.com/esper/release-5.2.0/esper-reference/html/epl_clauses.html#epl-group-by-versus-view
The group-by makes this an aggregation and Esper provides output when there is a change in the aggregation for the keys, even if you don't select aggregations.
You could do this one
select window(*).distinctOf(v=>client) from MovementEvent.win:time_batch(5 sec)