I'm trying to generate a stream of deduplicated events without specifying any window policy beyond that used for the deduplication. Using an output first every clause on my queries appears to have the desired effect, but not when those queries are inserting directly into a stream.
For the example given below, say that I'm trying to detect only the first honk from each car in a 4-hour window.
(define-event-type! "CarEvent"
{:license_plate java.lang.String})
(define-event-type! "HonkEvent"
{:volume java.lang.Integer}
:supertypes #{"CarEvent"})
(define-variant! "HonkEventDeduplicated" "HonkEvent")
(define-statement! "context-IndividualCarContext"
"create context IndividualCarContext partition by license_plate from CarEvent")
(define-statement! "populate-HonkEventDeduplicated"
"context IndividualCarContext
insert into HonkEventDeduplicated
select * from HonkEvent
group by license_plate
output first every 4 hours")
However -- select * from HonkEventDeduplicated fires on every single honk event, even when the same car honks twice in a row.
Instead of using output first every clause filtering, this can be done with the std:firstunique view:
(define-statement!
"populate-HonkEventDeduplicated"
"insert into HonkEventDeduplicated
select * from HonkEvent.win:time(4 hours).std:firstunique(license_plate)")
Related
Here are two events:AppStartEvent and AppCrashEvent.
I need to count the number of two events over a period of time, and then to calculate the count(AppStartEvent)/count(AppCrashEvent).
My EPL is here
create context ctx4NestInCR
context ctx4Time initiated #now and pattern [every timer:interval(1 minute)] terminated after 15 minutes,
context ctx4AppName partition by appName from AppStartEvent, appName from AppCrashEvent
<------------------->
context ctx4NestInCR select count(s),count(c) from AppStartEvent as s, AppCrashEvent as c output last when terminated
And it does not work
Error starting statement: Joins require that at least one view is specified for each stream, no view was specified for s
Your post doesn't have the join? It only has the context and that wouldn't produce the message. I would suggest to correct the post.
You can also join streams by merging the two streams and treating them as one.
insert into AppEvent select 'crash' as condition from AppCrashEvent;
insert into AppEvent select 'start' as condition from AppStartEvent;
select count(condition='crash')/count(condition='start') from AppEvent;
I am looking for an EPL statement which fires an event each time a certain value has increased by a specified amount, with any number of events in between, for example:
Considering a stream, which continuously provides new prices.
I want to get a notification, e.g. if the price is greater than the first price + 100. Something like
select * from pattern[a=StockTick -> every b=StockTick(b.price>=a.price+100)];
But how to realize that I get the next event(s), if the increase is >= 200, >=300 and so forth?
Diverse tests with context and windows has not been successful so far, so I appreciate any help! Thanks!
The contexts would be the right way to go.
You could start by defining a start event like this:
create schema StartEvent(threshold int);
And then have context that uses the start event:
create context ThresholdContext inititiated by StartEvent as se
terminated after 5 years
context ThresholdContext select * from pattern[a=StockTick -> every b=StockTick(b.price>=context.se.threshold)];
You can generate the StartEvent using "insert into" from the same pattern (probably want to remove the "every") or have the listener send in a StartEvent or declare another pattern that fires just once for creating a StartEvent.
I am using esper and when I query my named window with timer:interval(10 sec) i get the same event duplicated after every 10 secs. I doesn't outputs other events present in the window.
here is the EPL
on pattern[every timer:interval(20 sec)] select DeviceIP, parent , child, Supress, Status from testZabbixProblem"
The engine outputs all rows in one call to the listener and delivering all rows at once as an array and not just one row.
I currently have an order object. We can assume it has three fields called orderId, state and price.
class Order
{
public int orderId;
public String state;
public int filled;
}
Through out the life time of the order the state and filled quantity will change. Each time there is a field change we push it to the esper runtime via:
Order o .....;
epService.EPRuntime.SendEvent(o);
Now each time the order is added via SendEvent its a different object than the previous order object( ie not a reference). This means the old order object should no longer be in stream for statements to see
I would like statements like the one below to only operate on the most recent version of the Order in the stream, ie conceptually there should only be one order object for each physical order in the stream.
"select filled from OrderStream.win:keepall() where orderId= 1234"
Is there a way to remove old Order objects?
Can I use a reference so I just update the old order object and then push it again?
Is there another way??
I'm currently using Nesper
You could create a named window that holds unique events (older duplicates are evicted)
something like :
"create window OrderWin.std:unique(orderId) as Order"
"insert into "OrderWin select * from Order"
"select * from OrderWin where ..."
Another answer is to change the window. Keeping all events probably isn't what you wanted. Try using a different window, for example...
Let's say you only wanted the last trade per symbol. You could do the following:
select * from tradeEvent.std:unique(symbol)
This keeps only the last event for each event matching a given symbol.
I have a non real time Esper configuration where I feed a stream that I read from a file. I'm trying to create an expression that computes a statistic over the entire stream and outputs one value at the very end. Esper has semantics for forcing a view to output every X seconds, for instance, but is there a semantic for asking the view or the engine to "flush" the output when you know there are no more events to feed.
Turns out that at least one way to do this is to use the output clause with a variable trigger.
The expression would be:
select count(*) as totalCount from events output last when OutputSummary = true
The OutputSummary variable would be initialized like so:
epConfiguration.addVariable("OutputSummary", Boolean.class, "false");
When you're ready to flush, set the variable to true like so:
epRuntime.setVariableValue("OutputSummary", true);
long currentTime = epService.getEPRuntime().getCurrentTime();
epRuntime.sendEvent(new CurrentTimeEvent(currentTime));
It's necessary to send another time event to force the expression to evaluate.
When output requires at every 60 sec then the expression would be:
select emplyee_id from employees output snapshot every 60 sec
and when the output requires at every 10000 events then the expression would be:
select emplyee_id from employees output snapshot every 10000 events