This is my Event schema
EventA (FieldA string, FieldB string, FiledC string)
EventB (FieldD string, FieldE string)
I want the query like:
devide events into group window, group EventA by FieldA, FieldB ,group EventB by FieldE
If events occured in this order: EventA -> EventA -> EventA -> EventB -> EventB,and distinct( EventA.FieldC ) >= 3,and every EventB.FieldE equals every EventA.FieldB(join part),then generate a correlate event.
The problem is that:the count of EventA (3) is for the group window.For example:
["1","2","3"],["1","2","4"],["1","2","5"] can trigger "EventA -> EventA -> EventA"(3 events in one group window ),and ["1","2","3"],["1","3","4"],["1","4","5"] can not(3 group window , each has 1 event) and so as the 2 EventB.
so the complete example is:
["1","2","3"],["1","2","4"],["1","2","5"],["a","2"],["b","2"]
I find it's hard even in naturl language to descript the situation, and I have no idea how to descript this in CEP;
I use context in esper to solve problem that the count is for the group window not for the stream(in siddhi,partition).It is suit for the single EventA or EventB,but if EventA and EventB has to do a patten, the two independent context can not use together.And I tried Context Nesting like:
create context PartAB
context PartA partition by FieldA and FieldB from EventA,
context PartB partition by FieldE from EventB
and it seems doesn't work.
Thanks for anyone that can help this situation.
First question comes to mind is how are "FieldA, FieldB" and "FieldE" related? I.e. its is string-append for example where "FieldA || FieldB == FieldE" or what is the operation to compare these fields?
In Esper use a context like this one:
create context PartAB
partition by to_be_determined_A_B from EventA,
to_be_determined_E from EventB
This not-nested partitioned context makes sure events land in the same partition (nested context would create all possible combinations and that is not what you want).
You could compute the "to_be_determined_A_B" with an insert-into or by adding a new field to your event. Here is a sample insert-into:
insert into EventAWithFieldCombined select FieldA||FieldB combinedAB, * from EventA
... and the context could look like this...
create context PartAB
partition by combinedAB from EventAWithFieldCombined,
FieldE from EventB
Related
I'm new using Esper and I have been able to manage with it until now. There is something that I can't find solution and I have tried to find everywhere with no success.
I have three Classes extending one to other one.
Using Esper Online:
create schema A(symbol string, price double);
create schema B() inherits A;
create schema C() inherits B;
Now I want to count how many of each events in a time window of 2 seconds I have received, using the next EPL statement:
select COALESCE(a.symbol,b.symbol,c.symbol) as symbol, count(a) as total_a, count(b) as total_b, count(c) as total_c from pattern [every a=A or every b=B or every c=C]#time(2 seconds) group by COALESCE(a.symbol,b.symbol,c.symbol);
And I run the next events:
A={symbol='X', price=1}
B={symbol='X', price=1}
C={symbol='X', price=1}
C={symbol='X', price=1}
The problem is when I send a B event, it counts a B event and an A event due to inheritance , and obviously if I send a C event it counts a C a B and also an A event.
I have used the pattern-level annotation #SuppressOverlappingMatches it works with inheritances, but Time Windows doesn't work with it.
Try this.
select count(*, typeof(a)='A') as cnt_a, count(*, typeof(a)='B') as cnt_b,
count(*, typeof(a)='C') as cnt_c from A#time(2 sec) as a
I have a User node with two properties: partitaionA and partitionB which I generated from unionFind in graph algorithms package.
I want to do something like below but keep expanding like union operation.
partitionA -> partitionB -> partitionA -> partitionB -> ......... until the number of users in the group does not grow.
Match (u: User)
WHERE u.partitionA = 123
WITH collect(u.partitaionB) as next_level
MATH (uu: User)
WHERE uu.partitionB in next_level
RETURN uu
Is there any stored procedure can do something like this?
Not sure what is the best way to describe this question.
For the data model, I only have User node in this case and User node have 2 properties: partitionA and partitionB.
I have a quite simple problem to modelize and I don't have experience in Esper, so I may be headed the wrong way so I'd like some insight.
Here's the scenario: I have one stream of events "ParkingEvent", with two types of events "SpotTaken" and "SpotFree". So I have an Esper context both partitioned by id and bordered by a starting event of type "SpotTaken" and an end event of type "SpotFree". The idea is to monitor a parking spot with a sensor and then aggregate data to count the number of times the spot has been taken and also the time occupation.
That's it, no time window or whatsoever, so it seems quite simple but I struggle aggregating data. Here's the code I got so far:
create context ParkingSpotOccupation
context PartionBySource
partition by source from SmartParkingEvent,
context ContextBorders
initiated by SmartParkingEvent(
type = "SpotTaken") as startEvent
terminated by SmartParkingEvent(
type = "SpotFree") as endEvent;
#Name("measurement_occupation")
context ParkingSpotOccupation
insert into CreateMeasurement
select
e.source as source,
"ParkingSpotOccupation" as type,
{
"startDate", min(e.time),
"endDate", max(e.time),
"duration", dateDifferenceInSec(max(e.time), min(e.time))
} as fragments
from
SmartParkingEvent e
output
snapshot when terminated;
I got the same data for min and max so I'm guessing I'm doing somthing wrong.
When I'm using context.ContextBorders.startEvent.time and context.ContextBorders.endEvent.time instead of min and max, the measurement_occupation statement is not triggered.
Given that measurements have already been computed by the EPL that you provided, this counts the number of times the spot has been taken (and freed) and totals up the duration:
select source, count(*), sum(duration) from CreateMeasurement group by source
I am looking for an EPL statement which fires an event each time a certain value has increased by a specified amount, with any number of events in between, for example:
Considering a stream, which continuously provides new prices.
I want to get a notification, e.g. if the price is greater than the first price + 100. Something like
select * from pattern[a=StockTick -> every b=StockTick(b.price>=a.price+100)];
But how to realize that I get the next event(s), if the increase is >= 200, >=300 and so forth?
Diverse tests with context and windows has not been successful so far, so I appreciate any help! Thanks!
The contexts would be the right way to go.
You could start by defining a start event like this:
create schema StartEvent(threshold int);
And then have context that uses the start event:
create context ThresholdContext inititiated by StartEvent as se
terminated after 5 years
context ThresholdContext select * from pattern[a=StockTick -> every b=StockTick(b.price>=context.se.threshold)];
You can generate the StartEvent using "insert into" from the same pattern (probably want to remove the "every") or have the listener send in a StartEvent or declare another pattern that fires just once for creating a StartEvent.
I currently have an order object. We can assume it has three fields called orderId, state and price.
class Order
{
public int orderId;
public String state;
public int filled;
}
Through out the life time of the order the state and filled quantity will change. Each time there is a field change we push it to the esper runtime via:
Order o .....;
epService.EPRuntime.SendEvent(o);
Now each time the order is added via SendEvent its a different object than the previous order object( ie not a reference). This means the old order object should no longer be in stream for statements to see
I would like statements like the one below to only operate on the most recent version of the Order in the stream, ie conceptually there should only be one order object for each physical order in the stream.
"select filled from OrderStream.win:keepall() where orderId= 1234"
Is there a way to remove old Order objects?
Can I use a reference so I just update the old order object and then push it again?
Is there another way??
I'm currently using Nesper
You could create a named window that holds unique events (older duplicates are evicted)
something like :
"create window OrderWin.std:unique(orderId) as Order"
"insert into "OrderWin select * from Order"
"select * from OrderWin where ..."
Another answer is to change the window. Keeping all events probably isn't what you wanted. Try using a different window, for example...
Let's say you only wanted the last trade per symbol. You could do the following:
select * from tradeEvent.std:unique(symbol)
This keeps only the last event for each event matching a given symbol.