How can I remove elements from a stream - esper

I currently have an order object. We can assume it has three fields called orderId, state and price.
class Order
{
public int orderId;
public String state;
public int filled;
}
Through out the life time of the order the state and filled quantity will change. Each time there is a field change we push it to the esper runtime via:
Order o .....;
epService.EPRuntime.SendEvent(o);
Now each time the order is added via SendEvent its a different object than the previous order object( ie not a reference). This means the old order object should no longer be in stream for statements to see
I would like statements like the one below to only operate on the most recent version of the Order in the stream, ie conceptually there should only be one order object for each physical order in the stream.
"select filled from OrderStream.win:keepall() where orderId= 1234"
Is there a way to remove old Order objects?
Can I use a reference so I just update the old order object and then push it again?
Is there another way??
I'm currently using Nesper

You could create a named window that holds unique events (older duplicates are evicted)
something like :
"create window OrderWin.std:unique(orderId) as Order"
"insert into "OrderWin select * from Order"
"select * from OrderWin where ..."

Another answer is to change the window. Keeping all events probably isn't what you wanted. Try using a different window, for example...
Let's say you only wanted the last trade per symbol. You could do the following:
select * from tradeEvent.std:unique(symbol)
This keeps only the last event for each event matching a given symbol.

Related

Esper EPL statement each time a value has increased a multiple

I am looking for an EPL statement which fires an event each time a certain value has increased by a specified amount, with any number of events in between, for example:
Considering a stream, which continuously provides new prices.
I want to get a notification, e.g. if the price is greater than the first price + 100. Something like
select * from pattern[a=StockTick -> every b=StockTick(b.price>=a.price+100)];
But how to realize that I get the next event(s), if the increase is >= 200, >=300 and so forth?
Diverse tests with context and windows has not been successful so far, so I appreciate any help! Thanks!
The contexts would be the right way to go.
You could start by defining a start event like this:
create schema StartEvent(threshold int);
And then have context that uses the start event:
create context ThresholdContext inititiated by StartEvent as se
terminated after 5 years
context ThresholdContext select * from pattern[a=StockTick -> every b=StockTick(b.price>=context.se.threshold)];
You can generate the StartEvent using "insert into" from the same pattern (probably want to remove the "every") or have the listener send in a StartEvent or declare another pattern that fires just once for creating a StartEvent.

Recognize the type of the parameters of a user defined sql to be used in a Delphi TQuery at runtime

I'm writing a delphi(7 ver) application and in some place I want to execute parameterized queries (for BDE and Paradox) which will be loaded at runtime into a TQuery by the user. These queries will be stored in text files (one text file for one query). The application then, will construct for any parameter of the query, one input control (Tedit) in order to be able to accept values by the user. Also there will be a button for the execution of query. My question is how can I recognize the datatype of the query's parameter? Is there a way to get this type without of cause to be included in some way in the text file containing the query?
Create a second query from the first, but modify its where clause to ensure no rows.
SELECT * FROM MYTABLE WHERE PKFIELD IS NULL
Name your parameters so that you can establish their datatypes from the fieldtypes of this second query.
I realise this only works for relatively simple cases, but it should get you some of the way.
the advantage of using a parameter is that you don't need to know its data type.
Use the string value from the tedit
"select * from mytable where myfield = :param1"
"parambyname('param1').asstring := edit1.text"
I've made this with MySQL database. you must define some parameters, Exemple:
SELECT * FROM MyTable WHERE MyField=[ANNEE];
in this case, i have an other table, called balise, that look like this
"ID" "BALISE" "CAPTION" "DEFAULT_VALUE" "CNDT" "COMPOSANT"
"1" "ANNEE" "Année" "2014" "Properties.MaxValue=2014||Properties.MinValue=2007" 1;
in runtime, this mean that:
Make in my Panel, a TLablel that have caption Année
Make in the same line an other component type 1 (That mean in my case TcxSpinEdit), this component have défault value 2014, have Two properties Max Value=2014 and Min Value=2007, (I use RTTI to modifie this value of parameters, in Delphi ver7, use TypeInfo).
An Other Button with function called Actualise, this function have Original query, must browse an array of TBalise that i have created, take the value (In my case, take TcxSpinEdit(MyObject).Value), and replace it in the copy of my query (AnsiReplaceStr(Requete, '[ANNEE]', MyValue)), so i have the final query to execute it.
I have module in complete projet, worked with this methode, and it workk fine.

Filtering by aggregate function

I am trying to raise an event when the average value of a field is over a threshold for a minute. I have the object defined as:
class Heartbeat
{
public string Name;
public int Heartbeat;
}
My condition is defined as
select avg(Heartbeat) , Name
from Heartbeat.std:groupwin(Name).win:time(60 sec)
having avg(Heartbeat) > 100
However, the event never gets fired despite the fact that I fire a number of events with the Heartbeat value over 100. Any suggestions on what I have done wrong?
Thanks in advance
It confuses many people, but since time is the same for all groups you can simplify the query and remove the groupwin. The documentation note in this section explains why: http://esper.codehaus.org/esper-4.11.0/doc/reference/en-US/html_single/index.html#view-std-groupwin
The semantics with or without groupwin are the same.
I think you want group-by (and not groupwin) since group-by controls the aggregation level and groupwin controls the data window level.
New query:
select avg(Heartbeat) , Name from Heartbeat.win:time(60 sec) group by Name having avg(Heartbeat) > 100

Generating a deduplicated event stream without a window

I'm trying to generate a stream of deduplicated events without specifying any window policy beyond that used for the deduplication. Using an output first every clause on my queries appears to have the desired effect, but not when those queries are inserting directly into a stream.
For the example given below, say that I'm trying to detect only the first honk from each car in a 4-hour window.
(define-event-type! "CarEvent"
{:license_plate java.lang.String})
(define-event-type! "HonkEvent"
{:volume java.lang.Integer}
:supertypes #{"CarEvent"})
(define-variant! "HonkEventDeduplicated" "HonkEvent")
(define-statement! "context-IndividualCarContext"
"create context IndividualCarContext partition by license_plate from CarEvent")
(define-statement! "populate-HonkEventDeduplicated"
"context IndividualCarContext
insert into HonkEventDeduplicated
select * from HonkEvent
group by license_plate
output first every 4 hours")
However -- select * from HonkEventDeduplicated fires on every single honk event, even when the same car honks twice in a row.
Instead of using output first every clause filtering, this can be done with the std:firstunique view:
(define-statement!
"populate-HonkEventDeduplicated"
"insert into HonkEventDeduplicated
select * from HonkEvent.win:time(4 hours).std:firstunique(license_plate)")

Select certain number of records for batch processing

Hi is it possible using Entity Framework and/or linq to select a certain number of rows? For example i want to select rows 0 - 500000 and assign these records to the List VariableAList object, then select rows 500001 - 1000000 and assign this to the List VariableBList object, etc. etc.
Where the Numbers object is like ID,Number,DateCreated, DateAssigned, etc.
Sounds like you're looking for the .Take(int) and .Skip(int) methods
using (YourEntities db = new YourEntities())
{
var VariableAList = db.Numbers
.Take(500000);
var VariableBList = db.Numbers
.Skip(500000)
.Take(500000);
}
You may want to be wary of the size of these lists in memory.
Note: You also may need an .OrderBy clause prior to using .Skip or .Take--I vaguely remember running into this problem in the past.

Resources