I have a non real time Esper configuration where I feed a stream that I read from a file. I'm trying to create an expression that computes a statistic over the entire stream and outputs one value at the very end. Esper has semantics for forcing a view to output every X seconds, for instance, but is there a semantic for asking the view or the engine to "flush" the output when you know there are no more events to feed.
Turns out that at least one way to do this is to use the output clause with a variable trigger.
The expression would be:
select count(*) as totalCount from events output last when OutputSummary = true
The OutputSummary variable would be initialized like so:
epConfiguration.addVariable("OutputSummary", Boolean.class, "false");
When you're ready to flush, set the variable to true like so:
epRuntime.setVariableValue("OutputSummary", true);
long currentTime = epService.getEPRuntime().getCurrentTime();
epRuntime.sendEvent(new CurrentTimeEvent(currentTime));
It's necessary to send another time event to force the expression to evaluate.
When output requires at every 60 sec then the expression would be:
select emplyee_id from employees output snapshot every 60 sec
and when the output requires at every 10000 events then the expression would be:
select emplyee_id from employees output snapshot every 10000 events
Related
I am trying to find the timestamp of the first occurrence of max value(the cursor point in below image)
I wrote query
min(timestamp(max(jmeter_count_total{label="GET - Company Updates - ua_users_company-updates"})))
But it's returning the max timestamp of the max value
I am not able to grab the value highlighted by cursor in below image(minimum value). Instead I am getting highest value when I use above query.
I've played with this for a bit and I think this may work (take it with a grain of salt, due to limited testing).
TLDR - the query (using only foo for brevity):
min_over_time((timestamp(foo) and (foo == scalar(max_over_time(foo[2h]))))[1h:])
This portion of the query:
foo == scalar(max_over_time(foo[2h]))
returns only values where foo matches the max value of foo in the last 2h interval. For retrieving the timestamp of those cases we use the timestamp function and use these previous clause as a conditional:
timestamp(foo) and (foo == scalar(max_over_time(foo[2h])))
Finally we only want to get the first/lowest timestamp value over the time window, which is what the outer min_over_time with the nested subquery should do.
I fiddled with the online Prometheus demo using one of the metrics present there. You can check the queries here.
Here are two events:AppStartEvent and AppCrashEvent.
I need to count the number of two events over a period of time, and then to calculate the count(AppStartEvent)/count(AppCrashEvent).
My EPL is here
create context ctx4NestInCR
context ctx4Time initiated #now and pattern [every timer:interval(1 minute)] terminated after 15 minutes,
context ctx4AppName partition by appName from AppStartEvent, appName from AppCrashEvent
<------------------->
context ctx4NestInCR select count(s),count(c) from AppStartEvent as s, AppCrashEvent as c output last when terminated
And it does not work
Error starting statement: Joins require that at least one view is specified for each stream, no view was specified for s
Your post doesn't have the join? It only has the context and that wouldn't produce the message. I would suggest to correct the post.
You can also join streams by merging the two streams and treating them as one.
insert into AppEvent select 'crash' as condition from AppCrashEvent;
insert into AppEvent select 'start' as condition from AppStartEvent;
select count(condition='crash')/count(condition='start') from AppEvent;
I have a quite simple problem to modelize and I don't have experience in Esper, so I may be headed the wrong way so I'd like some insight.
Here's the scenario: I have one stream of events "ParkingEvent", with two types of events "SpotTaken" and "SpotFree". So I have an Esper context both partitioned by id and bordered by a starting event of type "SpotTaken" and an end event of type "SpotFree". The idea is to monitor a parking spot with a sensor and then aggregate data to count the number of times the spot has been taken and also the time occupation.
That's it, no time window or whatsoever, so it seems quite simple but I struggle aggregating data. Here's the code I got so far:
create context ParkingSpotOccupation
context PartionBySource
partition by source from SmartParkingEvent,
context ContextBorders
initiated by SmartParkingEvent(
type = "SpotTaken") as startEvent
terminated by SmartParkingEvent(
type = "SpotFree") as endEvent;
#Name("measurement_occupation")
context ParkingSpotOccupation
insert into CreateMeasurement
select
e.source as source,
"ParkingSpotOccupation" as type,
{
"startDate", min(e.time),
"endDate", max(e.time),
"duration", dateDifferenceInSec(max(e.time), min(e.time))
} as fragments
from
SmartParkingEvent e
output
snapshot when terminated;
I got the same data for min and max so I'm guessing I'm doing somthing wrong.
When I'm using context.ContextBorders.startEvent.time and context.ContextBorders.endEvent.time instead of min and max, the measurement_occupation statement is not triggered.
Given that measurements have already been computed by the EPL that you provided, this counts the number of times the spot has been taken (and freed) and totals up the duration:
select source, count(*), sum(duration) from CreateMeasurement group by source
I am looking for an EPL statement which fires an event each time a certain value has increased by a specified amount, with any number of events in between, for example:
Considering a stream, which continuously provides new prices.
I want to get a notification, e.g. if the price is greater than the first price + 100. Something like
select * from pattern[a=StockTick -> every b=StockTick(b.price>=a.price+100)];
But how to realize that I get the next event(s), if the increase is >= 200, >=300 and so forth?
Diverse tests with context and windows has not been successful so far, so I appreciate any help! Thanks!
The contexts would be the right way to go.
You could start by defining a start event like this:
create schema StartEvent(threshold int);
And then have context that uses the start event:
create context ThresholdContext inititiated by StartEvent as se
terminated after 5 years
context ThresholdContext select * from pattern[a=StockTick -> every b=StockTick(b.price>=context.se.threshold)];
You can generate the StartEvent using "insert into" from the same pattern (probably want to remove the "every") or have the listener send in a StartEvent or declare another pattern that fires just once for creating a StartEvent.
I'm trying to generate a stream of deduplicated events without specifying any window policy beyond that used for the deduplication. Using an output first every clause on my queries appears to have the desired effect, but not when those queries are inserting directly into a stream.
For the example given below, say that I'm trying to detect only the first honk from each car in a 4-hour window.
(define-event-type! "CarEvent"
{:license_plate java.lang.String})
(define-event-type! "HonkEvent"
{:volume java.lang.Integer}
:supertypes #{"CarEvent"})
(define-variant! "HonkEventDeduplicated" "HonkEvent")
(define-statement! "context-IndividualCarContext"
"create context IndividualCarContext partition by license_plate from CarEvent")
(define-statement! "populate-HonkEventDeduplicated"
"context IndividualCarContext
insert into HonkEventDeduplicated
select * from HonkEvent
group by license_plate
output first every 4 hours")
However -- select * from HonkEventDeduplicated fires on every single honk event, even when the same car honks twice in a row.
Instead of using output first every clause filtering, this can be done with the std:firstunique view:
(define-statement!
"populate-HonkEventDeduplicated"
"insert into HonkEventDeduplicated
select * from HonkEvent.win:time(4 hours).std:firstunique(license_plate)")