I'll first try to describe the non-technical problem:
There are many services, each service can have multiple instances and each of those instances is scraped for data and that data is stored in influxdb. Now the point in time where that data is scraped from each instance of the service is (obviously) not exactly the same point in time.
What i like to query (or display) is the maximum value for each service, over all instances. And i did not find a way to "quantify" time points. For examle to always move the value to the next full minute or something so that timescales get comparable.
Not the technical problem is that the reported values are all totals so to get a sense of change in that values i need difference oder derivative but these functions are in my case often aplied to one value of instance 1 and one value of instance 2 which reflects the difference between the instances and not between two points in time.
heres what i tried so far but i gives me almost flatlines since the instances report pretty much the same value but at alternating points in time.
from(bucket: "dpl4")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r._measurement == "prometheus")
|> filter(fn: (r) => r._field == "http_server_requests_seconds_sum")
|> group(columns: ["service"], mode: "by")
|> aggregateWindow(every: 1m, fn: max)
|> derivative(unit: 1m, nonNegative: true)
I hope i was able to describe the problem.
I've got an InfluxDB database with measurements of Grid power usage. The Grid power is negative when our Solar PV is not enough to power up the house and we are importing from the grid. Likewise it's the measurement is positive when we've got surplus Solar PV power and we are exporting to the grid.
Now I would like to calculate (perhaps using integral()) the cost of power exported, separately from the cost of power imported. Because there are different rates I can't simply integrate the it all together, I need the above zero and below zero considered separately to calculate the energy in kWh and subsequently the cost in each direction.
I was hoping to use InfluxDB min() and max() but that seem to select the min/max value from a given interval, not quite what I need I think.
Can I somehow split this measurement into two for further calculations?
I'm on InfluxDB 1.8 but considering an upgrade to 2.x eventually.
Since you are going to upgrade to 2.x, you could try Flux.
In v1.8, you can turn on Flux following this doc.
Use the filter operator to filter the positive and negative values, then apply the integral function.
Power Surplus:
from(bucket: "yourDatabaseName/autogen")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "yourMeasurementName" and r._field == "yourFieldName")
|> filter(fn: (r) => r._value > 0)
|> integral(unit: 10s)
Power Deficit:
from(bucket: "yourDatabaseName/autogen")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "yourMeasurementName" and r._field == "yourFieldName")
|> filter(fn: (r) => r._value < 0)
|> integral(unit: 10s)
from(bucket: "metrics")
|> range(start: -5m)
|> filter(fn: (r) => r["_measurement"] == "cpu")
|> filter(fn: (r) => r["_field"] == "usage")
|> last()
Running this query will return the data only if it was saved in the last 5 minutes.
What I am looking for is, if there is not data for the time range provided, then get the latest data (could be 10m old or 5d old). I know that prometheus does return last data and we are trying to move from prometheus to influxDB and are stuck with this problem.
Also, just increasing the range to say -10d would not work because the volume of data is very high (hundreds of record per second are being written).
We are experimenting with down sampling as well to see if that will work for us, but wanted to know if there was a way to get it from the source bucket itself.
I am recording period between application heartbeats into Influxdb.
The "target" period is 2000ms.
If the period is above 2750ms, then it is defined as a "lag event".
My end objective is to run statistics on "how long" we are running without lag events.
I switched to Flux from Influxql, so that i could use the stateDuration() method.
Using the below method, i am able to collect the increasing durations. At lag events, the state_duration is then reset to -1.
from (bucket: "sampledb/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "timers" and
r._field == "HeartbeatMs" and
r.character == "Tarek"
)
|> stateDuration(fn: (r) =>
r._value<=2750,
column: "state_duration",
unit: 1s
)
|> keep(columns: ["_time","state_duration"])
At this point, I would like to be able to collect 'max(state_duration)' for each duration between lag events, and this is where i get stuck. Trying to "group by every new stateDuration sequence"/"group by increasing stateDurations"...
I was thinking that it might be possible to use "reduce()" or "map()" to inject a sequence number that i can use to group by, somehow increasing that sequence number whenever i have a -1 in the state_duration.
Below is a graph of the "state_duration" when running the flux query, i am basically trying to capture the value at the top of each peak.
Any help is appreciated, including doing this e.g. in InfluxQL or with Continuous Queries.
Data looks like below when exported to csv:
"time","timers.HeartbeatMs","timers.character"
"2021-01-12T14:49:34.000+01:00","2717","Tarek"
"2021-01-12T14:49:36.000+01:00","1282","Tarek"
"2021-01-12T14:49:38.000+01:00","2015","Tarek"
"2021-01-12T14:49:40.000+01:00","1984","Tarek"
"2021-01-12T14:49:42.000+01:00","2140","Tarek"
"2021-01-12T14:49:44.000+01:00","1937","Tarek"
"2021-01-12T14:49:46.000+01:00","2405","Tarek"
"2021-01-12T14:49:48.000+01:00","2312","Tarek"
"2021-01-12T14:49:50.000+01:00","1453","Tarek"
"2021-01-12T14:49:52.000+01:00","1890","Tarek"
"2021-01-12T14:49:54.000+01:00","2077","Tarek"
"2021-01-12T14:49:56.000+01:00","2250","Tarek"
"2021-01-12T14:49:59.000+01:00","2360","Tarek"
"2021-01-12T14:50:00.000+01:00","1453","Tarek"
"2021-01-12T14:50:02.000+01:00","1952","Tarek"
"2021-01-12T14:50:04.000+01:00","2108","Tarek"
"2021-01-12T14:50:06.000+01:00","2485","Tarek"
"2021-01-12T14:50:08.000+01:00","1437","Tarek"
"2021-01-12T14:50:10.000+01:00","2421","Tarek"
"2021-01-12T14:50:12.000+01:00","1483","Tarek"
"2021-01-12T14:50:14.000+01:00","2344","Tarek"
"2021-01-12T14:50:17.000+01:00","2437","Tarek"
"2021-01-12T14:50:18.000+01:00","1092","Tarek"
"2021-01-12T14:50:20.000+01:00","1969","Tarek"
"2021-01-12T14:50:22.000+01:00","2359","Tarek"
"2021-01-12T14:50:24.000+01:00","2140","Tarek"
"2021-01-12T14:50:27.000+01:00","2421","Tarek"
There are two ways I can think of. One is to look for the inverted state. The other is to use elapsed() to find interval + timeShift() to emulate LAG().
I don't like the latter though I think the first is not intuitive neither :-(. Really hope features like LAG() or CurrentRecordIndex() would be available in Flux.
from (bucket: "sampledb/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "timers" and
r._field == "HeartbeatMs" and
r.character == "Tarek"
)
|> stateDuration(fn: (r) =>
r._value>2750, // Look for the inverted state
column: "inverted_state_duration",
unit: 1s
)
|> keep(columns: ["_time","inverted_state_duration"])
// Clear out records of the periods you are after
|> filter(fn: (r) => r["inverted_state_duration"] == -1)
// Calculate the gap duration with elapsed()
|> elapsed(columnName: "state_duration")
|> filter(fn: (r) => r["state_duration"] > ${ max(stateDuration.unit, record interval) })
So I'm trying to find any documentation on more complex Flux queries but after days of searching I'm still lost. I want to be able to calculate average values for each hour of the week and then when new data comes in I want to check if it deviates by x standard deviations for that hour.
Basically I want to have 24x7 array fields each representing the mean/median value for each hour of the week for the last 1 year. Then I want to compare last days values for each hour against these averages and report an error. I do not understand how to calculate these averages. Is there some hidden extensive documentation on Flux?
I don't really need a full solution, just some direction would be nice. Like, are there some utility functions for this in the standard lib or whatever
EDIT: After some reading, it really looks like all I need to do is use the window and aggregateWindow functions but I haven't yet found how exactly
Ok, so, this is what worked for me. Needs some cleaning up but gets the values successfully grouped per hour+weekday and the mean of all the values
import "date"
tab1 = from(bucket: "qweqwe")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r["_measurement"] == "asdasd")
|> filter(fn: (r) => r["_field"] == "reach")
|> aggregateWindow(every: 1h, fn: mean, createEmpty: false)
mapped = tab1
|> map(fn: (r) => ({ r with wd: string(v: date.weekDay(t: r._time)), h: string(v: date.hour(t: r._time)) }))
|> map(fn: (r) => ({ r with mapped_time: r.wd + " " + r.h }))
grouped = mapped
|> group(columns: ["mapped_time"], mode: "by")
|> mean()
|> group()
|> toInt()
|> yield()