I have two write points for InfluxDB, one is the start and the other is the end. I just need to determine the duration between those two events, and make queries around it. InfluxDB has difference() aggregate method, but it doesn't work on the time meta field.
Is supplying a custom timestamp value the only way to accomplish this?
As per "Can I perform mathematical operations against timestamps?"
No:
"Currently, it is not possible to execute mathematical operators against timestamp values in InfluxDB. Most time calculations must be carried out by the client receiving the query results."
and yes, maybe:
The function ELAPSED() returns the difference between subsequent timestamps in a single field.
So it depends on the shape of your data.
If you write only the mentioned two entries then you can follow the below steps -
Limit the result to two (Eg: select * from timeseries limit 2)
Extract the time from the result set
Take the difference between the time
Related
We are measuring throughput using Grafana and Influx. Of course, we would like to measure throughput in terms how many requests, approximately, happens every single second (rps).
The typical request is:
SELECT sum("count") / 10 FROM "http_requests" GROUP BY time(10s)
But we are loosing possibility to use astonishing dynamic $__interval that very useful when graph scope is large, like a day of week. When we are changing interval we should change divider into SELECT expression.
SELECT sum("count") / $__interval FROM "http_requests" GROUP BY time($__interval)
But this approach does not work, because of empty result returns.
How to create request using dynamic $__interval for throughput measuring?
The reason you get no results is that $__interval is not a number but a string such as 10s, 1m, etc. that is understood by influxdb as a time range. So it is not possible to use it the way you are trying.
However, what you want to calculate is the mean which is available as a function in InfluxQL. The way to get the behavior that you want is with something like this.
SELECT mean("count") FROM "http_requests" GROUP BY time($__interval)
EDIT: On a second thought that is not quite what you want.
You'd probably need to use derivative. I'll come back to you on that one later.
Edit2: Do you think this answers the question that you have Calculating request per second using InfluxDB on Grafana
Edit3: Third edit's a charm.
We use your starting query and wrap it in another one as such:
SELECT sum("rps") from (SELECT sum("count") / 10 as rps FROM "http_requests" GROUP BY time(10s)) GROUP BY time($__interval)
I have a Prometheus metric called device_number. What I want is to show the difference in value between now and one day/week/month etc ago. Which means subtracting two values with two different timestamps.
Checking around I don't find any useful documentation on how to do it.
Something I would do, but doesn't work is:
sum(device_number) - sum(device_number[$__range])
I found offset is the correct keyword.
Query like this:
sum(vss_device_number) - sum(vss_device_number offset 1d)
Will return difference between now and yesterday.
Docs.
PromQL also provides delta() function, which can be used for returning the delta between the current time and the time specified in square brackets passed to this function. For example, the following query should return the delta for vss_device_number over the last day (see [1d]):
delta(vss_device_number[1d])
The query returns deltas per each matching time series. If you need summary delta across all the matching time series, then wrap the query into sum():
sum(delta(vss_device_number[1d]))
I'm sending metrics in StatsD format to Telegraf, which forwards them to InfluxDB 0.9.
I'm measuring execution times (of some event) from multiple hosts. The measurement is called "execTime", and the tag is "host". Once Telegraf gets these numbers, it calculates mean/upper/lower/count, and stores them in separate measurements.
Sample data looks like this in influxdb:
TIME...FIELD..............HOST..........VALUE
t1.....execTime.count.....VM1...........3
t1.....execTime.mean......VM1...........15
t1.....execTime.count.....VM2...........6
t1.....execTime.mean......VM2...........22
(So at time t1, there were 3 events on VM1, with mean execution time 15ms, and on VM2 there were 6 events, and the mean execution time was 22ms)
Now I want to calculate the mean of the operation execution time across both hosts at time t1. Which is (3*15 + 6*22)/(3+6) ms.
But since the count and mean values are in two different series, I can't simply use "select mean(value) from execTime.mean"
Do I need to change my schema, or can I do this with the current setup?
What I need is essentially a new series, which is a combination of the execTime.count and execTime.mean across all hosts. Instead of calculating this on-the-fly, the best approach seems to be to actually create the series along with the others.
So now I have two timer stats being generated on each host for each event:
1. one event with actual hostname for the 'host' tag
2. second event with one tag "host=all"
I can use the first set of series to check mean execution times per host. And the second series gives me the mean time for all hosts combined.
It is possible to do mathematical operations on fields from two different series, provided both series are members of the same measurement. I suspect your schema is non-optimized for your use case.
Is it possible to aggregate measurements or create custom queries beyond the standard dateFrom dateTo queries?
As an example, I have measurements which have a time delta of 1 minute (2015-01-01T05:05:00, 2015-01-01T05:05:00, 2015-01-01T05:05:00, ...) and I would like to query the measurements at 15 minute intervals (2015-01-01T05:15:00, 2015-01-01T05:30:00, 2015-01-01T05:45:00, ...)
So far I have only come up with these solutions:
Using the standard api request as in
https://tenant.cumulocity.com/measurement/measurements?dateFrom=2015-10-01&dateTo=2015-11-05
and then throwing away most of the data will use a massive amount of time loading the data.
Using cep (cumulocity event language) to generate a new measurement every 15 minutes using the nearest 1 minute measurement seems like a bit of overkill and not very elegant.
Batch requesting the exact minute
https://tenant.cumulocity.com/measurement/measurements?dateFrom=2015-11-05T05:15:00%2B01:00&dateTo=2015-11-05T05:16:00%2B01:00
which will in a massive amount of API requests and also does not seem very efficient.
Use the /measurements/series endpoint which will only give me all series, even those I do not want, as well as only having the aggregation options hourly and daily (as far as I can tell).
Is there a better way of doing this?
you have captured nearly all of the mechanisms that are currently available. There is one more possibility -- not sure if this is an option for you:
Mark the fifteenth measurement when sending it from the device, using e.g. a different type.
I would normally use 2. It's actually quite efficient, it's similar to a materialized view in traditional SQL, plus you can use the data everywhere and in all widgets.
Good luck :-)
Cheers,
André
I would prefer the CEP solution. The rule wouldn't be that complicated. You would of course then store these measurements twice which is not that nice but having your desired measurement with a specific type or fragment will give you the fastest way to query it.
Instead of copying the measurement you could just add a special fragment to the measurement every 15 min in the CEP rule. You cannot update measurements so you would have to delete the measurement incoming every 15 min and then create a new measurement with exactly the same values but add a fragement (e.g. "aggregatedMeasurement": {}).
Your query then looks like this:
https://tenant.cumulocity.com/measurement/measurements?dateFrom=2015-10-01&dateTo=2015-11-05&fragmentType=aggregatedMeasurement
One more idea for point 3:
You could use SmartREST to create a template with the query string and leave the dateFrom and dateTo as placeholders.
From the client side you then would have to make only one request using the bulking feature in SmartREST.
On the server side this would still be transformed into the single requests so you wouldn't gain anything in speed.
I was reading about Time-Based Versioned Graphs and came across the following example:
CREATE (s1:Shop{shop_id:1})
-[:STATE{from:1388534400000,to:9223372036854775807}]->
(ss1:ShopState{name:'General Store'})
My question: how do I calculate this date? from:1388534400000,to:9223372036854775807
Those two values are timestamps which in java are the number of milliseconds since the Epoch (1/1/1970) began. The second value is the maximum Long value, the end of Java time, a long way away.
There are ways in all languages for generating these values for specific dates (beware that some will be based on seconds), there is quite a handy list on this site.
If you are not working in any particular programming language and just want to enter queries then you can use an online date converter like this one.
You can also calculate timestamps in Cypher if you are working with dates that relate to Now somehow using the timestamp() function:
CREATE (s1:Shop{shop_id:1})
-[:STATE{from:timestamp(),to:9223372036854775807}]->
(ss1:ShopState{name:'General Store'})
IIUC to is just a Long.MAX_VALUE, and from can be a result of either calling timestamp() function via Cypher or setting the property with the value of System.currentTimeMills() via Java API.
Take a look at the example: http://console.neo4j.org/?id=43uoyt (Note that you can skip setting rel.to and use coalesce when querying instead).