I am using grafana to display metrics based on prometheus data source. When using single stat panel with delta configuration I am getting wrong values. The values on prometheus are stored correctly, it looks something wrong in grafana when query with date filter, it show a lot of non sense results. Has something similar happened to someone?
Related
Grafana does not display any data ( failed to fetch ) after 60s for large datasets but when the interval is smaller dashboard loads fine any help here?
Tried tweaking timeouts in grafana.ini does not seem to help here looks like Grafana has a hard - limit on those parameters
Grafana version > 7.0.3
Data source : influxdb
dashboard loads fine for smaller intervals
any help here would be appreciated here?
Use time groupping GROUP BY time($__interval) in your InfluxDB query - https://grafana.com/docs/grafana/latest/datasources/influxdb/#query-editor - Grafana already has macro $__interval which will select "optimal" time aggregation based on current dashboard time range.
It doesn't make sense to load huge datasets with original granularity. You may solve it on the Grafana level somehow, but then you may have a problem in the browser - it may not have so much memory or it will take ages.
I'm new to Prometheus but familiar with Influx (currently running 1.6).
My understanding is it's possible to configure Prometheus to remotely read data from influx with the following configuration in prometheus.yml:
remote_read:
url: "http://localhost:8086/api/v1/prom/read?db=bulkstats"
"bulkstats" is the database I'm trying to read data from in Prometheus. An example query that would work in influx would be:
SELECT "sess-curaaaactive" FROM "PDSNSYSTEM1" WHERE ("Nodename" = 'ALPRGAGQPNC') AND time >= now() - 6h"
However I cannot find one example of how to query that data from PromQL. Please help!
Here is the link which matches prometheus format with influxdb's one.
In terms of prometheus's jargon, in your example, sess-curaaaactive is the metric name (measurement in influx) and ("Nodename" = 'ALPRGAGQPNC') is just a label which prometheus attaches to the measurement to create a time series.
https://github.com/influxdata/telegraf/pull/1557
Apparently some people have been asking for this, and this Github PR is the closest thing I can find to a solution, but it was ultimately denied(I think?).
Basically, I have a JSON object I'm getting from Stackdriver, which includes a Timestamp in ISO8601, which I convert to Unix time. I can insert the entire JSON response into Influx fine, but the timestamp from Stackdriver appears as a tag for a series, rather than the index of the time series itself. As a result, it is unfeasible to query by Stackdriver's provided timestamp. I could simply just drop it, and use the Influx provided timestamp, but it is essentially querying incorrect/imprecise data.
Does anyone have a clever way to approach this?
tl;dr How can I use Telegraf to override InfluxDB's timestamps with my own timestamps?
I'm using Telegraf/InfluxDB/Grafana to register and view metrics for my servers. Occasionally one of these components crash and metrics stop flowing into InfluxDB.
To be able to notice when this happens (on top of using Monit to restart the service) I would like to create a Grafana dashboard where I have a singlestat panel for each host that shows the most recent timestamp (or better, how much time has passed) since the last metric was received. I'd also like to colorize the background of the singlestat depending on how long it's been. I would like to be able to do this for any InfluxDB metric, as different metrics can have different reasons for lagging behind.
Right now, I've tried something like this in InfluxQL, but I just get an error that at least one non-time field must be present in the query:
SELECT last(time) FROM "system" WHERE "load1" > -1 GROUP BY "host"
If I try to change it to this I get a "Multiple series error":
SELECT last(time), last("load1") FROM "system" GROUP BY "host"
Is what I'm trying to do not easily doable or am I missing something obvious?
...the query syntax was taken from somewhere in this issue.
I haven't yet looked at the singlestat panel, but I suggest creating a 'scripted dashboard'.
Start by manually creating a singlestat dash with a dummy number. Then export it to see what the json looks like.
Then, recreate that same json, with the live result from the above query, using a scripted dashboard.
Look in grafana's /public/dashboards for 'scripted.js' or 'scripted_templated.js'. Make a copy of one of those in the same folder, then hack away to generate your json. Here and here and here are some nice examples for the javascript magic to submit the query to influxdb and parse the result into the json for your singlestat dash. Good luck.
I am trying to write an influxdb query such that a measurement looks like
dummydata value=1,2,3,4
and influxdb doesn't like this format. I'm guessing influxdb cannot do this, but I can't find any documentation that says it cannot, nor do I see a feasible workaround. I have to write 500 points per timestamp: it seems to me that 500 separate measurements per timestamp would get hairy quick.
So:
Can influxdb accept an array/list as a value
If not, is there a workaround
Or is influxdb just the wrong tool for this job?
Thanks in advance.
InfluxDB accepts strings, float64s, int64s, and booleans as field values.
it seems to me that 500 separate measurements per timestamp would get hairy quick.
That's where you are mistaken. InfluxDB 0.10+ is specifically designed to encourage multiple fields per point, where a field is a measured value. What you want to write is a point like this:
dummydata value=1,value2=2,value3=3,value4=4...