I would like to write points into an influx 0.8 database with the time values given in seconds through HTTP. Here's a sample point in JSON format:
[
{
"points": [
[
1435692857.0,
897
]
],
"name": "some_series",
"columns": [
"time",
"value"
]
}
]
The documentation is unclear what the format of time values should be (nano or milli seconds?) and how to specify to influxdb what to expect. Currently I'm using a query parameter: precision=s
That seems to work fine, the server returns HTTP Status code 200 as expected. When querying against the database using influx' admin interface using select * from some_series the datapoints in the table are returned with the expected timestamp. On the graph however, the time axis is indexed with fractions of seconds and queries like select * from some_series where time > now() - 1h dont yield any results.
I assume that there is something wrong with the timestamps. I tried multiplying my value by 1000 but then nothing gets inserted into the database with no visible errors.
Whats the problem?
By default, supplied timestamps are assumed to be in milliseconds. I think your writes are defaulting to milliseconds because the query string parameter should be time_precision=s, not precision=s.
See the details under "Time Precision on Written Data" on https://influxdb.com/docs/v0.8/api/reading_and_writing_data.html.
I also think the time value should be an integer rather than a float. I'm not sure how to explain the other behaviors, where the timestamp seems to be the right date and multiplying by 1000 doesn't solve the issue, but I wonder if it's related to writing floats.
Please contact the InfluxDB support team at support#influxdb.com for further assistance.
I found the solution! The problem was only in part with the precision. Your answer was correct, the query parameter is called time_precision and I should post integers instead of floats. Which was probably the first thing I attempted with no results...
However, due to some time zone problems, my time values where in the future relative to server time and by default, any select statement includes a where time < now() statement. So, in fact values were written into the database, but not displayed because of that hidden where statement. The solution was to tell the database to return "future" values, too:
select value from some_series where time < now() + 1h
Related
Using InfluxDB: Is there any way to build a time-bucketed report of a field value representing a state that persists over time? Ideally in InfluxQL query language
More specifically as an example: Say a measurement contains points that report changes in the light bulb state (On / Off). They could be 0s and 1s as in the example below, or any other value. For example:
time light
---- -----
2022-03-18T00:00:00Z 1
2022-03-18T01:05:00Z 0
2022-03-18T01:55:00Z 0
2022-03-18T02:30:00Z 1
2022-03-18T04:06:00Z 0
The result should be a listing of intervals indicating if this light was on or off during each time interval (e.g. hours), or what percentage of that time it was on. For the given example, the result if grouping hourly should be:
Hour
Value
2022-03-18 00:00
1.00
2022-03-18 01:00
0.17
2022-03-18 02:00
0.50
2022-03-18 03:00
1.00
2022-03-18 04:00
0.10
Note that:
for 1am bucket, even if the light starts and ends in On state, it was On for only 10 over 60 minutes, so the value is low (10/60)
and more importantly the bucket from 3am to 4am has value "1" as the light was On since the last period, even if there was no change in this time period. This rules out usage of simple aggregation (e.g. MEAN) over a GROUP BY TIME(), as there would not be any way to know if an empty/missing bucket corresponds to an On or Off state as it only depends on the last reported value before that time bucket.
Is there a way to implement it in pure InfluxQL, without retrieving potentially big data sets (points) and iterating through them in a client?
I consider that raw data could be obtained by query:
SELECT "light" FROM "test3" WHERE $timeFilter
Where "test3" is your measurement name and $timeFilter is from... to... time period.
In this case we need to use a subquery which will fill our data, let's consider grouping (resolution) time as 1s:
SELECT last("light") as "filled_light" FROM "test3" WHERE $timeFilter GROUP BY time(1s) fill(previous)
This query gives us 1/0 value every 1s. We will use it as a subquery.
NOTE: You should be informed that this way does not consider if beginning of data period within $timeFilter has been started with light on or off. This way will not provide any data before hour with any value within $timeFilter.
In next step you should use integral() function on data you got from subquery, like this:
SELECT integral("filled_light",1h) from (SELECT last("light") as "filled_light" FROM "test3" WHERE $timeFilter GROUP BY time(1s) fill(previous)) group by time(1h)
This is how it looks on charts:
And how Result data looks in a table:
This is not a perfect way of getting it to work but I hope it resolves your problem.
First time posting.
Our fire response agency just moved to Cognos 11 after qRep. I know little to nothing about SQL, modules, cubes, or data analytics, but I'm one of the few in our agency that can write an excel function so I guess I am considered the in-house data expert.
With that in mind, I need some help.
I want to compare a calculated result (time interval) to a static goal. For instance, if the calculated result is 01:00, how do I express a static goal of 01:20 as a time interval to make the calculation work?
Here is the formula I used for my "Static Goal" column:
IF([Inc Cat]='EMS') then ('60')) else
IF([Inc Cat]='FIRE') then ('80')) else
NULL
I need to convert the 60 and 80 above as seconds so I can compare a calculated time interval to these values.
I appreciate any help, but please review the second sentence of this post prior to replying.
Solved. I removed the single quotes and wrote out the complete time.
IF([Inc Cat]='EMS') then (0 00:01:10.000) else
IF([Inc Cat]='FIRE') then (0 00:01:20.000) else
NULL
my table has 3 columns: data type timestamp,
|created_At | final_time| duracion(difference between created at and final_time)
| | |
the column difference should save the difference in hours and minutes, in this format HH:MM
this is my controller:
def horario
horario.update(duracion: params[:duracion]) // this params is "00:59"
end
but in the table Horarios, in column duracion i have this:
2017-12-24 03:59:00
so i want to save 00:59 (59 minutes) but postgres save all current date and add 3 hours more.
i want to save so in the future i will be able tu sum column duracion. Or should i change data type for this column? In this case which datatype you recomend me for rails to save HH:MM??
thanks.
Rails 5 supports PostgreSQL's interval type to some extent. You can create interval columns in the usual way and they will be properly represented in db/schema.rb. You can also assign them values in the usual way so you can say things like:
model.some_interval = '6 hours'
and get 06:00:00 inside the database. However, there is nothing in Ruby or Rails that properly represents a time interval (we only have various timestamp and date classes) so when that interval comes out of the database, you'll have a string on your hands, i.e:
> model = Model.find(some_id)
> model.some_interval.class
=> String
so you might end up having to manually parse some strings in Ruby. For simple intervals like '6 hours', this will be easy but it won't be so easy with more complicated intervals like '6 years 23 days 11 hours'.
If you'll only be working with your time intervals inside the database then interval would be natural and easy, you can say things like:
select some_timestamp + some_interval
and
where some_timestamp + some_interval < some_other_timestamp
and everything will work nicely.
However, if you need to work with the intervals back in Ruby then you'd probably be better off storing the interval as a number of seconds in an integer column (or whatever resolution you need). Then you could say things like:
where some_timestamp + (some_interval_in_seconds || 'seconds')::interval < some_other_timestamp
inside the database and
some_time + model.some_interval_in_seconds
back in Ruby.
In any case, strings are probably the wrong approach unless you really like parsing strings everywhere all the time.
As others already pointed out, Rails handles the Postgres Interval type as a string. A string that, unfortunately, is not easy to parse.
If you do this:
u = Users.select('edited_at - created_at as time_dif')
puts u.first['time_dif']
You can get something like 168 days 12:51:20.851115. Ugly right?
Well, using Ruby to convert this string into an useful number is not easy, but you can use Postgres to do the job for you. You will need to do a plain SQL query though, but it's the best method I've found so far:
query = ActiveRecord::Base.connection.execute("
SELECT EXTRACT(epoch FROM time_dif)/3600 as hours_dif
FROM
(
SELECT (edited_at - created_at) as time_dif
FROM users
) AS MainQuery
")
In this example, Postgres' EXTRACT function will convert the Interval type into a number which represents the total seconds of the interval. If you divide this number by 3600 you will get the different in hours as in the example above.
Then, if you want to iterate over the results:
query.each do |r|
puts r['hours_dif']
end
You could save duracion as a float type, where duracion would equal something like final_time - created_at and this value would be the difference in seconds. You can then perform arithmetic with these values and always convert back to minutes, hours, or whatever you need.
I'm trying to display a simple tableview in IOS with data from Sqlite. My database date is stored as a timestamp. I thought was an unix timestamps but if i try to use dateWithTimeIntervalSince1970 i've really strange result.
Examples of date rows stored:
1352208510267
1352208512266
1352208514266
1352208516266
1352208530266
1352208532265
Use a query like this
SELECT datetime(timestamp, 'unixepoch') from YOURTABLENAME
WHERE id = someId;
This should convert it to some readable value.
Have a look here
I found the answer here. I compared the results with the previous answers:
SELECT strftime('%Y-%m-%d %H:%M:%S', datetime(ZDATE+978307200, 'unixepoch', 'localtime')), datetime(ZDATE, 'unixepoch', 'localtime') FROM ZTABLE
The query with the adjustment for Apple's epoch (Jan 1 2001) gives me the correct date:
"2015-09-29 20:50:51", "1984-09-28 20:50:51"
"2015-09-29 21:03:10", "1984-09-28 21:03:10"
"2015-09-29 21:25:30", "1984-09-28 21:25:30"
Unix timestamps are defined as the number of seconds since Jan 1 1970.
Just now, this would be about 1365525702.
Your values are one thousand times larger, i.e., they are measured in milliseconds.
Decide whether you actually need the millisecond precision, and then add * 1000 or / 1000 at the appropriate places.
For some reason:
Analytic.where({:ga_date.gte => '2010-09-01'}).count() # greater than or equal to
gives back 0, but
Analytic.where({:ga_date.gte => Time.parse('2010-09-01')}).count()
gives back 230, which is the number of records (documents).
Actually, the first line on the top works in another case, so it is quite strange.
Can only the date be compared, because if it is
Analytic.where({:ga_date.lte => Time.parse('2010-09-10')}).count() # less than or equal to
then all the records with date 2010-09-10 will not be counted because Time.parse('2010-09-10') will give 2010-09-10 00:00:00, so the records will all have to be 2010-09-09 before the midnight. In other words, 2010-09-10 2am won't be included because 2am is not "less than or equal to" 00:00:00. It can be hacked by using
Analytic.where({:ga_date.lte => Time.parse('2010-09-10 23:59:59')}).count()
but it is kind of ugly. If there is a way to compare by date only like the first line of code in this post?
I think that you have two separate issues here.
Different data types
The following two lines are not equivalent. The first is a string comparison. The second is a comparison with a date object.
Analytic.where({:ga_date.gte => '2010-09-01'}).count()
Analytic.where({:ga_date.gte => Time.parse('2010-09-01')}).count()
I think you have figured this out, but it's important to be clear here. If you are storing date objects in the DB, you need to perform comparisons with date objects.
MongoDB will compare types and data.
Mismatch date storage
You are storing dates that have information for hours, minutes and seconds. However, you don't like the following notation:
:ga_date.lte => Time.parse('2010-09-10 23:59:59')
The workaround here is to use $lt and the day after.
:ga_date.lt => (Time.parse('2010-09-10') + 1.day) # or (60 * 60 * 24)
to add,
it is not strangely works, its coincidentally works when it just happens the string representation of the date happens to also lexicographically be 'greater than' the other date
other issue,
try to use only as much data fields as needed
if you meant it to be "within the calendar day",
what I usually like is to call beginning_of_day in both cases to equalize
this has the effect of neutralizing the minutes
else if you really meant within a 24h strike zone,
use ActiveSupport's '+ 1.day'