I want to read my InfluxDB measurements in RFC3339 formatting via the CLI for debugging purposes, but since there is already an API that handles read/write operations, I don't want to mess with the actual format they are stored (Unix epoch nanoseconds).
If I issue "influx -precision rfc3339" as suggested here, will it also affect the format the timestamps are stored in the database, or just the format they are printed in the terminal? I only want the latter.
I just found the answer, in the official documentation:
Note: Setting the precision to rfc3339 (-precision rfc3339) works with the -execute option, but it does not work with the -import option. All other precision formats (e.g., h,m,s,ms,u, and ns) work with the -execute and -import options.
So it seems that even if I actively tried to import data with rfc3339 format, I wouldn't be able to. :)
Related
I use telegraf to send some data from a database to InfluxDB in regular intervals which works fine apart from one issue:
I need to replace telegraf's auto-generated timestamp (which is the current time at the moment of telegraf reading the data to transmit) with a field from the data.
(To answer the "why?" question: So the data I get in InfluxDB as a result actually matches the time of the event I want to record).
I would have thought there's some standard configuration parameter or an easy to find processor plugin, that let's me replace the default timestamp with the content of a field, but I didn't find any.
It does not seem to me to be a very exotic request and Telegraf's "Metric" does have a "SetTime" function, so I hope someone already solved that and can answer that.
You can use a starlark processor to accomplish this, at least if you are able to put the correct timestamp into a tag first.
Here an example of how I use a starlark processor to replace the timestamp of the measurement with the content of a tag that I already populated with the correct timestamp in the input plugin:
[[processors.starlark]]
source = '''
def apply(metric):
if "nanoseconds" in metric.tags:
metric.time = int(metric.tags["nanoseconds"])
return metric
'''
Due to the fact that
no one had an idea so far,
that feature isn't available in all plugins coming with telegraf, and
I have to solve this pretty urgently,
I wrote a telegraf processor plugin that does exactly what I needed.
(I will likely offer to contribute this to telegraf in the future, when I have a bit more time to breathe than right now.)
There exists a method to do this both for JSON format input and CSV input.
Here's a link to the JSON format description and how to set timestamps based on a value that's in the payload (to the format you can specify).
use
[[processors.date]]
tag_key = "name_of_column of your timestamp"
date_format = {time format applicable in Go language}
I have millions of entries that are already in epoch ms format as longs.
I don't want to waste time converting to a string like "2016-..." if possible.
Does influx-python's InfluxDBClient have a precision arg, similar to how the native influx client allows -precision when it imports? I looked i github but didn't find it.
Or is the native client just converting everything into rfc3339 under the covers anyhow? I would think that the underlying wire protocol would want stamps as simple longs not strings.
I faced similar trouble. Its in documentation while writing points you can pass time_precision as parameter in the write_point function. If you are using series Helper look at
https://github.com/influxdata/influxdb-python/issues/502
Pull request I made. It helps in setting it in one place and not having to pass it everywhere. I opened up an issue for this.
https://github.com/influxdata/influxdb-python/issues/501
I can't find it specified anywhere. I did find a Microsoft example that had "5/5/1955". Is that d/m/y or y/m/d.
I'm guessing that I probably ought to use ISO 8601, but it would be nice to know for sure.
According to the OASIS spec, the type associated with URI
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/dateofbirth is xs:date, so it should be a normal XML date format... which is indeed ISO-8601, according to XML Schema Part 2. (That talks about the time zone for a date being representable, which strikes me as a little odd, but never mind.)
I format dates with the command line option -f %Y-%m-%d or even %d-%b-%y
but each date comes out four years and one day ahead of the date I input
for example, date 01.06.2012 after parsing without -f option comes as 2016-06-02
toying with -f gives same result
What is the reason? Are there any workarounds,
except hardcode and substract back these 4 years and 1 day?
I am using xls2csv (by V.B.Wagner, comes with catdoc package in debian)
and switching to another parser can be very expensive option
Tools xls2csv is a Perl application that uses Spreadsheet::ParseExcel library.
Based on such library documentation, one of known problems is:
If Excel has date fields where the specified format is equal to the system-default for the short-date locale, Excel does not store the format, but defaults to an internal format which is system dependent. In these cases ParseExcel uses the date format 'yyyy-mm-dd'.
So you probably manipulate with Excel file that does not contain date formating due to above listed issue.
That's a known bug. A patch is available at
https://www.wagner.pp.ru/cgi-bin/cvstrac/catdoc/tktview?tn=14,4
It works.
By the way, there are two programs called xls2csv, we're talking about the one from the catdoc package, not the Perl program.
I am looking at the <xforms:input> formatting documentation and am curious if it is at all possible to display the date as "3 Jul 2011". This should formatting very simple given the use of Java's SimpleDateFormat with the mask [d] [MMM] [yyyy]. The <xforms:input> documentation makes it seem possible to change the canonical format but only references Regex expressions.
Or am I restricted to the masks [M], [D] and [Y]?
You can choose pretty much any format you want when displaying a date or time with <xforms:output>. However, when capturing a date or time with <xforms:input>, Orbeon Forms limits you to just a few formats, as documented.
The reason for this is somewhat technical: for inputs, Orbeon Forms needs to be able to both format the date/time in the format you specify, and to parse it. And the parsing is implemented to accept as many reasonable date or time formats entered by the user. For instance, if you choose a [M]/[D]/[Y] format (typical in the US), you can enter 12/2/2011, but also 12/2 (skipping the year), or even 2 (skipping both the year and the month), or today, as well as several other formats.
The bottom line is that because of this "smart parsing", the <xforms:input> can only support a number of predefined formats. Additional formats can be added, but this requires Orbeon Forms itself to be changed to support those additional formats.