I am trying to create a time series chart in Superset using a field named "created_at" which is a timestamp. But in the Superset interface, I can't change the "Time Grain" at all but created_at was picked up automatically.
Any idea how to resolve this issue? I wanted to draw a daily timeseries chart.
It was a connection string issue. postgresql+psycopg2 should be used instead of postgresql
Related
I'm using Google Sheets, and wanted to do a Data Studio "real time" report, based on the data in the sheet.
The sheet contains data from an issue management program.
My data in the sheet exactly:
Issue type (string), Work time (number), issue created at (date), issue resolved at (date)
I successfully created a pie chart that's dimension is: Issue type and metric: Work time
I can see that all the data in the pie chart.
Now my problem is:
I can't filter this chart with a date range.
The date is in a correct format: YYYYMMDD
but I can't describe to data studio, which date column it should filter when I set a date range in the control.
Is it possible some way to create two date range filter and one of them filter on created date, and the other one filter on resolved at date?
Thank all of you!
Roland
Check to make sure that the date range you're filtering by is the first date column from the left in your Sheet. If it's not, then Data Studio gets confused and doesn't work properly.
I'm not sure if you can have two different dates in two different filters. The graphs wouldn't know which filter to follow.
Finally I deleted the connection between the sheet and the report, and started again everything, and it works fine. I think I created the connection when the date was not in a correct format, I changed it after making the connection.
https://github.com/influxdata/telegraf/pull/1557
Apparently some people have been asking for this, and this Github PR is the closest thing I can find to a solution, but it was ultimately denied(I think?).
Basically, I have a JSON object I'm getting from Stackdriver, which includes a Timestamp in ISO8601, which I convert to Unix time. I can insert the entire JSON response into Influx fine, but the timestamp from Stackdriver appears as a tag for a series, rather than the index of the time series itself. As a result, it is unfeasible to query by Stackdriver's provided timestamp. I could simply just drop it, and use the Influx provided timestamp, but it is essentially querying incorrect/imprecise data.
Does anyone have a clever way to approach this?
tl;dr How can I use Telegraf to override InfluxDB's timestamps with my own timestamps?
I am building a rails app, where the user picks up a date from a date picker and a time from the time picker. Both the date and time have been formatted using moment js to show the date and time in the following way:
moment().format('LL'); //January 23,2017
moment().format('LTS'); //1:17:54 PM
I read this answer with guidelines about selection of a proper column type.
Is there documentation for the Rails column types?
Ideally, I should be using :date, :time or :timestamp for this. But since the dates are formatted, should I be using :string instead?
Which would be the correct and appropriate column type to use in this situation?
If you want to store a time reference in your database you should use one of the types the database offers you. I'll explain this using MySQL (which is the one I have used the most) but the explanation should be similar in other database servers.
If you use a timestamp column you will be using just 4 bytes of storage, which is always a good new since it makes smaller indexes, uses less memory in temporal tables during the internal database operations and so on. However, timestamp has a smaller range than datetime so you will only be able to store values from year 1970 up to year 2038 more or less
If you use datetime you will be able to store a wider range (from year 1001 to year 9999) with the same precision (second). The bad consequence is that a higher range needs more memory, making it a bit slower.
There are some other differences between these two column types that don't fit in this answer, but you should keep an eye on before deciding.
If you use varchar, which is the default column type for text attributes in Ruby on Rails, you will be forced to convert from text to datetime and vice-versa every time you need to use that field. In addition, ordering or filtering on that column will be very inefficient because the database will need to convert all strings into dates before filtering or sorting, making it impossible to use indexes on that column.
If you need sub-second precision, you can use bigint to meet your requirements, as MySQL does not provide a date specific type for this purpose
In general, I recommend using timestamp if your application requirements fit the timestamp limitation. Otherwise, use datetime, but I strongly discourage you to use varchar for this purpose.
EDIT: Formatting
The way you store dates in database is completely different from the way you display it to the user. You can create a DateTime object using DateTime.new(year, month, day, hour, minute, second) and assign that object to your model. By the time you save it into database, ActiveRecord will be in charge of converting the DateTime object into the appropiate database format.
In order to display a value that is already stored in database in a specific format (in a view, API response, etc.) you can hava a look at other posts like this one.
You can have a timestamp column in your database, and then parse the request to a ruby datetime object like this:
d = Time.parse(params[:date])
t = Time.new(params[:time])
dt = DateTime.new(d.year, d.month, d.day, t.hour, t.min, t.sec, t.zone)
#now simply use dt to your datetime column
On Postgres you can save a ruby DateTime object straight into a postgres timestamp field, e.g
User.first.update_attribute('updated_at', dt )
Another option is to concatenate your date and time strings into one and then u can do a one-liner:
User.last.update_attribute('created_at', Time.parse('January 23,2017 1:17:54 PM'))
I'm pretty sure this will work on MySQL datetime or timestamp as well.
Credit to david grayson Ruby: combine Date and Time objects into a DateTime
Is it possible to write a InfluxDB query that will give me the number of milliseconds since the last entry in a time series? I'd like to add a single-stat panel in Grafana displaying how old the data is.
I don't think it is possible since you are not able to query the time alone. A influxdb query needs at least one non-time field in a query. You could workaround that by double saving the time in a extra field which you are able to query alone.
But you still want to use now() - "the extra time field". But as far as I found out you also can't use now() inside grafana.
Update: there is a [Feature-Request] now on grafanas github. Make sure to vote it up so it gets implemented one day: https://github.com/grafana/grafana/issues/6710
Update 2: The feature got finaly implemented -> See my answer here: How to show "33 minutes ago" on Grafana dashboard with InfluxDB?
In my database migration file I inserted the line:
t.timestamps
Two columns, as I expected, were created: "updated_at" and "created_at". However, their type is "datetime" and not "timestamp".
I am using MySQL and the "timestamp" type, as I understand, is designed exactly for such cases, as it uses less space and is independent of timezone.
So, is there any reason, why Rails 3 uses "datetime" and not "timestamp"? Should I try to fix that? If yes, is there any way to do this besides not using "t.timestamps" and defining "updated_at" and "created_at" columns separately every time for each new table?
From memory, the mysql timestamp column type behaves similar to updated_at in that it is updated with the current time whenever the record is updated.
While this is useful for the updated_at column, this is not the desired behaviour for created_at.
In addition, Rails handles the timezone as specified in your app's settings (should would normally be set to UTC), so using mysql's time may be inconsistent with other datetime records.
timestamp columns have a limited range which begins in 1970 and ends in 2038. You can google "Unix Millennium Bug" for more information, but it's basically because unix timestamps are stored as a 32-bit signed integer. A timestamp is expressed in seconds since Jan. 1, 1970, and the number wraps on itself in 2038. For this reason, I typically use datetime even when timestamp seems like an easier solution, especially to represent historical or forecasted data further off into the past or future.