Good Afternoon,
I want to make my life easier by querying SQLite databases I find on mobile devices, as opposed to manually putting the values in MFT Stampede.
I have a database with 70 tables that I extracted from an iOS device. There is a table I have a particular interest in which keeps a record of all images that have been stored on the iOS device in a particular directory I'm interested in. I ran a timestamp number "680204956.051849" in MFT and got the "MAC (CF) Absolute Time of "Fri, 22 Jul 2022 17:49:16". I ran a query to extract all the dates:
SELECT
datetime(ZADDEDDATE, 'unixepoch')
FROM ZASSET
LIMIT 15;
For the same field I ran in MFT, I get "1991-07-22 17:49:16". The year is wrong, any idea how I can get the correct year?
Related
My app lets people log the movies they see (for example). Each logged movie usually (but not always) has a date and sometimes has a time. It's not unusual to have one but not the other. Occasionally the dates are only a year ("I watched a Dumbo sometime in 1984"), but could realistically be any combination of day/month/year/time.
I am used to modeling dates as date objects in my app and my backend. But is it a viable approach to store each component separately? When I need to reference an actual date from the components (e.g. for sorting the log) this will be built client-side, or perhaps be stored as a derived property sortDate and updated whenever any of the components change.
My reservation is that the information the user is saving is truly a 'moment in time' and I will have to take care of some things myself - for example what time zone are my components stored relative to? This would be captured automatically as part of a real Date object.
The alternative seems to be assuming some sort of 'default' for missing components (e.g. year 0000 if no year, time 00:00 if no time). But those defaults have meaning and I won't be able to distinguish them from 'not provided'.
What are the limitations and/or pitfalls of this approach? Does anyone have experience modeling their dates this way?
If it's of any consequence, my app is for iOS written in Swift and uses a Parse Server backend.
I've successfully used question marks to represent ambiguous and unknown timestamp parts in legal systems. Try to keep in mind that you're really not modeling dates here ('1984' isn't a date); you're modeling facts about dates.
So, if one of your users saw a movie some time in 1984, you might record the value '1984-??-?? ??:??:??' in a text column in a database. Values like this sort sensibly.
See also this answer on dba. Comments on that answer are also good to read.
We have a fact table which collects information detailing when an employee selected a benefit. The problem we are trying to solve is how to count the total benefits selected by all employee's.
We do have a BenefitSelectedOnDay flag and ordinarily, we can do a SUM on this to get a result, but this only works for benefit selections since we started loading the data.
For Example:
Suppose Client#1 has been using our analytics tool since October 2016. We have 4 months of data in the platform.
When the data is loaded in October, the Benefits source data will show:
Employee#1 selected a benefit on 4th April 2016.
Employee#2 selected a benefit on 3rd October 2016
Setting the BenefitSelectedOnDay flag for Employee#2 is very straight forward.
The issue is what to do with Employee#1 because we can’t set a flag on a day which doesn’t exist for that client in the fact table. Client#1's data will start on 1st October 2016.
Counting the benefit selection is problematic in some scenarios. If we’re filtering the report by date and only looking at benefit selections in Q4 2016, we have no problem. But, if we want a total benefit selection count, we have a problem because we haven’t set a flag for Employee#1 because the selection date precedes Client#1’s dataset range (Oct 1st 2016 - Jan 31st 2017 currently).
Two approaches seem logical in your scenario:
Load some historical data going back as far as the first benefit selection date that is still relevant to current reporting. While it may take some work and extra space, this may be your only solution if employees qualify for different benefits based on how long the benefit has been active.
Add records for a single day prior to the join date (Sept 30 in this case) and flag all benefits that were selected before and are active on the Client join date (Oct 1) as being selected on that date. They will fall outside of the October reporting window but count for unbounded queries. If benefits are a binary on/off thing this should work just fine.
Personally, I would go with option 1 unless the storage requirements are ridiculous. Even then, you could load only the flagged records into the fact table. Your client might get confused if he is able to select a period prior to the joining date and get broken data, but you can explain/justify that.
I process data that arrives on an amazon server, put the data in an influxDB (0.9.5) and visualise on Grafana. Now suddenly data from two measurements is not selected any-more before a certain timestamp (11 July), e.g.
SELECT * FROM cable WHERE time < '2016-07-11 00:00:00'
would return no results.
However, the data is still there: I re-precessed one data-file from 15 May 2016, and now the data from 9 May till 15 May shows again.
This is quite a strange behaviour, would anyone have a clue what is going on here?
Thanks,
Bart
I'm working on an app that connects to a mysql backend. It's a little simliar to snapchat in that once the current user gets the pics from the users they follow and see them they can never again see these pics. However, I can't just delete the pics from the database, the user who uploaded the pic still needs to see them. So I've come up with an interesting design and I want to know if its good or not.
When uploading the pic I would also create a mysql event that would run the same time exactly one day after the pic was uploaded deleting itself. If I have people uploading pics all the time events would be created all the time. How does this effect the mysql database. Is this even scalable?
No, not scalable: Deleting of single records is quick, however if your volume increases, you run into trouble. You do however have a classic case for using partitioning:
Create table your_images (insert_date DATE,some_image BLOB, some_owner INT)
ENGINE=InnoDB /* row_format=compressed key_block_size=4 */
PARTITION BY RANGE COLUMNS (insert_date)
PARTITION p01 VALUES LESS THAN ('2015-07-12'),
PARTITION p02 VALUES LESS THAN ('2015-07-03'),
PARTITION p0x VALUES LESS THAN (ETC),
PARTITION p0n VALUES LESS THAN (MAXVALUE));
You can then insert just as you are used to, drop the partitions once per day (using 1 event for all your data), and create new partitions also once per day (using the same event which is dropping your old partitions).
To make certain a photo lives for 24 hours (minimum), the partition cleanup has to occur with a 1 day delay (So cleanup the day before yesterday, not yessterday itself).
A date filter in your query getting the image from the database is still needed to prevent the images from older then a day being displayed.
Right, this is a bit confusing for me, so I'm going to try and explain from the top!
I have a rails web app. It's an internal company app and will only be used in the UK.
One of the things the app does is manage meetings.
Meetings have a date & time when they start. There's a date/time picker on the form which allows the user to pick the date & time the meeting is for. I save this date AS IS into the database. All meetings last 2 hours, so the end time is simply start + 2 hours.
Example:
2013-06-23 6:45PM in the form is stored in the db as 2013-06-23 18:45:00
2013-12-23 6.45pm in the form is stored in the db as 2013-12-23 18:45:00
Note that the first date is during Daylight Savings (BST) and the second is during GMT. I don't actually care whether it is GMT or BST: the meeting happens at that time, absolutely.
Inside the rails webapp, I simply print out the exact date & time from the DB - formatted nicely, of course!
Now, at some point I send an email to the organiser of the meeting, and the person they're meeting with. This email tells them the the date & time of the meeting etc, and also includes an iCal (.ics) file for them to put into their (Outlook usually, but also Apple or gmail) calendar.
The issue I am having is that (using the above examples) Outlook shows the meetings like this:
Meeting #1: Start: 23/06/2013 7:45pm, End: 23/06/2013 9:45pm
Meeting #2: Start: 23/12/2013 6:45pm, End: 23/12/2013 8:45pm
Note that it has adjusted the first one because of the BST/GMT thing.
The text of the .ics file contains this code:
Meeting #1:
BEGIN:VCALENDAR
...
DTEND:20130623T204500Z
DTSTART:20130623T184500Z
...
END:VCALENDAR
Meeting #2:
BEGIN:VCALENDAR
...
DTEND:20131223T204500Z
DTSTART:20131223T184500Z
...
END:VCALENDAR
So I am encoding the dates/times using the Z timezone (UTC). I understand this is why Outlook mis converting the UTC time into the BST time for #1 and leaving #2 alone (because GMT == UTC)
My question is: how do I stop this happening? I want the time the meeting is scheduled for to be the absolute, actual time, regardless of GMT/BST: 6:45pm
Should I be storing the date-times as UTC in the DB? How would this be done (I assume it would apply to all dates, not just meeting start dates). And how to re-convert them back into the actual datetime when I display them in the webapp?
Extra:
I have an entry in my initializers/time_formats.rb like this:
:ical => "%Y%m%dT%H%M00Z"
So dates come out like "20130623T184500Z". I use this when building the ics. And this I think is the issue - if the date/time is during BST I don't want to be using Z, but something else?
Your problem is your date/time format. You have:
DTSTART:20130623T184500Z
in your .ics file and this corresponds to 19:45 BST (as British summer time is UTC+1).
There are a few things you should do. First, you can simply remove the 'Z' from the end of your dates. This means that the times inherit the timezone of the calendar, or the underlying application.
This will work assuming that the machines which are running Outlook are all in the Europe/London timezone. If not, or if you want to be a bit safer, you should also specify the following after your BEGIN: VCALENDAR line:
X-WR-TIMEZONE:Europe/London
This specifies the default timezone for all dates which are not specified explicitly.
Finally, if this does not work for any reason then you need to define your datetimes explicitly. First you need to add a timezone definition for Europe/London to the calendar. The info you need is available at http://www.tzurl.org/zoneinfo-outlook/Europe/London.ics. Then you need to ensure that all datetimes are of the format:
DTSTART;TZID=Europe/London:20130623T184500
This last approach is the best, as it means that if your requirements expand to other timezones you will be able to handle them relatively easily.
Sorry to answer this myself, but in case anyone else runs into this here's what I found was the cause of my particular issue. Note that the answer above re timezones also makes sense!
My rails app is storing UTC datetimes in the DB (as is default)
But, it also thought it's own timezone was UTC, which also seems to be the default.
The upshot of that is essentially it was storing local dates, local to UTC anyway. Changing the app to know it was sitting in Europe/London made it so the dates in the DB are all now accurately UTC (meaning, they're an hour off if I'm currently in BST)
I can now use the Z datetime format in iCals, and outlook and the rails app both convert the UTC date back into the actual datetime for the viewing-user's locale (Europe/London for everyone at the moment). This is what I wanted.