As described in https://developer.apple.com/reference/cloudkit/ckserverchangetoken, the CloudKit servers return a change token as part of the CKFetchRecordZoneChangesOperation callback response. For what set of subsequent record fetches should I include the given change token in my fetch calls?
only fetches to the zone we fetched from?
or would it apply to any fetches to the db that that zone is in? or perhaps the whole container that the db is in?
what about app extensions? (App extensions have the same iCloud user as the main app, but have a different "user" as returned by fetchUserRecordIDWithCompletionHandler:, at least in my testing) Would it be appropriate to supply a change token from the main app in a fetch call from, say, a Messages app extension? I assume not, but would love to have a documented official answer.
I, too, found the scope of CKServerChangeToken a little unclear. However, after reviewing the documentation, both CKFetchDatabaseChangesOperation and CKFetchRecordZoneChangesOperation provide and manage their own server change tokens.
This is particularly useful if you decide to follow the CloudKit workflow Dave Browning outlines in his 2017 WWDC talk when fetching changes (around the 8 minute mark).
The recommended approach is to:
1) Fetch changes for a database using CKFetchDatabaseChangesOperation. Upon receiving the updated token via changeTokenUpdatedBlock, persist this locally. This token is 'scoped' to either the private or shared CKDatabase the operation was added to. The public database doesn't offer change tokens.
2) If you receive zone IDs via the recordZoneWithIDChangedBlock in the previous operation, this indicates there are zones which have changes you can fetch with CKFetchRecordZoneChangesOperation. This operation takes in it's own unique server change token via it's rather cumbersome initializer parameter: CKFetchRecordZoneChangesOperation.ZoneConfiguration. This is 'scoped' to this particular CKRecordZone. So, again, when receiving an updated token via recordZoneChangeTokensUpdatedBlock, it needs persisting locally (perhaps with a key which relates to it's CKRecordZone.ID).
The benefit here is that it probably minimises the number of network calls. Fetching database changes first prevents making calls for each record zone if the database doesn't report any changed zone ids.
Here's a code sample from the CloudKit team which runs through this workflow. Admittedly a few of the APIs have since changed and the comments don't explicitly make it clear the 'scope' of the server change tokens.
Related
I am currently creating an iOS application with Swift. For the database I use Firebase Realtime Database where I store among other things information about the user and requests that the user sends me.
It is very important for my application that the data in the database is not corrupted.
For this I have disabled data persistence so that I don't have to store the requests locally on the device. But I was wondering if it was possible for the user to directly modify the values of the variables during the execution of my application and still send erroneous requests.
For example the user has a number of coins, can he access the memory of the application, modify the number of coins, return to the application and send an erroneous request without having to modify it himself.
If this is the case then is it really more secure to disable data persistence or is this a misconception?
Also, does disabling access to jailbroken devices solve my problems? Because I've heard that a normal user can still modify the request backups before they are sent.
To summarize I would like to understand if what I think is correct? Is it really useful to prevent requests to save locally or then anyway a malicious user will be able to modify the values of variables directly during the execution and this without jailbreak?
I would also like to find a solution so that the data in my database is reliable.
Thank you for your attention :)
PS : I also set the security rules of the db so that only a logged in user can write and read only in his area.
You should treat the server-side data as the only source of truth, and consider all data coming from the client to be suspect.
To protect your server-side data, you should implement Firebase's server-side security rules. With these you can validate data structures and ensure all read/writes are authorized.
Disabling client-side persistence, or write queues as in your previous question, is not all that useful and not necessary once you follow the two rules above.
As an added layer of security you can enable Firebase's new App Check, which works with a so-called attestation provider on your device (DeviceCheck on iOS) to detect tampering, and allows you to then only allow requests from uncorrupted devices.
By combining App Check and Security Rules you get both broad protection from abuse, and fine-grained control over the data structure and who can access what data.
I am working on CloudKit sync in my app ("Tiny data, all devices" model, with a custom zone in the private database).
CKModifyRecordsOperation contains clientChangeTokenData property of NSData type which is described in the docs as follows:
When you modify records from a fetch operation, specify a client-generated data token using this property to indicate which version of the record you last modified. Compare the data token you supplied to the data token in the next record fetch to confirm the server has successfully received the device’s last modify request.
I don't get why I should bother given that with each request, I get a completion block which tells me whether the server has successfully received my request. Why do I need to manually compare this client token?
Is specifying clientChangeTokenData required to handle my use case correctly? I track local data changes and push everything in the queue on each data change. Remote changes are tracked via zone subscription.
If it is required, how do I generate this token correctly given that I have all kinds of record changes in my CKModifyRecordsOperation (my API usage aims for batch operations). What is the general workflow here?
Thank you.
It's unclear from the docs so I'd guess the clientChangeTokenData is useful in the case of sending up a large modify records operation, e.g. deleting 100 records. Then say your app sends a fetch request in another operation with a query (or fetch changes) result set that would be affected by the modifications which either:
is started and is running concurrently to the existing modify operation which hasn't finished yet.
is started before the server has finished processing the results of the previous modifications (the docs tend to allude to a processing delay).
If the fetch completion contains a different clientChangeTokenData to the one sent with the modify then you know it hasn't received (or finished processing?) the changes yet. In this situation you could either error, with an alert to say the server needs more time, or automatically retry the fetch after some time.
By the way in my tests, this token is per-device.
You would only have a reason to check the token if you had local changes that you want to write to CloudKit and you want to make sure that your changes are based on the latest version of the data in CloudKit.
You could also just ignore the token and save the data anyway. If the data has changed in the mean time, you will get a CloudKit error and you could handle it then.
Our app supports offline activity. Meaning we want to persist locally the creation of new core data objects as well as any modifications on existing objects. Then when the app goes online again we automatically push those changes (and any dependencies) up to the server.
I would think that RestKit would support such an operation, but currently when offline we store creations/modifications in a local cache. If I kill the app, those changes are not persisted. And also there is no attempt by RestKit to post those items to their originally intended endpoints.
I cannot find any documentation to support what we need here.
Is there a way for RestKit to do what we need?
If not, how do I get offline changes to persist to the disk (and not cache)? Then would it be appropriate to flag those as not uploaded to server, and then try uploading them when we are back online?
Any other important things I should consider?
At the time of writing RestKit does not support that feature.
To save to disk you need to call saveToPersistentStore: instead of just save: on the MOC.
You need to implement a scheme yourself, observing the 'online' status of the app and scanning the data store for things that need to be uploaded (which means maintaining a flag to indicate if it's happened yet).
I solved this issue by adding another field called 'updated' to my object. This field is set to true or 1 when the object is created or modified. Each time the application is started or synchronized, it iterates through the local core data copy and sends the objects with 'updated' set. On the web service, the response ALWAYS clears 'updated' to false when returning a response. This works well in the case where the web service and app are both online.
I am working on a project that requires an automated SSIS package to
connect to SurveyMonkey data store via API to incrementally download survey
results for the day or specified time period for custom reporting and low scoring task assignment.
Via OAuth I can collect a long lived access token, but due to the automated
and infinite nature of my projects lifespan, I cannot manually initiate
OAuth2 token refreshes or complete manual re-authentication cycles.
Is there another method to automatically export this data upon a scheduled
request?
Additionally, for clarification for how long is a long lived access token
valid? 60 days?
Miles from surveymonkey.com support responded to me with a great answer. I hope it can help someone down the line.
Hi Rob,
Currently our tokens should not expire - this is not guaranteed and
may change in future, but we will send out an update well ahead of
time if this does ever change. The token you receive on completion of
OAuth lets you know how long the token will last for without user
intervention, currently it returns 'null' in the 'expires_in' field.
There is no other automated way to schedule the data to be exported
currently, however it sounds like our current setup should suit your
needs
In addition to Miles's reply, it is very straightforward to pull diffs from surveymonkey using modified dates. we keep "last sync" timestamp per-survey in our database, and update it after each successful data pull.
Use the REST api directly, or (if you're using PHP) try https://github.com/oori/php-surveymonkey. We run it in production.
*note: actually, you're interested in setting the start_modified_date option for the "getRespondentList" function. but in general - see the API docs, modified date filter is available in more functions.
I am writing a simple IMAP client that will be able to sync w/ any Google email account. I don't want to have to read the ENTIRE set of message headers on the server every time I sync in order to be assured that I do not miss something. I would prefer to not ever have to do that, and to rely on some field that ensures total order. For example, I would prefer to rely on Google extended Message ID field or even just on Receieved-Date and have my logic be: "keep reading backwards until you hit something you have previously read". But alas, it does not seem to be that simple.
What is the preferred way to do sync such that it is both efficient (in terms of time + bandwidth) and guaranteed (i.e., no missed messages)?
Thanks!