I am writing a client application that fetches emails from an IMAP server and then stores them in a database. The problem is that once I have checked the mail, the next time I only want to download the mail that has arrived since. So if I had checked the server for mail two hour ago, I only want to get the mail that has arrived in the last two hours.
I could use SEARCH with SINCE DATE, but there's no support for time + date could be easily spoofed.
I also tried the RECENT flag, but that doesn't seem to work with gmail (in ruby it shows nil everytime).
You want to use the UniqueId (UID) for the messages. This is specifically why it was created.
You will want to keep track of the last UID requested, and then, to request all new messages you use the message set "[UID]:*", where [UID] is the actual UID value.
For example, lets say the last message feteched had a unique id of "123456". You would fetch
123456:*
Then, discard the first returned message.
UIDs are 'supposed' to be stable across sessions, and never change, and always increase in value. The catch to verify this, is to check the UIDValidity when you select the folder. If the UIDValidity number hasn't changed, then the UIDs should still be valid across sessions.
Here are the relevant parts from the RFC:
2.3.1.1. Unique Identifier (UID) Message Attribute
A 32-bit value assigned to each message, which when used with the
unique identifier validity value (see below) forms a 64-bit value
that MUST NOT refer to any other message in the mailbox or any
subsequent mailbox with the same name forever. Unique identifiers
are assigned in a strictly ascending fashion in the mailbox; as each
message is added to the mailbox it is assigned a higher UID than the
message(s) which were added previously. Unlike message sequence
numbers, unique identifiers are not necessarily contiguous.
The unique identifier of a message MUST NOT change during the
session, and SHOULD NOT change between sessions. Any change of
unique identifiers between sessions MUST be detectable using the
UIDVALIDITY mechanism discussed below. Persistent unique identifiers
are required for a client to resynchronize its state from a previous
session with the server (e.g., disconnected or offline access
clients); this is discussed further in [IMAP-DISC].
Note: The next unique identifier value is intended to
provide a means for a client to determine whether any
messages have been delivered to the mailbox since the
previous time it checked this value.
Here is the link with more info:
http://www.faqs.org/rfcs/rfc3501.html
What I would do, is also keep track of the InternalDate of the messages downloaded. This way, if you ever lose UID sync, you can at least iterate through the messages, and find the last one you downloaded, based upon the InternalDate of the message.
There's an imap flag called "seen". Most clients would mark a message seen when viewing the message, so you'd want to iterate over messages on the server which do not have that flag set.
Here's a code snippet which should give you the right idea. The operative bit of course is
imap.search(["NOT", "SEEN"]).each do bla.bla.bla
If you are you able to filter incoming mail into a specific IMAP folder on the server side, your app
can read new messages in that folder and then move them into the standard INBOX folder after it's done.
Related
So the skip token I get from graph API is a number, based on my understanding(I might be wrong), it indicates how many emails need to be skipped.
In our application, we store that skip token in our db/memory so we can fetch next page of emails. So if say a users current skip token is 100, and before we send a request to the server with skip token 100, that user delete 10 emails, then what gonna happen if still use that 100 skip token?
Since I am not sure how to deal with this kind of user delete emails case, the way our application works is: we always do a minus on the skip token(like -10), and check if we can find any email or timestamp overlap between current response and previous response, if there is no overlap, we do another minus to the skip token. It is kind of like walk backward. We stop doing minus till we can find an overlap.
Does it make sense? So far, I noticed some skip tokens's responses give nextLink as null while there are still new emails in user's inbox. Also, we missed a couple of emails for around half year(meaning that the email is in user's inbox but not fetched by our application).
The Delta Query (Track Changes) API might be better suited for your needs. It effectively allows you to keep a "bookmark" in a change log of someones inbox.
E.g. Instead of keeping the skip token you would keep the deltaLink you get back from calling /messages/delta. When you call the API again with the deltaLink you will get a set of changes back since the last time you called the API + a new deltaLink. This allows you to keep "in sync" with the changes going on in the inbox you are monitoring.
The API reference docs are here:
https://learn.microsoft.com/en-us/graph/delta-query-overview
I created an interactive message that gets called via a slash command which gets distributed to the entire channel. Everything works fine. But I can't for the life of me figure out how to limit the message to the entire channel but to the person evoking it. Right now the message goes to everyone including the person that invoked it. If you've ever used /poll, the poll goes out to everyone but the person who created it.
If anyone knows how to do this, can you point me in the right direction?
The response message from a slash command can be only one of two things:
an ephemeral response, which can be seen by the user who issued the command only
an in_channel response, which can be seen by everyone in the channel
There is no feature or switch so that the response message would not be visible to the issuing user.
You can however build a workaround and manually send every user except the one issuing the command an ephemeral message. Here is an outline of how that would work:
Get the list of users of the channel via conversations.members
Send each user an ephemeral message via chat.postEphemeral
There are some significant caveat to this workaround:
You app needs a lot of additional scopes to be allowed to retrieve the member list and send messages (see documentation of those API methods for details)
There is rate limit of about 1 message per second, so it might take quite some time to send those message to all user depending on how big the group is
your slash command needs to respond with 3 seconds. So you will need to implement some fancy multi-threading to be able to send all those messages.
To make that work in private channels you have to work with an additional bot user
That would get you the result you are asking for, but there are some caveats:
As described in https://developer.apple.com/reference/cloudkit/ckserverchangetoken, the CloudKit servers return a change token as part of the CKFetchRecordZoneChangesOperation callback response. For what set of subsequent record fetches should I include the given change token in my fetch calls?
only fetches to the zone we fetched from?
or would it apply to any fetches to the db that that zone is in? or perhaps the whole container that the db is in?
what about app extensions? (App extensions have the same iCloud user as the main app, but have a different "user" as returned by fetchUserRecordIDWithCompletionHandler:, at least in my testing) Would it be appropriate to supply a change token from the main app in a fetch call from, say, a Messages app extension? I assume not, but would love to have a documented official answer.
I, too, found the scope of CKServerChangeToken a little unclear. However, after reviewing the documentation, both CKFetchDatabaseChangesOperation and CKFetchRecordZoneChangesOperation provide and manage their own server change tokens.
This is particularly useful if you decide to follow the CloudKit workflow Dave Browning outlines in his 2017 WWDC talk when fetching changes (around the 8 minute mark).
The recommended approach is to:
1) Fetch changes for a database using CKFetchDatabaseChangesOperation. Upon receiving the updated token via changeTokenUpdatedBlock, persist this locally. This token is 'scoped' to either the private or shared CKDatabase the operation was added to. The public database doesn't offer change tokens.
2) If you receive zone IDs via the recordZoneWithIDChangedBlock in the previous operation, this indicates there are zones which have changes you can fetch with CKFetchRecordZoneChangesOperation. This operation takes in it's own unique server change token via it's rather cumbersome initializer parameter: CKFetchRecordZoneChangesOperation.ZoneConfiguration. This is 'scoped' to this particular CKRecordZone. So, again, when receiving an updated token via recordZoneChangeTokensUpdatedBlock, it needs persisting locally (perhaps with a key which relates to it's CKRecordZone.ID).
The benefit here is that it probably minimises the number of network calls. Fetching database changes first prevents making calls for each record zone if the database doesn't report any changed zone ids.
Here's a code sample from the CloudKit team which runs through this workflow. Admittedly a few of the APIs have since changed and the comments don't explicitly make it clear the 'scope' of the server change tokens.
I am working on CloudKit sync in my app ("Tiny data, all devices" model, with a custom zone in the private database).
CKModifyRecordsOperation contains clientChangeTokenData property of NSData type which is described in the docs as follows:
When you modify records from a fetch operation, specify a client-generated data token using this property to indicate which version of the record you last modified. Compare the data token you supplied to the data token in the next record fetch to confirm the server has successfully received the device’s last modify request.
I don't get why I should bother given that with each request, I get a completion block which tells me whether the server has successfully received my request. Why do I need to manually compare this client token?
Is specifying clientChangeTokenData required to handle my use case correctly? I track local data changes and push everything in the queue on each data change. Remote changes are tracked via zone subscription.
If it is required, how do I generate this token correctly given that I have all kinds of record changes in my CKModifyRecordsOperation (my API usage aims for batch operations). What is the general workflow here?
Thank you.
It's unclear from the docs so I'd guess the clientChangeTokenData is useful in the case of sending up a large modify records operation, e.g. deleting 100 records. Then say your app sends a fetch request in another operation with a query (or fetch changes) result set that would be affected by the modifications which either:
is started and is running concurrently to the existing modify operation which hasn't finished yet.
is started before the server has finished processing the results of the previous modifications (the docs tend to allude to a processing delay).
If the fetch completion contains a different clientChangeTokenData to the one sent with the modify then you know it hasn't received (or finished processing?) the changes yet. In this situation you could either error, with an alert to say the server needs more time, or automatically retry the fetch after some time.
By the way in my tests, this token is per-device.
You would only have a reason to check the token if you had local changes that you want to write to CloudKit and you want to make sure that your changes are based on the latest version of the data in CloudKit.
You could also just ignore the token and save the data anyway. If the data has changed in the mean time, you will get a CloudKit error and you could handle it then.
I'm working about an email manager that can manage more than one mailboxes.
This is my scenario: I get the mails by a mailbox and I store their properties in a database, uid included. The mailbox can be configured to be downloaded with imap or pop3 protocol, it depends on the mailbox itself.
POP3 uids: The unique-id of a message is an arbitrary server-determined string, consisting of one to 70 characters in the range 0x21 to 0x7E, which uniquely identifies a message within a maildrop and which persists across sessions.
IMAP uids: a 32-bit value assigned to each message, which when used with the unique identifier validity value (see below) forms a 64-bit value that MUST NOT refer to any other message in the mailbox or any subsequent mailbox with the same name forever.
So, it is almost impossible to find two equal POP3 uids, although between different mailboxes. But is more likely to happen with IMAP uids.
So, I need the certainty that the uids are all different.
I don't need the code to do it, I need only to know if it is possible to do it, and how to do it, the need for a correct reasoning do.
The POP3 standard does not guarantee the UIDs to be unique -- you can see different messages with a common UID.
IMAP has no persistent UIDs which would work in the way you want them to work. The UIDs are guaranteed to be unique within a single mailbox, but under certain circumstances they might get changed (see RFC 3501 and the UIDVALIDITY response). In short, the only guarantee that IMAP provides is that a triplet of (mailbox name, UIDVALIDITY, UID) will ever refer to a single message, no matter what happens. This is useful for e.g. caching message parts, because they are guaranteed to be immutable. Please take care to realize that this does not mean that a single message will be always assigned the same triplet -- not at all, even if it remains in the same mailbox.
There are non-standard extensions which attempt to provide some kind of a GUID for you, but these differ between the IMAP server implementations and are not available everywhere.