So the skip token I get from graph API is a number, based on my understanding(I might be wrong), it indicates how many emails need to be skipped.
In our application, we store that skip token in our db/memory so we can fetch next page of emails. So if say a users current skip token is 100, and before we send a request to the server with skip token 100, that user delete 10 emails, then what gonna happen if still use that 100 skip token?
Since I am not sure how to deal with this kind of user delete emails case, the way our application works is: we always do a minus on the skip token(like -10), and check if we can find any email or timestamp overlap between current response and previous response, if there is no overlap, we do another minus to the skip token. It is kind of like walk backward. We stop doing minus till we can find an overlap.
Does it make sense? So far, I noticed some skip tokens's responses give nextLink as null while there are still new emails in user's inbox. Also, we missed a couple of emails for around half year(meaning that the email is in user's inbox but not fetched by our application).
The Delta Query (Track Changes) API might be better suited for your needs. It effectively allows you to keep a "bookmark" in a change log of someones inbox.
E.g. Instead of keeping the skip token you would keep the deltaLink you get back from calling /messages/delta. When you call the API again with the deltaLink you will get a set of changes back since the last time you called the API + a new deltaLink. This allows you to keep "in sync" with the changes going on in the inbox you are monitoring.
The API reference docs are here:
https://learn.microsoft.com/en-us/graph/delta-query-overview
Related
Microsoft recommends to use delta function in combination with Subscriptions/Notifications to synchronize mailbox. So my plan is:
Create subscription
Receive notification about new mail in inbox
Use delta function to get latest changes in the inbox
My mailbox already has several thousands of letters. If I run the query
https://graph.microsoft.com/v1.0/users/{id}/mailFolders/inbox/messages/delta
It will return in response #odata.nextLink with $skiptoken param many times and only after I get all the thousands of emails in my mailbox I will receive response with $deltatoken to track new changes.
Is there a way to get deltatoken after the first request? I don't want to synchronize the old messages. I want to skip all old messages in inbox and have a fresh start.
Today the delta query functionality does not support this scenario. To request new features please post ideas to uservoice
This is supported for some endpoints. You can use $deltaToken=latest to get just a deltaToken without any resource data. It's not, as far as I can tell, available for mailboxes… but who knows, maybe it will be soon.
This is not documented anywhere in the documentation for the specific APIs that do support it, but is instead documented in the Overview for change tracking. Why? Because Microsoft wants you to be sad all the time.
According to this documentation Synchronize messages API users have ability to synchronize messages with pretty simple skipToken mechanics. And it works well for fetching new messages in folders.
But! What I'm also interested is how to sync flags and status like is message read or not.
For example I synced all messages from Inbox folder. After that user goes to his Outlook account and reads message and set some flag for this message.
How can I get this info? Should I resync all messages to get only those changes?
Also how I get notion about message removal? If some user deleted message from inbox, how I get to know which message was deleted without fetching all messages again?
You should use the Microsoft Graph API and move away from using the Outlook Rest API as it has been deemphasized with no more effort being put into the developer experience.
Use Microsoft Graph to synchronize and track changes by using the Delta query feature.
An initial sync call will happen with a Delta query. Make sure you select the properties you care about:
GET https://graph.microsoft.com/v1.0/me/mailfolders/AQMkADNkNAAAgEMAAAA/messages/delta?$select=subject,sender,isRead
Prefer: odata.maxpagesize=50
If the response has an #odata.nextlink in the response JSON object, GET that URL to the next page of results.
If the response has an #odata.deltaLink in the response JSON object, cache that URL until the next time you want to check for changes.
I've been using Google refresh/access token in my in-app purchase verification on server-side.
When refreshing the token, I get a new access token, valid for 3600 secs.
Why can't I just re-ask for a new access token when needed, rather than storing and re-using it?
Why can't I just re-ask for a new access token when needed, rather than storing and re-using it?
Simple answer is you can. Technically speaking you could just ask for a new access token every time you want to make a request. As long as you store your refresh token you will then be able to access the server when ever you need.
I think a better question would be why wouldn't you want to. Well if you are using an application and that application is running for 30 minutes there is really no reason to request a new access token when you can just use the one that was returned to you the first time. You are making an extra round trip to the server with ever request you make if you are also requesting an access token first.
However if you have an application that say runs once every five minutes as a cron job or a windows service well then one could argue that its a pain trying to keep track of it. Actually its not you could just add a date to when it was generated and check the date before using it.
Now google is not going to stop you from requesting a new one every time so feel free. However I cant remember if Google gives you a new access token or if they just return the same one that they generated last time. If they generate a new one every time. Remember that they will all work for an hour so dont leave them laying around.
I am writing a simple IMAP client that will be able to sync w/ any Google email account. I don't want to have to read the ENTIRE set of message headers on the server every time I sync in order to be assured that I do not miss something. I would prefer to not ever have to do that, and to rely on some field that ensures total order. For example, I would prefer to rely on Google extended Message ID field or even just on Receieved-Date and have my logic be: "keep reading backwards until you hit something you have previously read". But alas, it does not seem to be that simple.
What is the preferred way to do sync such that it is both efficient (in terms of time + bandwidth) and guaranteed (i.e., no missed messages)?
Thanks!
I am writing a client application that fetches emails from an IMAP server and then stores them in a database. The problem is that once I have checked the mail, the next time I only want to download the mail that has arrived since. So if I had checked the server for mail two hour ago, I only want to get the mail that has arrived in the last two hours.
I could use SEARCH with SINCE DATE, but there's no support for time + date could be easily spoofed.
I also tried the RECENT flag, but that doesn't seem to work with gmail (in ruby it shows nil everytime).
You want to use the UniqueId (UID) for the messages. This is specifically why it was created.
You will want to keep track of the last UID requested, and then, to request all new messages you use the message set "[UID]:*", where [UID] is the actual UID value.
For example, lets say the last message feteched had a unique id of "123456". You would fetch
123456:*
Then, discard the first returned message.
UIDs are 'supposed' to be stable across sessions, and never change, and always increase in value. The catch to verify this, is to check the UIDValidity when you select the folder. If the UIDValidity number hasn't changed, then the UIDs should still be valid across sessions.
Here are the relevant parts from the RFC:
2.3.1.1. Unique Identifier (UID) Message Attribute
A 32-bit value assigned to each message, which when used with the
unique identifier validity value (see below) forms a 64-bit value
that MUST NOT refer to any other message in the mailbox or any
subsequent mailbox with the same name forever. Unique identifiers
are assigned in a strictly ascending fashion in the mailbox; as each
message is added to the mailbox it is assigned a higher UID than the
message(s) which were added previously. Unlike message sequence
numbers, unique identifiers are not necessarily contiguous.
The unique identifier of a message MUST NOT change during the
session, and SHOULD NOT change between sessions. Any change of
unique identifiers between sessions MUST be detectable using the
UIDVALIDITY mechanism discussed below. Persistent unique identifiers
are required for a client to resynchronize its state from a previous
session with the server (e.g., disconnected or offline access
clients); this is discussed further in [IMAP-DISC].
Note: The next unique identifier value is intended to
provide a means for a client to determine whether any
messages have been delivered to the mailbox since the
previous time it checked this value.
Here is the link with more info:
http://www.faqs.org/rfcs/rfc3501.html
What I would do, is also keep track of the InternalDate of the messages downloaded. This way, if you ever lose UID sync, you can at least iterate through the messages, and find the last one you downloaded, based upon the InternalDate of the message.
There's an imap flag called "seen". Most clients would mark a message seen when viewing the message, so you'd want to iterate over messages on the server which do not have that flag set.
Here's a code snippet which should give you the right idea. The operative bit of course is
imap.search(["NOT", "SEEN"]).each do bla.bla.bla
If you are you able to filter incoming mail into a specific IMAP folder on the server side, your app
can read new messages in that folder and then move them into the standard INBOX folder after it's done.