I would like to create an email client that can access multiple IMAP mailboxes. I'd also like a copy of all emails for processing. What is the best way to do this using IMAP commands?
Right now I have a script that iterates over the folders, FETCHing FLAGS on 1:* to see what's been read and if any previously read messages have been marked as new, then FETCH BODY.PEEK on all of the messages I don't have in my database. Is there a better way?
A better way would be to fetch UIDs of all messages (UID FETCH 1:* FLAGS), compare the resulting UID list with your database and then download any messages you don't have and remove any messages you have but the server doesn't (deleted by other IMAP clients or using a web interface, for example). This is the only reliable method to duplicate an IMAP folder, AFAIK.
(And don't forget to take UIDVALIDITY into account as well!)
Your original method would not work correctly if other IMAP clients were accessing the mailbox in addition to your application. In theory, it would work OK if you can stay connected to the IMAP server continuously, using NOOP and IDLE to check new and deleted messages, but this is never possible in practice - even GMail doesn't have 100% uptime :-)
An ultimate IMAP client would combine both these approaches.
Related
I'm downloading my folders with a MCOIMAPFetchFoldersOperation which gives me a nice list of all my folders:
INBOX
[Gmail]
[Gmail]/All Mail
[Gmail]/Drafts
[Gmail]/Important
[Gmail]/Sent Mail
[Gmail]/Starred
[Gmail]/Trash
When I'm fetching the messages for my folders I get a good chunk of duplicates, since the same message can exist in INBOX, [Gmail]/All Mail and [Gmail]/Important et al. at the same time. I check for dupes with the messages UID, but a UID is only unique in a particular folder so that's useless in this case.
What would the most compatible approach be?
As noted in Gmail IMAP extensions, you can fetch the X-GM-MSGID attribute for a message. This value is unique across folders.
You could fetch the X-GM-MSGID value for all new messages, check which messages you have downloaded already, and download the ones that are missing.
This is what fetching a message id looks like in IMAP; not sure exactly how to do that in Mailcore2.
a006 FETCH 1 (X-GM-MSGID)
* 1 FETCH (X-GM-MSGID 1278455344230334865)
a006 OK FETCH (Success)
This is specific to Gmail. You can check whether the server supports it by looking for X-GM-EXT-1 in the CAPABILITY response. As far as I know, there is no standard way to do this; the IMAP RFCs don't have the concept of the same message being present in multiple mailboxes.
I am implementing an IMAP server, operating on messages shared with other systems and due to business needs users are not allowed to create or delete messages or mailboxes via the IMAP server.
In our production environment however, we often see the following three commands occur very often
15 EXAMINE "Drafts"
16 STATUS "Drafts"
17 CREATE "Drafts"
For all three we reply with a NO response, but when the user logs in again next time, we see the same commands being issued.
Is there a way to discourage the client from issuing these commands? The only folder that we have is INBOX.
In principle, a response code of CANNOT ("The operation violates some invariant of the server and can
never succeed") should get the message across, though I suspect few clients would react appropriately to such feedback.
I'm trying to implement a webmail in PHP. I would like to write a PHP CLI script which is run on every email arrival to store some parts of (not all of) incoming email into database for search purposes. Then when the user finished searching and chose an email to show, a connection is made to mail server to retrieve the complete email. In order to implement this scenario I need to make some sort of connection among emails within database and mail server.
Since my knowledge of working with mail servers is limited to Zend Framework's API, what I believe I need in order to retrieve an email from an IMAP server is a message number or a message unique id (this later one seems not to be supported by all mail servers).
To this point, I've managed to find .forward (and some other ways) to introduce my PHP CLI script to MTAs to be run on every email arrival. This way I can store emails to database. But this won't do since message unique id is created by MDA so MTA do not know of it and they can not provide it to me. This means I can not find emails later when I want to retrieve them from mail server.
At last, here's my question: Is there a way to introduce a PHP CLI script to a MDA for emails' arrival? If this is dependent on the mail server, which servers do support this and how? My personal choice would be Dovecot or Courier, but any other mail server would do as well.
This is tricky -- there are many ways on how to setup delivery. Some of them work with the underlying mail store directly, bypassing your IMAP server altogether, while others use e.g. Dovecot's facilities.
Have you considered building on top of the notify plugin which ships with Dovecot?
It seems like it's impossible to introduce such a PHP CLI script to IMAP server (at least I'm sure of Dovecot). Anyway, the work around I found for this problem is to use my own PHP script to insert the new mails into IMAP server and retrieve their id's and then store the id in database for future references. To be clear, email are given to my PHP CLI script by MTA, not MDA. As I said before this is done easily using .forward file.
[UPDATE]
Unfortunately it seems this solution can not be implemented as well. The way to insert a new email to IMAP server is APPEND command, and to have the UID of the recently added mail server must support UIDPLUS extension. Neither Dovecot nor Courier supports this extension at the moment! If they did it seems the server would return the UID with a APPENDUID response.
[UPDATE]
It is my bad since Courier does support UIDPLUS. So this solution is valid and the one I'm going to implement.
What's the best general technique for creating an IMAP client and keeping its local message store in sync with the server?
I guess I'm looking for the right way to figure out what's changed in an IMAP folder on the server since the last time I checked, and download those changes, to persist them to my local database... This would include messages no longer in the folder (deleted or moved), new messages, and changed messages...
I guess new messages is easy, I can grab the highest UID i have for a folder and then find messages since that UID. I'm not so sure about detecting messages that were deleted or moved though, or changed (maybe some flags changed on a message).
Thanks!
For sync, probably you need each folder all messages UID and flags.
You can compare local cached UIDs to server returned, with this you can dedect new messages and deleted(
Probably you should use some kind of hastable for search/compare, this will speed up all.
I am looking for the optimal way to acquire a list of the top X most recently active Gmail threads.
Background:
I am using Java accessing IMAP with OAuth in a Google Apps for Education domain. A Gmail Atom inbox feed is available which can list the last 20 threads containing unread messages. Access to this feed seems to be very fast much faster than anything I having managed to produce using OAuth/IMAP.
The advantage of using the IMAP approach over the Gmail Atom inbox feed is with IMAP I can access an arbitrary number of messages (not just 20), see read messages, get thread size, get any associated google labels, fetch quota details and check for flags. Essentially this will give my users a much more Gmail like experience (I only need a read only experience for our portal). My problem is IMAP access is significantly slower than the Atom feed. Comparison wise the IMAP method takes around 10 seconds whilst the Atom feed is usually returned within 2 seconds.
I am aware and have been working with the Gmail IMAP Extensions and Gmail Advanced Search syntax .
Current Method:
Imagine I want the top 40 threads from my IMAP inbox. Currently I download some arbitrary number of messages say (40 * 4), fetching only the X-GM-THRID. I iterate through these messages storing the thread id as I go (fetching more messages if required) until I either exhaust the list of inbox messages or I reach my target number of threads.
I then have a list of Gmail thread ids which I can use to perform an IMAP search (with an appropriate FetchProfile.Item's, depending on what message details are required).
I iterate through the search results producing something like (using the wonderful Google Guava/Google Collections Multimap):
Multimap<Long, Message> threadMultiMap = LinkedListMultimap.create();
and this is easily massaged into:
LinkedHashMap<Long, Message[]> threadMap;
Is there a better way than iterating through the INBOX until X distinct message threads have been identified?
Not actually an answer but a relevant query.
Mark, as per your api request, I'm posting a comment as an answer (http://code.google.com/p/java-gmail-imap/wiki/DisplayingGmailThreadBasedView)
Does your lib support 3 legged oauth, I tried looking for XoauthAuthenticator on the source in the repo and could not find it.
Thanks
Hi agallego,
I use java-gmail-imap with XOAUTH. There is nothing explicitly in JavaMail that requires any changes to work with XOAUTH.
If you look at the XOAUTH projects (google-mail-xoauth-tools and google-mail-xoauth-tool-java-two-legged) you will see that you can create a SASL provider that can be used to authenticate against Gmail. See e.g. XoauthAuthenticator.java
I hope this helps,
Mark