Fetching gmail emails using .NET MVC - asp.net-mvc

Question

I'm trying to create a little web application to act as a web mail client for Gmail...
I've used the following code to fetch the emails from my inbox:
public ActionResult Index()
{
using (var client = new ImapClient())
{
using (var cancel = new CancellationTokenSource())
{
ServicePointManager.ServerCertificateValidationCallback += (o, c, ch, er) => true;
client.Connect("imap.gmail.com", 993, true, cancel.Token);
// If you want to disable an authentication mechanism,
// you can do so by removing the mechanism like this:
client.AuthenticationMechanisms.Remove("XOAUTH");
client.Authenticate("********#gmail.com", "****", cancel.Token);
// The Inbox folder is always available...
var inbox = client.Inbox;
inbox.Open(FolderAccess.ReadOnly, cancel.Token);
m = new List<string>();
// download each message based on the message index
for (int i = 0; i < inbox.length; i++)
{
var message = inbox.GetMessage(i, cancel.Token);
m.Insert(i, message.TextBody);
}
client.Disconnect(true, cancel.Token);
}
}
return View(m.ToList());
}
The reason why I dislike this is way of doing is that this part of code:
for (int i = 0; i < inbox.length; i++)
{
var message = inbox.GetMessage(i, cancel.Token);
m.Insert(i, message.TextBody);
}
It takes so long to fetch all the emails, approximately 40 emails are fetched each 5 seconds... So if someone has 2000 emails, it'd take 20 minutes to load all the emails...
Is there any faster way to load all the emails into my MVC application? :/
P.S. I've tried doing it with email which has 10000 emails, and it takes forever to fetch all the emails....

Answer 1

If all you want is the text body of the message, you could potentially reduce IMAP traffic by using the following approach:
var messages = inbox.Fetch (0, -1, MessageSummaryItems.UniqueId | MessageSummaryItems.BodyStructure);
int i = 0;
foreach (var message in messages) {
var part = message.TextBody;
if (part != null) {
var body = (TextPart) inbox.GetBodyPart (message.UniqueId, part);
m.Insert (i, body.Text);
} else {
m.Insert (i, null);
}
i++;
}
What this does is send a batched FETCH request to the IMAP server requesting an "outline" (aka body structure) of the message and its unique identifier.
The loop that follows it then looks through the structure of the message to locate which MIME part contains the message's text body and then fetches only that particular sub-section of the message.
In general, you do not watch to download every message over IMAP. The purpose of IMAP is to leave all of the messages on the IMAP server and just fetch the least amount of data possible that you need in order to display whatever it is you want to display to the user.
It should also be noted that you don't actually need to use a CancellationTokenSource unless you are actually planning on being able to cancel the operations.
For example, your code snippet could be replaced with:
public ActionResult Index()
{
using (var client = new ImapClient())
{
ServicePointManager.ServerCertificateValidationCallback += (o, c, ch, er) => true;
client.Connect("imap.gmail.com", 993, true);
// If you want to disable an authentication mechanism,
// you can do so by removing the mechanism like this:
client.AuthenticationMechanisms.Remove("XOAUTH");
client.Authenticate("********#gmail.com", "****");
// The Inbox folder is always available...
var inbox = client.Inbox;
inbox.Open(FolderAccess.ReadOnly);
m = new List<string>();
var messages = inbox.Fetch (0, -1, MessageSummaryItems.UniqueId | MessageSummaryItems.BodyStructure);
int i = 0;
foreach (var message in messages) {
var part = message.TextBody;
if (part != null) {
var body = (TextPart) inbox.GetBodyPart (message.UniqueId, part);
m.Insert (i, body.Text);
} else {
m.Insert (i, null);
}
i++;
}
client.Disconnect(true);
}
return View(m.ToList());
}
Since you are writing your own webmail front-end to GMail, you may find the following suggestion useful:
When you look at the GMail webmail user interface or Yahoo Mail!'s user interface, you've probably noticed that they only show you the most recent 50 or so messages and you have to specifically click a link to show the next set of 50 messages and so on, right?
The reason for this is because it is inefficient to query the full list of messages and download them all (or even just the text bodies of all of the messages).
What they do instead is ask for just 50 messages at a time. And in fact, they don't ask for the messages at all, they ask for the summary information like so:
var all = inbox.Search (SearchQuery.All);
var uids = new UniqueIdSet ();
// grab the last 50 unique identifiers
int min = Math.Max (all.Count - 50, 0);
for (int i = all.Count - 1; i >= min; i--)
uids.Add (all[i]);
// get the summary info needed to display a message-list UI
var messages = inbox.Fetch (uids, MessageSummaryItems.UniqueId |
MessageSummaryItems.All | MessageSummaryItems.BodyStructure);
foreach (var message in messages) {
// the 'message' will contain a whole bunch of useful info
// to use for displaying a message list such as subject, date,
// the flags (read/unread/etc), the unique id, and the
// body structure that you can use to minimize your query when
// the user actually clicks on a message and wants to read it.
}
Once the user clicks a message to read it, then you can use the message.Body to figure out which body parts you actually need to download in order to display it to the user (i.e. avoid downloading attachments, etc).
For an example of how to do this, check out the ImapClientDemo sample included in the MailKit GitHub repo: https://github.com/jstedfast/MailKit