I'm trying to implement a webmail in PHP. I would like to write a PHP CLI script which is run on every email arrival to store some parts of (not all of) incoming email into database for search purposes. Then when the user finished searching and chose an email to show, a connection is made to mail server to retrieve the complete email. In order to implement this scenario I need to make some sort of connection among emails within database and mail server.
Since my knowledge of working with mail servers is limited to Zend Framework's API, what I believe I need in order to retrieve an email from an IMAP server is a message number or a message unique id (this later one seems not to be supported by all mail servers).
To this point, I've managed to find .forward (and some other ways) to introduce my PHP CLI script to MTAs to be run on every email arrival. This way I can store emails to database. But this won't do since message unique id is created by MDA so MTA do not know of it and they can not provide it to me. This means I can not find emails later when I want to retrieve them from mail server.
At last, here's my question: Is there a way to introduce a PHP CLI script to a MDA for emails' arrival? If this is dependent on the mail server, which servers do support this and how? My personal choice would be Dovecot or Courier, but any other mail server would do as well.
This is tricky -- there are many ways on how to setup delivery. Some of them work with the underlying mail store directly, bypassing your IMAP server altogether, while others use e.g. Dovecot's facilities.
Have you considered building on top of the notify plugin which ships with Dovecot?
It seems like it's impossible to introduce such a PHP CLI script to IMAP server (at least I'm sure of Dovecot). Anyway, the work around I found for this problem is to use my own PHP script to insert the new mails into IMAP server and retrieve their id's and then store the id in database for future references. To be clear, email are given to my PHP CLI script by MTA, not MDA. As I said before this is done easily using .forward file.
[UPDATE]
Unfortunately it seems this solution can not be implemented as well. The way to insert a new email to IMAP server is APPEND command, and to have the UID of the recently added mail server must support UIDPLUS extension. Neither Dovecot nor Courier supports this extension at the moment! If they did it seems the server would return the UID with a APPENDUID response.
[UPDATE]
It is my bad since Courier does support UIDPLUS. So this solution is valid and the one I'm going to implement.
Related
I'm in the process of planning the development of a mail-server to hand the sending of email across our multiple websites. Below is a description of what we are planning to implement and I'd like your opinion/suggestions.
We use ASP.NET MVC and have many web-sites hosted on Azure. We currently send mail internally within each of the web applications using SMTPServer.Send(). Obviously this is not the ideal way to send emails when you have a decently busy set of websites because the send mail call is blocking and cannot guarantee mails are sent. With this I'm worried out getting an influx of mail requests when we launch our next website (we think it'll get a decent amount of traffic and lots of emails will be sent).
My plan of action : to build a centralised mail-server that runs in the background (we use azure and this will be simply another web-application). When each one of our web applications wants to send a mail, instead of doing this internally, it'll call a web method on the mail-server called sendMail() this function will accept certain parameters and insert the mail parameters, content etc. into a database. The mail server will then poll the database at fixed time intervals, select a set of unsent emails and attempt to send them using the same SMTPServer.Send() function. If an email fails for some reason we won't flag it as sent and in the next poll interval the email will be selected again and another send attempt will be made. (we will cap the number of send attempts to say 20).
This will allow each of the websites to run smoothly without having loads of blocking send mail calls internally and the mail server will handle all the sending sequentially and in a controlled environment as a separate standalone web-application.
Thanks in advance!
Looks like a good design, Don't know the entire scenario which let to you building something like an email server. The problem has been solved well by using a service that already exist like Office 365.
Your design is good, My suggestions would be the following,
You can use Azure WebJobs to build the polling agent. You can make the web job run as a scheduled web job that does the polling and sending the mail and it can be written very clean as a simple console app.
You can use Azure API App to build the SendMail() call and you can use the Azure AD Auth on the API to authenticate the caller of the API using the Authentication and Authorization feature to easily secure your email server. You can also enable CORS easily as well to make sure you receive requests from other websites and process it.
Some issues I foresee for you,
Volume and Scaling : You can only process so much email between each polling. If you cannot then you will need to create another polling agent which will making things complicated as they need to know they are picking different sets of emails to send. If you volume is going to be low you should be fine.
Challenge : Why can't the websites send the mail them selves ? And then record it on the database for tracking. All you have to do build module or a component that they use on their web page to create and send the mail. Polymer 1.0 works well for this scenario.
Hope this helps to get you started.
I want an E-Mail to be sent using a background process whenever an Invite was generated.
What I currently have is this approach: The Invite model has the method send_mail, which sends an E-Mail using the Mandrill API and gem. It also has the method queue_mail adds InviteMailer with the invite's ID to the queue using Resque.
However… Since I'm having sort of a really hard time writing specs for this, I assume this might not be the best approach to send mails.
What I mainly want and need to test:
was the mail added to the queue?
is InviteMailer working properly?
does the mail contain the correct vital information?
Vital informations are: sent to the correct person, contains a link to a specific site and some specific data/text; also I'm not sure how to get the current host to the link.
I don't think this is a rare thing to do, so I wonder what the best practices are.
My testing environment: rspec, capybara, factory girl. I already added VCR, to cache the API-request.
You can use Mailcatcher to fake your mail server, and check received mail via web API:
Features
Catches all mail and stores it for display.
Shows HTML, Plain Text and Source version of messages, as applicable.
Rewrites HTML enabling display of embedded, inline images/etc and open links in a new window. (currently very basic)
Can send HTML for analysis by Fractal.
Lists attachments and allows separate downloading of parts.
Download original email to view in your native mail client(s).
Command line options to override the default SMTP/HTTP IP and port settings.
Mail appears instantly if your browser supports WebSockets, otherwise updates every thirty seconds.
Growl notifications when you receive a new message.
Runs as a daemon run in the background.
Sendmail-analogue command, catchmail, makes using mailcatcher from PHP a lot easier.
Written super-simply in EventMachine, easy to dig in and change.
How
gem install mailcatcher
mailcatcher
Go to http://localhost:1080/
Send mail through smtp://localhost:1025
API
A fairly RESTful URL schema means you can download a list of messages
in JSON from /messages, each message's metadata with
/messages/:id.json, and then the pertinent parts with
/messages/:id.html and /messages/:id.plain for the default HTML
and plain text version, /messages/:id/:cid for individual
attachments by CID, or the whole message with /messages/:id.source.
How would i go about to receive mails in a Ruby on Rails application without going through a mail server like PostFix or to fetch them by pop3 etc.
What i was to do is to catch all mails sent to #mydomain.com and just do something with them in my application. I don't need to store the mails or anything like that.
Is this posible?
I just implemented this for my SAAS to autoprocess mailer-bounce notification messages.
Call me, call you?
You call me
You can set up a local mail server. It would then respond to an incoming email, and start up a rails executable to process the email. This method is NOT recommended since starting up rails is a big task (takes multiple secs and lots of memory). You don't want a Rails bad boy started up just because you received an email. You'd be writing your own DDOS attack. (Attacking yourself.)
I call you
Instead, poll for email on your own schedule by using a single job to process all currently waiting emails. You need to set up a background job handler since stock rails is focused on responding to web requests. I use delayed_job, but there are other alternatives including kicking off a cron job every so in often.
Another benefit is that you don't need to manage a mail server. Leave that headache to someone else. Then use the Ruby library net::imap to read the incoming mail and process it.
If your process doesn't recognize the email format, then forward the msg to a human for processing.
And be sure that if the process sends mail in addition to reading/processing it, that the process uses a different email address as its From address. Otherwise, odds are good that sometime along the way, you'll end up in an email loop and many gigabytes of messages going back and forth. For example, your process receives a message, responds to it, but in the meantime the sender (a human) has switched on vacation response. And your robot then responds to the vacation response..... oops....
Writing your own mail server
Re:
How would i go about to receive mails in a Ruby on Rails application without going through a mail server like PostFix or to fetch them by pop3 etc.
What i was to do is to catch all mails sent to #mydomain.com and just do something with them in my application. I don't need to store the mails or anything like that.
Direct answer: Yes, you could do this by writing an smtp server and setting up dns so your machine will be the mail destination for the domain. Your smtp server would process the messages on the fly, they would not be stored on your system at any point.
Is this a good idea? No, not at all. While appearances may be to the contrary, email is a store and forward system. Trying to avoid storing the messages before your app processes them is not smart. It would be a very very poor "optimization." However, using an access protocol (POP3 or IMAP) is a good way to avoid the costs of installing, configuring and managing a mail server.
You can do this if you write your own mail server, or if your mail server supports hooks to run external programs upon receipt of mail (e.g. procmail).
If you don't have procmail available (or, if on something like Exchange Server, don't feel like writing custom rules or extensions), you're simply better off using a pop3 library to fetch mail.
Obviously, writing a mail server is more difficult than any of the alternatives.
If you're mostly worried about checking potentially hundreds of email accounts, that's solvable by configuring your email server properly. If you're on a hosted provider, ask your server administrator about creating a "catch-all" account that routes all mail to unknown addresses to a single account.
If you're aiming to avoid having to poll a server, consider the IMAP IDLE command. I've successfully written a Ruby client that opens a connection to an IMAP server, and gets told by the server when new mail arrives.
I would like to create an email client that can access multiple IMAP mailboxes. I'd also like a copy of all emails for processing. What is the best way to do this using IMAP commands?
Right now I have a script that iterates over the folders, FETCHing FLAGS on 1:* to see what's been read and if any previously read messages have been marked as new, then FETCH BODY.PEEK on all of the messages I don't have in my database. Is there a better way?
A better way would be to fetch UIDs of all messages (UID FETCH 1:* FLAGS), compare the resulting UID list with your database and then download any messages you don't have and remove any messages you have but the server doesn't (deleted by other IMAP clients or using a web interface, for example). This is the only reliable method to duplicate an IMAP folder, AFAIK.
(And don't forget to take UIDVALIDITY into account as well!)
Your original method would not work correctly if other IMAP clients were accessing the mailbox in addition to your application. In theory, it would work OK if you can stay connected to the IMAP server continuously, using NOOP and IDLE to check new and deleted messages, but this is never possible in practice - even GMail doesn't have 100% uptime :-)
An ultimate IMAP client would combine both these approaches.
I have this rails app that's running on our local intranet, with a thousand regular users. I am looking to integrate it with our email server(MS Exchange). Basically -
1) For each user, the app should fetch any new messages in their inbox from the mail-server, parse it, and file it in the database.
I could implement it with ruby/net-imap, but I am worried about performance. As I have mentioned there are a thousand users, each with a different email address. Polling the email server every x minutes wouldn for a thousand users could drastically reduce performance.
It looks like IMAP IDLE with ruby/net-imap could be an option, the mail server pushing a notification to my app whenever there's a new mail(IPhone/Outlook style). But it seems, for that i'll have to keep an imap connection open with the server for that. So if there are 100 users logged in at a time, that would mean 100 connections to mail-server. Will that drastically slow the app?
Or does anyone know of any other options (Btw, I don't have control over the mail-server)
We are using MRI, but could easily switch to jruby too.
This might help... I know it is for gmail or not exactly what you're looking for, but you might be able to adapt it to your problem.
There are also some libraries to handle imap with ruby such as Chilkat Ruby IMAP Library 2.1, but I haven't tried any yet.