In other apps i've used ar_mailer to queue up mails and ar_sendmail to take them out of the queue in a seperate process. I've not been massively happy with this setup: the ar_sendmail process seems to silently die fairly often, and in the default configuration mails are deleted out of the emails table when sent: i'd rather keep them for future reference and just mark them has having been sent.
I'm adding emailing into another project now (which uses rails 2.3.8) and am wondering what other solutions have people used in rails for bulk emailing?
Grateful for any advice - max
I think the prevailing opinion is to use a bulk e-mail service like MailChimp (http://mailchimp.com) or Amazon Simple Email Service (http://aws.amazon.com/ses)
Related
I have some questions about ActionMailer :
How does Actionmailer connect to a smtp server ?
Are the connections concurrent or parallel if the number of emails high > 1000 ?
How will sending out emails like facebook does ( 1000's in numbers ) as immediate emails affect the ruby on rails application and how would actionmailer handle it ?
Any other solution/plugin to send out large number emails from a RoR application apart ActionMailer?
------------------------------------------------added :
We need to send out at least 1000 emails per 15 minutes . We are using a Notes Domino server as our smtp server .! what is the possible architecture for this kind of problem. We are already storing the emails in the database to send them later , but what is needed is the sending approach !
The usual thing is to create a background job to send email. ActionMailer is very good for single emails but does tend to run into trouble after sending multiple emails as each one can take several seconds to complete. That's why I created PostageApp to help solve those problems.
Some services on the market to help you with sending lots of email from Rails:
MailGun
SendGrid
PostmarkApp
MailChimp
Mailjet
PostageApp
All of these have ways of sending multiple messages with a single API call or SMTP transaction.
1) Actionmailer connects to your smtp server via a set of parameters including a host, port and protocol.
3) The effect will be a slow site as a result of the many synchronous tasks being executed.
2 & 4)
Actionmailer is a bit too slow to be sending out a ton of emails under load, remember that it is a synchronous operation and as such its not really the sort of thing you want to be doing a lot on a busy site.
To be honest you're better off not sending that quantity of email from your website. It's not really designed to be used in such a way. If I had to send that sort of volume I'd look at doing the work in the background, something like Delayed Job would work well here or one of the many async rails mailers found here would do the trick.
What you really want to look at here is the requirement that you're trying to fulfil, is it absolutely necessary that the website be responsible for sending the mail in a synchronous fashion? In most cases the answer to that question is no. If you can, you'll be far better off deferring this sort of task to another part of your system, keep your site as lean and focused as you can.
Simple solution here for you...
Sidekiq or Resque
I'd highly recommend Sidekiq as it's not near as server intense for running multiple workers for this one - only be careful with concurrency issues (make sure you don't have 2 workers pick up the same job and send duplicate emails that is).
Say you set 20 Sidekiq workers, each should be able to send an email every 2-4 seconds, you're looking at an easy 300-600 per minute.
DO NOT try to do this without background workers like Sidekiq, Resque, or DelayedJob. You will freeze your entire app if you try sending in app with any large amount of emails. Even sending activation emails in app and what not will cause you unnecessary slow down issues.
I'd have one Worker that handles the queueing periodically and another Worker class that handles the sending. We're using Resque (6 workers maybe?) for this on an older app (pre-sidekiq) to send around 500 emails every 5 minutes with no issues.
You can aways use a third party like someone mentioned. Sendgrid is decent. But that wasn't the question, this is how you do it yourself simply and easily.
You define the SMTP settings in a config file if left blank it uses sendMail local
concurrent
multiple handlers
Is there a bulk email plugin for Rails apps?
you may also do 1000.times do email.deliver but it will probably collapse ur server
I've read most of the other answers on this topic, but a lot of them related to either third-party services like MailChimp (which I'm not necessarily opposed to) or how not to upset the host's email server.
I believe this case is unique so that it'll contribute...
I have my own DigitalOcean droplet running a rails app. I need to send out 100-1000 emails every so often, each with a unique message (a link I'm using for tracking clicks originating from the email).
I'm also operating my own iRedMail server.
Can someone recommend how to best-handle this task? I was going to simply cycle through the list of emails and use the template.html.erb to drop in my link, but what types of problems might I run into?
Thank you!
You should decouple your Rails App from the mail sending so that you don't have to wait in your view for the mails to be sent (assuming that you click on something that triggers the start of your mail sending). Use something like delayed_job or another queueing mechanism that Rails offers and only queue up the sending job of the e-mails. Then when the queue comes to execute the particular job you can customize the message with an HTML part and a text part or whatever else you need and pass them on individually to your MTA.
I want to have a stable way to allow users to email posts to my app. I've been trying CloudMailin but need something more reliable and high performance / fast to respond.
Any Rails 3 - heroku solutions out there that can handle Incoming Email Ingest?
thank you
SendGrid is another option. Here's the page from the documentation about parsing incoming e-mail.
Hi I can only apologise if you have been having issues with CloudMailin. Perhaps we could try and rectify the issues you have been experiencing. Contact us via email and we can look into things. Emails should be arriving within milliseconds of sending. If they are not then there is something going on.
How would i go about to receive mails in a Ruby on Rails application without going through a mail server like PostFix or to fetch them by pop3 etc.
What i was to do is to catch all mails sent to #mydomain.com and just do something with them in my application. I don't need to store the mails or anything like that.
Is this posible?
I just implemented this for my SAAS to autoprocess mailer-bounce notification messages.
Call me, call you?
You call me
You can set up a local mail server. It would then respond to an incoming email, and start up a rails executable to process the email. This method is NOT recommended since starting up rails is a big task (takes multiple secs and lots of memory). You don't want a Rails bad boy started up just because you received an email. You'd be writing your own DDOS attack. (Attacking yourself.)
I call you
Instead, poll for email on your own schedule by using a single job to process all currently waiting emails. You need to set up a background job handler since stock rails is focused on responding to web requests. I use delayed_job, but there are other alternatives including kicking off a cron job every so in often.
Another benefit is that you don't need to manage a mail server. Leave that headache to someone else. Then use the Ruby library net::imap to read the incoming mail and process it.
If your process doesn't recognize the email format, then forward the msg to a human for processing.
And be sure that if the process sends mail in addition to reading/processing it, that the process uses a different email address as its From address. Otherwise, odds are good that sometime along the way, you'll end up in an email loop and many gigabytes of messages going back and forth. For example, your process receives a message, responds to it, but in the meantime the sender (a human) has switched on vacation response. And your robot then responds to the vacation response..... oops....
Writing your own mail server
Re:
How would i go about to receive mails in a Ruby on Rails application without going through a mail server like PostFix or to fetch them by pop3 etc.
What i was to do is to catch all mails sent to #mydomain.com and just do something with them in my application. I don't need to store the mails or anything like that.
Direct answer: Yes, you could do this by writing an smtp server and setting up dns so your machine will be the mail destination for the domain. Your smtp server would process the messages on the fly, they would not be stored on your system at any point.
Is this a good idea? No, not at all. While appearances may be to the contrary, email is a store and forward system. Trying to avoid storing the messages before your app processes them is not smart. It would be a very very poor "optimization." However, using an access protocol (POP3 or IMAP) is a good way to avoid the costs of installing, configuring and managing a mail server.
You can do this if you write your own mail server, or if your mail server supports hooks to run external programs upon receipt of mail (e.g. procmail).
If you don't have procmail available (or, if on something like Exchange Server, don't feel like writing custom rules or extensions), you're simply better off using a pop3 library to fetch mail.
Obviously, writing a mail server is more difficult than any of the alternatives.
If you're mostly worried about checking potentially hundreds of email accounts, that's solvable by configuring your email server properly. If you're on a hosted provider, ask your server administrator about creating a "catch-all" account that routes all mail to unknown addresses to a single account.
If you're aiming to avoid having to poll a server, consider the IMAP IDLE command. I've successfully written a Ruby client that opens a connection to an IMAP server, and gets told by the server when new mail arrives.
For my side project kwiqi, I use ActionMailer's 'receive' method to process incoming email messages for tracking my expenses. Heroku doesn't have a local mail server running that same code will not work. One solution I've thought of is to periodically hit a controller action that will pull messages from Gmail. Are there other solutions that are reasonable? Is anyone processing incoming emails in Heroku?
You can use sendgrid addon, and their parse api (http://wiki.sendgrid.com/doku.php?id=parse_api). I've written a short tutorial on how to do so here: http://nanceskitchen.com/2010/02/21/accept-incoming-emails-into-a-heroku-app-using-sendgrid/
I know that this is a little late but for anyone else that might find this useful in future we created the http:///CloudMailin.com addon for Heroku that should help you to receive email on Heroku Rails apps really easily.
Heroku support running workers using DelayedJob. Workers are resourced just like Dynos (you pay by the hour) and for this you get a dedicated resource to process your emails.
In the past I have used Cron calling a controller in my app. It's pretty effective.
If the hourly limitation is an issue, you can call your app from another location ... I have a cheap Dreamhost account for some of my non-priority sites that I have used as Cron systems.
There are also a number of ping and uptime services that you can use for this purpose as well ... simply pass these services your email controller.
A real limitation of Heroku currently is that the most rapid frequency they support for cron jobs is hourly.
I'd recommend using Gmail and using delayed job as an alternative to cron to set a more reasonable frequency. There is a good tutorial on setting this up at WiseJive