I have a system that reprocesses some code in some rare cases when a database transaction cannot be committed.
This can create situations where a call to our Mandrill API to send e-mails can be duplicated.
I was wondering if Mandrill had a way to identify each request? Ideally, I am looking for something exactly similar to Stripe's implementation of idempotent.
Thanks!
Related
I am using rails 5 along with Amazon SES.
I wanted to throttle emails before sending them because SES isn't able to handle bulk emails in a short span and mail jobs are failing.
So I wanted to add a sidekiq worker with rate limiting.
From where should I call the worker, mail_interceptor or after_actions.
PS. In general what is the difference between them?
There is vast difference.
mail_interceptor
When using an ActionMailer in Rails, there is a way to hook into an outbound message after calling "Deliver" method, but before delivery agents actually send it. You can think of these interceptors in the controller as before_action filter.
If you need to do any sort of modification before actually deliver, you will need to use an Interceptor.
after_action
This is a callback for controllers, use for perform something after completion of method or action.
Please refer this for more understanding.
The Problem
Based on my understanding of your question, the problem here is a low level design question of where to place the code to throttle and send emails.
Difference between after_action and mail_interceptor as per my understanding
The use case of after_action,fairly straight forward, is to do something after certain action is executed.
The mail_interceptor on the other hand, is a hook that can be added in action mailer's mail delivery life cycle. As I understand, the purpose of the interceptor, is to modify certain email content or restrict sending email to users or restricting sending emails in certain environment before handling to the delivery agent. The use case is more concerned with whether to send or what to send or to whom to send rather than how to send.
Where to place the code?
Also, the interceptor gets called on all email deliveries, since it is hooked to the life cycle of . For example, in case of a reset password email, I am assuming there is no necessity to throttle and is not necessary to call this piece of code.
Based on the above assumptions and understanding, my opinion is placing the code to throttle emails in the after_action makes more sense than placing it in the mail_interceptor.
Hope this was helpful!
I've read most of the other answers on this topic, but a lot of them related to either third-party services like MailChimp (which I'm not necessarily opposed to) or how not to upset the host's email server.
I believe this case is unique so that it'll contribute...
I have my own DigitalOcean droplet running a rails app. I need to send out 100-1000 emails every so often, each with a unique message (a link I'm using for tracking clicks originating from the email).
I'm also operating my own iRedMail server.
Can someone recommend how to best-handle this task? I was going to simply cycle through the list of emails and use the template.html.erb to drop in my link, but what types of problems might I run into?
Thank you!
You should decouple your Rails App from the mail sending so that you don't have to wait in your view for the mails to be sent (assuming that you click on something that triggers the start of your mail sending). Use something like delayed_job or another queueing mechanism that Rails offers and only queue up the sending job of the e-mails. Then when the queue comes to execute the particular job you can customize the message with an HTML part and a text part or whatever else you need and pass them on individually to your MTA.
I'm building a web application were users need to ability to send out mass emails. The application is a ticket site where individuals sell tickets to various events. In turn, events have promoters that sell tickets. The sponsor of the event needs to ability to send mass emails to their promoters. An event may have hundreds of promoters. So I'm assuming looping through each promoter and sending an email wont cut it.
Does rails 3.0 provide some kind of mass email functionality?
Thanks,
Brian
I don't think you should have a problem looping through all the promoters and sending each one an email. However, you would want to do that as a background process.
Check out delayed-job or http://www.simpleworker.com/. Both of these will let you load the email into a background job, and allow you to redirect the user in a timely manner while the emails get sent in the background.
Solomon is right, you'll definitely want to do this in a background task, which is pretty simple with something like delayed job. Have you thought about how the messages will actually be delivered though? What mail server you are going to use? Many providers won't let you send out hundreds of messages at a time, and you are likely to run into spam issues if you try to send that volume of messages from a personal email account. You may want to take a look at a service like Mailgun that specializes in this. From their FAQ:
Why not just use Sendmail + Postfix + Courier IMAP?
You can but you should be aware that there is a constant battle raging
between good and evil (i.e., spam) in the email universe. In order to
be on the 'good' side of that battle and get your email delivered
there are numerous things you need to do. You need to have the right
infrastructure and register your IP and Domain appropriately. Also,
you need to have a history of email sending that complies with ESPs
rules in order to build a good reputation.
Moreover, if you are going to receive, store and host emails, you
better be prepared for maintaining this orchestra of software, take
care of backups, hardware failures, security patches and monitoring.
Stop kidding yourself, it's not 1998 anymore.
I'm not affiliated with them in any way, but in my experience getting rails to send email is trivial compared to dealing with getting it delivered successfully by the mail server.
In other apps i've used ar_mailer to queue up mails and ar_sendmail to take them out of the queue in a seperate process. I've not been massively happy with this setup: the ar_sendmail process seems to silently die fairly often, and in the default configuration mails are deleted out of the emails table when sent: i'd rather keep them for future reference and just mark them has having been sent.
I'm adding emailing into another project now (which uses rails 2.3.8) and am wondering what other solutions have people used in rails for bulk emailing?
Grateful for any advice - max
I think the prevailing opinion is to use a bulk e-mail service like MailChimp (http://mailchimp.com) or Amazon Simple Email Service (http://aws.amazon.com/ses)
How would i go about to receive mails in a Ruby on Rails application without going through a mail server like PostFix or to fetch them by pop3 etc.
What i was to do is to catch all mails sent to #mydomain.com and just do something with them in my application. I don't need to store the mails or anything like that.
Is this posible?
I just implemented this for my SAAS to autoprocess mailer-bounce notification messages.
Call me, call you?
You call me
You can set up a local mail server. It would then respond to an incoming email, and start up a rails executable to process the email. This method is NOT recommended since starting up rails is a big task (takes multiple secs and lots of memory). You don't want a Rails bad boy started up just because you received an email. You'd be writing your own DDOS attack. (Attacking yourself.)
I call you
Instead, poll for email on your own schedule by using a single job to process all currently waiting emails. You need to set up a background job handler since stock rails is focused on responding to web requests. I use delayed_job, but there are other alternatives including kicking off a cron job every so in often.
Another benefit is that you don't need to manage a mail server. Leave that headache to someone else. Then use the Ruby library net::imap to read the incoming mail and process it.
If your process doesn't recognize the email format, then forward the msg to a human for processing.
And be sure that if the process sends mail in addition to reading/processing it, that the process uses a different email address as its From address. Otherwise, odds are good that sometime along the way, you'll end up in an email loop and many gigabytes of messages going back and forth. For example, your process receives a message, responds to it, but in the meantime the sender (a human) has switched on vacation response. And your robot then responds to the vacation response..... oops....
Writing your own mail server
Re:
How would i go about to receive mails in a Ruby on Rails application without going through a mail server like PostFix or to fetch them by pop3 etc.
What i was to do is to catch all mails sent to #mydomain.com and just do something with them in my application. I don't need to store the mails or anything like that.
Direct answer: Yes, you could do this by writing an smtp server and setting up dns so your machine will be the mail destination for the domain. Your smtp server would process the messages on the fly, they would not be stored on your system at any point.
Is this a good idea? No, not at all. While appearances may be to the contrary, email is a store and forward system. Trying to avoid storing the messages before your app processes them is not smart. It would be a very very poor "optimization." However, using an access protocol (POP3 or IMAP) is a good way to avoid the costs of installing, configuring and managing a mail server.
You can do this if you write your own mail server, or if your mail server supports hooks to run external programs upon receipt of mail (e.g. procmail).
If you don't have procmail available (or, if on something like Exchange Server, don't feel like writing custom rules or extensions), you're simply better off using a pop3 library to fetch mail.
Obviously, writing a mail server is more difficult than any of the alternatives.
If you're mostly worried about checking potentially hundreds of email accounts, that's solvable by configuring your email server properly. If you're on a hosted provider, ask your server administrator about creating a "catch-all" account that routes all mail to unknown addresses to a single account.
If you're aiming to avoid having to poll a server, consider the IMAP IDLE command. I've successfully written a Ruby client that opens a connection to an IMAP server, and gets told by the server when new mail arrives.