How to to mass email (500,000) using rails [duplicate] - ruby-on-rails

I will be sending bulk emails from a Rails app and plan on using SendGrid. I am assuming that it is best to send a separate email to each recipient (as opposed to using BCC for all the recipients). If that is true, should I be using something like DelayedJob to queue the messages going over to SendGrid, or would it be safe to throw 500 messages at it all at once? Thanks!

500 messages really isn't that much to SendGrid. It's not even a blip on their radar. I worked for a company that sent out 2.7 million emails in a single month, and even then it's only just a blip.
With the SendGrid API's capabilities, you wouldn't be sending out 500 emails, you would send one email which has a specific SendGrid API header set. Why? Because have you ever tried to send 500 individual email messages and timed how long that takes? How about a single email? The single email's going to be quicker.
The SendGrid API has a Ruby example which is here:
https://sendgrid.com/docs/Integrate/Code_Examples/SMTP_API_Header_Examples/ruby.html.
That's quite long winded and messy, so let me simplify it for you. Basically, you set this in your email:
headers["X-SMTPAPI"] = { :to => array_of_recipients }.to_json
SendGrid will then parse this and then send that one email you sent it out to that array of recipients. I seem to recall that they ask you to limit this to about 1000 recipients per email, so it would be wise to split it up over multiple emails if you wanted that. That is when you would bring in something like the delayed_job or resque gems to deal with it.
Oh, and by the way you'll still need to specify a to address for this email just to make the Mail gem happy. We had info#ourcompany.com for that.
The SendGrid API will also support filters in their emails, so you can have placeholder strings such as {{ firstname }} and, assuming you send it through with the SMTPAPI header, it will do the "mail merge" on the email and customize them.
It would do you a great deal of good if you read the SendGrid API documentation. It's really useful and what they provide is super powerful.

I recommend using the sendgrid gem ( https://github.com/stephenb/sendgrid ) as it simplifies your calling code.
Here's an example rails 3 action mailer example:
class UserAnnouncementMailer < ActionMailer::Base
include SendGrid
default reply_to: "test#test.com", return_path: "test#test.com", from: "Test"
# bulk emailer
# params - opts a hash of
# emails: array of emails
#
def notice(opts={})
raise "email is nil" unless opts[:emails]
sendgrid_category :use_subject_lines
sendgrid_recipients opts[:emails]
name = "The Man"
to = "test#test.com"
from_name = "#{name} <theman#test.com>"
subject = "Important"
mail({from: from_name, to: to, subject: subject})
end
end
And the corresponding calling code. It's recommended to have the emails array to be < 1000 emails.
emails = ["alice#test.com", "bob#test.com"]
UserAnnouncementMailer.notice({:emails => emails}).deliver
See the sendgrid gem github readme for more details.

Delayed Job and SendGrid sound like the best option from what you say, but have you considered using one of the campaign mailers like Mailchimp instead? If you're sending out a lot of mails that are basically the same, they'll let you setup and campaign template and then fire a CSV of all the variables at it. They then effectively mail merge and fire them all out.
If however, you're only talking a few hundred you're on the right lines. SendGrid can easily handle the load, and you want to use Delayed Job so that you're not impacted by the performance of the SendGrid API should it not be favorable. Alternatively, look at Resque instead for sending mail as it may be more efficient.

I would imagine SendGrid can handle that kind of load. Most relay systems can. Also I would imagine if you sent the 500 in a CC API call, that their system would parse it and send them individually. I use Elastic Email (http://elasticemail.com) - and I know that this is how they handle it and it works great.

This is how I've done it in Rails 4
class NewsMailer < ApplicationMailer
include SendGrid
sendgrid_category :use_subject_lines
default from: 'My App! <support#myapp.com>'
def mass_mailer(news)
# Pass it in template
#news = news
# Custom method to get me an array of emails ['user1#email.com', 'user2#email.com',...]
array_of_emails = #news.recipients.pluck(:email)
# You can still use
# headers["X-SMTPAPI"] = { :to => array_of_emails }.to_json
sendgrid_recipients array_of_emails
mail to: 'this.will.be.ignored#ignore.me', subject: 'Weekly news'
end
end

Related

How to handle forwarded e-mails in Rails ActionMailbox?

We are using Rails 7 to build an application which, amongst other features, should perform some actions when e-mails are sent to one of its e-mail addresses (which have, for instance, the format ticket-{uuid}#ourdomain.com).
Rails' ActionMailbox's routing works fine for direct e-mails. However, when e-mails are forwarded, they are not recognized by ActionMailbox at all.
How can we ensure that forwarded e-mails are also handled and routed correctly with ActionMailbox?
EDIT: A simplified version of the code we are using:
class ApplicationMailbox < ActionMailbox::Base
routing /^ticket-(.+)#ourdomain.com$/i => :service_tickets
end
class ServiceTicketsMailbox < ApplicationMailbox
def process
puts "processing email: #{mail.inspect}"
# ... and then we extract its fields
# and store some of them in the database.
end
end
Ok I think I found the issue:
When you send a normal e-mail the To header looks like this To: ticket-123##ourdomain.com and this matches with /^ticket-(.+)#ourdomain.com$/i
However when you forward the e-mail the header looks like this To: John Doe <ticket-123#ourdomain.com> and this will not match with your regex.
Change the regex to /ticket-(.+)#ourdomain.com/i and it should work.
You can try it out on https://regexr.com/
The API never includes functionality to see any other mail-address than the latest one in the delivery stack.
The only option the API offers by itself is to use InboundEmail::source to get the raw message for further parsing.
After this process I could imagine to use RoutingJob to forward the mail to the correct receiver.
I'm not sure if Callbacks could help.
Also concerning MessageId I don't know if it's possible to extract the correct Ids.
As far as I see the whole challenge requires at least some work and I see there no simple solution.

Is it possible to forward the mail object in Action Mailer

We're currently letting users email each other without seeing each other's actual email addresses (double blind) by letting them send emails to username#parse.example.com which works great.
class ForwardsMailbox < ApplicationMailbox
before_processing :ensure_users
def process
content = mail.multipart? ? mail.parts.first.body.decoded : mail.decoded
UserMailer.with(sender: sender, recipient: recipient, subject: mail.subject, content: content).forward_email.deliver_later
end
private
def sender
#sender ||= User.find_by(email: mail.from.first)
end
def recipient
#recipient ||= User.find_by(username: mail.to.first.split('#').first)
end
def ensure_users
bounce_with UserMailer.invalid_user(inbound_email) if sender.nil? or recipient.nil?
end
end
Is it possible to forward the whole mail object instead of extracting its contents, checking if it is multipart etc?
Try to give this a shot. Reuse the mail object in your process method and directly deliver the message yourself. You'll need to reach into ActionMailer to get your delivery methods configured correctly, but I believe it'll work.
def process
mail.to = sender.email
mail.from = recipient...
ActionMailer::Base.wrap_delivery_behavior(mail) # this sets delivery to use what we specified in our rails config.
mail.deliver # This delivers our email to the smtp server / API
end
How this works:
Behind the scenes Mailers are just calling deliver on a Mail object to send emails. You can go peruse through ActionMailer::MessageDelivery if you'd like to see that in action. We're just using that functionality directly here.
I'd recommend staying away from using Mailers in this instance because copying over all the fields from your original mail object to the mailer's mail object is going to require a lot of trial and error.
One thing to note: the headers remain unchanged when the message is re-delivered, so things like Message-ID will still be the same (which may or may not be a problem, just something to consider).
Last, if you're concerned at all about deliver being a blocking call to an API/SMTP server like I was, worry not! It looks like ActionMailbox already ensures that the process method runs via ActiveJob, so you shouldn't have to worry about an SMTP/API request taking a while and blocking a web request (see ActionMailbox guides).

How to get email reply on rails app from email server

I need to implement messaging feature. When I send message to any user from my rails app then it also goes to users email. But what I need to implement if user who got the email make reply from gmail,yahoo..etc then the reply should also come into rails app. could anyone guide me some way.. so I can search it on google.
For Example:
If I send email to user from this email "mc-6bckm434ls#reply.xyz.com" and user replied on gspq5diedss#reply.xyz.com which I set Reply-To in header. Then I need user's reply in my rails application so that I can add this user's reply into my messaging thread.
By this feature which I want to implement User do not need to login in my application to do message, user can do message on current conversation via email reply also.
Since you tagged the question SendGrid, I'm assuming you're using it. You can use SendGrid's Inbound Parse Webhook to handle parsing incoming messages.
We also have a recent tutorial that goes through using the webhook in a rails app: http://sendgrid.com/blog/two-hacking-santas-present-rails-the-inbound-parse-webhook/
It's possible !
You can use mailman for example.
All you have to do is set a Reply-To header in your e-mail to make it unique, so when you fetch messages you know to what it corresponds.
For example, let's say you own the e-mail address foo#bar.com
You could send e-mails with a reply-to header "foo+content_id#bar.com", so you know what the user is replying to.
Then, mailman can fetch the messages from the mailbox and parse the content id in them.
Some services do that as well, handling all the emailing part and sending you notifications for incoming e-mails, for example, postmark
Ruby on Rails introduced Action Mailbox in version 6.0
With ActionMailbox you can easily configure rules where to route incoming emails (examples from the documentation):
# app/mailboxes/application_mailbox.rb
class ApplicationMailbox < ActionMailbox::Base
routing /^save#/i => :forwards
routing /#replies\./i => :replies
end
And how to handle specific messages in a mailbox:
# app/mailboxes/forwards_mailbox.rb
class ForwardsMailbox < ApplicationMailbox
# Callbacks specify prerequisites to processing
before_processing :require_forward
def process
if forwarder.buckets.one?
record_forward
else
stage_forward_and_request_more_details
end
end
private
def require_forward
unless message.forward?
# Use Action Mailers to bounce incoming emails back to sender – this halts processing
bounce_with Forwards::BounceMailer.missing_forward(
inbound_email, forwarder: forwarder
)
end
end
def forwarder
#forwarder ||= Person.where(email_address: mail.from)
end
def record_forward
forwarder.buckets.first.record \
Forward.new forwarder: forwarder, subject: message.subject, content: mail.content
end
def stage_forward_and_request_more_details
Forwards::RoutingMailer.choose_project(mail).deliver_now
end
end
Find the documentation about how to configure Action Mailbox and some examples in the Rails Guides.
See https://github.com/titanous/mailman/blob/master/USER_GUIDE.md
There is a railscast about using the gem but its pay-only :-/ http://railscasts.com/episodes/313-receiving-email-with-mailman,
however, there is the github repo from the railscast which can showcase you an example app that uses mailman (with before & after so you can track the changes) - https://github.com/railscasts/313-receiving-email-with-mailman

ActionMailer workflow

I am newbie in Rails, therefore sorry for silly question. For sending emails I use ActionMailer with Rails 2.3.5. The syntax is following
deliver_maintest1
deliver_maintest2
in model of instance of ActionMailer I have
def maintest1
end
def maintest2
end
Inside definitions I set recipient, subject, headers,...As I understand there is no any explicity defined method mail which is actually sent email. Emails are sent from def maintest1 and maintest2. The problem is before sending email I need to define few counters how many emails were sent thought maintest1 and maintest2. Now take into account I have tens defs like maintest. So I need a common place for all those defs. In your opinion what's the best solution?
Thanks!
On rails 3 and above you could use an observer. These get called after every mail delivery, passing through the message object. You just need to implement a delivered_email class method and register it.
class EmailObserver
def self.delivered_email(message)
# do something with message
end
end
Then, hook it into mail with
Mail.register_observer(EmailObserver)
This doesn't work on rails 2.x, which doesn't use the mail gem (it uses tmail from the ruby standard library.)
On 2.3.x I would try something like
class MyMailer < ActionMailer::Base
def deliver!(mail=#mail)
super
# do your logging here
end
end
You would be calling "Mailer.deliver_maintest" to send the mails out to anyone, to count the nos of times you sent a particular email you just need to keep track of it each time you call "Mailer.deliver_maintest" .
You can store that counter either the database or somewhere. something like this.
// Some lines of code to Update the counter for the mailer
Mailer.deliver_maintest
You can also use a third party email tool like PostMark to send your email ( with them you can associate each email with tags, and I just generally use those tags to keep track of emails sent out ).

Checking for new emails from within Rails

Within Rails, how can I listen for new mail coming into the boxes of certain accounts. I want to implement a simple "post-by-email" service, which gets the new content of the email and makes a blog post or sth.
You could try this gem.
I wrote a post explaining some of the options. A later post also covered my way of testing these options. The mail gem is great for parsing. You just have to decide the best option for you to optain the messages.
http://steve.dynedge.co.uk/2010/09/07/incoming-email-in-rails-3-choosing-the-right-approach/
I use this code to parse my emails.
class Receiver < ActionMailer::Base
def self.parse(email)
reply_separator = /(.*?)\s?== ADD YOUR REPLY ABOVE THIS LINE ==/m
comment_text = reply_separator.match(email.body.to_s)
# ...
end
end
The email object here is just a Mail::Message object which I get from using the gmail gem to read an inbox. If you're not using GMail then you should be able to use plain ol' vanilla Mail gem to connect to the mail server and then get the Mail::Message objects that way.

Resources