I want to automatically send out emails to a list of users from my Rails app.
The volume ranges from really a few users (5-10) to groups of users (50-70).
The maximum would be all users (currently 5000).
I have understood that sending emails can block the Rails app, so sending emails should be done with the help of a queueing system and a background job, e.g. by using DelayedJob, Resque or Sidekiq.
Unfortunately, having such a background job requires a worker process on the Heroku platform. And I want to avoid that due to the increased cost (at least for the beginning).
Is there any alternative approach that I could take? E.g., a second Rails app with just one worker process which only does the email handling (well I guess that would then also result in a paid worker process)?
Is it possible to send out mass emails without such a worker process via SendGrid, MailGun or any other service that integrates nicely with Heroku?
Props to #phoet for his link
We've got a similar idea working on Heroku which we thought would be free. Turns out they billed us for the scheduler hours... but here's what we did:
Resque
Resque is a queueing system which runs on Rails to queue items in Redis. It's totally recommended to use this on Heroku, and is very efficient & scalable
It works like this:
Send "data" (typically ID's) to resque queue
Resque sends ID's to Redis
The Resque rake job processes the Redis Queue
Perform your mailout when queue is processed (sending the email to Mandrill / SendGrid)
The reason for having a queue is as #apneadiving said - your controller will timeout AND (more importantly), your Rails app will lock up until the process has completed
There is a very good Railscast on Resque here:
Code
This is just basic code - can add more if you want:
#app/controllers/messages_controller.rb
def send_message
id = params[:id]
message = Message.find(id).broadcast!
flash[:notice] = "Broadcast Queued!"
redirect_to root_path
end
#app/models/message.rb
def broadcast!
self.subscribers.each do |subscriber|
Resque.enqueue(MailoutQueue, id, subscriber.id, queue.id)
end
end
#app/workers/mailout_queue.rb
class MailoutQueue
#queue = :mailer
def self.perform(message_id, recipient_id, queue_id)
MessageMailer.send_message(message_id, recipient_id).deliver
end
end
class MessageMailer < ActionMailer::Base
default from: '****************'
def send_message(message_id, subscriber_id)
#Declarations
#message = Message.find(message_id)
#subscriber = Subscriber.find(subscriber_id)
#Send
mail(:to => #subscriber.email, :subject => #message.title)
end
end
Take a look here - https://github.com/stephenb/sendgrid#delivering-to-multiple-recipients
This allows you to send multiple emails with only one SMTP call.
Related
We're currently letting users email each other without seeing each other's actual email addresses (double blind) by letting them send emails to username#parse.example.com which works great.
class ForwardsMailbox < ApplicationMailbox
before_processing :ensure_users
def process
content = mail.multipart? ? mail.parts.first.body.decoded : mail.decoded
UserMailer.with(sender: sender, recipient: recipient, subject: mail.subject, content: content).forward_email.deliver_later
end
private
def sender
#sender ||= User.find_by(email: mail.from.first)
end
def recipient
#recipient ||= User.find_by(username: mail.to.first.split('#').first)
end
def ensure_users
bounce_with UserMailer.invalid_user(inbound_email) if sender.nil? or recipient.nil?
end
end
Is it possible to forward the whole mail object instead of extracting its contents, checking if it is multipart etc?
Try to give this a shot. Reuse the mail object in your process method and directly deliver the message yourself. You'll need to reach into ActionMailer to get your delivery methods configured correctly, but I believe it'll work.
def process
mail.to = sender.email
mail.from = recipient...
ActionMailer::Base.wrap_delivery_behavior(mail) # this sets delivery to use what we specified in our rails config.
mail.deliver # This delivers our email to the smtp server / API
end
How this works:
Behind the scenes Mailers are just calling deliver on a Mail object to send emails. You can go peruse through ActionMailer::MessageDelivery if you'd like to see that in action. We're just using that functionality directly here.
I'd recommend staying away from using Mailers in this instance because copying over all the fields from your original mail object to the mailer's mail object is going to require a lot of trial and error.
One thing to note: the headers remain unchanged when the message is re-delivered, so things like Message-ID will still be the same (which may or may not be a problem, just something to consider).
Last, if you're concerned at all about deliver being a blocking call to an API/SMTP server like I was, worry not! It looks like ActionMailbox already ensures that the process method runs via ActiveJob, so you shouldn't have to worry about an SMTP/API request taking a while and blocking a web request (see ActionMailbox guides).
I have a rails app in which I have a function that send out quite a few emails. I would like to do it asynchronously and I thought that the method deliver_later would do that. Currently I have some delay from when the user clicks submit until the form is submitted - which is leads to a bad user experience (It is a quite simple form). My implementation looks like this:
def create
respond_to do |format|
if #competition.save
[...]
send_notification_to_team_members
end
end
def send_notification_to_team_members
#team.members.each do |member|
unless member.user.eql?(current_user)
Mailer.deliver_new_competition_notification(member.user, #competition).deliver_later
end
end
end
Currently it takes ~ 4 seconds for the action to be finished. I have also tried:
Mailer.deliver_new_competition_notification(member.user, #competition).deliver_later(wait: 1.minute)
Then it takes even longer - I would estimate ~1 minute.
So, am I using deliver_later wrongly, or is the method not doing what I expect. In that case, is there another method I can use to improve my performance?
deliver_later uses ActiveJob to provide asynchronous execution.
However ActiveJob does not provide asynchronicity itself - it is a unifying api layer that can be fulfilled by many backends. The default one just runs everything inline it is not async.
To get async usage you need to pick an asynchronous backend. You can configure the backend in your application's config
config.active_job.queue_adapter = ...
In general most adapters require a corresponding gem (eg delayed_job, sidekiq, sucker_punch) which may have their own dependencies too (for example sidekiq requires that you use redis.
I have a typical ActionMailer with a method specifying an email delivery.
def some_email(user_id)
#user = User.find(user_id)
if #user.eligible_for_email?
mail(to: #user.email, from: "me#me.com", subject: "The Subject")
#user.email_sent = Date.today
#user.save
end
end
I want to delay the sending of this using Sidekiq so I use:
Mailer.delay_for(2.days).some_email(user.id)
The eligible_for_email method:
def eligible_for_email?
!unsubscribed? && email_sent.nil?
end
In the meantime, the user could have unsubscribed, which is why there is a method in the User model called eligible_for_email? which I can use to conditionally send the email - but obviously this condition needs to be tested just before the email is sent, not when the job is scheduled.
So the problem is that when I use Sidekiq to process this, the conditional logic doesn't seem to be run when the job is done.
Does Sidekiq work by executing the some_email method on runtime and then queuing the resulting email to be sent out two days later, thereby negating my conditional code?
Your understanding is 100% correct and that's exactly what you want to do.
I'd guess you aren't restarting Sidekiq to pick up your code changes. Sidekiq does not auto-reload changed code like Rails does.
I'm new in Rails developing, and I have one question. There the following code:
def create
#order = current_user.orders.create!(order_params)
OrderMailer.send_order_info(#order).deliver
end
This code creates a new order, render json result and send e-mail about it. Mail sending takes some time, and I think I should do it in another thread or something similar. Please, give me advice how I can do it good. Thanks!
You should use delay the email sending. You could do it using Sidekiq, Delayed Job or Resque for example.
You will also be able to delay any other jobs with these gems.
You should look at the docs and see which one is the best for your use.
I personally use Sidekiq but Delayed Job is the easiest to install if you only want to use it for mailer.
We use Spawn for this: it's changed its name to "Spawnling" now
https://github.com/tra/spawnling
Very easy to use: (in the controller)
#user = User.create(params[:user])
spawn do
#user.do_some_slow_background_stuff
end
or, if you want to monitor whether the background process has finished yet (#spawn_id is the pid)
#user = User.create(params[:user])
spawner = spawn do
#user.do_some_slow_background_stuff
end
#spawn_id = spawner.handle
I will be sending bulk emails from a Rails app and plan on using SendGrid. I am assuming that it is best to send a separate email to each recipient (as opposed to using BCC for all the recipients). If that is true, should I be using something like DelayedJob to queue the messages going over to SendGrid, or would it be safe to throw 500 messages at it all at once? Thanks!
500 messages really isn't that much to SendGrid. It's not even a blip on their radar. I worked for a company that sent out 2.7 million emails in a single month, and even then it's only just a blip.
With the SendGrid API's capabilities, you wouldn't be sending out 500 emails, you would send one email which has a specific SendGrid API header set. Why? Because have you ever tried to send 500 individual email messages and timed how long that takes? How about a single email? The single email's going to be quicker.
The SendGrid API has a Ruby example which is here:
https://sendgrid.com/docs/Integrate/Code_Examples/SMTP_API_Header_Examples/ruby.html.
That's quite long winded and messy, so let me simplify it for you. Basically, you set this in your email:
headers["X-SMTPAPI"] = { :to => array_of_recipients }.to_json
SendGrid will then parse this and then send that one email you sent it out to that array of recipients. I seem to recall that they ask you to limit this to about 1000 recipients per email, so it would be wise to split it up over multiple emails if you wanted that. That is when you would bring in something like the delayed_job or resque gems to deal with it.
Oh, and by the way you'll still need to specify a to address for this email just to make the Mail gem happy. We had info#ourcompany.com for that.
The SendGrid API will also support filters in their emails, so you can have placeholder strings such as {{ firstname }} and, assuming you send it through with the SMTPAPI header, it will do the "mail merge" on the email and customize them.
It would do you a great deal of good if you read the SendGrid API documentation. It's really useful and what they provide is super powerful.
I recommend using the sendgrid gem ( https://github.com/stephenb/sendgrid ) as it simplifies your calling code.
Here's an example rails 3 action mailer example:
class UserAnnouncementMailer < ActionMailer::Base
include SendGrid
default reply_to: "test#test.com", return_path: "test#test.com", from: "Test"
# bulk emailer
# params - opts a hash of
# emails: array of emails
#
def notice(opts={})
raise "email is nil" unless opts[:emails]
sendgrid_category :use_subject_lines
sendgrid_recipients opts[:emails]
name = "The Man"
to = "test#test.com"
from_name = "#{name} <theman#test.com>"
subject = "Important"
mail({from: from_name, to: to, subject: subject})
end
end
And the corresponding calling code. It's recommended to have the emails array to be < 1000 emails.
emails = ["alice#test.com", "bob#test.com"]
UserAnnouncementMailer.notice({:emails => emails}).deliver
See the sendgrid gem github readme for more details.
Delayed Job and SendGrid sound like the best option from what you say, but have you considered using one of the campaign mailers like Mailchimp instead? If you're sending out a lot of mails that are basically the same, they'll let you setup and campaign template and then fire a CSV of all the variables at it. They then effectively mail merge and fire them all out.
If however, you're only talking a few hundred you're on the right lines. SendGrid can easily handle the load, and you want to use Delayed Job so that you're not impacted by the performance of the SendGrid API should it not be favorable. Alternatively, look at Resque instead for sending mail as it may be more efficient.
I would imagine SendGrid can handle that kind of load. Most relay systems can. Also I would imagine if you sent the 500 in a CC API call, that their system would parse it and send them individually. I use Elastic Email (http://elasticemail.com) - and I know that this is how they handle it and it works great.
This is how I've done it in Rails 4
class NewsMailer < ApplicationMailer
include SendGrid
sendgrid_category :use_subject_lines
default from: 'My App! <support#myapp.com>'
def mass_mailer(news)
# Pass it in template
#news = news
# Custom method to get me an array of emails ['user1#email.com', 'user2#email.com',...]
array_of_emails = #news.recipients.pluck(:email)
# You can still use
# headers["X-SMTPAPI"] = { :to => array_of_emails }.to_json
sendgrid_recipients array_of_emails
mail to: 'this.will.be.ignored#ignore.me', subject: 'Weekly news'
end
end