I just learned/started using Sidekiq today to handle background processing of incoming email messages and attachments, but am a bit lost on the best way to get the email body and attachments into the worker for processing.
My RoR app is hosted on Heroku and receives incoming emails via Mailgun to a controller, which then kicks off my worker. Within the worker is a call to a 3rd party API to upload my email messages and attachments (think DropBox.)
Mailgun pre-parses everything and sends it over as parameters, but from what I understand about Sidekiq, I don't want to pass along entire objects such as the email body and/or attachments as shown here.
#attach_count = params["attachment-count"]
#from = params["from"]
#subject = params["subject"]
#msgbody = params["body-html"]
ProcessEmailWorker.perform_async(#id, #attach_count, #from, #subject, #msgbody)
What's the best practice for getting these items over to my worker?
I assume Mailgun is POSTing to your controller.
You can send the POST body as a single string parameter to Sidekiq and have it re-parse everything.
You can save the data to Redis or your database for processing in Sidekiq.
You can send the email content as a Hash of Strings:
{ 'subject' => ..., 'body' => ... }
After speaking with another developer I chose to do the following:
Set up a route in Mailgun to store the incoming email message, but to post a notification to my controller
Have my controller grab the incoming message ID and pass that along to my worker
From within my worker, use the message ID to perform a GET to Mailgun to retrieve the stored message (and its attachments)
Process the message/attachments and upload them to my cloud storage provider.
Related
I'm using this repo to create a chat system between 2 users in a Rails and React project. I've been able to log the user input into the console, and I have created messages_controller and message_threads_controller according to the repo.
However, I'm unable to persist the message to Rails db and then authenticate a user with Pusher before sending it to Pusher. Mainly because the from_uid, to_uid and thread_uid are not present by the time the message is been sent to Rails. Sending the message to rails like this:
sendMessage = (event) => {
event.preventDefault();
const {message} = this.state;
axios.post('/api/v1/messages', {message: message})
.then((response) => {
console.log(response);
})
console.log('send Message')
this.setState({'message': message});
console.log(this.state.message);
}
In my routes.rb file I have this
resources :messages
get 'threads/:id', to: 'message_threads#index'
post '/pusher/auth', to: 'pusher#auth'
I'm missing some required parameters, this is the error I get.
Pusher::Error - Bad request: Missing required parameter:
The flow according to this tutorial is that the message needs to be persisted first by the rails database before sending it to Pusher.
My question now is how do I produce the extra parameters (from_uid, thread_uid, to_uid) being used on the React side of the app here, to enable messages to be created?
Also, how do I authenticate the user using Pusher?
According to this Stack Overflow link they are getting from Rails the CSRF value like this - csrf = $('meta[name=csrf-token]').attr('content'). But I could not implement the same in React.
Answer from the author of the git repo.
The example I created here was pretty bare bones but below are a few bullet points that I hope will explain how you could expand on it.
Add a User model and Thread model to the Rails app
When a User is created, generate a public UID for the user (you can use Ruby's built-in SecureRandom.uuid to generate the id) and save that to the DB. This would become the ID for that user that you would expose in your javascript to allow for communications between users. All users would have a UID.
When a Thread is Created, generated a thread UID this would become the unique id for the conversation taking place
Add a Users_Threads has_and_belongs_to_many relationship so that you can eventually track the users that are subscribed to which threads
When React app loads, use an Ajax request to get a list of the current User's friends (returns list of user details + their uid) and a request to get all threads for current User
So let's say for example a User named Fred clicked on a User named Bob and wanted to send Bob a message but they do not currently have a thread. Fred types the message, clicks Submit and you send an Ajax request containing the message text, from_uid (Fred) and to_uid (Bob) with thread_uid as null (since there is no existing convo and no existing thread).
Your Rails app then receives that requests at a controller and sees that Fred is trying to send Bob a message and the thread ID is null, so the controller create a new thread (with its own UID) and then add two entries to users_threads one for the new thread_uid and bob's uid and another for the new thread_uid and Fred's uid. After that, you'd create a Message with that thread_uid and the participant details.
You'd also probably want users to see that they are part of a new thread without having to reload the page so you'd I think you'd want a Pusher channel subscription just for notifying users of a new thread. So I'd say in the UserThreads model after create a new thread you could send something like Pusher.trigger('threads_channel', user_secret_uid, { thread: new_thread_uid }). You'd also need to make sure in the react app that each user subscribes to the threads_channel for their user_secret_uid. For security, i'd make sure this is a different uid than the messaging otherwise people could subscribe to a list of a different users threads.
I'm using ruby to send transactional emails via Mandrill.
As part of my product, I want to be able to send the same email to two recipients and have them see each other's email address.
(Like an introduction mail between two people).
So I filled he "to" field with both emails, and on my dashboard is seems that both are sent.
But unfortunately only one of the recipients receive the mail and the details of the second recipient is hidden.
In conclusion, I have two problems:
Only one recipient gets the mail
Hidden details of the second recipient.
I approached Mandrill support and this is what they replied:
If you'd like to enable this option globally for all of the messages you send, then you'll want to ensure that you have the
"Expose The List Of Recipients When Sending To Multiple Addresses"
option enabled in your Sending Defaults.
If, instead of making that change globally, you'd like to enable it
for individual messages, you'll want to use the
X-MC-PreserveRecipients (SMTP header), or the preserve_recipients (API
parameter), and set it to 'true'.
If you set this option to true, we'll expose the list of recipients to
each other as you would see happen when sending mail from a typical
email client program.
It worked!
If you want both recipients to be able to see each other, you can pass an array of e-mails in the to option.
If you do not want either of them to see each other, you can, in a loop over the users, send said e-mail.
If using ActionMailer it can be done like so:
mail(
to: ['person1#gmail.com', 'person2#gmail.com']
)
Or in a loop:
[user1, user2].each do |user|
UserMailer.some_email(user).deliver_now
end
mail(
to: user.email
)
Post your code, I have an idea of what your problem may be. Remember that a method in an ActionMailer class should only return mail() and must not be looped over inside of that method.
tldr: do everything unrelated to e-mail outside mailer, pass through necessary data as params to the method, end method with mail() call.
How can I send an asynchronous email with rails 4? By that I mean, when I click the 'send' button of a contact form, the page doesn't refresh, but the email is validated and sent, and a message is flashed to the user.
I've configured action_mailer correctly, and have a ContactForm mailer with one contact action that takes an email address as a parameter.
As a result,
ContactForm.contact("test#gmail.com").delivers #=> delivers email perfectly
But that's working on the command line. I don't really know the correct way to do this with a link. I mean, I could create a button that naviagates to send_email, and then I could have a route like this:
get 'send_email', to: 'contact#sendemail'
Then I would have a sendemail action which contains this method chain as shown above.
But this isn't asynchronous, and, also, I have no idea how I could validate the email's fields before sending the email, or highlighting invalid fields.
Is Ajax and JSON responses the key to highlighting the fields? What about the validation?
The resque_mailer seems to be a good way to send asyncronous emails. But why do I need this external gem when ajax is handled so well by vanilla rails?
The concept would be to have the form submit remotely. i.e submit to a create method in ContactsController. The method would then call a worker (resque/sidekiq) to send the email.
The create action can also respond to json. The json, response can either be a success or a fail (with errors).
On the AJAX success callback, you can trigger an alert, display div, or whatever notifying the user that the email was sent.
If the json results are returned with erros, then you can display the error message via JS.
This Railscasts Episode #171 Demonstrates sending emails using a background process with the help of DelayedJob
I will be sending bulk emails from a Rails app and plan on using SendGrid. I am assuming that it is best to send a separate email to each recipient (as opposed to using BCC for all the recipients). If that is true, should I be using something like DelayedJob to queue the messages going over to SendGrid, or would it be safe to throw 500 messages at it all at once? Thanks!
500 messages really isn't that much to SendGrid. It's not even a blip on their radar. I worked for a company that sent out 2.7 million emails in a single month, and even then it's only just a blip.
With the SendGrid API's capabilities, you wouldn't be sending out 500 emails, you would send one email which has a specific SendGrid API header set. Why? Because have you ever tried to send 500 individual email messages and timed how long that takes? How about a single email? The single email's going to be quicker.
The SendGrid API has a Ruby example which is here:
https://sendgrid.com/docs/Integrate/Code_Examples/SMTP_API_Header_Examples/ruby.html.
That's quite long winded and messy, so let me simplify it for you. Basically, you set this in your email:
headers["X-SMTPAPI"] = { :to => array_of_recipients }.to_json
SendGrid will then parse this and then send that one email you sent it out to that array of recipients. I seem to recall that they ask you to limit this to about 1000 recipients per email, so it would be wise to split it up over multiple emails if you wanted that. That is when you would bring in something like the delayed_job or resque gems to deal with it.
Oh, and by the way you'll still need to specify a to address for this email just to make the Mail gem happy. We had info#ourcompany.com for that.
The SendGrid API will also support filters in their emails, so you can have placeholder strings such as {{ firstname }} and, assuming you send it through with the SMTPAPI header, it will do the "mail merge" on the email and customize them.
It would do you a great deal of good if you read the SendGrid API documentation. It's really useful and what they provide is super powerful.
I recommend using the sendgrid gem ( https://github.com/stephenb/sendgrid ) as it simplifies your calling code.
Here's an example rails 3 action mailer example:
class UserAnnouncementMailer < ActionMailer::Base
include SendGrid
default reply_to: "test#test.com", return_path: "test#test.com", from: "Test"
# bulk emailer
# params - opts a hash of
# emails: array of emails
#
def notice(opts={})
raise "email is nil" unless opts[:emails]
sendgrid_category :use_subject_lines
sendgrid_recipients opts[:emails]
name = "The Man"
to = "test#test.com"
from_name = "#{name} <theman#test.com>"
subject = "Important"
mail({from: from_name, to: to, subject: subject})
end
end
And the corresponding calling code. It's recommended to have the emails array to be < 1000 emails.
emails = ["alice#test.com", "bob#test.com"]
UserAnnouncementMailer.notice({:emails => emails}).deliver
See the sendgrid gem github readme for more details.
Delayed Job and SendGrid sound like the best option from what you say, but have you considered using one of the campaign mailers like Mailchimp instead? If you're sending out a lot of mails that are basically the same, they'll let you setup and campaign template and then fire a CSV of all the variables at it. They then effectively mail merge and fire them all out.
If however, you're only talking a few hundred you're on the right lines. SendGrid can easily handle the load, and you want to use Delayed Job so that you're not impacted by the performance of the SendGrid API should it not be favorable. Alternatively, look at Resque instead for sending mail as it may be more efficient.
I would imagine SendGrid can handle that kind of load. Most relay systems can. Also I would imagine if you sent the 500 in a CC API call, that their system would parse it and send them individually. I use Elastic Email (http://elasticemail.com) - and I know that this is how they handle it and it works great.
This is how I've done it in Rails 4
class NewsMailer < ApplicationMailer
include SendGrid
sendgrid_category :use_subject_lines
default from: 'My App! <support#myapp.com>'
def mass_mailer(news)
# Pass it in template
#news = news
# Custom method to get me an array of emails ['user1#email.com', 'user2#email.com',...]
array_of_emails = #news.recipients.pluck(:email)
# You can still use
# headers["X-SMTPAPI"] = { :to => array_of_emails }.to_json
sendgrid_recipients array_of_emails
mail to: 'this.will.be.ignored#ignore.me', subject: 'Weekly news'
end
end
I am building the Email where I am parsing the .csv file which consists of email id's.Here is the code
File.open("#{Rails.root.to_s}/public/files/#{params["file"].original_filename}", "wb"){|file| file.write(params["file"].read) }
arr_of_arrs = CSV.read("#{Rails.root.to_s}/public/files/#{params["file"].original_filename}")
puts arr_of_arrs
arr_of_arrs.each do |i|
Here is the Mail sending process which is called in my Controller
Class.method(i[0]).deliver)
And I call the ActionMailer to send emails which are in the .csv file.And I am using AWS SES to send the mails.
My problem is when ever it fails to send an email to a particular address the whole email sending stops and it would not send emails to the rest of the address. But even after it fails it should be able to send the email to the rest of the addresses, how can I handle this issue since I am newbie to the ruby on rails.
This doesn't seem to have anything to do with delayed_job.
To not stop the remaining emails, catch the exception which breaks the loop:
arr_of_arrs.each do |i|
begin
Class.method(i[0]).deliver
rescue => e
# perhaps you'd like to log e's detail here
end
end