Overview
From our MVC web app we are sending emails. We send 2,000 each time, once per day.
We send the emails one at a time to Postmark to process and deliver.
The connection with Postmark is via SMTP which is setup in Web.Config:
<mailSettings>
<smtp deliveryMethod="Network" from="name#domain.com">
<network host="smtp.postmarkapp.com" port="587" enableSsl="true" defaultCredentials="false" userName="username" password="password" />
</smtp>
</mailSettings>
We're using Postal in our app for email templating, etc.
foreach(var user in users)
{
var email = new Postal.Email;
{
EmailTo = user.Email,
Subject = "Subject",
//other email content
};
//Send email
email.Send();
}
Our web app is on Azure in a Web App service.
The processing and sending is being done in a background method using Hangfire with no automatic retries.
Problem
There is a small percentage (exact percentage not known) of recipients that are consistently receiving duplicates or triplicates of these emails each day. We haven't noticed a pattern with the ones receiving more than one (i.e. that they're from the same domain, etc.).
In our code, the email details and content are created in a loop and fired off one at a time. We've added logging in our code to verify that we're really only creating one email to the people receiving duplicate and triplicate emails.
We've talked to Postmark about any ideas they have. They said they didn't think it was them. I've looked in the headers of the emails via the Postmark UI and the duplicates have unique message Id's.
Question
Are there things that can happen with the way emails are handled when lots of them are being fired off in short succession using SMTP in an MVC app? Is there a way to log what is actually being sent from the app/server?
Related
I'm developing an application that will send thousands of emails per day based on a CSV file using Amazon SES.
My client would like to send around 50,000 in one hit.
The app is almost done however i would like to test it with thousands of emails. What a shame would it be that on my first try with live data it doesn't work.
Right now i have only tested it using about 10 emails in my CSV file.
Does anyone know of a way to test sending around 10,000 emails? Obviously i can't use real data.
I believe you can use real data. To test 10,000 emails for example, you can do the following (assuming you've already increased your SES sending limits):
Generate 10,000 email recipients in your CSV file for a target domain that you already manage. Those 10,000 email addresses should be random and non-existent (e.g., 1eed1417-cc2c-4f7c-8ddb-7e0dfbcacb31#mydomain.com).
Configure a catch-all email address for mydomain.com (e.g., catch-all#mydomain.com) to receive emails sent to any non-existent address.
Add SES SMTP servers to mydomain.com's trusted senders/addresses and make sure your mailserver's throttling rules allow this sending volume.
Test during off-peak hours and check the inbox of your catch-all mailbox.
What i ended up doing is using https://mailtrap.io/
I made 25 emails. Then sent 600 emails to each of the 25 emails. That equals 15,000.
I didnt use action mailer to send the emails. I used the Amazon SES send email method
def send_email!
client.send_email(email_attributes)
end
private
def client
Aws::SES::Client.new(region: Rails.application.secrets.ses_region,
access_key_id: Rails.application.secrets.aws_access_key_id,
secret_access_key: Rails.application.secrets.aws_secret_access_key)
end
Used Kabana for tracking the emails. If they bounced, delivered and so on.
SES won't track email opens so we have a small Node-JS app that will track opens and clickable links.
The app sends around 15000 emails in 20minutes.
I am assuming you are not doing a full end-to-end test (which includes checking to ensure the email was received) but rather are just performing a functional test of your application logic.
If that is the case,
Point your application at a test email server and disable outbound email for all queues. You could also do this on a production server, at say 3:00 am, I guess, but be careful!
Run the application
Check the email server's outbox and count the number of items that have queued up.
If the count matches, your test has passed.
Don't forget to purge the queue before turning it back on!
We are using Rails 2.3.8 version. In our application, we tend to send individual email to all users ( around 1500 users - Bulk email ). We are not using any services other than Actionmailer. Hence, we cannot use any service (like mailGun, sendGrid, mailchimp, etc) Also, we have our own SMTP server, through which all the mails are sent.
We have added our mailing functionality to be executed in two scenarios.
Rake Task with CRON JOB ( Auto mails to be sent at specific time)
Via controller ( Mails are sent when a button is clicked)
In both the cases, Initially, the mails are forwarded to the users without any issues. As the user count increases, suddenly the mail functionality stops and throws either of the two following errors.
535 5.7.3 Authentication Unsuccessful
Execution expired
Could anyone please help me sort out the issue?
Is there any timeout value that has to be set for SMTP server connectivity? Or are there any restrictions (like number of mails sent per minute) ?
Thanks in Advance
I am using free mandrill account and sending email via using template and API
When i send message it returns okay it is sent as status
However even after several days there is still no smtp events at mandrill interface and the email is not arrived
So i am 100% suree mandrill is ghosting accounts
Or there is something else that i do not know?
Thank you for answers
PS: At the beginning emails were arriving but after some point no smtp events and no emails ever arrived even though no error message parsed. Also when i send to non existing email no bounce message returned.
Also account reputation is 61 : excellent
Yes i believe mandrill certainly ghosting accounts
After trying with several accounts i am now sure of it
Even if you send to non existing gmail email, it says delivered in its interface
However it should have displayed hard bounced
So beware of their free service. I believe this is not an ethical way of working. People would think that their emails are arriving however they are ghosted and mandrill did not even try to send them and yet displays delivered
Have you check it on Mandrill account? It shows all the report at outbound tag. The reason is, a queued response in the Mandrill API is not the same as a queued response from a recipient server.
When you send a message through Mandrill, you first relay it to Mandrill, Mandrill processes it, and then Mandrill relays it to the recipient server. This all happens quite quickly, but the two relaying steps are separate and distinct. The KB article you've linked to is providing additional details on that last step, relaying to recipient servers, not a queued status for the Mandrill API.
There are a number of reasons the Mandrill API may respond with queued including if you've added attachments or if you're sending to a bunch of recipients in a single API call.
Without seeing the actual API call that's being made, it's hard to say why you're getting a queued response. But if you're using the sample messages/send API call, you'll want to remove all of the optional parameters that you're not actually setting. For example, the sample has fake attachments, and a subaccount specified. The attachment will cause the call to be processed async. The subaccount probably doesn't exist, and would then cause the call to fail. So if that's the case, try removing all of those optional params. If not, please provide the API call you're making with sensitive data redacted (API key, actual email addresses).
I have a gmail for business emailId and would like to use it for sending out messages to my clients. I am using smtp.gmail.com to send out messages from an asp.net mvc application. This works fine on my localhost. But when I deployed the code to Amazon EC2, the functionality broke. I searched and found that EC2 IPs are blackisted by google. How should I fix this? I read that one way is to use Amazon SES, but can anyone tell how does it work? Is it just a verification policy enforcement thing and does it support sending out email from the said gmail account? Is there any other workaround?
Edit 1: As suggested I have contacted Amazon to provide production access. It is in process.
Meanwhile, I am seeing that I am able to send mails from my application. These are my settings -
WebMail.SmtpServer = "smtp.gmail.com";
WebMail.SmtpPort = 587;
WebMail.EnableSsl = true;
WebMail.UserName = "me#example.com";
WebMail.From = "me#example.com";
WebMail.Password = "MyPassword";
WebMail.SmtpUseDefaultCredentials = false;
Is there a reason why suddenly it is working. Is this intermittent or can I move on to using this as such?
SES is the solution for this problem, and then good thing is, unless you are sending 10's of thousands of emails it will only cost you pennies to do - it will allow you to send an email, and make it come from (or appear to come from) your gmail account.
First step is to verify to SES that you own/control the email address you want to send from - that can be done by clicking on a link in an email they send to that address, or else adding some TXT entries to your DNS settings. If you control the domain, I prefer to use the DNS method so that I can verify the entire domain for sending emails, not just the one account - then you can do things like send from 'sales#mydomain.com', 'billing#mydomain.com' etc w/out having to verify each email address individually.
Once you account is verified, sending email from SES using the SMTP option is exactly the same as sending it thru any other SMTP server - use the SMTP server name, username and password from amazon, and away you go.
Very easy, very inexpensive and emails are sent very fast.
Be aware there are limits by default - I believe 5/second and 10,000/month by default - but that can be upped if you have a legitimate need.
Also, once your account is setup with SES, you are not limited to sending your emails just from EC2 instance, you can use it anywhere - for example my outlook email, my mac and my iphone all use SES as my SMTP provider.
I was actually able to send an email using an EC2 instance. Try editing your SMTP Server to ssl://smtp.gmail.com
When using 'mail' command to send email to a gmail user, the email goes through fine. When sending an email using a Rails app, the email is sent to the spam folder for the gmail user. Can someone help me think through this?
Emails landing in SPAM can happen due to many reasons:
Wrong Mail Server setup: Checkout here on how to setup
Email content: Content of the email can also invite SPAM. Sites like SpamCheck helps to check whether the content of the email is ok.
As mentioned by #Noli above, using services like Sendgrid, Critsend etc for sending out emails, chances of landing them in Inbox will be more. You can use them as relay servers from Postfix. But the first two steps are anyway necessary.
Use Mailchimp if you want to sent emails to many people, for eg: for sending out newsletters, marketing emails etc.
Mail deliverabillty is extraordinarily hard to get right. You should consider leaving this to the specialists like Sendgrid or Mailchimp, and not spend tooooo much development time thinking about it
Another thing to check is that if this is a new server, you may need to set up Domain Keys to authenticate to Gmail. This happend to me and I was able to get my mail removed from the spam folder by following these:
https://help.ubuntu.com/community/Postfix/DomainKeys
https://help.ubuntu.com/community/Postfix/DKIM