I'm testing the Mandrill API and sent an email to my GMail account. In the API logs, it says:
"status": "queued"
According to https://mandrill.zendesk.com/hc/en-us/articles/205582717-Why-does-a-delivered-message-say-queued- :
most times Mandrill can send email much faster than recipient servers
are able to accept or process it
GMail is not able to handle my one email that I sent?
A queued response in the Mandrill API is not the same as a queued response from a recipient server.
When you send a message through Mandrill, you first relay it to Mandrill, Mandrill processes it, and then Mandrill relays it to the recipient server. This all happens quite quickly, but the two relaying steps are separate and distinct. The KB article you've linked to is providing additional details on that last step, relaying to recipient servers, not a queued status for the Mandrill API.
There are a number of reasons the Mandrill API may respond with queued including if you've added attachments or if you're sending to a bunch of recipients in a single API call.
Without seeing the actual API call that's being made, it's hard to say why you're getting a queued response. But if you're using the sample messages/send API call, you'll want to remove all of the optional parameters that you're not actually setting. For example, the sample has fake attachments, and a subaccount specified. The attachment will cause the call to be processed async. The subaccount probably doesn't exist, and would then cause the call to fail. So if that's the case, try removing all of those optional params. If not, please provide the API call you're making with sensitive data redacted (API key, actual email addresses).
The reason might be either hourly/monthly quota is over or you are using same multiple accounts for single public ip server.
Related
As per sendgrid documentation, the way Inbound parser webhook responds to failures are that it will retry to post the email to the configured endpoint for 3 days and will drop the email if undelivered.
The Parse API will POST the parsed email to a URL that you specify. If a POST is unsuccessful, SendGrid automatically queues and retries any POSTs that respond with a 5XX status. This prevents data loss for customers who have misconfigured their website or POST URL.
Respond with a 2xx status to the POST request to stop the email from retrying.
To avoid returning an error your link must return a 2xx HTTP code when the email is received. This response lets our system know that your link has received the email. It is then removed from our send queue. If we do not get a valid 2xx HTTP response, our servers will believe they have failed to deliver your message. Messages that cannot be delivered after 3 days will be dropped.
I would like to know if before dropping the undelivered emails, will sendgrid notify the sender that their email could not be delivered?
Based on a few tests I ran, it does not appear SendGrid will ever send a notification to the sender, nor is there any easy way to ascertain whether SendGrid has dropped any inbound emails.
Although the documentation wording is slightly ambiguous, my understanding based on testing (which aligns with the documentation wording) is this:
Server returns 2xx: email is deemed accepted, no further attempts made.
Server returns 5xx: server has error, attempts made until 2xx is returned (see below for timing of subsequent attempts), or 72 hours lapses
Server returns any other response, or DNS records do not exist: email is deemed failed, no further attempts made.
My conclusions are based on a number of tests I ran over the course of a week, from which I ascertained the following:
Case 1. Parse server returns 400 or 403 error
The email is dropped after one attempt.
No further attempts are made to POST the email.
No notification is sent to the sender or to the SendGrid account.
(Test method: Configure SendGrid to a URL that returns one of the above error codes. Checked server logs over 1 week, and noted only one attempt was made.)
Case 2. The parse hook URL does not have a DNS record
The email is dropped, presumably after one attempt.
No notification is sent to the sender or to the SendGrid account.
(Test method: configure the hook URL to a subdomain that does not have any DNS records. Run DNS search and try opening the hook URL to confirm the DNS records do not point anywhere. Send an email. Then, after 12 hours, add an appropriate record to the subdomain to point it to a script. Checked server logs and confirmed no attempts were made to POST the email. A subsequent email attempt successfully POSTed.)
Case 3. Parse server returns 500 error
SendGrid attempts to POST to the hook URL at the following intervals:
+0 +5m +10m +15m +20m
+25m +35m +50m +1h20m +2h20m
...then every 3 hours
The last attempt occurs +71h20m
The 20 min offset is a bit unusual, but it falls on the hour, so it may be because the message is queued to attempt POSTing on the hour.
No further attempts are made to POST the email after 72 hours.
No notification is sent to the sender or the SendGrid account.
Statistics
The documentation refers to the availability of statistics, however, I have found the numbers to be inaccurate.
For example, during my testing, I had a number of emails go through the API, including some that were intended to be successful (and did indeed POST to my server) and a few that were intended to fail (as part of the above testing), however, the numbers returned did not align.
Takeaways
Care should be exercised in the following scenarios:
Server downtime: this would ultimately depend on how the server is configured. If the server returns a 5xx response code, SendGrid will keep trying to POST the email. I also tried to test a timeout scenario, however, SendGrid seems to be very patient (e.g. I made my script pause for 10 minutes, SendGrid kept the connection for 10 minutes, though interestingly, due to the second attempt occurring after 5 mins, the email POSTed twice).
Server misconfiguration that returns 4xx error codes. SendGrid will drop the email.
It should also be noted if an email is dropped, there doesn't appear to be any reliable method of finding this out.
SendGrid won't send a notification before the email is dropped. However, the message will be dropped only after receiving an error from your Webhook URL and retrying to post the message for three days.
We got a confirmation from the Sendgrid support team, that the emails that are dropped in the Inbound Webhook Parser feature, will not have any notification or being saved in Sendgrid. So, we won't be able to get any info regarding the emails which are dropped.
I'm developing an application that will send thousands of emails per day based on a CSV file using Amazon SES.
My client would like to send around 50,000 in one hit.
The app is almost done however i would like to test it with thousands of emails. What a shame would it be that on my first try with live data it doesn't work.
Right now i have only tested it using about 10 emails in my CSV file.
Does anyone know of a way to test sending around 10,000 emails? Obviously i can't use real data.
I believe you can use real data. To test 10,000 emails for example, you can do the following (assuming you've already increased your SES sending limits):
Generate 10,000 email recipients in your CSV file for a target domain that you already manage. Those 10,000 email addresses should be random and non-existent (e.g., 1eed1417-cc2c-4f7c-8ddb-7e0dfbcacb31#mydomain.com).
Configure a catch-all email address for mydomain.com (e.g., catch-all#mydomain.com) to receive emails sent to any non-existent address.
Add SES SMTP servers to mydomain.com's trusted senders/addresses and make sure your mailserver's throttling rules allow this sending volume.
Test during off-peak hours and check the inbox of your catch-all mailbox.
What i ended up doing is using https://mailtrap.io/
I made 25 emails. Then sent 600 emails to each of the 25 emails. That equals 15,000.
I didnt use action mailer to send the emails. I used the Amazon SES send email method
def send_email!
client.send_email(email_attributes)
end
private
def client
Aws::SES::Client.new(region: Rails.application.secrets.ses_region,
access_key_id: Rails.application.secrets.aws_access_key_id,
secret_access_key: Rails.application.secrets.aws_secret_access_key)
end
Used Kabana for tracking the emails. If they bounced, delivered and so on.
SES won't track email opens so we have a small Node-JS app that will track opens and clickable links.
The app sends around 15000 emails in 20minutes.
I am assuming you are not doing a full end-to-end test (which includes checking to ensure the email was received) but rather are just performing a functional test of your application logic.
If that is the case,
Point your application at a test email server and disable outbound email for all queues. You could also do this on a production server, at say 3:00 am, I guess, but be careful!
Run the application
Check the email server's outbox and count the number of items that have queued up.
If the count matches, your test has passed.
Don't forget to purge the queue before turning it back on!
I am using free mandrill account and sending email via using template and API
When i send message it returns okay it is sent as status
However even after several days there is still no smtp events at mandrill interface and the email is not arrived
So i am 100% suree mandrill is ghosting accounts
Or there is something else that i do not know?
Thank you for answers
PS: At the beginning emails were arriving but after some point no smtp events and no emails ever arrived even though no error message parsed. Also when i send to non existing email no bounce message returned.
Also account reputation is 61 : excellent
Yes i believe mandrill certainly ghosting accounts
After trying with several accounts i am now sure of it
Even if you send to non existing gmail email, it says delivered in its interface
However it should have displayed hard bounced
So beware of their free service. I believe this is not an ethical way of working. People would think that their emails are arriving however they are ghosted and mandrill did not even try to send them and yet displays delivered
Have you check it on Mandrill account? It shows all the report at outbound tag. The reason is, a queued response in the Mandrill API is not the same as a queued response from a recipient server.
When you send a message through Mandrill, you first relay it to Mandrill, Mandrill processes it, and then Mandrill relays it to the recipient server. This all happens quite quickly, but the two relaying steps are separate and distinct. The KB article you've linked to is providing additional details on that last step, relaying to recipient servers, not a queued status for the Mandrill API.
There are a number of reasons the Mandrill API may respond with queued including if you've added attachments or if you're sending to a bunch of recipients in a single API call.
Without seeing the actual API call that's being made, it's hard to say why you're getting a queued response. But if you're using the sample messages/send API call, you'll want to remove all of the optional parameters that you're not actually setting. For example, the sample has fake attachments, and a subaccount specified. The attachment will cause the call to be processed async. The subaccount probably doesn't exist, and would then cause the call to fail. So if that's the case, try removing all of those optional params. If not, please provide the API call you're making with sensitive data redacted (API key, actual email addresses).
I am sending email using sendgrid.
Are there any options in sendgrid to get email analytics (open/bounced/replied emails) for each users email address?
I've done this two ways before:
1) Use SendGrid's Event API to collect and aggregate the data yourself.
2) Use a service like sendwithus on top of your SendGrid account, which will collect your SendGrid analytics for you and provide analytics by recipient, template, segment, etc. They also provide an API and Ruby client.
Once you're logged into SendGrid, navigate to the Stats Dashboard.
You should then be able to see something like this:
I hope this helps.
Yes, There are option available for this, to get the status of individual emails sent via sendgrid.
Send a unique arguments with every email you sent and then later on you can get status of these emails with reference to unique argument.
Some more explanation of your question will help us a-lot to describe you the best possible solution.
i have recently done this work, so if you explain the whole scenario, i will come up with the solution.
There are two ways to get hold of this data:
Use the SendGrid Event Webhook to get near realtime information about every event for every email you send via SendGrid. It works by issuing a POST request to a URL that you specify.
You can then store all this data for consumption later on and it's by far the most granular information you can get. Key/Value stores like Redis are a good option for this, but if you want a simple option to get going, check out the SendGrid EventKit to get up and running quickly.
Or:
Grab the data via the Stats API. This endpoint allows you to grab the numbers you need in a multitude of ways for both main and subuser accounts.
Based on your question, you should investigate the Event Webhook as it will give you most clarity on what happens with each email you send.
You can now use Thinbox to get analytics from Sendgrid, Mailgun, Mandrill, or just by bcc'ing an email address they give you. It's basically like EventKit but for the rest of the ESPs too.
in github when a user sends you a message two things happen. You get a "new Message" on your github dashboard and you receive an email.
if you reply to that email it triggers a new Github message internally... so the users can actually have a full conversation through their email client without going into github even though Github is managing it all.
I know Malgun/Sendgrid have apis to manage receiving of emails (they send a POST request to your app when an email is received) but I need a little more info on how to do it... how exactly can I set up my app so that when a user receives a message they can just hit reply on their favorite email client while still maintaining track of that conversation. (they can still check their messaging history through my site)
Does anyone have an idea how exactly they do it?
Please help.
How this is implemented really depends on how you can handle incoming messages. If you're able to receive your emails as a POST to your application, then the email is really no different to a user sending the message on your site, you just need to parse the From: header from the email, and look up the user, and strip the fluff out of the email.
If you're writing your own code to handle the emails (eg. that polls a mailbox), then you could just POST them over to your app in the same way, or parse them up and POST more structured data.
Once you have the data, it's easy to construct a message and write it to your DB (and fire off email notification to the user, remembering to set the Reply-To: or From: headers so your script gets the replies). Most of these kind of messaging systems don't keep track of conversations/threads, but just store a string subject (and use "Re: ...") to keep things simple, though you could obviously add this if you're feeling ambitious!
If you're doing this, you should be security in mind - malicious users may POST to your email script, and email headers can easily be forged. Spammers will also use any possible scripts they can find to relay mail through other peoples servers.