I am trying to implement sendgrid parse API using the 'griddler' gem in my rails application. The problem is how can I test it locally? And how to receive email in my local machine.
The are a couple ways to test SendGrid's Parse Webhook locally.
The easiest is to just simulate the Webhook by POSTing data to your endpoint yourself (via cURL or some other mechanism, like Postman [example]). The SendGrid Parse Webhook Docs has an example of the payload that will be posted to your server, so you may mimic that.
The second option, which allows for end-to-end testing rather than just simulation is by creating a tunnel your local machine and provide SendGrid with the URL of the tunnel. There are a number of free services that allow you to do just that very easily:
https://ngrok.com/
http://progrium.com/localtunnel/
If I may self-promote, I created a Ruby gem that does this:
https://github.com/ccallebs/pokey-sendgrid
It runs alongside your server and generates SendGrid webhook requests at regular intervals.
Related
Parsehub provides the webhook feature. But currently I'm testing my Rails app locally. So how could I provide the webhook url for a project on Parsehub to point to my local server or any specific method in my controller.
Parsehub Doc Webhook:
https://www.parsehub.com/docs/ref/api/v2/#webhooks
Webhook Url option Screenshot:
In order to use a webhook, you need to provide a publicly visible IP address for ParseHub to make requests to. You can get one by registering for a cheap VPS host (e.g. DigitalOcean for $5/month).
On that host, you want to run a webserver, and put the endpoint that the webserver listens on into the webhook textbox in ParseHub. To inspect the details of what ParseHub sends, you can just make your webserver log all the request data. You can also check out our API docs which have a description of all the fields: https://www.parsehub.com/docs/ref/api/v2/#run
The way I usually test webhooks is by using Request Bin http://requestb.in/
Essentially you get a url from them, you give this as your webhook address and anything that is posted to this URL will be caught by the website for further inspection!
You can then get these parameters and post them to your application manually, thus mimicking the entire process.
I'm in the process of planning the development of a mail-server to hand the sending of email across our multiple websites. Below is a description of what we are planning to implement and I'd like your opinion/suggestions.
We use ASP.NET MVC and have many web-sites hosted on Azure. We currently send mail internally within each of the web applications using SMTPServer.Send(). Obviously this is not the ideal way to send emails when you have a decently busy set of websites because the send mail call is blocking and cannot guarantee mails are sent. With this I'm worried out getting an influx of mail requests when we launch our next website (we think it'll get a decent amount of traffic and lots of emails will be sent).
My plan of action : to build a centralised mail-server that runs in the background (we use azure and this will be simply another web-application). When each one of our web applications wants to send a mail, instead of doing this internally, it'll call a web method on the mail-server called sendMail() this function will accept certain parameters and insert the mail parameters, content etc. into a database. The mail server will then poll the database at fixed time intervals, select a set of unsent emails and attempt to send them using the same SMTPServer.Send() function. If an email fails for some reason we won't flag it as sent and in the next poll interval the email will be selected again and another send attempt will be made. (we will cap the number of send attempts to say 20).
This will allow each of the websites to run smoothly without having loads of blocking send mail calls internally and the mail server will handle all the sending sequentially and in a controlled environment as a separate standalone web-application.
Thanks in advance!
Looks like a good design, Don't know the entire scenario which let to you building something like an email server. The problem has been solved well by using a service that already exist like Office 365.
Your design is good, My suggestions would be the following,
You can use Azure WebJobs to build the polling agent. You can make the web job run as a scheduled web job that does the polling and sending the mail and it can be written very clean as a simple console app.
You can use Azure API App to build the SendMail() call and you can use the Azure AD Auth on the API to authenticate the caller of the API using the Authentication and Authorization feature to easily secure your email server. You can also enable CORS easily as well to make sure you receive requests from other websites and process it.
Some issues I foresee for you,
Volume and Scaling : You can only process so much email between each polling. If you cannot then you will need to create another polling agent which will making things complicated as they need to know they are picking different sets of emails to send. If you volume is going to be low you should be fine.
Challenge : Why can't the websites send the mail them selves ? And then record it on the database for tracking. All you have to do build module or a component that they use on their web page to create and send the mail. Polymer 1.0 works well for this scenario.
Hope this helps to get you started.
I have a Rails app running on Heroku that uses Mailgun to process incoming emails. I haven't been able to figure how I can debug my email processing locally (on localhost) instead of having to push everything up to heroku every time I make a change. (this is just a test app - I'm the only one using it)
Is it possible to work with Mailgun locally? If so, how do I go about it?
Thank you in advance
Mailgun gives you the option to store a message for later retrieval. If you configure it that way, you'll be able to fetch messages from development for processing without having to set up a publicly-accessible webhook for Mailgun to hit.
But I'm assuming you have production configured with an HTTP endpoint, and it's no fun to do things differently between environments. There are a few tools that will let you set up a public endpoint that routes to localhost:
ngrok, which I've used to good effect to test Twilio. You can set up a permanent subdomain so you don't have to constantly change your Mailgun configuration.
UltraHook, which I haven't personally used, but looks the same.
Localtunnel which looks easiest to start up, but like you get a different host at every boot.
If you have a permanent publicly-accessible server, you can also maintain your own tunnel.
mailgun provides a sandbox that you can use for localhost the only downside to this is that you have to add the test email to valid recipient.
using this gem might be another possible solution:
https://github.com/ryanb/letter_opener/ or https://github.com/fgrehm/letter_opener_web for more advanced features
follow installation from repo
mail will open in new tab
Has anyone seen any examples of how to run an smtp server with Rails hosted on heroku?
I'm interested to send mail to my own smtp server, manipulate the content with rails by adding a footer, adding link tracking etc, then send it out again.
Once I have the data, easy. Not sure where to begin in terms of setting up smtp.
Alternatively, are there any smtp services out there that will simply POST the data to my app?
Take a look at SendGrid here http://sendgrid.com/
They've got some great APIs, and should be able to enable what you want.
If you want inbound email via a post to your application look at cloudmailin.com
It's python, not RoR, but the Lamson project is pretty close to what you are talking about.
http://lamsonproject.org/
(Thanks Zed!)
How would i go about to receive mails in a Ruby on Rails application without going through a mail server like PostFix or to fetch them by pop3 etc.
What i was to do is to catch all mails sent to #mydomain.com and just do something with them in my application. I don't need to store the mails or anything like that.
Is this posible?
I just implemented this for my SAAS to autoprocess mailer-bounce notification messages.
Call me, call you?
You call me
You can set up a local mail server. It would then respond to an incoming email, and start up a rails executable to process the email. This method is NOT recommended since starting up rails is a big task (takes multiple secs and lots of memory). You don't want a Rails bad boy started up just because you received an email. You'd be writing your own DDOS attack. (Attacking yourself.)
I call you
Instead, poll for email on your own schedule by using a single job to process all currently waiting emails. You need to set up a background job handler since stock rails is focused on responding to web requests. I use delayed_job, but there are other alternatives including kicking off a cron job every so in often.
Another benefit is that you don't need to manage a mail server. Leave that headache to someone else. Then use the Ruby library net::imap to read the incoming mail and process it.
If your process doesn't recognize the email format, then forward the msg to a human for processing.
And be sure that if the process sends mail in addition to reading/processing it, that the process uses a different email address as its From address. Otherwise, odds are good that sometime along the way, you'll end up in an email loop and many gigabytes of messages going back and forth. For example, your process receives a message, responds to it, but in the meantime the sender (a human) has switched on vacation response. And your robot then responds to the vacation response..... oops....
Writing your own mail server
Re:
How would i go about to receive mails in a Ruby on Rails application without going through a mail server like PostFix or to fetch them by pop3 etc.
What i was to do is to catch all mails sent to #mydomain.com and just do something with them in my application. I don't need to store the mails or anything like that.
Direct answer: Yes, you could do this by writing an smtp server and setting up dns so your machine will be the mail destination for the domain. Your smtp server would process the messages on the fly, they would not be stored on your system at any point.
Is this a good idea? No, not at all. While appearances may be to the contrary, email is a store and forward system. Trying to avoid storing the messages before your app processes them is not smart. It would be a very very poor "optimization." However, using an access protocol (POP3 or IMAP) is a good way to avoid the costs of installing, configuring and managing a mail server.
You can do this if you write your own mail server, or if your mail server supports hooks to run external programs upon receipt of mail (e.g. procmail).
If you don't have procmail available (or, if on something like Exchange Server, don't feel like writing custom rules or extensions), you're simply better off using a pop3 library to fetch mail.
Obviously, writing a mail server is more difficult than any of the alternatives.
If you're mostly worried about checking potentially hundreds of email accounts, that's solvable by configuring your email server properly. If you're on a hosted provider, ask your server administrator about creating a "catch-all" account that routes all mail to unknown addresses to a single account.
If you're aiming to avoid having to poll a server, consider the IMAP IDLE command. I've successfully written a Ruby client that opens a connection to an IMAP server, and gets told by the server when new mail arrives.