A client needs to send out eNewsletters.
We tried using their Sitefinity CMS, but it hangs for reasons that are hard to determine.
As a result, I am inclined to "roll my own".
However, various pitfalls yawn deep and dark along the imagined path.
For example:
Imagined Pitfalls
My client's domain gets blacklisted.
The emails get rejected as Spam.
Presumed ways of avoiding imagined pitfalls:
a. Send out emails with a delay of, say, 2 seconds between each one.
b. Send out emails individually, ie, no CC or BB.
Questions:
Are my imagined pitfalls and presumed
ways of avoiding them correct?
Can you think of other pitfalls?
Or better ways of avoiding them?
Do you know of reliable software that
just does this (ASP.NET, MVC by
preference, drawing form SQL Server
db).
It may be beneficial to use a third party like Constant Contact to handle sending your email campaigns. They expose an API that you can tap into via REST and it won't have any effect on your client's domain. Constant Contact is a reputable company and email campaigns they send out rarely get caught in Spam filters. My company currently uses the service, but not the API and its been very successful.
Constant Contact's Website
CC's API Web Site
Example of creating a Campaign
You should never do this. Your client's IP(s) and domain(s) will find themselves blacklisted to the point that even their legitimate, hand-crafted emails won't make it past spam filters at large mail services.
Use something like SendGrid if you want to stay closer to the metal than a service like MailChimp or Constant Contact.
There are other companies out there which do it, such as BriefYourMarket ... never roll your own :-)
Related
https://developer.microsoft.com/en-us/office/blogs/end-of-support-for-basic-authentication-access-to-exchange-online-apis-for-office-365-customers/
Our organization is finding this announcement somewhat problematic! We use an IMAP library extensively to read various service based email accounts in o365. Any guidance on how to address this would be greatly appreciated.
Note, we have many console apps written in .NET (4.8) that run on a server based fired by many scheduled tasks. I understand we'd need to somehow register our "application" (I'm assuming that can be a generic one for our company), but we cannot involve any "user" interaction. These are utility apps. Glancing at the existing sample code for OAuth, they all seem to involve popping up a browser window to get someone to interact with "asking permission" which is exactly what we need to avoid.
We've used IMAP all this time to simply read and parse service based email accounts. I'm not sure I understand why IMAP over a secure connection is "less secure" than a more complex solution. Why take the option away?
On the other hand, the Microsoft Graph API looks significantly more complicated and appears to be OAuth based which, again, seems to involve quite a bit of authentication complexity.
Most REST APIs we've interacted w/ in other .NET console apps use a simple set of API "keys." Why not offer that at least?
As I say, we're looking for a way to write some process that run programmatically to automate a number of operations related to certain mailboxes. IMAP has worked like a charm so far, so we're looking for direct guidance on what to migrate to.
We understand your concerns. While a secure IMAP connection protects the data that's being transported, Basic Authentication exposes your Exchange Online accounts to attack techniques like phishing or brute forcing.
The primary objective of this change is to protect our customers from these threats. In addition, Modern Auth enables admin visibility into app access and enables fine-grained control of these apps.
To answer your question on implementation guidance, there is an existing approach in Graph to achieve exactly what you're looking to do. It's called "OAuth 2.0 client credentials flow". You can read more at https://learn.microsoft.com/en-us/graph/auth-v2-service
(Disclosure - I'm a Senior PM at Microsoft)
I have implemented a custom email server and web client. The server is just a REST API (similar to google's gmail API) that uses a 3rd party (sendgrid) for sending and receiving. The emails are stored in a database. The web client just talks to the REST client for sending and receiving.
The problem with this approach is it doesn't implement IMAP anywhere, which makes it impossible for standard clients (outlook, iphone, etc.) to connect to and use our email API. This limits customers to using only our client for email.
What I need is some sort of IMAP Server "facade" that will manage the connections to clients and make calls to my REST API for actually handling the requests (get email, send email, etc.).
How can an IMAP facade be implemented? Is there maybe a way to take an existing MailServer and gut it and point all it's "events" to making calls to my API?
tl:dr; write your gateway in Perl; use Net::IMAP::Server; override Net::IMAP::Server::Mailbox; and use one of the many Perl REST clients to talk to your server.
Your best bet for doing this quickly, while maintaining a reasonable amount of code security, is with Perl. You'll need two Perl modules. The first is Net::IMAP::Server, and here is the Github repository for that module. This is a standards-compliant RFC 3501 server that was purposely designed to have a configurable mail store. You will override the default Net::IMAP::Server::Mailbox implementation with your own code that talks to your custom email backend.
For your second module, choose your favorite Perl module(s) to use to speak to your REST server. Your choice depends on how much fine grained control you want to have over the construction and delivery of the REST messages.
Fortunately, here you have tons of choices. One possibility is Eixo::REST, which has a Github repository here. Eixo::REST seems to deal well with asynchronous vs. synchronous REST API calls, but it doesn't provide a lot of control over X509 key management. Depending on how googley your API is, there's also the REST::Google module. Interestingly, this family also has a REST::Google::Apps::EmailSettings module, specifically for setting Gmail-specific funkiness like labels and languages. Lastly, the REST::Consumer module seems to encapsulate a lot of https-specific things like timeout and authentication as parameters to Perl object instantiation.
If you use these existing frameworks, then about 90% of the necessary code should already be done for you.
Don't do this by hacking Dovecot or any other mail server written in C or C++. If you hack together a mail server quickly using a compiled language, your server will sooner or later experience all the joy of buffer overflows and stack smashing and everything else that the Internet does to fuck over mail servers. Get it working safely first, then optimize later.
(This is basically my comment again, but elaborated quite a bit more.)
Some IMAP servers, most notably Dovecot, are structured such that the file access is in a separate module with a defined interface. Dovecot isn't the only one, but it's by far the most popular and its backend interface is known to be appropriate, so I'd take that absent specific concerns.
There already exist non-file modules such as imapc, which proves that it can be done. When a client opens a mailbox backed by imapc, Dovecot parses IMAP commands, calls message access functions in imapc, imapc issues new IMAP commands, parses the server responses, returns C structs to Dovecot, Dovecot fashions new IMAP responses and returns them to the client.
I suggest that you take the dovecot source, look at src/lib-storage/inbox/index/imapc and the other backends in that directory, and implement one that speaks your REST API as a client.
Since you're familiar with .NET, I would suggest hacking either of the following implementations of IMAPv4 servers to your liking:
Lumisoft Mail Server - a very old project indeed (let's call it "mature", huh?). Don't be too turned off by the decade-old website and the lack of a github link - the source is provided under "other downloads".
McNNTP - also an older project and with a major focus on NNTP (as the name says) but very close to what you're trying to achieve in terms of the IMAP component. Take a look, you'll probably find this a good starting point.
I'm just starting iOS app development with Swift (and in general) and I'm looking to get some information on popular practices when creating apps that require communication over arbitrary networks (i.e. not necessarily on the same network). I tried searching this on google but the answers weren't entirely what I was looking for; hopefully somebody can point be in the right direction. I wouldn't mind paying for a service, but unfortunately I don't know the first thing about backends and don't want to end up overpaying for services that I don't need. For example, I found an API called Parse, but I think it has far too many features that wouldn't benefit my app. Here's the main premise of the app:
There are two versions of the app - one for Admins and one for Employees
The Admins have the ability to post notes to a central list of notes for the employees to see
The Employees can access this list and scroll through it to pick which one they want to open. After a certain number of time, the notes expire and are removed from the list automatically
It's as simple as that. There likely won't be too many notes getting sent at once, so a large database isn't needed. My questions are as follows:
Do I need a database to store the notes, or can I handle it in some other way?
How is communication generally handled? The only things I've come across are ways to communicate when you're on the same WiFi or Bluetooth, but I haven't seen anything outside of that. How does an app like GroupMe communicate to users?
This is more of a general question, but how can you tell if you need a backend or not? I'm still kinda confused on the interaction between the frontend and backend.
Any help for any of the questions is greatly appreciated. I feel as though I don't even know where to start with a project like this.
EDIT: To clarify, I'm just looking for a place to start, not code or any implementation.
It's as simple as that. There likely won't be too many notes getting sent at once, so a large database isn't needed. My questions are as follows:
Do I need a database to store the notes, or can I handle it in some other way?
Yes you need some kind of database. That could be something complex like MySQL or something simple like writing a txt file for each note to the disk, with the filename being the date of the note.
You could use a service like Parse or run your own PHP server and write the software yourself. Parse is cheaper for a small database, running your own PHP server is cheaper for a big one and it gives you more control.
(You don't have to use PHP, but that is the most popular language for these things and it's what I use).
How is communication generally handled? The only things I've come across are ways to communicate when you're on the same WiFi or Bluetooth, but I haven't seen anything outside of that. How does an app like GroupMe communicate to users?
Usually your the phone sends a HTTP POST request to the server with some text in JSON format in the body of the HTTP request.
The server then responds with more text in JSON format in the response.
On the phone you use NSURLSession to handle to do the network communication and NSJSONSerialization to encode/decode the content. On the server, there will be something equivalent available.
Usually there would be a username and password or some other authentication system in the HTTP POST JSON text that tells the server wether or not the user is allowed to do whatever they're trying to do.
All communication between the phone and the server must be encrypted using SSL to protect your users. Do your homework and make sure you get this part right before you deploy your app to the store.
Parse will handle all of that stuff for you, but it's good to at least understand what's going on.
This is more of a general question, but how can you tell if you need a backend or not? I'm still kinda confused on the interaction between the frontend and backend.
You know you need a backend if you want two devices to communicate without being on the same WiFi/Bluetooth network. This is a security feature that cell network carriers (and home broadband ISPs) enforce to prevent malicious activity.
Generally only a commercial internet connection (and commercial router) will allow anonymous incoming network packets to get through to a phone/computer connected via that internet connection. Consumer internet connections only allow traffic coming in from a known source (for example, if you ask Google for some data, the router will temporarily allow Google to send some data to you. But if Google just sends some data without a phone/computer in your home asking for it, then it will be rejected).
You should be able to take what I've written and do a bunch of research.
If you decide to go with writing your own system in PHP, it comes pre-installed with OS X (just has to be enabled) and you can access it by IP address from the phone as long as you're on the same IP address. That should get you started for testing/development purposes at least.
The only part you won't have is SSL. Starting in iOS 9 (it's almost here!) you will need to disable NSURLSession's built in check for SSL or else it won't let you connect to the test server.
I have a rails app that just launched and I have over a thousand users signed up to the site. We are interested in keeping in contact with the community through email newsletters. We have been using a third party tool and migrating the email addresses from one database to another. Is there an easier way to do this from rails? I am open to suggestions.
http://github.com/adzap/ar_mailer
ar_mailer will allow you to queue up mail and send it out over time so you don't violate any mail rules put in place by your host.
If you really, absolutely, definitely need to send to these emails yourself, and you're dealing with a larger user base, you'd probably want to go through a distributed setup employing something like Starling, RabbitMQ, or the all-new Resque to distribute the task of sending these emails across a multitude of workers.
It's a topic far from trivial, so all in all I'd recommend using an external service provider for this.
Use ActionMailer to generate emails from templates (views) using Erb. I would do this unless you wish track clickthroughs back to your site from those mails, then you may be better of using a third party.
As an aside (I'm sure you know this but), be aware that you'll need to inform your ISP and demonstrate to them that you have permission from your users to send these emails, else they may block you from sending without warning.
You can simply loop through all the emails and send individual emails. It's pretty simple using rails and there are tutorials everywhere.
You could also use the third party API to add automatically your users emails to their database. If you could provide the third party you're using, I could maybe give you more information.
Some newsletter website give you the option to add emails directly in their system via a given form. Why not use this form? You would put it somewhere on the website and the system will add the new emails to the third party software and not to yours. Of course this only works if you have a "enter your email here to subscribe" kind of logic.
I'm looking for an alternative to Google Apps for sending email from my rails app. (Google limits the number of messages you can send).
Do most people roll their own, or is their a preferred provider?
I'd love to be able to dynamically create addresses for my customers: customer_name#myapp.com
We use AuthSMTP which does a quality job of informing you when you've been reported as SPAM and keeping you off of block lists.
It costs, but is very reasonable.
Just to give an update, it looks like the new hotness is SendGrid. Take a look if you're in the market for hosted email for your rails app.
If you're hosting on a Linux server and just looking for something to send emails from your rails application, rolling your own SMTP server really isn't rocket science. You'll find tons of manuals out there on how to deploy a SMTP server on your Linux distribution of choice.
Postfix would be the de-facto-standard and Qmail is my personal weapon of choice when it comes to security, reliability and speed.
If you want to provide your customers with mailboxes (POP and/or IMAP) for incoming emails, it wouldn't be rocket science either to deploy your own server, but the administration of this servers and the support calls from your customers could become a pain. :-) In this case I would go for Google Apps. The $50/year/customer for the business option probably would be the choice for your corporate customers.
I've had good success with Postfix with Rails and Linux in production. Once the site got beyond small, connecting to Google Apps for each email became impractical (it would fail or timeout somehow once every couple hundred emails). For doing a mass mail to all your users, very impractical.
There is a good series of articles for getting postfix set up on SliceHost's pages
http://articles.slicehost.com/2008/9/2/mail-server-overview