I have a site with many "channels" customers can subscribe to. I want to send each customer a unique weekly email based on any new content from their subscriptions, plus some suggestions of additional subscriptions. Feels like a very common use-case.
I'm not sure the best approach or service to use here. I could code everything myself and run a recurring job to send the emails. It feels like there should be a service which has a list of my user ids and pings my server for all the custom fields to inject into an email. But I can't find such a service.
Can anyone recommend a best practice for doing this kind of thing? (I'm using RoR on Heroku and have an Mandrill account.)
From looking over your question, it seems that you're looking for the "missing piece of the puzzle" to help you get this setup. So I'll give you some insights that we've found:
Queuing With Redis & Resque
Resque & Redis are technologies for handling queuing with web-based applications
Resque is a gem which allows you to host "queues", and process them in the background (with cron jobs & the like). On the Heroku platform, you'll be able to use the Heroku Scheduler to process the queue as-and-when required
Redis is a technology which runs parallel to your application, allowing for the storing of JSON-type data objects (key/value pairs). Heroku has several third-party redis addons you can use, which will get you started & allow you to scale. Redis is best described as "RAM" for your app - allowing you to store a series of data objects that can be used independently to your app's RESTful infrastructure
Resque Railscast
Ryan Bates actually explains the Resque process very well on one of his RailsCasts. I'd check it out & implement the code to get it something down; you can then iterate on top of it later!
I can post live code if you want?
Related
I have a web and worker tier running on Elastic Beanstalk, the worker tier is used for background processing such as delivering email. However, what I want to do is the following.
Each day, look at who should be receiving an email (triggered by a complex set of data in the web tier), the fetch the required information and email it. This also needs to be able to work at scale.
The thoughts I have at the moment are:
Configure the worker tier to have access to the web tier database, have it periodically (via a cron.yaml) directly access the database to build the emails.
Have the web tier set up and manage a list of scheduled emails, which contains all the required information denormalised, in which the worker tier (via a cron.yaml) periodically polls and actions.
The second option does sound a lot like SQS, but the difference is these emails could be up to a month in the future at a specific time, which it sounds like SQS couldn't offer.
How would you do this? Any better ideas?
Thanks,
Dan
I'm hosting a small pet-project Ruby on Rails application and I have a small feature:
I want to send an email to every person who bid in an auction every time they are outbid.
If 10 people bid on an item, you can imagine how this will scale if I were to just send the emails willy-nilly from my RoR application.
What do I need to use for this scenario if I want to queue up emails that need to be sent? Since this is hosted on a DigitalOcean VPN ease of installation and configuration is a must for me, as well as a great RoR integration.
Any suggestions on what I need to look for?
For the Emails services you might consider, I think this is pretty good price comparison: CLOUD EMAIL SERVICE PRICE COMPARISON, so the services which are pretty mature and easy to integration(you can find the gem in github) is following four: Sendgrid, Postmark, Amazon SES, Mailgun.
Amazon SES provide a very basic email sending solution, but other services like Mailgun provide extra services like mail list or campaign etc.
Cause the sending mail might be a time consuming task, so you can use delayed job(queue service gem) to delay the mail sending, now the most popular one should be Sidekiq, other available options are Resque, delayed_job. Sidekiq and Resque need Redis installed(it need to serialize the jobs into Redis), the delayed_job can just serialize the jobs into the database.(If you just need a very simple delayed job processing, you can choose delayed_job).
So the simplest combination for now can be Amazon SES(https://aws.amazon.com/ses/) + delayed_job(https://github.com/tobi/delayed_job).
Good luck!
I'm building a Rails 3 application that I plan on provided as a subscription-based SaaS (Software as a Service) product. Basically, I want users to be able to hit my "Sign up" page, create a new account, and immediately start using the software.
A good example of what I'm trying to accomplish is: http://www.getharvest.com/
Here's what I need to happen when someone signs up:
A MySQL database for them is generated on the db server
A sub-domain is created (e.g., companyx.awesomeapp.com)
The Rails app should know the appropriate database to connect to based on the sub-domain
Are there any good guides out there for setting this stuff up? Even better, are there services that you can purchase to automate this type of thing? Ideally, I'd like to just worry about writing my Rails app and then be able to plop it atop some awesome Rails SaaS infrastructure.
(Also, I need a way to bill them monthly, but I think that's a separate question/problem.)
Heroku would let you get up and running quickly. You can manage the infrastructure using the heroku gem. Here is the documentation to the client which should allow you to manage heroku applications remotely. Using heroku would allow you to scale applications on an individual level and let you focus on the code of the application instead of the hardware.
I'm trying to create a ruby on rails ecommerce application, where potential customers will be able to place an order and the store owner will be able to receive the order in real-time.
The finalized order will be recorded into the database (at the moment SQLite), and the storeowner will have a browser window open, where the new orders will appear just after the order is finalized.
(Application info: I'm using the HOBO rails framework, and planning to host the app in Heroku)
I'm now considering the best technology to implement this, as the application is expected to have a lot of users sending in a lot of orders:
1) Each browser window refreshes the page every X minutes, polling the server continuously for new records (new orders). Of course, this puts a heavy load on the server.
2) As above, but poll the server with some kind of AJAX framework.
3) Use some kind of server push technology, like 'comet' asynchronous messaging. Found Juggernaut, only problem is that it is using Flash and custom ports, and this could be a problem as my app should be accessible behind corporate firewalls and NAT.
4) I'm also checking node.js framework, seems to be efficient for this kind of asynchronous messaging, though it is not supported in Heroku.
Which is the most efficient way to implement this kind of functionality? Is there perhaps another method that I have not thought of?
Thank you for your time and help!
Node.js would probably be a nice fit - it's fast, loves realtime and has great comet support. Only downside is that you are introducing another technology into your solution. It's pretty fun to program in tho and a lot of the libraries have been inspired by rails and sinatra.
I know heroku has been running a node.js beta for a while and people were using it as part of the recent nodeknockout competition. See this blog post. If that's not an option, you could definitely host it elsewhere. If you host it at heroku, you might be able to proxy requests. Otherwise, you could happily run it off a sub domain so you can share cookies.
Also checkout socket.io. It does a great job of choosing the best way to do comet based on the browser's capabilities.
To share data between node and rails, you could share cookies and then store the session data in your database where both applications can get to it. A more involved architecture might involve using Redis to publish messages between them. Or you might be able to get away with passing everything you need in the http requests.
In HTTP, requests can only come from the client. Thus the best options are what you already mentioned (polling and HTTP streaming).
Polling is the easier to implement option; it will use quite a bit of bandwidth though. That's why you should keep the requests and responses as small as possible, so you should definitely use XHR (Ajax) for this.
Your other option is HTTP streaming (Comet); it will require more work on the set up, but you might find it worth the effort. You can give Realtime on Rails a shot. For more information and tips on how to reduce bandwidth usage, see:
http://ajaxpatterns.org/Periodic_Refresh
http://ajaxpatterns.org/HTTP_Streaming
Actually, if you have your storeowner run Chrome (other browsers will follow soon), you can use WebSockets (just for the storeowner's notification though), which allows you to have a constant connection open, and you can send data to the browser without the browser requesting anything.
There are a few websocket libraries for node.js, but i believe you can do it easily yourself using just a regular tcp connection.
I'm looking for an alternative to Google Apps for sending email from my rails app. (Google limits the number of messages you can send).
Do most people roll their own, or is their a preferred provider?
I'd love to be able to dynamically create addresses for my customers: customer_name#myapp.com
We use AuthSMTP which does a quality job of informing you when you've been reported as SPAM and keeping you off of block lists.
It costs, but is very reasonable.
Just to give an update, it looks like the new hotness is SendGrid. Take a look if you're in the market for hosted email for your rails app.
If you're hosting on a Linux server and just looking for something to send emails from your rails application, rolling your own SMTP server really isn't rocket science. You'll find tons of manuals out there on how to deploy a SMTP server on your Linux distribution of choice.
Postfix would be the de-facto-standard and Qmail is my personal weapon of choice when it comes to security, reliability and speed.
If you want to provide your customers with mailboxes (POP and/or IMAP) for incoming emails, it wouldn't be rocket science either to deploy your own server, but the administration of this servers and the support calls from your customers could become a pain. :-) In this case I would go for Google Apps. The $50/year/customer for the business option probably would be the choice for your corporate customers.
I've had good success with Postfix with Rails and Linux in production. Once the site got beyond small, connecting to Google Apps for each email became impractical (it would fail or timeout somehow once every couple hundred emails). For doing a mass mail to all your users, very impractical.
There is a good series of articles for getting postfix set up on SliceHost's pages
http://articles.slicehost.com/2008/9/2/mail-server-overview