I've used straight Mongrel, I've used Mongrel clusters behind Apache, I've looked at Thin, and I'm becoming very intrigued by Passenger. I've looked at Nginx, too. I've looked at MRI, Ruby Enterprise Edition, Rubinius, and JRuby. There are a lot of options, each claiming to be the new holy grail.
What is the best option out there for a brand new, fully up-to-date deployment? The only assumptions are this:
The app is Rails 2.2 based. (I know 2.2 isn't fully released yet, but neither is this deployment.)
The server is Linux-based. Probably Ubuntu Hardy, but really, whatever works best in this case.
Rails will need to be fully functional and probably talk to a MySQL database.
Everything else is negotiable.
Given these especially broad constraints, which combination of software will yield the best result, in terms of concurrency and low overhead?
I'm leaning toward Apache with the "worker" mpm and Passenger + Ruby Enterprise Edition, simply because it offers immediate stability and simplicity of setup and maintenance.
Am I likely to be particularly better off with another option?
I switched from Mongrel Cluster to Passenger two weeks ago (Debian Linux Server). I didn't look back for a second. Passenger is probably the easiest way to get your new server up and running. Performance and reliability are reasonable too.
Personally, I like to spend my time working on exciting new Rails projects rather than dealing with deployment issues - Passenger enables me to do exactly that. However, Mongrel or something else may still be preferable if you have some kind special requirements (doesn't apply for most products).
This morning, DHH talks about this very topic on his own blog:
But somehow the message of Passenger has been a little slow to sink in. There's already a ton of big sites running off it. Including Shopify, MTV, Geni, Yammer, and we'll be moving over first Ta-da List shortly, then hopefully the rest of the 37signals suite quickly thereafter.
So while there are still reasons to run your own custom multi-tier setup of manually configured pieces, just like there are people shying away from mod_php for their particulars, I think we've finally settled on a default answer. Something that doesn't require you to really think about the first deployment of your Rails application. Something that just works out of the box. Even if that box is a shared host!
http://www.loudthinking.com/posts/30-myth-1-rails-is-hard-to-deploy
Tobias Lütke on the topic of switching Shopify (million requests/day) to Passenger:
All this means that the total amount of memory that is used by Shopify during normal operations went from average of 9GB to an average of 5GB. We evenly distributed the savings amongst more Shopify processes and more memcached space which moved our average response time from 210ms to 130ms while traffic grew 30% in the last few months.
In conclusion: I cannot see any reason to choose a different deployment strategy at this point. Its simple, complete, fast and well documented.
http://blog.leetsoft.com/2008/11/15/passenger
We've been using the old standard nginx -> mongrel stack for the last 18 months, and although it was not trivial to set up the first time around, it's proven flexible, and has dealt with some very high traffic sites for us. Nginx in particular has been absolutely rock solid and fast, and if you can get your app page-caching you can deal with a lot of requests.
Stuck mongrels have been an issue, so we use monit to kill them when they misbehave. Again, it was not totally trivial to set up, but we've used the same process on many many sites at this point.
We haven't played with passenger yet, so perhaps it's easier and more stable, I'll defer to the other responders on that one, all I can say is that there's no reason at all you can't build a solid stack with nginx and mongrel.
We have switched fron NginX+Mongrel to Passenger.
I fully believe that Passenger is going to be the new standard for rails, despite NginX and Mongrel cluster being endorsed by some very smart people. Recent advances in Passenger have really propelled it forward.
Our current configuration is something like this:
Web servers
Ubuntu 8.04 LTS
Phusion Passenger on Apache2
MRI Ruby 1.8.6 and friends (form apt)
Ruby Gems 1.3.0 (Installed from source)
Database servers
Centos 5
MySQL Cluster (we just switched to this, but it is promising)
Having standardized on the exact linux distro we've been able to write Capitrano recipes to help deployment (slight variations in configuration have been the source of MANY service outages) and otherwise simplify our lives.
Have a look at Litespeed. You can get a free version that runs on 1 cpu or pay to get multi cpu. It is a bit expensive but is rock solid and handles rails brilliantly (i.e. uses less memory and is less of an overhead to monitor and setup). I run a massive amount of apps on it and it doesn't miss a beat.
We also switched from Mongrel to mod_passenger and found stability greatly improved with this effort required to setup and maintain. Good choice.
Another bit of gold:
Josh Peek's Slicehost gem is full of Capistrano recipes that are much simpler and much more organized than Deprec. Nothing in there is especially Slicehost-specific, either.
I'm hosting my new apps with Apache2 and Passenger on Ubuntu Hardy. Seems like the easiest and best option for most scenarios. I have just joined Slicehost.com for that purpose. They seem to get good reviews and have the most competitive prices of first class hosts.
I can't really endorse them yet because I'm a new client but the set of guides and range of support options is impressive.
What you are not mentioning is how large and popular your app is/will be. This criteria could affect the decision process.
Capistrano + Deprec for actually setting up my stack on Ubuntu and physically managing the deployment.
Nginx proxying to Mongrel clusers for the server architecture. It isn't the newest, bleeding edge technique but it works well, it is getting well-documented, and it is very, very high performance even when working on small VPSes. Assuming you haven't borked the application you can Slashdot a 128 MB Slicehost VPS and it just keeps coming back for more.
Having said that: there were a lot of gotchas the first time around, until I figured out how Nginx actually worked. After that its amazing -- like a little Apachelet with a slight Russian accent.
Related
I've been playing around with Rails (version 3) for a few months now and I understand the framework fairly well. However, I have yet to develop a large website which offers lots of database access and user interaction.
I'm fairly skeptical of the following:
The Speed and Scalability of Ruby (I've heard that its up to 10 times slower than most other server-side languages).
The extra background processing that Rails as a framework (multiple levels of abstration).
The lack of enterprise-level web apps that run on Rails (the only ones that I can think of Groupon, Github and Hulu).
The complexity of the environment (nginx > mongrel > rails > ruby > website).
The behind-the-scenes SQL operations (I know that these can be optimized, but I'm sure that I'll miss some).
For these reasons, I'm unsure whether to continue using Rails or to switch to something that is built ontop a more performant language .. say Java Spring.
Please advise :)
There are tons of large sites and infrastructures in production that use Rails. This question has also been asked to death over the years of Rails being actively used for all manner of web apps, large and small.
Short version is that it is not the fastest language around but despite that scales fine if you know what you are doing. And you should have enough money to hire people that know what they are doing if you actually have any problems of scale.
Scaling any webapp is hard, use the language/framework you know. Programmer happiness is king.
You can get good performance with Ruby. Easy:
require 'inline'
inline :C do |builder|
builder.c <<-C_CODE
void run() {
// Write your entire application in C here
}
C_CODE
end
run
Problem solved ;)
Not the speed and scaling discussion again?
In webdevelopment the things that are the slowest is the network communication (receiving the request, getting al your data back), the database (getting all your data from the database), and most of the times it is not about the computation time at all.
While it is true that Ruby and Ruby on Rails seem more focused on programmer happiness, I think that every decent web-application built in .NET or Java has as many levels of abstraction.
The complexity of the environment? I think you mean deploying? There are a lot of options, but the most used options are Passenger (very easy deployment on top of an apache or nginx), or Torquebox.
Torquebox for the moment is the fastest, best scaling solution (based on JBoss Application server), and several big names in the Ruby community are calling Jruby the implementation of choise to deploy your applications. While AFAIK the commonest deploy still is using REE (Ruby Enterprise Edition) and Passenger.
Unless you know you are going to have to do serious mathematical, cpu-intensive operations, I think the question you should ask yourself is: which framework/language will give me the quickest result?
If you are very proficient in Java/Spring, that might be the answer for you. But if your only worry is performance in general, I would say: do not hesitate and go for Ruby on Rails. It is a pure joy to develop in. The ruby community is really awesome if you would encounter any issues: support is just a post away.
And to conclude, I want to add a few very big sites using Rails: LinkedIn is using rails (and jruby), and Twitter still is using Rails for their frontend.
We have a very simple function (We look something up from a third party database and return an answer. It's literally five lines of code.) We would like to offload this task from our main server because we expect a high volume of traffic for this one function and would like to optimize it.
We are thinking about testing the promise of many cloud/PaaS providers, where they handle scaling and performance responsibilities.
We're most interested in Rails environments, but are curious to hear experiences from others about any company in the space.
Here are the PaaS companies we found that supports Rails:
1) Heroku
2) DotCloud
3) Duostack
Questions:
1) Do you know of other Rails-specific companies? Also feel free to list non-Rails companies since we're interested in following other companies in case they eventually provide Rails support.
2) How has your experience been with these companies?
Foreword and disclaimer: I work for DotCloud; so the following might be biased. You've been warned.
DotCloud could be interesting for you if you like the following features:
run something else than Ruby (what about some Django or Pylons code talking with your SQL DB? Or even some PHP blog like WordPress or Drupal, using the same user authentication database?)
experiment with databases like Redis or MongoDB, or background ruby workers, without paying for add-ons
SSH access, crontab access (without requiring an add-on)
cheaper workers (I didn't come up with this one; some of our users coming from the Heroku world told us that workers were insanely expensive there)
Duostack is indeed very nice if you want to mix specifically Rails and Node.js. I've been told that they had awesome auto-configuration facilities.
Finally, if you only plan to do Rails and nothing else, ever, you might as well stick with Heroku since they've been in that business for a while, and are probably more mature than the first two of the batch.
Shameless plug: DotCloud is offering a beta test drive; so if you want to see what it looks like, just subscribe to the beta and you will be quickly enough be able to see for yourself. Heroku has a free tier as well.
You could add EngineYard in the mix - but i'd be inclined to use Heroku as my first choice, Dotcloud second (it's a newish product, and is very good but still in development)
If you want more control over your app/servers or want to run it on any cloud or your own infrastructure without having to download/deploy anything, you can try Cloud 66 (www.cloud66.com)
Disclaimer: I work for Cloud 66
A lot has changed on the scene since this question was asked. We recently looked into these services and settled on Heroku, but even more recently decided to continue managing my own deployments directly on EC2. Here are some points not mentioned in the other answers.
Heroku
Now supports much more than just ruby
Has really great-looking support for PostgreSQL
Uses LXC for process containers, like DotCloud
DotCloud
Is now Docker, and is putting a lot of manpower into developing docker.io
Doesn't have a free tier any more
I'm not sure if DotCloud is using Docker internally or not, since the docs say explicitly it isn't production-ready yet.
Our decision to stick with plain EC2 was motivated by the fact that it's cheaper and affords a lot more flexibility. For example, we use local-only http servers behind our public server to do some of our request processing, which doesn't really fit into the PaaS models out there. We would have had to reimplement all our back-end components as redis workers, and pay for them as additional dynos. The fact that Amazon RDS now supports PostgreSQL was also a compelling factor. Incidentally, Amazon has a full-stack PaaS offering as well, Elastic Beanstalk.
Just stumbled upon the question. There are similar ones around here. The problem is also: The PaaS scene is changing very quickly. New vendors are popping in every week or so.
Nowadays OpenShift from Red Hat might also be mentioned here as a Ruby PaaS.
OFFTOPIC + shameless plug: I have compiled a list of PHP PaaS here: http://blog.fortrabbit.com/comparing-cloud-hosting-platforms/
I have 4 servers behind a load balancer and a staging server, a db server, and a utility server for a web application that hosts a number of web sites.
Should I make the jump to Chef to manage these servers or should I just maintain them manually? The servers were built using sprinkle but at that time there were only two. Now that there are four maintenance is becoming more of an issue.
I'd like to hear experiences and the pros and cons of chef and other chef-like tools.
Thanks!
We moved to Chef, and we now have a 1 minute redeployment for our app. So it certainly pays off.
However it took a long time (a few months) to get to the point where we were happy with the chef deployment strategy. With hindsight we would have had several spare boxes around to try out a 'from scratch' deployment. I certainly wouldn't advise trying chef in a production environment without an exact mirror of the setup and lots and lots of tests, nor would I advise using chef on a setup that hasn't been 'cheffed' from scratch.
Having said that, Chef is far better than the other options we looked at and now that we are out the other side it's a breeze deploying a new version of the app on multiple servers. In future I'll be using it for any staging or production environment I have.
In summary, yes, but only if your client/employer is aware that it may take some time before they see the benefits, which will be considerable.
Chef has a steep learning curve, so it will take a while - at least a few weeks - to become familiar with how to use it.
But once you pick up the basics, it is a very handy system, and can simplify any number of tasks - even for the smallest of infrastructures.
A few notes for when you start.
You will be setting up and tearing down cloud servers dozens of times, just to get the hang of it. Experiment.
The standard opscode cookbooks (github.com/opscode/cookbooks) are very useful. But you will need to extend/customize many of them for your particular case. And you will need to search the 'net for cookbooks that are missing from the opscode/cookbooks repository.
Read the opscode cookbooks, and read the 37signals cookbooks too.
The application and database cookbooks are geared towards standard Rails apps with MySQL and Memcached. To the extent that this describes you, you are way in luck.
Is there a "best" rails server to use for development (mongral, webrick, etc..)? Or does it not really matter?
Short answer: develop on what you will deploy on.
Long answer: good Rails containers, like Unicorn, Thin, etc, really don't impact your development much but will impact production. And, each has just enough configuration related quirks that interact with your code and choice of gems that just diving in and working in that environment can save a lot of time, even if using continuous integration, but especially if not.
The "best" in my opinion is Passenger in conjunction with your choice of Apache or nginx, whichever you're familiar with. It's perhaps the only game in town when it comes to getting an application running quickly and reliably. The Phusion team has invested a lot of time and effort into building a complete package for a scalable deployment platform.
Mongrel is only designed to be part of your stack and requires a number of supporting pieces to work properly. It also has to be managed with a separate process launcher and that can be frustrating and can cause serious issues if done wrong. Mongrel2 is a better platform but will need some time to be properly assimilated by the Rails community.
Webrick was never intended as a production web server. It is only a toy server for testing. It is painfully slow and can only service one request at a time.
See my answer here Recommendations (and Differences) between different Ruby on Rails Production Web Servers
What is the ideal rails server? Lets assume its on a VPS, such as Linode. Lets assume that any of the sites won't become the next twitter, but they should scale well. It must also support multiple sites and all sites are rails 3. And the database must be on the same server (for now).
Should it use apache or nginx?
Ruby Enterprise Edition or just normal Ruby?
Ideal linux distro?
MySQL, PostgreSQL or something else?
How should the directories be laid out (where to put your rails sites or anything else)?
Deployment options?
Anything else?
Should it use apache or nginx?
Nginx seems to be the preferred route here, unless you need specific Apache features. If you're using Passenger, both Apache and Nginx are supported. More on Passenger here.
Ruby Enterprise Edition or just normal Ruby?
I'm pretty sure REE is only available for Ruby 1.8, which isn't really the preferred version for Rails 3 anymore. Rails 3 had some problems with 1.8, but they might have fixed them by now. Generally, though, Ruby 1.9.2 runs Rails 3 well.
Ideal linux distro?
This really doesn't matter. If you're not sure, Ubuntu is a good choice as there's a lot of knowledge out there, and it's pretty easy to use. Slicehost has a bunch of great articles on getting started with a VPS, and a lot of them focus specifically on Ubuntu: http://articles.slicehost.com/.
MySQL, PostgreSQL or something else?
This is definitely subjective. MySQL is definitely the most common out there, and if you're really unsure, it's a good starting point. However, people often argue that PostgreSQL is cleaner and easier to use that MySQL. If you're just starting, I would recommend MySQL just because of the amount of information out there already.
How should the directories be laid out (where to put your rails sites or anything else)?
You can put your rails project anywhere you like, so I like to just put it in my home directory. Just make sure that your web server has access to your static assets.
Deployment options?
Capistrano is popular. You just commit your changes and cap deploy, and you'll be up and running.
Anything else?
If this all seems overwhelming, look into a simpler solution like Heroku. They set everything up for you, and, while you loose some flexibility, you won't have to worry about any of this. Their pricing isn't too bad, and they offer a free option.
I think kyl summed it up pretty well. But I figured I would let you know exactly what I have been using since rails beta4 (and now with RC). This setup has worked well for me:
Rackspace Cloud servers - Gives you full control of the server. You can resize your servers on the fly. You can also take snapshot images if you want to duplicate your setup for another site. Its cheap and in my opinion better than Amazon cloud.
CentOS 5.4 - solid, but as kyl mentioned probably any distro will work fine.
Ruby 1.9.2RC 2 - no problems so far on Rails 3 for me. Will definitely use Ruby Enterprise when they port it to 1.9.2 (not sure if that is in the works, though??)
Nginx - fast and lightweight. I like it much better than apache. Works well in front of Passenger, mongrel and thin.
MySQL - just personal preference for this. I have been using it for years. It is easy to set up a master/slave or master/master config if you need to scale. Some people have success just using sqlite, but I prefer something a bit more robust.
Github - a must for me for source code control. Bundler works very well with github
Application Server - I am still debating what to do about this. I was happy with Passenger 2.2.15 until I saw how long it took to spawn new ruby processes to handle concurrency. It takes up to 30 seconds to spawn a new process for me and the app is locked, so no requests can go thru while it is spawning. I am investigating right now if it is my app or Rails 3 that takes so long to load. This problem is fixed with Passenger 3, though. Hopefully that will be released soon. As a result of this, I am probably going to use Thin or Mongrel until Passenger 3 comes out.
Capistrano - works great for Rails 3. I would recommend finding some cap recipes for versioning your app with git tags... or just write your own.
Anything else? Not really related to the server, but I would recommend using the new plugin API for any part of your app that is reusable. Read up on railties and engines. Its simple to create a gem with Jeweler and version it with github using jeweler rake tasks. Then you can deploy from a github tag or from master, by adding the gem and github source in your Gemfile and bundle install or update it. I recently ported all of my common app code (blog, authentication, etc..) to Rails 3 engines and it is working great. And any time I need to reuse that code, I just drop it in the new app's Gemfile.