So I have a Rails application. It currently runs separate as front-end and back-end + database.
I need to scale it to have several back-end servers.
The backend server has Resque background workers running (spawned by user front-end requests). It also relies heavily on callbacks.
I am planning the following setup:
|front-end| --- |load-balancer (haproxy or AWS ELB)| --- Server 1 ---- Postgresql Database (+++ other DBs added via replication later if needed)
\___ Server 2 ---/
++ (other servers added in the same fashion later )
I have concerns about how to deal with putting Database on a separate machine in this case.
1) I intend to create a new empty Rails app with schema identical to initial back-end. Have it running and accepting updates / posts via HTTP and keep connected via remote SSH (to trigger :after_commit callbacks in back-end). Is it a bettergood idea?
2) I am using Postgresql and intend to switch to an enterprise DB once the need arises. Currently the need is to scale the part of back-end that does processing not the database.
3) Does this approach seem scalable?
I'm not sure I really understand your question. Generally in production applications the database layer is separate from the application layer. I can't quite tell if this pertains to you, but it's definitely an interesting watch. http://vimeo.com/33263672 . It talks about using a redis layer between the rails and db layers to facilitate queuing, and creating a zero downtime environment. It seems like it would be a better solution than using a second rails stack? I think it should look something like this;
|ELB| Web Servers |ELB| Application Servers |RRDNS| Redis Servers | PostGreSQL Servers |
If I am understanding your meaning. If not, that video link is still worth a watch :)
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
My team and I are currently building a Ruby on Rails web application (SaaS model) that will be used by at least 5k users, and there will be probably more after.
While we reach the end of the development process I was wondering what kind of hosting we were going to use.
I first thought about using a standard web server with Nginx and Puma, and a second server dedicated to the database, but the scalability is a thing that we really want. So I started to check for an alternative solution (knowing the fact that no one of our team is an admin), and I end up on Heroku.
So, my question is why should I use something like Heroku ? After watching the prices I've the feeling it's going to cost more than a standard server for less performances, but i'm still intrigued by it : if people use it, there must be a reason.
Heroku & DigitalOcean etc, are just services which make deploying to "cloud" hosting easier. Heroku uses AWS's computing infrastructure to host your apps, but provides an environment to help deploy them easier:
Why do people use Heroku when AWS is present? What distinguishes Heroku from AWS?
Heroku vs...
Some of the important points to note about Heroku is you don't have direct access to your database, as well as relying on their infrastructure to hold up (E.G if something happens to their datacenters -- which has happened before -- you're at the mercy of Heroku & AWS's engineers).
There are two issues we have with them:
You're obliged to use their AWS Postgres DB
You've got to route through Heroku's domain
--
Choice
The choice you really have is whether to use something like RackSpace (like us), or use a service such as Heroku or DigitalOcean. To me, there is not much difference, as Heroku / DO just provide an environment to push to VPS's running on their respective partner hosting providers.
With the ease of deployment (git push), you get the downside, which is your database, environment etc will not be directly configurable. I.E if you use Heroku, it's very difficult to get off it (transferring data etc).
By using a direct solution, such as RackSpace or AWS, you are in control of the environment, which means you get direct access to your database, dependent services & other aspects of the system. Of course, it comes with more maintenance, but you aren't locked into one system, as you are with Heroku
--
System
In terms of the system, Heroku locks you into using their domain infrastructure:
The domain example.herokuapp.com will always remain active, even if
you’ve set up a custom domain. If you want users to use the custom
domain exclusively, your app should send HTTP status 301 Moved
Permanently to tell web browsers to use the custom domain. The Host
HTTP request header field will show which domain the user is trying to
access; send a redirect if that field is example.herokuapp.com.
This is coupled with the lock-in of using the likes of different dependencies & other aspects of the Heroku system. It's basically a store-front for Amazon's AWS infrastructure
--
Recommendation
If you expect to have 5k users, I would highly recommend looking at
setting up your own environment, probably with RackSpace (as we
use them).
It's my opinion that Heroku is great for developers who just want to get an app up-and-running, but it's not very good with providing an environment you can control.
Some examples of this include your database - setting up on Rackspace etc means you get to define your database yourself. On Heroku, the db is kept in a third party datacenter, which you never see (try setting up PHPMyAdmin on Heroku)
Another example would be how you're going to scale the application. If you have memory leaks etc, you'll be far better positioned to have your own stack in place, rather than relying on someone else's. It allows you to define the specific details of how the app should operate - giving the ability to scale much easier
We have a Rails app that acts HTTP API only. On the client side, Ember.js is currently used. We are not overly impressed by Ember and really like the approach Meteor.js takes. So we'd like to exchange the client side with Meteor.js and communicate with the Rails server via websockets that speak the Data Distribution Protocol (DDP), so we can keep using the models, mailers and controllers in Rails. Implementing server side of DDP should be easy.
However, we're unsure how to make Rails talk websockets. We found Reel, which seems to make it easy to accept websocket requests in a standalone environment. Reel seems great as we'd like to implement the DDP on top of the Celluloid stack anyway. But what about running Reel in the Rails environment? Would we need "rails runner" for that? And we'd like to keep using the existing controllers to dispatch incoming requests (like, to add/change/remove resources). Is that even possible without having the request coming through Rack?
Any input is appreciated.
It's a bit late, but I've implemented DDP in Ruby, you can check it out here:
https://github.com/d-snp/ruby-ddp-server
It includes an implementation of EJSON as well. It's built on top of celluloid-websocket, and can be ran simply as a rack app.
I've made an integration with RethinkDB that can be used as a reference to build your own collections implementation.
https://github.com/d-snp/ruby-ddp-server-rethinkdb
I've also made a sample chat application that can be found here:
https://github.com/d-snp/celluloid-rethinkdb-chat
It's something that I have been longing to do as well, to integrate old "legacy" Rails code. Here is the best way I have found:
Since you would not be using any of Rails router/controller/views, but just the ability to read data and push it to the client, I recommend you use Rails to create JSON apis to the database, and deploy the code, then in Meteor you can consume the data via the http package, this would happen on the server at a regular interval and populate the MongoDB with the normalized data you need, then it would serve the browser client.
I am working on such an application that will keep a normalized version of the data in Mongo, and a relational version of the data in mySql (through Rails) this way I can preserve the legacy Rails functionality that I dont want to rewrite in JS, and get the benefit of Meteor for the one page that I need it most.
Hi I've been learning rails for the past half year and have a few apps up on Heroku. So for me I thought deploying apps onto the world wide web was just as simple as heroku push. However, I just got my first internship doing Rails and one of my seniors is talking about Apache and Nginx and I'm not sure how they fit in the picture, since I thought apps consisted of only Rails + cloud app platform. I have looked it up but I still don't get how and where it affects my app life cycle. Can someone explain what/where/when of using web servers?
So you've got your Rails app, and as you know you've got controllers and actions and view and what not.
When a user in their browser goes to your app on Heroku, they type in the URL which points to the Heroku servers.
The Heroku servers are web servers that listen to your users that type in the URL and connect them to your Rails application. The rails application does its thing (gets a list of blog posts or whatever) and the server sends this information back to your user's browser.
You've been using a web server the whole time, just it was abstracted away from you and made super simple thanks to Heroku.
So the life cycle is somewhat like this:
Whilst you've been building your applications on your development machine you've probably come across the command rails server. This starts a program called WEBrick which is a web server, and listens on port 3000. You go to your app via http://localhost:3000.
WEBrick listens on port 3000 and responds to requests from users, such as the "hey give me a list of posts" command.
When you push your code into production (in your experience via heroku push) you're sending your code a provider who takes care of the production equivalent of rails server for you.
A production setup (which your senior developers are talking about) is a bit more complex than your local rails server setup on your development machine.
In production you have your Rails server (often things like Unicorn, Passenger) which takes the place of WEBrick.
In a lot of production setups, another server, such as Apache or nginx is also used, and is the server that the user connects to when they go to your application.
This server exists often as a bit of a router to work out how different types of requests should be handled. For instance, requests to static files (css, images, javascript etc) that are storted on the server might just be processed directly by Apache or nginx, since it does a fantastic (and fast) job of sending static assets back to the client.
Other requests, such as "get me a list of all blog posts" get passed onto the Rails server (Unicorn, Passenger etc) who in turn do the required work and send the response to Apache/nginx, who send it back to the client.
Heroku does all this for you in a nice easy to use package, but it sounds like the place your working at manages this themselves, rather than using Heroku. They've setup their own bunch of web servers, and will have their own way doing an equivalent of heroku push which will send the code to the servers, and make sure they're up and running ready to respond to user requests.
Hope that helps!
Web Pages need a Web Server to make them available on the Internet.
So a site that is all static content (all just .html pages) just needs a web server and that's where Apace, nginx, etc come in. They are web servers.
When you use frameworks like rails, an additional component is added, an application server. This pre-processes the pages using the rails framework and then (still) uses the above mentioned web server to make the final pages (which are .html of course) available to the end users through their browser.
Passenger Phusion is an application server that, with rails will help manage and automate the deployment of code.
Heroku is a cloud service, meaning they take care of hardware and software allowing you to seamlessly publish you application without worrying about what is going on behind the scene. So the only thing you have to do is push your code to their Git and voila.
On the other hand, Rails can also be deployed on a system built by you completely from scratch, and you will be the responsible not only for the app development but also for the server maintenance and choice of the hardware and/or software. You could then choose between several application servers capable of running rails such as ngix.
Hope that helps.
I'm building a Rails 3 application that I plan on provided as a subscription-based SaaS (Software as a Service) product. Basically, I want users to be able to hit my "Sign up" page, create a new account, and immediately start using the software.
A good example of what I'm trying to accomplish is: http://www.getharvest.com/
Here's what I need to happen when someone signs up:
A MySQL database for them is generated on the db server
A sub-domain is created (e.g., companyx.awesomeapp.com)
The Rails app should know the appropriate database to connect to based on the sub-domain
Are there any good guides out there for setting this stuff up? Even better, are there services that you can purchase to automate this type of thing? Ideally, I'd like to just worry about writing my Rails app and then be able to plop it atop some awesome Rails SaaS infrastructure.
(Also, I need a way to bill them monthly, but I think that's a separate question/problem.)
Heroku would let you get up and running quickly. You can manage the infrastructure using the heroku gem. Here is the documentation to the client which should allow you to manage heroku applications remotely. Using heroku would allow you to scale applications on an individual level and let you focus on the code of the application instead of the hardware.
I'm very new to web programming (or actually, very old to it, since the last time I messed with the web was HTML 1.1), but now need to deploy a web application quickly. It seems like every time I turn around, there's new acronyms and technologies to learn (JSON, XMLRPC, GWT, Javascript, Rails, etc).
Here's what my app must do:
Given a username and password, authenticate (easy enough, everything does that, apparently).
Allow the user to upload a large glob of data for processing.
Process that data.
Allow the user to download their processed data.
I've already got Java scripts and a database for handling the data. On one machine, I can run a series of command-line programs to process an incoming datablock and put the results back into a mysql database. That is already present and working.
I want to construct a web front-end to this task, using these existing and tested methods. I'm currently leaning to this approach:
Have two machines, a database machine and a web server. That approach allows for later scalability, if necessary, but also requires that I can't assume that the programs that I use to access the data and manipulate it are locally stored.
Use a Ruby DRb application to create a server and a client. The client would pass data to the server which would in turn call these applications.
Use some other Ruby interface to interact with the DRb for the web frontend.
Here's my problem: it looks like most Ruby applications for the web automatically try to build some kind of local database. All the Rails tutorials I've found start with making your own database and interacting with that, which is exactly what I don't want to do.
Is Rails the right technology for me, or using Ruby DRb? Is there some other technology I should be exploring?
If Rails or Ruby is the Right Thing here, what should I be looking at? I already have the Programming Ruby book, and have used that for some of the backend stuff as well as getting basic DRb stuff working.
Sounds like Rails might be a bit heavyweight for your situation. Perhaps Sinatra might be a better fit? It's an ultra-lightweight framework: a hello world app might look something like:
require 'sinatra'
get '/' do
"Hello World!"
end
Rails is fine. You can have development and test databases on your local machine and the production database on a remote machine. It doesn't have to be the web server. Get a copy of Agile Web Development with Rails. It'll teach you all you need to know.