I am beginning work on a new Rails project that is based on the premise of allowing users to create their own "sites." Each "site" would be a subdomain of the root domain (we'll use example.com). So if user Foo wants to create his own site at bar.example.com, each page request to a bar.example.com page would require fetching the a row in a sites table based on the subdomain.
My question is not how to code a multisite app, I think I have a pretty good grasp on that. My question is, from a scalability and performance perspective, would it be better to simply generate a new rails project for each site a user creates? Or is it ok run all sites out of one rails app. If numbers are necessary, let's assume I have 1 million users, each with a maximum of 5 sites, with each site bringing in around 1,000 hits a day.
I realize this is kind of a broad question, and mostly depends on my implementation of either method to reach a feasible solution, but any suggestions in terms of the best way to write this, including optimizing the DB, etc. would be appreciated.
It would be exponentially easier to have 1 rails app with millions of subdomains compared to millions of rails apps.
Check out this railscast for how to start with subdomains: http://railscasts.com/episodes/221-subdomains-in-rails-3
I wouldn't ever consider doing something like this with multiple Rails projects, because of the need to maintain all the code. By keeping it centralized, you can change the functionality of everybody's sites at once.
I think you might also run into memory issues by having all of those copies of Rails instantiated, too.
#Solomon is right. Heroku.com is using same concept for it's users to demonstrate users' applications.
Related
I got my app ready, which rely on Ruby on Rails as backend.
Now I am going to publish another app, which has similar functionality, but different topic. like StackExchange, it has multiple sites under this big umbrella, like Stackoverflow, Superuser, Game Development etc.
I have these few approaches in mind:
Same code base, deploy to multiple Rails apps.
Same code base, handle by same Rails app, but with flagging. to identify.
How do Stackoverflow handle this kind of variation?
It will be much easier for you, initially, to have a single app handling the different sites, and to use the url to set some settings for each site, such as which data to load up, which stylesheets etc. In this way you're designing a single rails app, and can test it easily with various localhost urls which trigger the different variants etc.
If your site becomes very successful you will want to split it across multiple servers anyway. At that point, you will need to consider a strategy for doing so: you may decide that you will have one server per site, or it might be the case that one of the sites is so massively successful that you need multiple servers for that and one to handle the others, or something else. That's a nice problem to have, and you might not have it, so keep it simple for now is my advice.
EDIT - for info on how stackexchange works, go to http://blog.serverfault.com/ . Searching for "architecture" could be a good start for you.
I am running Ruby on Rails 3 and I have an application that makes use of namespaces in order to handle more "internal concepts". With "internal concepts" I mean that each namespace is used to handle a specific resource of my application. For example a namespace is "users" and it is used to handle user's sessions and authorizations, another is "blogs" and it is used to handle all about posts and comments.
I think this is a "convenient" solution to avoid a lot of problems, but not the best.
At this time my RoR application consists of this file system structure:
# "users" and "blogs" are namespaces
RAILS_ROOT/app/controllers/users
RAILS_ROOT/app/controllers/blogs
RAILS_ROOT/app/models/users
RAILS_ROOT/app/models/blogs
RAILS_ROOT/app/views/users
RAILS_ROOT/app/views/blogs
...
I would like to switch the "users" and "blogs" namespace in two RoR applications using subdomains to have something like this:
http://main.com # This is the main RoR application
http://users.main.com # This is another RoR application used to handle users
http://blogs.main.com # This is another RoR application used to handle blogs
In few words, I think I am trying to Scale Out* my application or maybe to create a Webservice for each RoR application, but my issues are:
1. What problems I may encounter?
I noticed of problems about maintaining sessions (in my case I handle those with cookies) between applications but I think it isn't the only one problem.
2. How to handle communication between the three RoR applications in my case?
I noticed that I can use ActiveResource to share information, but I must pay attention to information such as user authentication.
I have to implement the OpenID/Oauth protocol in order to maintain user authentications?
I think I have to ensure the user authentication information with a HTTPS connection also if the comunication is between subdomains. Is it true?
3. How do I organize my work and resources?
With all that being said, I would like to don't use (absolutely) plugins or gems, but, if I need, I would like to implement my own handler.
At the end I would like to have 3 RoR "easy" and separated applications without use namespaces in each of them and that can communicate between each other:
# "Main" application for http://main.com
ROOT_MAIN/app/controllers/
ROOT_MAIN/app/models/
ROOT_MAIN/app/views/users
...
# "Users" application for http://users.main.com
ROOT_USERS/app/controllers/
ROOT_USERS/app/models/
ROOT_USERS/app/views/users
...
# "Blogs" application for http://blogs.main.com
ROOT_BLOGS/app/controllers/
ROOT_BLOGS/app/models/
ROOT_BLOGS/app/views/users
...
BTW: is a good approach the usage of namespaces that I'm doing?
P.S.: If you need some other information, let me know and I will update the question.
*From The O2 Software Process: "Scale Out" refers to the concept of adding more servers to an existing park, as opposed to "Scale Up" which means to replace existing (slow) servers with newer (and faster) servers.
You problem is lot more simpler than you think. It all depends on how you handle your routes.
Ruby On Rails 3 has better support for Subdomains. So, you need not separate them into three/more RoR apps. You can put all your code in one single RoR app. And redirect user.abc.com to any controller like "users/sessions", redirect blog.abc.com to "blogs/blogs" controller. namespaces are convenient in apps like yours where they make your job really quick to separate out contextually different parts of your app in different folders and route formats.
Try the namespaces to your hearts content, I believe you won't get any errors you are imagining right now. I'd suggest you write code for it and come here if you face problems in it.
Is your app really so big that you need to use multiple apps to handle the different concerns? It could be that your post just lacks enough detail to convey the real magnitude of what you are doing but it seems like you are trying to modularize a small enough app that it would be fine without "scaling out" as you say. Or maybe I am just missing something?
I think that is going to be a tricky problem but there may be some way to store session data in the database and either share it the way you handle #2 or you'll have to roll a custom solution for that. I think the biggest problem will be sharing resources across your app, and also if you are breaking user management out into its own app you'll need to implement your own OpenID/Oauth. This post describes this with Devise/OAuth.
You can use activeresource to connect to each app's respective rest api. This post describes one guy's solutions to sharing data across rails apps.
This question is somewhat vague. You described using multiple apps to separate your concerns (blogging vs user management), so I imagine you'll have your resources at the root of each application without any namespacing, as you've done already in your existing application.
Now for a more general response to your entire question, recently I read a blog post regarding Data, Context and Interaction (wikipedia article) on Rails, and I think this might be a better solution for what you are trying to accomplish if you feel like your app is getting out of control.
Sorry for answering this late.
Actually if you want to scale your rails application you need not create different apps for each unit(I mean as you are trying to separate out users and blogs here), you are jumping a step ahead in the process of scaling your app you should first put all individual units as mountable engines and require them as gem in your core application and mount them in your core app routes.As in your case blogs can be moved to a separate mountable engine.If in future you require to scale more then you can move futher to use engines as separate application.Here's a link to a video that may give you idea what I am trying to explain here https://www.youtube.com/watch?v=pm94BsoMGik
I am trying to build a CMS I can use to host multiple sites. I know I'm going to end up reinventing the wheel a million times with this project, so I'm thinking about extending an existing open source Ruby on Rails CMS to meet my needs.
One of those needs is to be able to run multiple sites, while using only one code-base. That way, when there's an update I want to make, I can update it in one place, and the change is reflected on all of the sites. I think that this will be able to scale by running multiple instances of the application.
I think that I can use the domain/subdomain to determine which data to display. For example, someone goes to subdomain1.mysite.com and the application looks in the database for the content for subdomain1.
The problem I see is with most pre-built CMS solutions, they are only designed to host one site, including the one I want to use. So the database is structured to work with one site. However, I had the idea that I could overcome this by "creating a new database" for each site, then specifying which database to connect to based on the domain/subdomain as I mentioned above.
I'm thinking of hosting this on Heroku, so I'm wondering what my options for this might be. I'm not very familiar with Amazon S3, or Amazon SimpleDB, but I feel like there's some sort of "cloud database" that would make this solution a lot more realistic, than creating a new MySQL database for each site.
What do you think? Am I thinking about this the wrong way? What advice do you have to offer in this area?
I've worked on a Rails app like this, and the way it was done there was named-based virtual hosts, with db entries for each site running. Each record was scoped to a site if necessary (blog posts, etc.) while users would have access to all sites running out of that db. Administrator permissions could be global or scoped to one or more sites.
You're absolutely correct when you say you'll reinvent the wheel a million times during the project. Plugins will likely require hacking on top of the CMS itself.
In my situation, it ended up being a waste of almost a million dollars of company money to build that codebase to run multiple sites while still being able to cater to the whims of each client site. It worked, but was not very maintainable due to the number of site-specific hacks that subsequently entered the codebase. You may be able to make it work if you don't have to worry about catering to specific client sites running on your platform.
In the end, you're going to need a layer of indirection to handle the different sites regardless of methodology. We ended up putting it in the database itself. If you go with the different-db-for-each-site method you mentioned, you'll put that layer in your code instead. I'm not sure which one is the better method.
I hope you're able to pull this off. I failed.
Also, as I learned today, Heroku offers postgres instead of mysql for rails apps.
There's James Stewart's Theme Support Plugin for Rails 2.3, and lucasefe's themes_for_rails gem for Rails 3+.
I just started using the 2.3 version and it's working well so far.
I was wondering if somebody has some insight on this issue.
A little background:
We've been using Rails to migrate from an old dBase and Visual Basic based system
to build internal company IntrAnet that does things like label printing,
invetory control, shipping, etc - basically an ERP
The Dilemma
Right now we need to replace an old customer-facing website that was done in Java, that
would connect to our internal system for our clients to use. We want to be able to pull information like inventory, order placement, account statements from our internal system and expose it to site live. The reason is that we take orders on the website, through fax & phone and sometimes we have walk-ins. So sometimes (very rarely thou) even a short delay in inventory update on our old Java site causes us to put an order on backorder, because we sell the same item to 2 customers within half an hour. It's usually fixed within one day but we want to avoid this in the future.
Actual Question
Does anyone have any suggestion on how to accomplish this in a better
way?
Here are three options that I see:
a) Build a separate Rails app on a web server, that will connect to the same DB that our internal app connects to.
+++ Pluses:Live data - same thing that our internal apps see, i.e. orders are created in real time, inventory is depleted right away
--- Minuses: Potential security risk, duplication of code - i.e. I need to duplicate all the controllers, models, views, etc. that deal with orders.
b) Build a separate Rails app on a web server, that will connect to a different DB from our internal app.
+++ Pluses: Less security exposure.
--- Minuses:Extra effort to sync web DB and internal DB (or using a web service like REST-API), extra code to handle inventory depletion and order # creation, duplication of code - i.e. I need to duplicate all the controllers, models, views, etc. that deal with orders.
c) Expose internal app to the web
+++ Pluses: all the problems from above eliminated. This is much "DRY"er method.
--- Minuses: A lot more security headaches. More complicated login systems - one for web & one for internal users using LDAP.
So any thoughts? Anyone had similar problem to solve? Please keep in mind that our company has limited resources - namely one developer that is dedicated to this. So this has to be one of those "right" and "smart" solutions, not "throw money/people/resources at this" solutions.
Thank you.
I would probably create separate controllers for the public site and use ActiveResource to pull data from you internal application. Take a look at
http://blog.rubybestpractices.com/posts/gregory/rails_modularity_1.html
http://api.rubyonrails.org/classes/ActiveResource/Base.html
Edit - fixed link and added api link
I would go for a. You should be able to create the controllers so that they are re-usable.
Internal users are as likely to duplicate data as external users.
It's likely that a public UI and an internal, for-the-staff, UI will need to be different. The data needs to be consistent so I would put quite a bit of effort into ensuring that there is exactly one, definitive database. So: one database two UIs?
Have a "service" layer that both UIs can use. If this was Java I would be pretty confident of getting the services done quickly. I wonder how easy it is in Ruby/Rails.
The best outcome would be that your existing Customer Java UI can be adapted to use the Rails service layer.
Assuming you trust your programmers to not accidentally expose things in the wrong place, the 'right' solution seems to me to have a single application, but two different sets of controllers and views, one for internal use, and one for public-facing. This will give you djna's idea of one database, two UIs.
As you say having two separate databases is going to involve a lot of duplication, as well as the problem of replication.
It doesn't make sense to me to have two totally separate apps using the same database; the ActiveRecord part of a Rails app is an abstraction of the database in Ruby code, therefore having two abstractions for a single database seems a bit wrong.
You can also then have common business rules in your models, to avoid code duplication across the two versions of the site.
If you don't completely trust your programmers, then Mike's ActiveResource approach is pretty good - it would make it a lot harder to expose things by accident (although ActiveResource is a lot less flexible and feature rich than ActiveRecord)
What version of Rails are you using? Since version 2.3 Rails Engines is included, this allows to share common code (models/views/controllers) in a Rails plugin.
See the Railscast for a short introduction.
I use it too. I have developed three applications for different clients, but with all the shared code in a plugin.
My question here is seeking best practice, general advice and insight, rather than a solution to a specific problem.
I am in the early stages of planning out a Rails project which I consider fairly large. At its simplest level it offers a cookie-cutter CMS to the target users. So users sign up and choose a subdomain and are given a pretty basic website with CMS.
Therefore the entire app has about 4 different 'sides' to it:
A sales website selling the product to end users - www.myapp.com
A central admin area where staff can log in and manage accounts etc - www.myapp.com/superadmin
The users' own websites - subdomain.myapp.com
The users' admin area/CMS - subdomain.myapp.com/admin
So really what I'm looking for is best practice for structuring the app. i.e. should it all be rolled into one huge application or should it be split over 2 (or more) smaller apps?
If deployed as one application, I can see issues surrounding routing as both the sales website and the users' websites will need a root path set, plus I would not want the routes I set for the sales website being accessible through the users' websites. Can anything be done either within Rails or at Apache level (mod rewrites ?) to ensure no mixup of routes?
If split over 2 or more applications, how do you get the applications sharing the same database? Is that even a good idea? Are there any benefits from splitting the application (like isolating problems in one area of the app, rather than bringing everything down)?
I realise this post raises quite a few different questions, but appreciate any advice and insight you can give me.
I believe the benefits of isolating your concerns into separate apps outweigh the costs. I would probably start off with just 2 apps (one for the main site and superadmin, one for the client sites and admins), accessing the same database, but you could do 4.
The downside is you don't really have isolation since all your apps are tied to one database. You will eventually run into scaling problems with your database, but starting off simple with one database will get you launched. One strategy for scaling later would be to add a slave db that the client site and main site apps use, while the admin apps use the master db. This along with a TON of caching will get you pretty far.
There is nothing wrong with having multiple rails apps access one db, however you will need a way to share common code across your apps. Your models for the most part. I've done this before by tossing all my models in a plugin that I share as a sub-module in git or as an external in svn. Having separate apps will make each app smaller and easier to maintain.
However, where do you keep your migrations? Where do you test your models? I would opt for the superadmin app. Also, you make a change to a model or the schema, and now you have to check 2-4 apps and make sure they still work!
Better isolation, separate db's and inter-app communication through web APIs (SOA) and you don't have to worry about that. SOA I think is the way to go after a certain point, but SOA might be premature when you first start out.
At any rate, having separate apps sets you up for SOA but you don't have to jump beyond a single db to start.
I would bundle this all into the same app because you won't be duplicating the classes (models, plugins, etc.) across all the apps. Also: running 4 apps means that you'll have 4 processes all consuming memory due to the 4 separate Rails stacks they have loaded.
Compiling it into one application eliminates this issue. For the issue between the sales site and the users site having to have different roots that can be solved, as mentioned earlier, by subdomain_fu. Let me expand with some sample code from an application I have:
map.with_options :conditions => {:subdomain => 'logs'} do |admin|
admin.resources :channels do |channel|
channel.resources :logs
end
map.root :channels
map.connect ':id', :controller => "channels", :action => "show"
end
As we see here, the :conditions for the with_options method sets :subdomain to be logs which means that anything coming in to logs.mysite.com will fufill these conditions and therefore be routed this way.
Now further on in this routing file I have everything else wrapped up in a similar block:
map.with_options :conditions => {:subdomain => nil} do |admin|
# shebang!
end
Everything going to mysite.com will go to these routes.
Lastly, compiling it all into one mega-super-hyper-app will eliminate the database-sharing issues.
The biggest issue I see with separating into several apps is that you lose flexibility. What happens if, in the future, a previously administrative task (eg. uploading a type of file) becomes a "user task"? You would have to be moving code from one application to the other.
I'd keep everything on single application - and use roles for filtering what each user can see and do. It might be a bit more difficult at the begining, but it pays up in the near future.
Have a look at authorization frameworks, such as declarative_authorization or cancan.
Well, since nobody else has spoken up, I'd encourage you to do some reading on Service-Oriented Architecture. The book Enterprise Rails by Dan Chak has some great material on this, and you can read a lot of it through Google Books. Try chapter 13, here. I think it'll put you on the right track.
I see the kind of problem you are facing is, trying to build an application which will have various sub domains, so account_manager a plugin can solve your problem.
also if your application is large enough to maintain than splitting them in two or three would be good idea, with restfull resources you can make your applications talk to each other and so.
while if you are thinking of having them under one database, thats quite simple in rails using the establish_connection.
I think you can split the application in three to four different applications where set of clusters will handle each applications request, so the speed will be good. also you can bundle similar kind of functionality in one app to make sure maintaining them is easy.
http://www.railslodge.com/plugins/1113-subdomain-fu
As far as my research has taken me, most companies at high scale would opt for SOA with multiple databases. Here are links to some information on how Linked In and EBay think about this. And to echo PreciousBodilyFluids, I highly recommend the Enterprise Rails book by Dan Chak.