Scaling Out: how to handle communication between Ruby on Rails applications? - ruby-on-rails

I am running Ruby on Rails 3 and I have an application that makes use of namespaces in order to handle more "internal concepts". With "internal concepts" I mean that each namespace is used to handle a specific resource of my application. For example a namespace is "users" and it is used to handle user's sessions and authorizations, another is "blogs" and it is used to handle all about posts and comments.
I think this is a "convenient" solution to avoid a lot of problems, but not the best.
At this time my RoR application consists of this file system structure:
# "users" and "blogs" are namespaces
RAILS_ROOT/app/controllers/users
RAILS_ROOT/app/controllers/blogs
RAILS_ROOT/app/models/users
RAILS_ROOT/app/models/blogs
RAILS_ROOT/app/views/users
RAILS_ROOT/app/views/blogs
...
I would like to switch the "users" and "blogs" namespace in two RoR applications using subdomains to have something like this:
http://main.com # This is the main RoR application
http://users.main.com # This is another RoR application used to handle users
http://blogs.main.com # This is another RoR application used to handle blogs
In few words, I think I am trying to Scale Out* my application or maybe to create a Webservice for each RoR application, but my issues are:
1. What problems I may encounter?
I noticed of problems about maintaining sessions (in my case I handle those with cookies) between applications but I think it isn't the only one problem.
2. How to handle communication between the three RoR applications in my case?
I noticed that I can use ActiveResource to share information, but I must pay attention to information such as user authentication.
I have to implement the OpenID/Oauth protocol in order to maintain user authentications?
I think I have to ensure the user authentication information with a HTTPS connection also if the comunication is between subdomains. Is it true?
3. How do I organize my work and resources?
With all that being said, I would like to don't use (absolutely) plugins or gems, but, if I need, I would like to implement my own handler.
At the end I would like to have 3 RoR "easy" and separated applications without use namespaces in each of them and that can communicate between each other:
# "Main" application for http://main.com
ROOT_MAIN/app/controllers/
ROOT_MAIN/app/models/
ROOT_MAIN/app/views/users
...
# "Users" application for http://users.main.com
ROOT_USERS/app/controllers/
ROOT_USERS/app/models/
ROOT_USERS/app/views/users
...
# "Blogs" application for http://blogs.main.com
ROOT_BLOGS/app/controllers/
ROOT_BLOGS/app/models/
ROOT_BLOGS/app/views/users
...
BTW: is a good approach the usage of namespaces that I'm doing?
P.S.: If you need some other information, let me know and I will update the question.
*From The O2 Software Process: "Scale Out" refers to the concept of adding more servers to an existing park, as opposed to "Scale Up" which means to replace existing (slow) servers with newer (and faster) servers.

You problem is lot more simpler than you think. It all depends on how you handle your routes.
Ruby On Rails 3 has better support for Subdomains. So, you need not separate them into three/more RoR apps. You can put all your code in one single RoR app. And redirect user.abc.com to any controller like "users/sessions", redirect blog.abc.com to "blogs/blogs" controller. namespaces are convenient in apps like yours where they make your job really quick to separate out contextually different parts of your app in different folders and route formats.
Try the namespaces to your hearts content, I believe you won't get any errors you are imagining right now. I'd suggest you write code for it and come here if you face problems in it.

Is your app really so big that you need to use multiple apps to handle the different concerns? It could be that your post just lacks enough detail to convey the real magnitude of what you are doing but it seems like you are trying to modularize a small enough app that it would be fine without "scaling out" as you say. Or maybe I am just missing something?
I think that is going to be a tricky problem but there may be some way to store session data in the database and either share it the way you handle #2 or you'll have to roll a custom solution for that. I think the biggest problem will be sharing resources across your app, and also if you are breaking user management out into its own app you'll need to implement your own OpenID/Oauth. This post describes this with Devise/OAuth.
You can use activeresource to connect to each app's respective rest api. This post describes one guy's solutions to sharing data across rails apps.
This question is somewhat vague. You described using multiple apps to separate your concerns (blogging vs user management), so I imagine you'll have your resources at the root of each application without any namespacing, as you've done already in your existing application.
Now for a more general response to your entire question, recently I read a blog post regarding Data, Context and Interaction (wikipedia article) on Rails, and I think this might be a better solution for what you are trying to accomplish if you feel like your app is getting out of control.

Sorry for answering this late.
Actually if you want to scale your rails application you need not create different apps for each unit(I mean as you are trying to separate out users and blogs here), you are jumping a step ahead in the process of scaling your app you should first put all individual units as mountable engines and require them as gem in your core application and mount them in your core app routes.As in your case blogs can be moved to a separate mountable engine.If in future you require to scale more then you can move futher to use engines as separate application.Here's a link to a video that may give you idea what I am trying to explain here https://www.youtube.com/watch?v=pm94BsoMGik

Related

What is the conventional architecture for a Rails 5 API with an Administrative UI as well?

I was setting out to start a new Rails 5 API, and realized I also need a content-administration "site" of some sort. The admin tool is very simple, just a UI for very basic CRUD operations.
I have an instinct to create two separate Rails applications - one web application for the content-admin tool, and another web application for the API.
This brings about the problem with sharing data models, which is solvable by using rails engines, or including the models as a gem.
As I was researching solutions, I seemed to observe a pattern of including the content-admin portion within the API app itself. There are some middleware includes and controller inheritances involved in this, but its quite simple to get a content-admin UI to run within the same app as an API. Its much less work, and I dont see much of a problem with scale, since the content-admin UI is lightly utilized and the API is the core of the business.
Is this the accepted convention? I might be gaining a bias due to web search results, but it seems like the simplest and most common approach. I plan to have a separate server for accessing the content-admin vs accessing the API, which is what led me to originally plan this as two separate apps. Now I am thinking I was just getting sucked into the "microservices" hype, and it seems more conventional to just include the content-admin UI with my API app.
On the other side, everything I read about Rails engines is 3-4 years out of date. There is little information (that I am stumbling upon) within the last year or so, and more specifically, little-to-no information concerning Rails 5. I am wondering if this sort of architecture has fallen by the wayside.
Is there a typical convention for Rails 5 API applications that also need a content-admin UI?
The approach I've used before is to have the api running out of /app/controllers/api and then have the ActiveAdmin gem installed, with the admin interface files in /app/admin. You can set up the routes to serve the admin interface at https://api.yourapp.com/admin and the API at https://api.yourapp.com/api/v1/ or similar.
I don't know how much of an accepted convention this is, but it works fine.

Rails application architecture and common setup

So I am starting out on company project that will have several components:
At first...
Job list
Client profile creation and management
User administration and access (login, signup, roles, etc)
later...
Messaging
Schedule
Basic reporting
way later...
Deeper analysis and bi
I'm wondering if it makes sense for each bullet item to be its own rails project, self contained and modular (if that is indeed the case); or if it's just best for it to be in the same app. I could envision a situation where each module could operate so independently of each other that it wouldn't need the rest (except for the user funcionality) and another situation where all modules would be used together.
It seems like to me that many tasks can be handled with a lighter framework like Sinatra (and then situated physically under the rails app). It also seems like it would be a lot of overhead to have several rails apps running on a server. But I am not totally aware of all the pluses and minuses to operating each scenario.
I know this is kind of a general question that is bound to get a lot of "it depends" kind of responses (and rightfully so) I was looking for opinions/examples of how you setup this kind/your kind of project in rails. I am a quasi noob so be gentle.
Thanks in advance!
Generally speaking I would consider a website to be a suitable target for a Rails app. Each part of the app can have its own namespaces within the app, so the app has some structure internally, but they should all be one application. Consider things like sessions, where you want a user to login and use whatever features of the site you want. You want those sessions in one application without a user having to login to different sections.
Saying that, if there is complex or extended functionality that isn't part of the MVC architecture (say talking to an external API, data-mining etc), then you could offset that to a separate project and a include it as a Gem in your application. You would still have one main Rails application that includes those Gems.
You might also want to bundle together a section of your project into a reusable Rails engine that can be loaded into multiple projects. For example, Devise handles user login and management. It is a Rails engine, bundled as a Gem, that you include in your project.
Another example from Meducation (one of my sites). I'm in the process of extracting our email tracking system out into its own Rails engine as I feel its functionality sits alongside Meducation and is not a core part of it. I can then use it in other projects as well.
In your specific example, I think your requirements fit fine in one Rails application.

Distributed Resource in Rails

I am planning a system that requires collaboration between many local Rails apps. The design calls for a global app to relay RESTful requests between these servers.
For example, imagine each school having a local Rails app, including a Teacher resource. I propose a global app which provides access to teachers as: /school/42/teacher/3
The School resource on the global server has a field for the base URL of the app at each school, so it can relay such a request to school_42_url/teacher/3.
The design calls for relaying, rather than having school servers connecting directly to each other.
I can think of several ways to achieve this but, being new to Rails, can't work out which one is 'the Rails way'.
ActiveResource is appealing, but seems to require a fixed 'site' rather than setting it for each request.
Routing, perhaps with URL globbing might work, but I need to see an example of how to achieve this.
A third approach would be a custom Controller, but this doesn't feel like the Rails way.
To be clear, the combination of {school_id, teacher_id} is globally unique, but the global server need not store details of teachers, those are treated as a web resource.
Comments or suggestions welcome.
You should use rails RESTful architecture on your client apps to expose apis (xml, json), and some library like Net::HTTP to call those apis from your server(main) app.
It sounds like Pow may be what your looking for. It allows you to configure URL's for stack and rails application via symbolic links. It fits well in the rails model and may allow you to build your independent RESTful applications in the manor specified. I know their are methods for having independent rails servers work together but based this is a very low overhead approach and may be the way to go at-least for testing and development purposes.

Scalability of multi-site rails app

I am beginning work on a new Rails project that is based on the premise of allowing users to create their own "sites." Each "site" would be a subdomain of the root domain (we'll use example.com). So if user Foo wants to create his own site at bar.example.com, each page request to a bar.example.com page would require fetching the a row in a sites table based on the subdomain.
My question is not how to code a multisite app, I think I have a pretty good grasp on that. My question is, from a scalability and performance perspective, would it be better to simply generate a new rails project for each site a user creates? Or is it ok run all sites out of one rails app. If numbers are necessary, let's assume I have 1 million users, each with a maximum of 5 sites, with each site bringing in around 1,000 hits a day.
I realize this is kind of a broad question, and mostly depends on my implementation of either method to reach a feasible solution, but any suggestions in terms of the best way to write this, including optimizing the DB, etc. would be appreciated.
It would be exponentially easier to have 1 rails app with millions of subdomains compared to millions of rails apps.
Check out this railscast for how to start with subdomains: http://railscasts.com/episodes/221-subdomains-in-rails-3
I wouldn't ever consider doing something like this with multiple Rails projects, because of the need to maintain all the code. By keeping it centralized, you can change the functionality of everybody's sites at once.
I think you might also run into memory issues by having all of those copies of Rails instantiated, too.
#Solomon is right. Heroku.com is using same concept for it's users to demonstrate users' applications.

Best practice for structuring a 'large' Rails app

My question here is seeking best practice, general advice and insight, rather than a solution to a specific problem.
I am in the early stages of planning out a Rails project which I consider fairly large. At its simplest level it offers a cookie-cutter CMS to the target users. So users sign up and choose a subdomain and are given a pretty basic website with CMS.
Therefore the entire app has about 4 different 'sides' to it:
A sales website selling the product to end users - www.myapp.com
A central admin area where staff can log in and manage accounts etc - www.myapp.com/superadmin
The users' own websites - subdomain.myapp.com
The users' admin area/CMS - subdomain.myapp.com/admin
So really what I'm looking for is best practice for structuring the app. i.e. should it all be rolled into one huge application or should it be split over 2 (or more) smaller apps?
If deployed as one application, I can see issues surrounding routing as both the sales website and the users' websites will need a root path set, plus I would not want the routes I set for the sales website being accessible through the users' websites. Can anything be done either within Rails or at Apache level (mod rewrites ?) to ensure no mixup of routes?
If split over 2 or more applications, how do you get the applications sharing the same database? Is that even a good idea? Are there any benefits from splitting the application (like isolating problems in one area of the app, rather than bringing everything down)?
I realise this post raises quite a few different questions, but appreciate any advice and insight you can give me.
I believe the benefits of isolating your concerns into separate apps outweigh the costs. I would probably start off with just 2 apps (one for the main site and superadmin, one for the client sites and admins), accessing the same database, but you could do 4.
The downside is you don't really have isolation since all your apps are tied to one database. You will eventually run into scaling problems with your database, but starting off simple with one database will get you launched. One strategy for scaling later would be to add a slave db that the client site and main site apps use, while the admin apps use the master db. This along with a TON of caching will get you pretty far.
There is nothing wrong with having multiple rails apps access one db, however you will need a way to share common code across your apps. Your models for the most part. I've done this before by tossing all my models in a plugin that I share as a sub-module in git or as an external in svn. Having separate apps will make each app smaller and easier to maintain.
However, where do you keep your migrations? Where do you test your models? I would opt for the superadmin app. Also, you make a change to a model or the schema, and now you have to check 2-4 apps and make sure they still work!
Better isolation, separate db's and inter-app communication through web APIs (SOA) and you don't have to worry about that. SOA I think is the way to go after a certain point, but SOA might be premature when you first start out.
At any rate, having separate apps sets you up for SOA but you don't have to jump beyond a single db to start.
I would bundle this all into the same app because you won't be duplicating the classes (models, plugins, etc.) across all the apps. Also: running 4 apps means that you'll have 4 processes all consuming memory due to the 4 separate Rails stacks they have loaded.
Compiling it into one application eliminates this issue. For the issue between the sales site and the users site having to have different roots that can be solved, as mentioned earlier, by subdomain_fu. Let me expand with some sample code from an application I have:
map.with_options :conditions => {:subdomain => 'logs'} do |admin|
admin.resources :channels do |channel|
channel.resources :logs
end
map.root :channels
map.connect ':id', :controller => "channels", :action => "show"
end
As we see here, the :conditions for the with_options method sets :subdomain to be logs which means that anything coming in to logs.mysite.com will fufill these conditions and therefore be routed this way.
Now further on in this routing file I have everything else wrapped up in a similar block:
map.with_options :conditions => {:subdomain => nil} do |admin|
# shebang!
end
Everything going to mysite.com will go to these routes.
Lastly, compiling it all into one mega-super-hyper-app will eliminate the database-sharing issues.
The biggest issue I see with separating into several apps is that you lose flexibility. What happens if, in the future, a previously administrative task (eg. uploading a type of file) becomes a "user task"? You would have to be moving code from one application to the other.
I'd keep everything on single application - and use roles for filtering what each user can see and do. It might be a bit more difficult at the begining, but it pays up in the near future.
Have a look at authorization frameworks, such as declarative_authorization or cancan.
Well, since nobody else has spoken up, I'd encourage you to do some reading on Service-Oriented Architecture. The book Enterprise Rails by Dan Chak has some great material on this, and you can read a lot of it through Google Books. Try chapter 13, here. I think it'll put you on the right track.
I see the kind of problem you are facing is, trying to build an application which will have various sub domains, so account_manager a plugin can solve your problem.
also if your application is large enough to maintain than splitting them in two or three would be good idea, with restfull resources you can make your applications talk to each other and so.
while if you are thinking of having them under one database, thats quite simple in rails using the establish_connection.
I think you can split the application in three to four different applications where set of clusters will handle each applications request, so the speed will be good. also you can bundle similar kind of functionality in one app to make sure maintaining them is easy.
http://www.railslodge.com/plugins/1113-subdomain-fu
As far as my research has taken me, most companies at high scale would opt for SOA with multiple databases. Here are links to some information on how Linked In and EBay think about this. And to echo PreciousBodilyFluids, I highly recommend the Enterprise Rails book by Dan Chak.

Resources