I'm in the early stages of prototyping a Rails 3 application that will expose a public API. The site has three separate concerns which I am planning to split across three subdomains.
api.mysite.com
The publicly exposed API.
admin.mysite.com
The admin portal for creating blogs (using the public API).
x.mysite.com
The public blog site created at admin.mysite.com where x is the name of the blog. This too will make use of the public API.
All three will share domain objects. For example, you should be able to login to admin.mysite.com using an account you created on api.mysite.com or x.mysite.com.
Questions
Should I attempt to build one rails application to handle all three concerns or should I split this in multiple applications each handling a specific concern?
What are the Pros & Cons of each?
Does anyone have any insight into how some of the larger sites (basecamp, github, shopify) are organized?
Your question is fairly general so I'll try and answer in general terms. And the fact that you mention "larger sites" leads me to the conclusion that you're concerned about scaling.
In the beginning it is definitely going to be easier to build one application - especially since the domain is shared. You can do separate controllers for the various interfaces (api, html, etc) but with a shared back-end. This will reduce code duplication and the complexity of keeping 3 apps in sync. Also remember that you might change your mind about features based on user feedback and you want to be nimble enough to respond quickly.
The main benefit I can see of separating out three different deployables is that you can have an independent deploy schedule for each. For example, a bug fix in the api won't have to wait for admin to be ready to deploy. Or that you can have separate teams working in parallel.
If you're careful about what you keep in your session you'll be able to deploy multiple instances of your application on multiple servers, pointing at the same database (a.k.a. horizontal scaling). Each of these instances are identical to the others and a load balancer (either dedicated hardware or virtual) directs traffic between them. Eventually this approach runs out of steam when your database can't handle the load. At that point you can look at more caching, sharding, no-sql and all sorts of clever scaling techniques.
Most (but not all) larger sites end up doing some sort of horizontal scaling with some sharding of data.
All told, focus on getting a useful application to your users. If things take off you can worry about scaling. More applications fail because the user experience is awful rather than not being able to scale.
Related
I have a few questions on appropriate folder structure in cucumber:
I think I am going to organize my feature folders according to type_of_user/type_of_feature.feature, i.e. main_admin/add_a_customer.feature or franchisee/schedule_job.feature. The only slight issue with this is that of the user types I have: cleaners, customers, franchisees and main admin/franchisor, the latter two users share many features. For example, both franchisees and franchisor have the ability to add new customers and schedule jobs, the only difference being that the franchisor has the ability to schedule a job for anyone, anywhere - i.e. the only real difference is permissions, not functionality. Does it matter that I will be essentially duplicating tests for these two users, given the proposed folder structure? Or should I be looking to seperate folders by functionality only, then type of user?
For my mobile app, should I have these feature folders separate from the web app or should these go in the root as well: mobile/ios/cleaner_login.feature, mobile/android/cleaner_login.feature etc?
Regarding user types:
Organizing at the top level by user type has worked well for me. However, I would only consider user types separate if they actually used different features, not just if they differed in permissions with respect to specific objects as in the example you gave. You could consider both franchisees and franchisors "administrators", make a top-level folder for those, and just write scenarios for franchisees and franchisors for features that had different permissions for those roles.
If you're a developer and writing RSpec specs in addition to Cucumber features, you might even just write specs instead of features to cover the difference between franchisees and franchisors. (I would only do that if the differences between franchisee and franchisor were fairly trivial and not worth exposing in Cucumber.) If you're QA and testing only from the outside of course it'll all have to be in Cucumber.
I would certainly not systematically duplicate entire scenarios for the sake of any organization. The extra work required to maintain the duplication and the errors when you forgot would be far worse than the bit of extra work required to follow a slightly more complicated system that minimizes duplication!
Regarding web and mobile: How to handle different platforms depends on how different they are.
If you have a web app and a native (Android, iOS) mobile app the step implementations will be completely different and your tests will need to be in different projects altogether. That probably won't mean that much duplication, since the users and features in the web and mobile apps will probably be rather different.
If you have two web apps, one for desktop and one for mobile, there are no technology issues. But again it will depend on how similar the two apps are. If they're different, separate them at the top level (even before users). If they're very similar, separate them only when necessary and only at the scenario level.
Each Area will have its own config etc. So as the areas increases, the complexity and maintainability increases as well. Will it be good choice to modularise or partition and MVC application functionality in to Areas or continue with traditional Controller/View approach?.
Please suggest a common solution or better way to architect a large scale MVC application.
Areas shouldn't be confusing, and certainly aren't redundant. As you say, they allow you to partition your web app into smaller functional groupings. This is extremely helpful when the size of your applications grow and a single application umbrella becomes unwieldy.
As an example, I have just completed a large application that stored promotional data for various retailers across North America. The US and Canada sales teams are isolated, but are executing their tasks in nearly the same business contexts.
It made a lot of sense to partition the US and Canada parts of the web app into Areas, which organized things a lot better for us. Each area could still use the same components where they make sense (repositories, services, etc...), but the isolation Areas brought allowed us to build separate controllers and views specific to each business group, instead of trying to run a bunch of logic checks to accommodate whatever region the user was in.
Here is possible alternative to your approach from "Programming ASP.NET MVC 4" by Jess Chadwick, Todd Snyder, and Hrusikesh Panda:
There are many different approaches to take when designing an ASP.NET
MVC application. A developer can decide to keep all the application’s
components inside the website assembly, or separate components into
different assemblies. In most cases it’s a good idea to separate the
business and data access layers into different assemblies than the
website. This is typically done to better isolate the business model
from the UI and make it easier to write automated test that focuses on
the core application logic. In addition, using this approach makes it
possible to reuse the business and data access layers from other
applications (console applications, websites, web services, etc.).
A common root namespace (e.g., company.{ApplicationName}) should be
consistently used across all assemblies. Each assembly should have a
unique sub-namespace that matches the name of the assembly. Figure 5-4
shows the project structure of the Ebuy reference application. The
functionality for the application has been divided into three
projects: Ebuy.WebSite contains the view, controllers, and other
web-related files; Ebuy.Core contains the business model for the
application; and the CustomExtentions project contains the custom
extensions used by the application for model binding, routing, and
controllers. In addition, the project has two testing projects (not
shown): UnitTests and IntegrationTests.
There is no rule on whether to use Areas or not, it's basically up to solution architect to do an estimate should using areas provide benefit or not.
Our latest project that involved areas, included 3 different types of users working on the website. We used controller naming scheme where controller name matched the resource name (i.e. CategoryController).
However, certain resources could have been accessed by all 3 user groups in completely different manner: one user group could only list resource, other user group could manage them (basic crud) while the 3rd user group (admin-like) could do advanced features such as exporting, importing, etc...
By separating the functionalities in areas, we've reduced security problems by simply decorating the controller in area to request user type specific for that area, in order to avoid mixing of permissions. Doing it on the base controller for the area, made things even more centralized.
That is one reason why we would pick separation of areas.
On the other hand, we've often been in situation where we have request for a high-demanding public website and "back-office" internal configuration website. For the performance and scalability + concurrency issues, we've quite often designed public website that could be load balanced as a one project, and back-office website that would be only hosted once as a second project - instead of using areas.
Again, this is just one approach from the industry, not necessarily the optimal approach.
I am the primary developer on a Rails application that allows customers to manage artist portfolio websites. There are four very different types of behavior that the system entails:
Admin behavior, which allows logged-in users to manage the content of
their portfolios
Portfolio Display behavior, which renders users'
portfolios to general visitors
Conversion Funnel behavior, which
displays information about the service and entices new users to sign
up
Super Admin behavior, which displays statistics about the other
three behaviors to the service owners
Currently, these four sets of behaviors are broken up into namespaced controllers—but they all share the same models. I am wondering, since the four parts of the system share virtually no behavior, if there would be any benefit to splitting the application up into four separate applications or engines (and extracting any shared behavior into gems.)
As examples, the part of the system that deals with user statistics doesn't need to "know about" rendering YouTube embeds in portfolios, the part of the system that deals with displaying portfolios doesn't need to "know about" A/B testing, and the part of the system that deals with signing up new users doesn't need to "know about" much else besides signing up new users.
Additionally, a specific problem that I'd like to address is that I'd like for an inexperienced team member to be able to contribute a little bit of code to the part of the site that deals with signing up new users. It wouldn't be the end of the world if a bug or two got pushed to production in that part of the site but it's extremely important to not allows bugs in production in the part of the system that displays users' portfolios.
So, in terms of the maintainability and legibility of the codebase, would it make sense to separate these four components into separate applications? To what extent would doing so simply entail pushing complexity into a different layer of the system without eliminating it? Is this type of separation better accomplished by creating more cleanly decoupled classes and modules?
Thanks much!
A semi-isolated engine would achieve your goals of allowing for silo code work and you can easily shares all of the resources, such as stylesheets (which is likely important since you still offer one service).
A mountable engine also achieves your goals, but the sharing is more difficult, since it seems best to serve as isolated.
A service-oriented architecture with a number of functionally-oriented apps delivering APIs to one or more front-end apps may work. It'd be more work upfront, but it could payoff in the long run if you see the total service getting more complex over time.
Fun choices!
So, my company is talking about building an ecommerce platform that will serve many different clients. Each client would have a different look and feel, and its own set of users, but the backing code (ie: adminstration services, authentication servers, checkout services, possibly admin pages, etc.), and some users would be shared, so a bug fix could be applied to all sites at the same time, a primary admin could log into all websites.
As the entire StackExchange set of websites (with pretty high traffic) run off a small number of servers (two I believe) I wonder what it would involve to serve up many unrelated (but similar) websites through one webapp, or even one database.
In order to have one database, I imagine that every table would have columns identifying which realm the entity belonged to, and every SQL call would filter by that column. That seems like it would become an maintenance nightmare, and (less importantly for me) a DBA's hell.
Another option, with one webapp, but multiple databases, I imagine the realm could be tied to a specific data source, where all non-shared data could be specified. Then when any request was made, the appropriate datasource could be loaded and the webapp would run as if there was only single source. This would have an added benefit of being easily horizontally scalable since the exact same webapp, but a different set of realms and datasources, could be spawned when necessary. Websites could be easily moved to new servers as well by simply copying the webapp and moving the database.
I wondering what other possibilities there are, as well as, specific examples if they're out there.
Note: I'm not talking about Twitter-scale scalability, nor about hardware/languages/etc, but rather design methodologies and patterns.
The architecture you are talking about is called "multi-tenant" architecture. There are diff. ways of going about architecting a multi-tenant application. Broadly speaking, data tier can be architected in 3 ways:
seperate database for each client - easier to code, diff. to maintain
one database, different schema
one database, one schema (with clientid in each table; except for meta-data tables) - more time in coding, easier to maintain
Each has its own advantages/ disadvantages. Take a look at this article by Microsoft on multi-tenancy. http://msdn.microsoft.com/en-us/library/aa479086.aspx
Broadly speaking, I would suggest option 3 as it offers true multi-tenancy. If you have some tables that you expect to become very large, you can partition that table based on the clientid (for eg: if you want 10 partitons, you could do partition based on mod of clientid)
We have a very lage Rails app that has two distinct sections: the front end and the CMS/Admin. We would like to break up the app into two pieces (for maintenance, as we have distinct teams that work on the front end vs. back end and they could have different release cycles).
One thought was to start a new Admin 2.0 app that has access to the models/schema from the original application, but has its own controllers/views and its own models that extend the original models until it is safe to fully decouple. Is this advisable? If not, what would be an appropriate plan to migrate away from one monolithic codebase?
warning, this is a bit ranty, and does not go anywhere.
Having worked on a very large app that operates in the manor you describe (for scalability reasons), I still have mixed opinions (an no conclusive answers).
Currently we operate 3 major apps (+ one or two smaller ones that use a fragment of the schema).
RVW (our admin app): This is the only app that writes, runs on a single server, and is responsible for maintaining the schema.
reevoo.com: ecommerce, price comparison, stuff like that. This (for historic reasons runs on a slightly different schema, run on a read only slave of RVW, with database views to map the schemas. All writes are done by sticking things on queues that RVW picks up and acts on. This works very well, although the number of random db related issues (mostly related to the views) is an issue. The main problem with this app is the difficulty sharing code (gems work well, I've often dreamed of bringing the schemas into line and sharing the core models in a gem!). We share code between apps using ruby gems. And test using lots of integration tests that cross app boundaries (using drunit (presentation on this available)).
reevoomark: very high load b2b app. This has many servers each with a full stack (db server, app server one per node). These have their databases populated with a db export - import batch job. This works very well in the short term, the shear flexibility of it is just ace, but integration testing between apps is very hard.
My advice would be to avoid splitting the apps at all costs, keeping things DRY quickly becomes a major challenge. My advice would be to stick with one app, two sets of routes (selected at startup by environment variables).
This gives you all the advantages of the other solutions, while making code sharing implicit. Splitting your test packs out would make your test cycles shorter and make things more manageable for the two teams. I would avoid working on different code bases, as doing this promotes the apps drifting apart and making code sharing tricky (as in .com).
If you decide do split, have a good set of high level cross app tests. Custom (per app) extensions to a core set of models sounds like a good plan, although with distinct code bases and teams you may still end up with duplicate code. Rails engines should be a good way of sharing the models, but be prepared for model reloading to become a little schizophrenic.
Good luck!
Have you namespaced your admin controllers? That would be a relatively easy point of subdivision and also avoid many of the negative side effects of forking your code into two apps.
Have you considered looking at Rails Engines? Added to Rails in 2.3.