We have a very lage Rails app that has two distinct sections: the front end and the CMS/Admin. We would like to break up the app into two pieces (for maintenance, as we have distinct teams that work on the front end vs. back end and they could have different release cycles).
One thought was to start a new Admin 2.0 app that has access to the models/schema from the original application, but has its own controllers/views and its own models that extend the original models until it is safe to fully decouple. Is this advisable? If not, what would be an appropriate plan to migrate away from one monolithic codebase?
warning, this is a bit ranty, and does not go anywhere.
Having worked on a very large app that operates in the manor you describe (for scalability reasons), I still have mixed opinions (an no conclusive answers).
Currently we operate 3 major apps (+ one or two smaller ones that use a fragment of the schema).
RVW (our admin app): This is the only app that writes, runs on a single server, and is responsible for maintaining the schema.
reevoo.com: ecommerce, price comparison, stuff like that. This (for historic reasons runs on a slightly different schema, run on a read only slave of RVW, with database views to map the schemas. All writes are done by sticking things on queues that RVW picks up and acts on. This works very well, although the number of random db related issues (mostly related to the views) is an issue. The main problem with this app is the difficulty sharing code (gems work well, I've often dreamed of bringing the schemas into line and sharing the core models in a gem!). We share code between apps using ruby gems. And test using lots of integration tests that cross app boundaries (using drunit (presentation on this available)).
reevoomark: very high load b2b app. This has many servers each with a full stack (db server, app server one per node). These have their databases populated with a db export - import batch job. This works very well in the short term, the shear flexibility of it is just ace, but integration testing between apps is very hard.
My advice would be to avoid splitting the apps at all costs, keeping things DRY quickly becomes a major challenge. My advice would be to stick with one app, two sets of routes (selected at startup by environment variables).
This gives you all the advantages of the other solutions, while making code sharing implicit. Splitting your test packs out would make your test cycles shorter and make things more manageable for the two teams. I would avoid working on different code bases, as doing this promotes the apps drifting apart and making code sharing tricky (as in .com).
If you decide do split, have a good set of high level cross app tests. Custom (per app) extensions to a core set of models sounds like a good plan, although with distinct code bases and teams you may still end up with duplicate code. Rails engines should be a good way of sharing the models, but be prepared for model reloading to become a little schizophrenic.
Good luck!
Have you namespaced your admin controllers? That would be a relatively easy point of subdivision and also avoid many of the negative side effects of forking your code into two apps.
Have you considered looking at Rails Engines? Added to Rails in 2.3.
Related
I am creating a management system and I want to know how "Ruby on Rails" can support me in the mission of ensuring that each customer has their information, records and tables independent from other customers.
Is it better to put everything in a database and put a customer identifier to pull information through this parameter in queries or create a database for each customer automatically?
I admit that the second option attracts me more ... And I know that putting everything in one database will be detrimental to performance, because I assume that customers and their data will increase exponentially!
I want to know which option is more viable in the long run. And if the best option is to create separate databases, how can I do this with Ruby on Rails ??
There are pro and cons for both solutions which really depend on your use case.
Separating each customer in its own database has definitely advantages for scaling, running in different data centres or even onsite. However, this comes with higher complexity. For instance you can't query across customers anymore, you would need to run queries for each customers and aggregate the results. This approach is called multi tenancy (or shardening). There is a good gem called Apartment available (https://github.com/influitive/apartment).
Keeping everything in one database might be simpler to start of as it's less complex but it really depends on your use case.
Edit
Adding some more information based on the questions.
There are several reasons to use a one db per client architecture.
You have clearly separated tenants. In case it might make sense to go with the one db approach.
Scale. Having separated databases for each tenant makes scaling of course easier.
If 2) is the main reason you want to go for a one db per client approach I would strongly advise you against it. You add so much more complexity to your app which you might not need for years to come (if ever).
If scaling is your main concern I recommend reading Designing Data Intensive Applications by Martin Kleppmann. But basically, don't worry about scale for the first few years and focus on your product.
I have a few questions on appropriate folder structure in cucumber:
I think I am going to organize my feature folders according to type_of_user/type_of_feature.feature, i.e. main_admin/add_a_customer.feature or franchisee/schedule_job.feature. The only slight issue with this is that of the user types I have: cleaners, customers, franchisees and main admin/franchisor, the latter two users share many features. For example, both franchisees and franchisor have the ability to add new customers and schedule jobs, the only difference being that the franchisor has the ability to schedule a job for anyone, anywhere - i.e. the only real difference is permissions, not functionality. Does it matter that I will be essentially duplicating tests for these two users, given the proposed folder structure? Or should I be looking to seperate folders by functionality only, then type of user?
For my mobile app, should I have these feature folders separate from the web app or should these go in the root as well: mobile/ios/cleaner_login.feature, mobile/android/cleaner_login.feature etc?
Regarding user types:
Organizing at the top level by user type has worked well for me. However, I would only consider user types separate if they actually used different features, not just if they differed in permissions with respect to specific objects as in the example you gave. You could consider both franchisees and franchisors "administrators", make a top-level folder for those, and just write scenarios for franchisees and franchisors for features that had different permissions for those roles.
If you're a developer and writing RSpec specs in addition to Cucumber features, you might even just write specs instead of features to cover the difference between franchisees and franchisors. (I would only do that if the differences between franchisee and franchisor were fairly trivial and not worth exposing in Cucumber.) If you're QA and testing only from the outside of course it'll all have to be in Cucumber.
I would certainly not systematically duplicate entire scenarios for the sake of any organization. The extra work required to maintain the duplication and the errors when you forgot would be far worse than the bit of extra work required to follow a slightly more complicated system that minimizes duplication!
Regarding web and mobile: How to handle different platforms depends on how different they are.
If you have a web app and a native (Android, iOS) mobile app the step implementations will be completely different and your tests will need to be in different projects altogether. That probably won't mean that much duplication, since the users and features in the web and mobile apps will probably be rather different.
If you have two web apps, one for desktop and one for mobile, there are no technology issues. But again it will depend on how similar the two apps are. If they're different, separate them at the top level (even before users). If they're very similar, separate them only when necessary and only at the scenario level.
I am working on an administrative web app in Rails. Because of various implementation details that are not really relevant, the database backing this app will have all of the content needed to back another separate website. It seems like there are two obvious options:
Build a web app that somehow reads from the same database in a read-only fashion.
Add a RESTful API to the original app and build the second site in such a way as for it to take its content from the API.
My question is this: are either of these options feasible? If so, which of them seems like the better option? Do Rails, Sinatra, or any of the other Rack-based web frameworks lend themselves particularly well to this sort of project? (I am leaning towards Sinatra because it seems more lightweight than Rails and I think that my Rails experience will carry-over to it nicely.)
Thanks!
Both of those are workable and I have employed both in the past, but I'd go with the API approach.
Quick disclaimer: one thing that's not clear is how different these apps are in function. For example, I can imagine the old one being a CRUD app that works on individual records and the new one being a reporting app that does big complicated aggregation queries. That makes the shared DB (maybe) more attractive because the overlap in how you access the data is so small. I'm assuming below that's not the case.
Anyway, the API approach. First, the bad:
One more dependency (the old app). When it breaks, it takes down both apps.
One more hop to get data, so higher latency.
Working with existing code is less fun than writing new code. Just is.
But on the other hand, the good:
Much more resilient to schema changes. Your "old" app's API can have tests, and you can muck with the database to your heart's content (in the context of the old app) and just keep your API to its spec. Your new app won't know the difference, which is good. Abstraction FTW. This the opposite side of the "one more dependency" coin.
Same point, but from different angle: in the we-share-the-database approach, your schema + all of SQL is effectively your API, and it has two clients, the old app and the new. Unless your two apps are doing very different things with the same data, there's no way that's the best API. It's too poorly defined.
The DB admin/instrumentation is better. Let's say you mess up some query and hose your database. Which app was it? Where are these queries coming from? Basically, the fewer things that can interact with your DB, the better. Related: optimize your read queries in one place, not two.
If you used RESTful routes in your existing app for the non-API actions, I'm guessing your API needs will have a huge overlap with your existing controller code. It may be a matter of just converting your data to JSON instead of passing it to a view. Rails makes it very easy to use an action to respond to both API and user-driven requests. So that's a big DRY win if it's applicable.
What happens if you find out you do want some writability in your new app? Or at least access to some field your old app doesn't care about (maybe you added it with a script)? In the shared DB approach, it's just gross. With the other, it's just a matter of extending the API a bit.
Basically, the only way I'd go for the shared DB approach is that I hated the old code and wanted to start fresh. That's understandable (and I've done exactly that), but it's not the architecturally soundest option.
A third option to consider is sharing code between the two apps. For example, you could gem up the model code. Now your API is really some Ruby classes that know how to talk to your database. Going even further, you could write a Sinatra app and mount it inside of the existing Rails app and reuse big sections it. Then just work out the routing so that they look like separate apps to the outside world. Whether that's practical obviously depends on your specifics.
In terms of specific technologies, both Sinatra and Rails are fine choices. I tend towards Rails for bigger projects and Sinatra for smaller ones, but that's just me. Do what feels good.
I'm in the early stages of prototyping a Rails 3 application that will expose a public API. The site has three separate concerns which I am planning to split across three subdomains.
api.mysite.com
The publicly exposed API.
admin.mysite.com
The admin portal for creating blogs (using the public API).
x.mysite.com
The public blog site created at admin.mysite.com where x is the name of the blog. This too will make use of the public API.
All three will share domain objects. For example, you should be able to login to admin.mysite.com using an account you created on api.mysite.com or x.mysite.com.
Questions
Should I attempt to build one rails application to handle all three concerns or should I split this in multiple applications each handling a specific concern?
What are the Pros & Cons of each?
Does anyone have any insight into how some of the larger sites (basecamp, github, shopify) are organized?
Your question is fairly general so I'll try and answer in general terms. And the fact that you mention "larger sites" leads me to the conclusion that you're concerned about scaling.
In the beginning it is definitely going to be easier to build one application - especially since the domain is shared. You can do separate controllers for the various interfaces (api, html, etc) but with a shared back-end. This will reduce code duplication and the complexity of keeping 3 apps in sync. Also remember that you might change your mind about features based on user feedback and you want to be nimble enough to respond quickly.
The main benefit I can see of separating out three different deployables is that you can have an independent deploy schedule for each. For example, a bug fix in the api won't have to wait for admin to be ready to deploy. Or that you can have separate teams working in parallel.
If you're careful about what you keep in your session you'll be able to deploy multiple instances of your application on multiple servers, pointing at the same database (a.k.a. horizontal scaling). Each of these instances are identical to the others and a load balancer (either dedicated hardware or virtual) directs traffic between them. Eventually this approach runs out of steam when your database can't handle the load. At that point you can look at more caching, sharding, no-sql and all sorts of clever scaling techniques.
Most (but not all) larger sites end up doing some sort of horizontal scaling with some sharding of data.
All told, focus on getting a useful application to your users. If things take off you can worry about scaling. More applications fail because the user experience is awful rather than not being able to scale.
I am working on one big project. Now we need to add new functionality: scheduler managment.
It's not a main task of application, but it is quite complicated part of it.
Is it good idea to extract it as a separate application?
It will share some data (users and some other personalities) and it will use the same database.
The main reason I want to do it is to simplify main application.
I understand, that it is mayby too wide question. But maybe you can share your expirience of developing this kind of applications and maybe there are any articles I can read and world-wide best practices.
While others have mentioned some of the benefits of separating the applications, I'll touch on a couple of reasons why you might NOT want to separate the code.
You're going to need to maintain a single set of tests, especially if both applications are sharing the same database. If you don't do this, it's hard to predict when changing one application would break the other, especially if the applications start to need different things out of the database.
The two applications are obviously going to have a lot of overlap (users, for example). Separating into two applications could potentially force you to duplicate code, since rails by default has some pretty specific ideas about how a rails application should be structured. If your applications are sharing certain views, for example, what will you do to coordinate change in both applications when one application wants to modify the view?
Neither of these is insurmountable, but rails is easiest to develop when you follow rails conventions. As you begin to deviate, you end up having to do more work. Also, don't take either of these points as invalidating the other answers here, but merely counterpoints that you need to think about.
When you can use the functionality in other projects too, then I would separate it.
Maybe you can create a rails engine to share it easily between projects.
Consider asking yourself "What about re-usability?" Is the new scheduling functionality likely to be re-usable in another context with another application? If the answer is "yes," then perhaps making the scheduling management more modular in design will save you time in the future. If the answer is "no," then I would think you have more leeway in how tightly you integrate scheduling management with your existing app.
The practical difference here would be writing generalized scheduling management functionality that has assignable tables and methods upon which to act versus more 'hard coding' it with the data/code scheme of your 'onebig project.'
hth -
Perry
Adding management-tools into a web-app often complicate deployment, is my experience. Especially when the use of your application grows, and you need to performance-tune it, dragging along a huge "backend" may be problematic.
For sake of deploy-, scale- and test-ability, I prefer my applications to be small and focused. Sometimes it even pays off to have the entire admin-enviroment over REST-XML-services.
But as other answers point out: this is more a "it depends" solution. These are my €0.02.