Our Rails suite is comprised of three independent Rails apps:
JSON API (Rails app)
Admin dashboard (Rails app)
Shared data models (Rails engine)
Both the API and Admin dashboard require the shared data models engine in their Gemfiles. All models and custom classes are stored in the engine, and both apps make heavy use of the shared components. The API lives on one Heroku server, and the Admin dashboard lives on another separate Heroku server (two separate Heroku apps). Each use their own respective Postgres databases. All three apps have their own GIT repos.
The API database stores information pertinent to our public users, and the Admin database stores mostly statistical information for admin eyes only.
A caveat of the setup is that the Admin dashboard app has direct access to the API database, and vice-versa. I understand that this is bad practice and may not seem to make sense, but there was a reason for this (mainly because the Admin dashboard needed to access all records of certain API tables, and the use of a custom API to communicate over the wire was not feasible). A similar reason exists for the API-to-Admin database communication.
This setup works for our purposes, nothing is broken, and resources are allocated efficiently. However, productivity is beginning to suffer due to the slow and uncomfortable development process. An example: a change to the API is required. Chances are that the shared models engine needs a change and therefore a feature branch is needed in both repos. After committing and pushing, the Admin dashboard now contains an old version of the models engine (is behind by one patch version). The problem lies in trying to coordinate all three Rails apps, when only one app needs a change. Another problem is migrations. Since the models engine contains two different database connections, I must create the migration once in the models engine then create it again in the appropriate app (API or Admin).
My ideal setup would involve one large Rails container app with separate engines contained within. The separate engines would be: API, Admin, Models. Also, I’m beginning to think that using only one database might make things easier. I would also like to keep the API on its own server instance, and the Admin on a separate server. The reason for this is that the API is public facing (communicates with a public iOS app) and the Admin is used mainly as a CMS and reporting engine.
I am looking for solutions and advice from experience managing similar Rails / Heroku architectures. Specific questions:
Should I attempt to combine the three Rails apps into one container
app and use the engine approach?
Should I attempt to combine the two
databases into one database?
Is it possible to have one Rails
container app, and allocate different servers to different engines?
If I should keep all apps separate, is their an easier and more
productive way to implement new features and fixes on a daily basis?
Related
Suppose we have this scenario:
www.main.com - Main interface where admin (foo, bar, etc..) can store products, based on their own e-commerce
www.foo.com - Sample store that sells items from the "foo" store
www.bar.com - Sample store that sells items from the "bar" store
The problem is finding a way to centralize the database structure and models.
I prefer to keep every single store in separated apps (so I exclude rails engines).
For instance, if a user buys something in the "foo" store, I need to interact with the main db and update it.
How can I do this?
Rails works much better with a "one database, one app" model. There are lots of ways how you can share models across apps (gems, engines, git submodules, etc), but none of those ways is great. You end up introducing lots of overhead in your development, deployment, and testing process. You also invite lots of hidden dependencies between code, as Rails doesn't give you easy way to keep clean abstraction (for example, you wrote a helper for store Foo, and then your coworker used that helper in store Bar, and then every time you change Foo, Bar breaks).
I recommend a centralized API approach instead:
api.foobarmain.com - a app/service that provides RESTful API for all the functionality of all stores.
This app has all the db models, and it exposes them as resources in the API for other apps to interact with.
This app can have an admin UI for all the stores, if you need it. Alternatively, admin UI may be another client of an API.
www.foo.com - a full stack app that interacts with API at api.foobarmail.com
There is no shared database connection to API, everything that you need to interact has to be exposed via API.
There will be no shared code between www.foo.com and www.bar.com. Code reuse happens only by virtue of using the same API app/service.
From the perspective of www.foo.com, the model layer (in MVC) is powered by API, not by database.
You can still have its own database on www.foo.com if you need to store data specific to www.foo.com only.
www.bar.com - another full stack app that interacts with API
so on ...
Another way is using multiple schema if you are using Postgresql. I have a similar issue with a new project that I about to start.
You can use gem 'apartment' to deal with different schemas. The queries will be a bit complicated but with different schemas you will end up with one database and you can create different namespaces to respond accordingly.
You can set up so app select the correct schema based on the Domain or Subdomain.
Here is the link: https://github.com/influitive/apartment
I am developing a multitenant Rails app using Postgresql schemas. All is going great, but my situation is a little different from the conventional multitenant apps out there; namely my app will require that I pull customer data for each tenant from their database to mine.
Here is where it gets tricky. I wrote a jRuby gem that connects to each customer's database and pulls data to my server, and then it processes that data and loads it into my Rails app (each customer set of data will end up in the appropriate tenant schema). Therefore this gem is the only place that it aware of all my tenants and their configuration (database info, which tables to pull, and so on).
My question is: What do you think of this design choice? Some of the problems I am already seeing is that this forced the app to be in two Ruby states, i.e. it normally functions in Ruby, but when I need to do a pull, I have to switch to jRuby. Furthermore, it is hard to inspect into tenant configuration without resort to this gem.
Any comments or feedback on this? Is there another path I could have taken with this?
I am planning on using Devise and Apartment in my upcoming application to create subdomains for each organization that creates an account. I would like to host my application on Heroku, but ran across the following quote:
The most common use case for using multiple schemas in a database is
building a software-as-a-service application wherein each customer has
their own schema. While this technique seems compelling, we strongly
recommend against it as it has caused numerous cases of operational
problems. For instance, even a moderate number of schemas (> 50) can
severely impact the performance of Heroku’s database snapshots tool,
PG Backups.
What technique would work well with Heroku to host basecamp-style subdomains in rails 4 where many users can log in to the subdomain which they are part of?
If Heroku does not work, what other PaaS options are there that would do this well?
Domain
Firstly, you need to be sure that you're using your own custom domain for the subdomains.
Heroku's standard xxx.herokuapp.com won't be able to handle another subdomain on top of that - so you'll basically need to use your custom domain from the get-go
It will be good to reference this documentation for more information!
Multi Tenancy
Although I don't have experience with PGSQL's schemas, I do have some with multi tenancy as a whole.
There are a number of great resources here:
Basecamp-style Subdomains (by DHH)
Multitenancy Railscasts (Pro)
Apartment Gem Documenatation
Essentially, multi-tenancy is just a way to scope the data so that it's only the tenant's that you see / interact with. In the sense of the DB, the two ways to achieve this are either to use different DB's (as you would with MYSQL), or use a schema (like with PGSQL)
Whilst I can't give you a direct fix for your issue, I can help you with some ideas:
Models
One way to achieve multi-tenancy, especially with the likes of MYSQL, is to do it through the model:
How do i work with two different databases in rails with active records?
#lib/admin.rb
class Admin < ActiveRecord::Base
self.abstract_class = true
establish_connection "#{Rails.env}_admin"
end
#app/models/option.rb
Class Option < Admin
# do stuff
end
This works very well for us, although we have not got it working for scoped accounts yet. We've been thinking of setting a ##class_variable for the Account or something, but haven't been working on that right now.
This works very well for MYSQL - powered databases, but also means you'll have to create db's for every account, which will not work with PGSQL (as far as I'm aware)
PGSQL Schemas
I feel this is kind of a cheat way to do this, as all the data is still stored in 1 database - it's basically just scoped around different types of data.
The problem here is that real multi tenancy should be where you completely separate the user's data, so you could cut it out of the app completely if they wanted. From a security & access perspective, it's the most flexible & modular way.
The problem for Heroku is they can only use one database (they give everyone access to their AWS database instances), meaning they can't allow you to create 50+ free databases (it just won't work very well).
You can, of course, use your own stack to create the databases you require, but in terms of PGSQL, it's just about creating the schemas for your data & then using something like -Apartment to make it happen:
PostgreSQL works slightly differently than other databases when
creating a new tenant. If you are using PostgreSQL, Apartment by
default will set up a new schema and migrate into there. This provides
better performance, and allows Apartment to work on systems like
Heroku, which would not allow a full new database to be created.
So I am starting out on company project that will have several components:
At first...
Job list
Client profile creation and management
User administration and access (login, signup, roles, etc)
later...
Messaging
Schedule
Basic reporting
way later...
Deeper analysis and bi
I'm wondering if it makes sense for each bullet item to be its own rails project, self contained and modular (if that is indeed the case); or if it's just best for it to be in the same app. I could envision a situation where each module could operate so independently of each other that it wouldn't need the rest (except for the user funcionality) and another situation where all modules would be used together.
It seems like to me that many tasks can be handled with a lighter framework like Sinatra (and then situated physically under the rails app). It also seems like it would be a lot of overhead to have several rails apps running on a server. But I am not totally aware of all the pluses and minuses to operating each scenario.
I know this is kind of a general question that is bound to get a lot of "it depends" kind of responses (and rightfully so) I was looking for opinions/examples of how you setup this kind/your kind of project in rails. I am a quasi noob so be gentle.
Thanks in advance!
Generally speaking I would consider a website to be a suitable target for a Rails app. Each part of the app can have its own namespaces within the app, so the app has some structure internally, but they should all be one application. Consider things like sessions, where you want a user to login and use whatever features of the site you want. You want those sessions in one application without a user having to login to different sections.
Saying that, if there is complex or extended functionality that isn't part of the MVC architecture (say talking to an external API, data-mining etc), then you could offset that to a separate project and a include it as a Gem in your application. You would still have one main Rails application that includes those Gems.
You might also want to bundle together a section of your project into a reusable Rails engine that can be loaded into multiple projects. For example, Devise handles user login and management. It is a Rails engine, bundled as a Gem, that you include in your project.
Another example from Meducation (one of my sites). I'm in the process of extracting our email tracking system out into its own Rails engine as I feel its functionality sits alongside Meducation and is not a core part of it. I can then use it in other projects as well.
In your specific example, I think your requirements fit fine in one Rails application.
It's an application that we use internally at the office that I would like to offer as a hosted service for anyone.
How can I do that without making major code changes?
The first thing that occurs to me is to have the app select which database to connect to based on the domain.
So each instance of the app would have its own database, but all instances would share the same code.
The only changes required to the code would be the database selection.
Is this approach maintainable? I've heard wordpress.com does this and that it offers a couple of advantages. I'm mainly looking to do it this way to avoid have to scope my entire set of database queries to a certain site within the same database.
Thanks!
The simplest way to do this is to clone the application, and create another server instance to handle it. This actually the way I handle multiple wordpress blogs on my server
Pro:
This process can be streamlined into a utility script.
Can be easily maintained if symlinks are used for the common code. IE: Everything but branding and some of the things in the config directory.
Cons:
- If you're using passenger it will require an apache restart for each new instance.
- Same if you're using Apache to route subdomains on different virtual hosts to different mongrel clusters.
However the better way comes from the question: Rails - Separate Database Per Subdomain
The method in the accepted answer is much more robust. It might require more changes than you're looking for, but it has all the benefits without the drawbacks of any other methods. Each new instance requires a new entry in the master database with the table name and other instance specific information. You'll also want custom rake task to build the database for each new instance.
I would suggest switching the database connection and adding a view_path based on the domain, I have posted code in this question.
I hope this helps!
I wouldn't do this with multiple databases as you mentioned. Keeping all your schemas/migrations in sync with all the db's could become painful.
I would look into simply making it a multi-tenant app where you have some sort of "Account" model and then all your existing models are scoped to it ... in other words, if this was a blog app, your Account has_many :posts, etc.
With this approach, you can identify accounts by subdomain ... have people choose their subdomain when they create an account and go from there.
It's pretty straightforward to do. If you need add billing into the mix, you might look at the SaaS Railskit (which handles all the signup and subdomain stuff) or Chargify.
You can also identify accounts Twitter-style ... with http://myapp.com/someuser