Suppose we have this scenario:
www.main.com - Main interface where admin (foo, bar, etc..) can store products, based on their own e-commerce
www.foo.com - Sample store that sells items from the "foo" store
www.bar.com - Sample store that sells items from the "bar" store
The problem is finding a way to centralize the database structure and models.
I prefer to keep every single store in separated apps (so I exclude rails engines).
For instance, if a user buys something in the "foo" store, I need to interact with the main db and update it.
How can I do this?
Rails works much better with a "one database, one app" model. There are lots of ways how you can share models across apps (gems, engines, git submodules, etc), but none of those ways is great. You end up introducing lots of overhead in your development, deployment, and testing process. You also invite lots of hidden dependencies between code, as Rails doesn't give you easy way to keep clean abstraction (for example, you wrote a helper for store Foo, and then your coworker used that helper in store Bar, and then every time you change Foo, Bar breaks).
I recommend a centralized API approach instead:
api.foobarmain.com - a app/service that provides RESTful API for all the functionality of all stores.
This app has all the db models, and it exposes them as resources in the API for other apps to interact with.
This app can have an admin UI for all the stores, if you need it. Alternatively, admin UI may be another client of an API.
www.foo.com - a full stack app that interacts with API at api.foobarmail.com
There is no shared database connection to API, everything that you need to interact has to be exposed via API.
There will be no shared code between www.foo.com and www.bar.com. Code reuse happens only by virtue of using the same API app/service.
From the perspective of www.foo.com, the model layer (in MVC) is powered by API, not by database.
You can still have its own database on www.foo.com if you need to store data specific to www.foo.com only.
www.bar.com - another full stack app that interacts with API
so on ...
Another way is using multiple schema if you are using Postgresql. I have a similar issue with a new project that I about to start.
You can use gem 'apartment' to deal with different schemas. The queries will be a bit complicated but with different schemas you will end up with one database and you can create different namespaces to respond accordingly.
You can set up so app select the correct schema based on the Domain or Subdomain.
Here is the link: https://github.com/influitive/apartment
Related
I've started off with a single Rails application. Very simple, largely read-only, front end for a line of business application (using view backed to retrieve data) - with a few standard tables to augment the views.
I now have a need to use the same set of data in a new application (the 2 applications, whilst sharing the same data, work differently enough its not trivial to try and merge them into the same application).
I figured it would be easiest to split the models I can reuse into their own engine and have the 2 applications share the database.
Adding an API and having both applications query that is an option, but in practice I'm not sure I can provide an API that will satisfy both applications properly, as they use the data in different ways.
On that basis I figured if I gave each application a table prefix, or use different schemas to namespace them - that way each application has it's own distinct tables for stuff they don't share, but I can easily reuse the existing views without having to duplicate them.
Both options seems to work great, except I forgot about migrations for the common views and data.
So the only things I can think of are:
Since the 2 applications are kinda tightly coupled anyway, I have no migrations in my common data engine at all - any changes to the views/tables will be dealt with by the "first" application. This seems a bit nasty, since the models are now contained in a separate engine.
I dislike the idea of having migrations in the engine and then copying them into one or the other, since that's basically the same thing.
I use pivotal labs advice, but add some code to detect if it's in the "first" application, and only apply to that. If I fail to do that I end up with both applications including the engine migrations, which results in both applications trying to run the same migrations, and causing nothing but pain.
I actually split off the common data into it's own database. So application #1 uses db #1, application #2 uses db #2, and the common data is housed in db #3 and accessible by both applications. With a bit of faffing, I'm guessing I can end up with 3 dbs, 3 schema_migrations, and I can just blindly leave my migrations in the engine, and include them in both applications as per pivotal labs - my plan is to do something like this to make all this work, and have the common models setup to connect to their own DB, rather than the application DB
Stick with 1 db and multiple schemas, and somehow setup a task to run the engine migrations only, using an account locked down to it's own schema only - that way it creates it's own schema_migrations.
I kinda feel paralyzed as I'm not sure what is the least shitty option. 3 or 4 feels "best", but not great.
I think 3 is the best option. However, I'd expose the common db #3 data via an api. That is, have an application that is used to manage the shared data (HTML admin interface, and json feeds for the other apps to connect on).
Keep the shared data app really simple. Just CRUD actions on the data. So the other two apps will grab the data from the shared data app, manipulate to match local requirements, and then display it; or allow input - manipulate the input to match the shared structure, and then persist via the shared data app's API update/create actions.
As an update to this, due to how tightly coupled the 2 applications are, and other needs that came up yesterday, I've ended up pulling the migrations out of both and using active_record_migrations to manage and orchestrate the migrations for both applications.
Effectively I could've done the same by having a dummy or master application, however this feels somewhat cleaner.
However, this comes at the expense of being able to use models effectively in the migrations. For my use case this doesn't matter at all.
I have a rails 4 app that exposes API to external users. The users also get access to a web dashboard where they can see & manage data related to API calls, similar to stripe. Stripe dashboard also allows you to switch between live & test data. I am looking to replicate similar behavior. Are there any design recommendations or a Rails way on how to do this? Use separate database (db_live vs db_test) or use separate tables inside db_live, and then use *_test table naming convention to access test data inside live database.
Whats the Rails/ActiveRecord way to do this? I am using Postgres as the database.
One potential solution would be to simply add a live (or test) boolean column to the appropriate database tables and use scopes to apply the desired where condition. An index on the column would also help with performance.
The practicality of this solution depends on exactly how test data is generated and how much of it you expect there to be per user/account.
Was searching for the same answer as well. Till now, the best option I can think of is to use a multi-tenant system. You can set a session variable as test|live and based on it connect to different databases OR in case of postgres different schemas. This way, all our code will remain DRY and all the switching logic between test and live systems can be moved in a single place.
Here's a basic idea on multi-tenant systems:
http://jerodsanto.net/2011/07/building-multi-tenant-rails-apps-with-postgresql-schemas/
Our Rails suite is comprised of three independent Rails apps:
JSON API (Rails app)
Admin dashboard (Rails app)
Shared data models (Rails engine)
Both the API and Admin dashboard require the shared data models engine in their Gemfiles. All models and custom classes are stored in the engine, and both apps make heavy use of the shared components. The API lives on one Heroku server, and the Admin dashboard lives on another separate Heroku server (two separate Heroku apps). Each use their own respective Postgres databases. All three apps have their own GIT repos.
The API database stores information pertinent to our public users, and the Admin database stores mostly statistical information for admin eyes only.
A caveat of the setup is that the Admin dashboard app has direct access to the API database, and vice-versa. I understand that this is bad practice and may not seem to make sense, but there was a reason for this (mainly because the Admin dashboard needed to access all records of certain API tables, and the use of a custom API to communicate over the wire was not feasible). A similar reason exists for the API-to-Admin database communication.
This setup works for our purposes, nothing is broken, and resources are allocated efficiently. However, productivity is beginning to suffer due to the slow and uncomfortable development process. An example: a change to the API is required. Chances are that the shared models engine needs a change and therefore a feature branch is needed in both repos. After committing and pushing, the Admin dashboard now contains an old version of the models engine (is behind by one patch version). The problem lies in trying to coordinate all three Rails apps, when only one app needs a change. Another problem is migrations. Since the models engine contains two different database connections, I must create the migration once in the models engine then create it again in the appropriate app (API or Admin).
My ideal setup would involve one large Rails container app with separate engines contained within. The separate engines would be: API, Admin, Models. Also, I’m beginning to think that using only one database might make things easier. I would also like to keep the API on its own server instance, and the Admin on a separate server. The reason for this is that the API is public facing (communicates with a public iOS app) and the Admin is used mainly as a CMS and reporting engine.
I am looking for solutions and advice from experience managing similar Rails / Heroku architectures. Specific questions:
Should I attempt to combine the three Rails apps into one container
app and use the engine approach?
Should I attempt to combine the two
databases into one database?
Is it possible to have one Rails
container app, and allocate different servers to different engines?
If I should keep all apps separate, is their an easier and more
productive way to implement new features and fixes on a daily basis?
The iTunes Enterprise Partner Feed is "a data feed of the complete set of metadata from iTunes and the App Store" and "is available in two different formats - either as the files necessary to build a relational database or as stand-alone flat files that are country and media dependent."
I need to consume the data from this feed (which is essentially exported into flat files) and allow linking of my own Model objects (User, Activity, etc.) to data provided by the feed (App, Developer, etc.) The data is provided as a weekly full export and a daily incremental export.
I have two ideas for ways to implement this:
Create all of the models in my rails app and write my own importer that will insert/update records directly into my app's database daily via cron using models I've created (App, Developer, etc.)
Keep this database entirely separate and open up REST API that my own app will consume
My naive approach with #1 to keep everything in the Rails app is based on the need to be able to observe changes in the data I'm getting from the EPF. For example, if an App's description is updated, I want to be able to create an Activity object via an observer to track that update.
On one hand #2 feels like a better approach because it creates a standalone API into the data that can be consumed from several different apps I create. On the other hand, I'm just not sure how I'd accomplish the data change notifications without using observers directly on my own models. Or, how I would even consume the data in an object oriented way that could still be used with my own models. It feels like a lot of duplicate work to have to query the API for, say, an App's data, create a proper Active Record object for it and then save it just so it can be linked to one of my own models.
Is there a standard way to do this that I'm totally missing? Any pointers?
EDIT: Rails engines sound interesting but it would mean that each app would still need to consume and insert the data separately. That doesn't sound so DRY. It sounds more and more like a REST API is the way to go. I just don't know how to bridge the gap from API to Active Record model.
Rails Engines might be a good fit for this. You can create a Rails Engine gem and add all of your models and rake tasks to consume the data. Then you can include this gem in any app that uses it and also create an API app which includes the gem. You should be able to create observers in your other apps that interact with the gem.
I have quite a few apps that interact with each other and this approach works well for me. I have one central app that includes all the engines that consume data and I run all of my cronjobs from this app. I use the use_db plugin which allows my app to communicate with different databases. Each engine has use_db as a dependency and I keep the database configuration inside the gem. One example:
Engine Gem = transaction_core
This gem consumes transaction data from a source and inserts it into my transaction database.
The gem is included in my central app and I pull the transaction data using a rake task on the cron
I include this gem in several other apps that need to use the transaction data. Since the engine automatically adds the models and database config to the app, there is no additional work required to use the models in the app.
I have not used observers inside an app that includes my engines, but I see no reason why it would not work. With the engine the models work as if they are in your app/models directory. Hope this helps!
Modest Rubyist has a good 4 part tutorial on Rails 3 plugins that includes Engines:
http://www.themodestrubyist.com/2010/03/05/rails-3-plugins---part-2---writing-an-engine/
It's an application that we use internally at the office that I would like to offer as a hosted service for anyone.
How can I do that without making major code changes?
The first thing that occurs to me is to have the app select which database to connect to based on the domain.
So each instance of the app would have its own database, but all instances would share the same code.
The only changes required to the code would be the database selection.
Is this approach maintainable? I've heard wordpress.com does this and that it offers a couple of advantages. I'm mainly looking to do it this way to avoid have to scope my entire set of database queries to a certain site within the same database.
Thanks!
The simplest way to do this is to clone the application, and create another server instance to handle it. This actually the way I handle multiple wordpress blogs on my server
Pro:
This process can be streamlined into a utility script.
Can be easily maintained if symlinks are used for the common code. IE: Everything but branding and some of the things in the config directory.
Cons:
- If you're using passenger it will require an apache restart for each new instance.
- Same if you're using Apache to route subdomains on different virtual hosts to different mongrel clusters.
However the better way comes from the question: Rails - Separate Database Per Subdomain
The method in the accepted answer is much more robust. It might require more changes than you're looking for, but it has all the benefits without the drawbacks of any other methods. Each new instance requires a new entry in the master database with the table name and other instance specific information. You'll also want custom rake task to build the database for each new instance.
I would suggest switching the database connection and adding a view_path based on the domain, I have posted code in this question.
I hope this helps!
I wouldn't do this with multiple databases as you mentioned. Keeping all your schemas/migrations in sync with all the db's could become painful.
I would look into simply making it a multi-tenant app where you have some sort of "Account" model and then all your existing models are scoped to it ... in other words, if this was a blog app, your Account has_many :posts, etc.
With this approach, you can identify accounts by subdomain ... have people choose their subdomain when they create an account and go from there.
It's pretty straightforward to do. If you need add billing into the mix, you might look at the SaaS Railskit (which handles all the signup and subdomain stuff) or Chargify.
You can also identify accounts Twitter-style ... with http://myapp.com/someuser