I know it's gonna be a very broad question and I'm pretty sure this is not anything new in todays world, but I really don't now where to start on this one.
I've started working on a rails-based service that would allow users to create, configure and manage their projects (basicaly copies of a single app with customization). The projects would be run on a subdomain with an option of connecting their own domain. (Quite similar to how SquareSpace and Wix and many other website building services handle their websites)
I want these projects to be as flexible and customizable as possible (including the visual design, some additional features etc) while maintaing ease of setup and updates for the codebase itself.
So the questions I start with:
Should each project be a separate app (basicaly a copy out of the
same repo) with its own configs, database, nginx and unicorn
configs, etc. or should I keep all in 1 app and differenciate them
with different configs and maybe databases and assets?
If it's better to keep everything separated...
... what's the best setup process (provided that I have basic configuration to start with)?
... how would I be able to update them all (with bugfixes, new features, performance improvements etc.) from one place?
If I keep all the projects in the same app...
... how should I manage the configuration concurency between projects?
... how do I separate the data and assets?
As always, I'd be grateful for any sufficient help I can get.
I have no experience building these kind of things and maybe there is an easy solution I'm not aware of. But my answer may give you some ideas/pointers to start with.
1) It's hard to give an accurate answer but based on the high personalization criteria and the "copies of a single app" you talk about, I would go for an architecture more PAAS-like than SAAS-like. So separated apps.
2.1) You may want to consider a setting using:
A main rails app, with a main webserver and a main nginx conf. A wildcard domain.
A bunch of dynamically managed docker containers. Each container is initialized from your single app, and then personalized by the user. User is associated to its containers through the main rails app. When a container is created from the main app, the main nginx config is updated (i.e. by adding a file in sites-enabled which define a new 'server' that bind the open port of the new container, and the right 'server_name').
Each contained rails app ship an unique and shared rails engine. The rails engine is shipped by adding the corresponding gem in the Gemfile so it can be updated.
2.2) You update the 'shared rails engine' gem. You can then run bundle install in each containers.
People interested in these kind of setups may consider tools like dokku and deis.
Consider a multi-tenant architecture. We've used it effectively for fleet-genius.com.
The following link provides a good overview of the architecture from a Rails point of view.
http://blog.elbowroomstudios.com/zero-to-multitenant-in-15-minutes-a-rails-walkthrough/
Related
THE PROBLEM
First a little bit of context: I am currently working as a freelancer, developping webapps using Ruby on Rails. Because I am working solo, the need to optimize my workflow is pretty important.
That's why I have always had the same question since I begun working with Rails:
How can I share code/configuration/conventions/tests between my apps?
More precisely, I want to share:
common gems that I always use, with their configuration
common integration tests, to ensure some conventions
common view helpers, test helpers, extensions to the core classes, javascript/sass partials
common files: .gitignore, git hooks, .eslintrc, configuration files of my CI etc
Some concrete examples of what I need in all my apps:
disable turbolinks by default, to add it later if need be
use javascript instead of coffeescript
use slim instead of ERB
install/configure capistrano
install a CSS framework (bootstrap, bourbon + neat + refills)
So far I don't really have the need to share models nor controllers.
I don't want to share behavior or functional components of the system itself, I am not looking for a micro-services architecture.
I have found that so far when creating new applications, all this setup work does take me a lot of time. Also, I would like to apply it retro-actively to existing apps when I add something new.
I have done quite a bit of research, but I haven't found a lot of answers. Many people are trying to share models, but few people seem to want to share a common ground between all their apps. Finding the right keywords may have been the problem though.
It seems to me that Rails is really good at DRY within an application, not so easy when trying to DRY between applications.
POSSIBLE SOLUTIONS
1 - Rails application template
The solution I am using right now, described in the Rails Application Templates guide, using the same API than the Rails generators described in the Creating and Customizing Rails Generators & Templates guide.
That's the solution used by Thoughtbot, with their popular suspenders gem. In the case of Thoughbot though, they have years of experience to draw from, have many employees and their common setup does not change that much.
Pros:
saves a lot of time when creating a new application
really nice API
Cons:
a lot of duplication: all the apps have the same common code, with the problem of this code getting out-of-sync
not retroactive: useless to add a common feature to already created apps
heavy maintenance work: my current workflow is to go through the git log of my apps once per month, and for each commit that could be common to all the apps I have, add it to the application template, and add it to the other apps manually
So far this solution is not that bad, because I only have two applications. But once I will have more, I will suffer from more and more overhead.
A better solution would be maybe to create a generator/rake task for every common new feature, to be able to apply it quickly to existing apps, and call it directly in the application template for new apps.
I haven't tried it though, and I am not really sure it will work. For example what if I want to propagate a one-line change in an existing common file in all the apps?
2 - Rails Engine
I have tinkered a bit with the Rails Engines to share code.
I have not understood from the Getting Started with Engines guide if I should better use a --full engine or a --mountable one for this specific purpose.
Pros:
once I update the gem version, all the changes are made available to the app
DRY: all the common code is in a unique place (the gem)
Cons:
the gems I would like to share are put in the *.gemspec file, which has not as many features as the Gemfile (from what I understand)
overhead caused by the need to update the version of the gem in all the apps, migrate to the new API of helpers, etc
no way to share non-rails files (.gitignore, git hooks, .eslintrc)
This solution has too many important shortcomings.
3 - Hybrid solution: Rails Application Template + Rails Engine
Maybe the best would actually to use both the above solutions.
In the gem share helpers and tests, in the rails template share the gems, their configuration/files and other files (for git, linters, etc)
It is indeed adding more complexity and overhead...
4 - Use git subtrees
Some people use git subtrees to share folders between multiple webapps.
Cons:
one has to share whole folders, not easy to share everything I would need, in their different target directories in the rails app
seems a bit "hacky" to me
Conclusion
Is there another solution than the ones I mentionned above?
What do you think would be the best way to do it?
How about having a blank "Master" rails app in the git somewhere. With all the settings and configurations you'd like to share. When creating a new app from scratch, you can merge the "Master" into it to apply the defaults. When you have an existing app, same thing, just merge and resolve conflicts as needed.This also gives you the ability to override the merged code if needed to.
Few things I can see wrong with this approach though:
Rails application name could cause a lot of headaches
Any updates could cause merge conflicts
I created a tool to deal with this when working in nodejs projects. But the tool is really just a command line tool so you should be able to use it.
https://github.com/tomasbjerre/dictator-builder
It is a concept with creating a dictator that dictates parts of your code base.
TL;DR:
Assets, partials, layouts, gem configurations, other code shared among apps
Different database for every app
Ability to update codebase across all apps at once, not affecting custom code on client's site
Apartment? Rails Engine? CDN?
I'd like to have certain elements (admin navbar and sidebar, devise configuration (aside from :host), as well as other bits) available on all of my client's sites. That's great and all, but I'd also like to be able to update my codebase in once, and push it out (preferably with git), to ALL of my customer's sites, to insure they have the latest code I've written.
I've been looking into creating an Engine for base functionality and creating a CDN for assets, but I'd still manually have to do any interface changes (such as adding a link in the admin navbar) by hand with every customer's project.
Can apartment solve this problem, and still allow each customer to have their own unique codebase/git repo that I use for their own website?
I know I'm not doing the best job of accurately describing my issue, so if you'd like me to clarify anything, please ask.
Any reading material would be appreciated.
EDIT: I should add I'm currently using Deis on an AWS cluster for production.
I've been using nodeJS + expressJS for several years now developing a custom Application Platform for our organization. Our central framework provides a common set of services (authentication, language, administration, etc...) for any installed Modules/Applications under it.
I would like to switch our framework out with compoundJS. However I'm not familiar with the design constraints imposed by it (and Rails apps in general) and can't seem to figure out how to accomplish what I'm after.
I would like to only have a single server instance running: all
requests first process through our common authentication checking.
Then are passed on to an application's controllers.
I would also like to have each application separated out: preferably
under a separate site/applications/ directory. Each of these
applications could be designed using compoundJS normally. And I would like to install them like:
cd site/applications
npm install site-hr
npm install site-finance
npm install site-payroll
this would then have all the routes from /hr, /finance, /payroll operational.
How do I accomplish this?
Is there a way to get compoundJS to search the nonstandard /applications/* folders for models/controllers/views and load them while keeping the central /site configurations?
Or is there a better way?
Sorry for the late answer, but I needed something similar: I needed to put together tool applications in a portal.
I found a way to include child applications in a parent's Compound application as node modules. I wrote a guide on how to do it and sent a pull request to add it in the advanced folder of CompoundJS' guides. It is also available here. It requires a bit of work, but it works fine with 4 applications for us.
Hope it can help.
It's simple. Just use app.use in config/environment.js to map your sub-apps:
var mod = require('your-compound-module');
app.use('/subroot', mod());
When you visit /subroot/any-path, then it will be handled by /anypath route of your sub-app. Note, that you don't need any additional work on path helpers, as they will start with '/subroot' automatically (handled on compound side).
This is a good point, but we haven't seen any implementation yet. May be years later there would be some.
Using a proxy layer in front of the instances would be a general method, usually with Nginx, Vanish Cache etc. For the bleeding edge techs, I've heard Phusion Passenger has implemented Node.js support, but I haven't successfully tested yet. If you are familiar with Ruby, it would be a good try.
If you really want to construct a big project with many modules, you can try out some industrialized frameworks for instance Architect for Cloud9 IDE project.
For authentication, I think it's necessary to use independent methods in each application, but they can share with one user database.
We are going to be hosting around 20 client sites for Ruby on Rails. We need the best Control Panel for our Server/VPS what do you recommend? I've researched and can find no good blogs or posts about this exact subject.
Some of my findings:
1) Cpanel does not seem good as it doesn't support Rails 3.2.8
2) Would just SSH manual config management be enough? Some people have told me that is what people do with capistrano. But how do you control resources like bandwidth, cpu usage?
3) In the future we would like to offer a free trial signup that auto-creates the accounts and installs our app.
The ideal solution would be similar to cpanel, but allow Rails 3.2.8 running with Apache/Passenger
I think many Rails firms build out their own control panel in Rails. If you're on one VPS, this should be very easy. (If you have multiples VPSs, it's still doable, but your control panel app will have to do everything via SSH, perhaps using Capistrano.)
In your control panel Rails app, you can create some models that know how to do various server administration tasks. For example, if you're using Apache, one of your models will probably have to know how to edit http.conf to add a new virtual host.
Your idea about automatically creating the apps on signup can be done this way too. Write some model logic that checks the app out from source control, creates the virtual hosts or whatever, creates the database (if each client gets their own), etc.
Also consider pushing some of the tasks onto a job queue, rather than handling them synchronously. (I'm a big fan of resque). This is a good idea for tasks that take a long time, such as checkout from source control.
You may even want to extract all this server admin functionality into a gem for your private use. Then, you can easily perform the same tasks from custom scripts outside the Rails app, should you need to.
I am trying to build a CMS I can use to host multiple sites. I know I'm going to end up reinventing the wheel a million times with this project, so I'm thinking about extending an existing open source Ruby on Rails CMS to meet my needs.
One of those needs is to be able to run multiple sites, while using only one code-base. That way, when there's an update I want to make, I can update it in one place, and the change is reflected on all of the sites. I think that this will be able to scale by running multiple instances of the application.
I think that I can use the domain/subdomain to determine which data to display. For example, someone goes to subdomain1.mysite.com and the application looks in the database for the content for subdomain1.
The problem I see is with most pre-built CMS solutions, they are only designed to host one site, including the one I want to use. So the database is structured to work with one site. However, I had the idea that I could overcome this by "creating a new database" for each site, then specifying which database to connect to based on the domain/subdomain as I mentioned above.
I'm thinking of hosting this on Heroku, so I'm wondering what my options for this might be. I'm not very familiar with Amazon S3, or Amazon SimpleDB, but I feel like there's some sort of "cloud database" that would make this solution a lot more realistic, than creating a new MySQL database for each site.
What do you think? Am I thinking about this the wrong way? What advice do you have to offer in this area?
I've worked on a Rails app like this, and the way it was done there was named-based virtual hosts, with db entries for each site running. Each record was scoped to a site if necessary (blog posts, etc.) while users would have access to all sites running out of that db. Administrator permissions could be global or scoped to one or more sites.
You're absolutely correct when you say you'll reinvent the wheel a million times during the project. Plugins will likely require hacking on top of the CMS itself.
In my situation, it ended up being a waste of almost a million dollars of company money to build that codebase to run multiple sites while still being able to cater to the whims of each client site. It worked, but was not very maintainable due to the number of site-specific hacks that subsequently entered the codebase. You may be able to make it work if you don't have to worry about catering to specific client sites running on your platform.
In the end, you're going to need a layer of indirection to handle the different sites regardless of methodology. We ended up putting it in the database itself. If you go with the different-db-for-each-site method you mentioned, you'll put that layer in your code instead. I'm not sure which one is the better method.
I hope you're able to pull this off. I failed.
Also, as I learned today, Heroku offers postgres instead of mysql for rails apps.
There's James Stewart's Theme Support Plugin for Rails 2.3, and lucasefe's themes_for_rails gem for Rails 3+.
I just started using the 2.3 version and it's working well so far.