I've got an existing webapp running on Rails. The plan is to setup a new server which will provide an API service, and eventually update the webapp to be a client of this API.
It seems like a good approach to achieving this would be packaging all the models as gems and sharing them between the two applications. Eventually the API service would be monolithic - containing all the models, but there is a period of development/migration where models will need to be shared.
Both the API and the webapp will be using the same database.
What do I need to consider before jumping into packaging up all my
models?
Would it be worthwhile to package all the models into a
single gem, individually package each model, or do some sort of
logical grouping of models?
How would I approach dependencies for
gems?
For reference, here's a similar-ish question:
Sharing models between Rails apps using gems
I'm also just getting familiar with packaging Ruby code as a Rubygem. (This might explain some of my questions above.)
[edit] I'm using Rails 2.3.14, not Rails 3.X.
We have done packaging models to a gem in the past. We started out by moving a group of models that are associated with each other but have no other dependency of the rest of the models. Then we slowly move another group. It is a bit painful but we eventually did it. The model gems are shared among several apps.
Related
THE PROBLEM
First a little bit of context: I am currently working as a freelancer, developping webapps using Ruby on Rails. Because I am working solo, the need to optimize my workflow is pretty important.
That's why I have always had the same question since I begun working with Rails:
How can I share code/configuration/conventions/tests between my apps?
More precisely, I want to share:
common gems that I always use, with their configuration
common integration tests, to ensure some conventions
common view helpers, test helpers, extensions to the core classes, javascript/sass partials
common files: .gitignore, git hooks, .eslintrc, configuration files of my CI etc
Some concrete examples of what I need in all my apps:
disable turbolinks by default, to add it later if need be
use javascript instead of coffeescript
use slim instead of ERB
install/configure capistrano
install a CSS framework (bootstrap, bourbon + neat + refills)
So far I don't really have the need to share models nor controllers.
I don't want to share behavior or functional components of the system itself, I am not looking for a micro-services architecture.
I have found that so far when creating new applications, all this setup work does take me a lot of time. Also, I would like to apply it retro-actively to existing apps when I add something new.
I have done quite a bit of research, but I haven't found a lot of answers. Many people are trying to share models, but few people seem to want to share a common ground between all their apps. Finding the right keywords may have been the problem though.
It seems to me that Rails is really good at DRY within an application, not so easy when trying to DRY between applications.
POSSIBLE SOLUTIONS
1 - Rails application template
The solution I am using right now, described in the Rails Application Templates guide, using the same API than the Rails generators described in the Creating and Customizing Rails Generators & Templates guide.
That's the solution used by Thoughtbot, with their popular suspenders gem. In the case of Thoughbot though, they have years of experience to draw from, have many employees and their common setup does not change that much.
Pros:
saves a lot of time when creating a new application
really nice API
Cons:
a lot of duplication: all the apps have the same common code, with the problem of this code getting out-of-sync
not retroactive: useless to add a common feature to already created apps
heavy maintenance work: my current workflow is to go through the git log of my apps once per month, and for each commit that could be common to all the apps I have, add it to the application template, and add it to the other apps manually
So far this solution is not that bad, because I only have two applications. But once I will have more, I will suffer from more and more overhead.
A better solution would be maybe to create a generator/rake task for every common new feature, to be able to apply it quickly to existing apps, and call it directly in the application template for new apps.
I haven't tried it though, and I am not really sure it will work. For example what if I want to propagate a one-line change in an existing common file in all the apps?
2 - Rails Engine
I have tinkered a bit with the Rails Engines to share code.
I have not understood from the Getting Started with Engines guide if I should better use a --full engine or a --mountable one for this specific purpose.
Pros:
once I update the gem version, all the changes are made available to the app
DRY: all the common code is in a unique place (the gem)
Cons:
the gems I would like to share are put in the *.gemspec file, which has not as many features as the Gemfile (from what I understand)
overhead caused by the need to update the version of the gem in all the apps, migrate to the new API of helpers, etc
no way to share non-rails files (.gitignore, git hooks, .eslintrc)
This solution has too many important shortcomings.
3 - Hybrid solution: Rails Application Template + Rails Engine
Maybe the best would actually to use both the above solutions.
In the gem share helpers and tests, in the rails template share the gems, their configuration/files and other files (for git, linters, etc)
It is indeed adding more complexity and overhead...
4 - Use git subtrees
Some people use git subtrees to share folders between multiple webapps.
Cons:
one has to share whole folders, not easy to share everything I would need, in their different target directories in the rails app
seems a bit "hacky" to me
Conclusion
Is there another solution than the ones I mentionned above?
What do you think would be the best way to do it?
How about having a blank "Master" rails app in the git somewhere. With all the settings and configurations you'd like to share. When creating a new app from scratch, you can merge the "Master" into it to apply the defaults. When you have an existing app, same thing, just merge and resolve conflicts as needed.This also gives you the ability to override the merged code if needed to.
Few things I can see wrong with this approach though:
Rails application name could cause a lot of headaches
Any updates could cause merge conflicts
I created a tool to deal with this when working in nodejs projects. But the tool is really just a command line tool so you should be able to use it.
https://github.com/tomasbjerre/dictator-builder
It is a concept with creating a dictator that dictates parts of your code base.
I'm creating an architecture of the app and trying to find the best one. Few words about it:
Box solution (each app installed on customer's machine)
There is a Main app and gems to extend it (private)
Main app has default list of gem
Other functionality extending by adding additional gems by request (our gems)
But some customers no need full functionality, so we don't need to include all gems to it's Gemfile.
What is the best way to organize it? What do you think about this way? Maybe you can offer more effective way?
You may want to consider making your "main app" or core functionality a mountable engine and not a Rails application.
That means packing the controllers, models, and routes into a gem that is added to a stock Rails application which is customized to the specific customer. This is how CMS:es like Refinery and Radiant are structured.
This allows full freedom for each "Box" to have whatever settings and gems it wants. This is especially important if you intend to let clients self host (which is a nightmare).
It also makes developing and updating the core easier as bundler will resolve the gem dependencies. You would add the "default" gems as dependencies of your mountable engine gem.
If you want to make creating/updating/developing the individual apps easier you could set this up as GIT forks of a "template" application.
I want to build my application in a modular way: For example I might have:
Users->has_and_belongs_to_many->Projects
Users->has_and_belongs_to_many->Tasks
Projects->has_many->Tasks
Tasks->belongs_to->Project (1 task belongs to 1 project.)
Note: Projects AND tasks can have many users.
The way I was thinking of building this is by doing:
User = Gem
Projects = Gem
Tasks = Gem
Each can return either json (for ember) or html. Neither depend on each other, The idea is that each is just a detail. Rails, ember and html are all just details.
How could I go about this and at the end of the day hook them together like a puzzle? or is this even feasible?
Right now I have the user piece of this concept almost done. It contains models, tests, controllers and views. Can I bundle that as a gem or should I only bundle the controller and the models? This "User" gem would make use of friendlyId gem to make the urls look nice, infact all the "gems" (project, tasks and users) will depend on this gem.
What would you recommend to do to have the kind of modularity I want? is it good practice? bad practice?
Pivotal Labs made a presentation (Euruko 2013 in Athens) and they showed a rails engines approach similar to what you are describing. Here is the presentation http://www.ustream.tv/recorded/35107339/highlight/377037
I haven't seen it in practice though, but I would like to.
I believe use of Rails engines is the way to go.
Each Engine can be an app in itself.
For instance Devise - gem that adds a lot of authentication functionality into your app is a Rails Engine. It provides its own controllers, helpers, mailers and views that you can use from your app.
I personally was working in projects that were using Engines to plugin i18n web backends (admin page that allows entering and saving translations to Redis) into many apps and to plugin users' bug reports and Question&Answer functionality into several existing projects.
Each engine is being included in a project as a gem, it can consist of anything that normal app does - controllers, views, models, assets, other gems...
I personally haven't heard of any success stories with such approach (but I don't know if anyone tried though), so if you have a solid concept in mind maybe it will work for you.
In computing terms, an engine is something that is continually producing output based on input. But in Ruby, the term seems a little bit loose. The people who've created Refinery CMS have taken to calling gems that extend the functionality of their system, 'engines'.
Basically, what I want to know is, is Spree, the open source ecommerce cms written in ruby, an engine? Would calling it the 'spree engine' be correct?
As defined by Rails, an Engine is a lot like an application within an application. Spree is one of these, and there are others. Each engine has its own app folder with the same sort of structure you'd see in a top-level application.
You can combine one or more engines together into an application, then add your own functionality on top. That's what makes systems like Spree far more flexible than a fixed-puropose Rails app that you have limited control over.
Things that extend Rails are only truly engines if they are self-contained applications. Many gems add functionality that's much more specific than that, so are better termed "plug-ins" or "modules" depending on the phrasing.
It's actually pretty easy to build your own engine and can be useful for separating and re-using code across several different applications.
I think that there is quite good explanation in guide Getting Started with Engines.
Spree is actually comprised of many engines...
The overcooked version: Engines come kitted with (many of) the guts of a typical rails application, with a few bonuses: namespacing out of the box, generators for easily copying migrations, and the ability to mount it in another rails application.
From http://guides.rubyonrails.org/engines.html
1 What are engines?
Engines can be considered miniature applications that provide functionality to their host applications. A Rails application is actually just a "supercharged" engine, with the Rails::Application class inheriting a lot of its behavior from Rails::Engine.
Try to read this guide: http://guides.rubyonrails.org/engines.html and also this cast: http://railscasts.com/episodes/277-mountable-engines
I have a set of models I need to share among several rails3/sinatra/etc applications. I haven't seen anything like this yet and I'm curious what is the most effective way to go about this in a DRY manner? Can I specify a central area for models, can I create a gem that only brings in the models I need? How have other people approached this issue. I was thinking of making a smallish gem or something I don't yet know about to handle this.
I'm using Datamapper, but this is also a more general issue of structuring multiple rails/non rails ruby apps.
You might find it's easier to have a submodule in your repository that contains your models and associated unit tests than to create a full-fledged gem. This way you can patch from one app to the other instead of having to drive everything top down.
You can create public API to models you want to use in other apps.
In rails you should already have REST controllers, so you can use ActiveResource in the other apps.
You can pack your ActiveResource classes in a gem just to simplify thing and to keep things DRY.
You can check: http://api.rubyonrails.org/classes/ActiveResource/Base.html for some examples.
And it doesn't matter if you're using ActiveRecord or DataMapper.