Selecting Ruby gems at run-time based on configuration - ruby-on-rails

Question

We're writing a Rails app that we want to be able to talk to any of several external data repositories via a uniform adapter interface, and we'd like to be able to add more later just by writing new implementations of that adapter interface (cf. ActiveRecord adapters).
Any one installation of the app will only need one adapter running, and we don't want to have to rev the code or even the Gemfile whenever we introduce a new adapter. Assuming we write each adapter as a standalone gem, what's the proper way to incorporate just one adapter gem at runtime, based on configuration?

Answer 1

Normally in Rails, you can only access gems that are listed in the Gemfile; Bundler enforces this. However that doesn't stop you from loading gems manually.
If your adapter gems are installed in a well-known location, then loading an adapter gem could be as simple as this:
$LOAD_PATH << "/path/to/adapters/my_adapter/lib"
require "my_adapter"
Where "my_adapter" is something you discover at runtime via configuration.

Answer 2

Put your configuration in the environment, as you're probably already doing.
For example, I use the dotenv gem.
Set an environment variable to your adapter gem name.
The result is an ENV var that Rails can access, such as:
ENV["adapter"] #=> "my_custom_gem_name"
Use the environment to pick the adapter, e.g. in your Rails config:
require ENV['adapter']
This presumes your gem name is the same as the require name. If your gem name is different, use the require name.
When you deploy a new adapter, you can put it anywhere on the Rails load path. Or you can adjust the Rails load path as you like e.g. adding a path to $LOAD_PATH.
An example of how to adjust the load path:
# /config/application.rb
module MyApp
class Application < Rails::Application
$LOAD_PATH << "/path/to/custom/gem"
require("my_custom_gem_name")
…
For example, put the gem code in ./lib, or install the gem into ./vendor or systemwide, using any tool you like such as scp or rsync or ansible, thus bypassing the typical bundle command.
You won't need to update the Gemfile.

Answer 3

I don't think there's a good way to do quite exactly what you ask. Part of the bundler system (that's what uses Gemfiles) is that only gems mentioned in your gemfile (and their dependencies) are available to your app, it is isolated to only them. If this sounds like a bad thing, we could have a long conversation about the pre-bundler dependency management problems that this system has successfully solved.
Option 1 -- what are you worried about anyway? There is probably no downside to including all the possible gems in your Gemfile, even if a given installation will only use one. If using Rails, you will want to make sure these gems are not require'd on launch. (If not using Rails, this might not be neccesary, as other environments don't necessarily ask bundler to require all gems on boot (which is wise), but won't hurt).
So all the gems will get installed in every installation, yes, but only the one you want to use will be loaded, when you issue the require at runtime -- you'll still want to do this on application load, perhaps in response to an ENV variable, to avoid any concurrency or load time weirdness. Anyway, the unused gems will just be sitting there on disk un-loaded. How big a downside is having them installed but not used? How much are you willing to increase the confusingness of your setup to get rid of this downside?
Option 1 is what I'd do.
Option 2 -- separate Gemfiles. Another option is preparing a separate Gemfile for each type of setup. Gemfiles are just ruby code, so you could have one 'base' gemfile including common gems, and then a separate Gemfile for each type of setup, which uses ruby to load/include the base gemfile and then adds the setup-specific gems. You'd give each Gemfile a separate name, Gemfile_adapter1 or whatever.
You're going to have to commit something to your source when you add support for a new adapter type, aren't you? Where does this adapter come from? I don't understand how you could do it without revising any code. Anyway, adding a new Gemfile for that adapter type when you add a new adapter doesn't seem like a huge barrier, but I dunno.
You can launch Rails specifying which of these gemfiles to use with the BUNDLE_GEMFILE env variable: BUNDLE_GEMFILE=./Gemfile_one rails server. And in every other command you do that will use the Gemfile. BUNDLE_GEMFILE=./SOME_GEMFILE bundle install. BUNDLE_GEMFILE=./SOME_GEMFILE RAILS_ENV=production bundle exec rake assets:precompile. If you use capistrano that's doing some of these things for you, figure out how to make sure cap uses the right BUNDLE_GEMFILE when it executes it. Figure out how to make your app server do that when it launches your app. Etc.
This will work out fine -- but is going to end up being a pain to keep track of and make sure it's working right in your entire devops stack.
I suppose you could even generate the Gemfiles at install time, instead of having them in your source repo, for even more confusing situation and another thing that can go wrong and be confusing to debug! (I wouldn't).
I'd consider this Option 2, but would prefer option 1 unless there were really good reasons it wasn't going to work.
Option 3 -- Don't use Bundler. You could abandon bundler and gemfiles entirely. Maybe, if you can convince Rails to do this somehow. It might be hard to convince Rails to do this.
If you could, you are in a situation where you have to install all your gems on your deploy system by hand, making sure they are the right versions and figuring out what versions are compatible with what other versions, and that no more recent versions you don't want are installed.
Then, at runtime, you just need to require all the gems you want, and the app will get the most recent version of that gem installed on the system.
I would never, ever, ever, do this. I don't know if you can get Rails to do it, but even if you could, I remember the dependency hell from before Bundler existed, and would never want to go back.