Bypassing bundler for auxiliary development gems - ruby-on-rails

In the Gemfile of my Rails project, I am starting to have auxiliary gems like "ruby-debug19", "perftools.rb", or "irbtools". All of these really have nothing to do with the project, but rather are part of my local development setup. But since I'm using bundler, I cannot load these gems (even though they are installed system-wide) unless I add them to the Gemfile. In my view that is a bit of a code smell.
For example, I would like to be able to require 'irbtools' in rails console without adding "irbtools" to my Gemfile.
Is there a way to keep auxiliary gems out of the Gemfile and still be able to load them for debugging, profiling, etc. when I need them?

Actually, you can create a group in you Gemfile like:
group :auxiliary do
gem 'irbtools'
end
And then use bundle install --without auxiliary if you don't want to use irbtools. Why do you think adding them to Gemfile is a code smell? And if it possible to do this without adding gems to the Gemfile it will be many more code smell I think.

Thanks to this post I have a great solution.
Add this line at the end of your Gemfile:
eval(File.read(File.dirname(__FILE__) + '/Gemfile.local'), binding) rescue nil
Create a file called Gemfile.local.
Add your development gems to Gemfile local. For example:
group :development do
gem 'cucumber'
end
Add Gemfile.local to .gitignore.
Now you can add your auxiliary development gems without changing the Gemfile for other folks on the team. Very cool.

I put the code below in a file in my app root, so it's easy to load from irb.
If you want it in something like a rails server, you probably need to add the load statement to environments/development.rb etc. That still creates problems if you accidentally check that in, but it's less annoying than having to add it to the Gemfile and causing your Gemfile.lock to change also.
# Example usage:
# add_more_gems("ruby-debug-base19-0.11.26", "linecache19-0.5.13")
# or
# add_more_gems(%w(ruby-debug-base19-0.11.26 linecache19-0.5.13))
#
# Note that you are responsible for:
# - adding all gem dependencies manually
# - making sure manually-added gem versions don't conflict with Gemfile.lock
# To list deps, run e.g. "gem dep ruby-debug-base19 -v 0.11.26"
#
def add_more_gems(*gem_names_and_vers)
gem_names_and_vers.flatten!
gem_base = File.expand_path(Gem.dir)
gem_names_and_vers.each do |gem_name_and_ver|
# uncomment if desired
###puts "Adding lib paths for #{gem_name_and_ver.inspect}"
spec_file = File.join(gem_base, 'specifications', "#{gem_name_and_ver}.gemspec")
spec = Gem::Specification.load spec_file
this_gem_dir = File.join(gem_base, 'gems', gem_name_and_ver)
spec.require_paths.each {|path|
dir_to_add = File.join(this_gem_dir, path)
$: << dir_to_add unless $:.member?(dir_to_add)
}
end
end
# put your often-used gems here
add_more_gems(
%w(
ruby-debug-base19-0.11.26
ruby-debug-ide19-0.4.12
linecache19-0.5.13
)
)

Not sure if this would work for you. It depends on whether or not you're using RVM. If you are, then you could install those auxiliary gems into the #global gemset that is created automatically for every Ruby interpreter. The gems in the #global gemset are available to all project-specific gemsets by default. This way you won't need to clutter up your Gemfiles.

Related

How to use gems locally?

I would like to use gems 'better_errors' and 'binding_of_caller' for my debugging in rails app, but i DON'T want to include those in Gemfile. Is it possible to do? My first thought was to simply
gem install better_errors
gem install binding_of_caller
but it doesnt work, i mean installation finishes without problems, but thats it, gem doesnt seem to work at all when i run my app on localhost. Do I need some kind of config set, anybody?
but i DON'T want to include those in Gemfile. Is it possible to do?
Yes, it is possible. You can just download the respective directories in desire folder (ex. lib) and add that gem class in your initializer so it will be loaded at the time of starting. Configuration varies as per gem.
My first thought was to simply .... but it doesnt work,
Ofcourse, it wont. How can your rails app magically detects without knowing it that you have better way to show error. It is simply saying like you have cancer formula and doctors automatically applied that formula to there patient without you telling them. There should be some commucaition between two parties rails-app and gem so they can coordinate and work better.
Do I need some kind of config set, anybody?
Yes, explained above.
i dont want to force those gems on my coworkers. KRUKUSA any more details? // said in comment
Yes, including this gems in your rails app can do this job. This extension will be available automatically to your worked. (no force applied :P)
it looks like all you want to not show those gems to other co-worker, if so, you can use this trick with git.
To achieve this thing, first simply add the gems in your gemfile, run bundle and then make it untrackable with git. You can put Gemfile and Gemfile.lock in your .gitignore file. or you can add the first add the gems and mark it ignore with below command. Read more here
git update-index --assume-unchanged Gemfile Gemfile.lock
Another possibility would be to create your own environment and use it accordingly.
Have your own configuration for myenv:
$ cp config/environments/{development,myenv}.rb
In config/database.yml, add the environment myenv and use the same config as development:
development: &development
<rest of the code you have on config/databases.yml>
...
myenv:
<< *development
In Gemfile add your custom gems to use on your mydev group:
group :myenv do
gem 'better_errors'
gem 'binder_of_caller'
end
Run rails and rake with RAILS_ENV like this: RAILS_ENV=myenv rails c
The advantage of this approach is that you still get the updates from Gemfile from the repo, and if you need to add a gem in the Gemfile for everybody to see, you still can.
Also, nobody will see the gems you installed inside the myenv group in your Gemfile.

Best options for deploying a Ruby standalone script and dependencies?

for a Ruby standalone script what Rails like deployment features such as Gemfile / "bundle install" etc
that is assuming you are developing a Ruby script that you want to test and then deployment, and perhaps ship to others, what Rails like deployment approach would you use for say:
a) GEM - marking GEM requirements & having them installed as required - e.g. Rails "Gemfile" where you mark what gems you need and then "bundle install" to install them
b) File Require - automatically loading *.rb files if they are in your script directory (I'm thinking of in Rails where if you put a class file in the apps/model directory Rails automatically load/require's the file for you)
depends on whether it's a tool you expect people to use on every host they find it on or not. also depends on whether the tool can be shared with pubic repos.
if it just has to work, without worrying about whether you've installed the gems via bundler already or not, you can use something like the following from within your standalone script to install gems if not already present (be mindful of system vs. user ruby):
#!/usr/bin/env ruby
require 'rubygems'
def install_gem(name, version=Gem::Requirement.default)
begin
gem name, version
rescue LoadError
print "ruby gem '#{name}' not found, " <<
"would you like to install it (y/N)? : "
answer = gets
if answer[0].downcase.include? "y"
Gem.install name, version
else
exit(1)
end
end
end
# any of the following will work...
install_gem 'activesupport'
install_gem 'activesupport', '= 4.2.5'
install_gem 'activesupport', '~> 4.2.5'
require 'active_support/all'
...
In my humble opinion, a gem is the way to go. Bundler makes it easy to get started; it starts a skeleton for you when you run the command…
bundle gem <GEM_NAME>
Take a look. As long as you specify your dependencies in your gem's .gemspec file, and somebody installs your packaged gem (they won't need bundler, just RubyGems' gem command), the dependencies will be installed as gems along with it.

Loading gem at runtime in Rails 3

I have Rails 3.0.x application. I would like to load gems on runtime, without using Gemfile.
What I would like to accomplish is to load my application as usual, with regular gems being loaded by Bundler. After that I would like to load all gems (Rails Engines) located in a specific directory (but until runtime, I don't know what gems will that be).
Does anybody know if this is possible in Rails, maybe using Bundler API?
What you're trying to do is dangerous. If each of your Rails Engines are also gems - then they would also have Gemfiles with other dependencies, and those would in turn have other dependencies, etc. If you allow Bundler to resolve those, then you would have lesser problems at runtime.
Here's how you would do it without any hacks. Remember that your Gemfile is just Ruby code, and you can have gems which are not loaded by default.
# In your Gemfile, add at the end:
Dir[YOUR_RAILS_ENGINES_SUBFOLDER + "/*/*.gemspec"].each do |gemspec_file|
dir_name = File.dirname(gemspec_file)
gem_name = File.basename(gemspec_file, File.extname(gemspec_file))
# sometimes "-" and "_" are used interchangeably in gems
# for e.g. gemspec_file is "engines/my-engine/my_engine.gemspec"
# dir_name will be engines/my-engine
# gem_name will be my_engine
# Register that engine as a dependency, *without* being required
gem gem_name, :path => dir_name, :require => false
# e.g. this is similar to saying
# gem 'my_engine', :path => 'engines/my-engine', :require => false
end
Now you have all your dynamic Rails engines registered as gem dependencies. Bundler will resolve them, and all their sub-dependencies, so you don't have to worry about anything. Just run bundle install once before running the application, or whenever you add/remove any engine in that folder.
The good thing is, these gems will just be registered, and not loaded. So in your production code, you can now load whatever gem that you choose at runtime simply by saying require <your-engine-name>
Edit: Extra code comments
Try this:
Bundler.with_clean_env do
# require gems...
end

Rails 3, Bundler, LoadError

I'm writing an app that will run scripts in a specified folder, and then record the numbers and graph them.
My problem is that if the script is a ruby file, the require statements fail inside the script because bundler seems to have done something funky with the load path.
Running rails runner Datasource.run_jobs fails:
class Datasource < ActiveRecord::Base
def self.run_jobs
require 'aws_sdb'
access_key_id = "REDACTED"
secret_key = "REDACTED" # In all actuality these woudln't be here.
#sdb = AwsSdb::Service.new(:access_key_id => access_key_id, :secret_access_key => secret_key)
Datasource.all.each do |ds|
puts "Updating #{ds.name}..."
unless #sdb.list_domains.include? ds.name
puts "#{ds.name} doesn't exist in SDB, creating a domain for it..."
#sdb.create_domain ds.name
end
#data = "TEST"
data = `#{RAILS_ROOT}/lib/plugins/#{ds.name}`
#sdb.put_attributes(ds.name, Time.now.to_i, data)
puts "Data Collected: #{data.inspect}"
end
end
has_many :graphs
end
(Basing this off your comments on the question)
You will need to add hpricot (and any other gem this needs) to your Gemfile so that they are made available by Bundler. Bundler is by far the easiest way to avoid gem conflicts and tomfoolery.
Imagine this situation: You somehow lose the gems that you have currently. Be this happening through a format or system change or any other reason. Whatever it is, you've lost your gems. How are you going to re-install all your gems? You could keep a list of them somewhere else yourself, but is this truly likely?
Bundler solves this problem by making you state what gems your application requires and only adding those gems to the load path, which is why you can't find hpricot. When you run bundle install the first time, this creates a Gemfile.lock which contains something like this:
GEM
remote: http://rubygems.org/
specs:
abstract (1.0.0)
actionmailer (3.0.0)
...
Because you commit this file to your source control "solution" of choice (be it Git, SVN, FTP, whatever, it's not important) you have a solid way of specifying the precise gems and precise versions of those gems that your application uses.
When/If your gems are wiped, you can simply clone your project again and run bundle install. Because the Gemfile.lock file exists, you'll have exactly the same gems you had originally, even if there were updates.
If you don't want the exact same gems, just run bundle update and this will ignore the specifications in Gemfile.lock and instead revert to depending on Gemfile to define them. This will check for new versions of gems and install them, updating the Gemfile.lock when it's done.
Honestly, I don't understand the Bundler hate. If you could explain in wider terms than "OMG IT SUCKS YEHUDA IS SATAN", I'd be much obliged.
Edit: WedTM asked for a sample Gemfile and related code:
In the Gemfile you'd have this:
group(:scripts) do
gem 'gem1'
end
To require these gems for your scripts:
require 'bundler'
Bundler.require(:scripts)
You may also wish to require the default gems too which you can do by just adding default anywhere to the arguments of require:
Bundler.require(:default, :scripts)
If this for some reason doesn't work I would imagine it would be because it can't locate the Gemfile. This can be fixed by setting the ENV['BUNDLE_GEMFILE'] to the path to the Gemfile.
I wonder if you might be able to use RVM to set up the ruby environment before running your scripts. Maybe something with a gemset like:
data = `rvm gemset use scripts; #{RAILS_ROOT}/lib/plugins/#{ds.name}`

Rails optional gem config

What do you do when you want to use a gem for development/testing that you don't want to force other devs to use? Right now I have
begin
require 'redgreen'
rescue LoadError
end
in test_helper.rb and no gem config, but that seems like a clumsy approach, albeit a functional one. I'd like to do something like the following:
config.gem "redgreen", :optional => true
Any other suggestions? Or should I just vendor those pretty superficial gems...?
EDIT
To be clear, I am only talking about those specific gems, like redgreen, which aren't actually used in the functional code, but only in the coding process. There is no need to vendor these at all, except to avoid the conditional require.
Gems that are specific to your development environment should be installed in your gemset or local gems, but not in the Gemfile.
A classic example is the ruby-debug-base19x which Rubymine needs for debugging. This is installed in your local gemset, but not in the Gemfile because not all coders use Rubymine.
[EDIT]
Indeed, everything is run in the context of the bundle, and outside gems are not reachable. There do exist some workarounds indeed. Most of them are dirty :)
I found a lot of good solutions in this bundler issue.
The nicest solution was to add this to your .irbrc :
# Add all gems in the global gemset to the $LOAD_PATH so they can be used even
# in places like 'rails console'.
if defined?(::Bundler)
global_gemset = ENV['GEM_PATH'].split(':').grep(/ruby.*#global/).first
if global_gemset
all_global_gem_paths = Dir.glob("#{global_gemset}/gems/*")
all_global_gem_paths.each do |p|
gem_path = "#{p}/lib"
$LOAD_PATH << gem_path
end
end
end
require 'irb/completion'
require 'rubygems'
require 'wirble'
Wirble.init
Wirble.colorize
If you then install wirble to the global gemset, it can then be found.
Original source: https://gist.github.com/794915
Hope this helps.
I answered a similar question of my own here
User-level bundler Gemfile
One way to do this is to create different environments:
group :scott do
end
Then
bundle --with-env=scott
Ok, I think I've come up with something. Basically, the idea is to only execute a secondary Gemfile when a Rails app is executing. To do this we add two things:
First, we alter the rails script a little:
# in ./script/rails
Kernel::IN_RAILS_APP = true
APP_PATH = File.expand_path('../../config/application', __FILE__)
require File.expand_path('../../config/boot', __FILE__)
require 'rails/commands'
Second, we tell bundler to pull in the secondary Gemfile if we're in a rails app and a secondary file exists:
# Gemfile
if Kernel.const_defined?(:IN_RAILS_APP)
local_gemfile = File.dirname(__FILE__) + "/Gemfile.local"
if File.exists?(local_gemfile)
puts 'using local gemfile'
self.instance_eval(Bundler.read_file(local_gemfile))
end
end
Now you can add a Gemfile.local to your project and run specific gems on a per-machine basis. bundle install works normally since the IN_RAILS_APP constant doesn't exist.
** Make sure to add Gemfile.local to your .gitignore.
In my opinions this is what environments are for. Fortunately there is also a way provided to do it with what is in your Gemfile, this is also how rails use it: groups
Pretty much use the environments the same way rails use it. Here is what you could find in your Gemfile:
group :test do
# Pretty printed test output
gem 'turn', :require => false
end
And here is what you can find in your config/application.rb
Bundler.require(:default, Rails.env) if defined?(Bundler)
All you would need to do is to change your local environment settings and the others working with you won't be affected unless they decide to. Everything gets committed and nothing gets lost.
Here some links :
http://yehudakatz.com/2010/05/09/the-how-and-why-of-bundler-groups/
http://gembundler.com/groups.html
If you want it to be optional, it's better to freeze the gem as a plugin. However, it's not a good idea to use different gems than the rest of a development team, as it creates some inconsistencies in the codebase that can be hard to track down later. I would say add it to config.gem, and just tell the other developers to do:
rake gems:install
And you're done.
This is how I tackled the same problem under Rails 3.1. In my Gemfile:
if File.exists? './tmp/eric_dev_gems'
gem 'redgreen'
gem 'awesome_print'
gem 'wirble'
gem 'wirb'
gem 'hirb'
end
Create a file in ./tmp/ (or in some folder which is in your .gitignore) of your choosing. I used eric_dev_gems. This should be ignored by git, and will only exist on your system unless one of your teammates decides he wants to create that file too.
I solved it by putting this in my gem file:
$gem_names ||= ENV['GEM_PATH'].split(':').map{|g| Dir.glob("#{g}/gems/*").map{|p|p.split('/gems/').last}}.flatten
gem 'redgreen' if $gem_names.any?{|n| n=~/redgreen/ }
That way the gem will only be used if you manually installed it on your system.
This works well but has the downside that it puts the gem name in the Gemfile.lock. This is of little consequence because the gem does not get installed with bundle install but it does make your lock file a bit messy and can cause the lock file to change a bit from one developer to the next.
If that is an issue for you another option is to keep the gemfile clean and require the gem by its full path, or you can add the path for just that gem. Like this:
$gem_paths ||= ENV['GEM_PATH'].split(':').map{|g| Dir.glob("#{g}/gems/*")}.flatten
$gem_paths.grep(/redgreen/).each {|p|$LOAD_PATH << p+'/lib'}
require 'redgreen' rescue nil

Resources