I am using bundler on my gemfile for executing the application, and found it convenient to use bundler-only to be able to pick only a subset of gems that are needed for deployments only.
So on my deployment machine, I use bundle-only that will only install gems namespaced under the deploy group.
However, because of this I need to duplicate some gem (eg. my deployments need to send a notification to slack, so I have some notifier gems both in my global namespace and my :deploy-only namespace.
This cause several warnings
Your Gemfile lists the gem slack-notifier (>= 0) more than once. You
should probably keep only one of them. While it's not a problem now,
it could cause errors if you change the version of one of them later
Is there a way to suppress the warnings ? (if possible those gems only)
Don't list gems more than once. That warning is there for good reason.
You can group gems under multiple namespaces at once within a Gemfile, like so:
group :deploy, :somethingelse do
gem 'slack-notifier'
end
group :deploy do
# Deploy-ONLY gems
end
group :somethingelse
# Somethingelse-ONLY gems
end
Or if you prefer, you can do this grouping inline:
gem 'slack-notifier', group: [:deploy, :somethingelse]
For more information, read the bundler documentation on Gemfile groups.
One alternative is to maintain a list of all groups, and systematically include , groups: groups
# Gemfile
groups = [:deploy, :x, :y, :z, ...] # Maintain this list as you add groups
# Gems needed except in deploy
gem :a
gem :b
...
# Gems that are also required for deploy
gem :d1, groups: groups
gem :d2, groups: groups
# Gems that are required ONLY in deploy
group :deploy do
gem :dep_only1
gem :dep_only2
end
So the following works without warning
bundle --without deploy # Will ignore deploy group
bundle-only deploy # Will install only deploy gems including those that are also needed by the app
Related
I do bundle install --without development test before my RoR application working in production because I want to remove gems only used in development or test, but the other day, this caused the problem.
I wrote some code and it works in development, but it contained module provided by the gem, which was installed as development gem's dependence. I used unintentionally so I cannot noticed by deployment failed. So I want to detect it. I'm using CI, so maybe I can notice if I do same bundle install as in production and something rails kicked, but if I do so, CI will take a long time so I don't really want to. I'd love to hear what you think.
edit: I think you haven't got my point yet, so let me explain it again.
for example, my Gemfile is like below;
ruby '2.5.7'
gem 'rails', '5.2.2.1'
gem 'pg', '>= 0.18', '< 2.0'
gem 'puma', '~> 3.11'
...
group :development, :test do
gem 'overcommit'
gem 'rails_best_practices'
gem 'rubocop' # <- this gem also install unicode_display_width(which has `Unicode::DisplayWidth`) as dependency
end
group :development do
gem 'brakeman'
gem 'debase'
gem 'rack-mini-profiler', require: false
gem 'ridgepole'
gem 'ruby-debug-ide', '0.6.0'
...
end
group :test do
gem 'simplecov'
...
end
and I used Unicode::DisplayWidth in my application because I totally thought it was the library ruby originally has(like csv). I don't want to do something like this again, but I may do carelessly, so I want to detect it.
It's not really clear what your problem is from your description. But have a look at bundle list to view installed gems https://bundler.io/man/bundle-list.1.html
Also, try bundle config and if you see any without's that you were not expecting, you can run bundle config --delete without to remove them.
I'm going to try and paraphrase what I think you're asking based on your updated question:
You want your CI pipeline to detect if you used a library in your code that wasn't made available via bundler*, but you don't want to slow your CI pipeline down with another bundle install command.
If that's the case, we have 3 separate pipelines/processes that we use:
CI/CD pipeline on Semaphore to run our automated tests. This pipeline won't catch the type of error described above
Separate pipeline using Heroku's review app feature which builds a "live" application using dummy/seed data, but production-like settings (e.g. bundle install --without development text). This may catch the type of error described above; you may have to use the review app to trigger the error and sometimes we don't
We also have a separate staging environment where user testing occurs, which is also production-like. This is where that type of error should definitely be identified (because we have users test features here by using the site)
This has been a common strategy at a number of projects I've worked on for catching this kind of error before it deploys to production.
* Because bundler in production is run bundle --without development test
By some reason some of the developers use a gem and the others don't.
Sometimes I met code like that:
if File.exist?('USE_MY_GEM')
gem 'my_gem'
end
and USE_MY_GEM written in .gitignore. But I believe this is not very good practice.
How to turn a gem off / on for a specific machine?
I would go with Bundler groups feature. This way the Gemfile might contain the group, containing these superfluous gem[s] some developers use.
From the documentation:
Restrict the groups of gems that you want to add to the load path. Only gems in these groups will be require'able:
require 'rubygems'
require 'bundler'
Bundler.setup(:default, :ci)
require 'nokogiri'
Well, the solution was obvious.
We can manage gems in a way we usually manage groups.
Just wrap gem in a group and add --without option:
Gemfile
group :my_gem do
gem 'my_gem'
end
In console:
$ bundle install --without my_gem
Ensure that .gitignore contains:
/.bundle
Sometimes you make a gem that is specific to a project. This helps abstract and pull some of the "responsibility" out of the main Rails app and into a more modular place.
The gem would be located here on you application:
gem 'example_gem', path: './example_gem'
You bundle and everything is OK. Now, you git init the gem and store it in its own repo on github. You try doing this to keep it developer friendly:
group :development, :test do
gem 'example_gem', path: './example_gem'
end
group :production do
gem 'example_gem', github: 'company/example_gem'
end
You pat yourself on the back for increasing your workflow, but after running bundle you get:
Your Gemfile lists the gem example_gem (>= 0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.
You cannot specify the same gem twice coming from different sources.
You specified that example_gem (>= 0) should come from source at ./example_gem and git://github.com/company/example_gem.git
The workflow here is to be able to edit the gem in development and when you're done, commit those changes and push them to Github. BUT, when on development, you don't want to have to do a git commit, git push, and a bundle update on your main app just to see a small change.
Does anyone know of a better way to solve this issue?
Yes, there is a better way: First, have the gem as a git gem in all environments
gem :example_gem, :git => 'git#github.com:foo/example_gem', :branch => :master #you need to set a branch
Then in your app's folder run
bundle config --local local.example_gem /path/to/gem
This will edit .bundle/config to set this option (make sure this file isn't checked into source control!) and tells bundler to get the gem from that path.
At the point when you are going to push your commits you have to be a little bit careful: if your app depends on not yet committed changes to the gem then clearly things will break. In addition as you commit to the repository for the example gem the rails app's Gemfile.lock will get updated. If you push a Gemfile.lock that references a commit that only exists in your copy of the example_gem repo then other users will be stuck.
this works for me.
foo = "https://github.com/me/my-development-version-gem.git"
group :production do
foo = "https://github.com/me/my-production-version-gem.git"
end
gem "foo", :git => foo
in my case i need different branchs for envs to engine
I have Rails 3.0.x application. I would like to load gems on runtime, without using Gemfile.
What I would like to accomplish is to load my application as usual, with regular gems being loaded by Bundler. After that I would like to load all gems (Rails Engines) located in a specific directory (but until runtime, I don't know what gems will that be).
Does anybody know if this is possible in Rails, maybe using Bundler API?
What you're trying to do is dangerous. If each of your Rails Engines are also gems - then they would also have Gemfiles with other dependencies, and those would in turn have other dependencies, etc. If you allow Bundler to resolve those, then you would have lesser problems at runtime.
Here's how you would do it without any hacks. Remember that your Gemfile is just Ruby code, and you can have gems which are not loaded by default.
# In your Gemfile, add at the end:
Dir[YOUR_RAILS_ENGINES_SUBFOLDER + "/*/*.gemspec"].each do |gemspec_file|
dir_name = File.dirname(gemspec_file)
gem_name = File.basename(gemspec_file, File.extname(gemspec_file))
# sometimes "-" and "_" are used interchangeably in gems
# for e.g. gemspec_file is "engines/my-engine/my_engine.gemspec"
# dir_name will be engines/my-engine
# gem_name will be my_engine
# Register that engine as a dependency, *without* being required
gem gem_name, :path => dir_name, :require => false
# e.g. this is similar to saying
# gem 'my_engine', :path => 'engines/my-engine', :require => false
end
Now you have all your dynamic Rails engines registered as gem dependencies. Bundler will resolve them, and all their sub-dependencies, so you don't have to worry about anything. Just run bundle install once before running the application, or whenever you add/remove any engine in that folder.
The good thing is, these gems will just be registered, and not loaded. So in your production code, you can now load whatever gem that you choose at runtime simply by saying require <your-engine-name>
Edit: Extra code comments
Try this:
Bundler.with_clean_env do
# require gems...
end
In the Gemfile of my Rails project, I am starting to have auxiliary gems like "ruby-debug19", "perftools.rb", or "irbtools". All of these really have nothing to do with the project, but rather are part of my local development setup. But since I'm using bundler, I cannot load these gems (even though they are installed system-wide) unless I add them to the Gemfile. In my view that is a bit of a code smell.
For example, I would like to be able to require 'irbtools' in rails console without adding "irbtools" to my Gemfile.
Is there a way to keep auxiliary gems out of the Gemfile and still be able to load them for debugging, profiling, etc. when I need them?
Actually, you can create a group in you Gemfile like:
group :auxiliary do
gem 'irbtools'
end
And then use bundle install --without auxiliary if you don't want to use irbtools. Why do you think adding them to Gemfile is a code smell? And if it possible to do this without adding gems to the Gemfile it will be many more code smell I think.
Thanks to this post I have a great solution.
Add this line at the end of your Gemfile:
eval(File.read(File.dirname(__FILE__) + '/Gemfile.local'), binding) rescue nil
Create a file called Gemfile.local.
Add your development gems to Gemfile local. For example:
group :development do
gem 'cucumber'
end
Add Gemfile.local to .gitignore.
Now you can add your auxiliary development gems without changing the Gemfile for other folks on the team. Very cool.
I put the code below in a file in my app root, so it's easy to load from irb.
If you want it in something like a rails server, you probably need to add the load statement to environments/development.rb etc. That still creates problems if you accidentally check that in, but it's less annoying than having to add it to the Gemfile and causing your Gemfile.lock to change also.
# Example usage:
# add_more_gems("ruby-debug-base19-0.11.26", "linecache19-0.5.13")
# or
# add_more_gems(%w(ruby-debug-base19-0.11.26 linecache19-0.5.13))
#
# Note that you are responsible for:
# - adding all gem dependencies manually
# - making sure manually-added gem versions don't conflict with Gemfile.lock
# To list deps, run e.g. "gem dep ruby-debug-base19 -v 0.11.26"
#
def add_more_gems(*gem_names_and_vers)
gem_names_and_vers.flatten!
gem_base = File.expand_path(Gem.dir)
gem_names_and_vers.each do |gem_name_and_ver|
# uncomment if desired
###puts "Adding lib paths for #{gem_name_and_ver.inspect}"
spec_file = File.join(gem_base, 'specifications', "#{gem_name_and_ver}.gemspec")
spec = Gem::Specification.load spec_file
this_gem_dir = File.join(gem_base, 'gems', gem_name_and_ver)
spec.require_paths.each {|path|
dir_to_add = File.join(this_gem_dir, path)
$: << dir_to_add unless $:.member?(dir_to_add)
}
end
end
# put your often-used gems here
add_more_gems(
%w(
ruby-debug-base19-0.11.26
ruby-debug-ide19-0.4.12
linecache19-0.5.13
)
)
Not sure if this would work for you. It depends on whether or not you're using RVM. If you are, then you could install those auxiliary gems into the #global gemset that is created automatically for every Ruby interpreter. The gems in the #global gemset are available to all project-specific gemsets by default. This way you won't need to clutter up your Gemfiles.