I do bundle install --without development test before my RoR application working in production because I want to remove gems only used in development or test, but the other day, this caused the problem.
I wrote some code and it works in development, but it contained module provided by the gem, which was installed as development gem's dependence. I used unintentionally so I cannot noticed by deployment failed. So I want to detect it. I'm using CI, so maybe I can notice if I do same bundle install as in production and something rails kicked, but if I do so, CI will take a long time so I don't really want to. I'd love to hear what you think.
edit: I think you haven't got my point yet, so let me explain it again.
for example, my Gemfile is like below;
ruby '2.5.7'
gem 'rails', '5.2.2.1'
gem 'pg', '>= 0.18', '< 2.0'
gem 'puma', '~> 3.11'
...
group :development, :test do
gem 'overcommit'
gem 'rails_best_practices'
gem 'rubocop' # <- this gem also install unicode_display_width(which has `Unicode::DisplayWidth`) as dependency
end
group :development do
gem 'brakeman'
gem 'debase'
gem 'rack-mini-profiler', require: false
gem 'ridgepole'
gem 'ruby-debug-ide', '0.6.0'
...
end
group :test do
gem 'simplecov'
...
end
and I used Unicode::DisplayWidth in my application because I totally thought it was the library ruby originally has(like csv). I don't want to do something like this again, but I may do carelessly, so I want to detect it.
It's not really clear what your problem is from your description. But have a look at bundle list to view installed gems https://bundler.io/man/bundle-list.1.html
Also, try bundle config and if you see any without's that you were not expecting, you can run bundle config --delete without to remove them.
I'm going to try and paraphrase what I think you're asking based on your updated question:
You want your CI pipeline to detect if you used a library in your code that wasn't made available via bundler*, but you don't want to slow your CI pipeline down with another bundle install command.
If that's the case, we have 3 separate pipelines/processes that we use:
CI/CD pipeline on Semaphore to run our automated tests. This pipeline won't catch the type of error described above
Separate pipeline using Heroku's review app feature which builds a "live" application using dummy/seed data, but production-like settings (e.g. bundle install --without development text). This may catch the type of error described above; you may have to use the review app to trigger the error and sometimes we don't
We also have a separate staging environment where user testing occurs, which is also production-like. This is where that type of error should definitely be identified (because we have users test features here by using the site)
This has been a common strategy at a number of projects I've worked on for catching this kind of error before it deploys to production.
* Because bundler in production is run bundle --without development test
Related
Why does my spring gem load in the wrong (or all) environment(s)?
I have this in my Gemfile and spring gem is not listed anywhere else in the file:
group :development do
gem 'listen', '~> 3.1.5'
# Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
gem 'spring'
gem 'spring-watcher-listen', '~> 2.0.0'
end
When I ran bundle exec rails console test (for the test environment), spring processes started and the Listen module was loaded in the rails console. I made sure all spring processes were stopped beforehand.
To do a sanity check, I removed the whole development group above and bundled. Spring and listen gems were no longer loaded, as I expected.
I faced this misunderstanding in production.
Here's how I solved it:
You can also fix this issue permanently by upspringing (removing the spring gem from) your bin/ executables:
bin/spring binstub --remove --all
Or
spring binstub --remove --all
You can now run the command below to get into rails console in production
rails c --environment=production
Also, to avoid this from occurring in subsequent occasions endeavour to make sure that the spring gem is only present in development and test groups in your Gemfile.
Moreso, when you're in production make sure you always provide the --without development test argument to the bundle install command
bundle install --without development test
and not the usual or common
bundle install
Please note: As an indication, whenever you run the command rails c or rails console and you see the output below:
Running via Spring preloader in process 26651
WARNING: Spring is running in production. To fix this make sure the spring gem is only present in development and test groups in your Gemfile and make sure you always use bundle install --without development test in production
It's an indication that the spring gem is running in your production environment, and it should be stopped or removed entirely from your bin executables.
That's all.
I hope this helps
Spring is generally used through binstubs - did you install the binstubs? If so this is the file your rails command is running through.
#!/usr/bin/env ruby
begin
load File.expand_path('../spring', __FILE__)
rescue LoadError => e
raise unless e.message.include?('spring')
end
APP_PATH = File.expand_path('../config/application', __dir__)
require_relative '../config/boot'
require 'rails/commands'
As you can see it will load spring anytime you use the rails command. There is no check for the environment. If you do not want to load spring you can use DISABLE_SPRING=1 rails c test.
According the spring gem github page, it looks like rails console will load up spring:
rails console, rails generate, rails runner
These execute the rails command you already know and love. If you run
a different sub command (e.g. rails server) then spring will
automatically pass it through to the underlying rails executable
(without the speed-up).
Also, this is worrying:
You must not install Spring on your production environment. To
prevent it from being installed, provide the --without development
test argument to the bundle install command
rails console (development) would make sense, but not rails console test (or another environment). It seems buggy to me, and a reason now why I don't like the gem.
Sometimes you make a gem that is specific to a project. This helps abstract and pull some of the "responsibility" out of the main Rails app and into a more modular place.
The gem would be located here on you application:
gem 'example_gem', path: './example_gem'
You bundle and everything is OK. Now, you git init the gem and store it in its own repo on github. You try doing this to keep it developer friendly:
group :development, :test do
gem 'example_gem', path: './example_gem'
end
group :production do
gem 'example_gem', github: 'company/example_gem'
end
You pat yourself on the back for increasing your workflow, but after running bundle you get:
Your Gemfile lists the gem example_gem (>= 0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.
You cannot specify the same gem twice coming from different sources.
You specified that example_gem (>= 0) should come from source at ./example_gem and git://github.com/company/example_gem.git
The workflow here is to be able to edit the gem in development and when you're done, commit those changes and push them to Github. BUT, when on development, you don't want to have to do a git commit, git push, and a bundle update on your main app just to see a small change.
Does anyone know of a better way to solve this issue?
Yes, there is a better way: First, have the gem as a git gem in all environments
gem :example_gem, :git => 'git#github.com:foo/example_gem', :branch => :master #you need to set a branch
Then in your app's folder run
bundle config --local local.example_gem /path/to/gem
This will edit .bundle/config to set this option (make sure this file isn't checked into source control!) and tells bundler to get the gem from that path.
At the point when you are going to push your commits you have to be a little bit careful: if your app depends on not yet committed changes to the gem then clearly things will break. In addition as you commit to the repository for the example gem the rails app's Gemfile.lock will get updated. If you push a Gemfile.lock that references a commit that only exists in your copy of the example_gem repo then other users will be stuck.
this works for me.
foo = "https://github.com/me/my-development-version-gem.git"
group :production do
foo = "https://github.com/me/my-production-version-gem.git"
end
gem "foo", :git => foo
in my case i need different branchs for envs to engine
Still diving into the interesting (but a bit confusing) world of developing my own gem/engine.
I have successfully deployed it to my (private) GitHub account, but as the gem is in continuous development, I don't want to deploy it again and again and then do bundle update, etc., so I found the option to require a gem by specifying a path like this:
gem 'my_gem', path: 'path/to/my/gem'
This works nicely when developing (it even seems to reload changed files from the gem when rails s is running [at least for helpers and views it seems to do it, didn't use controllers and models yet, but hoping this will be the case for them, too], which is niiiice for developing the engine while using it straight away in the host Rails app).
But when deploying to the production server, I'd like to get the gem from github:
gem 'my_gem', git: "git#github.com:my_company/my_gem"
Sadly, bundler doesn't allow to have a gem required more than once (even when they are in separate development/test and production groups), so I'm doing the following:
if File.directory?('../my_gem')
gem 'my_gem', path: '../my_gem'
else
gem 'my_gem', git: "git#github.com:my_company/my_gem.git"
end
This seems to work nicely so far (didn't test deployment using Capistrano yet, though). But it leaves some questions:
1) Is this the "right" way to solve my problem? Seems a bit odd that I didn't find a much simpler way yet, something like:
gem 'my_gem', path: '../my_gem', git: "git#github.com:my_company/my_gem.git"
...where I'd expect bundler to simply try the :path option first, and if it's not available, it's try the :git option. But as far as I see, when doing this, it only uses the :git option.
2) Is it good practice to develop gems like this? I'm extracting some functionality from my host app into an engine, and at the same time, I'm enhancing this functionality. So I have the engine gem loaded within my host app, and from within my host app I change the engine's codes which are (thank God!) reflected within my host app without having to restart the rails server. This feels quite "right", but as there is a dummy Rails app within the created engine, I'm a bit unsure whether I rather should develop my engine's strictly isolated from my host app, or not. I think this is a bit of a philosophical question, but maybe someone has an opinion about it and can post pros and contras?
3) How do I tell Capistrano to always update my own gem to the newest version from GitHub automatically?
You can do
if ENV['RAILS_ENV'] != "development"
gem 'my_gem', git: "git#github.com:my_company/my_gem.git"
else
gem 'my_gem', path: '../my_gem'
end
assuming that RAILS_ENV is being passed into the bundle command (which is so when using capistrano).
To update your gem to the latest, try making a capistrano task that essentially does this
/path/to/bundler update my_gem
I can't really comment on your other points. Just my 2 cents.
I am developing on an open source rails application and as a rails 3 project, it includes a Gemfile. The application is also dependent on a database adapter for active record. I can specify a specific adapter (like gem 'sqlite3') within the Gemfile but this is not what I want to do. I advertise the app as beeing compatible with all the adapters active record is compatible with. Is there any way to specify that the app depends on a single database adapter (out of a selection) but that one is sufficient? Something like
any_of do
gem 'sqlite'
gem 'mysql'
etc.
end
Thanks for any help or suggestions how to approach the problem differently.
I just recently installed something that generated an error, until I realized how they handled this.
group :postgresql do
gem 'pg'
end
group :mysql do
gem 'mysql2'
end
and then when you run bundler:
bundle install --without postgresql
In the rails project I'm working on I inserted support for rspec, cucumber and autotest with this Gemfile (partial)
gem 'rspec-rails'
gem 'cucumber-rails'
gem 'autotest-standalone'
gem 'autotest-rails-pure'
gem 'zentest-without-autotest'
however in order to run tests with autotest i need to execute bundle exec autotest otherwise it fails with this message
$ autotest
loading autotest/cucumber_rails_rspec_rspec2
Error loading Autotest style autotest/cucumber_rails_rspec_rspec2 (no such file to load -- autotest/cucumber_rails_rspec_rspec2). Aborting.
Now I'm developing on a Mac and I'd like to enable autotest-growl and autotest-fsevents gem, but if I insert those lines in my ~/.autotest
require 'autotest/growl'
require 'autotest/fsevent'
then I need to insert the corresponding gems in the Gemfile and everything works, but it breaks builds on my CI server (which is on Linux)
How to solve this without maintaining a different Gemfile for local and CI environments?
EDIT:
For the moment I solved with these lines in Gemfile
if RUBY_PLATFORM.downcase.include?("darwin") # I'm on Mac
gem 'autotest-fsevent'
gem 'autotest-growl'
end
It works both locally and on the CI server, I don't know if it mess something, for the moment it seems to work flawlessly.
Any cleaner way to do that is still welcome.
EDIT2:
I switched to groups solutions. While the previous monkeypatch works pretty well both in development and for continuous integration, it will gives you an error in production if you use capistrano bundler tasks for deployments or if you use bundle install --deployment option (which is advised in production)
When using the if RUBY_PLATFORM.downcase.include?("darwin") line you'll get this error on deploy.
# bundle install --deployment --without development test
You are trying to install in deployment mode after changing
your Gemfile. Run `bundle install` elsewhere and add the
updated Gemfile.lock to version control.
You have deleted from the Gemfile:
* autotest-fsevent
* autotest-growl
So my final solution to this problem is to include platform specific gems in a given group, say osx, and then in production and on CI server exclude it using bundle.
If you use capistrano to deploy put this in your config.rb
set :bundle_without, [:development, :test, :osx]
# capistrano bundler task
require "bundler/capistrano"
You might want to use groups in your gemfile, something like:
group :development do
gem "autotest-growl"
gem "autotest-fsevents"
end
and on the server you use: $ bundle install --without development
You can handle this by taking advantage of the different Gemfile environments (testing, development, production).
Your local box can be development while the CI server is your "production" environment.
With this in mind you can edit your Gemfile to use the appropriate gems depending on the environment.
Edit: Sorry, I think I scanned your post too quickly. But you can add your ~/.autotest to .gitignore so it wont be included on your CI server.