After a long overdue reading of the bunder documents, I pretty much understand why it's a great thing compared to manual gem install.
But this came to me (and others) only recently, and now we have a hybrid environment. Our development machines are mostly fine. But when we deploy to production, Bundler tells capistrano to put the gems in shared/bundle/gems, which makes sense. But we still have manually installed gems in the normal .rvm location.
So now we know not to use "gem install" on production, but we're dealing with version mismatches when we run Rake tasks directly. We can use "bundle exec rake foo:bar" to force the correct behavior, I think. But for now we have a bunch of outdated gems:
Outdated gems installed by Bundler, and
Gems we manually installed using gem install
I would like to get things pristine. Is there any reason I shouldn't use gem cleanup?
Hope this isn't too convoluted :-)
Thanks in advance.
gem cleanup will keep the latest version of all gems, and remove older versions. What I think you want is this:
(Optional) create an RVM gemset for your project. (I like to do this; some people rely entirely on Bundler, which also works.)
Make sure all your top-level dependencies are specified in your Gemfile.
Remove all gems (rvm gemset empty may be helpful) except Bundler.
bundle install
Run everything with bundle exec from now on.
Related
I am new to Ruby, Rails and gem management. One thing what I have noticed is that whenever I run bundle install (even within the project directory) it seems to install gems which affects other gem based Rails projects too.
For instance, assume that after running bundle install within one gem based Rail project (which installs a bunch of gems from the gemfile) I run into gem incompatibility issues. Now, these issues will manifest in all the other gem based projects too and will present themselves everytime I attempt to rackup
My questions are:
1) Is there a way to localize this damage (gem version incompatibility) to the current project and not have other projects affected?
2) Is there a good way to obtain a compatible set of gems or is the only way to look at the gem dependencies on a gem-by-gem basis (look at the tree) and figure out the compatible ones? I seem to be wasting a lot of time on this and if I fix one something else seems to break.
Please let me know what I'm missing here or point me to resources.
Thanks
It sounds like you're missing use of bundle exec
Description
This command executes the command, making all gems specified in the Gemfile(5) available to require in Ruby programs.
Essentially, if you would normally have run something like rspec spec/my_spec.rb, and you want to use the gems specified in the Gemfile(5) and installed via bundle install(1), you should run bundle exec rspec spec/my_spec.rb.
bundle install will install gems to $GEM_HOME which is shared by any apps using the same ruby and can result in multiple versions of the same gem being installed (this is expected and normal). If you don't then also use bundle exec to load only the gem versions specified in your Gemfile you can get incompatibility errors or unexpected behavior as ruby doesn't know which version of a particular gem to require.
RVM, rbenv, and other ruby version managers are useful tools for both isolating gems per project (which isn't really necessary when also using bundler) and for allowing multiple ruby versions to be installed on the same machine (which is not handled by bundler and can be very useful when developing multiple apps with different ruby version requirements).
I use RVM to manage Ruby versions.
In my project I use Bundler to manage gems for the project.
RVM also have gemsets.
Gem in gemset don't have a connection with Bundler's gem. ← Is this correct?
I came to this conclusion because gem files stored in different locations:
RVM gemset: ~/.rvm/gems/ruby-2.0.0-p247#myApp
Bundler: [my_app_dir]/vendor/bundle/gems
So app uses Bundler gems, not RVM gemset gems.
But when I add gem to my Gemfile, RubyMine IDE shows me warning, that this gem is not in RVM gemset. So I add this gem to RVM gemset also (just to get rid of this warning).
So the questions are:
Is there any good reason to add gems in both places (RVM Gemset and Gemfile)?
If no, then why RubyMine warning me about this?
Is there any good reason to add gems in both places (RVM Gemset and Gemfile)?
The gemset is incidental, the Gemfile is absolutely the place to declare your dependencies. Where you store those gems is up to you.
It sounds like Bundler is configured to store them in a project-local path, but you're expecting them to be in a gemset. Bundler got that configuration by running bundle install --path vendor/bundle/gems at some point. It stores that configuration in its project configuration file at project_dir/.bundle/config:
BUNDLE_PATH: vendor/bundle/gems
I'm unfamiliar with Rubymine, but if you run the Rails server using Bundler (i.e. bundle exec rails server) you can ignore that warning. Bundler will correctly load the gems listed in the Gemfile.
If you want to use a gemset instead of the Bundler cache, you can just remove that line from the Bundler configuration file and reinstall your gems with bundle install.
If no, then why RubyMine warning me about this?
My guess is that Rubymine is not reading the Bundler project configuration (in project_path/.bundle/config) and does not understand where the gems are installed.
You (or if you are working in a team, somebody of your team) has once done a bundle install and specified a installation-folder. In your case vendor/bundle/gems. Bundle remembers this setting and all next invocations of the bundle command will use the same path.
There is a good reason to do it that way: your application-folder will contain all requirements and will be easier to redistribute (for instance).
Now if you want that bundle installs your gems in the normal locations, you can do the following:
run bundle install --system which will use the default location
alternatively: bundle stores it settings in a config file, I think .bundle/config and you can
check that one as well. Normally it is not needed, since bundle install --system will set that
correctly again.
then you can safely remove the vendor/bundle/gems folder
No, something's wrong, you shouldn't have anything under vendor/bundle, it should all be under ~/.rvm/gems/ruby-2.0.0-p247#myApp and perhaps ~/.rvm/gems/ruby-2.0.0-p247#global assuming your .rvmrc (or.ruby-version) is setup correctly.
What does "gem env" look like? Also "bundle env"?
So this just took me 3 days, since nothing else I was finding here was helping. I also run multiple projects through RubyMine at the same time (and different versions) so setting my GEM_PATH and launching from command-line doesn't work for me. I use IntelliJ with RM plugin, this should work on RM standalone.
Bundler seems to install custom gems, or gems from custom repos, in a different directory than gems from rubygems, or github.
/Users/YOURUSER/.rvm/environments/ruby-{version}\#yourgemset/bundler/gems
One thing I wasn't able to fix is in the GEMFILE, I have some custom git_sources, and rubymine highlights those and gives me the warning that it cannot find the gem in my bundle (you can ignore this warning; unless the gem doesn't install at all):
gem 'somegem', custom_git:'gituser/repo'
is highlighted and warning is "Gem x cannot be found... in SDK'
However Bundler installed it, and ruby is able to load it.
# TLDR: Steps to have RUBYMine find extra gems, and show up in external libs
vim ~/.rvm/environments/ruby-{your-verion-here}\#{your-gemset}
add the bundler gems path to GEM_PATH entry
export GEM_PATH='/Users/YOURUSER/.rvm/gems/ruby-{version}#yourgemset/bundler/gems:{the rest}'
save the file
Restart RubyMine/IntelliJ, reopen your project (if not open automatically)
Open the Project Structure dialog > Platform SDKs > Choose the GEMSET you're working with
add /Users/YOURUSER/.rvm/environments/ruby-{version}\#yourgemset/bundler/gems to your classpath
Hit OK, then REOPEN the Project Structure Dialog > Project Settings > Project
Your project will likely have no SDK So select the one you're using again and hit OKAY
RM/IJ will now reindex files
You're done, any broken/missing inspection links should now be fixed. And you should be able to introspect into your gems.
In the above instructions that when you run bundle install (from terminal or RM) it works successfully, and that you have RVM correctly setup, and gemset already created
I hope this helps! Let me know if I should clarify anything (happy NYE)
We would test this, but don't want to risk ruining our dev environment if this isn't supposed to happen.
Is it okay to delete Gemfile.lock?
We're on Rails 3.0.6.
You're probably not going to ruin your dev environment. However, you might end up with newer versions of gems than you had before. It depends on how you have defined them in Gemfile.
If you're using entries like:
gem "rails"
Then you'll get the latest rails gem, whatever that might be.
If you're using entries like:
gem "rails", "3.2.11"
Then you'll get 3.2.11 again.
Having said all of that, this is what branches are for. Make a branch in git, hg, or whatever you're using, blow away Gemfile.lock, run bundle install, and then check your test suite. If it's horrible, then you can abandon the branch while you figure out what went wrong.
Another tip: Any time I've ever wanted to do this, I found that it was useful to clear out all of my installed gems as well. If you're using rvm with gemsets this is as simple as running
rvm gemset empty [gemset_name]
It's ok to delete Gemfile.lock, just run
bundle install
to generate a new Gemfile.lock. Take note that if you didn't specify any version of a gem on your Gemfile, you will always get the latest
I know this has been answered already, but for everyone else that happens to come across this post on Google, you should know that command bundle init will regenerate the Gemfile.
What is the difference between installing a gem from the command line
sudo gem install gem-name
and writing your gem into the Gemfile and running bundle install?
I think the problem is that I don't understand the exact purpose of the Gemfile. So far it seems like it is a place to list all of the gems that your app is dependent on.
Installing a gem via:
sudo gem install gem-name
is going to install that gem system wide.
Whereas installing them via the Gemfile is specific for your rails app(to keep track of dependencies, version, app portability etc).
The best source of the whats and whys about Bundler, is probably this page:
http://gembundler.com/rationale.html
That page has great examples and explanation about why Bundler is useful and in some cases, necessary.
I always thought you write all gems that your app is dependent on in it, and then if you want to port your application somewhere else, you can run the bundle install and it'll grab the gems you need for you so you don't manually have to do it.
This might clear things up, I quote:
'It holds information about all the project dependencies so that you don't need to struggle to figure out what gems you need to install.'
http://blog.despo.me/42762318
I want to freeze a specific gem into my Rails application.
In rails 2 there was this command:
rake gems:unpack
I can't find that command in Rails 3.
So, the short answer is, you don't.
When you modify your Gemfile, and then run bundle install or bundle update, bundler handles the dependency resolution for you and determines the best (newest) versions of each gem you've required that satisfies the entire dependency chain (you won't get a new version that breaks another gem in the dependency list, etc.). You can also of course place a specific version, or a '>= 1.2.3' specification or whathaveyou in the Gemfile using the familiar syntax from the config.gem days, and bundler will make sure to satisfy that as well (or won't produce a Gemfile.lock if there is no valid resolution).
When Bundler does its business, it creates the Gemfile.lock file, which (and this is provided you use bundler alone for managing your gem on all workstations/environments/deployments) performs the same function as freezing all of the gems you've required. For free! (Check this file into version control!) If your new development intern pulls down your source on a fresh machine, it takes one bundle install and the exact same versions of the gems that you have installed are on her machine. Push to deployment, and do a bundle install --deployment there (or more likely, throw it in your Capfile), and the same gems are installed (this time into vendor/bundle, configurable). Bundler is used in Rails 3 to manage loading all of the gems, so wherever you've told bundler to install them (whatever your normal gem install location is by default, or BUNDLE_PATH (which is recorded in .bundle/config if you install with bundle install --path=foo otherwise), bundler will load the right ones, even when they differ from system gems.
You don't need to unpack the gems and check them in to your app, because it doesn't matter: you're guaranteeing the same versions are being called regardless of where they are installed, which will likely vary from machine to machine anyways (.bundle/ should not be checked in to the repo) - so why stick another 60-80 MB of files into your repo that you won't ever be changing or using? (incidentally, this is why I wouldn't recommend a bundle install --path=vendor/gems like nfm suggested - it's not necessarily wrong, there's just no benefit to it over the normal bundler workflow, and now your repo size just ballooned up).
DO NOT USE THE "RECOMMENDED" ANSWER BY NFM!
Instead, review the Bundler site, particularly the page on deployments:
http://gembundler.com/deploying.html
The short summary is to use specific versions in your Gemfile and run bundle install --deployment on each target system where you need the exact gem versions.
Using the --path option will install the gems, but it's not really what you want to do. As Matt Enright said, you just bloat your SCM with stuff that bundler can handle smartly on each target environment.
I haven't had to do this yet, but I believe it's all handled by bundler.
When you create a new rails3 app, the rails dependencies are put into your Gemfile. You can run bundle install to install them. By default, they are installed into your BUNDLE_PATH.
If you want to install them within your app, you can specify where: bundle install vendor/gems.
I had to do this for typus gem deployment on Heroku as you can't run a heroku rails generate typus on Heroku given it's a read only file system. I didn't want ALL gems put into my app, just the one that was causing me grief. Here are the steps that lead to success:
create directory in app_name/vendor/gems/gem_name (optional) ... in my case /app_name/vendor/gems/typus
add the following to gemfile (this tells bundle where to find and put the gem source):
gem 'typus', :git => 'https://github.com/fesplugas/typus.git', :path => "vendor/gems/typus"
then from within your app directory (this installs the gem into your app):
'gem unpack typus --target vendor/gems/typus'
then bundle install
then .. in my case... commit and push to repository and then deploy up to heroku... you may have to run a heroku rake db:migrate
Assuming you already have bundler gem installed:
$ bundle lock
$ git add Gemfile.lock
You can bundle install on dreamhost without any issues. If you are on shared the environment is already setup to store them locally in your home directory. If you are on a VPS or Dedicated you can run bundle install as root or just add this to your .bash_profile
export GEM_HOME=$HOME/.gems
export GEM_PATH=$GEM_HOME:/usr/lib/ruby/gems/1.8
I think what you are looking for is
bundle package
checkout the man pages here:
http://gembundler.com/man/bundle-package.1.html
I second the answer by tsega (updated by coreyward). "bundle package" is the generic answer.
The poster didn't ask WHETHER to freeze his gems. He wanted to know HOW. Answers like "Just don't do it" aren't helpful at all. Yes, it turned out his specific problem was a little different than that, but while "bundle package" might have been overkill it still solves the problem.
I have worked on a lot of systems, and on some you just don't have full access. Installing gems on some systems just isn't an option. So unless you package them, in general you're screwed. There are different workarounds for different hosts and systems, but none for some.
Pod - If you need to modify the gem, the best practice to do this would be forking the project, making the change, then using the 'git' flag in bundler:
git 'some_gem', :git => 'git://github.com/me/my_forked_some_gem.git'
This way you'll be notified when the gem is updated.
The command that you want is bundle package which just unpacks the gems and dependencies at vendor/cache folder.
But just a notice, the :git => .... kind of gems wont get packaged. You have to hack a way out for :git => ... related gems to get packed.
Cleaner instructions for the gem unpack and :path => option:
https://stackoverflow.com/a/8913286/555187
A lot of comments are somewhat saying that it's not useful to use the bundle install --path vendor/gems, but those people who are using Dreamhost, it should note that you cannot use bundle install in Dreamhost.
The solution is to get all the gems into the vendor folder and upload the whole thing to the Dreamhost directory.
There are other solutions to turn around this, but it's much more complicated to do.
Well I have to modify slightly one of the gems I need. So I need to keep it inside my Repo. So what NFM mentioned is what I probably need.