We have many gems in several rails projects. We use rvm and bundler and rubygems and gemsets. Is there a way to locally cache the gems that I need instead of having bundler ask gem to go get them? We change the gem name each time we have a new major branch... My system has many copies of many of the gems I am downloading. I have tried using a proxy, but that does not seem any faster, and requires running the vm and sometimes I have to manually flush the cache because it gets mixed up about what is available.
You can use bundle install --path vendor/bundle in development mode
to set your gems in vendor folder, and bundle package to grab the gems and packages them in vendor/cache. Read more about bundle package.
Read this question, and answers, I hope it'll be helpful for you.
Bundler does not currently use a shared, local cache, but you might have better luck by not using gemsets.
By default, Bundler installs gems to a shared location, and if you use the same shared location for all of your projects, it will reuse the same installed gems for any projects that use the same version of those gems (assuming that they use the same version of Ruby, too).
The isolation provided by gemsets is largely unnecessary with Bundler, which sets up load paths correctly so that only the correct version of each gem is visible to the application.
Related
I'm using Ruby 2.1 and Rails 4.1 on Windows 7. Whenever I run bundle install, all gems are installed in the system path c:/Ruby21/lib/ruby/gems/2.1.0/gems/. I also found the vendor directory in my project.
Coming from PHP composer and node.js npm background, all dependencies should be locally installed in the project vendor folder or node_modules folder. So, my questions are:
Should I install gems in the system path or vendor/bundle?
If all gems or some gems should be installed in the system path, how could it affect the production environment where I may not have shell access?
Should all gems or specific gems be installed in vendor/bundle?
How can I install gems in vendor/bundle?
When you run bundle install, you are using a tool called Bundler.
Bundler takes care of managing your dependencies in a similar way as Composer, but instead of installing everything in the project folder, it installs your gems system-wide, that are shared among all your projects. It keeps track of what project requires which libraries by using the Gemfile in your project folder. So, you should just let Bundler do its thing, it does it very well and is the standard package manager for Rails.
If your host supports Ruby and Rails applications (for example, a PaaS like Heroku), it definitely will support Bundler and all the necessary gems will be installed. If you're talking about a cheap shared hosting without shell access, you won't be able to deploy a Ruby application there anyway because you will need to install the actual Ruby interpreter and other things, which would require shell access.
No.
You shouldn't. There's this article describing how to do it, but it seems to me that
countless times where installing gems globally leaked into other projects on the same machine and led to weird behavior that was annoying to debug
has only ever happened to the author of this article, and I don't think Bundler is at fault. In any case, you should always prepend gem commands with bundle exec (as in bundle exec rspec) and you will never have the mentioned problem. bundle exec makes sure that when you execute a command from a gem, the correct version defined in your Gemfile is called, it is important if you have several version of the same gem installed in your system.
A few years ago when RVM was popular, gemsets achieved a similar goal but got mostly deprecated by rbenv and Bundler.
I am new to rails and am learning about bundler. I understand that bundle install installs all the gems specified in the gemfile but where does it install them to?
Does it install them on my computer permanently so that they can be used by any future project?
If so doesnt this mean my computer gets filled with random gem versions and gem installs that I needed for one example project but may never use again?
By default, bundle install is going to install at the root level so all users on a computer could have access to the gems. So 'yes' it is permanent (at least not tied to your application, you can remove them whenever you want).
Take a look at the man pages for bundler. In here, you'll notice that you can specify to install to a local directory.
Install your dependencies, even gems that are already installed to your system gems, to a location other than your system's gem
repository. In this case, install them to vendor/bundle.
$ bundle install --path vendor/bundle
Further bundle commands or calls to Bundler.setup or Bundler.require
will remember this location.
This will let you install the gems to a location inside your application. So when you delete the example app, you also delete the associated gems.
Also, if you would like to see where a specific gem is installed (say you want to look at its source code), type bundle show <gemname>. This will spit out the path to that gem.
The short answer is 'yes'. The longer answer is that there are some technologies which will reduce or eliminate the problems associated with this effect.
If you install 'RVM':
https://rvm.io/
this will allow you to install multiple versions of Ruby and create individual 'gemsets'. As you enter the directory that contains your project, the ruby version and gemset settings are automatically picked up and the active Ruby version will change. This way you can keep gems separate between projects - and use several Ruby versions at once, including JRuby and other esoteric versions.
To find out where gems are stored, type:
gem environment
into your command line and look for the INSTALLATION_DIRECTORY entry in the response.
I know that if application uses bundler, I can easily find all the gems installed by looking at the Gemfile.
Say, I am looking at the Rails 3 application that doesn't use bundler, how do I know what gems it uses?
Thanks
If it's not using Bundler, I don't know of a definitive way to identify every gem being used. You could search the entire app tree for require statements to start with, but that's not going to show most of them. Gems also require other gems internally, and will install their own dependencies, but those gems won't be referenced directly from your app's require statements.
If the app works and the tests pass (meaning you've at least got all the required gems installed), you could approach the problem by creating a Gemfile, listing the gems you know are needed, and then running your tests (or the app itself) via bundle exec, which will ensure that only the gems listed in the Gemfile are visible. Then you'll get failures related to missing gems, and can add them to the Gemfile until it all works. Once it's working via bundle exec, you'll know that you've captured all the requirements there.
If you're using RVM, you'll probably find it helpful to create a gemset for your app, along with a .rvmrc file in the app root, to take advantage of RVM's automatic gemset switching and Bundler integration. It'll make it easier to maintain the gem state going forward.
In any case, running gem list with the app in a working state will show you all the gems that it might be using, but without being scoped to a gemset or wrapped in bundle exec, you'll also see gems that were installed for other reasons that potentially have nothing to do with your app's dependencies.
What are the benefits & drawbacks of installing things into the #global gemset in RVM?
Let's say I want to install different versions of rails on the same server. I then want the ability to install multiple ruby apps on the same server, with the least duplication of files to save on disk space. However, I still want to avoid dependency problems, gem conflict issues and other problems.
Let's also assume that each app has extra gems it needs that I only want in it's local project gemset.
Would I be better off:
Installing both Rails 3 and Rails 2 gems into the #global gemset
...And use project-local gemsets for their gems...
Installing Rails 3 into a #rails3 gemset, and Rails 2 into a #rails2 set... then cloning for each project I need?
For example:
rvm use ree#rails3 && rvm gemset export rails3.gems
rvm use ree#rails2 && rvm gemset export rails2.gems
rvm use --create ree#project1-on-rails3 && rvm gemset import rails3.gems
Install more project-local gems here...
rvm use --create ree#project2-on-rails2 && rvm gemset import rails2.gems
Install more project-local gems here...
Something else entirely...
NOTE: I wrote this whole response assuming that you are using Bundler to manage your gem dependencies. I realize that some people don't, and you didn't mention Bundler in your question. If you aren't using Bundler, I would point out that it probably is the best way to conserve disk space (only if you bundle install --system, though!). If you are using exported gemsets to manage dependencies, I think your scheme sounds reasonable, but I have no experience with it.
Both Rails 3, and Rails 2 with Bundler will set their load path appropriately such that they will not load any gems (or any versions of any gems) that are not in the Gemfile.lock. There's not really any way that I've experienced to have a "gem dependency problem" on the server. It is important that you run bundle install on your development machine whenever you modify the Gemfile, and that you check your Gemfile.lock into source control, as described on the Bundler homepage.
I spent some time digging into the use-cases of gemsets back in January. The reasons I found to use separate gemsets for each project were:
Your shell environment is the same as your application environment (scripts run correctly without bundle exec).
You can easily browse and grep through the source code of all your dependencies, by navigating to the gemset install directory.
It prevents some reported ‘heisenbugs’, according to the author of RVM. I have experienced something like this where a gem executable wasn't available and bundle exec didn't seem to help.
I don't think any of these benefits are very compelling on the server, so if you are aiming to conserve disk space, I'm not sure why you would use gemsets at all.
Actually, the only reason I've used rvm at all on the server was because it was a convenient way to build ruby from source (we needed a version that wasn't available in the native package manager).
I want to freeze a specific gem into my Rails application.
In rails 2 there was this command:
rake gems:unpack
I can't find that command in Rails 3.
So, the short answer is, you don't.
When you modify your Gemfile, and then run bundle install or bundle update, bundler handles the dependency resolution for you and determines the best (newest) versions of each gem you've required that satisfies the entire dependency chain (you won't get a new version that breaks another gem in the dependency list, etc.). You can also of course place a specific version, or a '>= 1.2.3' specification or whathaveyou in the Gemfile using the familiar syntax from the config.gem days, and bundler will make sure to satisfy that as well (or won't produce a Gemfile.lock if there is no valid resolution).
When Bundler does its business, it creates the Gemfile.lock file, which (and this is provided you use bundler alone for managing your gem on all workstations/environments/deployments) performs the same function as freezing all of the gems you've required. For free! (Check this file into version control!) If your new development intern pulls down your source on a fresh machine, it takes one bundle install and the exact same versions of the gems that you have installed are on her machine. Push to deployment, and do a bundle install --deployment there (or more likely, throw it in your Capfile), and the same gems are installed (this time into vendor/bundle, configurable). Bundler is used in Rails 3 to manage loading all of the gems, so wherever you've told bundler to install them (whatever your normal gem install location is by default, or BUNDLE_PATH (which is recorded in .bundle/config if you install with bundle install --path=foo otherwise), bundler will load the right ones, even when they differ from system gems.
You don't need to unpack the gems and check them in to your app, because it doesn't matter: you're guaranteeing the same versions are being called regardless of where they are installed, which will likely vary from machine to machine anyways (.bundle/ should not be checked in to the repo) - so why stick another 60-80 MB of files into your repo that you won't ever be changing or using? (incidentally, this is why I wouldn't recommend a bundle install --path=vendor/gems like nfm suggested - it's not necessarily wrong, there's just no benefit to it over the normal bundler workflow, and now your repo size just ballooned up).
DO NOT USE THE "RECOMMENDED" ANSWER BY NFM!
Instead, review the Bundler site, particularly the page on deployments:
http://gembundler.com/deploying.html
The short summary is to use specific versions in your Gemfile and run bundle install --deployment on each target system where you need the exact gem versions.
Using the --path option will install the gems, but it's not really what you want to do. As Matt Enright said, you just bloat your SCM with stuff that bundler can handle smartly on each target environment.
I haven't had to do this yet, but I believe it's all handled by bundler.
When you create a new rails3 app, the rails dependencies are put into your Gemfile. You can run bundle install to install them. By default, they are installed into your BUNDLE_PATH.
If you want to install them within your app, you can specify where: bundle install vendor/gems.
I had to do this for typus gem deployment on Heroku as you can't run a heroku rails generate typus on Heroku given it's a read only file system. I didn't want ALL gems put into my app, just the one that was causing me grief. Here are the steps that lead to success:
create directory in app_name/vendor/gems/gem_name (optional) ... in my case /app_name/vendor/gems/typus
add the following to gemfile (this tells bundle where to find and put the gem source):
gem 'typus', :git => 'https://github.com/fesplugas/typus.git', :path => "vendor/gems/typus"
then from within your app directory (this installs the gem into your app):
'gem unpack typus --target vendor/gems/typus'
then bundle install
then .. in my case... commit and push to repository and then deploy up to heroku... you may have to run a heroku rake db:migrate
Assuming you already have bundler gem installed:
$ bundle lock
$ git add Gemfile.lock
You can bundle install on dreamhost without any issues. If you are on shared the environment is already setup to store them locally in your home directory. If you are on a VPS or Dedicated you can run bundle install as root or just add this to your .bash_profile
export GEM_HOME=$HOME/.gems
export GEM_PATH=$GEM_HOME:/usr/lib/ruby/gems/1.8
I think what you are looking for is
bundle package
checkout the man pages here:
http://gembundler.com/man/bundle-package.1.html
I second the answer by tsega (updated by coreyward). "bundle package" is the generic answer.
The poster didn't ask WHETHER to freeze his gems. He wanted to know HOW. Answers like "Just don't do it" aren't helpful at all. Yes, it turned out his specific problem was a little different than that, but while "bundle package" might have been overkill it still solves the problem.
I have worked on a lot of systems, and on some you just don't have full access. Installing gems on some systems just isn't an option. So unless you package them, in general you're screwed. There are different workarounds for different hosts and systems, but none for some.
Pod - If you need to modify the gem, the best practice to do this would be forking the project, making the change, then using the 'git' flag in bundler:
git 'some_gem', :git => 'git://github.com/me/my_forked_some_gem.git'
This way you'll be notified when the gem is updated.
The command that you want is bundle package which just unpacks the gems and dependencies at vendor/cache folder.
But just a notice, the :git => .... kind of gems wont get packaged. You have to hack a way out for :git => ... related gems to get packed.
Cleaner instructions for the gem unpack and :path => option:
https://stackoverflow.com/a/8913286/555187
A lot of comments are somewhat saying that it's not useful to use the bundle install --path vendor/gems, but those people who are using Dreamhost, it should note that you cannot use bundle install in Dreamhost.
The solution is to get all the gems into the vendor folder and upload the whole thing to the Dreamhost directory.
There are other solutions to turn around this, but it's much more complicated to do.
Well I have to modify slightly one of the gems I need. So I need to keep it inside my Repo. So what NFM mentioned is what I probably need.