A rails 3.0.x project I am working with uses a gem that contains a large amount of test data that isn't needed for the heroku deployment. I would like to exclude this from the heroku slug, as it adds a few dozen megs to the slug (and has pushed us past the 100mb size limit several times, our slug is large for other reasons.)
I've tried doing this using the .slugignore mechanism, but I can't find a way to have it exclude files in gems as opposed to files in the app. This is a rails 3.0.x application running on the bamboo stack, but I would upgrade to rails 3.1 and/or the cedar stack if there was a workaround / procedure in those versions.
Other suggestions about fixing this that aren't 'make the huge gem smaller' are also great and extremely welcome.
On cedar stack the gems are installed into your vendor folder (the actual path is something like vendor/bundle/ruby/1.9.1/gems/), so I believe you can slugignore the subpaths you need, though I haven't tried it.
In general use case this will not work because .slugignore matched files are being deleted before installing gem files. See the following bit of heroku deployment output:
-----> Deleting 2 files matching .slugignore patterns.
-----> Ruby/Rails app detected
-----> Installing dependencies using Bundler version 1.3.0.pre.5
Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin --deployment
Using rake (10.0.3)
Using Platform (0.4.0)
Using open4 (1.3.0)
...
Related
I've just set up a Ruby on Rails app on Heroku, but, I'm experiencing a CI setup performance problem, which is quite annoying.
During the setup of our unit tests, Heroku is always re-installing our gems, producing logs like:
Fetching activestorage 6.0.2.2
Installing activestorage 6.0.2.2
This is not the case when setting up our app, as it shows lines like this instead:
Using activestorage 6.0.2.2
In both cases, bundler looks to be called with the correct arguments:
(for CI setup)
Running: bundle install --without development --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
(for app setup)
Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
In both cases, we're having the line:
Bundled gems are installed into `./vendor/bundle`
So, I'm wondering: do you know why Heroku does not cache gems between CI test setups, while it's the case during consequent builds of our app?
If yes, is there a way to force the reuse of gems previously installed in our CI pipeline?
This is taking a lot of time and can benefit from that.
Thanks a lot
After asking Heroku's team, it was a bug in the ruby's buildpack.
This is now fixed, and works well.
Buildpack creators can cache data between CI build, it was just turned off during one of their upgrade.
Heroku does not cache gems between CI test setups
Heroku filesystem is an ephemeral filesystem. When you run the tests, a dyno is spun up with a brand new operating system. Then heroku install all the dependencies and starts your tests.
I think it is a common scenario with all the CI platforms. Everytime we run tests, they will spin up a new environment from start by installing all the dependencies.
One justification for this may be that tests change a lot of data on different levels. So the CI providers start from scratch so that previous test residue data can't interfere with current run.
I'm running multiple of my Rails apps on Heroku. I use most of the standards including Gemfile and Bundler. Gemfile.lock is commited with Git.
However, when I push to Heroku, instead of reading Gemfile.lock, it seems to just run bundle install --without development:test --path vendor/bundle --binstubs bin/ from afresh, which means it updates the gems every time I push, causing discrepancies between my dev and prod environments.
I get no error message, but that's not the behavior I expect. What am I doing wrong?
Thanks!
I think it does this s part of it's 'build everything from the ground up in case the developers system has any differences" philosophy.
For example if you were running "ibuntu" (made up) and your environment had different dependencies between gems due to the way ruby is implemented on your system. Its safer just to build everything from scratch o their system and make sure that any dependencies that come up in their stack get met.
I have an interesting error when installing gems directly from github (:git => 'whatever').
Firstly, when I remove all gems and run bundle install command, I get the following:
Installing gem1
Installing gem2
Using gem3 (the one from github)
Then when I want to check what I've got I see the following by using gem list:
gem1 (x.x.x)
gem2 (y.y.y)
No gem3... now, looking closer to the file system, I see the following:
ls -l ~/.rvm/gems/ruby-1.9.3-p125/gems
gem1
gem2
So where is gem 3? Not where I'm expecting it to be:
ls -l ~/.rvm/gems/ruby-1.9.3-p125/bundler/gems
gem3-213213213
So it goes under bundler/gems and is not visible to gem list... and by Capistrano deploy, which gives me following:
git://github.com/author/gem3.git (at master) is not checked out. Please run `bundle install`
I'm more worried about Capistrano unable to deploy... Anyone has any clues?
Bundler gets its gems from various sources on your system. As long as they are the correct version, it will pull them in.
When deploying, it has more strict/conservative behavior.
From bundle help install, in the section about Deployment Mode, which is used when the --deployment flag is specified:
Gems are installed to vendor/bundle not your default system loca-
tion
In development, it's convenient to share the gems used in your
application with other applications and other scripts run on the
system.
In deployment, isolation is a more important default. In addition,
the user deploying the application may not have permission to
install gems to the system, or the web server may not have permis-
sion to read them.
As a result, bundle install --deployment installs gems to the ven-
dor/bundle directory in the application. This may be overridden
using the --path option.
I tried this tip: https://github.com/thoughtbot/laptop/pull/14#issuecomment-3192270 .
On deploy I see
-----> Using RUBY_VERSION: ruby-1.9.3-p0
But my logs show the environment variable is not respected
INFO ruby 1.9.2 (2011-07-09) [x86_64-linux]
Hacky / experimental solutions accepted!
Edit: I am on the cedar stack.
Here's an update for everyone referencing this question... Heroku now allows you to specify your ruby version in your Gemfile, thanks to their addition to the latest version of bundler.
First run:
gem install bundler --pre
then add ruby '1.9.3' to your Gemfile as such.
source 'http://rubygems.org'
ruby '1.9.3'
gem 'rails', '3.2.3'
The full release from Heroku with details is at http://blog.heroku.com/archives/2012/5/9/multiple_ruby_version_support_on_heroku/
Right, so Ruby 1.9.3 is definitely possibly. As you said hacky/experimental solutions were accepted - this is definitely one of them.
It's actually really simple;
Enable Heroku Labs's user_env_compile feature for your application.
Set a heroku config variable to RUBY_VERSION to ruby-1.9.3-p0 (heroku config:add RUBY_VERSION=ruby-1.9.3-p0)
ENSURE that the heroku PATH config variable has bin at the front (heroku config:add PATH=bin:vendor/bundle/ruby/1.9.1/bin:/usr/local/bin:/usr/bin:/bin)
When you next deploy you should see your application using 1.9.3 - the deployment output will show this too;
-> Heroku receiving push
-----> Ruby/Rails app detected
-----> Using RUBY_VERSION: ruby-1.9.3-p0
-----> Installing dependencies using Bundler version 1.1.rc.7
Running: bundle install --without development:test --path vendor/bundle --binstubs bin/ --deployment
Since May 10 it is possible to use Ruby 1.9.3 in an easier way. Just take a look at Heroku's blog:
http://blog.heroku.com/archives/2012/5/9/multiple_ruby_version_support_on_heroku/
I was able to get a Rails 3.2 app running on Ruby 1.9.3-p0 on Heroku cedar today
I followed the steps on
http://railsapps.github.com/rails-heroku-tutorial.html
ran into a problem locally with 1.9.3 and openssl
but was able to get an app deployed and working
http://mccaffrey-rails32-193.herokuapp.com
no Procfile
thin and pg in Gemfile
I added some debug output so you can see the rails and ruby version info
Its crazy that a basic rails app slug size is 28.9MB!
I had the same problem: Specifying 1.9.3 in my Gemfile, but it was using 1.9.2. This fixed it:
heroku config:add PATH=bin:vendor/bundle/ruby/1.9.1/bin:/usr/local/bin:/usr/bin:/bin
According to the Heroku Cedar stack definition here Cedar currently only supports MRI 1.9.2. Have you tried contacting Heroku support? They're generally pretty responsive.
Edit: it's possible to 'fool' the platform sometimes into running some binaries such as running Haskell on Heroku so if you're feeling really dedicated you might be able to package something up. I imagine this isn't too easy with MRI though.
I have a lot of gems (in a Rails 2.3 app) in the vendor/ directory. These are frozen in using rake. I'd like to move to Bundlr.
What's the best way of migrating these gems?
Bundler will allow you to install specific versions of your gems: http://gembundler.com/gemfile.html
While I don't know of a way to auto-generate a Gemfile from your existing frozen gems, you can definitely do a quick audit of the versions of the gems you are using and then setup a Gemfile which will bundle in the exact same gems.
Here's info on setting up Rails 2.3 with Bundler: http://gembundler.com/rails23.html
Remember to remove all of your config.gem declarations from your environments. These should all be specified in the Gemfile now in the right format for bundler.
If you want your new gems to be stored in your repository for easy access (and faster install/deployment), check out bundle package: http://gembundler.com/bundle_package.html