When using Rails inside a Docker container
several posts, (including one on docker.com) use the following pattern:
In Dockerfile do ADD Gemfile and ADD Gemfile.lock, then RUN bundle install.
Create a new Rails app with docker-compose run web rails new.
Since we RUN bundle install to build the image, it seems appropriate to docker-compose build web after updating the Gemfile.
This works insomuch as the gemset will be updated inside the image, but:
The Gemfile.lock on the Docker host will not be updated to reflect the changes to the Gemfile. This is a problem because:
Gemfile.lock should be in your repository, and:
It should be consistent with your current Gemfile.
So:
How can one update the Gemfile.lock on the host, so it may be checked in to version control?
Executing the bundle inside run does update the Gemfile.lock on the host:
docker-compose run web bundle
However: You must still also build the image again.
Just to be clear, the commands to run are:
docker-compose run web bundle
docker-compose up --build
where web is the name of your Dockerized Rails app.
TL;DR - make the changes on the container, run bundle on the container and restart for good measure. Locally these changes will be reflected in your app and are ready to test/push out to git, and your production server will use it to rebuild.
Long; Read:
docker exec -it name_of_app_1 bash
vim Gemfile and put something like gem 'sorcery', '0.9.0' I feel this ensures you get the version you're looking for
bundle to get just this version in the current container's Gemfile and Gemfile.lock
This has been semi normal "Rails" type stuff here, just you are doing it on the container that is running. Now you don't have to worry about git and getting these changes onto. your git repo because these changes are all happening on your local copy. So like, open a terminal tab and go into your app and less Gemfile and you should see the changes.
Now you can restart your running container. Your Dockerfile will get rebuilt (in my case by docker-compose up locally, tests should pass. Browser test at will.
Commit your changes to git and use your deploy process to check it out on staging.
Notes:
Check that Dockerfile like the OP's links say. I'm assuming that you have some kind of bundle or bundle install line in your Dockerfile
Run bundle update inside the container, then rebuild.
$ docker-compose run web bundle update
$ docker-compose build
Related
Before I begin: This is not a post about speeding up bundle install that runs when I build the container.
I am building a Docker application that needs to run bundle install during runtime. It may take a while to explain this specific use-case, but the important component is: my running container will download rails projects, and run bundle install. Currently, this takes an extremely long time (likely because of nokogiri).
Is there a way to build my container, such that anytime my script runs bundle install during runtime, it uses cached gems?
I am using:
Docker Compose Version 3
Fargate
ECS
Set your BUNDLE_PATH env var to vendor/bundle
Mount a volume in Fargate to the bundle path
The first run will be slow since it has to build up the bundle cache, but after that it should only update gems if necessary.
I use gitlab.com and CI with the shared docker runner that runs tests for my Ruby on Rails project on every commit to master. I noticed that about 90% of the build time is spent on 'bundle install'. Is it possible to somehow cache the installed gems between commits to speed up the 'bundle install'?
UPDATE:
To be more specific, below is the content of my .gitlab-ci.yml. The first 3 lines of the 'test' script take about 90% of the time making the build run for 4-5 minutes.
image: ruby:2.2.4
services:
- postgres
test:
script:
- apt-get update -qy
- apt-get install -y nodejs
- bundle install --path /cache
- bundle exec rake db:drop db:create db:schema:load RAILS_ENV=test
- bundle exec rspec
I don't know if you have special requirements for doing a apt-get all the time, if that is not needed create your own dockerfile with those commands in it. So that your base has already those updates/nodejs packages. If you want to update later on, you can always update your dockerfile again.
For your gems, if you want it quicker you can cache them in between builds too. Normally this is per job and per branch. See example here http://doc.gitlab.com/ee/ci/yaml/README.html#cache
cache:
paths:
- /cache
I prefer to add key: "$CI_BUILD_REF_NAME" so that for that particular branch my files are cached. See environments to see what keys you can use more.
You can setup BUNDLE_PATH environment variable and point it to a folder where you want your gems to be installed. The first time you run bundle install it will install all gems and consequent runs will only check if there are any new gems and install only those.
Note: That's supposed to be the default behavior. Check your BUNDLE_PATH env var value. Is it being changed to a temporary per commit folder or something? Or, is 90% of the build time being spent on downloading gem meta information from rubygems.org? In which case you might want to consider using --local flag (but not sure this is a good idea on CI server).
Fetching source index for https://rubygems.org/
After looking at your .gitlab-ci.yml I noticed that your --path options is missing =. I think it is supposed to be:
- bundle install --path=/cache
I'm attempting to deploy a Rails app which I have developed on Windows, to Openshift. However, the bundle is not complete, because Gemfile.lock contains lines like:
pg (0.17.0-x86-mingw32)
which therefore don't get installed on the Linux instance at Openshift.
Knowing that Heroku works around this by detecting a Windows Gemfile.lock and removing it, I have tried adding a `.openshift/action_hooks/pre_build script doing either
rm Gemfile.lock
or
sed -i /mingw32/d Gemfile.lock
But neither helps. How can I deploy my app to Openshift and have it pick up suitable Linux versions of all gems?
Try including a full path to the file, like this: rm $OPENSHIFT_REPO_DIR/Gemfile.lock
You should be able to connect to the server via rhc ssh in order to test the solution.
Anybody knows how to make BUNDLE INSTALL Cache'ing work in latest DOCKER release?
I've tried so far:
1.
WORKDIR /tmp
ADD ./Gemfile Gemfile
ADD ./Gemfile.lock Gemfile.lock
RUN bundle install
2.
ADD . opt/railsapp/
WORKIDR opt/rails/app
RUN bundle install
None of them work, it still runs "BUNDE INSTALL" everytime from scratch without Gemfile being changed.
Anyone knows how to make Caching for bundle install work correctly?
Cheers,
Andrew
Each time you change any file in your local app directory, the cache will be wiped out, forcing every step afterwards to be re-run, including the last bundle install.
The solution is don't run bundle install in step 2. You have already installed your gems in step 1 and there is little chance the Gemfile will change between step 1 and step 2 ;-).
The whole point of step 1 is to add your Gemfile, which should not change to often, so you can cache it and the subsequent bundle command before adding the rest of your app, which will probably change very often if you are still developing it.
Here's how the Dockerfile could look like:
1.
WORKDIR /tmp
ADD ./Gemfile Gemfile
ADD ./Gemfile.lock Gemfile.lock
RUN bundle install
2.
ADD . opt/railsapp/
WORKIDR opt/rails/app
Versions of Docker before 0.9.1 did not cache ADD instructions. Can you check that you're running a version of Docker 0.9.1 or greater?
Also, which installation of Docker are you using? According to this GitHub issue, some users have experienced cache-busting ADD behavior when using unsupported Docker builds. Make sure you're using an official Docker build.
ADD caching is based on all the metadata of the file, not just the contents.
if you are running docker build in a CI-like environment with a fresh checkout, then it is possible the timestamps of the files are being updated which would invalidate the cache.
I don't understand why i have to update my gems localy and push it to heroku, to get the updated version of them?
why there is no heroku bundle update command?
When you bundle update or run any of the equivalent CLI commands, I believe Bundler updates your Gemfile.lock file - which keeps a tree of all your gem dependencies - and the lock file is tracked by your git repository (see here for more info).
If you were able to run the command directly on Heroku, then you'd have to pull your repository again, otherwise you'd have a git fast-forward issue on your hands.
So really, you're not running any more commands by having to do it locally and push it back up.
The real reason why should run bundle update localy first is to test if your application is still working with the newer gem version. heroku bundle update would be a dangerous command.