Installed gems not found by bundler when BUNDLE_PATH changed with Docker - ruby-on-rails

I'm using docker to develop a rails application. The docker file looks like this:
FROM ruby:1.9.3
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev vim
ENV APP_HOME /next-reg
RUN mkdir $APP_HOME
WORKDIR $APP_HOME
ENV BUNDLE_PATH /box
ADD . $APP_HOME
RUN gem install gem1.gem gem2.gem
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN bundle install
As you can see I change the BUNDLE_PATH, this is because of an article showing how we can persist gem downloads. So that overtime when the docker cache gets warm, it re-bundles and takes FOREVER.
When I docker build it successfully installs the gems, then it fails to find them on bundle. Could someone give me a hand with persisting gems, installing my own gems, and getting it to work?
Before I changed the BUNDLE_PATH the build worked, it just re bundled frequently without changes to the gem file (because, I guess the docker image cache got warm).
My docker-compose is like this:
db:
image: postgres
volumes:
- ~/.docker-voumes/postgres/data:/var/lib/postgresql/data
# This is to hold and persist ruby gems, referenced in web and in web's dockerfile.
gem_files:
image: busybox
volumes:
- /box
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/next-reg
volumes_from:
- gem_files
ports:
- "3000:3000"
- "8000:8000"
links:
- db
env_file:
- .myenv.env

I think that there is a lack of GEM_HOME/GEM_PATH in your code.
GEM_HOME/GEM_PATH will be used by gem install xxx to install gems in a specific folder.
BUNDLE_PATH will be used by bundle install to install gems in a specific folder but not by gem install xx
To have a working system you should do :
FROM ruby:1.9.3
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev vim
ENV APP_HOME /next-reg
RUN mkdir $APP_HOME
WORKDIR $APP_HOME
ENV BUNDLE_PATH /box
ENV GEM_PATH /box
ENV GEM_HOME /box
ADD . $APP_HOME
RUN gem install bundler
RUN gem install tzinfo -v 1.2.2
COPY Gemfile Gemfile
RUN bundle install
With this Gemfile :
source 'https://rubygems.org'
gem 'tzinfo', '1.2.2'
Wich will produce :
Step 11/13 : RUN gem install tzinfo -v 1.2.2
---> Running in 8a87fa54fa19
Successfully installed thread_safe-0.3.6
Successfully installed tzinfo-1.2.2
2 gems installed
---> 3c91d59bde8a
Removing intermediate container 8a87fa54fa19
Step 13/13 : RUN bundle install
---> Running in 20f1e4ec93b1
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...
Fetching version metadata from https://rubygems.org/.
Resolving dependencies...
Rubygems 1.8.23.2 is not threadsafe, so your gems will be installed one at a time. Upgrade to Rubygems 2.1.0 or higher to enable parallel gem installation.
Installing rake 12.0.0
Using thread_safe 0.3.6
Using bundler 1.14.6
Using tzinfo 1.2.2
Bundle complete! 2 Gemfile dependencies, 4 gems now installed.
Bundled gems are installed into /box.
As you can see in the result output, the bundle install re-use the preloaded gems from gem install

Related

"Could not find rake-12.3.3 in any of the sources."

I tried running these commands:
gem uninstall rake
rm Gemfile.lock
bundle install
gem install rake --version=12.3.3
I also tried adding rake 12.3.3 to the gemfile and running bundle install:
gem 'rake', '12.3.3'
This is my Dockerfile:
FROM ruby
RUN apt-get update -qq && apt-get install -y build-essential
# for postgres
RUN apt-get install -y libpq-dev
# for nokogiri
RUN apt-get install -y libxml2-dev libxslt1-dev
# for a JS runtime
RUN apt-get install -y nodejs
ENV APP_HOME /readpaths
WORKDIR $APP_HOME
ADD Gemfile* $APP_HOME/
RUN bundle install
ADD . $APP_HOME
I've just noticed something curios: No matter how often I run gem uninstall rake (I tried five times) I always get a message saying:
Successfully uninstalled rake-12.3.1
When I run gem install rake I get:
Successfully installed rake-12.3.3
Then, when I run gem uninstall rake again, I get again:
Successfully uninstalled rake-12.3.1
If you come to this error because of Heroku/Jekyll, this is how I fixed it:
Make sure you only have this buildpack: https://github.com/heroku/heroku-buildpack-ruby
Make sure your dyno is actually on
Your Procfile should have this: web: jekyll serve --no-watch --port $PORT --host 0.0.0.0
Make sure your have PORT configured in the config vars section of your settings
Try rebuilding the docker container.

Install Bundler failed in Ruby Docker image

I have a very old project I'm giving support. I'm using docker to make my builds and everything was working until this last one try.
This is my Dockerfile:
FROM ruby:2.1
# Install apt based dependencies required to run Rails as
# well as RubyGems. As the Ruby image itself is based on a
# Debian image, we use apt-get to install those.
RUN apt-get update && apt-get install -y \
git \
ghostscript \
libgs-dev \
build-essential \
libmysqlclient-dev \
locales \
nodejs
# Use en_US.UTF-8 as our locale
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
# Configure the main working directory. This is the base
# directory used in any further RUN, COPY, and ENTRYPOINT
# commands.
RUN mkdir -p /app
WORKDIR /app
# Copy the Gemfile as well as the Gemfile.lock and install
# the RubyGems. This is a separate step so the dependencies
# will be cached unless changes to one of those two files
# are made.
COPY Gemfile Gemfile.lock ./
COPY vendor/oohlalog_gem ./vendor/oohlalog_gem
RUN gem install bundler && bundle install
# Copy the main application.
COPY . ./
# Expose port 3000 to the Docker host, so we can access it
# from the outside.
EXPOSE 3000
# Configure an entry point, so we don't need to specify
# "bundle exec" for each of our commands.
ENTRYPOINT ["bundle", "exec"]
# The main command to run when the container starts. Also
# tell the Rails dev server to bind to all interfaces by
# default.
CMD ["rails", "server", "-b", "0.0.0.0"]
When I try to run:
docker build . -t my_image
I'm getting:
Step 11 : RUN gem install bundler && bundle install
---> Running in 7f8e7442b423 ERROR: Error installing bundler:
bundler requires Ruby version >= 2.3.0.
I don't know how to fix this.
This is a result of a release of a new Bundler version 2.0.0 that happened on January 03, 2019. According to GitHub issue this is a desired behaviour.
As this is an old and not maintained anymore project the best option for you would be using older bundler.
You can modify the line from your Dockerfile:
RUN gem install bundler && bundle install
to the following:
RUN gem install bundler -v '$OLD_BUNDLER_VERSION' && bundle install
where $OLD_BUNDLER_VERSION should be set to last known working version, i.e. 1.16.1
Try a newer version of ruby image as your base. You are using 2.1 ( FROM ruby:2.1 ) but you need ruby version 2.3.0
My way is to use the correct bundler version and run it like this:
ENV BUNDLER_VERSION 2.2.26
RUN gem update --system \
&& gem install bundler -v $BUNDLER_VERSION \
&& bundle install -j 4

Docker failed to build on Rails bundle Install rake

I use Docker for Windows which I use to run a Image for a Rails(v5) App.
When I do docker-compose build I get this error. I don't known where to look to debug and/or resolve this.
Here is my Dockerfile:
FROM ruby:2.3.3-slim
RUN apt-get update && apt-get install -y build-essential
RUN apt-get install -y libmysqlclient-dev
RUN apt-get install -y mysql-client
RUN apt-get install -y libmagickwand-dev imagemagick
RUN apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup_6.x | bash -
RUN apt-get install -y nodejs
RUN apt-get install -y git
#RUN npm install -g phantomjs
RUN gem update --system
RUN mkdir -p /MyImage
WORKDIR /MyImage
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN gem install bundler
RUN bundle install
COPY package.json package.json
RUN npm update
RUN npm install
COPY . /MyImage
CMD [ "foreman", "start" ]
Here is a log if my error:
Step 15/20 : RUN bundle install
---> Running in 4bf360b89cb7
The git source `git://github.com/acrogenesis/owlcarousel-rails.git` uses the `git` protocol, which transmits data without encryption. Disable this warning with `bundle config git.allow_insecure true`, or switch to the `https` protocol to keep your data secure.
The git source `git://github.com/sinatra/sinatra.git` uses the `git` protocol, which transmits data without encryption. Disable this warning with `bundle config git.allow_insecure true`, or switch to the `https` protocol to keep your data secure.
Fetching source index from https://rubygems.org/
Fetching source index from https://rails-assets.org/
Fetching git://github.com/sinatra/sinatra.git
Fetching git://github.com/acrogenesis/owlcarousel-rails.git
Fetching gem metadata from https://rails-assets.org/..
Fetching rake 12.1.0
Installing rake 12.1.0
Errno::ENOENT: No such file or directory # rb_sysopen -
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/templates/Executable
An error occurred while installing rake (12.1.0), and Bundler cannot continue.
Make sure that `gem install rake -v '12.1.0'` succeeds before bundling.
In Gemfile:
autonumeric-rails was resolved to 2.0.0.1, which depends on
jquery-rails was resolved to 4.3.1, which depends on
railties was resolved to 5.0.6, which depends on
rake
ERROR: Service 'MyImage' failed to build: The command '/bin/sh -c bundle install' returned a non-zero code: 5
But when I do gem install rake -v '12.1.0' directly into the shell of my container, everything install just fine.
Just try to install rake by yourself as described in the error message:
gem install rake -v '12.1.0'
To debug:
Comment out all lines in your Dockerfile starting from RUN bundle install and to end;
Then run docker build -t test:0.0 .;
To start interactive session: docker run -it test:0.0 /bin/bash;
Try to execute gem install rake -v '12.1.0' manually...

Issue with docker and bundler

I'm running some issues with docker/docker-compose and bundler. After building my image I can run the rails server correctly without any issue, but, when I try to run a console with rails console I constantly get:
Could not find i18n-0.7.0 in any of the sources
Run `bundle install` to install missing gems.
If I try to run bundle install in the container there is no problem, all seems to be correctly installed.
docker-compose run web bundle install
...
Using spring 1.3.6
Using therubyracer 0.12.2
Using turbolinks 2.5.3
Using uglifier 2.7.2
Using web-console 2.2.1
Updating files in vendor/cache
Bundle complete! 24 Gemfile dependencies, 119 gems now installed.
Bundled gems are installed into /usr/local/bundle.
My docker-compose.yml looks like this:
db:
image: postgres
web:
build: .
command: rails s -p 3000 -b 0.0.0.0
ports:
- "3000:3000"
volumes:
- .:/app
- ./github_rsa:/root/.ssh/id_rsa
links:
- db
I need to mount a volume with the ssh key because there are some gems that need to be pulled from private repositories.
My Dockerfile looks like this:
FROM ruby:2.2.0
RUN apt-get update -qq && apt-get install -y build-essential \
libpq-dev \
libxml2-dev libxslt1-dev \
nodejs
ENV HOME /root
ENV APP_HOME /app
RUN mkdir $APP_HOME
RUN mkdir $APP_HOME/tmp
RUN mkdir $APP_HOME/log
# Copy the Gemfile and Gemfile.lock into the image.
# Temporarily set the working directory to where they are.
WORKDIR /tmp
ADD Gemfile Gemfile
ADD Gemfile.lock Gemfile.lock
# Copy the github key for pulling gems
COPY github_rsa /root/.ssh/id_rsa
RUN \
chown -R root:root /root/.ssh && \
chmod 700 $HOME/.ssh && \
chmod 600 $HOME/.ssh/id_rsa
RUN echo "Host github.com\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config
# Start ssh agent and add keys
RUN eval "$(ssh-agent)" && \
ssh-add && \
ssh-add -l
# Install bundler
RUN gem install bundler -v '~> 1.10'
# Install ruby dependencies
RUN bundle install
# Remove the ssh key now, we don't want to ship secrets on our images
RUN rm /root/.ssh/id_rsa
# Add app to container
ADD . $APP_HOME
# Add container working directory
WORKDIR $APP_HOME
# Expose puma port
EXPOSE 3000
You need to take this image down immediately. The following does not work:
RUN rm /root/.ssh/id_rsa
You cannot remove files like this; they will still exist in previous layers and be accessible to anyone who has the image. At the moment, you are shipping your secrets.
Regarding the actual question, I suspect it's just to do with the working directory and paths. Try moving RUN bundle install to after the WORKDIR instruction.

bundler installed gems not persisting in fig/docker

I am trying to use fig and docker to set up a local development environment for an existing rails app.
In the build process, I clearly see bundler installing the app's gems, but when I try to start the container with fig up or even reopen it with the /bin/bash command, the gems are not visible.
Here is the Dockerfile:
FROM ubuntu:14.04
# REPOS
RUN apt-get -qq update
RUN apt-get install -y software-properties-common
RUN add-apt-repository -y "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) universe"
RUN add-apt-repository -y ppa:chris-lea/node.js
RUN apt-get -y update
#INSTALL
RUN apt-get install -y -q build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison pkg-config libpq-dev make wget unzip git vim nano nodejs gawk libgdbm-dev libffi-dev
#RUBY
RUN mkdir -p /download
WORKDIR download
RUN wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz
RUN tar xvfz ruby-2.1.2.tar.gz
WORKDIR /download/ruby-2.1.2
RUN ./configure
RUN make
RUN make install
RUN gem update --system
RUN gem install bundler
RUN mkdir /rent
WORKDIR /rent
ADD Gemfile /rent/Gemfile
ADD Gemfile.lock /rent/Gemfile.lock
RUN bundle install --deployment
And here is the fig.yml file:
web:
build: .
command: bundle exec rails s -p 3000
volumes:
- .:/rent
ports:
- "3000:3000"
Running fig build clearly shows the app's gems being installed.
Running fig up fails with the message
bundler: command not found: rails
If I run fig run web /bin/bash and check the contents of
/local/lib/ruby/gems/2.1.0/gems
it only has bundler, rdoc, rake and a few others installed.
If I navigate to the app's directory and run the bundle command, it will install the app's gems and I can see that they are installed in the directory above. I can even start the app with rails server.
Why aren't the bundled gems being persisted in the image(container?).
I ran the rails tutuorial from the fig website and didn't have this problem.
Thanks
It seems that the problem was caused by using the --deployment flag with bundle install.
This flag's main effect is to deploy the gems to the vendor/bundle/ directory instead of the normal gem location. I checked and the gems were there, so I'm not sure why ruby couldn't find them.
Anyways, removing --deployment fixed the problem.
If you're using fig and mounted volumes, I found a solution which allows updates to the mounted files in development (like Gemfile.lock when you do fig run web bundle install) while still keeping container caching behavior.
See this for the full thing: https://gist.github.com/fotinakis/04077671bec4edf77c08
It's slightly convoluted, but basically you always install bundler and the gems as the non-root application user:
# Add 'web' user which will run the application.
RUN adduser web --home /home/web --shell /bin/bash --disabled-password --gecos ""
# Add directory where all application code will live and own it by the web user.
RUN mkdir /app
RUN chown -R web:web /app
# Install gems separately here to take advantage of container caching of `bundle install`.
# Also, ensure that gems are installed as the web user and not system-wide so that we can run
# `fig web bundle install` and the web user will have permissions to update the shared Gemfile.lock.
ADD Gemfile /app/
ADD Gemfile.lock /app/
RUN chown -R web:web /app
USER web
ENV HOME /home/web
ENV PATH $PATH:/home/web/.gem/ruby/2.1.0/bin
ENV GEM_HOME /home/web/.gem/ruby/2.1.0
ENV GEM_PATH $GEM_HOME
RUN gem install --user-install bundler
WORKDIR /app/
RUN bundle install
USER root
# Add the whole application source to the image and own it all by web:web.
# Note: this is overwritten in development because fig mounts a shared volume at /app.
ADD . /app/
RUN chown -R web:web /app
Running this will now work and update your local Gemfile.lock:
$ fig run web bundle install

Resources