I am trying to run a small Rails app in a docker container. I am getting close, however I am struggling with my Dockerfile.
I have added the following command to my Dockerfile to recursively add all files in my project folder.
ADD .
After this, I run
RUN bundle install --deployment
However, because my ADD command also adds the Dockerfile, it means that my image cache breaks every time I edit my Dockerfile forcing me to rebundle.
According to https://docs.docker.com/reference/builder/#the-dockerignore-file, I can use a .dockerignore file to ignore Dockerfile, but this causes the docker build command to fail with
2014/09/17 22:12:46 Dockerfile was excluded by .dockerignore pattern 'Dockerfile'
How can I easily add my project to my image, but exclude the Dockerfile, so I don't break the docker image cache?
Your clarification in a comment to the answer from #Kuhess, saying that the actual problem is "the invalidation of the docker image cache causing bundle install to run again", is helpful in providing you an answer.
I've been using a Dockerfile that looks like the following for my rails 4.1.* app. By ADDing Gemfile* first, then running bundle install, and only then ADDing the rest of the app, the bundle install step is cached unless one of the Gemfile* files changes.
FROM ruby:2.1.3
RUN adduser --disabled-password --home=/rails --gecos "" rails
RUN gem install bundler --no-ri --no-rdoc
RUN gem install -f rake --no-ri --no-rdoc
RUN mkdir /myapp
WORKDIR /myapp
ADD Gemfile /myapp/Gemfile
ADD Gemfile.lock /myapp/Gemfile.lock
RUN bundle install
ADD . /myapp
RUN chown -R rails:rails /myapp
USER rails
EXPOSE 3000
ENV RAILS_ENV production
CMD bundle exec rails server -p 3000
My Dockerfile is in the root dir of the app, but changes to it (or any other part of the app) do not cause the image cache for bundle install to break because they are added after it is RUN. The only thing that breaks it are changes to Gemfile*, which is correct.
FYI, my .dockerignore file looks as follows:
.git
log
vendor/bundle
There is an issue for that on Github: https://github.com/docker/docker/issues/7969
Your main problem is the eviction of the cache because of the ADD and the modification of the Dockerfile. One of the maintainer explains that, for the moment, the .dockerignore file is not designed to deal with it:
It — .dockerignore — skips some files when you upload your context from client to daemon and daemon needs Dockerfile for building image. So main idea(for now) of .dockerignore is skipping big dirs for faster context upload, not clean context. full comment on Github
I am afraid that the image cache will break when you ADD Dockerfile while these lines are not modified.
Maybe one way to deal with the cache is to place all the files you want to add in a different directory than the Dockerfile:
.
├── Dockerfile
└── files_to_add/
Then if you ADD files_to_add, the Dockerfile will not be included and the cache will not be evicted.
But, I do not consider that this trick is a solution. I also want to have my Dockerfile next to other files at the root of my projects.
Related
I have the following Dockerfile
ARG JEKYLL_VERSION=4
FROM jekyll/jekyll:$JEKYLL_VERSION as BUILD
COPY --chown=jekyll:jekyll . /srv/jekyll
RUN ls -lah /srv/jekyll
RUN jekyll build
RUN ls /srv/jekyll/_site/
FROM nginxinc/nginx-unprivileged:alpine
ADD build/nginx-site.conf /etc/nginx/conf.d/default.conf
COPY --chown=101:101 --from=BUILD /srv/jekyll/_site/ /var/www
Which does build perfectly locally, but not on the Linux Jenkins Buildslave:
Bundle complete! 1 Gemfile dependency, 28 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux-musl]
Configuration file: /srv/jekyll/_config.yml
Source: /srv/jekyll
Destination: /srv/jekyll/_site
Incremental build: disabled. Enable with --incremental
Generating...
done in 0.186 seconds.
Auto-regeneration: disabled. Use --watch to enable.
Removing intermediate container 2b064db0ccaa
---> 1e19e78f593a
Step 6/9 : RUN ls /srv/jekyll/_site/
---> Running in 194d35c3f691
[91mls: /srv/jekyll/_site/: No such file or directory
[0mThe command '/bin/sh -c ls /srv/jekyll/_site/' returned a non-zero code: 1
Changing RUN jekyll build to RUN jekyll build && ls /srv/jekyll/_site/ lists the jekyll output as expected.
Why is the output of the jekyll build not stored? If I remove the ls command, the output can't be found in the later stage.
Any hints?
That happens because the image declares that directory as a VOLUME. This is probably a bug and I'd suggest reporting it as a GitHub issue; it won't affect any of the documented uses of the image to remove that line.
At the end of the day Jekyll is just a Ruby gem (library). So while that image installs a lot of things, for your use it might be enough to start from a ruby image, install the gem, and use it:
FROM ruby:3.1
RUN gem install jekyll
WORKDIR /site
COPY . .
RUN jekyll build
If your directory tree contains a Ruby Gemfile and Gemfile.lock, you could also COPY those into the image and RUN bundle install instead.
Alternatively, since Jekyll is a static site generator, you should get the same results if you build the site on the host or in a container, and another good path could be to install Ruby and Jekyll on your host (maybe in an rbenv gemset) and then COPY the site from the host into the Nginx image; skip the first stage entirely.
I am have made a Docker container for my Ruby project. I want to have a bind mount for my project files but I can't seem to figure it out.
I usually use docker-compose and I would just include the volume service there and it would work fine.
Here is the Dockerfile:
FROM ruby:2.2.2
# throw errors if Gemfile has been modified since Gemfile.lock
RUN bundle config --global frozen 1
WORKDIR /usr/src/app
COPY Gemfile Gemfile.lock ./
RUN bundle install
COPY . .
CMD ["./script.rb"]
As it stands right now I have to rebuild my image every time I add new code and then change the path in the CMD line. Obviously, this is bad.
I read the official docs but in there it just shows a huge command line entry which is not ideal either.
Is there a way to have a bind mount and have the container run the code from various folders in my project?
Why would you copy Gemfile.lock, run bundle install to create a new Gemfile.lock, then immediately copy the current directory which contains the original Gemfile.lock and overwrite the Gemfile.lock that was just created by Bundler within the Docker container?
Also why can you get away with not having EXPOSE 3000?
https://docs.docker.com/compose/rails/#define-the-project
FROM ruby:2.3.3
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
RUN mkdir /myapp
WORKDIR /myapp
COPY Gemfile /myapp/Gemfile
COPY Gemfile.lock /myapp/Gemfile.lock
RUN bundle install
COPY . /myapp
That isn't the only place it does that. It's also done here, which seems pretty official. Maybe I'm missing a fundamental aspect of Docker?
https://hub.docker.com/_/ruby/
COPY Gemfile Gemfile.lock ./
RUN bundle install
COPY . .
More a guess than an answer, but sometimes you order the steps in Dockerfiles slightly differently to improve the caching mechanism. When you change things in your application, it's less likely that it will affect the Gemfiles, so you don't have to do a bundle install after everything you change. Ordering the steps in this way avoids having to execute bundle install for changes to your application that do not affect the Gemfiles.
Documentation on build caching: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#build-cache
In regards to the second part of this question:
Also why can you get away with not having EXPOSE 3000?
The full Dockerfile that you referenced does contain this line:
EXPOSE 3000
I am trying to run webpack inside a docker container for a node app. I get the following error.
sh: 1: webpack: Permission denied
The Dockerfile works fine on a normal build.
FROM node
# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install
# Bundle app source
COPY . /usr/src/app
EXPOSE 3001
#This launches webpack which fails.
CMD [ "npm", "start" ]
I had the same issue, as I was migrating an existing project to docker. I resolved it by not copying the entire project contents (COPY . /usr/src/app in your docker file) and instead only copying the files and directories actually required.
In my case, the unnecessary directories added when copying the whole project were, among other things, node_modules, the build directory and the entire .git repo directory.
I still don't know why copying the entire directory doesn't work (something conflicts with something? something has incorrect permissions?), but copying only what you need is better for image size anyway.
The official rails image on docker hub:
https://hub.docker.com/_/rails/
I create a Dockerfile like:
FROM rails:onbuild
ENV RAILS_ENV=production
ADD vendor/gems/my_gem /usr/src/app/vendor/gems/my_gem
CMD ["sh", "/usr/src/app/init.sh"]
My init.sh
#!/bin/bash
bundle exec rake db:create db:migrate
bundle exec rails server -b 0.0.0.0
My Gemfile
...
gem 'my_gem', path: './vendor/gems/my_gem'
...
When I build my docker image:
docker build -t myapp .
It said:
...
The path `/usr/src/app/vendor/gems/my_gem` does not exist.
The command '/bin/sh -c bundle install' returned a non-zero code: 13
The default path is /usr/src/app. How to add special files there?
docker ADD will add <src> (when it is a folder, not an url) relative to the source directory that is being built (the context of the build).
So you need to be sure your current directory when doing docker build . is also the one which includes vendor/gems/my_gem.
The OP scho reports in the comments
After I changed to FROM rails:4.2.1, it worked.
As documented in docker rails:
This image (rails:onbuild) includes multiple ONBUILD triggers which should cover most applications.
The build will COPY . /usr/src/app, RUN bundle install, EXPOSE 3000, and set the default command to rails server.
That means the ADD command was probably not needed and in conflict with the ONBUILD triggered COPY.
That differs from using rails:4.2.1, where the ADD or COPY is left to the Dockerfile specification (as opposed to ONBUILD triggers).