I've got a rails proxy app that runs with jupyter notebooks, and I'm trying to get a docker image up and running with supervisor as the entry point.
However, after including a FROM jupyter/notebook, everything I did to set up the rails is gone!
My dockerfile
#Installs Jupyter Notebook and IPython kernel from the current branch
# Another Docker container should inherit with `FROM jupyter/notebook`
# to run actual services.
# For RVM support - ultimately do out own solution here.
FROM tzenderman/docker-rvm:latest
# Update aptitude
RUN apt-get update
# Install software
RUN apt-get install -y git supervisor libgmp3-dev
# Set up loggin for now
RUN mkdir -p /var/log/supervisor
# USING TOKENS ###################################################
ARG DEPLOYMENT_TOKEN
RUN git clone https://$DEPLOYMENT_TOKEN:x-oauth-basic#github.com/proversity-org/edx-api-jupyter.git /tmpapp/
RUN mkdir /home/sifu/
RUN cp -R /tmpapp/* /home/sifu/
RUN cp -R -r /tmpapp/. /home/sifu/
RUN chown root:root -R /home/sifu/
#################################################################
WORKDIR /home/sifu
# Install ruby using RVM
RUN /bin/bash -l -c "rvm install $(cat .ruby-version) --verify-downloads"
RUN /bin/bash -l -c "rvm use $(cat .ruby-version) --default"
RUN /bin/bash -l -c "rvm list"
RUN rvm requirements
# run docker env for ruby apps
RUN /bin/bash -l -c "source .docker-ruby-version"
RUN echo $RUBY-VERSION
ENV GEM_HOME /usr/local
ENV PATH /usr/local/rvm/gems/ruby-2.2.3/bin:$PATH
ENV PATH /usr/local/rvm/rubies/ruby-2.2.3/bin:$PATH
# Install Bundler
RUN ruby --version
RUN gem install bundler
RUN bundle config --global silence_root_warning 1
# Install Sifu gems
RUN bundle install --gemfile=/home/sifu/Gemfile
# Alow for arugments to sifu & notebook (server ip & port etc)
#RUN /bin/bash -l -c "which bundle"
#RUN /bin/bash -l -c "cp $(which ruby) /usr/bin/"
# Set up supervisor config -- move this up later
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# Set as environment variables
FROM jupyter/notebook
CMD ["/usr/bin/supervisord"]
ENTRYPOINT ["/usr/bin/supervisord", "-c", "/etc/myapp/supervisord.conf"]
All the software I installed is gone, when I use a FROM jupyter/notebook to include the notebooks in the image.
Is there something crucial I am missing?
Related
I am having issues deploying a Ruby on Rails App to ECS Fargate. When I build the image locally (the same way it is done in the pipeline). I can easily start the web service with this command ["bundle", "exec", "unicorn", "-c", "config/unicorn.rb"]. However, running this same image in Fargate returns this: 2022-07-27 15:46:33bundler: failed to load command: unicorn (/usr/local/bundle/bin/unicorn)
Here is my Docker file:
FROM ruby:2.6.7
RUN apt-get update -qq && apt-get install -y nodejs postgresql-client sudo
WORKDIR /app
ENV RAILS_ENV="staging"
ENV NODE_ENV="staging"
ENV LANG="en_US.UTF-8"
ENV RACK_ENV ="staging"
ENV BUNDLE_WITHOUT='development:test'
ARG GITHUB_TOKEN
RUN gem install bundler -v '2.2.28'
RUN bundle config https://github.com/somename/somerepo someuser:"${GITHUB_TOKEN}"
COPY . /app
RUN rm -rf /app/tmp
RUN mkdir -p /app/tmp
RUN bundle install
RUN bundle exec rails assets:precompile
EXPOSE 3000
The CMD is missing from Docker because its'a added in the task definition. Has anyone run into this issue? I've tried a number of different approaches, but am unsure of what is being changed locally running and running in Fargate.
Update
Looking into this issue further and found some more information that will need to be updated here.
I am using CircleCI to build and push this image to ECR. The issue seems to be that when created in CircleCI, any artifacts created by the bundle install become unreachable on run time and Docker is unable to run any gems because the GEM path is not even accessible to the root user. I pulled the image created by CircleCI Docker, locally, and confirmed the same errors. EXECing into the container and running chmod 755 -R /usr/local/bundle/bin/ and then executing the bundle exec to start the service, properly starts unicorn.
Next Steps
As a result, I attempted to add those changes into the Dockerfile on build and the same behavior still persists.
RUN bundle install
RUN chmod 755 -R /usr/local/bundle/bin/
Then I tried changing permissions in an entrypoint script and the container won't start at all.
Finally figured this out a few days ago. The answer is to add VOLUME arguments at the end of your Dockerfile. This will maintain persistence with any changes you have made. My final Dockerfile:
FROM ruby:2.6.7
ARG NPM_TOKEN
RUN apt-get update -qq && apt-get install -y \
build-essential \
curl \
postgresql-client \
software-properties-common \
sudo
RUN curl -fsSL https://deb.nodesource.com/setup_9.x | sudo -E bash - && \
sudo apt-get install -y nodejs
RUN apt-get install -y \
npm \
yarn
ENV BUNDLE_WITHOUT='development:test'
ENV RAILS_ENV="staging"
ENV RACK_ENV="staging"
ENV NPM_TOKEN="${NPM_TOKEN}"
RUN mkdir -p /dashboard
WORKDIR /dashboard
RUN mkdir -p /dashboard/tmp/pids
COPY Gemfile* ./
COPY gems/rails_admin_history_rollback /dashboard/gems/rails_admin_history_rollback
ARG GITHUB_TOKEN
RUN sudo gem install bundler -v '2.2.28' && \
bundle config https://github.com/some-company/some-repo some-name:"${GITHUB_TOKEN}"
RUN bundle install
COPY . /dashboard
RUN bundle exec rails assets:precompile
RUN chmod +x bin/entrypoint.sh
VOLUME /dashboard/
VOLUME /usr/local/bundle
EXPOSE 3000
I am running a docker image and whenever I make any new changes inside the app folder such as in controllers or views the changes are not relfected in browser. Everytime I need to create a new image inorder to reflect the changes.
Dockerfile
FROM ruby:3.0
USER root
RUN apt-get update && apt-get install -y build-essential libpq-dev nodejs
RUN rm -rf .idea \
rm -rf .gitignore
ENV LC_ALL=en_US.UTF-8
ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US.UTF-8
RUN mkdir -p /app
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN gem install bundler && bundle install --jobs 20 --retry 5
COPY . ./
# Adding a non root user
RUN groupadd --system non-root-user
RUN useradd -u 1001 --system non-root-user -g non-root-user -d /home/non-root-user -m -s /bin/bash
RUN chown -R non-root-user /app
RUN chmod -R 777 /app
# Switching to non-root user 'non-root-user'
USER non-root-user
# Exposing port 3010
EXPOSE 3010
# Default entry point (i.e. will be appened before every command being executed)
ENTRYPOINT ["sh", "./config/docker/startup.sh"]
startup.sh
#! /bin/sh
echo "Creating tmp/pids..."
mkdir -p tmp/pids/
# Start Application
echo "Starting up puma(app server)..."
bundle exec puma -p 3010 -C ./config/puma.rb
I have updated the below configuration in my development.rb still no success
config.file_watcher = ActiveSupport::FileUpdateChecker
Ruby - 3.0.0
Rails - 6.1
I am running the container with the below command
docker run -it -p 3010:3010 -e RAILS_ENV=development my_image
scenario1
So the problem I am facing like after building the docker images. If I am going inside the docker container and doing GEM_PATH=$GEM_HOME gem list it is not showing all the gems installed at the time of building the image. it is showing only a few gems.
scenario2
If I am not sourcing the RVM path inside the Dockerfile. Then If I am going inside the docker container and doing GEM_PATH=$GEM_HOME gem list then it is showing all the gems.
Can anyone please explain to me why this is happening and how I should install and source RVM? So I can see all the gems inside the container. Basically, I want the scenario1 should work. Thanks
Below is my Dockerfile
FROM ruby:2.6.5
RUN apt-get update -qq && apt-get install -y sudo && apt-get install -y build-essential && apt-get install -y apt-utils && apt-get install -y git
RUN gpg --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB && \curl -sSL https://get.rvm.io | sudo bash -s stable --ruby && echo 'source /usr/local/rvm/scripts/rvm' >> /etc/bash.bashrc
RUN /bin/bash -l -c 'gem install bundler -v 1.17.3'
RUN /bin/bash -l -c 'gem install rails --version=5.2.4.1'
WORKDIR /app
RUN bundle config build.nokogiri --use-system-libraries
RUN bundle config set path 'vendor/bundle'
COPY Gemfile Gemfile.lock ./
ARG SSH_KEY
# Make ssh dir
RUN mkdir /root/.ssh/
# Copy ssh
RUN echo "$SSH_KEY" > /root/.ssh/id_rsa && \
chmod 0600 /root/.ssh/id_rsa
RUN touch /root/.ssh/known_hosts
# Add bitbuckets key
RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts
RUN bundle install
COPY . ./
EXPOSE 80:80
CMD rails s -p 80 -b '0.0.0.0' -e qa
TL;DR: don't use rvm\rbenv within a container
the whole thing when using containers, is to bundle\package all the dependencies inside a container. for instance, if you need ruby 2.9 then use a docker image which has ruby 2.9 installed rather than using rvm[rbenv](https://github.com/rbenv/rbenv).
since you are using bundler, i will advise you to add rails gem to it, and let bundler manage the dependencies.
i assume that you care about the directory where rubygems are being installed for caching propose, and i see that you instruct bundler exactly what is the rubygem home directory
RUN bundle config set path 'vendor/bundle'
We are running sphinx through supervisord inside docker. We are trying to enable autorestart, but it does work. We are trying to verify this by manually killing the searchd process.
Below is the configuration
[program:sphinx]
command=searchd --pidfile --config "config/sphinx.conf"
autostart=true
autorestart=unexpected
startsecs=5
startretries=3
exitcodes=0
We also attempted other configuration
program:sphinx]
command= rake ts:stop
command= rake ts:configure
command= rake ts:start
autostart=true
exitcodes=0,2
autorestart=unexpected
Are we missing something?
Dockerfile
FROM ruby:2.3
RUN apt-get update
#Install Sphinx
RUN wget -P /tmp http://sphinxsearch.com/files/sphinx-2.3.2-beta.tar.gz
RUN mkdir /opt/sphinx_src
RUN tar -xzvf /tmp/sphinx-2.3.2-beta.tar.gz -C /opt/sphinx_src
WORKDIR /opt/sphinx_src/sphinx-2.3.2-beta
RUN ./configure --with-pgsql --with-mysql
RUN make
RUN make install
RUN apt-get install -q -y supervisor cron
ADD supervisor-cron.conf /etc/supervisor/conf.d/cron.conf
RUN service supervisor stop
RUN apt-get install -y net-tools telnet
WORKDIR /opt/app
ADD start.sh /usr/local/sbin/start
RUN chmod 755 /usr/local/sbin/start
EXPOSE 9312
CMD ["/usr/local/sbin/start"]
start.sh
#!/bin/sh
export set BUNDLE_GEMFILE=Gemfile
cd /opt/app
bundle install
echo "About to perform Sphinx Start"
cp /opt/app/supervisor-sphinx.conf /etc/supervisor/conf.d/sphinx.conf
supervisord -n
I have a web application that uses the gulp-ruby-sass plugin and it runs within a container that installs the sass gem.
This is the Dockerfile:
FROM phusion/passenger-customizable:0.9.19
# Set correct environment variables.
ENV HOME /root
# Use baseimage-docker's init process.
CMD ["/sbin/my_init"]
# Build system and git.
RUN /pd_build/utilities.sh && \
# Ruby support.
/pd_build/ruby-2.3.*.sh && \
# Node.js and Meteor support.
/pd_build/nodejs.sh && \
npm install -g gulp jspm yarn && \
npm install phantomjs-prebuilt#2.1.9 && \
mv node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs /usr/bin && \
rm -r node_modules && \
# Clean up APT when done.
apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
EXPOSE 9000
WORKDIR /var/www/
CMD /bin/bash
This is the bootstrap script file (to bootstrap the application):
#! /bin/bash
bundle install
npm install
jspm install
npm config set bin-links false
npm rebuild node-sass
gulp watch
This is the Gemfile:
source "https://rubygems.org"
gem "sass"
And I launch the container this way:
docker run -it --name web -p 9000:9000 --volume $(pwd):/var/www brainy_web sh bootstrap.sh
The problem is that when the gulp task gulp build-css is executed using docker-compose, it doesn't find the sass gem as it is shown in the log: Gem sass is not installed (complete log here), but when I run the container with bash and execute the bootstrap.sh file everything runs smoothly.
Any thoughts on how to use sass with docker and gulp-ruby-sass?
UPDATE
I found that executing from the outside the gulp task, it doesn't find either:
docker exec web gulp build-css
I found the solution! I followed the advice from #webert-s-lima and in here: https://stackoverflow.com/a/29999734/532912
I just added the -l argument when launching the container. The statement would be like this:
docker run -it --name web -p 9000:9000 --volume $(pwd):/var/www brainy_web /bin/bash -l bootstrap.sh