I'm currently using whenever gem for running my Sidekiq worker. It's perfectly fine when I'm running the Sidekiq worker but when I run it in background using cron it's not working.
Dockerfile
FROM ruby:2.6.0-slim
RUN apt-get update -qq \
&& apt-get install -y \
# Needed for certain gems
build-essential \
# Needed for postgres gem
libpq-dev \
# install cron for cronjobs
cron \
# Others
nodejs \
vim-tiny \
# The following are used to trim down the size of the image by removing unneeded data
&& apt-get clean autoclean \
&& apt-get autoremove -y \
&& rm -rf \
/var/lib/apt \
/var/lib/dpkg \
/var/lib/cache \
/var/lib/log
RUN mkdir /app
WORKDIR /app
COPY Gemfile /app/Gemfile
COPY Gemfile.lock /app/Gemfile.lock
RUN bundle install
ADD . /app
RUN bundle exec whenever --update-crontab --set environment='development' && /etc/init.d/cron restart
CMD bash -c "rm -f tmp/pids/server.pid && rails s -p 3000 -b '0.0.0.0'"
docker-compose.yml
version: '3'
services:
db:
image: postgres:alpine
volumes:
- ./tmp/db:/var/lib/postgresql/data
ports:
- "5432:5432"
redis:
image: redis:5.0.7
command: redis-server
ports:
- "6379:6379"
sidekiq:
build: .
command: bundle exec sidekiq
depends_on:
- redis
volumes:
- .:/app
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/app
ports:
- "3000:3000"
depends_on:
- db
- redis
- sidekiq
schedule.rb
set :environment, "development"
every 1.minute do
runner "User.create"
end
When I run docker-compose build then docker-compose up and I access the web image using docker exec -it <container-id> sh
crontab -e returns
# Begin Whenever generated tasks for: /app/config/schedule.rb at: 2019-12-16 03:00:58 +0000
* * * * * /bin/bash -l -c 'cd /app && bundle exec bin/rails runner -e development '\''User.create'\'''
# End Whenever generated tasks for: /app/config/schedule.rb at: 2019-12-16 03:00:58 +0000
It should create a user every minute. I tried running this manually and it managed to create a user. I wonder why it's not running from cron
Related
Aasumption
I m using docker-compose and wanted to use the whenever gem to do a cron process that deletes at a certain time in Rails, but upon research I found that I have to install and run cron in docker. So I looked into it, but I can't find any information about Alpine regarding cron processing in Rails. Can an
yone tell me how to do this?
What we have achieved
I want to execute a specific process once a day.
Code
Here is my Dockerfile:
FROM ruby:2.7.1-alpine
ARG WORKDIR
ENV RUNTIME_PACKAGES="linux-headers libxml2-dev make gcc libc-dev nodejs tzdata postgresql-dev postgresql git" \
DEV_PACKAGES="build-base curl-dev" \
HOME=/${WORKDIR} \
LANG=C.UTF-8 \
TZ=Asia/Tokyo
RUN echo ${HOME}
WORKDIR ${HOME}
COPY Gemfile* ./
RUN apk update && \
apk upgrade && \
apk add --no-cache ${RUNTIME_PACKAGES} && \
apk add --virtual build-dependencies --no-cache ${DEV_PACKAGES} && \
bundle install -j4 && \
apk del build-dependencies
COPY . .
CMD ["rails", "server", "-b", "0.0.0.0"]
Here is my Docker Compose file:
version: '3.8'
services:
db:
image: postgres:12.3-alpine
environment:
TZ: UTC
PGTZ: UTC
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
volumes:
- ./api/tmp/db:/var/lib/postgresql/data
api:
build:
context: ./api
args:
WORKDIR: $WORKDIR
command: /bin/sh -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
environment:
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
API_DOMAIN: "localhost:$FRONT_PORT"
APP_URL: "http://localhost:$API_PORT"
volumes:
- ./api:/$WORKDIR
depends_on:
- db
ports:
- "$API_PORT:$CONTAINER_PORT"
mailcatcher:
image: schickling/mailcatcher
ports:
- "1080:1080"
- "1025:1025"
front:
build:
context: ./front
args:
WORKDIR: $WORKDIR
CONTAINER_PORT: $CONTAINER_PORT
API_URL: "http://localhost:$API_PORT"
command: yarn run dev
volumes:
- ./front:/$WORKDIR
ports:
- "$FRONT_PORT:$CONTAINER_PORT"
depends_on:
- api
Actual processing
/config/schedule.rb
require File.expand_path(File.dirname(__FILE__) + "/environment")
ENV.each { |k, v| env(k, v) }
set :output, "#{Rails.root}/log/cron.log"
set :environment, :development
every 1.days do
runner "User.guest_reset"
end
What we tried
I did a lot of research and found a lot of information on using cron with apt, but could not find any information on using apk.
Separate cron into another service in your docker-compose.yml by using the same image as your rails app image (which built by Dockerfile). Then run cron and whenever --update-crontab in the command.
docker-compose.yml
version: '3'
services:
app:
image: myapp
depends_on:
- 'db'
build:
context: .
command: bash -c "rm -f tmp/pids/server.pid &&
bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- ".:/myapp"
cron:
image: myapp
command: bash -c "touch log/cron.log && cron && whenever --update-crontab &&
crontab -l && tail -f log/cron.log"
volumes:
- '.:/myapp'
db:
image: postgres:13
ports: # 127.0.0.1 to only expose the port to loopback
- '127.0.0.1:5432:5432'
volumes:
- 'postgres_dev:/var/lib/postgresql/data'
Dockerfile
FROM ruby:3.0.1
RUN apt-get update -qq && apt-get install -y postgresql-client cron vim \
&& mkdir /myapp
WORKDIR /myapp
ENV BUNDLE_WITHOUT=development:test
COPY Gemfile Gemfile.lock ./
RUN bundle install --jobs 20 --retry 5
COPY package.json ./
RUN npm install --check-files --production
I have project which used docker-compose to provide environment to developers. The application is running fine on docker-compose build command and running on 0.0.0.0:3000 on docker-compose up command. When I am trying to run the command docker-compose run web rails g uploader or docker-compose run web rails g migration it's show in console thats they successfuly create but when I check project there are no files.
This is my Dockerfile:
# Base image
FROM ruby:2.7.0
# Set enviroment variables in docker
ENV INSTALL_PATH=/app \
RAILS_ENV=production \
RACK_ENV=$RAILS_ENV \
RAILS_LOG_TO_STDOUT=true \
RAILS_SERVE_STATIC_FILES=true \
SECRET_KEY_BASE=ad187ccccdf25beb51568211a26b0bff237385d79df37e08151acda85266f9a469f37926450ba18d9362ec5e83d1b612c09368bc59dc895cb5ce2798a3ab456b
RUN env
# Ensure gems are cached and only get updated when they change. This will
# drastically increase build times when your gems do not change.
ADD Gemfile* $INSTALL_PATH/
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list \
&& apt-get update \
&& apt-get install -qq -y build-essential nodejs libpq-dev cron htop vim sqlite3 yarn imagemagick netcat --fix-missing --no-install-recommends \
&& cd $INSTALL_PATH; bundle install --jobs 20 --retry 5
WORKDIR $INSTALL_PATH
ADD . .
RUN mv config/database.docker.yml config/database.yml \
# Fix windows line ending from windows runners
&& find ./ -type f -exec sed -i 's/\r$//' {} + \
&& chmod +x docker/* \
&& yarn install --check-files \
&& RAILS_ENV=$RAILS_ENV bundle exec rails assets:precompile \
&& chown -R nobody:nogroup $INSTALL_PATH
USER nobody
# Expose a volume so that nginx will be able to read in assets in production.
VOLUME ["$INSTALL_PATH/public"]
EXPOSE 3000
CMD ["docker/startup.sh"]
This one is my docker-compose.yml:
version: '2'
volumes:
database_data:
driver: local
web_rails_public: {}
services:
web:
restart: always
image: eu.gcr.io/academic-ivy-225422/joystree_web
container_name: joystree_web_app_container
build: .
volumes:
- web_rails_public:/app/public
env_file:
- '.env.web'
ports:
- "3000:3000"
links:
- "database:database"
depends_on:
- database
database:
restart: always
container_name: joystree_postgres_container
image: postgres:11
env_file:
- '.env.db'
ports:
- "5432:5432"
volumes:
- database_data:/var/lib/postgresql/data
I had this same problem and solved it by following these steps:
1 - in Dockerfile add the fallowing lines, for be able to create new files:
RUN mkdir /home/web \
&& chown $(id -un):$(id -gn) /home/web
WORKDIR /home/web
2 - in docker-compose.yml web the volume should be .:/home/web or just the same as you call in mkdir
web:
volumes:
- .:/home/web
I hope that solve your problem too
I am new to docker and was trying to integrate docker with my rails app. I was following this doc
I was able to setup the services but I am facing a problem.
Whenever I add a Gem to my Gemfile I get error in docker-compose up saying
sidekiq_1 | bundler: failed to load command: sidekiq (/usr/local/bundle/bin/sidekiq)
sidekiq_1 | /usr/local/bundle/gems/bundler-2.2.3/lib/bundler/resolver.rb:309:in `block in verify_gemfile_dependencies_are_found!': Could not find gem 'awesome_print' in any of the gem sources listed in your Gemfile. (Bundler::GemNotFound)
Dockerfile
FROM ruby:3.0.0-alpine3.12
ENV BUNDLER_VERSION=2.2.3
RUN apk add --update --no-cache \
binutils-gold \
build-base \
curl py-pip \
curl \
file \
g++ \
gcc \
git \
less \
libstdc++ \
libffi-dev \
libc-dev \
linux-headers \
libxml2-dev \
libxslt-dev \
libgcrypt-dev \
make \
netcat-openbsd \
nodejs \
openssl \
pkgconfig \
python3 \
tzdata \
yarn
RUN gem install bundler -v 2.2.3
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN bundle config build.nokogiri --use-system-libraries
RUN bundle check || bundle install
COPY package.json yarn.lock ./
RUN yarn install --check-files
COPY . ./
ENTRYPOINT ["./entrypoints/docker-entrypoint.sh"]
docker-compose.yml
version: '3.6'
services:
app:
build:
context: .
dockerfile: Dockerfile
container_name: sna-main
depends_on:
- redis
ports:
- "3000:3000"
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
environment:
RAILS_ENV: development
redis:
image: redis:5.0.7
sidekiq:
build:
context: .
dockerfile: Dockerfile
depends_on:
- app
- redis
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
environment:
RAILS_ENV: development
entrypoint: ./entrypoints/sidekiq-entrypoint.sh
volumes:
gem_cache:
node_modules:
It would be great if someone can help me out to understand how to run bundle install every time there is a change in Gemfile.
From the doc I have understood that because we are caching the Gems so we need to remove the volume explicitly if we add/remove a new gem.
I did remove the volumes by running docker-compose down -v.
It removed the volumes but still getting the same error.
There's a chance that this is because docker-compose cached your images; you need to re-build with docker-compose up --build to reflect the image changes in docker-compose.
I got it work by adding bundle install in my entrypoint file
----------------./entrypoints/docker-entrypoint.sh-----------------
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle install
bundle exec rails s -b 0.0.0.0
----------------./entrypoints/sidekiq-entrypoint.sh-----------------
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle install && bundle exec sidekiq -C config/sidekiq.yml
I tried the approach with tty: true stdin_open: true inside docker-compose.yml and attaching to the container id (following http://www.chris-kelly.net/2016/07/25/debugging-rails-with-pry-within-a-docker-container/) but it just hangs.
I also tried docker-compose run --service-ports web following this article https://blog.carbonfive.com/2015/03/17/docker-rails-docker-compose-together-in-your-development-workflow/ but it also hangs the request when binding.pry
Could supervisord affect this?
Here's the Dockerfile:
FROM ruby:2.3.3
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev supervisor
RUN curl -sL https://deb.nodesource.com/setup_9.x | bash - && apt-get install -yq nodejs
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install yarn
RUN mkdir /example
WORKDIR /example
COPY Gemfile /example/Gemfile
COPY Gemfile.lock /example/Gemfile.lock
RUN bundle install
COPY . /example
COPY docker/supervisor.conf /etc/supervisor/conf.d/example.conf
RUN cd client-app && npm install
CMD supervisord -n
And the docker-compose.yml:
version: '3'
services:
db:
image: postgres
web:
build: .
environment:
API_HOST: http://localhost:3000/api
volumes:
- .:/example
ports:
- "3000:3000"
- "4200:4200"
depends_on:
- db
And the supervisor.conf:
[program:rails]
directory=/example
command=rails server -b 0.0.0.0 -p 3000
autostart=true
autorestart=true
[program:npm]
directory=/example
command=/bin/bash -c "yarn && cd client-app && npm run docker-start"
autostart=true
autorestart=true
(assuming you're running postgres something like docker run -d --name=postgres ...)
I would ditch compose and try
docker build -t web .
docker run --link postgres:db -it -p 3000:3000 -p 4200:4200 web
If that fails, I'd punt and
docker run --link postgres:db -it -p 3000:3000 -p 4200:4200 web bash
Then try running rails s, rails c, etc manually within the container.
Since I installed Capybara-webkit, I can't launch my specs with docker compose. The next command stays on hold:
$ docker-compose run web xvfb-run -a bundle exec rspec
I thought I have a problem with Capybara-webkit, so I created a SO question and an issue on the repo, but it seems it's more a pb of interaction between docker-compose and xvfb.
If I do first
$ docker-compose run web bash
then
$ xvfb-run -a bundle exec rspec spec
it works fine. I have no clue.
Edit 31/08/17
As requested, here is the docker-compose file:
version: '2'
services:
db:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=XXXXX
volumes:
- mysql-data:/var/lib/mysql
redis:
image: redis
ports:
- "6379:6379"
volumes:
- redis:/data
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/app_dir
- app-gems:/usr/local/bundle
ports:
- "3000:3000"
depends_on:
- db
- redis
volumes:
mysql-data:
driver: local
redis:
driver: local
app-gems:
driver: local
And the Dockerfile:
FROM ruby:2.4.1
RUN apt-get update -qq && apt-get install -y \
build-essential \
libpq-dev \
nodejs \
xvfb \
qt5-default \
libqt5webkit5-dev \
gstreamer1.0-plugins-base \
gstreamer1.0-tools \
gstreamer1.0-x
RUN mkdir /app_dir
WORKDIR /app_dir
ADD Gemfile* /app_dir/
RUN bundle install
COPY . .
In docker-compose.yml
command: ./start.sh
And in start.sh file
#!/bin/bash
xvfb-run "run whatever"
Posting comments as answer since I need formatting
Can you try changing below
command: bundle exec rails s -p 3000 -b '0.0.0.0'
to
entrypoint: xvfb-run -a bundle exec rspec
and try docker-compose up
Also if that doesn't work then try adding tty: true to the service