docker-compose error connecting to redis + sidekiq - ruby-on-rails

Question

I am trying to build a container with docker but I cannot connect sidekiq + redis, the error says sidekiq_1 | Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED), seems to sidekiq is trying to connect to localhost but since I am building "in theory" redis + sidekiq + rails + postgres containers is not in the localhost, it should be in redis image.
My docker-compose.yml file is this:
version: '3'
services:
postgres:
image: postgres:10.5
volumes:
- my_app-postgres:/var/lib/postgresql/data
redis:
image: redis:4.0.11
volumes:
- my_app-redis:/var/lib/redis/data
web:
build: .
command: bundle exec rails server -p 3000 -b '0.0.0.0'
ports:
- '3000:3000'
depends_on:
- postgres
- redis
volumes:
- .:/my_app
env_file:
- .env
sidekiq:
build: .
command: bundle exec sidekiq -C config/sidekiq.yml
volumes:
- .:/my_app
depends_on:
- postgres
- redis
env_file:
- .env
volumes:
my_app-postgres:
my_app-redis:
another interesting 'info' I see in log is Booting Sidekiq 4.2.10 with redis options {:url=>nil} this url can be the cause of the issue?
in my development environment the app is working fine, I try to 'dockerize' what I have. How could I make this work?

Answer 1

By default, sidekiq tries to connect to 127.0.0.1:6379 but your sidekiq is in a different container than redis, so you need to configure sidekiq to use redis:6379 as redis host, e.g. by using an initializer:
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://redis:6379/12' }
end
Take a look at the docs for more details: https://github.com/mperham/sidekiq/wiki/Using-Redis
If you are planning on using Kubernetes for deployment later on, you can put all containers in a pod and then they would be able to connect via localhost because containers within the same Kubernetes pod share the network space. To program directly within a pod inside a Kubernetes cluster, you could work with a tool I recently open sourced on GitHub called DevSpace: https://github.com/covexo/devspace

Answer 2

Create two initializer files:
i) redis.rb
uri = "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0/your-app-cache" || 'redis://localhost:6379/0/your-app-cache'
Rails.application.config.cache_store = :redis_store, uri
ii) sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Sidekiq.configure_client do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end

Answer 3

Full Sample
./Dockerfile
FROM ruby:2.6.3-alpine
ENV BUNDLER_VERSION=2.0.2
RUN apk add --update --no-cache \
binutils-gold \
build-base \
curl \
file \
g++ \
gcc \
git \
less \
libstdc++ \
libffi-dev \
libc-dev \
linux-headers \
libxml2-dev \
libxslt-dev \
libgcrypt-dev \
make \
netcat-openbsd \
nodejs \
openssl \
pkgconfig \
postgresql-dev \
python \
tzdata \
yarn
ARG USER=root
ARG WORK_DIR_PATH=/home
RUN mkdir -p $WORK_DIR_PATH && chown -R $USER:$USER $WORK_DIR_PATH
WORKDIR $WORK_DIR_PATH
COPY Gemfile* ./
RUN gem install bundler
RUN bundle config build.nokogiri --use-system-libraries
RUN bundle check || bundle install
COPY package.json yarn.lock ./
RUN yarn install --check-files
COPY . .
./.env
APP_NAME=api
APP_PORT=3100
ENV=production
DATABASE_NAME=rails_db
DATABASE_USER=batman
DATABASE_PASSWORD=super_pass_123
DATABASE_PORT=5342
DATABASE_HOST=api_db # must be equal to the name of the postgres service in docker-compose.yml
SECRET_KEY_BASE=your_secret_string
REDIS_HOST=redis # must be equal to the name of the redis service in docker-compose.yml
REDIS_PORT=6379
./docker-compose.yml
version: '3.7'
services:
api:
build:
context: .
dockerfile: Dockerfile
container_name: ${APP_NAME}
#restart: unless-stopped
depends_on:
- api_db
- redis
ports:
- "${APP_PORT}:${APP_PORT}"
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
entrypoint: ./sh/entrypoints/api-entrypoint.sh
api_db:
image: postgres
command: postgres -p ${DATABASE_PORT}
ports:
- "${DATABASE_PORT}:${DATABASE_PORT}"
volumes:
- db_data:/var/lib/postgresql/data
- ./log/db:/logs
environment:
- POSTGRES_USER=${DATABASE_USER}
- POSTGRES_PASSWORD=${DATABASE_PASSWORD}
- POSTGRES_DB=${DATABASE_NAME}
redis:
image: redis
ports:
- "${REDIS_PORT}:${REDIS_PORT}"
command: redis-server
volumes:
- redis:/data
sidekiq:
build:
context: .
dockerfile: Dockerfile
depends_on:
- api_db
- redis
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
ENABLE_BOOTSNAP: 'false'
entrypoint: ./sh/entrypoints/sidekiq-entrypoint.sh
volumes:
redis:
gem_cache:
db_data:
node_modules:
./sh/entrypoints/api-entrypoint.sh
https://stackoverflow.com/a/59047028/4488252
#!/bin/sh
DB_INITED=0
if db_version=$(bundle exec rake db:version 2>/dev/null)
then
if [ "$db_version" = "Current version: 0" ]
then
echo "DB is empty"
else
echo "DB exists"
DB_INITED=1
fi
bundle exec rake db:migrate
else
echo "DB does not exist"
bundle exec rake db:setup
fi
if [ $DB_INITED == 0 ]
then
echo "Performing initial configuration"
# init some plugins, updated db if need, add initial data
fi
bundle exec rails assets:precompile
bundle exec rails s -b 0.0.0.0 -p $APP_PORT
./sh/entrypoints/sidekiq-entrypoint.sh
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle exec sidekiq
./config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
database: <%= ENV['DATABASE_NAME'] %>
username: <%= ENV['DATABASE_USER'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
port: <%= ENV['DATABASE_PORT'] || '5432' %>
host: <%= ENV['DATABASE_HOST'] %>
development:
<<: *default
test:
<<: *default
production:
<<: *default
secret_key_base: <%= ENV['SECRET_KEY_BASE'] %>
./config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
Sidekiq.configure_client do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
./.dockerignore
https://gist.github.com/neckhair/ace5d1679dd896b71403fda4bc217b9e
.git
.gitignore
README.md
#
# OS X
#
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear on external disk
.Spotlight-V100
.Trashes
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
#
# Rails
#
.env
.env.sample
*.rbc
capybara-*.html
log
tmp
db/*.sqlite3
db/*.sqlite3-journal
public/system
coverage/
spec/tmp
**.orig
.bundle
.ruby-version
.ruby-gemset
.rvmrc
# if using bower-rails ignore default bower_components path bower.json files
vendor/assets/bower_components
*.bowerrc
bower.json
# Logs
logs
*.log
# Runtime data
pids
*.pid
*.seed
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# node-waf configuration
.lock-wscript
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directory
# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git
node_modules
server/*.spec.js
kubernetes
Usage
https://docs.docker.com/compose/reference/down/
build and run: docker-compose up --build -d
https://docs.docker.com/compose/reference/down/
stop: docker-compose down
stop + delete images and volumes: docker-compose down --rmi all --volumes