rake operations with database using docker - ruby-on-rails

With the following files I can start rails application successfully
except rake operations with database. When I run
docker exec -it www_web_1 bundle exec rake db:migrate
I saw that it tried to connect to localhost (web) and fail
instead of conntecing to database host (db)
Any ideas how to solve it?
Dockerfile
FROM ruby:2.4.2
RUN mkdir -p /var/www/project
WORKDIR /var/www/project
COPY Gemfile /var/www/project/Gemfile
RUN bundle install
COPY entrypoint.sh /usr/local/bin
COPY . /var/www/project
CMD ["entrypoint.sh"]
entrypoint.sh
#!/usr/bin/env sh
bundle exec rake db:migrate
bundle exec rails s -p 3000 -b '0.0.0.0'
docker-compose.yml file
version: '2'
services:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: project
web:
depends_on:
- db
environment:
- PROJECT_DB_NAME=project
- PROJECT_DB_USER=root
- PROJECT_DB_PSWD=root
- PROJECT_DB_HOST=db
- RACK_ENV=production
- RAILS_ENV=production
volumes:
mysql:
web:
config/database.yml
default: &default
adapter: mysql2
pool: 5
timeout: 5000
database: <%= ENV["PROJECT_DB_NAME"] %>
encoding: utf8
username: <%= ENV["PROJECT_DB_USER"] %>
password: <%= ENV["PROJECT_DB_PSWD"] %>
host: <%= ENV["PROJECT_DB_HOST"] %>
development:
<<: *default
production:
<<: *default

You have compose file.
you have to run compose command
$ docker-compose up --build # run and build
$ docker-compose run web rake db:create # database create
$ docker-compose run web rake db:migrate # migreate
if you'd like more know docker-compose
visit https://github.com/x1wins/paper-flower/wiki/2.-Docker--compose-(Development-mode,-Standalone)

Related

db issues when dockerizing Ruby on Rails application

I'm trying to dockerize an existing rails application. But it's running into an error when trying to invoke rake db:create.
Here is the error:
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Couldn't create database for {"adapter"=>"postgresql", "encoding"=>"unicode", "pool"=>5,
"database"=>"app_development", "username"=>"postgres", "password"=>"postgres"}
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Here is my Dockerfile
FROM ruby:2.7.2
RUN apt-get update -qq && apt-get install -y nodejs postgresql-client
WORKDIR /app
COPY Gemfile /app/Gemfile
COPY Gemfile.lock /app/Gemfile.lock
RUN bundle install
COPY . /app
# COPY entrypoint.sh /usr/bin/
# RUN chmod +x /usr/bin/entrypoint.sh
# ENTRYPOINT ["entrypoint.sh"]
EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0"]
docker-compose.yml
version: "3.9"
services:
db:
image: postgres
volumes:
- db_data:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: app_development
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
volumes:
db_data:
database.yml
...
development:
<<: *default
database: app_development
# url: postgres://...
username: postgres
password: postgres
What am I doing wrong? I've tried everything online and still no luck. Any catches? Thanks in advance.
I think you will need to communicate over TCP/IP rather than a socket, so you should be able to sort this using the host setting in your database.yml file
...
development:
<<: *default
username: postgres
database: postgres
host: 127.0.0.1

Cannot establish connection between Rails and Postgres Docker containers

I am new to Docker and try to dockerize a Rails container and a Postgres container with docker-compose and both build and start but when I try to go to localhost:3009 to see the "Yay, you're on Rails" page, I get this error in the Rails logs:
Started GET "/" for 172.27.0.1 at 2020-10-05 12:40:18 +0000
backend_1 | Cannot render console from 172.27.0.1! Allowed networks: 127.0.0.0/127.255.255.255, ::1
backend_1 |
backend_1 | PG::ConnectionBad (could not connect to server: Connection refused
backend_1 | Is the server running on host "postgres" (172.27.0.2) and accepting
backend_1 | TCP/IP connections on port 5439?
backend_1 | ):
This is my docker-compose.yml:
version: '3'
volumes:
db_data:
driver: local
app_data:
driver: local
services:
# database container
postgres:
image: postgres
volumes:
- app_data:/var/lib/postgresql/data
ports:
- 5439:5432
environment:
POSTGRES_DB: db_name
POSTGRES_USER: user
POSTGRES_PASSWORD: xxxxxx
command: ["postgres", "-c", "log_statement=all"]
restart: always
app:
build: ./server
volumes:
- ./server:/code
ports:
- "3009:3000"
depends_on:
- postgres
environment:
RAILS_ENV: development
DATABASE_HOST: postgres
DATABASE_PORT: 5439
DATABASE_NAME: db_name
DATABASE_USERNAME: user
DATABASE_PASSWORD: xxxxxx
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
And this is the Dockerfile:
FROM ruby:2.7.1
WORKDIR /app
COPY . /app
# Install NodeJS and Yarn.
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y yarn
RUN yarn install --check-files
RUN bundle install
CMD ["rails", "server", "-b", "0.0.0.0"]
This is my database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
host: <%= ENV['DATABASE_HOST'] %>
port: <%= ENV['DATABASE_PORT'] %>
database: <%= ENV['DATABASE_NAME'] %>
username: <%= ENV['DATABASE_USERNAME'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
development:
<<: *default
test:
<<: *default
I can access the pg database in DBeaver with 172.27.0.1:5439, but not with 172.27.0.2:5439 and I have no clue why the Rails container tries to connect on that address.
As David Maze pointed out:
You need to set DATABASE_PORT to the ordinary PostgreSQL port 5432; ports: are ignored (and unnecessary) for inter-container communications.

docker-compose error connecting to redis + sidekiq

I am trying to build a container with docker but I cannot connect sidekiq + redis, the error says sidekiq_1 | Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED), seems to sidekiq is trying to connect to localhost but since I am building "in theory" redis + sidekiq + rails + postgres containers is not in the localhost, it should be in redis image.
My docker-compose.yml file is this:
version: '3'
services:
postgres:
image: postgres:10.5
volumes:
- my_app-postgres:/var/lib/postgresql/data
redis:
image: redis:4.0.11
volumes:
- my_app-redis:/var/lib/redis/data
web:
build: .
command: bundle exec rails server -p 3000 -b '0.0.0.0'
ports:
- '3000:3000'
depends_on:
- postgres
- redis
volumes:
- .:/my_app
env_file:
- .env
sidekiq:
build: .
command: bundle exec sidekiq -C config/sidekiq.yml
volumes:
- .:/my_app
depends_on:
- postgres
- redis
env_file:
- .env
volumes:
my_app-postgres:
my_app-redis:
another interesting 'info' I see in log is Booting Sidekiq 4.2.10 with redis options {:url=>nil} this url can be the cause of the issue?
in my development environment the app is working fine, I try to 'dockerize' what I have. How could I make this work?
By default, sidekiq tries to connect to 127.0.0.1:6379 but your sidekiq is in a different container than redis, so you need to configure sidekiq to use redis:6379 as redis host, e.g. by using an initializer:
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://redis:6379/12' }
end
Take a look at the docs for more details: https://github.com/mperham/sidekiq/wiki/Using-Redis
If you are planning on using Kubernetes for deployment later on, you can put all containers in a pod and then they would be able to connect via localhost because containers within the same Kubernetes pod share the network space. To program directly within a pod inside a Kubernetes cluster, you could work with a tool I recently open sourced on GitHub called DevSpace: https://github.com/covexo/devspace
Create two initializer files:
i) redis.rb
uri = "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0/your-app-cache" || 'redis://localhost:6379/0/your-app-cache'
Rails.application.config.cache_store = :redis_store, uri
ii) sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Sidekiq.configure_client do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Full Sample
./Dockerfile
FROM ruby:2.6.3-alpine
ENV BUNDLER_VERSION=2.0.2
RUN apk add --update --no-cache \
binutils-gold \
build-base \
curl \
file \
g++ \
gcc \
git \
less \
libstdc++ \
libffi-dev \
libc-dev \
linux-headers \
libxml2-dev \
libxslt-dev \
libgcrypt-dev \
make \
netcat-openbsd \
nodejs \
openssl \
pkgconfig \
postgresql-dev \
python \
tzdata \
yarn
ARG USER=root
ARG WORK_DIR_PATH=/home
RUN mkdir -p $WORK_DIR_PATH && chown -R $USER:$USER $WORK_DIR_PATH
WORKDIR $WORK_DIR_PATH
COPY Gemfile* ./
RUN gem install bundler
RUN bundle config build.nokogiri --use-system-libraries
RUN bundle check || bundle install
COPY package.json yarn.lock ./
RUN yarn install --check-files
COPY . .
./.env
APP_NAME=api
APP_PORT=3100
ENV=production
DATABASE_NAME=rails_db
DATABASE_USER=batman
DATABASE_PASSWORD=super_pass_123
DATABASE_PORT=5342
DATABASE_HOST=api_db # must be equal to the name of the postgres service in docker-compose.yml
SECRET_KEY_BASE=your_secret_string
REDIS_HOST=redis # must be equal to the name of the redis service in docker-compose.yml
REDIS_PORT=6379
./docker-compose.yml
version: '3.7'
services:
api:
build:
context: .
dockerfile: Dockerfile
container_name: ${APP_NAME}
#restart: unless-stopped
depends_on:
- api_db
- redis
ports:
- "${APP_PORT}:${APP_PORT}"
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
entrypoint: ./sh/entrypoints/api-entrypoint.sh
api_db:
image: postgres
command: postgres -p ${DATABASE_PORT}
ports:
- "${DATABASE_PORT}:${DATABASE_PORT}"
volumes:
- db_data:/var/lib/postgresql/data
- ./log/db:/logs
environment:
- POSTGRES_USER=${DATABASE_USER}
- POSTGRES_PASSWORD=${DATABASE_PASSWORD}
- POSTGRES_DB=${DATABASE_NAME}
redis:
image: redis
ports:
- "${REDIS_PORT}:${REDIS_PORT}"
command: redis-server
volumes:
- redis:/data
sidekiq:
build:
context: .
dockerfile: Dockerfile
depends_on:
- api_db
- redis
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
ENABLE_BOOTSNAP: 'false'
entrypoint: ./sh/entrypoints/sidekiq-entrypoint.sh
volumes:
redis:
gem_cache:
db_data:
node_modules:
./sh/entrypoints/api-entrypoint.sh
https://stackoverflow.com/a/59047028/4488252
#!/bin/sh
DB_INITED=0
if db_version=$(bundle exec rake db:version 2>/dev/null)
then
if [ "$db_version" = "Current version: 0" ]
then
echo "DB is empty"
else
echo "DB exists"
DB_INITED=1
fi
bundle exec rake db:migrate
else
echo "DB does not exist"
bundle exec rake db:setup
fi
if [ $DB_INITED == 0 ]
then
echo "Performing initial configuration"
# init some plugins, updated db if need, add initial data
fi
bundle exec rails assets:precompile
bundle exec rails s -b 0.0.0.0 -p $APP_PORT
./sh/entrypoints/sidekiq-entrypoint.sh
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle exec sidekiq
./config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
database: <%= ENV['DATABASE_NAME'] %>
username: <%= ENV['DATABASE_USER'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
port: <%= ENV['DATABASE_PORT'] || '5432' %>
host: <%= ENV['DATABASE_HOST'] %>
development:
<<: *default
test:
<<: *default
production:
<<: *default
secret_key_base: <%= ENV['SECRET_KEY_BASE'] %>
./config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
Sidekiq.configure_client do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
./.dockerignore
https://gist.github.com/neckhair/ace5d1679dd896b71403fda4bc217b9e
.git
.gitignore
README.md
#
# OS X
#
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear on external disk
.Spotlight-V100
.Trashes
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
#
# Rails
#
.env
.env.sample
*.rbc
capybara-*.html
log
tmp
db/*.sqlite3
db/*.sqlite3-journal
public/system
coverage/
spec/tmp
**.orig
.bundle
.ruby-version
.ruby-gemset
.rvmrc
# if using bower-rails ignore default bower_components path bower.json files
vendor/assets/bower_components
*.bowerrc
bower.json
# Logs
logs
*.log
# Runtime data
pids
*.pid
*.seed
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# node-waf configuration
.lock-wscript
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directory
# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git
node_modules
server/*.spec.js
kubernetes
Usage
https://docs.docker.com/compose/reference/down/
build and run: docker-compose up --build -d
https://docs.docker.com/compose/reference/down/
stop: docker-compose down
stop + delete images and volumes: docker-compose down --rmi all --volumes

fe_sendauth: no password supplied when creating database

I am in the process of dockerizing our Ruby on rails apps. One of them is throwing the following error when I run docker-compose run web bundle exec rake db:create
Starting sapi_db_1 ... done
[WARNING] The git gem requires git 1.6.0.0 or later, but only found
2.1.4. You should probably upgrade.
fe_sendauth: no password supplied
I am confirmed the config files are working as I have used them 3 times previously with oher apps.
Here's my dockerfile
FROM ruby:2.2.3
RUN apt-get update && apt-get -y install build-essential libpq-dev nodejs
RUN gem install bundler
RUN mkdir /app
WORKDIR /app
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock
RUN bundle install
ADD . /app
My docker-compose.yml
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/sapi
ports:
- "3000:3000"
environment:
DATABASE_URL: postgres://postgres#db
env_file:
- '.env'
depends_on:
- db
db:
image: postgres:latest
ports:
- "5432:5432"
env_file:
- '.env'
My database.yml file
default: &default
adapter: postgresql
pool: 5
timeout: 5000
url: <%= ENV['DATABASE_URL'] %>
username: <%= ENV['POSTGRES_USER'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
development:
<<: *default
database: app_development
How can I resolve this? I have already confirmed that the config files work by carrying the process out on other apps.
Resolved by changing the .env file

Docker-compose rails postgres

I've been following this tutorial to 'dockerize' my rails application and have hit a snag with connecting to the db after some searching around, no solutions seem to work. I've also tried the default user 'postgres' and no password, but still no luck. My error indicates that my password is incorrect, but everything I try doesn't change the error:
web_1 | I, [2017-06-02T00:58:29.217947 #7] INFO -- : listening on addr=0.0.0.0:3000 fd=13
postgres_1 | FATAL: password authentication failed for user "web"
postgres_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"
web_1 | E, [2017-06-02T00:58:29.230868 #7] ERROR -- : FATAL: password authentication failed for user "web"
Here's what I have:
.env
LISTEN_ON=0.0.0.0:3000
DATABASE_URL=postgresql://web:mypassword#postgres:5432/web?encoding=utf8&pool=5&timeout=5000
Dockerfile
FROM ruby:2.3.4
RUN apt-get update && apt-get install -qq -y build-essential nodejs libpq-dev postgresql-client-9.4 --fix-missing --no-install-recommends
ENV INSTALL_PATH /web
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN bundle install
COPY . .
# precompile assets using dummy data
RUN bundle exec rake RAILS_ENV=production DATABASE_URL=postgresql://user:pass#127.0.0.1/dbname SECRET_TOKEN=pickasecuretoken assets:precompile
VOLUME ["$INSTALL_PATH/public"]
VOLUME /postgres
CMD RAILS_ENV=development bundle exec unicorn -c config/unicorn.rb
docker-compose.yml
postgres:
image: postgres:9.4.5
environment:
POSTGRES_USER: web
POSTGRES_PASSWORD: mypassword
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
web:
build: .
links:
- postgres
volumes:
- .:/web
ports:
- "3000:3000"
env_file:
- .env
config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: 5
development:
<<: *default
url: <%= ENV['DATABASE_URL'] %>
The line in database.yml grabs the DATABASE_URL environment variable that is stored in the container from the .env file.
I spent the better part of a day fiddling with this. What finally worked for me was to fall back to the Postgres defaults.
docker-compose.yml
postgres:
image: postgres:9.4.5
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
.env
DATABASE_URL=postgresql://web:#postgres:5432/web?encoding=utf8&pool=5&timeout=5000
In the DATABASE_URL, keeping the password separator in the url but leaving the password blank finally made it work.

Resources