Rails docker postgres Connection refused - ruby-on-rails

I'm trying to dockerize my Rails application, but I have this error when I want to run it with docker-compose:
! Unable to load application: PG::ConnectionBad: could not connect to server: Connection refused
Is the server running on host "postgres" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
my docker-compose file is:
version: '2'
services:
web:
build: .
image: cda_app
container_name: "cda_app_web"
ports:
- '127.0.0.1:${WEB_PORT}:3000'
env_file: .env
volumes:
- gems:/gems
- ./:/var/www/app
logging:
driver: 'json-file'
options:
max-size: '100m'
max-file: '5'
links:
- redis
- postgres
entrypoint: 'bundle exec puma -C config/puma.rb'
redis:
image: 'redis'
volumes:
- redis:/data
- redis_log:/var/log/redis
postgres:
image: 'postgres:12.4'
volumes:
- postgres:/var/lib/postgresql/data
- postgres_log:/var/log/postgresql
volumes:
gems:
redis:
redis_log:
postgres:
postgres_log:
Dockerfile:
ARG BASE_IMAGE=ruby:2.7.1
FROM $BASE_IMAGE
RUN apt-get update && \
apt-get install -y \
apt-utils \
build-essential \
cmake \
curl \
ghostscript \
libmagic-dev \
libpq-dev \
openssh-client \
rename \
&& rm -rf /var/lib/apt/lists/*
RUN curl -sL https://deb.nodesource.com/setup_12.x | bash && apt-get install -y nodejs
RUN npm install -g yarn
WORKDIR /var/www/app
RUN mkdir /gems
ENV BUNDLE_PATH=/gems
RUN gem install bundler
ARG BUNDLE_WITHOUT=development:test
COPY Gemfile Gemfile.lock ./
RUN bundle install --jobs $(nproc) --with BUNDLE_WITHOUT
COPY package.json yarn.lock ./
RUN yarn install --check-files --ignore-optional
COPY . .
RUN rename -f -v 's/\.sample//' config/*sample.yml
EXPOSE 3000
CMD ./bin/puma -b tcp://0.0.0.0:3000
my database.yml:
default: &default
pool: <%= ENV["DB_POOL"] %>
template: 'template0'
adapter: 'postgresql'
database: <%= ENV["DB_NAME"] %>
username: <%= ENV["DB_USER"] %>
host: <%= ENV["DB_HOST"] %>
port: <%= ENV["DB_PORT"] %>
password: <%= ENV["DB_PASSWORD"] %>
timeout: 5000
encoding: 'utf8'
min_messages: WARNING
development:
<<: *default
database: <%= ENV["DB_NAME"] %>_development
test:
<<: *default
database: <%= ENV["DB_NAME"] %>_test<%= ENV['TEST_ENV_NUMBER'] %>
production:
<<: *default
database: <%= ENV["DB_NAME"] %>_production
.env file:
DB_NAME=cda_database
DB_HOST=postgres
DB_USER=postgres
DB_PASSWORD=postgres
DB_PORT=5432
DB_POOL=5
I can't understand what I'm doing wrong?!
It seems I did everything correctly, but I'm getting the error.
I'm using Digitalocean. Also, the database is working outside of docker, but it can't be accessible in docker.

You can try it in docker-compose file. I hope it working
postgres:
image: 'postgres:12.4'
volumes:
- postgres:/var/lib/postgresql/data
- postgres_log:/var/log/postgresql
environment:
- POSTGRES_USER= postgres
- POSTGRES_PASSWORD= postgres

You can write the environment variables each one for postgresql in docker-compose.yml, example:
version: '2'
services:
web:
build: .
image: cda_app
container_name: "cda_app_web"
ports:
- '127.0.0.1:${WEB_PORT}:3000'
env_file: .env
volumes:
- gems:/gems
- ./:/var/www/app
logging:
driver: 'json-file'
options:
max-size: '100m'
max-file: '5'
links:
- redis
- postgres
entrypoint: 'bundle exec puma -C config/puma.rb'
redis:
image: 'redis'
volumes:
- redis:/data
- redis_log:/var/log/redis
postgres:
image: 'postgres:12.4'
volumes:
- postgres:/var/lib/postgresql/data
- postgres_log:/var/log/postgresql
environment:
POSTGRES_PASSWORD: "${DB_PASSWORD}"
POSTGRES_USER: "${DB_USER}"
POSTGRES_DB: "${DB_NAME}"
volumes:
gems:
redis:
redis_log:
postgres:
postgres_log:
Be aware that the file .env must be in the same directory than your docker-compose.yml

Related

Cannot establish connection between Rails and Postgres Docker containers

I am new to Docker and try to dockerize a Rails container and a Postgres container with docker-compose and both build and start but when I try to go to localhost:3009 to see the "Yay, you're on Rails" page, I get this error in the Rails logs:
Started GET "/" for 172.27.0.1 at 2020-10-05 12:40:18 +0000
backend_1 | Cannot render console from 172.27.0.1! Allowed networks: 127.0.0.0/127.255.255.255, ::1
backend_1 |
backend_1 | PG::ConnectionBad (could not connect to server: Connection refused
backend_1 | Is the server running on host "postgres" (172.27.0.2) and accepting
backend_1 | TCP/IP connections on port 5439?
backend_1 | ):
This is my docker-compose.yml:
version: '3'
volumes:
db_data:
driver: local
app_data:
driver: local
services:
# database container
postgres:
image: postgres
volumes:
- app_data:/var/lib/postgresql/data
ports:
- 5439:5432
environment:
POSTGRES_DB: db_name
POSTGRES_USER: user
POSTGRES_PASSWORD: xxxxxx
command: ["postgres", "-c", "log_statement=all"]
restart: always
app:
build: ./server
volumes:
- ./server:/code
ports:
- "3009:3000"
depends_on:
- postgres
environment:
RAILS_ENV: development
DATABASE_HOST: postgres
DATABASE_PORT: 5439
DATABASE_NAME: db_name
DATABASE_USERNAME: user
DATABASE_PASSWORD: xxxxxx
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
And this is the Dockerfile:
FROM ruby:2.7.1
WORKDIR /app
COPY . /app
# Install NodeJS and Yarn.
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y yarn
RUN yarn install --check-files
RUN bundle install
CMD ["rails", "server", "-b", "0.0.0.0"]
This is my database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
host: <%= ENV['DATABASE_HOST'] %>
port: <%= ENV['DATABASE_PORT'] %>
database: <%= ENV['DATABASE_NAME'] %>
username: <%= ENV['DATABASE_USERNAME'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
development:
<<: *default
test:
<<: *default
I can access the pg database in DBeaver with 172.27.0.1:5439, but not with 172.27.0.2:5439 and I have no clue why the Rails container tries to connect on that address.
As David Maze pointed out:
You need to set DATABASE_PORT to the ordinary PostgreSQL port 5432; ports: are ignored (and unnecessary) for inter-container communications.

How to connect to other container from Dockerfile while docker-compose build

I try to configure docker compose for my php project. On deploy I want to update a source code, update composer dependencies and run database migrations.
So I have a docker-compose.yml file:
version: '3.0'
services:
php:
build:
context: .
dockerfile: php/Dockerfile
depends_on:
- postgres
postgres:
image: "postgres:13-alpine"
restart: always
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB_NAME}
Php container builds from the next Dockerfile:
# Inatall dependensies
RUN apt-get update \
&& apt-get install -y git libicu-dev postgresql-server-dev-all zip libzip-dev postgresql-client\
&& docker-php-ext-install intl pdo pdo_pgsql zip
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
# Copy source files
COPY ./app /var/www/my-site
# Update project files
WORKDIR /var/www/my-site
RUN composer install
RUN php ./yii migrate --interactive=0 # This command needs to connect to the database and fails
CMD [ "php-fpm"]
When I run docker-compose build, I have this error: could not translate host name "postgres" to address: Name or service not known.
How can I take access to database container while other is building?
Both php and postgres need to be on same network and php can access postgres using container_name which is postgres. depends_on will make sure postgres get starts before php.
version: '3.0'
services:
php:
build:
context: .
dockerfile: php/Dockerfile
restart: on-failure
depends_on:
- postgres
networks:
- test-network
postgres:
container_name: 'postgres'
image: "postgres:13-alpine"
restart: always
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB_NAME}
networks:
- test-network
networks:
test-network:
driver: bridge

docker-compose error connecting to redis + sidekiq

I am trying to build a container with docker but I cannot connect sidekiq + redis, the error says sidekiq_1 | Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED), seems to sidekiq is trying to connect to localhost but since I am building "in theory" redis + sidekiq + rails + postgres containers is not in the localhost, it should be in redis image.
My docker-compose.yml file is this:
version: '3'
services:
postgres:
image: postgres:10.5
volumes:
- my_app-postgres:/var/lib/postgresql/data
redis:
image: redis:4.0.11
volumes:
- my_app-redis:/var/lib/redis/data
web:
build: .
command: bundle exec rails server -p 3000 -b '0.0.0.0'
ports:
- '3000:3000'
depends_on:
- postgres
- redis
volumes:
- .:/my_app
env_file:
- .env
sidekiq:
build: .
command: bundle exec sidekiq -C config/sidekiq.yml
volumes:
- .:/my_app
depends_on:
- postgres
- redis
env_file:
- .env
volumes:
my_app-postgres:
my_app-redis:
another interesting 'info' I see in log is Booting Sidekiq 4.2.10 with redis options {:url=>nil} this url can be the cause of the issue?
in my development environment the app is working fine, I try to 'dockerize' what I have. How could I make this work?
By default, sidekiq tries to connect to 127.0.0.1:6379 but your sidekiq is in a different container than redis, so you need to configure sidekiq to use redis:6379 as redis host, e.g. by using an initializer:
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://redis:6379/12' }
end
Take a look at the docs for more details: https://github.com/mperham/sidekiq/wiki/Using-Redis
If you are planning on using Kubernetes for deployment later on, you can put all containers in a pod and then they would be able to connect via localhost because containers within the same Kubernetes pod share the network space. To program directly within a pod inside a Kubernetes cluster, you could work with a tool I recently open sourced on GitHub called DevSpace: https://github.com/covexo/devspace
Create two initializer files:
i) redis.rb
uri = "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0/your-app-cache" || 'redis://localhost:6379/0/your-app-cache'
Rails.application.config.cache_store = :redis_store, uri
ii) sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Sidekiq.configure_client do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Full Sample
./Dockerfile
FROM ruby:2.6.3-alpine
ENV BUNDLER_VERSION=2.0.2
RUN apk add --update --no-cache \
binutils-gold \
build-base \
curl \
file \
g++ \
gcc \
git \
less \
libstdc++ \
libffi-dev \
libc-dev \
linux-headers \
libxml2-dev \
libxslt-dev \
libgcrypt-dev \
make \
netcat-openbsd \
nodejs \
openssl \
pkgconfig \
postgresql-dev \
python \
tzdata \
yarn
ARG USER=root
ARG WORK_DIR_PATH=/home
RUN mkdir -p $WORK_DIR_PATH && chown -R $USER:$USER $WORK_DIR_PATH
WORKDIR $WORK_DIR_PATH
COPY Gemfile* ./
RUN gem install bundler
RUN bundle config build.nokogiri --use-system-libraries
RUN bundle check || bundle install
COPY package.json yarn.lock ./
RUN yarn install --check-files
COPY . .
./.env
APP_NAME=api
APP_PORT=3100
ENV=production
DATABASE_NAME=rails_db
DATABASE_USER=batman
DATABASE_PASSWORD=super_pass_123
DATABASE_PORT=5342
DATABASE_HOST=api_db # must be equal to the name of the postgres service in docker-compose.yml
SECRET_KEY_BASE=your_secret_string
REDIS_HOST=redis # must be equal to the name of the redis service in docker-compose.yml
REDIS_PORT=6379
./docker-compose.yml
version: '3.7'
services:
api:
build:
context: .
dockerfile: Dockerfile
container_name: ${APP_NAME}
#restart: unless-stopped
depends_on:
- api_db
- redis
ports:
- "${APP_PORT}:${APP_PORT}"
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
entrypoint: ./sh/entrypoints/api-entrypoint.sh
api_db:
image: postgres
command: postgres -p ${DATABASE_PORT}
ports:
- "${DATABASE_PORT}:${DATABASE_PORT}"
volumes:
- db_data:/var/lib/postgresql/data
- ./log/db:/logs
environment:
- POSTGRES_USER=${DATABASE_USER}
- POSTGRES_PASSWORD=${DATABASE_PASSWORD}
- POSTGRES_DB=${DATABASE_NAME}
redis:
image: redis
ports:
- "${REDIS_PORT}:${REDIS_PORT}"
command: redis-server
volumes:
- redis:/data
sidekiq:
build:
context: .
dockerfile: Dockerfile
depends_on:
- api_db
- redis
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
ENABLE_BOOTSNAP: 'false'
entrypoint: ./sh/entrypoints/sidekiq-entrypoint.sh
volumes:
redis:
gem_cache:
db_data:
node_modules:
./sh/entrypoints/api-entrypoint.sh
https://stackoverflow.com/a/59047028/4488252
#!/bin/sh
DB_INITED=0
if db_version=$(bundle exec rake db:version 2>/dev/null)
then
if [ "$db_version" = "Current version: 0" ]
then
echo "DB is empty"
else
echo "DB exists"
DB_INITED=1
fi
bundle exec rake db:migrate
else
echo "DB does not exist"
bundle exec rake db:setup
fi
if [ $DB_INITED == 0 ]
then
echo "Performing initial configuration"
# init some plugins, updated db if need, add initial data
fi
bundle exec rails assets:precompile
bundle exec rails s -b 0.0.0.0 -p $APP_PORT
./sh/entrypoints/sidekiq-entrypoint.sh
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle exec sidekiq
./config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
database: <%= ENV['DATABASE_NAME'] %>
username: <%= ENV['DATABASE_USER'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
port: <%= ENV['DATABASE_PORT'] || '5432' %>
host: <%= ENV['DATABASE_HOST'] %>
development:
<<: *default
test:
<<: *default
production:
<<: *default
secret_key_base: <%= ENV['SECRET_KEY_BASE'] %>
./config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
Sidekiq.configure_client do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
./.dockerignore
https://gist.github.com/neckhair/ace5d1679dd896b71403fda4bc217b9e
.git
.gitignore
README.md
#
# OS X
#
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear on external disk
.Spotlight-V100
.Trashes
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
#
# Rails
#
.env
.env.sample
*.rbc
capybara-*.html
log
tmp
db/*.sqlite3
db/*.sqlite3-journal
public/system
coverage/
spec/tmp
**.orig
.bundle
.ruby-version
.ruby-gemset
.rvmrc
# if using bower-rails ignore default bower_components path bower.json files
vendor/assets/bower_components
*.bowerrc
bower.json
# Logs
logs
*.log
# Runtime data
pids
*.pid
*.seed
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# node-waf configuration
.lock-wscript
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directory
# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git
node_modules
server/*.spec.js
kubernetes
Usage
https://docs.docker.com/compose/reference/down/
build and run: docker-compose up --build -d
https://docs.docker.com/compose/reference/down/
stop: docker-compose down
stop + delete images and volumes: docker-compose down --rmi all --volumes

fe_sendauth: no password supplied when creating database

I am in the process of dockerizing our Ruby on rails apps. One of them is throwing the following error when I run docker-compose run web bundle exec rake db:create
Starting sapi_db_1 ... done
[WARNING] The git gem requires git 1.6.0.0 or later, but only found
2.1.4. You should probably upgrade.
fe_sendauth: no password supplied
I am confirmed the config files are working as I have used them 3 times previously with oher apps.
Here's my dockerfile
FROM ruby:2.2.3
RUN apt-get update && apt-get -y install build-essential libpq-dev nodejs
RUN gem install bundler
RUN mkdir /app
WORKDIR /app
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock
RUN bundle install
ADD . /app
My docker-compose.yml
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/sapi
ports:
- "3000:3000"
environment:
DATABASE_URL: postgres://postgres#db
env_file:
- '.env'
depends_on:
- db
db:
image: postgres:latest
ports:
- "5432:5432"
env_file:
- '.env'
My database.yml file
default: &default
adapter: postgresql
pool: 5
timeout: 5000
url: <%= ENV['DATABASE_URL'] %>
username: <%= ENV['POSTGRES_USER'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
development:
<<: *default
database: app_development
How can I resolve this? I have already confirmed that the config files work by carrying the process out on other apps.
Resolved by changing the .env file

Docker-compose rails postgres

I've been following this tutorial to 'dockerize' my rails application and have hit a snag with connecting to the db after some searching around, no solutions seem to work. I've also tried the default user 'postgres' and no password, but still no luck. My error indicates that my password is incorrect, but everything I try doesn't change the error:
web_1 | I, [2017-06-02T00:58:29.217947 #7] INFO -- : listening on addr=0.0.0.0:3000 fd=13
postgres_1 | FATAL: password authentication failed for user "web"
postgres_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"
web_1 | E, [2017-06-02T00:58:29.230868 #7] ERROR -- : FATAL: password authentication failed for user "web"
Here's what I have:
.env
LISTEN_ON=0.0.0.0:3000
DATABASE_URL=postgresql://web:mypassword#postgres:5432/web?encoding=utf8&pool=5&timeout=5000
Dockerfile
FROM ruby:2.3.4
RUN apt-get update && apt-get install -qq -y build-essential nodejs libpq-dev postgresql-client-9.4 --fix-missing --no-install-recommends
ENV INSTALL_PATH /web
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN bundle install
COPY . .
# precompile assets using dummy data
RUN bundle exec rake RAILS_ENV=production DATABASE_URL=postgresql://user:pass#127.0.0.1/dbname SECRET_TOKEN=pickasecuretoken assets:precompile
VOLUME ["$INSTALL_PATH/public"]
VOLUME /postgres
CMD RAILS_ENV=development bundle exec unicorn -c config/unicorn.rb
docker-compose.yml
postgres:
image: postgres:9.4.5
environment:
POSTGRES_USER: web
POSTGRES_PASSWORD: mypassword
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
web:
build: .
links:
- postgres
volumes:
- .:/web
ports:
- "3000:3000"
env_file:
- .env
config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: 5
development:
<<: *default
url: <%= ENV['DATABASE_URL'] %>
The line in database.yml grabs the DATABASE_URL environment variable that is stored in the container from the .env file.
I spent the better part of a day fiddling with this. What finally worked for me was to fall back to the Postgres defaults.
docker-compose.yml
postgres:
image: postgres:9.4.5
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
.env
DATABASE_URL=postgresql://web:#postgres:5432/web?encoding=utf8&pool=5&timeout=5000
In the DATABASE_URL, keeping the password separator in the url but leaving the password blank finally made it work.

Resources