I am in the process of dockerizing our Ruby on rails apps. One of them is throwing the following error when I run docker-compose run web bundle exec rake db:create
Starting sapi_db_1 ... done
[WARNING] The git gem requires git 1.6.0.0 or later, but only found
2.1.4. You should probably upgrade.
fe_sendauth: no password supplied
I am confirmed the config files are working as I have used them 3 times previously with oher apps.
Here's my dockerfile
FROM ruby:2.2.3
RUN apt-get update && apt-get -y install build-essential libpq-dev nodejs
RUN gem install bundler
RUN mkdir /app
WORKDIR /app
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock
RUN bundle install
ADD . /app
My docker-compose.yml
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/sapi
ports:
- "3000:3000"
environment:
DATABASE_URL: postgres://postgres#db
env_file:
- '.env'
depends_on:
- db
db:
image: postgres:latest
ports:
- "5432:5432"
env_file:
- '.env'
My database.yml file
default: &default
adapter: postgresql
pool: 5
timeout: 5000
url: <%= ENV['DATABASE_URL'] %>
username: <%= ENV['POSTGRES_USER'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
development:
<<: *default
database: app_development
How can I resolve this? I have already confirmed that the config files work by carrying the process out on other apps.
Resolved by changing the .env file
Related
I'm trying to dockerize an existing rails application. But it's running into an error when trying to invoke rake db:create.
Here is the error:
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Couldn't create database for {"adapter"=>"postgresql", "encoding"=>"unicode", "pool"=>5,
"database"=>"app_development", "username"=>"postgres", "password"=>"postgres"}
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Here is my Dockerfile
FROM ruby:2.7.2
RUN apt-get update -qq && apt-get install -y nodejs postgresql-client
WORKDIR /app
COPY Gemfile /app/Gemfile
COPY Gemfile.lock /app/Gemfile.lock
RUN bundle install
COPY . /app
# COPY entrypoint.sh /usr/bin/
# RUN chmod +x /usr/bin/entrypoint.sh
# ENTRYPOINT ["entrypoint.sh"]
EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0"]
docker-compose.yml
version: "3.9"
services:
db:
image: postgres
volumes:
- db_data:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: app_development
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
volumes:
db_data:
database.yml
...
development:
<<: *default
database: app_development
# url: postgres://...
username: postgres
password: postgres
What am I doing wrong? I've tried everything online and still no luck. Any catches? Thanks in advance.
I think you will need to communicate over TCP/IP rather than a socket, so you should be able to sort this using the host setting in your database.yml file
...
development:
<<: *default
username: postgres
database: postgres
host: 127.0.0.1
I'm trying to dockerize my Rails application, but I have this error when I want to run it with docker-compose:
! Unable to load application: PG::ConnectionBad: could not connect to server: Connection refused
Is the server running on host "postgres" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
my docker-compose file is:
version: '2'
services:
web:
build: .
image: cda_app
container_name: "cda_app_web"
ports:
- '127.0.0.1:${WEB_PORT}:3000'
env_file: .env
volumes:
- gems:/gems
- ./:/var/www/app
logging:
driver: 'json-file'
options:
max-size: '100m'
max-file: '5'
links:
- redis
- postgres
entrypoint: 'bundle exec puma -C config/puma.rb'
redis:
image: 'redis'
volumes:
- redis:/data
- redis_log:/var/log/redis
postgres:
image: 'postgres:12.4'
volumes:
- postgres:/var/lib/postgresql/data
- postgres_log:/var/log/postgresql
volumes:
gems:
redis:
redis_log:
postgres:
postgres_log:
Dockerfile:
ARG BASE_IMAGE=ruby:2.7.1
FROM $BASE_IMAGE
RUN apt-get update && \
apt-get install -y \
apt-utils \
build-essential \
cmake \
curl \
ghostscript \
libmagic-dev \
libpq-dev \
openssh-client \
rename \
&& rm -rf /var/lib/apt/lists/*
RUN curl -sL https://deb.nodesource.com/setup_12.x | bash && apt-get install -y nodejs
RUN npm install -g yarn
WORKDIR /var/www/app
RUN mkdir /gems
ENV BUNDLE_PATH=/gems
RUN gem install bundler
ARG BUNDLE_WITHOUT=development:test
COPY Gemfile Gemfile.lock ./
RUN bundle install --jobs $(nproc) --with BUNDLE_WITHOUT
COPY package.json yarn.lock ./
RUN yarn install --check-files --ignore-optional
COPY . .
RUN rename -f -v 's/\.sample//' config/*sample.yml
EXPOSE 3000
CMD ./bin/puma -b tcp://0.0.0.0:3000
my database.yml:
default: &default
pool: <%= ENV["DB_POOL"] %>
template: 'template0'
adapter: 'postgresql'
database: <%= ENV["DB_NAME"] %>
username: <%= ENV["DB_USER"] %>
host: <%= ENV["DB_HOST"] %>
port: <%= ENV["DB_PORT"] %>
password: <%= ENV["DB_PASSWORD"] %>
timeout: 5000
encoding: 'utf8'
min_messages: WARNING
development:
<<: *default
database: <%= ENV["DB_NAME"] %>_development
test:
<<: *default
database: <%= ENV["DB_NAME"] %>_test<%= ENV['TEST_ENV_NUMBER'] %>
production:
<<: *default
database: <%= ENV["DB_NAME"] %>_production
.env file:
DB_NAME=cda_database
DB_HOST=postgres
DB_USER=postgres
DB_PASSWORD=postgres
DB_PORT=5432
DB_POOL=5
I can't understand what I'm doing wrong?!
It seems I did everything correctly, but I'm getting the error.
I'm using Digitalocean. Also, the database is working outside of docker, but it can't be accessible in docker.
You can try it in docker-compose file. I hope it working
postgres:
image: 'postgres:12.4'
volumes:
- postgres:/var/lib/postgresql/data
- postgres_log:/var/log/postgresql
environment:
- POSTGRES_USER= postgres
- POSTGRES_PASSWORD= postgres
You can write the environment variables each one for postgresql in docker-compose.yml, example:
version: '2'
services:
web:
build: .
image: cda_app
container_name: "cda_app_web"
ports:
- '127.0.0.1:${WEB_PORT}:3000'
env_file: .env
volumes:
- gems:/gems
- ./:/var/www/app
logging:
driver: 'json-file'
options:
max-size: '100m'
max-file: '5'
links:
- redis
- postgres
entrypoint: 'bundle exec puma -C config/puma.rb'
redis:
image: 'redis'
volumes:
- redis:/data
- redis_log:/var/log/redis
postgres:
image: 'postgres:12.4'
volumes:
- postgres:/var/lib/postgresql/data
- postgres_log:/var/log/postgresql
environment:
POSTGRES_PASSWORD: "${DB_PASSWORD}"
POSTGRES_USER: "${DB_USER}"
POSTGRES_DB: "${DB_NAME}"
volumes:
gems:
redis:
redis_log:
postgres:
postgres_log:
Be aware that the file .env must be in the same directory than your docker-compose.yml
I am trying to build a container with docker but I cannot connect sidekiq + redis, the error says sidekiq_1 | Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED), seems to sidekiq is trying to connect to localhost but since I am building "in theory" redis + sidekiq + rails + postgres containers is not in the localhost, it should be in redis image.
My docker-compose.yml file is this:
version: '3'
services:
postgres:
image: postgres:10.5
volumes:
- my_app-postgres:/var/lib/postgresql/data
redis:
image: redis:4.0.11
volumes:
- my_app-redis:/var/lib/redis/data
web:
build: .
command: bundle exec rails server -p 3000 -b '0.0.0.0'
ports:
- '3000:3000'
depends_on:
- postgres
- redis
volumes:
- .:/my_app
env_file:
- .env
sidekiq:
build: .
command: bundle exec sidekiq -C config/sidekiq.yml
volumes:
- .:/my_app
depends_on:
- postgres
- redis
env_file:
- .env
volumes:
my_app-postgres:
my_app-redis:
another interesting 'info' I see in log is Booting Sidekiq 4.2.10 with redis options {:url=>nil} this url can be the cause of the issue?
in my development environment the app is working fine, I try to 'dockerize' what I have. How could I make this work?
By default, sidekiq tries to connect to 127.0.0.1:6379 but your sidekiq is in a different container than redis, so you need to configure sidekiq to use redis:6379 as redis host, e.g. by using an initializer:
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://redis:6379/12' }
end
Take a look at the docs for more details: https://github.com/mperham/sidekiq/wiki/Using-Redis
If you are planning on using Kubernetes for deployment later on, you can put all containers in a pod and then they would be able to connect via localhost because containers within the same Kubernetes pod share the network space. To program directly within a pod inside a Kubernetes cluster, you could work with a tool I recently open sourced on GitHub called DevSpace: https://github.com/covexo/devspace
Create two initializer files:
i) redis.rb
uri = "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0/your-app-cache" || 'redis://localhost:6379/0/your-app-cache'
Rails.application.config.cache_store = :redis_store, uri
ii) sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Sidekiq.configure_client do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Full Sample
./Dockerfile
FROM ruby:2.6.3-alpine
ENV BUNDLER_VERSION=2.0.2
RUN apk add --update --no-cache \
binutils-gold \
build-base \
curl \
file \
g++ \
gcc \
git \
less \
libstdc++ \
libffi-dev \
libc-dev \
linux-headers \
libxml2-dev \
libxslt-dev \
libgcrypt-dev \
make \
netcat-openbsd \
nodejs \
openssl \
pkgconfig \
postgresql-dev \
python \
tzdata \
yarn
ARG USER=root
ARG WORK_DIR_PATH=/home
RUN mkdir -p $WORK_DIR_PATH && chown -R $USER:$USER $WORK_DIR_PATH
WORKDIR $WORK_DIR_PATH
COPY Gemfile* ./
RUN gem install bundler
RUN bundle config build.nokogiri --use-system-libraries
RUN bundle check || bundle install
COPY package.json yarn.lock ./
RUN yarn install --check-files
COPY . .
./.env
APP_NAME=api
APP_PORT=3100
ENV=production
DATABASE_NAME=rails_db
DATABASE_USER=batman
DATABASE_PASSWORD=super_pass_123
DATABASE_PORT=5342
DATABASE_HOST=api_db # must be equal to the name of the postgres service in docker-compose.yml
SECRET_KEY_BASE=your_secret_string
REDIS_HOST=redis # must be equal to the name of the redis service in docker-compose.yml
REDIS_PORT=6379
./docker-compose.yml
version: '3.7'
services:
api:
build:
context: .
dockerfile: Dockerfile
container_name: ${APP_NAME}
#restart: unless-stopped
depends_on:
- api_db
- redis
ports:
- "${APP_PORT}:${APP_PORT}"
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
entrypoint: ./sh/entrypoints/api-entrypoint.sh
api_db:
image: postgres
command: postgres -p ${DATABASE_PORT}
ports:
- "${DATABASE_PORT}:${DATABASE_PORT}"
volumes:
- db_data:/var/lib/postgresql/data
- ./log/db:/logs
environment:
- POSTGRES_USER=${DATABASE_USER}
- POSTGRES_PASSWORD=${DATABASE_PASSWORD}
- POSTGRES_DB=${DATABASE_NAME}
redis:
image: redis
ports:
- "${REDIS_PORT}:${REDIS_PORT}"
command: redis-server
volumes:
- redis:/data
sidekiq:
build:
context: .
dockerfile: Dockerfile
depends_on:
- api_db
- redis
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
ENABLE_BOOTSNAP: 'false'
entrypoint: ./sh/entrypoints/sidekiq-entrypoint.sh
volumes:
redis:
gem_cache:
db_data:
node_modules:
./sh/entrypoints/api-entrypoint.sh
https://stackoverflow.com/a/59047028/4488252
#!/bin/sh
DB_INITED=0
if db_version=$(bundle exec rake db:version 2>/dev/null)
then
if [ "$db_version" = "Current version: 0" ]
then
echo "DB is empty"
else
echo "DB exists"
DB_INITED=1
fi
bundle exec rake db:migrate
else
echo "DB does not exist"
bundle exec rake db:setup
fi
if [ $DB_INITED == 0 ]
then
echo "Performing initial configuration"
# init some plugins, updated db if need, add initial data
fi
bundle exec rails assets:precompile
bundle exec rails s -b 0.0.0.0 -p $APP_PORT
./sh/entrypoints/sidekiq-entrypoint.sh
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle exec sidekiq
./config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
database: <%= ENV['DATABASE_NAME'] %>
username: <%= ENV['DATABASE_USER'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
port: <%= ENV['DATABASE_PORT'] || '5432' %>
host: <%= ENV['DATABASE_HOST'] %>
development:
<<: *default
test:
<<: *default
production:
<<: *default
secret_key_base: <%= ENV['SECRET_KEY_BASE'] %>
./config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
Sidekiq.configure_client do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
./.dockerignore
https://gist.github.com/neckhair/ace5d1679dd896b71403fda4bc217b9e
.git
.gitignore
README.md
#
# OS X
#
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear on external disk
.Spotlight-V100
.Trashes
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
#
# Rails
#
.env
.env.sample
*.rbc
capybara-*.html
log
tmp
db/*.sqlite3
db/*.sqlite3-journal
public/system
coverage/
spec/tmp
**.orig
.bundle
.ruby-version
.ruby-gemset
.rvmrc
# if using bower-rails ignore default bower_components path bower.json files
vendor/assets/bower_components
*.bowerrc
bower.json
# Logs
logs
*.log
# Runtime data
pids
*.pid
*.seed
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# node-waf configuration
.lock-wscript
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directory
# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git
node_modules
server/*.spec.js
kubernetes
Usage
https://docs.docker.com/compose/reference/down/
build and run: docker-compose up --build -d
https://docs.docker.com/compose/reference/down/
stop: docker-compose down
stop + delete images and volumes: docker-compose down --rmi all --volumes
With the following files I can start rails application successfully
except rake operations with database. When I run
docker exec -it www_web_1 bundle exec rake db:migrate
I saw that it tried to connect to localhost (web) and fail
instead of conntecing to database host (db)
Any ideas how to solve it?
Dockerfile
FROM ruby:2.4.2
RUN mkdir -p /var/www/project
WORKDIR /var/www/project
COPY Gemfile /var/www/project/Gemfile
RUN bundle install
COPY entrypoint.sh /usr/local/bin
COPY . /var/www/project
CMD ["entrypoint.sh"]
entrypoint.sh
#!/usr/bin/env sh
bundle exec rake db:migrate
bundle exec rails s -p 3000 -b '0.0.0.0'
docker-compose.yml file
version: '2'
services:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: project
web:
depends_on:
- db
environment:
- PROJECT_DB_NAME=project
- PROJECT_DB_USER=root
- PROJECT_DB_PSWD=root
- PROJECT_DB_HOST=db
- RACK_ENV=production
- RAILS_ENV=production
volumes:
mysql:
web:
config/database.yml
default: &default
adapter: mysql2
pool: 5
timeout: 5000
database: <%= ENV["PROJECT_DB_NAME"] %>
encoding: utf8
username: <%= ENV["PROJECT_DB_USER"] %>
password: <%= ENV["PROJECT_DB_PSWD"] %>
host: <%= ENV["PROJECT_DB_HOST"] %>
development:
<<: *default
production:
<<: *default
You have compose file.
you have to run compose command
$ docker-compose up --build # run and build
$ docker-compose run web rake db:create # database create
$ docker-compose run web rake db:migrate # migreate
if you'd like more know docker-compose
visit https://github.com/x1wins/paper-flower/wiki/2.-Docker--compose-(Development-mode,-Standalone)
I've been following this tutorial to 'dockerize' my rails application and have hit a snag with connecting to the db after some searching around, no solutions seem to work. I've also tried the default user 'postgres' and no password, but still no luck. My error indicates that my password is incorrect, but everything I try doesn't change the error:
web_1 | I, [2017-06-02T00:58:29.217947 #7] INFO -- : listening on addr=0.0.0.0:3000 fd=13
postgres_1 | FATAL: password authentication failed for user "web"
postgres_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"
web_1 | E, [2017-06-02T00:58:29.230868 #7] ERROR -- : FATAL: password authentication failed for user "web"
Here's what I have:
.env
LISTEN_ON=0.0.0.0:3000
DATABASE_URL=postgresql://web:mypassword#postgres:5432/web?encoding=utf8&pool=5&timeout=5000
Dockerfile
FROM ruby:2.3.4
RUN apt-get update && apt-get install -qq -y build-essential nodejs libpq-dev postgresql-client-9.4 --fix-missing --no-install-recommends
ENV INSTALL_PATH /web
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN bundle install
COPY . .
# precompile assets using dummy data
RUN bundle exec rake RAILS_ENV=production DATABASE_URL=postgresql://user:pass#127.0.0.1/dbname SECRET_TOKEN=pickasecuretoken assets:precompile
VOLUME ["$INSTALL_PATH/public"]
VOLUME /postgres
CMD RAILS_ENV=development bundle exec unicorn -c config/unicorn.rb
docker-compose.yml
postgres:
image: postgres:9.4.5
environment:
POSTGRES_USER: web
POSTGRES_PASSWORD: mypassword
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
web:
build: .
links:
- postgres
volumes:
- .:/web
ports:
- "3000:3000"
env_file:
- .env
config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: 5
development:
<<: *default
url: <%= ENV['DATABASE_URL'] %>
The line in database.yml grabs the DATABASE_URL environment variable that is stored in the container from the .env file.
I spent the better part of a day fiddling with this. What finally worked for me was to fall back to the Postgres defaults.
docker-compose.yml
postgres:
image: postgres:9.4.5
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
.env
DATABASE_URL=postgresql://web:#postgres:5432/web?encoding=utf8&pool=5&timeout=5000
In the DATABASE_URL, keeping the password separator in the url but leaving the password blank finally made it work.