Docker-compose rails postgres - ruby-on-rails

I've been following this tutorial to 'dockerize' my rails application and have hit a snag with connecting to the db after some searching around, no solutions seem to work. I've also tried the default user 'postgres' and no password, but still no luck. My error indicates that my password is incorrect, but everything I try doesn't change the error:
web_1 | I, [2017-06-02T00:58:29.217947 #7] INFO -- : listening on addr=0.0.0.0:3000 fd=13
postgres_1 | FATAL: password authentication failed for user "web"
postgres_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"
web_1 | E, [2017-06-02T00:58:29.230868 #7] ERROR -- : FATAL: password authentication failed for user "web"
Here's what I have:
.env
LISTEN_ON=0.0.0.0:3000
DATABASE_URL=postgresql://web:mypassword#postgres:5432/web?encoding=utf8&pool=5&timeout=5000
Dockerfile
FROM ruby:2.3.4
RUN apt-get update && apt-get install -qq -y build-essential nodejs libpq-dev postgresql-client-9.4 --fix-missing --no-install-recommends
ENV INSTALL_PATH /web
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN bundle install
COPY . .
# precompile assets using dummy data
RUN bundle exec rake RAILS_ENV=production DATABASE_URL=postgresql://user:pass#127.0.0.1/dbname SECRET_TOKEN=pickasecuretoken assets:precompile
VOLUME ["$INSTALL_PATH/public"]
VOLUME /postgres
CMD RAILS_ENV=development bundle exec unicorn -c config/unicorn.rb
docker-compose.yml
postgres:
image: postgres:9.4.5
environment:
POSTGRES_USER: web
POSTGRES_PASSWORD: mypassword
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
web:
build: .
links:
- postgres
volumes:
- .:/web
ports:
- "3000:3000"
env_file:
- .env
config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: 5
development:
<<: *default
url: <%= ENV['DATABASE_URL'] %>
The line in database.yml grabs the DATABASE_URL environment variable that is stored in the container from the .env file.

I spent the better part of a day fiddling with this. What finally worked for me was to fall back to the Postgres defaults.
docker-compose.yml
postgres:
image: postgres:9.4.5
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
.env
DATABASE_URL=postgresql://web:#postgres:5432/web?encoding=utf8&pool=5&timeout=5000
In the DATABASE_URL, keeping the password separator in the url but leaving the password blank finally made it work.

Related

db issues when dockerizing Ruby on Rails application

I'm trying to dockerize an existing rails application. But it's running into an error when trying to invoke rake db:create.
Here is the error:
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Couldn't create database for {"adapter"=>"postgresql", "encoding"=>"unicode", "pool"=>5,
"database"=>"app_development", "username"=>"postgres", "password"=>"postgres"}
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Here is my Dockerfile
FROM ruby:2.7.2
RUN apt-get update -qq && apt-get install -y nodejs postgresql-client
WORKDIR /app
COPY Gemfile /app/Gemfile
COPY Gemfile.lock /app/Gemfile.lock
RUN bundle install
COPY . /app
# COPY entrypoint.sh /usr/bin/
# RUN chmod +x /usr/bin/entrypoint.sh
# ENTRYPOINT ["entrypoint.sh"]
EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0"]
docker-compose.yml
version: "3.9"
services:
db:
image: postgres
volumes:
- db_data:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: app_development
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
volumes:
db_data:
database.yml
...
development:
<<: *default
database: app_development
# url: postgres://...
username: postgres
password: postgres
What am I doing wrong? I've tried everything online and still no luck. Any catches? Thanks in advance.
I think you will need to communicate over TCP/IP rather than a socket, so you should be able to sort this using the host setting in your database.yml file
...
development:
<<: *default
username: postgres
database: postgres
host: 127.0.0.1

Docker compose with Rails and Postgres could not connect to server: No route to host Is the server

I'm currently having an issue with my docker-compose that have these services.
Rails app and Postgres. These are my configurations:
docker-compose.yml
version: '3'
services:
db:
image: postgres:alpine
restart: always
volumes:
- ./tmp/db:/var/lib/postgresql/data
ports:
- "5432:5432"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
app:
build: .
restart: always
command: bash -c "rm -f tmp/pids/server.pid && rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
- bundle_path:/bundle
ports:
- "3000:3000"
depends_on:
- db
volumes:
bundle_path:
Dockerfile
FROM ruby:2.5.3-slim
# install rails dependencies
RUN apt-get update -qq \
&& apt-get install -y \
# Needed for certain gems
build-essential \
# Needed for postgres gem
libpq-dev \
# Others
nodejs \
vim-tiny \
# The following are used to trim down the size of the image by removing unneeded data
&& apt-get clean autoclean \
&& apt-get autoremove -y \
&& rm -rf \
/var/lib/apt \
/var/lib/dpkg \
/var/lib/cache \
/var/lib/log
# Changes localtime to Singapore
RUN cp /usr/share/zoneinfo/Asia/Singapore /etc/localtime
# create a folder /myapp in the docker container and go into that folder
RUN mkdir /myapp
WORKDIR /myapp
COPY Gemfile /myapp/Gemfile
COPY Gemfile.lock /myapp/Gemfile.lock
# Run bundle install to install gems inside the gemfile
RUN bundle install
ADD . /myapp
CMD bash -c "rm -f tmp/pids/server.pid && rails s -p 3000 -b '0.0.0.0'"
database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
development:
<<: *default
database: myapp_development
host: db
username: postgres
password: postgres
port: 5432
I can build the app using docker-compose build but whenever I docker-compose up the service db exited but my rails app is running.
This is the logs I'm getting when I run docker-compose up
db_1 | The files belonging to this database system will be owned by user "postgres".
db_1 | This user must also own the server process.
db_1 |
db_1 | The database cluster will be initialized with locale "en_US.utf8".
db_1 | The default database encoding has accordingly been set to "UTF8".
db_1 | The default text search configuration will be set to "english".
db_1 |
db_1 | Data page checksums are disabled.
db_1 |
db_1 | initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
db_1 | If you want to create a new database system, either remove or empty
db_1 | the directory "/var/lib/postgresql/data" or run initdb
db_1 | with an argument other than "/var/lib/postgresql/data".
The error I'm getting when I access http://localhost:3000 is
could not connect to server: No route to host Is the server running on host "db" (172.18.0.2) and accepting TCP/IP connections on port 5432?
I think you should use volume for Postgres too.
services:
db:
image: postgres:alpine
restart: always
volumes:
- postgres_volume:/var/lib/postgresql/data
volumes:
postgres_volume:
I had similar issue and fixed it with that. Try also to restart Docker.

docker-compose error connecting to redis + sidekiq

I am trying to build a container with docker but I cannot connect sidekiq + redis, the error says sidekiq_1 | Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED), seems to sidekiq is trying to connect to localhost but since I am building "in theory" redis + sidekiq + rails + postgres containers is not in the localhost, it should be in redis image.
My docker-compose.yml file is this:
version: '3'
services:
postgres:
image: postgres:10.5
volumes:
- my_app-postgres:/var/lib/postgresql/data
redis:
image: redis:4.0.11
volumes:
- my_app-redis:/var/lib/redis/data
web:
build: .
command: bundle exec rails server -p 3000 -b '0.0.0.0'
ports:
- '3000:3000'
depends_on:
- postgres
- redis
volumes:
- .:/my_app
env_file:
- .env
sidekiq:
build: .
command: bundle exec sidekiq -C config/sidekiq.yml
volumes:
- .:/my_app
depends_on:
- postgres
- redis
env_file:
- .env
volumes:
my_app-postgres:
my_app-redis:
another interesting 'info' I see in log is Booting Sidekiq 4.2.10 with redis options {:url=>nil} this url can be the cause of the issue?
in my development environment the app is working fine, I try to 'dockerize' what I have. How could I make this work?
By default, sidekiq tries to connect to 127.0.0.1:6379 but your sidekiq is in a different container than redis, so you need to configure sidekiq to use redis:6379 as redis host, e.g. by using an initializer:
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://redis:6379/12' }
end
Take a look at the docs for more details: https://github.com/mperham/sidekiq/wiki/Using-Redis
If you are planning on using Kubernetes for deployment later on, you can put all containers in a pod and then they would be able to connect via localhost because containers within the same Kubernetes pod share the network space. To program directly within a pod inside a Kubernetes cluster, you could work with a tool I recently open sourced on GitHub called DevSpace: https://github.com/covexo/devspace
Create two initializer files:
i) redis.rb
uri = "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0/your-app-cache" || 'redis://localhost:6379/0/your-app-cache'
Rails.application.config.cache_store = :redis_store, uri
ii) sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Sidekiq.configure_client do |config|
config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/12" }
end
Full Sample
./Dockerfile
FROM ruby:2.6.3-alpine
ENV BUNDLER_VERSION=2.0.2
RUN apk add --update --no-cache \
binutils-gold \
build-base \
curl \
file \
g++ \
gcc \
git \
less \
libstdc++ \
libffi-dev \
libc-dev \
linux-headers \
libxml2-dev \
libxslt-dev \
libgcrypt-dev \
make \
netcat-openbsd \
nodejs \
openssl \
pkgconfig \
postgresql-dev \
python \
tzdata \
yarn
ARG USER=root
ARG WORK_DIR_PATH=/home
RUN mkdir -p $WORK_DIR_PATH && chown -R $USER:$USER $WORK_DIR_PATH
WORKDIR $WORK_DIR_PATH
COPY Gemfile* ./
RUN gem install bundler
RUN bundle config build.nokogiri --use-system-libraries
RUN bundle check || bundle install
COPY package.json yarn.lock ./
RUN yarn install --check-files
COPY . .
./.env
APP_NAME=api
APP_PORT=3100
ENV=production
DATABASE_NAME=rails_db
DATABASE_USER=batman
DATABASE_PASSWORD=super_pass_123
DATABASE_PORT=5342
DATABASE_HOST=api_db # must be equal to the name of the postgres service in docker-compose.yml
SECRET_KEY_BASE=your_secret_string
REDIS_HOST=redis # must be equal to the name of the redis service in docker-compose.yml
REDIS_PORT=6379
./docker-compose.yml
version: '3.7'
services:
api:
build:
context: .
dockerfile: Dockerfile
container_name: ${APP_NAME}
#restart: unless-stopped
depends_on:
- api_db
- redis
ports:
- "${APP_PORT}:${APP_PORT}"
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
entrypoint: ./sh/entrypoints/api-entrypoint.sh
api_db:
image: postgres
command: postgres -p ${DATABASE_PORT}
ports:
- "${DATABASE_PORT}:${DATABASE_PORT}"
volumes:
- db_data:/var/lib/postgresql/data
- ./log/db:/logs
environment:
- POSTGRES_USER=${DATABASE_USER}
- POSTGRES_PASSWORD=${DATABASE_PASSWORD}
- POSTGRES_DB=${DATABASE_NAME}
redis:
image: redis
ports:
- "${REDIS_PORT}:${REDIS_PORT}"
command: redis-server
volumes:
- redis:/data
sidekiq:
build:
context: .
dockerfile: Dockerfile
depends_on:
- api_db
- redis
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
env_file: .env
environment:
RAILS_ENV: ${ENV}
ENABLE_BOOTSNAP: 'false'
entrypoint: ./sh/entrypoints/sidekiq-entrypoint.sh
volumes:
redis:
gem_cache:
db_data:
node_modules:
./sh/entrypoints/api-entrypoint.sh
https://stackoverflow.com/a/59047028/4488252
#!/bin/sh
DB_INITED=0
if db_version=$(bundle exec rake db:version 2>/dev/null)
then
if [ "$db_version" = "Current version: 0" ]
then
echo "DB is empty"
else
echo "DB exists"
DB_INITED=1
fi
bundle exec rake db:migrate
else
echo "DB does not exist"
bundle exec rake db:setup
fi
if [ $DB_INITED == 0 ]
then
echo "Performing initial configuration"
# init some plugins, updated db if need, add initial data
fi
bundle exec rails assets:precompile
bundle exec rails s -b 0.0.0.0 -p $APP_PORT
./sh/entrypoints/sidekiq-entrypoint.sh
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle exec sidekiq
./config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
database: <%= ENV['DATABASE_NAME'] %>
username: <%= ENV['DATABASE_USER'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
port: <%= ENV['DATABASE_PORT'] || '5432' %>
host: <%= ENV['DATABASE_HOST'] %>
development:
<<: *default
test:
<<: *default
production:
<<: *default
secret_key_base: <%= ENV['SECRET_KEY_BASE'] %>
./config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
Sidekiq.configure_client do |config|
config.redis = { :url => "redis://#{ENV['REDIS_HOST']}:#{ENV['REDIS_PORT']}/" }
end
./.dockerignore
https://gist.github.com/neckhair/ace5d1679dd896b71403fda4bc217b9e
.git
.gitignore
README.md
#
# OS X
#
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear on external disk
.Spotlight-V100
.Trashes
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
#
# Rails
#
.env
.env.sample
*.rbc
capybara-*.html
log
tmp
db/*.sqlite3
db/*.sqlite3-journal
public/system
coverage/
spec/tmp
**.orig
.bundle
.ruby-version
.ruby-gemset
.rvmrc
# if using bower-rails ignore default bower_components path bower.json files
vendor/assets/bower_components
*.bowerrc
bower.json
# Logs
logs
*.log
# Runtime data
pids
*.pid
*.seed
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# node-waf configuration
.lock-wscript
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directory
# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git
node_modules
server/*.spec.js
kubernetes
Usage
https://docs.docker.com/compose/reference/down/
build and run: docker-compose up --build -d
https://docs.docker.com/compose/reference/down/
stop: docker-compose down
stop + delete images and volumes: docker-compose down --rmi all --volumes

Can't launch sphinxsearch in docker

I'm trying to dockerize a Rails application. It uses Sphinx for search, and I can't make it run through docker.
This is what happens when I run docker-compose up and try to perform search:
web_1 | [1fd79fbf-2e77-4af5-90ad-ae3637ada807] Sphinx Query (1.9ms) SELECT * FROM `field_core` WHERE MATCH('soccer') AND `sphinx_deleted` = 0 ORDER BY `name` ASC LIMIT 0, 10000
web_1 | [1fd79fbf-2e77-4af5-90ad-ae3637ada807] Completed 500 Internal Server Error in 27ms (ActiveRecord: 3.0ms)
web_1 | [1fd79fbf-2e77-4af5-90ad-ae3637ada807]
web_1 | ThinkingSphinx::ConnectionError (Error connecting to Sphinx via the MySQL protocol. Can't connect to MySQL server on '127.0.0.1' (111)):
web_1 | app/controllers/fields_controller.rb:7:in `search'
This is result of docker-compose run sphinx rake ts:index:
sh: 1: searchd: not found
The Sphinx start command failed:
Command: searchd --pidfile --config "/app/config/development.sphinx.conf"
Status: 127
Output: See above
There may be more information about the failure in /app/log/development.searchd.log.
docker-compose.yml:
version: '3'
services:
db:
image: circleci/mysql:5.7
restart: always
volumes:
- mysql_data:/var/lib/mysql
ports:
- "3309:3309"
expose:
- '3309'
web:
build: .
command: rails server -p 3000 -b '0.0.0.0'
ports:
- "3000:3000"
expose:
- '3000'
depends_on:
- db
- sphinx
volumes:
- app:/app
sphinx:
container_name: sociaball_sphinx
image: stefobark/sphinxdocker
restart: always
links:
- db
volumes:
- /app/config/sphinxy.conf:/etc/sphinxsearch/sphinxy.conf
- /app/sphinx:/var/lib/sphinx
volumes:
mysql_data:
app:
Dockerfile:
FROM ruby:2.4.1
RUN apt-get update && apt-get install -qq -y build-essential nodejs --fix-missing --no-install-recommends
RUN curl -s \
http://sphinxsearch.com/files/sphinxsearch_2.3.2-beta-1~wheezy_amd64.deb \
-o /tmp/sphinxsearch.deb \
&& dpkg -i /tmp/sphinxsearch.deb \
&& rm /tmp/sphinxsearch.deb \&& mkdir -p /var/log/sphinxsearch
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN gem install bundler && bundle install --jobs 20 --retry 5
COPY . ./
EXPOSE 3000
CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]
thinking_sphinx.yml:
development: &common
min_infix_len: 1
charset_table: "0..9, english, U+0021..U+002F"
port: 9306
address: sociaball_mysql_1
production:
<<: *common
So, rake isn't available in sphinx container, and sphinx scripts aren't available in app's container. What am I doing wrong?
Thinking Sphinx expects a copy of the Rails app when running the rake tasks, so you'll need to have a copy of your app within your Sphinx container. This will ensure (once you've bundled the gems) that rake exists in the Sphinx container, where the Sphinx binaries are also present.
So.
Instead of running rake task, I just did what it does directly in sphinx container. Like this:
docker-compose run --rm sphinx indexer \
--config "/etc/sphinxsearch/sphinxy.conf" --all --rotate
Regarding 500 error. It was caused by incorrect configuration in thinking_sphinx.yml. It should have pointed to remote host with sphinx instead of db:
development: &common
# ...
address: sociaball_sphinx_1

fe_sendauth: no password supplied when creating database

I am in the process of dockerizing our Ruby on rails apps. One of them is throwing the following error when I run docker-compose run web bundle exec rake db:create
Starting sapi_db_1 ... done
[WARNING] The git gem requires git 1.6.0.0 or later, but only found
2.1.4. You should probably upgrade.
fe_sendauth: no password supplied
I am confirmed the config files are working as I have used them 3 times previously with oher apps.
Here's my dockerfile
FROM ruby:2.2.3
RUN apt-get update && apt-get -y install build-essential libpq-dev nodejs
RUN gem install bundler
RUN mkdir /app
WORKDIR /app
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock
RUN bundle install
ADD . /app
My docker-compose.yml
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/sapi
ports:
- "3000:3000"
environment:
DATABASE_URL: postgres://postgres#db
env_file:
- '.env'
depends_on:
- db
db:
image: postgres:latest
ports:
- "5432:5432"
env_file:
- '.env'
My database.yml file
default: &default
adapter: postgresql
pool: 5
timeout: 5000
url: <%= ENV['DATABASE_URL'] %>
username: <%= ENV['POSTGRES_USER'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
development:
<<: *default
database: app_development
How can I resolve this? I have already confirmed that the config files work by carrying the process out on other apps.
Resolved by changing the .env file

Resources