Rails 6 Docker run rake db:seed command - ruby-on-rails

I am working on rails application and it uses Docker and Kubernetes.
I do not know Docker and Kubernetes and I need to run the seed file on the production server but it's not working.
Following is my Docker file
FROM ruby:2.6.5-alpine
RUN apk update
RUN apk add bash build-base libxml2-dev libxslt-dev postgresql postgresql-dev nodejs vim
yarn libc6-compat curl
RUN mkdir /app
WORKDIR /app
COPY template-app/Gemfile* ./
RUN gem install bundler && bundle config https://gem.abc.io/xyz/ nvHuX-
OXxLY2OpiQkFVfgnYgd4ertyufdds
RUN bundle install
#RUN bundle exec rake db:seed(not working)
COPY template-app/ .
EXPOSE 3000
CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0", "-p", "3000"]
In the Docker file, I have added RUN bundle exec rake db:seed but it's not working and my build is not passed(deployed)
I am using postgres database.
I do not have docker composer file.
Thanks.
Edit:
I got the following error when I write rake db:migrate in the docker file
Removing intermediate container 3634d992e850
---> 1bd575429f8e
Step 9/12 : RUN bundle exec rake db:seed
---> Running in f5d8c63f7db7
rake aborted!
No Rakefile found (looking for: rakefile, Rakefile, rakefile.rb, Rakefile.rb)
/usr/local/bundle/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
/usr/local/bundle/bin/bundle:23:in `load'
/usr/local/bundle/bin/bundle:23:in `<main>'
(See full trace by running task with --trace)
Edit
I have added following code in dockerfile
RUN yarn install --check-files
RUN bundle exec rake db:seed
I am able to run the seed command but get the following error
Edit
My application directory structure
Here my rails app is in template-app folder and dockerfile is outside the folder
Database.yml file
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV["RAILS_DB_POOL"] || ENV['RAILS_MAX_THREADS'] || 5 %>
idle_timeout: <%= ENV["RAILS_IDLE_CONNECTION"] || 60 %>
port: <%= ENV['RAILS_DATABASE_PORT'] || 5432 %>
# For details on connection pooling, see Rails configuration guide
# https://guides.rubyonrails.org/configuring.html#database-pooling
development:
<<: *default
adapter: postgresql
database: <%= ENV['TEMPLATEAPP_DATABASE'] %>
username: <%= ENV['TEMPLATEAPP_DATABASE_USER'] %>
password: <%= ENV['TEMPLATEAPP_DATABASE_PASSWORD'] %>
host: <%= ENV['TEMPLATE_DATABASE_HOSTNAME'] %>
test:
<<: *default
adapter: postgresql
database: <%= ENV['TEMPLATEAPP_DATABASE'] %>
username: <%= ENV['TEMPLATEAPP_DATABASE_USER'] %>
password: <%= ENV['TEMPLATEAPP_DATABASE_PASSWORD'] %>
host: <%= ENV['TEMPLATE_DATABASE_HOSTNAME'] %>
production:
<<: *default
adapter: postgresql
database: <%= ENV['TEMPLATEAPP_DATABASE'] %>
username: <%= ENV['TEMPLATEAPP_DATABASE_USER'] %>
password: <%= ENV['TEMPLATEAPP_DATABASE_PASSWORD'] %>
host: <%= ENV['TEMPLATE_DATABASE_HOSTNAME'] %>

Well there are lot of things going on in your Dockerfile, first you are installing postgresql but you are not doing its initial configuration. Normal we use docker-compose file and use pre build postgresql for it with following docker image
db:
image: postgres:11
ports:
- "5432:5432"
volumes:
- '../../tmp/data/pg11:/var/lib/postgresql/data:cached'
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: ''
POSTGRES_HOST_AUTH_METHOD: trust
PGDATA: /var/lib/postgresql/data/pgdata
Now another issue is you are using rails 6 mean you have to run yarn to install all packages. so I also added following line
RUN yarn install --check-files
The other issue is you are running your db migration before copying your project. you can use following Dockerfile
FROM ruby:2.6.5-alpine
RUN apk update
RUN apk add bash build-base libxml2-dev libxslt-dev postgresql postgresql-contrib postgresql-dev nodejs vim yarn libc6-compat curl
RUN mkdir /app
WORKDIR /app
COPY template-app/Gemfile* ./
RUN gem install bundler
# RUN gem install bundler && bundle config https://gem.abc.io/xyz/ nvHuX-
RUN bundle install
COPY template-app/ .
RUN yarn install --check-files
RUN bundle exec rake db:seed
EXPOSE 3000
CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0", "-p", "3000"]
But as I told you due to postgresql it wont' work, till you setup postgres and take consideration using its credentials in your config/database.yml file.
Please use following format
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV["RAILS_DB_POOL"] || ENV['RAILS_MAX_THREADS'] || 5 %>
idle_timeout: <%= ENV["RAILS_IDLE_CONNECTION"] || 60 %>
port: <%= ENV['RAILS_DATABASE_PORT'] || 5432 %>
# For details on connection pooling, see Rails configuration guide
# https://guides.rubyonrails.org/configuring.html#database-pooling
development:
<<: *default
adapter: postgresql
database: <%= ENV['TEMPLATEAPP_DATABASE'] %>
username: <%= ENV['TEMPLATEAPP_DATABASE_USER'] %>
password: <%= ENV['TEMPLATEAPP_DATABASE_PASSWORD'] %>
host: <%= ENV['TEMPLATE_DATABASE_HOSTNAME'] %>
test:
<<: *default
adapter: postgresql
database: <%= ENV['TEMPLATEAPP_DATABASE'] %>
username: <%= ENV['TEMPLATEAPP_DATABASE_USER'] %>
password: <%= ENV['TEMPLATEAPP_DATABASE_PASSWORD'] %>
host: <%= ENV['TEMPLATE_DATABASE_HOSTNAME'] %>

Related

PG: Operation timed out Is the server running on that host and accepting TCP/IP connections?

I have a rails app with docker setup. I have two containers and i have verified they are on the same network but still i am getting error
PG::ConnectionBad: connection to server at "192.168.160.2", port 5432 failed: Operation timed out Is the server running on that host and accepting TCP/IP connections?
I have changed network too, also removed all images and volumes but still no luck.
Everything was working fine until I composed down the containers and when i tried to run again make setup command i started facing the issue.
Here is my docker-compose.yml
x-toolbox-environment: &toolbox-environment
- PROJECT=frameworq
- APPLICATION=portal
- AWS_REGION
- AWS_DEFAULT_REGION
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_SESSION_TOKEN
- AWS_SECURITY_TOKEN
- AWS_SESSION_EXPIRATION
x-web-volumes: &web-volumes
- .:/app:cached
- bundle_cache:/usr/local/bundle
- rails_cache:/app/tmp/cache:cached
- public_cache:/app/public:cached
x-web-environment: &web-environment
- RAILS_PORT=4000
- MAILER_FROM=noreply#frameworq.com
- APP_DOMAIN=lvh.me
services:
web:
build: .
volumes: *web-volumes
ports:
- '4000:4000'
- '3035:3035'
environment: *web-environment
command: bash -c "bundle exec foreman s"
links:
- db
db:
image: postgres:12
ports:
- 5432:5432
volumes:
- postgres:/var/lib/postgresql:cached
environment:
POSTGRES_HOST_AUTH_METHOD: trust
toolbox:
image: partos/ecs_toolbox:latest
environment: *toolbox-environment
volumes:
- .:/app:cached
- /var/run/docker.sock:/var/run/docker.sock
- .aws:/root/.aws
remote:
build: ./docker/remote
environment: *toolbox-environment
entrypoint: ""
volumes:
- ./docker/remote:/app:cached
- .aws:/root/.aws
volumes:
bundle_cache:
rails_cache:
postgres:
public_cache:
localstack_data:
driver: local
database.yml
# # SQL Server (2012 or higher required)
# #
# # Install the adapters and driver
# # gem install tiny_tds
# # gem install activerecord-sqlserver-adapter
# #
# # Ensure the activerecord adapter and db driver gems are defined in your Gemfile
# # gem 'tiny_tds'
# # gem 'activerecord-sqlserver-adapter'
# #
# default: &default
# adapter: sqlserver
# encoding: utf8
# azure: false
# mode: dblib
# host: localhost
# username: sa
# password: this-is-the-kms-test-database-p2ssw0rd
# azure: &azure
# adapter: sqlserver
# encoding: utf8
# azure: true
# mode: dblib
# host: <%= Rails.application.credentials.database_host %>
# username: <%= Rails.application.credentials.database_user %>
# password: <%= Rails.application.credentials.database_password %>
# #----------
# development:
# <<: *default
# database: kmsDev
# legacy:
# <<: *azure
# database: kmsprod
# # Warning: The database defined as "test" will be erased and
# # re-generated from your development database when you run "rake".
# # Do not set this db to the same as development or production.
# test:
# <<: *default
# database: kmsTest
# production:
# <<: *azure
# database: <%= ENV['DATABASE_NAME'] %>
default: &default
adapter: postgresql
encoding: unicode
host: <%= ENV["DB_HOST"] %>
database: <%= ENV["DB_NAME"] %>
username: <%= ENV["DB_USERNAME"] %>
password: <%= ENV["DB_PASSWORD"] %>
port: <%= ENV["DB_PORT"] %>
# For details on connection pooling, see Rails configuration guide
# https://guides.rubyonrails.org/configuring.html#database-pooling
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
development:
<<: *default
host: <%= ENV.fetch("PGHOST") { 'db' } %>
database: <%= ENV.fetch("POSTGRES_DB") { 'frameworq_development' } %>
username: <%= ENV.fetch("POSTGRES_USER") { 'postgres' } %>
password: <%= ENV.fetch("POSTGRES_PASSWORD") { '' } %>
port: <%= ENV.fetch("PGPORT") { 5432 } %>
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: frameworq_test
host: <%= ENV.fetch("PGHOST") { 'db' } %>
database: <%= ENV.fetch("POSTGRES_DB") { 'frameworq_test' } %>
username: <%= ENV.fetch("POSTGRES_USER") { 'postgres' } %>
password: <%= ENV.fetch("POSTGRES_PASSWORD") { '' } %>
port: <%= ENV.fetch("PGPORT") { 5432 } %>
qa:
<<: *default
staging:
<<: *default
production:
<<: *default
Dockerfile
FROM ruby:3.0.4-alpine
RUN apk --update add --no-cache build-base bash git vim curl postgresql-dev openssl-dev nodejs npm yarn tzdata less \
imagemagick postgresql-client gcompat chromium chromium-chromedriver
RUN mkdir /app
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN bundle update --bundler
RUN bundle install --jobs 5
ADD . /app
RUN yarn install
RUN RAILS_SECRET_KEY_BASE=secret_key_base RAILS_ENV=production bundle exec rails assets:precompile apipie:cache
EXPOSE 5000
VOLUME ["/app/public", "/usr/local/bundle"]
CMD bash -c "bundle exec puma -C config/puma.rb"
Makefile
# DEVELOPMENT TARGETS
setup: destroy_shell build_shell prepare_shell up
destroy_shell:
docker-compose down -v
build_shell:
docker-compose build
prepare_shell:
docker-compose run --rm web bash -c "make prepare"
up:
docker-compose run --rm --service-ports --use-aliases web
shell:
docker-compose run --rm web bash
toolbox:
docker-compose run --rm toolbox bash
prepare_bundle:
bundle -j 4
yarn
prepare_db_first_time:
bundle exec rails db:create db:schema:load
prepare_db:
bundle exec rails db:migrate:with_data
prepare_test_db:
bundle exec rails db:create db:schema:load
prepare: prepare_bundle prepare_db_first_time
check: test_web fix_lint_web lint_front
lint_web:
bundle exec rubocop
fix_lint_web:
bundle exec rubocop -A
test_web:
RAILS_ENV=test bundle exec rails assets:precompile && bundle exec rails test
test_compiled_web:
RAILS_ENV=test bundle exec rails test
prepare_test_assets:
# We're not using webpacker now, importmaps instead
# cp -R public/packs public/packs-test
RAILS_ENV=test bundle exec rails assets:precompile
test_web_with_db: prepare_test_assets prepare_test_db test_compiled_web
lint_front:
# We're using importmaps, not yarn now
# yarn install && yarn run eslint --fix app/javascript/
generate_ts_routes:
docker-compose run --rm web bash -c 'bundle exec rails ts:routes'
db-reset:
bundle exec rails db:drop
bundle exec rails db:create
bundle exec rails db:schema:load
bundle exec rails db:migrate
bundle exec rails db:seed
eslint:
# We're using importmaps, not yarn now
# NODE_ENV=development yarn install
# yarn eslint --ext .js --ext .jsx /app/app/javascript/
tailwind:
docker-compose run --rm web bash -c 'bin/rails tailwindcss:watch'
guard:
docker-compose run --rm web bash -c 'bundle exec guard'
console:
docker-compose run --rm web bash -c 'rails c'
taildev:
docker-compose run --rm web bash -c 'tail -f log/development.log'
tailstaging:
docker-compose run --rm web bash -c 'tail -f log/staging.log'

EC2 backend docker run issues after ssh into instance

Im attempting to bring up the backend on this Ruby on Rails app using docker but am getting the following error after I try to run the following commands on my ssh'd ec2 instance.
[ec2-user#ip-10-10-3-155 ~]$ docker run --rm \
>-it --name *****-db-setup \
>839014*****.dkr.ecr.us-east-1.amazonaws.com/*****-backend:dev-latest \
>bundle exec rake db:drop db:create db:migrate
rake aborted!
PG::ConnectionBad: could not translate host name "db"
to address: Name or service not known
/usr/local/bundle/gems/pg-0.21.0/lib/pg.rb:56:in `initialize'
/usr/local/bundle/gems/pg-0.21.0/lib/pg.rb:56:in `new'
/usr/local/bundle/gems/pg-0.21.0/lib/pg.rb:56:in `connect'
/usr/local/bundle/gems/activerecord-5.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:695:in
`connect'
/usr/local/bundle/gems/activerecord-5.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:220:in
`initialize'
/usr/local/bundle/gems/activerecord-5.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:38:in `new'
/usr/local/bundle/gems/activerecord-5.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:38:in `postgresql_connection' /myapp/app/models/admin.rb:7:in `include'
/myapp/app/models/admin.rb:7:in `<class:Admin>'
/myapp/app/models/admin.rb:3:in `<top (required)>'
/usr/local/bundle/gems/activesupport-5.1.4/lib/active_support/dependencies.rb:476:in
`load' .......Tasks: TOP => db:drop =>
db:check_protected_environments => environment
this is my env file I created
$ export AWS_ACCESS_KEY_ID=***********
$ export AWS_SECRET_ACCESS_KEY=***************************
$ export AWS_DEFAULT_REGION=us-east-1
$ export DB_USER=*****
$ export DB_PASS=****test1
$ export DB_NAME=*****_backend
$ export DB_HOST=tf-*****-backend-2021051***********.c8o12l*****.us-east-1.rds.amazonaws.com
$ export DB_PORT=5432
Docker-compose.yml
version: '3'
services:
db:
image: postgres
backend:
build: backend
# https://stackoverflow.com/a/38732187/42559
entrypoint: /myapp/entrypoint.sh
command: bundle exec unicorn -p 3001
volumes:
- ./backend/:/myapp
ports:
- "3001:3001"
depends_on:
- db
env_file:
- ./.env
frontend:
build: frontend
command: yarn server
volumes:
- ./frontend/:/app
- /app/node_modules
ports:
- "3000:3000"
depends_on:
- backend
This is the Database.yml
default: &default
adapter: postgresql
encoding: unicode
host: <%= ENV.fetch("DB_HOST") { "db" } %>
username: <%= ENV.fetch("DB_USER") { "postgres" } %>
port: <%= ENV["DB_PORT"] %>
password: <%= ENV["DB_PASS"] %>
# For details on connection pooling, see Rails configuration guide
# http://guides.rubyonrails.org/configuring.html#database-pooling
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
development:
<<: *default
database: backend_development
test:
<<: *default
database: backend_test
production:
<<: *default
database: <%= ENV["DB_NAME"] %>
From my observations and attempts I believe it due to how the Docker-compose.yml and env file are interacting or something with my ruby server. I already have the backend image in the EC2 instance repository, now im trying to convert it to a container and bring up the backend. Thanks for any insight provided in advance !

connecting an aws elastic beanstalk ruby on rails app to an aws postgres rds

i have a ruby rails app i deployed to aws with elastic beanstalk, i need to connect it to aws rds
heres my database.yml
production:
adapter: postgresql
database: <%= ENV['RDS_DB_NAME'] %> #
username: <%= ENV['RDS_USERNAME'] %>
password: <%= ENV['RDS_PASSWORD'] %>
host: <%= ENV['RDS_HOSTNAME'] %>
port: <%= ENV['RDS_PORT'] %>
i am able to connect to my aws db from pgadmin but ones i deploy that database.yml
i get this error
+ cd /var/app/ondeck
+ su -s /bin/bash -c 'bundle exec /opt/elasticbeanstalk/support/scripts/check-for-rake-task.rb db:migrate' webapp
`/home/webapp` is not a directory.
Bundler will use `/tmp/bundler20200525-23041-9y70lv23041' as your home directory temporarily.
+ '[' false = true ']'
+ su -s /bin/bash -c 'leader_only bundle exec rake db:migrate' webapp
`/home/webapp` is not a directory.
Bundler will use `/tmp/bundler20200525-23045-1bhxksf23045' as your home directory temporarily.
rake aborted!
PG::ConnectionBad: could not connect to server: Connection timed out
Is the server running on host "aapujs6mg3rasw.cxytyfiwlama.us-east-1.rds.amazonaws.com" (172.31.39.107) and accepting
TCP/IP connections on port 3306?
/opt/rubies/ruby-2.6.6/bin/bundle:23:in `load'
/opt/rubies/ruby-2.6.6/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate

Codeship build failing with NoDatabaseError

The builds for my rails app are failing with the following error on running tests
9) FailedKeyChecks should contain a value
Failure/Error: ActiveRecord::Base.connection.execute(query)
ActiveRecord::NoDatabaseError:
FATAL: database "myapp_development" does not exist
10) User is invalid without name
Failure/Error: raise ActiveRecord::NoDatabaseError.new(error.message, error)
ActiveRecord::NoDatabaseError:
FATAL: database "myapp_development" does not exist
what am I doing wrong? My codeship.database.yml file
development:
adapter: postgresql
host: localhost
encoding: unicode
pool: 10
username: <%= ENV['PG_USER'] %>
template: template1
password: <%= ENV['PGPASSWORD'] %>
database: development<%= ENV['TEST_ENV_NUMBER'] %>
port: <%= ENV['DATABASE_PORT'] %>
sslmode: disable
test:
adapter: postgresql
host: localhost
encoding: unicode
pool: 10
username: <%= ENV['PG_USER'] %>
template: template1
password: <%= ENV['PGPASSWORD'] %>
database: test<%= ENV['TEST_ENV_NUMBER'] %>
port: <%= ENV['DATABASE_PORT'] %>
sslmode: disable
my setup commands
rvm use 2.4.1 --install
bundle install
cp codeship.database.yml config/database.yml
export RAILS_ENV=test
bundle exec rake db:create
#bundle exec rake db:migrate
bundle exec rake db:schema:load
and my test commands
RAILS_ENV=test bundle exec rake
RAILS_ENV=test bundle exec rspec
any idea why development database is being referenced? Thanks in advance
Try running bundle exec rake db:create:all
followed by bundle exec rake db:migrate
It might have failed because you forgot the 'all'

rake aborted:PG::ConnectionBad: fe_sendauth: no password supplied

I'm on Cloud9 Ubuntu Template and I installed postgres. I'm getting an error when I try to do a "rake db:migrate".
rake aborted:PG::ConnectionBad: fe_sendauth: no password supplied
Related settings in my database.yml file
default: &default
adapter: postgresql
encoding: unicode
pool: 5
username: <%= ENV['USERNAME'] %>
password: <%= ENV['PASSWORD'] %>
host: <%= ENV['IP'] %>
development:
<<: *default
database: app_development
It seems like Cloud9 issue. I refer https://community.c9.io/t/fe-sendauth-no-password-supplied-error-after-setting-up-postgrsql-on-rails/2206/2
Run below commands on your c9 terminal.
$ source ~/.profile
$ rake db:create
$ rake db:migrate
It works for me.

Resources