I have simple rails application with docker-compose.yml file.
It consists from two containers - db container with PostgreSQL and web container with rails app.
In dockerfile for web part I have such lines in CMD
CMD RAILS_ENV=production rake db:create db:migrate && \
bundle exec rails s -p 3000 -b '0.0.0.0' --environment=production
So in line rake db:create db:migrate I create db if it is a first run of db container, and run migrate.
But if it is only update of web part - I need only to run db:migrate, and db:create (as it should) give me error
ERROR: database "myapp_production" already exists
STATEMENT: CREATE DATABASE "myapp_production" ENCODING = 'unicode'
Everything working fine, but I think there is a better way.
What is a best way to handle this situation?
I have the same development stack and here is that I'm doing.
Here is a Dockerfile for postgres which I'm extend:
FROM postgres:9.4
ADD db/init.sql /docker-entrypoint-initdb.d/
EXPOSE 5432
CMD ["postgres"]
From the docker postgres documentation:
If you would like to do additional initialization in an image derived
from this one, add one or more *.sql or *.sh scripts under
/docker-entrypoint-initdb.d (creating the directory if necessary).
After the entrypoint calls initdb to create the default postgres user
and database, it will run any *.sql files and source any *.sh scripts
found in that directory to do further initialization before starting
the service.
My init.sql:
CREATE USER database_user;
CREATE DATABASE database_production;
GRANT ALL PRIVILEGES ON DATABASE database_production TO database_user;
After that my RUN command in the web container points to the run.sh script:
#!/usr/bin/env bash
echo "Bundling gems"
bundle install --jobs 8 --retry 3
echo "Clearing logs"
bin/rake log:clear
echo "Run migrations"
bundle exec rake db:migrate
echo "Seed database"
bundle exec rake db:seed
echo "Removing contents of tmp dirs"
bin/rake tmp:clear
echo "Starting app server ..."
bundle exec rails s -p 3000 -b '0.0.0.0'
That's it. My database created in the db container, and web app only does migration.
OS: Ubuntu 14.04
RAILS 3.2
In my gemfile, I have:
gem 'sunspot_rails'
gem 'sunspot_solr'
The application was running fine, until I cleaned it up, and part of the process included deleting the app/solr directory.
In my config/sunspot.yml file, I have:
staging:
solr:
hostname: localhost
port: 8983
log_level: INFO
path: /solr/staging
I ran:
RAILS_ENV=staging RAILS_GROUPS=assets bundle exec rake assets:precompile
then:
rm -Rf /home/app/solr
mkdir /home/app/solr
mkdir /home/app/solr/staging
rake sunspot:solr:stop
RAILS_ENV=staging bundle exec rake sunspot:solr:start
screen -d -m sh -c 'RAILS_ENV=staging bundle exec rakesunspot:solr:reindex[500] > /home/installio/app/solr/indexing_out.txt'
The app/solr directory has in it the following structuere
pids/staging/sunspot-solr-staging.pid
data (empty directory)
staging (empty directory)
And I am getting the "something went wrong message).
From the sunspot-solr-staging.log:
SEVERE: Unable to create core: collection1
org.apache.solr.common.SolrException: Could not load config for solrconfig.xml
Any ideas?
I have set up a RoR environement on AWS' elastic beanstalk. I am able to ssh into my EC2 instance.
My home directory is /home/ec2-user, which is effectively empty.
If I move up a directory, there is also a /home/webapp directory that i do not have access to.
Is there a way to run a rake command or rails console on my elastic beanstalk instance?
If I type rails console I get Usage: rails new APP_PATH [options]
If I type RAILS_ENV=production bundle exec rails console, I get "Could not locate Gemfile"
For rails, jump to /var/app/current then as #juanpastas said, run RAILS_ENV=production bundle exec rails c
Don't know why, but since EBS runs everything as root, this worked for me:
sudo su
bundle exec rails c production
None of these solutions mentioned here worked for me, so I cooked up a little script that I put in script/aws-console.
You can run it from the /var/app/current directory as root:
eb ssh
cd /var/app/current
sudo script/aws-console
My script can be found as a Gist here.
None of the other answers worked for me so I went looking - this is working for me now on an elastic beanstalk 64bit amazon linux 2016.03 V2.1.2 ruby 2.2 (puma) stack
cd /var/app/current
sudo su
rake rails:update:bin
bundle exec rails console
that returns me the expected console
Loading production environment (Rails 4.2.6)
irb(main):001:0>
For Ruby 2.7:
if you don't need environment variables:
BUNDLE_PATH=/var/app/current/vendor/bundle/ bundle exec rails c
It looks like environment variables are not loaded automatically anymore, which might prevent rails console from starting.
I solved it by creating this .ebextensions file:
# Simply call `sudo /var/app/scripts/rails_c`
commands:
create_script_dir:
command: "mkdir -p /var/app/scripts"
ignoreErrors: true
files:
"/var/app/scripts/export_envvars":
mode: "000755"
owner: root
group: root
content: |
#!/opt/elasticbeanstalk/.rbenv/shims/ruby
if __FILE__ == $0
require 'json'
env_file = '/var/app/scripts/envvars'
env_vars = env_vars = JSON.parse(`/opt/elasticbeanstalk/bin/get-config environment`)
str = ''
env_vars.each do |key, value|
new_key = key.gsub(/\s/, '_')
str << "export #{new_key}=\"#{value}\"\n"
end
File.open(env_file, 'w') { |f| f.write(str) }
end
"/var/app/scripts/rails_c":
mode: "000755"
owner: root
group: root
content: |
. ~/.bashrc
/var/app/scripts/export_envvars
. /var/app/scripts/envvars
cd /var/app/current
/opt/elasticbeanstalk/.rbenv/shims/bundle exec rails c
Create a .ebextension file named setvars.config and add those lines to it
commands:
setvars:
command: /opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' > /etc/profile.d/sh.local
packages:
yum:
jq: []
Then deploy your code again it should work.
reference: https://aws.amazon.com/ar/premiumsupport/knowledge-center/elastic-beanstalk-env-variables-shell/
For Ruby 2.7:
As someone said, if you don't need env vars, run the following
BUNDLE_PATH=/var/app/current/vendor/bundle/ bundle exec rails c
However, if you need ENV, I recommend doing this as per AWS doc:
https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-env-variables-linux2/
tl;dr
On Amazon Linux 2, all environment properties are centralised into a single file called /opt/elasticbeanstalk/deployment/env. No user can access these outside the app. So, they recommend to add some hook scripts after deploy to basically create a copy.
#!/bin/bash
#Create a copy of the environment variable file.
cp /opt/elasticbeanstalk/deployment/env /opt/elasticbeanstalk/deployment/custom_env_var
#Set permissions to the custom_env_var file so this file can be accessed by any user on the instance. You can restrict permissions as per your requirements.
chmod 644 /opt/elasticbeanstalk/deployment/custom_env_var
#Remove duplicate files upon deployment.
rm -f /opt/elasticbeanstalk/deployment/*.bak
If because of some reason you don't want to run as root, do the following to pass env vars from root into new user environment:
sudo -u <user> -E env "PATH=$PATH" bash -c 'cd /var/app/current/ && <wtv you want to run>'
I like to create an eb_console file at the root of my rails app, then chmod u+x it. It contains the following:
ssh -t ec2-user#YOUR_EC2_STATION.compute.amazonaws.com 'cd /var/app/current && bin/rails c'
This way, I just have to run:
./eb_console
like I would have run heroku run bundle exec rails c.
#!/bin/sh
shell_join () {
ruby -r shellwords -e 'puts Shellwords.join(ARGV)' "$#"
}
command_str () {
printf 'set -e; . /etc/profile.d/eb_envvars.sh; . /etc/profile.d/use-app-ruby.sh; set -x; exec %s\n' "$(shell_join "$#")"
}
exec sudo su webapp -c "$(command_str "$#")"
Put above file somewhere in your source code, deploy, eb ssh into the eb instance, cd /var/app/current, and then execute path/to/above/script bin/rails whatever argumeents you usually use.
Reason why I have written above script is:
When using sudo, it drops some environment variables which might actually be needed for your rails app; so manually load the profiles which the Elastic Beanstalk platform provides.
Current Beanstalk ruby platform assumes you run rails application on user webapp, a non-login-able user, so it would be wise to run your command in this user.
For the latest ruby version, please use the following command:
BUNDLE_PATH=/opt/rubies/ruby-2.6.3/lib/ruby/gems/2.6.0/ bundle exec rails c production
Running it with sudo is not needed.
add an eb extension shortcut:
# .ebextensions/irb.config
files:
"/home/ec2-user/irb":
mode: "000777"
owner: root
group: root
content: |
sudo su - -c 'cd /var/app/current; bundle exec rails c'
then:
$ eb ssh
$ ./irb
irb(main):001:0>
None of these were working for me, including the aws-console script. I finally ended up creating a script directory in /var/app/current and then creating a rails file in that directory as outline by this answer on another SO question.
eb ssh myEnv
cd /var/app/current
sudo mkdir script
sudo vim script/rails
Add this to file and save:
echo #!/usr/bin/env ruby
# This command will automatically be run when you run "rails" with Rails 3 gems installed from the root of your application.
APP_PATH = File.expand_path('../../config/application', __FILE__)
require File.expand_path('../../config/boot', __FILE__)
require 'rails/commands'
Then make it executable and run it:
sudo chmod +x script/rails
sudo script/rails console
And it worked.
You have to find the folder with your Gemfile :p.
To do that, I would take a look in you web server config there should be a config that tells you where your app directory is.
Maybe you know where your app is.
But in case you don't know, I would give a try to:
grep -i your_app_name /etc/apache/*
grep -i your_app_name /etc/apache/sites-enabled/*
To search files containing your_app_name in Apache config.
Or if you are using nginx, replace apache above by nginx.
after you find application folder, cd into it and run RAILS_ENV=production bundle exec rails c.
Making sure that your application is configured to run in production in Apache or nginx configuration.
I do the following on my server:
pg_dump -O -c register_production > register.sql
Then, after copying register.sql to my local environment, I try:
psql register_development < register.sql
This appears to work, but when I try to launch the Rails site locally, I get this:
PG::UndefinedTable: ERROR: relation "list_items" does not exist at character 28
How can I restore everything (including relations) from the server db to my local dev db?
I use this command to save my database:
pg_dump -F c -v -U postgres -h localhost <database_name> -f /tmp/<filename>.psql
And this to restore it:
pg_restore -c -C -F c -v -U postgres /tmp/<filename>.psql
This dumps the database in Postgres' custom format (-F c) which is compressed by default and allows for reordering of its contents. -C -c will drop the database if it exists already and then recreate it, helpful in your case. And -v specifies verbose so you can see exactly what's happening when this goes on.
Does the register_development database exist before you run the psql command? Because that form will not create it for you.
See http://www.postgresql.org/docs/8.1/static/backup.html#BACKUP-DUMP-RESTORE for more information.
I'm trying to import my production Heroku database into my development machine.
My local db is PostgreSQL.
First, I'm exporting the dump from Heroku to my machine
curl -o latest.dump `heroku pgbackups:url`
Then, I try to drop the local db with rake db:drop and then I create the empty database again by using rake db:create.
The problem I'm getting is when actually trying to import the dump to the database
psql -d app_development -U myusername -f mydumpfile.sql
I begin seeing errors like this
psql:latest.dump:24: ERROR: syntax error at or near "PGDMP"
LINE 1: PGDMP
^
psql:latest.dump:28: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
^
psql:latest.dump:36: ERROR: syntax error at or near ""
LINE 1: id integer NOT NULL,
^
psql:latest.dump:40: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
^
psql:latest.dump:45: ERROR: syntax error at or near ""
LINE 1: id integer NOT NULL,
^
psql:latest.dump:49: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
...
psql:latest.dump:1601: invalid command \S4???(?̭?A?|c?e0<00K?A?}FϚ?????A(??~?t?I?????G(? K???l??k"?H?ȁ?ͲS?,N*?[(#??a5J??j}
psql:latest.dump:1602: invalid command \??k???|??w???h?
psql:latest.dump:1603: invalid command \=??????o?h?
psql:latest.dump:1609: invalid command \????^.?????????E???/-???+??>#?ؚE?.2)Ȯ&???? g????"7},_??]?:?f?Tr|o???)?p????h?KO?08[Rqu???|3?cW?ڮ?ahbm??H?H8??$???2?a?-أ
psql:latest.dump:1613: invalid command \D!qVS???L??*??R??I!???
psql:latest.dump:1614: invalid command \??-?}Q
psql:latest.dump:12565: ERROR: invalid byte sequence for encoding "UTF8": 0xb0
Any idea what is happening this and how to solve it?
You see errors because psql tries to interpret SQL queries when you're actually giving him a compressed dump (that's what heroku uses).
While you can't read the dump, pg_restore -O latest.dump gives you valid SQL you could pipe to psql but the easy solution is the following one :
pg_restore -O -d app_development latest.dump
Notes :
Use -O because you probably don't use the random username of your remote heroku postgres db.
Heroku doesn't recommend to use taps but I don't know how really risky it is.
Follow these 4 simple steps in your terminal(Heroku Dev Center):
Create a backup copy of your database:
$ heroku pg:backups capture DATABASE_NAME
Download the copy from Heroku (to your local machine) using curl:
$ curl -o latest.dump `heroku pg:backups public-url`
Load it*:
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U YOUR_USERNAME -d DATABASE_NAME latest.dump
get YOUR_USERNAME and choose the desired database from your config/database.yml file.
DATABASE_NAME can be your development/test/production db (Ex. mydb_development)
That's it!
I wanted to avoid having to set up Postgres on my local machine (blowing away and recreating the database is a pain if you're just looking for quick instructions). I put together some exact instructions for doing this with a local Postgres database running Docker. I'm adding a link here, since Google kept bringing me to this question (and it's a possible solution, though probably not what you're looking for):
https://gist.github.com/locofocos/badd43131f14b3e40c760741d5a26471
Heroku export the .dump extension file of the db to import in any of the relational DB by having its own norms and conditions.
While importing it to the local postgres DB, first you download the file latest.dump into your local machine and then run
pg_restore -h localhost -p 5432 -U postgres_username -d db_name -v latest.dump
and restart the rails server.
Late 2021 update for the highest voted answer to date (works great):
$ rails db:drop db:create db:migrate
$ heroku pg:backups capture DATABASE_URL
$ curl -o latest.dump heroku pg:backups public-url
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U YOUR_USERNAME -d DATABASE_NAME latest.dump
get YOUR_USERNAME on your local machine
DATABASE_NAME can be your development/test/production db (Ex. rails_react_bootstrap_development) from your config/database.yml file.
DATABASE_URL is not a variable or example code you need to set. It is a valid heroku option