I am having a very difficult time trying to launch a sample Rails 6 application to Elastic Beanstalk. For context, I am following these instructions
ADD RDS to Ruby Application
ADD an RDS to Beanstalk
I have followed these instructions to a tee and am still unable to connect to the rds database that I have provisioned. I keep receiving the following error:
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Whenever I try to run RAILS_ENV=production rails db:migrate or any other rake task, I keep getting that error.
On my AWS console, under Configuration and Software, I have the following environment variables:
Also in my database.yml file I have the rds configured variables listed as such.
production:
adapter: postgresql
database: <%= ENV['RDS_DB_NAME'] %>
username: <%= ENV['RDS_USERNAME'] %>
password: <%= ENV['RDS_PASSWORD'] %>
host: <%= ENV['RDS_HOSTNAME'] %>
port: <%= ENV['RDS_PORT'] %>
I have mapped my values as instructed in the documentation and am certain that they are correct.
Finally, I have sshed into my beanstalk provisioned ec2 instance and have executed the following command:
psql -U username -p 5432 -h examplehost.rds.amazonaws.com -d ebdb
provided the password and am able to connect. I am really at my wits end, I've spent too much time trying to diagnose this and am running out of ideas. I don't know where to look too next for ideas on how to trouble shoot this. I've read so many stack overflow questions and blogs that my head is spinning. If anyone has any ideas on how to resolve this, I would greatly appreciate it.
---Update----
I have created a new environment variable on the elastic beanstalk console.
ENV['DATABASE_URL'] = postgres://YourUserName:YourPassword#YourHostname:5432/YourDatabaseName
I made the necessary configurations, uploaded my .zip file and the connection to the database failed.
---- UPDATE-----
printenv does not show the varialbes provided by beanstalk, however this command does sudo /opt/elasticbeanstalk/bin/get-config environment.
My first advice is that, in my opinion, it is a much better option to create an Amazon RDS on their own, and not tied to Beanstalk.
As the AWS documentation indicates (emphasis mine):
AWS Elastic Beanstalk provides support for running Amazon Relational Database Service (Amazon RDS) instances in your Elastic Beanstalk environment. To learn about that, see Adding a database to your Elastic Beanstalk environment. This works great for development and testing environments. However, it isn't ideal for a production environment because it ties the lifecycle of the database instance to the lifecycle of your application's environment.
And:
To decouple your database instance from your environment, you can run a database instance in Amazon RDS and configure your application to connect to it on launch. This enables you to connect multiple environments to a database, terminate an environment without affecting the database, and perform seamless updates with blue-green deployments.
In my opinion, even for testing or development, it is always advisable to configure a small database instance and give your application the ability of define the most appropriate mechanism for connecting to your database.
The only downside is that you will probably need to configure a VPC, although it should not be actually a problem and, in ay case, it is worth value.
If for any reason you need to use the Beanstalk provisioned RDS database perhaps you have some workarounds to your problem (it should be a workaround because your configuration looks fine - please, only, verify that the database configuration is defined for the right Beanstalk environment).
For instance, one thing you can try is to store the database connection configuration in a S3 bucket, as also suggested in the AWS documentation. The idea is basically create some configuration file with the necessary connectivity information, store it in S3, and read that configuration in your application, i.e., process that file, in order to initialize your database.
But maybe you can try another approach.
Please, consider this SO question, and the answer from Jon McAuliffe and others. As indicated, Beanstalk will provide your application with environment variables, but maybe this variables will not be exposed as shell variables, they will be exposed to your application in different ways depending on the runtime the application needs to be executed on.
In the case of Ruby, you are accessing these variables in the correct way but, for any reason, your program is not having access to that information.
This probably also explains why printenv does not print any if your variables but the get-config script does.
But maybe you can take advantage of the fact that get-config provides you the right information and, either define this variables in your ENV by executing the get-config script for every RDS* key, perhaps in your environment.rb - please, be aware that I programmed in Ruby when I was a student but there is a long time since that, do the task in the file you consider appropriate - or using .ebextensions and a custom configuration file. You can find several examples here.
For instance, consider the following (copy and paste, with minor modifications of this example configuration):
commands:
01_update_env:
command: "/tmp/update_environment_variables.sh"
files:
"/tmp/update_environment_variables.sh":
mode: "000755"
content : |
#!/bin/bash
RDS_HOSTNAME=$(/opt/elasticbeanstalk/bin/get-config environment -k RDS_HOSTNAME)
if [ -z "$RDS_HOSTNAME" ]; then
echo "Could not determine RDS hostname"
exit 1
fi
echo "RDS hostname $RDS_HOSTNAME..."
# Just export the variable at OS level, or make it visible to
# the rails env in some other way
export RDS_HOSTNAME=$RDS_HOSTNAME
# Process the rest of the variables...
# Probably we should create a list and iterate through it
A similar approach could be the one exposed in this stackoverrun question, but restricted to the container that Beanstalk will use to encapsulate your app. AFAIK, the container should receive as env variables the different RDS* ones corresponding to the database configuration.
Dan, be aware that I have not tested these solutions, they are only ideas: please, be careful with that, I do not want to cause any damage to your system.
I found an answer for this problem with a mysql server that might still help you. Basically, even though I followed all your steps, could see my envars using sudo /opt/elasticbeanstalk/bin/get-config environment and could connect directly to my database with the mysql command, I was still getting the following error:
Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) (Mysql2::Error::ConnectionError)
The solution turned out to be the fact that Elastic Beanstalk was not connecting my envars to my bundle exec rails console command in the eb ssh instance access. I solved the issue by prepending all of the required envars explicitly to any rails commands I ran from within the eb ssh instance access. So for example, in order to run rails console, I had to run the following:
RAILS_MASTER_KEY=xxxxxxx RAILS_ENV=production RDS_HOSTNAME=xxxxxxx RDS_PASSWORD=xxxxxxx RDS_USERNAME=xxxxxxx RDS_DB_NAME=xxxxxxx AWS_REGION=xxxxxxx AWS_BUCKET=xxxxxxx bundle exec rails c
Replace the xxxxxxxs above with the values from the corresponding variables in your EB > Configuration > Software tab, and you should be able to connect to the remote database and run migrations, rake tasks and other database-reliant functions.
For Linux2 instances I was having the same issue and just noticed that the env variables I set in the config just didn't exist for su that I had set myself to -- if I remain the default login after eb ssh env prints everything I expected
edit: sorry -- env printing of variables on linux 2 instance enabled by
https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-env-variables-shell/
so what I did was find where those env variables were being exported for default user shell, which was /etc/profile.d/sh.local as noted in the above aws knowledge center link and just source that file when I needed to start the rails console as su
I'd like to introduce per-machine configuration to my rails application. It's currently deployed with Capistrano, and I need to introduce at least one machine-specific variable -- a hostname, so that performance characteristics can be properly tracked. In the future, there are likely going to be further local differences. The code is deployed to a 3 node load balanced cluster from the same git repo.
What is the best way to manage machine specific config?
You can set environment variables in Capistrano config file, and fetch it in your rails app.
config/deploy/.rb
%w[
node1.example.com
node2.example.com
node3.example.com
].each do |host|
server host, roles: %w[app], default_env: {hostname: host}
end
Then you can access the hostname with ENV['HOSTNAME'].
My goal is to have 2 databases and 2 deployments of rails on the same server. I want to have the regular production server using the production database. Then I want to be able to deploy to a different web address that will use a different database. My goal is to be able to push the backup first and make sure all the migrations etc. work in the full environment. I would then push it to the main server.
The issue I seem to run into is that the database.ml file only lists the 3 database types. The passenger environment will also assume that its running in production and would migrate the main MySQL database even if I deploy the code to a different directory. Whats the best way around this? Was wondering if it is simple or if it involves setting lots of variables in lots of places? Any suggestions would be great!
You can add other database types to database.yml as you see fit.
staging:
adapter: postgresql
host: mydb_host
database: mydb_staging
etc...
You can copy config/environments/production.rb to config/environments/staging.rb and leave it as is so the two environments are exactly the same, or tweak staging.rb as you see fit.
Now you have a staging environment! Use it where appropriate, e.g.:
rake RAILS_ENV=staging db:migrate
I am not a passenger expert, but know that my shop has both staging and production instances of apps running on the same server under passenger, so it can be done. Google can probably instruct you better on configuring that than I can.
I want to run the same rails app several times with a few configuration differences on the same server. Each app must have its own:
database
ports
cookie_store key(not sure if needed)
secret_key_base
Let's say I want to run the same code multiple times to service different cities:
newyork.myapp.com and boston.myapp.com. I wonder what would be the best way to store and use the different configurations.
Use environments:
Add a file to config/environments, one for each site you want to host. Name it something along the lines of 'production_[city]', replacing [city] with the city name. Copy the production.rb file contents into each.
in config/database.yml find the 'production' block of yml and duplicate it once for each site you want to host. Rename the root node of each block to production_[city], matching the filenames above. For example:
production_ny:
adapter: mysql2
username: my_user
password: my_pa$$w0rd%&*#
database: production_ny
This takes care of the database settings per app.
Assuming Rails 4, your secret key base will be in config/secrets.yml under an environment node, as per config/database.yml so just add an entry per site:
production_ny:
secret_key_base: xxxxxxxx1111111122222223333333344444444...
All sites will need an end-point. Using a different domain for each would give you separate cookies and sessions for free. Or you could go the subdomain route:
http://guides.rubyonrails.org/configuring.html#deploy-to-a-subdirectory-relative-url-root
tldr:
In your config/environments/production_ny.rb:
config.relative_url_root = "/ny"
for example. Then set up your webserver to handle subdirectories. You may need to add a path to cookies in order to scope them to the virtual directory. Just use:
Rails.configuration.relative_url_root
Ports, again, will have to be set up at the webserver level. (Apache, nginx, etc.)
To see if it all works, try this in the command-line:
RAILS_ENV=production_ny bundle exec rails s
This should start a development style webserver for you to access, but use the production_ny environment.
You will need to create and set up your database as normal - create, migrate, seed.
The final step is setting the RAILS_ENV environment variable to production_[city] per app using your web server. The steps to do this will depend on your technology choice.
I have two Rails applications (lets call them APP-1 and APP-2), each of them has a dependancy on a third Rails application (APP-3).
I would like to be able to run the tests for APP-1 and APP-2 in parallel on my CI server. The problem is, both need to start up APP-3 and write to a DB via the APP-3. This causes conflicts and failures if the tests are run in parallel.
My idea for a solution is for APP-1 and APP-2 to each start their own instance of APP-3 and to have each instance point to a different DB. Is there a way to dynamically set the DB in the database.yml of APP-3 so that it connects to a different DB depending on which APP starts it up?
FYI. APP-1 and APP-2 currently start APP-3 via rake tasks.
Make two database.yml files for App-3. Call them database_A.yml and database_B.yml or whatever you want.
In your environment.rb file for App-3, add the following toward the end of the Initializer block.
Rails::Initializer.run do |config|
# ...
# Decide what database_FOO.yml file you care about.
#
db_config_file_name = "database_A.yml" # Put your logic here for choosing which yml file you want.
db_config_file_path = File.join("config", db_config_file_name)
config.database_configuration_file = db_config_file_path if File.exists? db_config_file_path
end
I pulled this from my blog, where I describe how I use a different database.yml file when I launch my app with JRuby. The concept is similar, so hopefully you find this helpful.
http://www.workingasintended.com/2010/05/03/choosing-a-different-rails-databaseyml-file-for-jruby/