I know many others have encountered this problem, but I'm limited in the Postgres configuration I can do since I am using Amazon RDS...
Here is the relevant section of my database.yml
default: &default
host: <%= ENV["POSTGRES_HOST"] %>
username: <%= ENV["POSTGRES_USER"] %>
password: <%= ENV["POSTGRES_PASSWORD"] %>
adapter: postgis
encoding: unicode
database: <%= ENV["POSTGRES_DB"] %>
port: 5432
pool: 5
When I hard code the host, leave everything else as is, and load localhost:3000, I get fe_sendauth: no password supplied error. However, when I hard code all of the values instead of using an ENV variable, everything loads as normal with no errors.
I'm using Amazon RDS, so I have no ability to edit pg_hba.conf, but is this just a simpler problem of RoR not having access to ENV variables?
Yup, it's pretty clear that your ENV values are returning nil. Keep in mind that environment variables are loaded a bit differently on your production server than localhost. In most deployment configurations you can't simply set a Unix environment variable and have it be detectable in ENV.
I can't really tell much about how your ENV variables are being set from your provided code, but you should consider using Figaro (https://github.com/laserlemon/figaro) or dotenv (https://github.com/bkeepers/dotenv) to manage your ENV variables for you. However, it will require the extra step of having to manage an environment variable file outside of source control.
Related
I want to get secret informations from other place beside rails application and the host. I made a global variable file under config/initializers to get the keys. Then set them in the database connection file. Like this:
config/initializers/get_secrets.rb
##username = 'A'
##password = 'B'
config/database.yml
development:
adapter: mysql2
encoding: utf8
database: my_db_name
username: <%= ##username %>
password: <%= ##password %>
host: 127.0.0.1
port: 3306
When ran rails console, got:
./config/initializers/get_secrets.rb:1: warning: class variable access from toplevel
(erb):10: warning: class variable access from toplevel
>
It seems not a good usage in this case. Then is there a right way?
In Rails 5.2+, we have a master.key file and a credentials YAML file. The master.key file encrypts your credentials (so never show anyone this key except your server).
You can activate this by doing:
EDITOR=vim rails credentials:edit
The file:
aws:
key: 123
google_key: 456
secret_key_base: f230ffddd1d1f151j3bchjas9a292j221...
If you want to use a credential, it looks like this:
Rails.application.credentials.google_key
or for nested resources like the AWS one:
Rails.application.credentials.aws[:key]
But if you're on Rails 5.1 or less, you don't have this ability. So, you do it in a different way:
Environment variables. Your program environment can always export an ENV and retrieve it. I always do this for simple nuisances like test email passwords:
export GMAIL_PASSWORD="1234" and its usage, ENV['GMAIL_PASSWORD']
Secrets.yml. This is a file that basically predates master.key + credentials by functioning as an in between credential handler for your environments. It's janky and often erroneously committed to repos by accident, but we do what we can:
development:
secret_key_base: 1112sdasae1d13d1tfd1d3...
GMAIL_PASSWORD: '1234'
test:
secret_key_base: _your_secret_ as above
production:
secret_key_base: <%= secure_token %>
And it is accessed like Rails.application.secrets.GMAIL_PASSWORD
If you cannot access your variables, it may be because the secrets.yml file is not being loading in or you are in the wrong environment. Notice those vars are nested in particular environments.
I am using postgresql as the database in my development environment, so that means that I setup my database.yml like the code below where the username and password is my postgresql's username and password (obviously, I'm using environment variables to avoid hard-coding) and will be inherited by development and test environments:
default: &default
adapter: postgresql
encoding: unicode
# For details on connection pooling, see rails configuration guide
# http://guides.rubyonrails.org/configuring.html#database-pooling
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
host: localhost
username: <%= ENV['FOO_DATABASE_USERNAME'] %>
password: <%= ENV['FOO_DATABASE_PASSWORD'] %>
I'm just curious about the production environment, which has the code below:
production:
<<: *default
database: foo_production
username: foo
password: <%= ENV['FOO_DATABASE_PASSWORD'] %>
Unlike in the development and test environments which uses my postgresql's username and password, the production environment have different values for username and password.
My question is, do I need to change the username and password in the production environment? or should I just leave it as it is?
What you're talking about here is essentially how you can make your code work in multiple environments (development and production), with multiple different database credentials.
The way this is typically handled is through environment variables.
In your example above, it looks like you've created two environment variables for your local database:
PINTERESTING_APP_DATABASE_USERNAME
PINTERESTING_APP_DATABASE_PASSWORD
In production, I'm assuming you have some different environment variables set which contain your production Postgres credentials.
What you need to do in your code is select one of these two options, and use them everywhere.
Here's some pseudo-code I use in my applications to accomplish this:
DATABASE_HOST = ENV['LOCAL_DB_HOST'] || ENV['PRODUCTION_DB_HOST']
DATABASE_PORT = ENV['LOCAL_DB_PORT'] || ENV['PRODUCTION_DB_PORT']
DATABASE_USERNAME = ENV['LOCAL_DB_USERNAME'] || ENV['PRODUCTION_DB_USERNAME']
DATABASE_PASSWORD = ENV['LOCAL_DB_PASSWORD'] || ENV['PRODUCTION_DB_PASSWORD']
As you can see above, what I'm doing is either using my development credentials, OR my production credentials, depending on which environment variables are available.
This allows me to seamlessly switch between development and production environments by just modifying environment variables (which is what Heroku does).
On my local machine, I want to set environment variables for all of the sensitive information in the database.yml file. My file looks like this:
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: <%= ENV['DATABASE_USERNAME'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
socket: <%= ENV['SOCKET'] %>
development:
<<: *default
database: <%= ENV['DEVELOPMENT_DATABASE'] %>
test:
<<: *default
database: #JetStreamIQ-2_0_test
production:
<<: *default
database: <%= ENV['PRODUCTION_DATABASE'] %>
username: <%= ENV['DATABASE_USERNAME'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
I thought that I could just set these environment variables in my .bashrc file, but that doesn't seem to be working. My .bashrc file looks like this:
export DATABASE_USERNAME="root"
export DATABASE_PASSWORD="*****"
export SOCKET="/var/run/mysqld/mysqld.sock"
export DEVELOPMENT_DATABASE="shoppe_development"
export PRODUCTION_DATABASE="#"
When I run my server with
rails s
I get an error that says:
Access denied for user 'root'#'localhost' (using password: YES)
I understand that there is a problem with the database username and password because of the way I've configured my database.yml file, but I'm just not sure what it is.
Is there something big I'm missing here? Any help would be much appreciated.
Thanks!
I'd like to offer a couple of tips here.
You can run database.yml through erb to test what you're getting. For example, erb config/database.yml. This will help you determine what the problem may be.
You can find out every environment variable set in your shell with the set command. This will allow you to confirm that the environment variables you're expecting are being set (in .bashrc).
Hope this is helpful.
It turns out that after I edited my bashrc file, I needed to exit out of that terminal session and open another terminal for the changes to be finalized :/ After I did that my application started perfectly.
If you dont want to restart your terminal, you just need to execute your .bashrc file:
source /path/to/.bashrc
in the same terminal as your rails server.
Is it a typo or are you really setting JETSTREAMIQ_DATABASE_PASSWORD but then using DATABASE_PASSWORD in your database.yml? Because that would do it.
Can you connect to mysql using the actual values directly using the mysql command line app?
you may need to put your environment variables at the top of .bashrc file. cause it has some return executed into the file. so it return before actually executing your export command
something like this:
nano ~/.bashrc
export DATABASE_VAR="my_own_var"
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
Simply set environment variable
DATABASE_URL=postgres://user:pass#localhost/dbname
It will override database.yml settings
Offcourse it supports more parameters, like
DATABASE_URL=mysql2://user:passwd#host:port/dbname?encoding=utf8mb4&collation=utf8mb4_unicode_ci
You can also source your ~/.bashrc file using the following command on your terminal:
exec $SHELL
Have you tried to access your local database with that data from the console?
Maybe you have to grant access for localhost
GRANT ALL ON shoppe_development.* TO 'root'#'localhost';
Besides that, I recommend you the gem dotenv to handle your env vars for development environment
I have the following config in my production config/database.yml file. I've noticed that this doesn't work - it fails with a postgres password authentication failure.
I also noticed that if I replace <%= ENV['SOME_PASSWORD'] %> with the actual password, that it works fine.
production:
adapter: postgresql
host: localhost
database: some_database
username: some_user
password: <%= ENV['SOME_PASSWORD'] %>
However, I only see this behavior on my production Ubuntu server. This insertion of the environment variable works find on my OS X dev box.
Just curious why this is - why doesn't this work on Ubuntu? If I go into an interactive ruby session on the production box via irb, and type <%= ENV['SOME_PASSWORD'] %> it outputs the password as expected.
I've tested this multiple ways, and I'm sure this is the problem. I'm just not sure why it happens. I'm using Ruby Enterprise Edition for my ruby interpreter in both places. Just seems weird, and I'd really like to be able to keep my password completely out of my config file - I just feel like I need to use a slightly different syntax for the production server, but I'm not sure what that syntax might be.
NOTE: Every single other article I've found anywhere on the internet that deals with Rails postgres authentication issues, at some point, tells you to change your pg_hba.conf file to trust mode, which not only ignores the issue by bypassing authentication, it's insecure, and not a valid solution if you care about security.
Experimenting with MongoID on a Rails server and confused about how/where to set the environment variables.
config/mongoid.yml default template provides:
defaults: &defaults
host: localhost
...
# set these environment variables on your prod server
production:
host: <%= ENV['MONGOID_HOST'] %>
port: <%= ENV['MONGOID_PORT'] %>
username: <%= ENV['MONGOID_USERNAME'] %>
password: <%= ENV['MONGOID_PASSWORD'] %>
database: <%= ENV['MONGOID_DATABASE'] %>
My question is are these set in Rails somewhere? or are they at the system level? and if so where/how to set so that no user account needs to be logged in for them to be valid?
The ENV hash will have values from the system environment from when the rails process was started.
These can be set on the command line prior to starting the server or program. For example in bash:
export MONGOID_USERNAME='username'
These are only good for the life of your shell, unless you add them to your profile, but it is likely that your web server won't use that profile, so it is only useful for local development.
They can also be set, for example, in Apache with SetEnv. For example:
<Location /app >
SetEnv MONGOID_HOST 'localhost'
SetEnv MONGOID_PORT '8883'
SetEnv MONGOID_USERNAME 'username'
</Location>
This could be anywhere SetEnv is legal in your apache config, and that is the same context that your application lives under.
Regarding you comment about best practices, some people put an example yml config file in source control, and ignore the config/*.yml files from source control. When cloning a repository, copying and correcting the examples to the correct values is part of the setup, like running rake tmp:create to make the tmp folder structure.
I wanted to add another option here. On boot, Rails will attempt to read DATABASE_URL as a url and connect to a database from that env variable (ignoring database.yml). You should specify the database as:
DATABASE_URL="mysql2://user:pass#localhost/app_development" rails server
and you can verify this via:
DATABASE_URL="..." rails runner "p ActiveRecord::Base.connection_config"
This is just another option instead of putting erb settings into database.yml.