When installing Figaro gem, an application.yml is automatically created. And inside this file I planned on storing login credentials for SendGrid.
But by default in the rails application, there is another secrets.yml file, with the secret_key_base.
I'm little confused on their relationship.
My question: Is it necessary to have both? Can I combine them? Should both be added to gitignore ?
You can leave it there as it is, just put the new variables to the application.yml and make sure you gitignore both files since you need those only for localhost. For production you have to put the keys to a different place based on the service. I'm using heroku and have to save production variables from terminal.
Related
I'm working on a Rails app with a few collaborators and we decided to begin using separate database.yml files for some time until we can a configuration that works for all of us.
After adding database.yml to the .gitignore file and pushing a version without it, I realized that this would likely prevent the Heroku app from running.
My confusion is that the deployment was successful and the database.yml file was not needed. Why is this? Is our old database.yml file cached?
This is actually the expected behavior. For more details see: https://devcenter.heroku.com/articles/rails-database-connection-behavior
Which boils down to (for Rails 4.1+):
While the default connection information will be pulled from
DATABASE_URL, any additional configuration options in your
config/database.yml will be merged in.
Heroku will always use DATABASE_URL and merge the rest from database.yml to the config contained in that url.
Ah yes the old db config developer war.
Heroku actually uses the solution to this issue - Rails merges the database configuration from database.yml with a hash created from parsing ENV["DATABASE_URL"]. The ENV var takes precedence over the file based configuration.
When you first push a Rails app, Heroku automatically attaches a Postgres addon and sets ENV["DATABASE_URL"] and presto your app magically connects to the database.
Even if you add complete nonsense settings like setting the database name in database.yml the ENV var still wins.
How can this solve our developer war?
Do the opposite of what you are currently doing. Strip everything except the bare minimum required to run the application out of database.yml and check it back into version control.
Developers can use direnv or one of the many tools available to set ENV[DATABASE_URL] to customize the settings while database.yml should be left untouched unless you actually need to tweak the db.
Is database.yml the right place to read the AWS keys from bashrc? database.yml sounds like a place only for database configs. Is there a more appropriate place where the AWS configs from bashrc could be read inside my Rails app?
Rails 5.2 onwards
Rails 5.2 has introduced the concept of encrypted credentials. Basically, from Rails 5.2 onwards, there is an encrypted credentials file that is generated on initializing the app in config/credentials.yml.enc. This file is encrypted, and hence, can be pushed to your source control tool. There is also a master.key file which is generated while initializing the app, which can be used to decrypt the credentials file, and make changes to it.
So, credentials for AWS could be added to it as:
aws:
access_key_id: 123
secret_access_key: 345
These keys could be accessed in your app as Rails.application.credentials.aws[:secret_access_key]. Other sensitive config, like credentials to other external services that are being used, can also be added to this config. Check out this blog by Marcelo Casiraghi for more details.
Pre Rails 5.2
There was no concept of a credentials system prior to Rails 5.2. There are a couple of ways in which you could try to come up with a solution to store your configuration.
A. You could create a YAML file for defining your config from scratch.
Create a file called my_config.yml and place it in the config folder. Path: config/my_config.yml
Add whatever configuration is required to the file, in YAML format (s described for AWS above)
Make changes in application.rb to load this file during initialization as follows:
APP_CONFIG = YAML.load(ERB.new(File.new(File.expand_path('../my_config.yml', __FILE__)).read).result)[Rails.env] rescue {}
Using this approach, you will then be able to use APP_CONFIG['aws']['access_key_id'] for the AWS configuration defined above. For this use case, it is strongly recommended to have separate configuration files for development and production environments. The production file should probably not be checked in to version control for security.
B. Another approach would be to use some gems for managing configurations like railsconfig/config
NOTE: To answer the bit about storing this configuration in database.yml, it is strongly recommended to not do so. database.yml is a configuration file for storing configuration related to databases. Separation of concerns really helps while scaling any application, and hence, it is recommended to place such configurations in a separate file, which can be independently maintained, without any reliance on the database config.
Absolutely. The standard place to configure things like AWS would be inside config/initializers. You can create a file in there called aws.rb.
app/
bin/
config/
|__ initializers/
|__ aws.rb
and inside this file you can configure your AWS setup using the environment variables from your bashr
Aws.config.update({
credentials: Aws::Credentials.new('your_access_key_id', 'your_secret_access_key')
})
Files inside this directory are executed on app start, so this configuration will be executed right when your app starts, before it starts handling requests.
It may also be useful to note that the AWS SDK for Ruby will automatically search for specific environment variables to configure itself with. If that's what you're using, and if you have the following environment variables set up in your bashrc
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
then you won't need any additional code in your Rails app to configure AWS. Check out more details here.
I updated my Rails app from 5.1.6 to 5.2.1 and we were using secrets before. We'd like to switch to credentials since it was supposed to replace secrets. There are a couple of things I'm wondering:
1) Is there a "Railsy" way to do this? I'm aware I can just edit the secrets and copy the contents over to the credentials but I'm not sure if that's the right way to do it since I couldn't find anything that talks about this. Also, I tried copy the encrypted content from secrets.yml.enc and paste it to credentials.yml.enc but that didn't work, ran into some issues with the encryption.
2) Related to the first point, am I supposed to still be able to use secrets in 5.2.1? I thought credentials was supposed to replace secrets so I was surprised I could still use secrets and all my tests are passing.
Thanks for any info or help on this!
Backup your config/secrets.yml. Scaffold a temporary vanilla Rails 5.2.1 project via rails new. Copy config/master.key and config/credentials.yml.enc from it to your existing Rails project. Edit these credentials e.g in Ubuntu via:
EDITOR="gedit --wait" bin/rails credentials:edit
Replace the secret_key_base (new flat format prefered, optional for all environments) from the old secrets.yml and / or paste whatever else you need as a secret into it and save it. Delete config/secrets.yml and the temporary Rails project. Access the secrets in your code e.g. via:
Rails.application.credentials.secret_key_base
Ensure that your upgraded Rails 5.1.6 project use the master key in config/environments/*.rb:
Rails.application.configure do
...
config.require_master_key = true
...
end
Restart the Rails server. Do'nt forget to .gitignore and .dockerignore the config/master.key!
With the release of the secrets.yml file, I removed my reliance on Figaro and moved all of my keys to secrets.yml and added that file to .gitignore.
But when I tried to push to Heroku, Heroku said they needed that file in my repo in order to deploy the website. which makes sense, but I don't want my keys in git if I can avoid it.
With Figaro, I would run a rake task to deploy the keys to heroku as env variables and keep application.yml in the .gitignore. Obviously, I can't do that any more. So how do I handle this?
Secrets isn't a full solution to the environment variables problem and it's not a direct replacement for something like Figaro. Think of Secrets as an extra interface you're now supposed to use between your app and the broader world of environment variables. That's why you're now supposed to call variables by using Rails.application.secrets.your_variable instead of ENV["your_variable"].
The secrets.yml file itself is that interface and it's not meant to contain actual secrets (it's not well named). You can see this because, even in the examples from the documentation, Secrets imports environment variables for any sensitive values (e.g. the SECRET_KEY_BASE value) and it's automatically checked into source control.
So rather than trying to hack Secrets into some sort of full-flow environment variable management solution, go with the flow:
Pull anything sensitive out of secrets.yml.
Check secrets.yml into source control like they default you to.
For all sensitive values, import them from normal environment variables into secrets ERB (e.g. some_var: <%= ENV["some_var"] %>)
Manage those ENV vars as you normally would, for instance using the Figaro gem.
Send the ENV vars up to Heroku as you normally would, for instance using the Figaro gem's rake task.
The point is, it doesn't matter how you manage your ENV vars -- whether it's manually, using Figaro, a .env file, whatever... secrets.yml is just an interface that translates these ENV vars into your Rails app.
Though it adds an extra step of abstraction and some additional work, there are advantages to using this interface approach.
Whether you believe it's conceptually a good idea or not to use Secrets, it'll save you a LOT of headache to just go with the flow on this one.
PS. If you do choose to hack it, be careful with the heroku_secrets gem. As of this writing, it runs as a before_initialize in the startup sequence so your ENV vars will NOT be available to any config files in your config/environments/ directory (which is where you commonly would put them for things like Amazon S3 keys).
An equivalent for secrets.yml of that Figaro task is provided by the heroku_secrets gem, from https://github.com/alexpeattie/heroku_secrets:
gem 'heroku_secrets', github: 'alexpeattie/heroku_secrets'
This lets you run
rake heroku:secrets RAILS_ENV=production
to make the contents of secrets.yml available to heroku as environment variables.
see this link for heroku settings
if u want to run on local use like this
KEY=xyz OTHER_KEY=123 rails s
I am using the figaro gem and have created an application.yml file with all of my variables as per the documentation. This application.yml file is located in a shared folder (I'm using capistrano) and is symlinked to config/application.yml within the current live app directory, however I can only access the variables in the rails console and not the app. My credentials are listed as follows (real details omitted):
Note: I have tried removing the "" speech marks and also prefixing this list with production: with each line having 2 spaces, not tabbed, and it doesn't solve anything. The permissions on the file are exactly the same, 777, as the databse.yml file which was implemented in the same way.
application.yml
FFMPEG_LOCATION: "/path/to/ffmpeg"
EMAIL_USERNAME: "me#gmail.com"
EMAIL_PASSWORD: "password"
S3_BUCKET: "my_bucket"
AWS_SECRET_KEY_ID: "my_secret_key"
AWS_ACCESS_KEY_ID: "my_access_key"
I can access these variables in the production console =>
Loading production environment (Rails 3.2.14)
irb(main):001:0> ENV["S3_BUCKET"]
=> "my-s3-bucket-name"
However they don't return anything in the app itself. I set my linux box up following Ryan's excellent Pro railscast episode http://railscasts.com/episodes/335-deploying-to-a-vps
How can I get these variables accessible in the app itself?
If anyone needs more code just shout.
EDIT
I removed the figaro gem implemented the yaml config shown in the following railscasts tutorial: http://railscasts.com/episodes/85-yaml-configuration-revised. I think this is effectively what the figaro gem was doing however instead of using ENV variables, the tutorial uses CONFIG[:variables] which seem to work great.
Per Comment:
Nginx runs as its own user, so the environment variables need to live in it's space. As a user when you log in and run console, you're accessing a different set of environment variables than the nginx user accesses.
You can do this if you choose by adding them to the nginx config in the main context. But it's probably easier to go with straight yaml and add your secret tokens to your yaml file.