I have a Rails 5 app and I am trying to import my Environment Variables in a yaml file, which I can then import in the environment.rb. My config/aws.yml looks as follows:
S3_BUCKET: <%= ENV["S3_BUCKET"] %>
AWS_ACCESS_KEY_ID: <%= ENV["AWS_ACCESS_KEY_ID"] %>
AWS_SECRET_ACCESS_KEY: <%= ENV["AWS_SECRET_ACCESS_KEY"] %>
Then in my environment.rb I have the following:
require_relative 'application'
aws_file = File.join(Rails.root, 'config', 'aws.yml')
if File.exist?(aws_file)
YAML.safe_load(File.open(aws_file)).each do |k, v|
ENV[k.to_s] = v
end
end
on running my rails c this is what I get:
[1] pry(main)> ENV.fetch('S3_BUCKET')
=> "<%= ENV[\"S3_BUCKET\"] %>"
How can I pass the actual environment variable rather than it pulling the name of the environment variable as a string?
EDIT: Adding more relevant information
My credentials are stored in my elastic-beanstalk configuration and I am trying to load the values from the configuration to my yaml file.
<%= ... %> is a feature of ERB, not YAML. You'd need to first run your YAML file through ERB to cause the <%= ... %> to execute and then load it as YAML.
But there's a deeper problem. Your YAML file is trying to get its values from environment variables. But then you set those same environment variables from the YAML values. It's circular.
Instead use something like Encrypted Credentials introduced in Rails 5.2 to manage your secrets. Then write config/initializer/aws.rb to set the necessary environment variables from those secrets. This is nice because it stores your secrets in your app rather than in the deployment platform. Then it can be run with full secrets anywhere.
Or you can set the environment variables in Elastic Beanstalk via the "Environment Properties" console.
In addition to what you have already, add this to config/application.rb:
config.x.aws = config_for(:aws)
And you'll be able to access it:
Rails.configuration.x.aws.S3_BUCKET # => ...
Have you tried fetch, it worked on my case:
S3_BUCKET: <%= ENV.fetch("S3_BUCKET") %>
AWS_ACCESS_KEY_ID: <%= ENV.fetch("AWS_ACCESS_KEY_ID") %>
AWS_SECRET_ACCESS_KEY: <%= ENV.fetch("AWS_SECRET_ACCESS_KEY") %>
Related
What's the correct way of defining secret_key_base on Rails 6 now that we have per-environment credentials?
My environment has the variable SECRET_KEY_BASE but Rails is not picking it up. I tried defining secret_key_base in config\credentials\production.yml.enc but it has no effect on Rails.application.credentials.secret_key_base
I know config/secrets.yml with
staging:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
works, but, is that the Rails 6 way?
The right way to access and check for secret_key_base in Rails 6 is no longer:~
Rails.application.credentials.secret_key_base
it now is:
Rails.application.secret_key_base
I'm not sure if this is Rails 6 or it's been like this forever. This becomes pretty clear when looking at this method, and its implementation:
https://github.com/rails/rails/blob/09a2979f75c51afb797dd60261a8930f84144af8/railties/lib/rails/application.rb#L410-L427
# The secret_key_base is used as the input secret to the application's key generator, which in turn
# is used to create all MessageVerifiers/MessageEncryptors, including the ones that sign and encrypt cookies.
#
# In development and test, this is randomly generated and stored in a
# temporary file in <tt>tmp/development_secret.txt</tt>.
#
# In all other environments, we look for it first in ENV["SECRET_KEY_BASE"],
# then credentials.secret_key_base, and finally secrets.secret_key_base. For most applications,
# the correct place to store it is in the encrypted credentials file.
def secret_key_base
if Rails.env.development? || Rails.env.test?
secrets.secret_key_base ||= generate_development_secret
else
validate_secret_key_base(
ENV["SECRET_KEY_BASE"] || credentials.secret_key_base || secrets.secret_key_base
)
end
end
Both development and test mode have their own way of generating and storing the secret key base. For everything else, it pics it up from the environment, or credentials or secrets, in that order.
Docker users in development might consider this in their entrypoint.sh:
if [ "$RAILS_ENV" = "development" ]; then
printf $SECRET_KEY_BASE > ./tmp/development_secret.txt
fi
I've tried to solve that problem few days ago.
And what I learned:
First attempt
I try to use credentials per environment with
$ EDITOR=nano rails credentials:edit --environment development
$ EDITOR=nano rails credentials:edit --environment staging
$ EDITOR=nano rails credentials:edit --environment production
My creds files and keys were placed in config/credentials.
I set necessary variables straight there. It's usable solution, but we met a problem with our deployment at Kubernetes cluster, when our devopses wants to use helm configs. So, predefined credentials is not applicable for that case.
Second attempt
After that I've tried to use ENV-variables in my credentials files.
Unfortunately, it's not works too:
secret_key_base: <%= ENV['SECRET_KEY_BASE'] %>
Final attempt
Finally, I did graceful degradation to gem config with default configuration, when you per-environment settings placed there:
config/settings.yml
config/settings/development.yml
config/settings/production.yml
config/settings/test.yml
And my settings.yml file consists only ENV-variables, like so:
secret_key_base: <%= ENV['SECRET_KEY_BASE'] %>
db:
host: <%= ENV['DB_HOST'] %>
port: <%= ENV['DB_PORT'] %>
pool: <%= ENV['DB_POOL'] %>
user: <%= ENV['DB_USER'] %>
password: <%= ENV['DB_PASSWORD'] %>
database: <%= ENV['DB_DATABASE'] %>
...
It's workable solution, but seems like step-backward.
As I know now, we cant use ENV-vars in credentials any simple way.
I'm setting up a new rails 5.2 app utilising Active Storage and using AWS for the hosting of images in production.
However, I'm having an issue with the app reading the credentials:
2018-07-06T08:11:52.625415+00:00 app[web.1]: ! Unable to load application: Aws::Sigv4::Errors::MissingCredentialsError: Cannot load `Rails.config.active_storage.service`:
2018-07-06T08:11:52.625432+00:00 app[web.1]: missing credentials, provide credentials with one of the following options:
2018-07-06T08:11:52.625435+00:00 app[web.1]: - :access_key_id and :secret_access_key
2018-07-06T08:11:52.625437+00:00 app[web.1]: - :credentials
2018-07-06T08:11:52.625479+00:00 app[web.1]: - :credentials_provider
This is an existing S3 Bucket which I created a new user just for this app. I'm happy with the CORS etc.
The user is set up under the S3FullAccess group.
I've edited the credentials in my app via $EDITOR="atom --wait" rails credentials:edit
The contents of the file:
aws:
access_key_id: [my access key]
secret_access_key: [my secrect key]
# Used as the base secret for all MessageVerifiers in Rails, including the one protecting cookies.
secret_key_base: [my secret key base]
Appreciate this is in YAML format, I have played with using one space, and one tab on the keys, but this doesn't seem to make a difference.
When I save and close the file, the terminal writes New credentials encrypted and saved.
I also have gem 'aws-sdk-s3', '~>1', require: false installed.
And config/storage.yml
test:
service: Disk
root: <%= Rails.root.join("tmp/storage") %>
local:
service: Disk
root: <%= Rails.root.join("storage") %>
# Use rails credentials:edit to set the AWS secrets (as aws:access_key_id|secret_access_key)
amazon:
service: S3
access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
region: eu-west-2
bucket: [mybucket]
Any suggestions on what I might be doing wrong?
I think you're missing the master.key file in your server. Check your local repo in config/master.key (this file is added to your .gitignore by default).
Add this file to your server or set ENV["RAILS_MASTER_KEY"].
This worked for me on Heroku: in "Settings > Config vars" add a RAILS_MASTER_KEY key, with the content of your your config/master.key file (from your Rails app) as the value.
Go into config/environments/development.rb and make sure you have this:
config.active_storage.service = :local
in config/environments/production you should have
config.active_storage.service = :amazon
amazon is for Amazon S3. It can be changed to whichever storage service you want to use. See the Rails docs for more info on storage services and Active Storage.
In Rails 5.2, do the following:
Step 1. In config/storage.yml add
amazon:
service: S3
access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
region: ap-south-1
bucket: my-bucket
Step 2:
Copy config/credentials.yml.example to config/credentials.yml
and add the following in config/credentials.yml
development:
AWS_ACCESS_KEY_ID: YOUR-KEY
AWS_SECRET_ACCESS_KEY: YOUR-SECRET
credentials.yml is already added to .gitignore by default.
Step 3:
In application.rb
Uncomment the following:
# Load ENV variables from credentials.yml file
config.before_configuration do
env_file = File.join(Rails.root, 'config', 'credentials.yml')
YAML.load(File.open(env_file))[Rails.env].each do |key, value|
ENV[key.to_s] = value
end if File.exists?(env_file)
end
Restart the server and try to upload again.
Another way of solving this issue (worked for me)
Run rake secret in the console
copy the key
go to config and open application.rb
inside the class type: config.secret_key_base = "paste the output of rake secrete"
I had the same error. In my case the problem was neither with configs, nor with master.key. Starting Redis server fixed the error. For MacOS:
$> redis-server
I created a new Rails app called sample_app and I use postgresql as my db (I already created a postgresql username and password). I use this setup guide https://gorails.com/setup/ubuntu/16.04
So I run this command rails new sample_app -d postgresql. And then I have to edit the config/database.yml to match the username and password to my postgresql's username and password I just created. But I don't want to hard-code because I will be using git.
I found this tutorial from digital ocean which suggest to use:
username: <%= ENV['APPNAME_DATABASE_USER'] %>
password: <%= ENV['APPNAME_DATABASE_PASSWORD'] %>
Is this the correct code? If so, since my app is called sample_app, my code should be?
username: <%= ENV['SAMPLE_APP_DATABASE_USER'] %>
password: <%= ENV['SAMPLE_APP_DATABASE_PASSWORD'] %>
If this is not the correct one, can you help me? Thank you!
There are many ways you can set the environment variables.
Here are two of them,
Option One: Setting ENV variables via a yml file
Create a file config/local_env.yml:
config/local_env.yml:
SAMPLE_APP_DATABASE_USER: 'your username'
SAMPLE_APP_DATABASE_PASSWORD: '******'
The above are the names you will use like,ENV['SAMPLE_APP_DATABASE_USER']. these can be names as your wish. you can take any name, but we should use the same name in the ENV reference.
add it to gitignore:
/config/local_env.yml
Change some code in application.rb
application.rb:
config.before_configuration do
env_file = File.join(Rails.root, 'config', 'local_env.yml')
YAML.load(File.open(env_file)).each do |key, value|
ENV[key.to_s] = value
end if File.exists?(env_file)
end
The code opens the config/local_env.yml file, reads each key/value pair, and sets environment variables.
Using Environment Variables:
username: <%= ENV['SAMPLE_APP_DATABASE_USER'] %>
password: <%= ENV['SAMPLE_APP_DATABASE_PASSWORD'] %>
Option Two: Use the Figaro Gem
The gem takes advantage of Ruby’s ability to set environment variables as well as read them. The gem reads a config/application.yml file and sets environment variables before anything else is configured in the Rails application.
Here’s how to use it. In your Gemfile, add:
gem 'figaro'
and run bundle install
The gem provides a generator:
$ bundle exec figaro install
The generator creates a config/application.yml file and modifies the .gitignore file to prevent the file from being checked into a git repository.
You can add environment variables as key/value pairs to config/application.yml:
SAMPLE_APP_DATABASE_USER: 'your username'
SAMPLE_APP_DATABASE_PASSWORD: '******'
The environment variables will be available anywhere in your application as ENV variables:
ENV["SAMPLE_APP_DATABASE_USER"]
Here are the remaining ways you can achieve the same.
You can call it anything you want...
username: <%= ENV['CARROTS'] %>
password: <%= ENV['BEANS'] %>
You just have to make sure your deploy script sets the variables CARROTS and BEANS correctly.
try this gem dotenv-rails
add this to Gemfile:
gem 'dotenv-rails', :groups => [:development, :test]
bundle it. Now create a .env file on your apps's directory with following content:
SAMPLE_APP_DATABASE_USER: "devuser"
SAMPLE_APP_DATABASE_PASSWORD: "devuser"
restart the server you're good to go. these variables are exported when you boot your app which you can access in your database.yml file
username: <%= ENV['SAMPLE_APP_DATABASE_USER'] %>
password: <%= ENV['SAMPLE_APP_DATABASE_PASSWORD'] %>
read dotenv-rails documentation for more info
Can someone help me understand how to retrieve an API key if I'm storing it into secrets.yml?
If I have some kind of google API key 'yt_key':
secrets.yml
development:
secret_key_base: 390257802398523094820 #some key
yt_key: A423092389042430 #some key
test:
secret_key_base: 43208947502938530298525#some key
yt_key: A423092389042430 #some key
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
yt_key: <%= ENV["YT_KEY"] %>
I'm just following the examples, this is how I would set it up right?
So if I publish this to production, I would save the A423092389042430 in heroku and under YT_KEY, correct?
But in development, would I do it this way to retrieve the data:
in /config/application.rb
Yt.configure do |config|
config.api_key = '<%= ENV["YT_KEY"] %>'
end
or should this be in the the class:
module Sample
class Application < Rails::Application
Yt.configure do |config|
config.api_key = '<%= ENV["YT_KEY"] %>'
end
config.active_record.raise_in_transactional_callbacks = true
end
end
Or did I set up the configure wrong?
ENV["YT_KEY"] references the 'YT_KEY' environment variable which you'll have to set with a Heroku config variable.
In your app, you can access your secrets like this:
Rails.application.secrets.key_name
Since you're storing the 'YT_KEY' as an environment variable in production only, you should configure Yt like so:
(You can do this in a initializer file located at app/initializers/yt.rb)
Yt.configure do |config|
config.api_key = Rails.application.secrets.yt_key
end
That way, the correct key will be set in each environment.
It's good practice to use different keys for each environment, so should get another key for your production environment. Also, you should avoid storing secret production environment keys in the code. That's why it's common to use ENV variables for production keys.
Let me know if you need any clarification!
Do it this way, we are doing this way since a long time and working very well for us and this is a good convention as well.
secrets.yml
development:
secret_key_base: 390257802398523094820 #some key
yt_key: A423092389042430 #some key
test:
secret_key_base: 43208947502938530298525#some key
yt_key: A423092389042430 #some key
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
yt_key: <%= ENV["YT_KEY"] %>
Add these line to your application.rb file
config_files = ['secrets.yml']
config_files.each do |file_name|
file_path = File.join(Rails.root, 'config', file_name)
config_keys = HashWithIndifferentAccess.new(YAML::load(IO.read(file_path)))[Rails.env]
config_keys.each do |k,v|
ENV[k.upcase] ||= v
end
end
and now you can access yt_key this way ENV["YT_KEY"] or any other key you add like some_key to ENV["SOME_KEY"].
It's often recommended to not put your custom keys in secret.yml instead make another file like app_keys.yml and put all keys there.
You can also use Figaro gem.
Once installed, you'll have a config/application.yml file. Inside it you can store your api keys etc.:
SENDGRID_USERNAME: a-name
SENDGRID_PASSWORD: password
Now, anywhere in your .rb files, you can reference it using vars:
# Noticed how I keep my vars uppercase throughout.
ENV["SENDGRID_USERNAME"]
ENV["SENDGRID_PASSWORD"]
# Production vars go below the `production` line
production:
ENV["MY_PRODUCTION_VAR"]
If using those env keys inside your html.erb then you'll need to wrap it with <%= ... %>
I'm currently trying to configure Paperclip with newest aws-sdk suggested gem.
On my S3.yml file I have something like this
development:
bucket: newmeeter-dev
access_key_id: ENV['S3_KEY']
secret_access_key: ENV['S3_SECRET']
But it is not recognizing the ENV variables. I'm getting the following error
AWS::S3::Errors::InvalidAccessKeyId in PhotosController#create
The AWS Access Key Id you provided does not exist in our records.
If I try to put both the access and secret directly into the file it works perfectly. At the same time I tried to print both ENV variables into the views or in the console I can see their values okay.
I'm not getting why it is not recognizing it.
Solved!
I found the reply to this question here
Ruby on Rails: Can you put Ruby code in a YAML config file?
Solution: YAML files understand code in ERB format.
Printing ENV variables inside <%= and %> works.
access_key_id: <%= ENV['S3_KEY'] %>
secret_access_key: <%= ENV['S3_SECRET'] %>