I am using the figaro gem and have created an application.yml file with all of my variables as per the documentation. This application.yml file is located in a shared folder (I'm using capistrano) and is symlinked to config/application.yml within the current live app directory, however I can only access the variables in the rails console and not the app. My credentials are listed as follows (real details omitted):
Note: I have tried removing the "" speech marks and also prefixing this list with production: with each line having 2 spaces, not tabbed, and it doesn't solve anything. The permissions on the file are exactly the same, 777, as the databse.yml file which was implemented in the same way.
application.yml
FFMPEG_LOCATION: "/path/to/ffmpeg"
EMAIL_USERNAME: "me#gmail.com"
EMAIL_PASSWORD: "password"
S3_BUCKET: "my_bucket"
AWS_SECRET_KEY_ID: "my_secret_key"
AWS_ACCESS_KEY_ID: "my_access_key"
I can access these variables in the production console =>
Loading production environment (Rails 3.2.14)
irb(main):001:0> ENV["S3_BUCKET"]
=> "my-s3-bucket-name"
However they don't return anything in the app itself. I set my linux box up following Ryan's excellent Pro railscast episode http://railscasts.com/episodes/335-deploying-to-a-vps
How can I get these variables accessible in the app itself?
If anyone needs more code just shout.
EDIT
I removed the figaro gem implemented the yaml config shown in the following railscasts tutorial: http://railscasts.com/episodes/85-yaml-configuration-revised. I think this is effectively what the figaro gem was doing however instead of using ENV variables, the tutorial uses CONFIG[:variables] which seem to work great.
Per Comment:
Nginx runs as its own user, so the environment variables need to live in it's space. As a user when you log in and run console, you're accessing a different set of environment variables than the nginx user accesses.
You can do this if you choose by adding them to the nginx config in the main context. But it's probably easier to go with straight yaml and add your secret tokens to your yaml file.
Related
This topic has a SOLUTION embeded at the end.
PROBLEM
I'm deploying for the first time a Rails app on a VPS on Ubuntu 18.04. with Nginx.
I followed the good tutorial of Gorails "Deploy Ruby on Rails To Production in 2019".
Everything worked, until I had the "Incomplete response received from application" page.
I checked the nginx logs on /var/log/nginx/error.logand saw the typical message "Missing secret_key_base for 'production' environment, set this string with rails credentials:edit"
As the method of Gorails didn't seems to work (after a bundle exec rails secret on his console app-side, he add a file /my_website/.rbenv-vars with a SECRET_KEY_BASE line, filled with the generated secret key), I decided to follow the multiples topics answering to this question.
Here is the thing, I'm not sure if the followings steps are the goods one.
I run bundle exec rails secreton my console, server-side, as deploy user. So I have my GENERATED_KEY_1
I add to ~/.bashrc : export SECRET_KEY_BASE="GENERATED_KEY_1"
I source ~/.bashrc
I check my key with echo $SECRET_KEY_BASE, and I have the good key displayed (GENERATED_KEY_1)
I edited my credential file as
development:
secret_key_base: ORIGINAL_KEY
test:
secret_key_base: ORIGINAL_KEY
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
and added Dotenv to my Gemfile, required it in application.rb
But none of this worked, after restarted nginx server.
So I restarted the previous step, with the root-user.
But again, it failed.
My questions are:
what I am missing ?
How can I know, if it's searching the key in the good place, as I have always the same error message ?
Which key am I suppose to generate ? App-side ? Server-side ? As root or deploy user ?
Do I have something else to configure in /etc/nginx/sites-available/default ? (I saw on this topic that this guys changed a rails_env production; to rails_env development; but I haven't any rails line)
Thank you, I'm a little bit desperate ^^
SOLUTION
During my many tests, I logged with the root user, and run EDITOR="vim" rails credentials:edit. This command had generated a master.key, which doesn't exist on your Github repo.
But first, I didn't modified it. I think that was the main problem, as the application use it to decrypt your credentials.yml.enc file. When I understood it, I edited the master.key with the content of the master.key on my computer app.
Even after editing credentials.yml.encwith <%= ENV["SECRET_KEY_BASE"] %>, this solution works. This corresponds to the answer of Lyzard Kyng, even if it's a bit different.
I can't run EDITOR="vim" rails credentials:editwith the deploy user, it doesn't work.
Rails 5.2 and later uses encrypted credentials for storing sensitive app's information, which includes secret_key_base by default. These credentials are encrypted with the key stored in master.key file. Git repository, generated by default Rails application setup, includes credentials.yml.enc but ignores master.key. After the deployment, which usually involves git push, Rails production environment should be augmented with this key some way.
So you have two options. You can securely upload master.key to production host via scp or sftp. Or you can establish shell environment variable RAILS_MASTER_KEY within the context of a user that runs rails server process. The former option is preferred, but as you have dotenv-rails gem installed, you'd create .env.production file under app's root and put there a line
RAILS_MASTER_KEY="your_master-key_content"
Don't forget to ensure that gem dotenv-rails isn't restricted within Gemfile by development and test Rails environments.
By the way since passenger module ver. 5.0.0 you can set shell environment variables right from nginx.conf
run rake secret in your local machine and this will generate a key for you
make config/secrets.yml file
add the generated secret key here
production:
secret_key_base: asdja1234sdbjah1234sdbjhasdbj1234ahds…
and redeploy the application after commiting
i had the same issue and resolved by this method.
It would be more secure to generate your key on the server and use it there, rather than push it to your repo from a local machine.
Instead of ~/.bashrc do this for using environment variables;
As root user, navigate to the # directory (can probably just use cd ..)
Enter nano home/<yourAppUser>/.bash_profile to navigate to (and create) the file to store the ENV
As you have already, just write this in the file: export SECRET_KEY_BASE="GENERATED_KEY_1"
You can store your database password here as well.
1_ Set credentials with
rails credentials:edit
2_ Upload master.key file to your production server.
If deploy with capistrano, copy master.key to shared folder (shared_path) and then add this to deploy.rb:
namespace :config do
task :symlink do
on roles(:app) do
execute :ln, "-s #{shared_path}/master.key #{release_path}/config/master.key"
end
end
end
after 'deploy:symlink:shared', 'config:symlink'
In my case, on rails credentials:edit, the file indentation were not accurate which gave the error on deployment. So make sure the indentation is correct on your local before deploying.
Is database.yml the right place to read the AWS keys from bashrc? database.yml sounds like a place only for database configs. Is there a more appropriate place where the AWS configs from bashrc could be read inside my Rails app?
Rails 5.2 onwards
Rails 5.2 has introduced the concept of encrypted credentials. Basically, from Rails 5.2 onwards, there is an encrypted credentials file that is generated on initializing the app in config/credentials.yml.enc. This file is encrypted, and hence, can be pushed to your source control tool. There is also a master.key file which is generated while initializing the app, which can be used to decrypt the credentials file, and make changes to it.
So, credentials for AWS could be added to it as:
aws:
access_key_id: 123
secret_access_key: 345
These keys could be accessed in your app as Rails.application.credentials.aws[:secret_access_key]. Other sensitive config, like credentials to other external services that are being used, can also be added to this config. Check out this blog by Marcelo Casiraghi for more details.
Pre Rails 5.2
There was no concept of a credentials system prior to Rails 5.2. There are a couple of ways in which you could try to come up with a solution to store your configuration.
A. You could create a YAML file for defining your config from scratch.
Create a file called my_config.yml and place it in the config folder. Path: config/my_config.yml
Add whatever configuration is required to the file, in YAML format (s described for AWS above)
Make changes in application.rb to load this file during initialization as follows:
APP_CONFIG = YAML.load(ERB.new(File.new(File.expand_path('../my_config.yml', __FILE__)).read).result)[Rails.env] rescue {}
Using this approach, you will then be able to use APP_CONFIG['aws']['access_key_id'] for the AWS configuration defined above. For this use case, it is strongly recommended to have separate configuration files for development and production environments. The production file should probably not be checked in to version control for security.
B. Another approach would be to use some gems for managing configurations like railsconfig/config
NOTE: To answer the bit about storing this configuration in database.yml, it is strongly recommended to not do so. database.yml is a configuration file for storing configuration related to databases. Separation of concerns really helps while scaling any application, and hence, it is recommended to place such configurations in a separate file, which can be independently maintained, without any reliance on the database config.
Absolutely. The standard place to configure things like AWS would be inside config/initializers. You can create a file in there called aws.rb.
app/
bin/
config/
|__ initializers/
|__ aws.rb
and inside this file you can configure your AWS setup using the environment variables from your bashr
Aws.config.update({
credentials: Aws::Credentials.new('your_access_key_id', 'your_secret_access_key')
})
Files inside this directory are executed on app start, so this configuration will be executed right when your app starts, before it starts handling requests.
It may also be useful to note that the AWS SDK for Ruby will automatically search for specific environment variables to configure itself with. If that's what you're using, and if you have the following environment variables set up in your bashrc
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
then you won't need any additional code in your Rails app to configure AWS. Check out more details here.
What started as a minor annoyance has now turned into a headache. I am building a Rails 4 app and am using Foreman for my dev setup with a Procfile and .env file for configuration. When I set an ENV variable in the .env file, it is correctly picked up by my app. In this case I am setting some ENV options for Paperclip in an initializer.
The problem surfaces when I go to change the value of the ENV variables. In the console, if I type ENV["MY_VAR"], it shows the new value. However, the value that was used in my initializer, which presumably was run when I started the console, shows the old value! Nowhere in my project is the old value listed anywhere. This leads me to believe that the environment is being cached somehow or that the env variables are being exported to my shell. I'm running out of places to look so any help would be greatly appreciated! I am developing on a Mac (10.9.4) with Ruby 1.9.3-p374 and Rails 4.1.0.
Example:
ROOT/.env
S3_BUCKET=mybucket
config/initializers/paperclip.rb
Paperclip::Attachment.default_options[:s3_credentials] = {bucket: ENV["S3_BUCKET"]}
If I change the value of S3_BUCKET to "newbucket" and run "foreman run rails c" or "rails c" to enter the console, this is what happens:
ENV["S3_BUCKET"] # => "newbucket"
Paperclip::Attachment.default_options[:s3_credentials] # => {bucket: 'mybucket'}
I should mention that this behavior also occurs in my classes that I've put in /lib. I imagine this is all due to something silly that I've overlooked. Any ideas?
If you're using Rails 4 out of the box, it comes with a gem called Spring that's intended to make your life easier by preloading an instance of your application in the background and reloading it as your code and configuration files change.
Spring, however, only monitors Rails' default configuration files, so you'll need to configure Spring to monitor additional any other files that you wish to trigger a reload.
Spring reads ~/.spring.rb and config/spring.rb for custom settings. You can add add the following line to the file of your choosing to watch your .env file for changes:
Spring.watch '.env'
See Spring's configuration documentation in the README for more info.
I am in the process of deploying a Rails app.
I get errors missing secret_key_base in the nginx log file when I have secret_key_base: <%= ENV["SECRET_KEY_BASE"] %> in the secrets.yml file.
I have generated the secret using rake secret in the console and placed in ~/.bashrc as
export SECRET_KEY_BASE="secret"
From the console I run echo $SECRET_KEY_BASE and copy secret to secrets.yml replacing <%= ENV["SECRET_KEY_BASE"] %> with secret.
Then everything works fine and the application runs fine in production environment.
I would rather not keep secret in secret.yml and I do not know how to correct this or what I am doing wrong.
Does anyone have any suggestions?
If you use passenger, add
passenger_set_cgi_param SECRET_KEY_BASE "yoursecret";
to your nginx configuration in the relevant section.
See this section in the passenger user guide.
In production ~/.bashrc might not be read - e.g., you run as a different user, nginx might not read ~/.bashrc before starting, etc. Lots of people run into this issue.
A common approach to this is to handle environment configuration like Rails handles database configuration. Create a config/something.yml file with settings for each environment, then read that yaml in a config/initializers/something.rb initializer and use the values for the specific environment. The rails_config and figaro gems automate this approach. I've often just done it without a gem, as it's not terribly difficult. The key, as with database.yml, is that you never want to check this into source control - use .gitignore with git.
If you want to stick with setting your secret key base using an environment variable, then how you do that depends on your production machine, and how you provision it and deploy your code. With Heroku, it's simple enough to just pop into the Heroku console and set it. For other situations, you could use something like Chef/Puppet/Ansible to set the environment variable for your server. Another approach would be to push that information using Capistrano.
You can put the secret base in /config/initializes/secret_token.rb:
SampleApp::Application.config.secret_key_base = 'Your_Base_here'
EDIT:
This is kind of discouraged in many cases, so edit your .env file and set your key base:
SECRET_KEY_BASE=Your_base_here
and put your secrets.yml back to:
<%= ENV["SECRET_KEY_BASE"] %>
You could alternatively use /config/initializes/secret_token.rb:
SampleApp::Application.config.secret_token = ENV['SECRET_TOKEN']
Which will give you the same result, being more secure.
If you are then planning on pushing this to Heroku:
heroku config:set SECRET_KEY_BASE=$SECRET_KEY_BASE
With the release of the secrets.yml file, I removed my reliance on Figaro and moved all of my keys to secrets.yml and added that file to .gitignore.
But when I tried to push to Heroku, Heroku said they needed that file in my repo in order to deploy the website. which makes sense, but I don't want my keys in git if I can avoid it.
With Figaro, I would run a rake task to deploy the keys to heroku as env variables and keep application.yml in the .gitignore. Obviously, I can't do that any more. So how do I handle this?
Secrets isn't a full solution to the environment variables problem and it's not a direct replacement for something like Figaro. Think of Secrets as an extra interface you're now supposed to use between your app and the broader world of environment variables. That's why you're now supposed to call variables by using Rails.application.secrets.your_variable instead of ENV["your_variable"].
The secrets.yml file itself is that interface and it's not meant to contain actual secrets (it's not well named). You can see this because, even in the examples from the documentation, Secrets imports environment variables for any sensitive values (e.g. the SECRET_KEY_BASE value) and it's automatically checked into source control.
So rather than trying to hack Secrets into some sort of full-flow environment variable management solution, go with the flow:
Pull anything sensitive out of secrets.yml.
Check secrets.yml into source control like they default you to.
For all sensitive values, import them from normal environment variables into secrets ERB (e.g. some_var: <%= ENV["some_var"] %>)
Manage those ENV vars as you normally would, for instance using the Figaro gem.
Send the ENV vars up to Heroku as you normally would, for instance using the Figaro gem's rake task.
The point is, it doesn't matter how you manage your ENV vars -- whether it's manually, using Figaro, a .env file, whatever... secrets.yml is just an interface that translates these ENV vars into your Rails app.
Though it adds an extra step of abstraction and some additional work, there are advantages to using this interface approach.
Whether you believe it's conceptually a good idea or not to use Secrets, it'll save you a LOT of headache to just go with the flow on this one.
PS. If you do choose to hack it, be careful with the heroku_secrets gem. As of this writing, it runs as a before_initialize in the startup sequence so your ENV vars will NOT be available to any config files in your config/environments/ directory (which is where you commonly would put them for things like Amazon S3 keys).
An equivalent for secrets.yml of that Figaro task is provided by the heroku_secrets gem, from https://github.com/alexpeattie/heroku_secrets:
gem 'heroku_secrets', github: 'alexpeattie/heroku_secrets'
This lets you run
rake heroku:secrets RAILS_ENV=production
to make the contents of secrets.yml available to heroku as environment variables.
see this link for heroku settings
if u want to run on local use like this
KEY=xyz OTHER_KEY=123 rails s