I am trying to secure my app properly by setting environment variables. I am doing it with figaro gem. I am not using heroku, just a vps I setup by myself.
Now on development all works good and makes sense. application.yml contains hardcoded database passwords which are not on database.yml anymore. application.yml is not pushed on repository and passwords are not shared. All good.
But I am having hard time making sense of it on production. If I set production section on database.yml with environment variables and then the application.yml is not on the server running on production, how can it possibly work? Or if I set them on another file that is then pushed later on repository, I just moved harcoded passwords from a file to another and are still accessible. Or are environment varibales supposed to be used only on test and development?
Could someone give me a clear explanation? I have been reading other questions and articles around but I can't make sense of it.
You want to set Unix environment variables in production. Check out this guide which states:
Variables in the config/application.yml file will override environment variables set in the Unix shell.
In other words, in development you would set these variables in application.yml which would then override any values set in Unix. In production, you would set them in the shell and that is where the values would come from. I suggest doing some more general reading regarding environment variables as they are a fundamental part of application development.
Related
TL; DR
How to use production.yml.enc and staging.yml.enc credential files in two production- and staging- Rails apps, while the app has only the regular development, test and production environments?
I'm using Heroku and refer to it in this question. Yet this is not specific to this vendor.
In detail
An application is often deployed multiple times. An instance serves as the production, while another is the staging app, expected to be put in production. Rails facilitates this pattern, since creating new environments is easy.
However, Heroku suggests not to do this, with good reasons. For example, one can be tempted to put some if Rails.env.production? here and there, paving the way for some "but it works in staging!?" on Friday evening. It is better to have a single production environment, with different sets of parameters to actually differentiate the stages (eg. a different AWS S3 bucket name, a different API key, etc.). To achieve this, Heroku's advice is to rely on environment variables.
Since Rails 5.2 and later in Rails 6, credentials are conveniently handled via encrypted Yaml files in config/credentials. This is typically where one would like to store all these variables that change from an environment to another, instead of using messy environment variables. This mechanism can be used in Heroku thanks to a single RAILS_MASTER_KEY environment variable that contains the key used to decrypt the credentials file.
But these pieces do not fit well. How can we have a single production environment, whereas credentials files are per-environment?
This can be implemented with a staging credentials file and key, and a switch in application.rb based on an environment variable:
Create and populate a staging credentials file and key with EDITOR=vi rails credentials:edit --environment staging.
In application.rb, add:
# Introduce the environment variable RAILS_CREDENTIALS_ENVIRONMENT to specify a custom
# environment name of which to use the credentials file. Specifically aimed to use in staging,
# where the environment is set to "production", but we need to use the "staging" environment variables.
# This environment variable is also used to select the corresponding key file or -if that does not exist-
# the corresponding environment variable.
if ENV['RAILS_CREDENTIALS_ENVIRONMENT'].present?
new_filename_no_ext = Rails.root.join 'config', 'credentials', ENV['RAILS_CREDENTIALS_ENVIRONMENT']
config.credentials.content_path = "#{new_filename_no_ext}.yml.enc"
if File.exist? "#{new_filename_no_ext}.key"
config.credentials.key_path = "#{new_filename_no_ext}.key"
end
end
Set an environment variable RAILS_CREDENTIALS_ENVIRONMENT to production or staging, depending on the case. For example, Heroku does this with:
heroku config:set -a theapp-staging RAILS_CREDENTIALS_ENVIRONMENT=staging
If on staging or produciton you want to store your key in an environment variable instead of a key file, then simply assign the key to the RAILS_MASTER_KEY environment variable. As documented, this takes precendence over keys stored in files. Note that on your development machine you wouldn't want to have a RAILS_MASTER_KEY set, otherwise the credential files for all environments still get the same key and are thus accessible by everyone that needs to have access to (e.g.) only the development credentials.
Have several files one by each environment in your source code repository o local server disk, works but you will have several files with hardcoded values.
But, if you expect several teams with several requirements on your single app, you will need several development and test environments in order to keep an independent teams which are fully responsible for their Services:
Development
Release / Deployment
Ops (not platform/system administration)
An approach to management this is : externalize your configurations on platforms called: Configurations Manager
This platforms, must have the following features:
key-vaue pair creation by app. Like heroku web dashboard
expose http endpoint to get this variables from remote apps
security
Your rails app must get the variables at the startup stage or instantaneous if your language support hot reload variables.
Here some Configurations Managers:
Zookeeper : http://www.therore.net/java/2015/05/03/distributed-configuration-with-zookeeper-curator-and-spring-cloud-config.html
Consul : https://www.consul.io
Spring Cloud : https://www.baeldung.com/spring-cloud-configuration
Confignet: https://github.com/software-architect-tools/confignet
Whit this approach you will have a platform to management several apps for several environments like heroku web variables creation but more sophisticated.
I was reading from this article that you can create a config/local_env.yml with environment variables and then use config/application.rb to read it to replace/take priority before the environment variables that you export in your .bashrc. What then is the .env used for then? Does it serve the same purpose as the config/local_env.yml?
All of these methods are used to feed environment variables to your rails application. So, from an app's point of view, it serves the same purpose whether you export it from .env or .bashrc or config/local_env.yml files.
The differences in these methods are really a matter of personal choice among the team members involved in maintaining the app's development and deployment environments. However, here are few things to consider while opting for either of these choices.
.bashrc - Use this if you manually configure deployment servers and really comfortable with linux/unix command line system administration. This configuration file is specific to bash shell (Bourne Again Shell). You need to configure different file if your server uses different shell (for example: .zshrc if it uses Z Shell)
.env - Use this if you want to keep your app centric environment variables within the app itself while maintaining different variations of environment variables for different runtime environments of your rails app. For example: .env.development and .env.test files with specific values of the environment variables for your development and test environments respectively. This gives you more control of your app's environment variables and do not have to rely on the platform (terminal shell) you want to deploy your app.
config/local_env.yml - This is similar to .env approach, which is provided out of the box by rails gem that allows you to configure environment variables for your app in yml format. This method also keeps your app's configuration within the app irrespective of the shell you are using to run your app.
In addition to the previous answer, another downside of using .bashrc is that it is specific to only one user, so if you're e.g. starting your app server as a systemd service then I believe it won't see your variables.
Meanwhile, .env's Github readme says that it is not the most recommended thing to use outside of the development environment, although it is OK for that purpose.
Another two options to consider are:
Rails secrets. The benefit of this approach is that you get to commit it to git since it's encrypted, so when working in a team you will all have access to the same file. The downside is that it is coupled to the RAILS_ENV variable, so you can't use it to set that e.g. to production on your production app (but you can manually pass it every time it's invoked). Another downside is that if you have a staging environment, then apparently Heroku discourages using RAILS_ENV=staging, which you sometimes really need, so if you need it then you can either do it anyway, or you'll need to set the differing variables via a different mechanism - for example my app has a variable which points to the URL of another part of my app, this URL needs to point to its staging variant on staging, and to its production variant on production, so it needs to differ between my production and staging environment.
Using an /etc/environment file - note that systemd services don't have access to it by default so you'd have to add the line EnvironmentFile=/etc/environment. Also if you're running a shell script from a non-login shell (which happens sometimes), they it won't load them either, but the solution is to just include in your script this: set -a; source /etc/environment; set +a. You should be careful not to commit this to git. If you're working in a team and you need to manage this file then it gets a little complicated since it's not committed to git, but maybe there's a way to have it encrypted. (systemd services note: you might optionally use LoadCredential= for sensitive variables such as private keys, so that other processes won't have access to them; if you do this then you can also commit to git /etc/environment which should now contain non-sensitive variables only).
In my opinion both of those are valid and it's fine to pick whatever is easier to do in your app.
I work with Rails 4 and Ruby 2.1 and sorry but I am working on Windows
I have read a lot about this topic "Missing secret token, secret key base" but actually I do not understang anything.
I do not use Heroku, Git, Puma, Passenger or everything else I've read. I just thought I could instead of running rails s as usual run rails s -e production and see what is the version of my web application in production.
But I have the error "Missing secret_token and secret_key_base for production environment, set these values in config/secrets.yml"
I read about solutions using openSSL, export SECRET_KEY_BASE=<the long string> but I do understand the solutions.
I thought it was a problem related to the system of connection by password I settled thanks to Rails tutorial of Micheal Hartl. So disabled SSL connection. But nothing change.
This is my config/secrets.yml :
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
Can someone explain how to concretely solve this issue ?
GDMN I am sorry that everyone gave you such poor explanations and instructions. Ok onto it then, shall we......
First everyone is right you no longer need "secret_token", you do however need "secret_key_base". What this does is it secures your visitors connection and keeps your system and app more secure. That is a simple explanation but all you need to worry about at the beginners level.
Second the ENV stands for "Environment Variable" they are used in all operating systems and they refer to variables on the OS level that hold information that you do not want to be accessible t someone trying to gain access to your site. For instance in Ruby On Rails if you HARD CODE the secrety_token_base string/hash then I hacker can gain access by using your security_token against you. I have seen this happen and it is not pretty, if the individual is skilled enough then they can gain access to even your root/admin account.
Now on to setting it all up. I only know the linux way and I know you are looking for the windows method but this should at least give you an understanding to seek out the information relevant to your operating system.
First thing you would need to do is generate your secret_token_base by running
bundle exec rake secret
To my knowlege this is the way you do it in all Operating Systems. After you run the above command the console will return a string and you would need to copy it. Once copied you would run the following command:
export SECRET_KEY_BASE=WhatYouJustCopied
Then we would check to make sure the Environment Variable SECRET_KEY_BASE is set by running:
env | grep -E "SECRET_TOKEN|SECRET_KEY_BASE"
If you do not have SECRET_TOKEN set you will only get the KEY_BASE.
If you want to learn more in depth information please visit this link it may be a little dated but most of it is still relevant and conceptually it is the same.
I wish you luck on your new found ROR Adventure! It is fun once you get the hang of it!
From your command prompt, run:
bundle exec rake secret
It will generate a long string of characters. Copy this string and paste it into config/secrets.yml as follows:
production:
secret_key_base: <paste the string here>
Note: only do this if you are not using a public repository. This key should not be accessible to anyone else. An alternate, and more secure way of doing this is using an environment variable. See this: http://daniel.fone.net.nz/blog/2013/05/20/a-better-way-to-manage-the-rails-secret-token/
So, if you look inside the secrets.yml file, you will see where your secret_key_base is set for each of your environments. When you look at the production setting, it wants an env variable to initialize your secret_key_base. Normally, in production, you would want your app server to fetch the value from a general place in case you need to spin multiple servers up, you wouldn't have to hard-code your secret_key_base everywhere since that is not a secure way of setting that variable.
Basically, you will have to have that env variable set up on the machine that will run your rails app in production. There are so many different ways to set this.
What I have set up to initialize my ENV variables for production is have a separate yml file that is constructed like this
# config/env_provider.yml
production:
SECRET_KEY_BASE: "KEY GOES HERE"
other_production_variables: #...etc
Then my separate servers will be told where to find this file before initializing the variables (This is not checked into version control). After the file is in place it will know to initialize the variables from the following code in environment.rb before your app initializes
#config/environment.rb
YAML.load_file("#{::Rails.root}/config/env_provider.yml")[::Rails.env].each {|k,v| ENV[k] = v }
# This is before Rails.application.initialize!
The thing with this set up is to make sure that you do not have this file accessible to everyone to see, only allow your application servers to use it. Anyway, this is how I deal with ENV variables and deploy them to production. I hope this helps you.
With the release of the secrets.yml file, I removed my reliance on Figaro and moved all of my keys to secrets.yml and added that file to .gitignore.
But when I tried to push to Heroku, Heroku said they needed that file in my repo in order to deploy the website. which makes sense, but I don't want my keys in git if I can avoid it.
With Figaro, I would run a rake task to deploy the keys to heroku as env variables and keep application.yml in the .gitignore. Obviously, I can't do that any more. So how do I handle this?
Secrets isn't a full solution to the environment variables problem and it's not a direct replacement for something like Figaro. Think of Secrets as an extra interface you're now supposed to use between your app and the broader world of environment variables. That's why you're now supposed to call variables by using Rails.application.secrets.your_variable instead of ENV["your_variable"].
The secrets.yml file itself is that interface and it's not meant to contain actual secrets (it's not well named). You can see this because, even in the examples from the documentation, Secrets imports environment variables for any sensitive values (e.g. the SECRET_KEY_BASE value) and it's automatically checked into source control.
So rather than trying to hack Secrets into some sort of full-flow environment variable management solution, go with the flow:
Pull anything sensitive out of secrets.yml.
Check secrets.yml into source control like they default you to.
For all sensitive values, import them from normal environment variables into secrets ERB (e.g. some_var: <%= ENV["some_var"] %>)
Manage those ENV vars as you normally would, for instance using the Figaro gem.
Send the ENV vars up to Heroku as you normally would, for instance using the Figaro gem's rake task.
The point is, it doesn't matter how you manage your ENV vars -- whether it's manually, using Figaro, a .env file, whatever... secrets.yml is just an interface that translates these ENV vars into your Rails app.
Though it adds an extra step of abstraction and some additional work, there are advantages to using this interface approach.
Whether you believe it's conceptually a good idea or not to use Secrets, it'll save you a LOT of headache to just go with the flow on this one.
PS. If you do choose to hack it, be careful with the heroku_secrets gem. As of this writing, it runs as a before_initialize in the startup sequence so your ENV vars will NOT be available to any config files in your config/environments/ directory (which is where you commonly would put them for things like Amazon S3 keys).
An equivalent for secrets.yml of that Figaro task is provided by the heroku_secrets gem, from https://github.com/alexpeattie/heroku_secrets:
gem 'heroku_secrets', github: 'alexpeattie/heroku_secrets'
This lets you run
rake heroku:secrets RAILS_ENV=production
to make the contents of secrets.yml available to heroku as environment variables.
see this link for heroku settings
if u want to run on local use like this
KEY=xyz OTHER_KEY=123 rails s
I've enjoyed using Rails on Heroku, and like that I can adjust the configuration property of a Heroku app without having to commit a change to xyz.yml and redeploy.
It would be nice to completely do away with the Yaml config files in my Rails app, and rely as much as possible on storing configuration in ENV. This goes along with the 12-factor config principle.
However, there are some trade-offs in switching from a Yaml-based configuration management to a Heroku/12-factor-based one.
While it's true that a proliferation of deployments (qa, stage, prod, dev, demo, labs) can lead to a proliferation of Yaml files, it's very easy to copy-paste to create a new configuration profile. I don't see a way to 'copy' configuration profiles from one deployment to another in Heroku.
Storing configuration data in the repo means that, in the case of Heroku, deploying and configuring and application are accomplished in a single operation. If I were to move my configuration out of Yaml files and into ENV variables, I would have to configure my application in a separate step after deployment.
Would like to hear from people who have used 12-factor style configuration in their private applications, and how they have managed lots of configuration variables across lots of deployments.
How do you quickly configure a new deployment?
Where do you keep your authoritative source of configuration variables, if not the repo? How do you distribute it among developers?
Thanks!
What I typically use is Yaml using the ENV and provide defaults. For instance, YAML can be ERB'ed happily to include your ENV vars:
foo:
var: ENV["MY_CONFIG"] || "default_value"
You just need to make sure that you load the Yaml with ERB when you read it:
YAML.load(ERB.new(File.read("#{Rails.root}/config/app_config.yml")).result)
By doing this your code works fine in dev, but also allows you to set config vars in the environment as well.
You can accomplish this relatively easy with some simple shell scripts, iterate existing variables via heroku config or heroku release:info v99, and then set heroku config:set k=v --app
But if its a problem/pain/friction perhaps you have too much inside your env var configuration.
A bit of a late answer, but I believe this is what you are looking for.
I developed a gem called settei, allowing you to use YAML files to configure the app. However the gem will serialize the YAML file as one environment variable during deploy. This way one get the best of both worlds: YAML for ease of management/creating derivative environment, and ENV for 12-factor compliance.