I'm using Twilio for an app and on production I set the auth token using heroku's CLI. I'm using sms-spec (https://github.com/monfresh/sms-spec) to test my app's Twilio integration locally. I want to set ENV['TWILIO_AUTH_TOKEN'] to my token in the test environment.
I use guard to auto-run my tests whenever I make changes so I don't want to have to manually set the ENV variable each time I run tests. I also don't want to put the token in my source code for security reasons.
Is there a way I can set the ENV variable for my local test environment such that it is permanent and not in my source? I've spent a few hours researching this and can't seem to find a good explanation of how to do this. Any help is much appreciated :)
Two approaches:
Use a gem like Dotenv (link). This is the approach I use in most of my applications for development. Simply include the gem in your gemfile, bundle install and then store any environment variable settings in a top level file called .env. Restart your rails server and ENV will be automatically loaded. Very easy to use and convenient.
If you are flexible on the ENV part, and you are running Rails 4.1+, you can use config/secrets/yml. This is documented very well in the Rails 4.1 release notes, Section 2.2. So, in your case, you would set it up like so:
development:
twilio_auth_token: verysecretstring
Then, in your initializer, instead of referencing ENV['TWILIO_AUTH_TOKEN'], you would use Rails.application.secrets.twilio_auth_token. I haven't tried this myself, but it is on my list as I would rather use native Rails functionality than a separate gem.
Of course, any files which contain your secrets needs to be safeguarded carefully. At a minimum, make sure you include in .gitignore so that your secrets do not find their way into your code respository.
Related
I'm working on a Rails application and I am attempting to organize my secret_key_base and related important secrets (Think API keys, database credentials, etc.)
I am trying to find a way to setup something like the following, under /config
/config
credentials.yml.enc
master.key
/config/credentials
development.yml.enc
development.key
production.yml.enc
production.key
test.yml.enc
test.key
First Question:
Is it that secret_key_base exists in /config/credentials.yml.enc, which is loaded (first?) and then the credentials are loaded for the environment rails is running in? Or should I create a different secret_key_base for each environment?
Second Question:
No matter what I do, when I run in development or test, tmp/development_secret loads first. In addition, whenever I try to access my secrets in development, (Rails.application.secret_key_base) as referenced here: What's the correct way of defining secret_key_base on Rails 6, I run into an issue where I only ever receive nil when looking for secrets I've defined in my development.yml.enc, which I assume is because it's not loading anything in that file, it's going to tmp/development_secret and not finding anything else (Maybe I'm wrong.)
Goals:
Stop tmp/development_secret from being created, and instead access
secrets using the specific .yml.enc file depending on the
environment.
Understand why /config/credentials.yml.enc exists if it doesn't
load in all the environments. If it doesn't, then it isn't clear when it loads.
Why?
Using config/database.yml as an example, I want to store different creds for each environment, but none of them in version control. (I want nobody but a few to have production.) However, I want to access them the exact same way in all of my environments. Not having creds load in production because of an issue with a .yml file will crash my app, so I don't want to load them differently in test/development.
Put together a blog post about this because searching for documentation on this feature is painful. It's a really easy thing because then only the master.key, and production.key would need loaded as ENV variables which is great. (Or possibility just one of them.)
This really should be a simple, out-of-the-box thing, but it's hard to find real documentation on best practices.
I've found the answer, at least the one I'm looking for. You can have your initializer load whatever file you want by overriding the defaults.
config.credentials.content_path = 'config/credentials/development.yml.enc'
config.credentials.key_path = 'config/credentials/development.key'
https://edgeapi.rubyonrails.org/classes/Rails/Application.html
There are several posts ans Stack Overflow questions about how to manage API tokens on the web, but I see a lot of people repeat what they read somewhere else, often with contradictions...
How do you deal with API Tokens, secrets and the like ?
Here's what I have read so far using 3 different gems
secrets.yml
Introduced with Rails 4.1, then updated to encrypted secrets around rails 5
When initially released on rails 4, they were (or were not ?) meant to be pushed on repositories. However I often saw examples using environment variables for production
# config/secrets.yml
development:
secret_key_base: super_long_secret_key_for_development
...
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
...
And at this point someone asked "Why do we use ENV for production ?". A legit question back then, often answered "We don't want production token to be hard coded in the application" (hence how it is not clear anymore if the secrets should have been committed or not).
Later, with Rails 5, secrets became encrypted with a master key, so the encrypted secrets.yml file could be committed to the repository, but then the problem remained the same with the master key used to read the secrets.
PROs:
Rails convention.
Easy deploy with capistrano-secrets gem and cap [stage] setup (and it only deploys stage secrets nice) or similar gems
YML data structure (array/hash ok) + can use Ruby code via ERB
With encrypted secrets since rails 5, easy to collaborate/share the secrets
CONs:
Need to use Rails.application.secrets.xxx
Many services like AWS still read from ENV to automatically setup their gems/services
Is not the 12 factors way (or is it ?)
Quite new, so not really used yet ?
If using encrypted secrets, need to manage the master key
Bkeepers dotenv
Simply defining a .env file at the root that is read by Rails on startup
Plugins like capistrano-env allow to easily inject environment specific ENV on servers, and secrets can still must be managed using .env.staging, .env.production
PROs
ENV is in 12factor rules
3.5k stars... maybe not for nothing ?
the dotenv approach is now available on almost all other languages (JS, Go, etc)
Recent versions allow reusing variables (API_ROOT=X, SOME_ENDPOINT=${X}/endpoint)
CONs
No Ruby interpolation (unless extra code is added to parse the .env with the ERB templating engine for example)
limited to string-string key/val
Figaro
Some sort of hybrid secrets/ENV. With 12factors/Rails/Heroku in mind, but in the end doesn't seem better than the rest...
From the above and the rest I didn't write, it would seem like secrets.yml would be a great winner if those secrets were put in ENV instead (and tbh I feel lazy about writing Rails.Application.secrets each time).
So, suppose I start a quite new Rails app, and also based on your experience. Which one would you choose ?
(My particular stack uses AWS + Capistrano, no Heroku)
I personally think that the "right" approach depends on your environment.
In my case, I have servers which are managed by IT and I don't have access to the vhost or anything else to easily set environment variables. Because of this, I commit a secrets.yml file which doesn't contain the production stanza, then set up Capistrano to add this file to shared_files. On each server, I add the file.
If I had easy access the the vhost, and I was managing the server vhosts with Puppet, Chef, Ansible, or similar, I would use the environment variable approach like the default secrets.yml file.
Your case with AWS appears to be the latter. Ultimately, either option is fine. There is little downside to committing the secrets.yml file without a production stanza.
First, all three methods can be 12-factor compatible. It is compatible if you pass the config value by ENV variable, instead of copying one file to the server first.
My thoughts are each of these solutions:
Rails secrets
Developers are forced to go 12-factor, either manually set it on production server, or have another file on local machine and then pass it as ENV every time during deploy. (Didn't know about capistrano-secrets, it probably handles this)
(I think what you said in CON #2 and #3 are the opposite to secret.yml solution)
The accessor is also quite long as you mentioned.
dotenv
It does not encourage 12-factor, and was not originally designed for production env anyways. Either you write code to pass its value as ENV to production during deploy (making it 12 factor compatible), or you copy your .env.production file to the production server.
Also it forces you to use the flat key:value configuration. No nesting.
Figaro
Though it uses YAML, it does not allow nested hash.
It is 12 factor compatible, e.g. it includes code to transfer the config to heroku.
My Solution
I wrote a gem, in which settings are stored in gitignored YAML file. Nesting is allowed. When accessing some value, do Setting.dig(:fb,:api).
It provides mechanism for 12-factor app deploy, by serializing the whole YAML file into a string and pass it to production as ENV.
I no longer have to distinguish between secret config and non-secret config. They are in one place and secret by default. I get benefit of 12-factor app while using easy to namespace YAML files.
Can someone explain to me what the Rails environments are and what they do? I have tried researching myself, but could not find anything. From what I gather, the environments are:
Development
Productions
Test
Each "environment" is really just a config. You can launch your app in various different modes, and the modes are called "environments" because they affect the app's behaviour in lots of different ways. Ultimately, though, they are just configs.
BTW you can't have looked very hard when you looked "everywhere", because i just googled "rails environment" and the top result was this
http://guides.rubyonrails.org/configuring.html
which is the official explanation of configuring the rails environment.
From what you have provided in your question, it seems that you are asking:
"What are the difference between each environment configuration in Rails?"
Rails comes packages with 3 types of environments. Each have its own server, database, and configuration. See Rails Guides: Configuration for more information on options available to you.
Setting up the environment
To set your Rails environment, you will want to enter in command line:
export RAILS_ENV=<env>
Where <env> can be test, development, or production. Setting this environment variable is crucial, as it will determine what gems are installed, or what env is touched when running rails console or rails server.
Included in configuration is the gemset used for the app. When you run rails new, you will find a Gemfile with groups test, development, and production. These groups correspond to the environment currently set. When the environment is set to one of those, running bundle install installs all gems related to that group (and gems not listed in a group).
Included environments
test is designed for running tests/specs. This database will likely be bare bones, except for seeds you may call before running the suite. After each test is complete, the database will rollback to its state before the test began. I do not recommend launching rails server, as running tests (via MiniTest or RSpec) will do this for you, and close the server once the suite is finished.
development allows you to "test" your app with a larger database, typically a clone of production. This allows you to test actual real-world data without breaking production (the version that customers or end-users will experience). To view the development environment in action, change the RAILS_ENV and launch rails server. This is good for deciding how you want your pages to look (CSS, HTML). It is also good practice to briefly "test" your app yourself, clicking around making sure everything "looks" good and the JavaScript works.
production is reserved for the customer and end-user. Configuration includes the actual domain of the app, which ports to use, and initializers or tasks to run. You do not want to play around with your database, as it may be customer-impacting. Ideally, the app should work as best as it can, since this is considered your "final product."
Here are some good reads about Rails Environments
http://teotti.com/use-of-rails-environments/
and
https://signalvnoise.com/posts/3535-beyond-the-default-rails-environments
good luck !!
I have secret configuration that I do not want to commit to version control. The way Heroku deals with this is to set environment variables that can then be read by the app. This is a nice solution, and I'd like to use the same principle when using Passenger/nginx on my own server.
I can set these globally, e.g. in my .bash_profile, but this makes them available globally. If I have, for instance, two sites, both interacting with Twitter, and using TWITTER_API_KEY, this should be set differently for each site.
What I've found through Google suggests this isn't possible, but thought it worth asking anyway.
The alternative is to have a private config file that isn't committed into version control. That works, but using environment vars just seems more elegant to me.
I have a url that I am using in one of the controllers. Is there a better place to put this url? The url uses an API key and I was wondering if there is a better place to add this url and/or api key such that its not added in the controller class code and ergo more editable? If i add it as an environment variable or anything else how do i access it from my controller class? thank you. ITS A RUBY AND RAILS PROJECT
Using environment variables might be a good idea if you want to keep things like API keys and passwords out of your source code. Accessing them from within your code is done with the ENV object:
my_api_key = ENV['MY_API_KEY']
To use this technique, you need to set up the variables in your environment before launching your app, and how you do this depends on your local setup, and will likely also vary between development and production.
In development, you can simply set the environment vars in your shell, e.g. with bash:
$ export MY_API_KEY=foobar123abc
$ rails s
Now rails will start and have access to this environment variable. You can also set variables for just a single command:
$ MY_API_KEY=foobar123abc rails s
Depending on what the sevice/api is, you could set some of them to default development/test values in config/environments/development.rb (or test.rb):
ENV['MY_API_KEY'] = 'non_secret_api_key_that_can_be_shared_around'
Setting up environment variables in production will depend on how you're deploying your app. Phusion have an article on using environment variables in Passenger if your using that. There's also a useful article on using environment variables with Heroku which is worth a read even if you're not using them for deployment.
You can add it to application.rb file as
config.service {
:api_key => 'api_key'
}
Or better yet, add it to development.rb and production.rb files so that you can test it better.
You can access this api_key from controller like
Rails.application.config.service[:api_key]