I am writing a Rails app with automated deployment using Capistrano. In the deploy.rb script I have:
set :deploy_to, '/my/deploy/path/'
And in production.rb I have:
server 'example.com', user: 'secret_user_name', roles: %w{web app db}
Currently the app is private. But assume I wanted an open source app like this. Then I wouldn't want secret_user_name and /my/deploy/path to be stored in the repo. In the Rails project if I encountered an issue like this I would store the secret values in secrets.yml and access them from there. However I don't have access to secrets.yml from Capistrano. So I could manually load the secrets file but I'm sure there is a better way to do this.
So my question is: How can I have an automated deployment process without exposing server information using Capistrano? Is there a recommended way to store secrets like there is in Rails?
If your intention is to maintain your own deployment environment (which you keep secret) but open source the code of the application itself, then I would simply move the Capistrano-related files to a separate private repository. Then you can open source the app itself, but keep the Capistrano config private.
There is no need for Capistrano's deploy.rb, etc. to live in the same directory structure or even the same repository as the app that is being deployed. After all, Capistrano deploys based on the :repo_url, which can be anything. It doesn't have to match repo where Capistrano's files are kept.
If you want to give other people (i.e. those that fork/clone the app) the ability to deploy to their own infrastructure, perhaps the easiest solution is to write up a wiki page explaining how they can set up their own Capistrano config. Deployment environments can vary widely and therefore it is probably not something you can do simply with environment variables or encrypted secrets.
In any case make sure you audit and rewrite your Git history if necessary to make sure you won't be leaking any sensitive config when you make the repo public.
Environment variables might help you.
You can put export SSH_PROD_USER=secret_user_name; in your preferred shell profile. For example if you use bash then it would be ~.bash_profile. Then use it in production.rb like this:
server 'example.com', user: ENV['SSH_PROD_USER'], roles: %w{web app db}
So basically ruby will have all your environment variables
Use something like Figaro gem: https://github.com/laserlemon/figaro
and don't push to your repo your application.yml
Or use an encrypted repo with credentials.
Related
I'm working in a team in a Rails project.
Each of us have a local database for development purpose. We have a problem: Everyone have different configuration for the local database. When someone make a commit without reset the /config/database.yml the other members of the team can't use their database because the access is not configured.
Can I have a local configuration not commited? To each one can works without problem and without the need of re-set the file every time? Sometime like the local_settings.py in Django
You can configure the database in Rails via both the config/database.yml file and the DATABASE_URL env var. They are merged but settings from DATABASE_URL take precedence.
A good solution is to check in the config/database.yml with a basic configuration thats just enough to support postgres.app or whatever is the most common solution.
Developers that need other settings can set the DATABASE_URL env var to customize the database configuration. Gems like figaro and dotenv make this easier.
If you have ever wondered this how Heroku manages to use the correct DB no matter what you throw into database.yml and the DATABASE_URL ENV var is how you should be configuring your production DB.
Why should I use ENV vars and not more database_*.yaml files?
The twelve-factor app stores config in environment variables (often
shortened to env vars or env). Env vars are easy to change between
deploys without changing any code; unlike config files, there is
little chance of them being checked into the code repo accidentally;
and unlike custom config files, or other config mechanisms such as
Java System Properties, they are a language- and OS-agnostic standard.
https://12factor.net/config
Add config/database.yml in to the .gitignore file at root path of your rails-app.
Copy config/database.yml with the values you need for production into config/database_example.yml.
Now you can modify your local database and in production you copy config/database_expample.yml to config/database.yml
If the config file is ignored by git, everyone can change it locally without getting tracked by git.
EDIT:
HERE YOU SEE HOW YOU CAN REMOVE FILE FROM TRACKING!!!
Ignore files that have already been committed to a Git repository
I have an app being developed on github. I want to opensource it. Currently we use a capistrano script to deploy to our staging & production servers.
I am trying to figure out how we can put our config files in a separate repo, and still use capistrano to execute one touch deploys. The goal is that we can open up our repo for anyone to use.
You have some options.
Use environment variables: You can set environment variables at both your local and production machines. In your app you'll read these vars by doing: <%= ENV['my_var'] %>. Doing this you can commit your app to a public repo without worrying about expose sensitive information like passwords and keys. For example, set a db_password environment var to store your database password and in your database.yml you could read it doing: password: <%= ENV['db_password'] %>
You can use gems link dotenv (https://github.com/bkeepers/dotenv) or figaro (https://github.com/laserlemon/figaro): By using these gems you won't need to set environment variables manually on your machines, you will define them in a .env file instead. You'll be able to read them the same way using <%= ENV['my_var'] %>. You will have to ignore your .env files at your .gitignore and tell capistrano to create the environment variables at your production server reading from your .env file.
Another alternative would be make different config files for development and production and ignore them in your .gitignore. You can store your config production files in a different repo and have it updated on your machine at the time of your deploy. You'll just need to copy your config files from your local machine to the production server (https://coderwall.com/p/wgs6gw/copy-local-files-to-remote-server-using-capistrano-3) after updating the app repo on your production server.
The last alternative is the one I use most (in my case I use ansible in place of capistrano).
If you wanna see an example I have an application and a deployment task currently running in production that you can checkout:
Application: https://github.com/josuelima/bolao
Provisoning and Deployment: https://github.com/josuelima/ansible-sandbox
Hope I made myself clear and you got the idea.
I've enjoyed using Rails on Heroku, and like that I can adjust the configuration property of a Heroku app without having to commit a change to xyz.yml and redeploy.
It would be nice to completely do away with the Yaml config files in my Rails app, and rely as much as possible on storing configuration in ENV. This goes along with the 12-factor config principle.
However, there are some trade-offs in switching from a Yaml-based configuration management to a Heroku/12-factor-based one.
While it's true that a proliferation of deployments (qa, stage, prod, dev, demo, labs) can lead to a proliferation of Yaml files, it's very easy to copy-paste to create a new configuration profile. I don't see a way to 'copy' configuration profiles from one deployment to another in Heroku.
Storing configuration data in the repo means that, in the case of Heroku, deploying and configuring and application are accomplished in a single operation. If I were to move my configuration out of Yaml files and into ENV variables, I would have to configure my application in a separate step after deployment.
Would like to hear from people who have used 12-factor style configuration in their private applications, and how they have managed lots of configuration variables across lots of deployments.
How do you quickly configure a new deployment?
Where do you keep your authoritative source of configuration variables, if not the repo? How do you distribute it among developers?
Thanks!
What I typically use is Yaml using the ENV and provide defaults. For instance, YAML can be ERB'ed happily to include your ENV vars:
foo:
var: ENV["MY_CONFIG"] || "default_value"
You just need to make sure that you load the Yaml with ERB when you read it:
YAML.load(ERB.new(File.read("#{Rails.root}/config/app_config.yml")).result)
By doing this your code works fine in dev, but also allows you to set config vars in the environment as well.
You can accomplish this relatively easy with some simple shell scripts, iterate existing variables via heroku config or heroku release:info v99, and then set heroku config:set k=v --app
But if its a problem/pain/friction perhaps you have too much inside your env var configuration.
A bit of a late answer, but I believe this is what you are looking for.
I developed a gem called settei, allowing you to use YAML files to configure the app. However the gem will serialize the YAML file as one environment variable during deploy. This way one get the best of both worlds: YAML for ease of management/creating derivative environment, and ENV for 12-factor compliance.
I have a few web services that require secure tokens/keys/passwords to be passed in. Where should I define these secure values for my rails app to see? I want the development keys in version control, but don't want the production keys in version control. How should I set this up? I'm new to rails.
You see the question properly.
Put your passwords and keys in some yml file excluded from version control.
Then on your production server, create the very same file and symlink your app to it every time you deploy.
EDIT.
Capistrano is almost made to fits these needs:
put your yml files in the shared folder
In your capistrano deploy.rbfile:
after 'deploy' do
run "ln -s #{shared_path}/database.yml #{release_path}/config/database.yml"
end
to work with yml files: http://railscasts.com/episodes/85-yaml-configuration-file
apneadiving is right, symlinking the files is a good idea. Another approach is to put the keys in the shell variables, accessible only to the user that runs the app. Then, in your rails app you'll have
login = ENV['SERVICE_LOGIN']
password = ENV['SERVICE_PASSWORD']
As of Rails 4.1.0, check out secrets.yml.
I have a number of Rails apps hosted on GitHub. They are all currently private, and I often will deploy them from their GitHub repository. I'd like to be able to make some of them open source, just like the ones you can find on http://opensourcerails.com.
My question is: How can I make these repositories public without giving away super secret credentials?
For example, I can look in /config/initializers/cookie_verification_secret.rb and see the cookie secret for nearly every one of them. I don't understand how this is acceptable. Are these users all changing these values in their deploy environments somehow?
Some users even expose their AWS secret and key! Others will instead set their AWS secret to something like:
ENV['aws-secret']
although I'm not sure at what point they're setting that value.
So, what are the best practices for open sourcing your Rails app without compromising your app's security.
I recently went through this with one of my own apps. My solution was to store anything secret in a git-ignored YAML config file, and then to access that file using a simple class in the initializers directory. The config file is stored in the 'shared' folder for the Capistrano deployment and copied to config at each deploy.
Config store: http://github.com/tsigo/jugglf/blob/master/config/initializers/juggernaut.rb
Example usage: https://github.com/tsigo/jugglf/blob/6b91baae72fbe4b1f7efa2759bb472541546f7cf/config/initializers/session_store.rb
You may also want to remove from source control all history of the file that used these secret values. Here's a guide for doing this in Git that I used: http://help.github.com/removing-sensitive-data/
If you're using foreman, put an .env file in the root of your app. (foreman docs)
.env will have
AWS_SECRET=xxx
AWS_ACCESS=yyy
Then when you need to use the keys, insert:
ENV['AWS_SECRET']
ENV['AWS_ACCESS']
Though it's important that you don't commit this .env to your version control. So if you're using git, add the .env to your .gitignore.
Bonus round! - Heroku
If deploying to Heroku, these environment variables need to be configured in the Heroku environment, too. There are two options:
Manually add the keys through the heroku config:add command
Use the heroku-config gem to synchronize your local environment variables, both ways.
Not storing any secret value at all. At any point in the history of a Git repo.
Those values should be stored elsewhere, leaving only template config files versioned, along with a script able:
to read the right values from the external repo
and build the final config file complete (with the secret values in it)
By keeping the tow set of data separate (sources on one side, secret values on the other), you can then open source the sources repo without comprising any secrets.
I actually took a hint from your question, using ENV.
I had three different secret values that I didn't want made available. They're the app's secret token of course, and Twitter's consumer key and secret. In my secret token initializer:
KinTwit::Application.config.secret_token = ENV['SECRET_TOKEN']
Twitter.consumer_key = ENV['CONSUMER_KEY']
Twitter.consumer_secret = ENV['CONSUMER_SECRET']
I'm hosting my project on Heroku, so I added these as configuration variables to Heroku.
[03:07:48] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add CONSUMER_KEY=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v7
CONSUMER_KEY => ub3rs3cr3tk3y
[03:08:40] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add CONSUMER_SECRET=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v8
CONSUMER_SECRET => ub3rs3cr3tk3y
[03:08:57] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add SECRET_TOKEN=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v9
SECRET_TOKEN => ub3rs3cr3tk3y
Now, the values are ready on my next push. But, what if you aren't using Heroku? I'm obviously not an expert on every single rails deployment (jeesh, not even a Heroku pro), but an example of this would be doing a db:migrate for testing.
$ RAILS_ENV=test rake db:migrate
The KEY=value pair before the command sets the environment variable, so running this command, echo ENV['RAILS_ENV'] would print test. So however this is set up in your environment is how you would do it. But, the environment variables aren't in your code, so that's the trick.
[EDIT - The following method has the annoyance of having to switch to the Production branch to run "rails server" in order to include necessary cookies. Thus, making edits while the server is difficult... and I'm still looking for a good solution]
After further investigation, I think the solution I was looking for was to exclude anything that stored a secret value from my Git repo's master branch (just as #VonC said). But instead of then reading from those files in a separate repo, I simply create a new "production" branch and add them to that.
This way they're excluded from Master and I can push that to Github or some other public repo just fine. When I'm ready to deploy, I can checkout the Production branch and merge Master into it and deploy Production.
I need to be able to do this because Heroku and other hosts require a single git repo to be pushed to their servers.
More information here:
http://groups.google.com/group/heroku/browse_thread/thread/d7b1aecb42696568/26d5249204c70574