I have an app being developed on github. I want to opensource it. Currently we use a capistrano script to deploy to our staging & production servers.
I am trying to figure out how we can put our config files in a separate repo, and still use capistrano to execute one touch deploys. The goal is that we can open up our repo for anyone to use.
You have some options.
Use environment variables: You can set environment variables at both your local and production machines. In your app you'll read these vars by doing: <%= ENV['my_var'] %>. Doing this you can commit your app to a public repo without worrying about expose sensitive information like passwords and keys. For example, set a db_password environment var to store your database password and in your database.yml you could read it doing: password: <%= ENV['db_password'] %>
You can use gems link dotenv (https://github.com/bkeepers/dotenv) or figaro (https://github.com/laserlemon/figaro): By using these gems you won't need to set environment variables manually on your machines, you will define them in a .env file instead. You'll be able to read them the same way using <%= ENV['my_var'] %>. You will have to ignore your .env files at your .gitignore and tell capistrano to create the environment variables at your production server reading from your .env file.
Another alternative would be make different config files for development and production and ignore them in your .gitignore. You can store your config production files in a different repo and have it updated on your machine at the time of your deploy. You'll just need to copy your config files from your local machine to the production server (https://coderwall.com/p/wgs6gw/copy-local-files-to-remote-server-using-capistrano-3) after updating the app repo on your production server.
The last alternative is the one I use most (in my case I use ansible in place of capistrano).
If you wanna see an example I have an application and a deployment task currently running in production that you can checkout:
Application: https://github.com/josuelima/bolao
Provisoning and Deployment: https://github.com/josuelima/ansible-sandbox
Hope I made myself clear and you got the idea.
Related
I am writing a Rails app with automated deployment using Capistrano. In the deploy.rb script I have:
set :deploy_to, '/my/deploy/path/'
And in production.rb I have:
server 'example.com', user: 'secret_user_name', roles: %w{web app db}
Currently the app is private. But assume I wanted an open source app like this. Then I wouldn't want secret_user_name and /my/deploy/path to be stored in the repo. In the Rails project if I encountered an issue like this I would store the secret values in secrets.yml and access them from there. However I don't have access to secrets.yml from Capistrano. So I could manually load the secrets file but I'm sure there is a better way to do this.
So my question is: How can I have an automated deployment process without exposing server information using Capistrano? Is there a recommended way to store secrets like there is in Rails?
If your intention is to maintain your own deployment environment (which you keep secret) but open source the code of the application itself, then I would simply move the Capistrano-related files to a separate private repository. Then you can open source the app itself, but keep the Capistrano config private.
There is no need for Capistrano's deploy.rb, etc. to live in the same directory structure or even the same repository as the app that is being deployed. After all, Capistrano deploys based on the :repo_url, which can be anything. It doesn't have to match repo where Capistrano's files are kept.
If you want to give other people (i.e. those that fork/clone the app) the ability to deploy to their own infrastructure, perhaps the easiest solution is to write up a wiki page explaining how they can set up their own Capistrano config. Deployment environments can vary widely and therefore it is probably not something you can do simply with environment variables or encrypted secrets.
In any case make sure you audit and rewrite your Git history if necessary to make sure you won't be leaking any sensitive config when you make the repo public.
Environment variables might help you.
You can put export SSH_PROD_USER=secret_user_name; in your preferred shell profile. For example if you use bash then it would be ~.bash_profile. Then use it in production.rb like this:
server 'example.com', user: ENV['SSH_PROD_USER'], roles: %w{web app db}
So basically ruby will have all your environment variables
Use something like Figaro gem: https://github.com/laserlemon/figaro
and don't push to your repo your application.yml
Or use an encrypted repo with credentials.
I'm working in a team in a Rails project.
Each of us have a local database for development purpose. We have a problem: Everyone have different configuration for the local database. When someone make a commit without reset the /config/database.yml the other members of the team can't use their database because the access is not configured.
Can I have a local configuration not commited? To each one can works without problem and without the need of re-set the file every time? Sometime like the local_settings.py in Django
You can configure the database in Rails via both the config/database.yml file and the DATABASE_URL env var. They are merged but settings from DATABASE_URL take precedence.
A good solution is to check in the config/database.yml with a basic configuration thats just enough to support postgres.app or whatever is the most common solution.
Developers that need other settings can set the DATABASE_URL env var to customize the database configuration. Gems like figaro and dotenv make this easier.
If you have ever wondered this how Heroku manages to use the correct DB no matter what you throw into database.yml and the DATABASE_URL ENV var is how you should be configuring your production DB.
Why should I use ENV vars and not more database_*.yaml files?
The twelve-factor app stores config in environment variables (often
shortened to env vars or env). Env vars are easy to change between
deploys without changing any code; unlike config files, there is
little chance of them being checked into the code repo accidentally;
and unlike custom config files, or other config mechanisms such as
Java System Properties, they are a language- and OS-agnostic standard.
https://12factor.net/config
Add config/database.yml in to the .gitignore file at root path of your rails-app.
Copy config/database.yml with the values you need for production into config/database_example.yml.
Now you can modify your local database and in production you copy config/database_expample.yml to config/database.yml
If the config file is ignored by git, everyone can change it locally without getting tracked by git.
EDIT:
HERE YOU SEE HOW YOU CAN REMOVE FILE FROM TRACKING!!!
Ignore files that have already been committed to a Git repository
I need to set up an environment variable for my rails app. Both in my local machine and in the production server. I read some tutorials on the internet but NONE has given the complete instruction on how to set and use these variable in the actual production server. I use digital ocean and linux server to host my rails app.
I have spent days trying to figure this out, but still haven't found a clear and complete instruction from setting the variables on my local machine -> push it to git repo -> set and use the variables in production server. So, hope somebody can help me here, thanks!
UPDATE:
This is how I currently setup the environment variables in my rails app by using figoro gem:
You can set system-wide environment variables in the /etc/rc.local file (which is executed when the system boots). If your Rails app is the sole user of the Linux system, that is a good place to store credentials such as API keys because there is no risk of including this file in a public Git repository, as it is outside the application directory. The secrets will only be vulnerable if the attacker gains shell access to your Linux server.
Set the environment variables within /etc/rc.local (do not include the <> characters):
export SOME_LOGIN=<username>
export SOME_PASS=<password>
To see the value of an environment variable, use one of the following commands in the Linux shell:
printenv MY_VAR
echo $MY_VAR
To access those environment variables within Rails, use the following syntax:
Inside .rb files or at the rails console
ENV['MY_VAR']
Inside .yml files:
<%= ENV['MY_VAR'] %>
For anyone still having this issue, figaro now has an easy method in setting the production variables in heroku. Just run:
$ figaro heroku:set -e production
ryzalyusoff.
For Unix
You can use LINUX ENV in rails application.
# .env
GITHUB_SECRET_KEY=SECRET
TWITTER_ACCESS_KEY=XXXXXXXXXXXX
# in rails code
puts ENV["TWITTER_ACCESS_KEY"] # => SECRET
Create .env files for local machine and your production server. Export environment variables like this(on server with ssh):
export GITHUB_SECRET_KEY="XXXXXXXXXXXXXXXXXX"
Anyway, storing keys in config - bad idea. Just add .env.example, others keys configs add to .gitignore. Goodluck.
Example with Rails
For Windows
Syntax
SET variable
SET variable=string
SET /A "variable=expression"
SET "variable="
SET /P variable=[promptString]
SET "
Key
variable : A new or existing environment variable name e.g. _num
string : A text string to assign to the variable.
expression : Arithmetic expression
Windows CMD
I believe we should not push a secret file on git.
To ignore such file use gitignore file and push other code on the git.
On the server side just copy the secret file and create a symlink for that file.
You can find demo here http://www.elabs.se/blog/57-handle-secret-credentials-in-ruby-on-rails
You can set your environment variables in production in the same way, you do it for local system. However, there are couple of gems, which make it easier to track and push to production. Have a look at figaro. This will help you in setting up and deployment of env vars.
You can do this with figaro gem
or in rails 4 there is a file named secret.yml in config folder where you can define your environment variables this file is by default in .gitignore file.For production you need to manually copy that file to server for security reason so that your sensitive information is not available to any one
First create your variable like:
MY_ENV_VAR="this is my var"
And then make it global:
export MY_ENV_VAR
You can check if the process succeeded with:
printenv
Or:
echo MY_ENV_VAR
I created a rails 4.1.4 app which I'm trying to host on heroku, but I get the following error - Missing secret_key_base for production environment, set this value in config/secrets.yml My secrets.yml file looks has secret_key_base: <%= ENV["SECRET_KEY_BASE"] %> under production
I ran rake secret and saved the result in the environment variable SECRET_KEY_BASE. When I log into my app on heroku, I can see that value stored in the Config Variables when I click on settings. Other solutions have suggested I add config/secrets.yml to git and redeploy to heroku - however I don't wish to add secrets.yml, or any yml file to version control.
Any ideas?Thanks!
When you deploy to Heroku, you're deploying via Git. Any files that aren't included in your git repository won't be pushed to your server. As a result, it doesn't matter what you have in your secrets.yml right now - it's not being deployed.
There's nothing wrong with committing configuration files or YAML files - the problem is in committing secrets. If you commit your API keys and passwords then you have to trust everyone with access to your source code. That's impossible if your code is on Github (because you don't trust everyone on earth with a computer), but is still a bad idea in a small company. You'll be far less stressed if someone leaves, loses their laptop, or gets infected with malware if they don't have production credentials on their machines.
You're already doing the right thing to avoid this. Using environment variables to configure your app keeps those secrets out of your repository, even if those configuration files are committed.
I have a number of Rails apps hosted on GitHub. They are all currently private, and I often will deploy them from their GitHub repository. I'd like to be able to make some of them open source, just like the ones you can find on http://opensourcerails.com.
My question is: How can I make these repositories public without giving away super secret credentials?
For example, I can look in /config/initializers/cookie_verification_secret.rb and see the cookie secret for nearly every one of them. I don't understand how this is acceptable. Are these users all changing these values in their deploy environments somehow?
Some users even expose their AWS secret and key! Others will instead set their AWS secret to something like:
ENV['aws-secret']
although I'm not sure at what point they're setting that value.
So, what are the best practices for open sourcing your Rails app without compromising your app's security.
I recently went through this with one of my own apps. My solution was to store anything secret in a git-ignored YAML config file, and then to access that file using a simple class in the initializers directory. The config file is stored in the 'shared' folder for the Capistrano deployment and copied to config at each deploy.
Config store: http://github.com/tsigo/jugglf/blob/master/config/initializers/juggernaut.rb
Example usage: https://github.com/tsigo/jugglf/blob/6b91baae72fbe4b1f7efa2759bb472541546f7cf/config/initializers/session_store.rb
You may also want to remove from source control all history of the file that used these secret values. Here's a guide for doing this in Git that I used: http://help.github.com/removing-sensitive-data/
If you're using foreman, put an .env file in the root of your app. (foreman docs)
.env will have
AWS_SECRET=xxx
AWS_ACCESS=yyy
Then when you need to use the keys, insert:
ENV['AWS_SECRET']
ENV['AWS_ACCESS']
Though it's important that you don't commit this .env to your version control. So if you're using git, add the .env to your .gitignore.
Bonus round! - Heroku
If deploying to Heroku, these environment variables need to be configured in the Heroku environment, too. There are two options:
Manually add the keys through the heroku config:add command
Use the heroku-config gem to synchronize your local environment variables, both ways.
Not storing any secret value at all. At any point in the history of a Git repo.
Those values should be stored elsewhere, leaving only template config files versioned, along with a script able:
to read the right values from the external repo
and build the final config file complete (with the secret values in it)
By keeping the tow set of data separate (sources on one side, secret values on the other), you can then open source the sources repo without comprising any secrets.
I actually took a hint from your question, using ENV.
I had three different secret values that I didn't want made available. They're the app's secret token of course, and Twitter's consumer key and secret. In my secret token initializer:
KinTwit::Application.config.secret_token = ENV['SECRET_TOKEN']
Twitter.consumer_key = ENV['CONSUMER_KEY']
Twitter.consumer_secret = ENV['CONSUMER_SECRET']
I'm hosting my project on Heroku, so I added these as configuration variables to Heroku.
[03:07:48] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add CONSUMER_KEY=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v7
CONSUMER_KEY => ub3rs3cr3tk3y
[03:08:40] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add CONSUMER_SECRET=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v8
CONSUMER_SECRET => ub3rs3cr3tk3y
[03:08:57] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add SECRET_TOKEN=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v9
SECRET_TOKEN => ub3rs3cr3tk3y
Now, the values are ready on my next push. But, what if you aren't using Heroku? I'm obviously not an expert on every single rails deployment (jeesh, not even a Heroku pro), but an example of this would be doing a db:migrate for testing.
$ RAILS_ENV=test rake db:migrate
The KEY=value pair before the command sets the environment variable, so running this command, echo ENV['RAILS_ENV'] would print test. So however this is set up in your environment is how you would do it. But, the environment variables aren't in your code, so that's the trick.
[EDIT - The following method has the annoyance of having to switch to the Production branch to run "rails server" in order to include necessary cookies. Thus, making edits while the server is difficult... and I'm still looking for a good solution]
After further investigation, I think the solution I was looking for was to exclude anything that stored a secret value from my Git repo's master branch (just as #VonC said). But instead of then reading from those files in a separate repo, I simply create a new "production" branch and add them to that.
This way they're excluded from Master and I can push that to Github or some other public repo just fine. When I'm ready to deploy, I can checkout the Production branch and merge Master into it and deploy Production.
I need to be able to do this because Heroku and other hosts require a single git repo to be pushed to their servers.
More information here:
http://groups.google.com/group/heroku/browse_thread/thread/d7b1aecb42696568/26d5249204c70574