I'm working in a team in a Rails project.
Each of us have a local database for development purpose. We have a problem: Everyone have different configuration for the local database. When someone make a commit without reset the /config/database.yml the other members of the team can't use their database because the access is not configured.
Can I have a local configuration not commited? To each one can works without problem and without the need of re-set the file every time? Sometime like the local_settings.py in Django
You can configure the database in Rails via both the config/database.yml file and the DATABASE_URL env var. They are merged but settings from DATABASE_URL take precedence.
A good solution is to check in the config/database.yml with a basic configuration thats just enough to support postgres.app or whatever is the most common solution.
Developers that need other settings can set the DATABASE_URL env var to customize the database configuration. Gems like figaro and dotenv make this easier.
If you have ever wondered this how Heroku manages to use the correct DB no matter what you throw into database.yml and the DATABASE_URL ENV var is how you should be configuring your production DB.
Why should I use ENV vars and not more database_*.yaml files?
The twelve-factor app stores config in environment variables (often
shortened to env vars or env). Env vars are easy to change between
deploys without changing any code; unlike config files, there is
little chance of them being checked into the code repo accidentally;
and unlike custom config files, or other config mechanisms such as
Java System Properties, they are a language- and OS-agnostic standard.
https://12factor.net/config
Add config/database.yml in to the .gitignore file at root path of your rails-app.
Copy config/database.yml with the values you need for production into config/database_example.yml.
Now you can modify your local database and in production you copy config/database_expample.yml to config/database.yml
If the config file is ignored by git, everyone can change it locally without getting tracked by git.
EDIT:
HERE YOU SEE HOW YOU CAN REMOVE FILE FROM TRACKING!!!
Ignore files that have already been committed to a Git repository
Related
Since database.yml file contains some private information, we shouldn't put it under version control.
I just dig out a bunch of solutions for sync database.yml to my server, such as this, I just want to know which is the safest way? Do I should only use scp to for that when I use Cap3 deploy?
You should just create the database.yml on your server manually when you first deploy, and then use Capistrano's linked_files directive to persist the file across deployments. I think the default capistrano configuration contains an example of using the linked_files directive for your database.yml.
Copy the database.yml during deployment from your development machine to your server to shared/db if it doesn't already exist. After deploy and before assets are compiled copy the database.yml to current/config/database.yml.
I have an app being developed on github. I want to opensource it. Currently we use a capistrano script to deploy to our staging & production servers.
I am trying to figure out how we can put our config files in a separate repo, and still use capistrano to execute one touch deploys. The goal is that we can open up our repo for anyone to use.
You have some options.
Use environment variables: You can set environment variables at both your local and production machines. In your app you'll read these vars by doing: <%= ENV['my_var'] %>. Doing this you can commit your app to a public repo without worrying about expose sensitive information like passwords and keys. For example, set a db_password environment var to store your database password and in your database.yml you could read it doing: password: <%= ENV['db_password'] %>
You can use gems link dotenv (https://github.com/bkeepers/dotenv) or figaro (https://github.com/laserlemon/figaro): By using these gems you won't need to set environment variables manually on your machines, you will define them in a .env file instead. You'll be able to read them the same way using <%= ENV['my_var'] %>. You will have to ignore your .env files at your .gitignore and tell capistrano to create the environment variables at your production server reading from your .env file.
Another alternative would be make different config files for development and production and ignore them in your .gitignore. You can store your config production files in a different repo and have it updated on your machine at the time of your deploy. You'll just need to copy your config files from your local machine to the production server (https://coderwall.com/p/wgs6gw/copy-local-files-to-remote-server-using-capistrano-3) after updating the app repo on your production server.
The last alternative is the one I use most (in my case I use ansible in place of capistrano).
If you wanna see an example I have an application and a deployment task currently running in production that you can checkout:
Application: https://github.com/josuelima/bolao
Provisoning and Deployment: https://github.com/josuelima/ansible-sandbox
Hope I made myself clear and you got the idea.
With the release of the secrets.yml file, I removed my reliance on Figaro and moved all of my keys to secrets.yml and added that file to .gitignore.
But when I tried to push to Heroku, Heroku said they needed that file in my repo in order to deploy the website. which makes sense, but I don't want my keys in git if I can avoid it.
With Figaro, I would run a rake task to deploy the keys to heroku as env variables and keep application.yml in the .gitignore. Obviously, I can't do that any more. So how do I handle this?
Secrets isn't a full solution to the environment variables problem and it's not a direct replacement for something like Figaro. Think of Secrets as an extra interface you're now supposed to use between your app and the broader world of environment variables. That's why you're now supposed to call variables by using Rails.application.secrets.your_variable instead of ENV["your_variable"].
The secrets.yml file itself is that interface and it's not meant to contain actual secrets (it's not well named). You can see this because, even in the examples from the documentation, Secrets imports environment variables for any sensitive values (e.g. the SECRET_KEY_BASE value) and it's automatically checked into source control.
So rather than trying to hack Secrets into some sort of full-flow environment variable management solution, go with the flow:
Pull anything sensitive out of secrets.yml.
Check secrets.yml into source control like they default you to.
For all sensitive values, import them from normal environment variables into secrets ERB (e.g. some_var: <%= ENV["some_var"] %>)
Manage those ENV vars as you normally would, for instance using the Figaro gem.
Send the ENV vars up to Heroku as you normally would, for instance using the Figaro gem's rake task.
The point is, it doesn't matter how you manage your ENV vars -- whether it's manually, using Figaro, a .env file, whatever... secrets.yml is just an interface that translates these ENV vars into your Rails app.
Though it adds an extra step of abstraction and some additional work, there are advantages to using this interface approach.
Whether you believe it's conceptually a good idea or not to use Secrets, it'll save you a LOT of headache to just go with the flow on this one.
PS. If you do choose to hack it, be careful with the heroku_secrets gem. As of this writing, it runs as a before_initialize in the startup sequence so your ENV vars will NOT be available to any config files in your config/environments/ directory (which is where you commonly would put them for things like Amazon S3 keys).
An equivalent for secrets.yml of that Figaro task is provided by the heroku_secrets gem, from https://github.com/alexpeattie/heroku_secrets:
gem 'heroku_secrets', github: 'alexpeattie/heroku_secrets'
This lets you run
rake heroku:secrets RAILS_ENV=production
to make the contents of secrets.yml available to heroku as environment variables.
see this link for heroku settings
if u want to run on local use like this
KEY=xyz OTHER_KEY=123 rails s
We have inherited a Rails project hosted on Heroku. The problem is that I don't understand how to get a copy of the database.yml or other various config files that have been ignored in the .gitignore file.
Sure I can look through the history but it seems like a hip shot when comparing it to what should be on the production server. Right now we are having to make changes to the staging environment which makes things a bit arduous and less efficient than having a local dev environment so we can really get under the hood. Obviously, having an exact copy of the config is a good place to start.
With other hosts it's easy to get access to all the files on the server using good ol' ftp.
This might be more easily addressed with some sort of git procedure that I am less familiar with.
Heroku stores config variables to the ENV environment
heroku config will display these list of variables
heroku config:get DATABASE_URL --app YOUR_APP
Will return the database url that Heroku as assigned to your app, using this string one can deduce the parameters necessary to connect to your heroku applications database, it follows this format
username:password#database_ip:port/database_name
This should provide you with all the values you'll need for the heroku side database.yml, its a file that is created for you each time you deploy and there is nothing it that can't be gotten from the above URL.
gitignoring the database.yml file is good practice
It's not something you can do - entries added to .gitignore means they've been excluded from source control. They've never made it into Git.
As for database.yml you can just create the file in app\config\database.yml with local settings, it should look something like;
development:
adapter: postgresql
host: 127.0.0.1
username: somelocaluser
database: somelocaldb
It's safe to say though, that if the file is not in Git then it's not on Heroku since that's the only way you can get files to Heroku. Any config is likely to be in environment variables - heroku config will show you that.
I have a number of Rails apps hosted on GitHub. They are all currently private, and I often will deploy them from their GitHub repository. I'd like to be able to make some of them open source, just like the ones you can find on http://opensourcerails.com.
My question is: How can I make these repositories public without giving away super secret credentials?
For example, I can look in /config/initializers/cookie_verification_secret.rb and see the cookie secret for nearly every one of them. I don't understand how this is acceptable. Are these users all changing these values in their deploy environments somehow?
Some users even expose their AWS secret and key! Others will instead set their AWS secret to something like:
ENV['aws-secret']
although I'm not sure at what point they're setting that value.
So, what are the best practices for open sourcing your Rails app without compromising your app's security.
I recently went through this with one of my own apps. My solution was to store anything secret in a git-ignored YAML config file, and then to access that file using a simple class in the initializers directory. The config file is stored in the 'shared' folder for the Capistrano deployment and copied to config at each deploy.
Config store: http://github.com/tsigo/jugglf/blob/master/config/initializers/juggernaut.rb
Example usage: https://github.com/tsigo/jugglf/blob/6b91baae72fbe4b1f7efa2759bb472541546f7cf/config/initializers/session_store.rb
You may also want to remove from source control all history of the file that used these secret values. Here's a guide for doing this in Git that I used: http://help.github.com/removing-sensitive-data/
If you're using foreman, put an .env file in the root of your app. (foreman docs)
.env will have
AWS_SECRET=xxx
AWS_ACCESS=yyy
Then when you need to use the keys, insert:
ENV['AWS_SECRET']
ENV['AWS_ACCESS']
Though it's important that you don't commit this .env to your version control. So if you're using git, add the .env to your .gitignore.
Bonus round! - Heroku
If deploying to Heroku, these environment variables need to be configured in the Heroku environment, too. There are two options:
Manually add the keys through the heroku config:add command
Use the heroku-config gem to synchronize your local environment variables, both ways.
Not storing any secret value at all. At any point in the history of a Git repo.
Those values should be stored elsewhere, leaving only template config files versioned, along with a script able:
to read the right values from the external repo
and build the final config file complete (with the secret values in it)
By keeping the tow set of data separate (sources on one side, secret values on the other), you can then open source the sources repo without comprising any secrets.
I actually took a hint from your question, using ENV.
I had three different secret values that I didn't want made available. They're the app's secret token of course, and Twitter's consumer key and secret. In my secret token initializer:
KinTwit::Application.config.secret_token = ENV['SECRET_TOKEN']
Twitter.consumer_key = ENV['CONSUMER_KEY']
Twitter.consumer_secret = ENV['CONSUMER_SECRET']
I'm hosting my project on Heroku, so I added these as configuration variables to Heroku.
[03:07:48] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add CONSUMER_KEY=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v7
CONSUMER_KEY => ub3rs3cr3tk3y
[03:08:40] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add CONSUMER_SECRET=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v8
CONSUMER_SECRET => ub3rs3cr3tk3y
[03:08:57] [william#enterprise ~/dev/rwc/kintwit]$ heroku config:add SECRET_TOKEN=ub3rs3cr3tk3y
Adding config vars and restarting app... done, v9
SECRET_TOKEN => ub3rs3cr3tk3y
Now, the values are ready on my next push. But, what if you aren't using Heroku? I'm obviously not an expert on every single rails deployment (jeesh, not even a Heroku pro), but an example of this would be doing a db:migrate for testing.
$ RAILS_ENV=test rake db:migrate
The KEY=value pair before the command sets the environment variable, so running this command, echo ENV['RAILS_ENV'] would print test. So however this is set up in your environment is how you would do it. But, the environment variables aren't in your code, so that's the trick.
[EDIT - The following method has the annoyance of having to switch to the Production branch to run "rails server" in order to include necessary cookies. Thus, making edits while the server is difficult... and I'm still looking for a good solution]
After further investigation, I think the solution I was looking for was to exclude anything that stored a secret value from my Git repo's master branch (just as #VonC said). But instead of then reading from those files in a separate repo, I simply create a new "production" branch and add them to that.
This way they're excluded from Master and I can push that to Github or some other public repo just fine. When I'm ready to deploy, I can checkout the Production branch and merge Master into it and deploy Production.
I need to be able to do this because Heroku and other hosts require a single git repo to be pushed to their servers.
More information here:
http://groups.google.com/group/heroku/browse_thread/thread/d7b1aecb42696568/26d5249204c70574