I´m using the gem 'paypal-sdk-adaptivepayments' to integrate Paypal in my Rails app. The configuration file is paypal.yml:
development:
# Credentials for Classic APIs
username: ENV["PAYPAL_CLASSIC_USERNAME_DEV"]
password: ENV["PAYPAL_CLASSIC_PASSWORD_DEV"]
signature: ENV["PAYPAL_CLASSIC_SIGNATURE_DEV"]
app_id: ENV["PAYPAL_CLASSIC_APP_ID_DEV"]
http_timeout: 30
# Mode can be 'live' or 'sandbox'
mode: sandbox
test:
<<: *default
production:
<<: *default
#mode: live
Cause this information is secret I want to use another gem called 'Figaro' that externalize this variables. I used this for another configuration files in my app but it doesn´t works with 'paypal.yml'. I know that doesn´t works because when I put the real information in the paypal.yml file it works.
development:
# Credentials for Classic APIs
username: *******#yahoo.com
password: *******
signature: ******
app_id: ******
http_timeout: 30
# Mode can be 'live' or 'sandbox'
mode: sandbox
test:
<<: *default
production:
<<: *default
#mode: live
Has anyone used Figaro with this file? Is there any other option to "secretize" this information in Rails?
Thanks in advance!
Okay, what you should do is replace your yaml config file with a config.rb file:
#config/initializers/paypal.rb
PayPal::SDK.configure(
username: ENV["PAYPAL_CLASSIC_USERNAME_DEV"],
password: ENV["PAYPAL_CLASSIC_PASSWORD_DEV"],
signature: ENV["PAYPAL_CLASSIC_SIGNATURE_DEV"],
app_id: ENV["PAYPAL_CLASSIC_APP_ID_DEV"],
http_timeout: 30
)
Then define these variables in a yaml file such as application.yml:
#config/application.yml
PAYPAL_CLASSIC_USERNAME_DEV: yourpaypalusername
PAYPAL_CLASSIC_PASSWORD_DEV: yourpaypalpassword
PAYPAL_CLASSIC_SIGNATURE_DEV: yourpaypaysignature
PAYPAL_CLASSIC_APP_ID_DEV: yourappid
Lastly assuming you are using git for your version control, go into your gitignore and tell it to ignore the yaml file with your secret info:
#.git_ignore
/config/application.yml
This will properly keep your environment variables out of your code base when you push it to github or other git repo.
Edit: To use different environment keys you merely specify the keys for different environments in your yaml file:
#application.yml
production:
PAYPAL_CLASSIC_USERNAME_DEV: yourpaypalusername
PAYPAL_CLASSIC_PASSWORD_DEV: yourpaypalpassword
PAYPAL_CLASSIC_SIGNATURE_DEV: yourpaypaysignature
PAYPAL_CLASSIC_APP_ID_DEV: yourappid
That being said, I highly suspect what you are doing is a violation of 12factor best practices. This yaml file is not a part of your application - when you push your code up to a server like Heroku, this file is not supposed to go with it. The yaml file with your keys should only ever exist on your local machine. It should not exist in Github, nor on your production server. To read more about this, check out this link: http://12factor.net/config
So how do you tell a server like Heroku what these config variables are? This will vary based on what service provider you use, but is typically done via the command line tools or through an admin management page. e.g., for Heroku you would run:
heroku config:set PAYPAL_CLASSIC_USERNAME_DEV=putyourusernamehere
Related
I want to get secret informations from other place beside rails application and the host. I made a global variable file under config/initializers to get the keys. Then set them in the database connection file. Like this:
config/initializers/get_secrets.rb
##username = 'A'
##password = 'B'
config/database.yml
development:
adapter: mysql2
encoding: utf8
database: my_db_name
username: <%= ##username %>
password: <%= ##password %>
host: 127.0.0.1
port: 3306
When ran rails console, got:
./config/initializers/get_secrets.rb:1: warning: class variable access from toplevel
(erb):10: warning: class variable access from toplevel
>
It seems not a good usage in this case. Then is there a right way?
In Rails 5.2+, we have a master.key file and a credentials YAML file. The master.key file encrypts your credentials (so never show anyone this key except your server).
You can activate this by doing:
EDITOR=vim rails credentials:edit
The file:
aws:
key: 123
google_key: 456
secret_key_base: f230ffddd1d1f151j3bchjas9a292j221...
If you want to use a credential, it looks like this:
Rails.application.credentials.google_key
or for nested resources like the AWS one:
Rails.application.credentials.aws[:key]
But if you're on Rails 5.1 or less, you don't have this ability. So, you do it in a different way:
Environment variables. Your program environment can always export an ENV and retrieve it. I always do this for simple nuisances like test email passwords:
export GMAIL_PASSWORD="1234" and its usage, ENV['GMAIL_PASSWORD']
Secrets.yml. This is a file that basically predates master.key + credentials by functioning as an in between credential handler for your environments. It's janky and often erroneously committed to repos by accident, but we do what we can:
development:
secret_key_base: 1112sdasae1d13d1tfd1d3...
GMAIL_PASSWORD: '1234'
test:
secret_key_base: _your_secret_ as above
production:
secret_key_base: <%= secure_token %>
And it is accessed like Rails.application.secrets.GMAIL_PASSWORD
If you cannot access your variables, it may be because the secrets.yml file is not being loading in or you are in the wrong environment. Notice those vars are nested in particular environments.
I would like to deploy my application via Dokku to a VPS.
Putting together the dokku-postgres documentation and the relative, scarce internet documentation on the matter (one at GitHub), it seems necessary to configure database.yml to use the url environment variable url: <%= ENV['DATABASE_URL'] %>
Since I could not find any other sources of information, I wonder how database.yml should be configured, and how Rails will connect to the postgres service created with Dokku.
For instance, taken for granted that linking url to the DATABASE_URL variable is necessary, will this be enough to establish a connection between my Rails application and the postgres service or would it be still necessary to use a username and a password? In the latter case, what username and password am I expected to use?
Below is how at present my database.yml looks like.
default: &default
adapter: postgresql
encoding: unicode
pool: 5
username: asarluhi
development:
<<: *default
database: fireworks_app_development
test:
<<: *default
database: fireworks_app_test
production:
<<: *default
database: fireworks_app_production
pool: 25
username: fireworks_app
password: <%= ENV['FIREWORKS_APP_DATABASE_PASSWORD'] %>
This file was created as it is (apart from a higher pool size for production) when I created the application. How would you suggest to edit the production section?
The dokku-postgres documentation states that the following (and nothing else) will be set on the linked application by default:
DATABASE_URL=postgres://postgres:SOME_PASSWORD#dokku-postgres-lolipop:5432/lolipop
In place of the lollipop postgres service example, I would use fireworks_app_production to match the name of the database in database.yml
Are username and password still necessary after pointing url to the DATABASE_URL variable? Am I expected to add or remove anything else?
You don't need to worry about the database.yml with dokku, just upload your app to the server, let's use "fireworks" as the name for example on this.
when you upload the first time the app, this is created automatically so you don't need to create it.
then you install the postgres plugin and run
# that will create the container for the database
$ dokku postgres:create fireworks
# and then you link both, the app with the database
$ dokku postgres:link fireworks fireworks
you don't have to worry about anything else, with that dokku will connect this
then you just have to run db:migrate and everything is ready to work!
I've been searching the web, and I can't find any good/recent examples of what to exclude from a new public rails app. I'm looking to open source my app on GitHub and am wondering what types of data should be removed from source control.
From what I can tell, there should be a config/config.yml file that has private information. I've been looking through the other files, and it looks like config/database.yml, config/intializers/secret_token.rb and config/initializers/session_store.rb should also be excluded?
Is it best practice to exclude all of these files separately? Or is there a way to have the info all defined in config/config.yml and be called in each of those files? Additionally, what files and data should be kept private and hidden? Is that all of them?
I'm just wondering what approach I should take and what is the best practice. Thanks for any help!
I've been looking into this recently as well; I wanted to keep sensitive information hidden throughout the process of pushing open source code to Github, then automatically pushed to Travis CI for testing, then from Travis being automatically deployed to Heroku. Here are all the details of what I've found so far looking at various StackOverflow Q&As, blogs etc, which will hopefully serve as a reference for you, even if only for config inside the Rails app (omit any {{ ... }} you see)
Disclaimer: I'm by no means an expert here, so please keep in mind there are likely better ways to do this than what I'm trying. I'd love to be able to learn some new tricks in this Q&A thread.
Inside the Rails App
I currently use the Figaro gem to hide sensitive information in ENV environment variables. In my (.gitignored) config/application.yml, I keep the following information:
# App keys
SECRET_TOKEN: # your rake secret generated token
development:
DB_NAME: # your dev db name here
DB_USER: # your dev db username here
DB_PASSWORD: # your dev db password here
test:
DB_NAME: # your test db name here
DB_USER: # your test db username here
DB_PASSWORD: # your test db password here
production:
DB_NAME: # your prod db name here
DB_USER: # your prod db username here
DB_PASSWORD: # your prod db password here
# Third Party keys that you will reference in their relevant files
THIRD_PARTY_API_OR_LICENSE_KEY: # list of whatever api/license keys you use
(DB_NAME, DB_USER, and DB_PASSWORD will be used dynamically depending on what environment your app is running in).
An empty version of the above file (config/application.example.yml) gets pushed to Github with some instructions on how to fill it in.
The files that are pushed to Github and reference these variables look like this:
config/database.yml
(Postgresql is used here, but you should be able to change the settings for whatever database you use)
postgresql: &postgresql
adapter: postgresql
database: <%= ENV['DB_NAME'] %>
username: <%= ENV['DB_USER'] %>
password: <%= ENV['DB_PASSWORD'] %>
min_messages: ERROR
defaults: &defaults
pool: 5
timeout: 5000
host: localhost
<<: *<%= ENV['DB'] || "postgresql" %>
development:
<<: *defaults
test:
<<: *defaults
production:
<<: *defaults
config/initializers/secret_token.rb
if Rails.env.production? && ENV['SECRET_TOKEN'].blank?
raise 'SECRET_TOKEN environment variable must be set!'
end
YourApp::Application.config.secret_token =
ENV['SECRET_TOKEN'] || {{WHATEVER_SECRET_TOKEN_RAILS_GENERATED_BY_DEFAULT}}
(Plus, whatever files would be referencing THIRD_PARTY_API_OR_LICENSE_KEY-type keys.)
Testing on Travis CI
Create encrypted travis variables using the Travis gem. The Heroku API key and Heroku Git URL are needed if you deploy direct to Heroku from a Travis worker (see this StackOverflow Q&A for details), otherwise you can omit them if you just use it for testing:
$ gem install travis
$ travis encrypt your_username/your_repo HEROKU_API_KEY={{YOUR_HEROKU_API_KEY}}
$ travis encrypt HEROKU_GIT_URL={{YOUR_HEROKU_GIT_URL}} # eg git#heroku.com:your_app.git
$ travis encrypt DB_NAME={{YOUR_DB_NAME_UNDER_TEST}} # eg your_app_test
$ travis encrypt DB_USER={{YOUR_DB_USER_UNDER_TEST}}
$ travis encrypt DB_PASSWORD={{YOUR_DB_PASSWORD_UNDER_TEST}}
(Plus, encrypt any other keys you may need during testing, if any...)
Then add them to .travis.yml
(once again Postgresql-focused, but you should be able to change the settings for whatever database you use)
env:
global:
- secure: {{YOUR_ENCRYPTED_HEROKU_API_KEY}}
- secure: {{YOUR_ENCRYPTED_HEROKU_GIT_URL}}
- secure: {{YOUR_ENCRYPTED_DB_NAME}}
- secure: {{YOUR_ENCRYPTED_DB_USER}}
- secure: {{YOUR_ENCRYPTED_DB_PASSWORD}}
matrix:
- DB: postgresql
before_script:
- psql -c "create database $DB_NAME;" -U $DB_USER
- RAILS_ENV=test bundle exec rake db:migrate
script:
- bundle exec rspec spec/
after_success:
- gem install heroku
- git remote add heroku $HEROKU_GIT_URL
# ... see link above for the rest of the config content
Multiple variables marked with the same name of secure are fine; they'll show up in the config as HEROKU_API_KEY=[secure] HEROKU_GIT_URL=[secure] etc.
Deployment to Heroku
Use the Figaro's Heroku rake task to automatically set the environment variables that Heroku needs to see in production:
$ rake figaro:heroku
Or, set them manually:
$ heroku config:set SECRET_TOKEN={{YOUR_SECRET_TOKEN}}
$ heroku config:set DB_NAME={{YOUR_DB_NAME_UNDER_PRODUCTION}} # eg your_app_production
$ heroku config:set DB_USER={{YOUR_DB_USER_UNDER_PRODUCTION}}
$ heroku config:set DB_PASSWORD={{YOUR_DB_PASSWORD_UNDER_PRODUCTION}}
$ heroku config:set THIRD_PARTY_API_OR_LICENSE_KEY={{YOUR_THIRD_PARTY_API_OR_LICENSE_KEY}}
Then, attempt a deployment.
That's all I have for now. Not sure at the moment if I should be hiding more info or if I'm not hiding it well enough, but it's a work in progress.
You'll get different opinions. IMHO, it's best-practice to include these files, but omit the secret content from them. Document what you're doing so developers who are new to your project know what they need to fill in.
Phusion has a good blog post about how to handle the Rails session secret, and the tradeoffs you can make to include or exclude information:
http://blog.phusion.nl/2013/01/04/securing-the-rails-session-secret/#.URYPXekTMak
My favorite way to document these is using a "rake setup" task. You can have the task print what the developer needs to do-- in other words, you don't need to automate it all (though that's nice if you're able to do it).
If you want to get fancy, have your files read the secret settings from a shared/ directory, which also enables deployment symlinking. This is described in the Phusion blog too. This is how I build apps that need to be deployed frequently.
I have a rails application which needs to connect to two different databases. These database definitions are both in the database.yml file in my repository, with the format:
production:
adapter: ...
database: ...
username: ...
password: ...
secondary-production:
adapter: ...
database: ...
username: ...
password: ...
I have that format for production, staging, and development databases. When running cap deploy or cap production deploy, I need to make both database definitions come through.
I wasn't the person who set up capistrano on this app, and I've never used it before, so I'm finding it hard to work out how to do this. Could anyone point me to a relevant piece of documentation or let me know what file and syntax I need to use for it?
I found the answer.
There was a database.yml.erb file in the /config/recipes/templates folder that I had to update.
Let's say I have s different database connections...
Right now for the purposes of this example let's only be concerned with development. When I run 'rake db:setup' it only creates development because from the rake task perspective the other connection is another environment. I could pass in the other connection as the RAILS_ENV to create the database. However, the problem comes in with how we've defined our database connections. I'd rather not undo this as it makes management and deployment much easier. What we've done is this...
In our database.yml we've added the following code.
databases_file = File.join(Rails.root.to_s, "config", "databases", "#{Rails.env.to_s}.yml")
IO.read(databases_file) if File.exist?(databases_file)
Then in a databases sub-folder under config we've created the different environment yml files.
So in development.yml we have...
development:
... details ...
logging:
... details ...
Now if I set RAILS_ENV to 'logging' it will never load the file since the file is called development.yml. Thus it will never find the logging connection.
If I consolidate all these settings back into the database.yml file then we're back to not checking in the database.yml into the git repo and having to manually create it on the development machines for new hires and such. We'd also have to change our deploy process to start putting the file in place on the lower environments. (Our production process already supports this)
You could use environment variables to manage these. You could have a single database.yml with things defined such as:
development:
database: ENV['DEVELOPMENT_DATABASE']
And then set the environment variables in your .bashrc/.zshrc.
We have multiple databases and we use branches heavily... so I came up with this solution to create multiple databases and tie their names to the branch being worked on...
# config/database.yml
<%=
databases_file = File.join(Rails.root.to_s, "config", "databases", "#{Rails.env.to_s}.yml")
if Rails.env.development? || Rails.env.test?
branch = ENV['BRANCH'] || `git branch --no-color 2> /dev/null`.chomp.split("\n").grep(/^[*]/).first[/(\S+)$/,1].sub("-", "_")
puts "Using databases for #{branch}"
IO.read(databases_file).gsub!("<branch>", branch) if File.exist?(databases_file)
else
IO.read(databases_file) if File.exist?(databases_file)
end
%>
And then you create a database file for each environment. In the development.yml and test.yml you use the token to let this script look for what it needs to in order to do replacement. So an example file would look like this.
# config/databases/development.yml
development:
adapter: mysql2
host: 127.0.0.1
database: <branch>_dev
username: user
password: pass
versions:
adapter: mysql2
host: 127.0.0.1
database: <branch>_versions
username: user
password: pass
reporting:
adapter: mysql2
host: 127.0.0.1
database: <branch>_reporting
username: user
password: pass
ods:
adapter: mysql2
host: 127.0.0.1
database: <branch>_ods
username: user
password: pass
I then extended the rake tasks to handle multiple databases
This is where most of the work went in. Therefore, I'm going to leave it out of the answer and make you work for it! No actually, it's just a big large mess that I'm not proud of and would love time to fix it, but haven't found any. I don't want to lead anyone down the wrong path but if you have questions just get me a message and I'll be happy to help.