I am deploying my first little app with MongoDB and Mongoid as a driver.
What is the right secure way to use MongoDB in production?
I mean in the development I have just started mongod and that's it - no username or password needed and that looks unsecure.
Also Mongoid sets default configurations
production:
host: <%= ENV['MONGOID_HOST'] %>
port: <%= ENV['MONGOID_PORT'] %>
username: <%= ENV['MONGOID_USERNAME'] %>
password: <%= ENV['MONGOID_PASSWORD'] %>
database: <%= ENV['MONGOID_DATABASE'] %>
How should I configure this options and entire MongoDB on my production server?
To create a production environment where you need to use a username and password to connect:
In the mongo console:
// Add an Admin User (to the admin db)
use admin
db.addUser("theadmin", "anadminpassword")
// Use your database
use supercool
// Add a user (to your database)
db.addUser("joe", "passwordForJoe")
// show all users:
db.system.users.find()
// add readonly user (kinda cool)
db.addUser("readonly", "passwordForJoe", true)
Now, all connections to your mongodb will require authentication -- http://www.mongodb.org/display/DOCS/Security+and+Authentication
Also: you can consider using your linux firewall to only allow 27017 from your web server(s).
MongoDB by default doesn't support authentication. This is by design and is expected to be handled by individual applications. But it is not too hard to enable authenticated access to MongoDB. I will describe the steps I took for my typical rails, mongoid, git, capistrano based setup.
First add a user to the admin database. Without which none of the below steps work.
use admin
db.addUser("heisenberg", "knock-knock")
Create a user to the db your application will use. In MongoDB authentication works on a per db level
use breaking_bad
db.addUser("gus", "fring")
Better yet, create a user for just read-only purposes for security and performance benefits
use breaking_bad
db.addUser("walter", "white", true)
Enable the auth flag for mongodb to respect all your authentication related hardwork. This can be done either through a --auth flag to the mongodb command. Or better uncomment this line in the /etc/mongodb.conf
auth = true #Uncomment me
Now restart your mongodb process to pickup the new changes.
service mongodb restart
Check if you are on the right track by ensuring that your CRUD application now fails! It lost access to read/write from your mongodb afterall. Now add the username: and password: attributes to your mongoid.yml under the default group.
production:
sessions:
default:
database: breaking_bad
hosts:
- albuquerque.com:27017
username: gus
password: fring
For bonus points, remove the mongoid.yml file from the git repository as this file now has security credentials
git rm mongoid.yml
Add capistrano tasks that copy the mongoid.yml file from your dev machine to your server and add appropriate symlinks. Run cap deploy after this
namespace :mongoid do
desc "Copy mongoid config"
task :copy do
upload "config/mongoid.yml", "#{shared_path}/mongoid.yml", :via => :scp
end
desc "Link the mongoid config in the release_path"
task :symlink do
run "test -f #{release_path}/config/mongoid.yml || ln -s #{shared_path}/mongoid.yml #{release_path}/config/mongoid.yml"
end
end
Use the bind_ip setting in your /etc/mongodb.conf to tell MongoDB to only accept connections from your webserver
Use iptables to setup firewall settings to further secure your setup. Or use it within a VPN.
Further reading:
http://docs.mongodb.org/manual/tutorial/control-access-to-mongodb-with-authentication/
http://docs.mongodb.org/manual/administration/security/
Related
I am using Rails 5.2.3 with PostGIS 2.5 running on Postgres 11, set up in a Docker Compose environment. I've set the following in config/application.rb:
config.active_record.schema_format = :sql
So that special features in the Schema can get dumped. However, doing so during db:migrate, I get the following error:
pg_dump: [archiver (db)] connection to database "development" failed: FATAL: password authentication failed for user "root"
The command that gets executed is:
pg_dump -s -x -O -f /var/www/dev/db/structure.sql -T geography_columns -T geometry_columns -T layer -T raster_columns -T raster_overviews -T spatial_ref_sys -T topology development
The reason for the error is that there is no root user here; in fact, I am using a different username in my database config:
default: &default
adapter: postgis
encoding: unicode
host: <%= ENV['DATABASE_HOST'] %>
user: <%= ENV['DATABASE_USER'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
# For details on connection pooling, see Rails configuration guide
# http://guides.rubyonrails.org/configuring.html#database-pooling
pool: <%= ENV.fetch("DATABASE_POOL") { 10 } %>
The user is not root, and everything works fine from the Rails app itself, which can connect to the database using the user specified in the environment variable.
It just seems that pg_dump, when called from Rails to dump the schema, is not using the proper username from the config, but instead falls back to root.
I see some configuration environment variables in the source code, but how should I set them and why does Rails not do that by default?
I think it works when when, in my docker-compose.yml file I create an additional environment variable PGUSER that has the same value as DATABASE_USER. But that seems like workaround — is this a bug I'm seeing or some expected behavior?
The only solution I found so far was to specify an additional environment variable named PGUSER for the Rails container.
I assume this is a bug or not properly documented.
Apparently when using the Octopus gem to do Postgres replication everything should be plug and play. However I can't seem to find what I'm doing wrong.
This is my config/shards.yml
octopus:
environments:
- development
replicated: true
development:
slave1:
adapter: postgresql
host: localhost
database: slaveapp_development
username: pguser
password: pgpass
The AR model Provider(I create the exact same tables in each app via Rake tasks) I'd like to sync/replicate to my slave:
class Provider < ActiveRecord::Base
has_many :products
replicated_model()
end
I boot both apps via Rails server and enter Masterapp's console and from there:
> Provider.using(:slave1).create({provider_params...})
#=> works! I get a new record in slave1's DB.
> Provider.using(:master).create({provider_params...})
#=> works partly. Creates record only in master's DB.
The problem is that when calling Provider.using(:master)... I'm expecting:
1 - Create record at master's DB.
2 - Replicate same record at slave1's DB. <--- This is NOT happening.
That is not the purpose of the Octopus gem.
It redirects the database requests, depending on whether it is a read or write operation. That way you can use your Rails models without thinking about the current database connection and if it fits the intended operation.
It does not copy the data to the slaves.
To perform the actual replication, i.e. the data transfer from the master to the slaves, you have to set it up yourself.
There are several options with PostgreSQL. If you are using a newer version (9.1+) you can use the integrated streaming replication in "hot standby" mode. There are tutorials on how to set it up, e.g.
http://www.rassoc.com/gregr/weblog/2013/02/16/zero-to-postgresql-streaming-replication-in-10-mins/
http://prongs.org/blog/postgresql-replication
If you are stuck with an older version of PostgreSQL have a look at the alternatives.
Try adding fully_replicated: false after replicated: true in your config.
As you know, you MUST provide the correct database name, username, and password for the database in the config/database.yml file, or your Rails app will refuse to work.
In the default setup, your password is in plain text in the config/database.yml file. If your app is on a free GitHub repository, then your password is public information. This is not a viable option for a serious app. (It's OK for a tutorial exercise, provided that you don't use this password for anything else.)
I have a solution that has worked for me so far, but I'm wondering if there is something better. You can see my deployed example at https://github.com/jhsu802701/bsf .
What I do is set up the config/database.yml file to provide the username and password for the development environment programatically. For the development environment, I add commands to the config/database.yml script to acquire the development environment username (which is my regular username for the Debian Linux setup I use) and a blank password. (I give my username Postgres superuser privileges.) For the production environment, I add a command in the deployment script that acquires the username and password from files elsewhere on my account and writes this information to the config/database.yml file.
Is there a better solution?
Is there a Ruby gem that covers this? If not, I'm thinking of creating one.
The way that heroku does it, and a vast majority of other rails shops are with ENV variables
Export two variables to your environment,
export POSTGRES_USERNAME='username'
export POSTGRES_PASSWORD='password'
then in your database.yml file you can do
username: <%= ENV['POSTGRES_USERNAME'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
This is how I make it work:
On terminal/cmd:
heroku config:set YOUR_DATABASE_PASSWORD=passywordy
Then, in /config/database.yml file;
production:
<<: *default
password: <%= ENV['YOUR_DATABASE_PASSWORD'] %>
(this password area is automatically generated when I used rails new my_app -d postgresql)
On other than heroku export you variables to system environment (linux) by typing in bash
export KEY=value
Then you can call it in Rails by ENV['KEY']
e.g.
in bash:
export CMS_DATABASE_PASSWORD=MySecurePassword
in secrets.yml:
password: <%= ENV['CMS_DATABASE_PASSWORD'] %>
Setting the environment variables as described in existing posts above, will only persist the environment variables for the duration of the current shell session.
To set the environment variables permanently, the export instruction(s) should be added to your shell config file. (Then run source ~/.bashrc to apply the changes to your current session)
TL;DR: If you're using BASH, add the export instruction(s) to ~/.bashrc.
While the above should suffice (if using BASH on most popular Linux distros), confidently identifying which config file to update for your shell can be quite tricky. The following post explains the reasons why and provides guidance on which config file to edit.
https://unix.stackexchange.com/questions/117467/how-to-permanently-set-environmental-variables
I've been searching the web, and I can't find any good/recent examples of what to exclude from a new public rails app. I'm looking to open source my app on GitHub and am wondering what types of data should be removed from source control.
From what I can tell, there should be a config/config.yml file that has private information. I've been looking through the other files, and it looks like config/database.yml, config/intializers/secret_token.rb and config/initializers/session_store.rb should also be excluded?
Is it best practice to exclude all of these files separately? Or is there a way to have the info all defined in config/config.yml and be called in each of those files? Additionally, what files and data should be kept private and hidden? Is that all of them?
I'm just wondering what approach I should take and what is the best practice. Thanks for any help!
I've been looking into this recently as well; I wanted to keep sensitive information hidden throughout the process of pushing open source code to Github, then automatically pushed to Travis CI for testing, then from Travis being automatically deployed to Heroku. Here are all the details of what I've found so far looking at various StackOverflow Q&As, blogs etc, which will hopefully serve as a reference for you, even if only for config inside the Rails app (omit any {{ ... }} you see)
Disclaimer: I'm by no means an expert here, so please keep in mind there are likely better ways to do this than what I'm trying. I'd love to be able to learn some new tricks in this Q&A thread.
Inside the Rails App
I currently use the Figaro gem to hide sensitive information in ENV environment variables. In my (.gitignored) config/application.yml, I keep the following information:
# App keys
SECRET_TOKEN: # your rake secret generated token
development:
DB_NAME: # your dev db name here
DB_USER: # your dev db username here
DB_PASSWORD: # your dev db password here
test:
DB_NAME: # your test db name here
DB_USER: # your test db username here
DB_PASSWORD: # your test db password here
production:
DB_NAME: # your prod db name here
DB_USER: # your prod db username here
DB_PASSWORD: # your prod db password here
# Third Party keys that you will reference in their relevant files
THIRD_PARTY_API_OR_LICENSE_KEY: # list of whatever api/license keys you use
(DB_NAME, DB_USER, and DB_PASSWORD will be used dynamically depending on what environment your app is running in).
An empty version of the above file (config/application.example.yml) gets pushed to Github with some instructions on how to fill it in.
The files that are pushed to Github and reference these variables look like this:
config/database.yml
(Postgresql is used here, but you should be able to change the settings for whatever database you use)
postgresql: &postgresql
adapter: postgresql
database: <%= ENV['DB_NAME'] %>
username: <%= ENV['DB_USER'] %>
password: <%= ENV['DB_PASSWORD'] %>
min_messages: ERROR
defaults: &defaults
pool: 5
timeout: 5000
host: localhost
<<: *<%= ENV['DB'] || "postgresql" %>
development:
<<: *defaults
test:
<<: *defaults
production:
<<: *defaults
config/initializers/secret_token.rb
if Rails.env.production? && ENV['SECRET_TOKEN'].blank?
raise 'SECRET_TOKEN environment variable must be set!'
end
YourApp::Application.config.secret_token =
ENV['SECRET_TOKEN'] || {{WHATEVER_SECRET_TOKEN_RAILS_GENERATED_BY_DEFAULT}}
(Plus, whatever files would be referencing THIRD_PARTY_API_OR_LICENSE_KEY-type keys.)
Testing on Travis CI
Create encrypted travis variables using the Travis gem. The Heroku API key and Heroku Git URL are needed if you deploy direct to Heroku from a Travis worker (see this StackOverflow Q&A for details), otherwise you can omit them if you just use it for testing:
$ gem install travis
$ travis encrypt your_username/your_repo HEROKU_API_KEY={{YOUR_HEROKU_API_KEY}}
$ travis encrypt HEROKU_GIT_URL={{YOUR_HEROKU_GIT_URL}} # eg git#heroku.com:your_app.git
$ travis encrypt DB_NAME={{YOUR_DB_NAME_UNDER_TEST}} # eg your_app_test
$ travis encrypt DB_USER={{YOUR_DB_USER_UNDER_TEST}}
$ travis encrypt DB_PASSWORD={{YOUR_DB_PASSWORD_UNDER_TEST}}
(Plus, encrypt any other keys you may need during testing, if any...)
Then add them to .travis.yml
(once again Postgresql-focused, but you should be able to change the settings for whatever database you use)
env:
global:
- secure: {{YOUR_ENCRYPTED_HEROKU_API_KEY}}
- secure: {{YOUR_ENCRYPTED_HEROKU_GIT_URL}}
- secure: {{YOUR_ENCRYPTED_DB_NAME}}
- secure: {{YOUR_ENCRYPTED_DB_USER}}
- secure: {{YOUR_ENCRYPTED_DB_PASSWORD}}
matrix:
- DB: postgresql
before_script:
- psql -c "create database $DB_NAME;" -U $DB_USER
- RAILS_ENV=test bundle exec rake db:migrate
script:
- bundle exec rspec spec/
after_success:
- gem install heroku
- git remote add heroku $HEROKU_GIT_URL
# ... see link above for the rest of the config content
Multiple variables marked with the same name of secure are fine; they'll show up in the config as HEROKU_API_KEY=[secure] HEROKU_GIT_URL=[secure] etc.
Deployment to Heroku
Use the Figaro's Heroku rake task to automatically set the environment variables that Heroku needs to see in production:
$ rake figaro:heroku
Or, set them manually:
$ heroku config:set SECRET_TOKEN={{YOUR_SECRET_TOKEN}}
$ heroku config:set DB_NAME={{YOUR_DB_NAME_UNDER_PRODUCTION}} # eg your_app_production
$ heroku config:set DB_USER={{YOUR_DB_USER_UNDER_PRODUCTION}}
$ heroku config:set DB_PASSWORD={{YOUR_DB_PASSWORD_UNDER_PRODUCTION}}
$ heroku config:set THIRD_PARTY_API_OR_LICENSE_KEY={{YOUR_THIRD_PARTY_API_OR_LICENSE_KEY}}
Then, attempt a deployment.
That's all I have for now. Not sure at the moment if I should be hiding more info or if I'm not hiding it well enough, but it's a work in progress.
You'll get different opinions. IMHO, it's best-practice to include these files, but omit the secret content from them. Document what you're doing so developers who are new to your project know what they need to fill in.
Phusion has a good blog post about how to handle the Rails session secret, and the tradeoffs you can make to include or exclude information:
http://blog.phusion.nl/2013/01/04/securing-the-rails-session-secret/#.URYPXekTMak
My favorite way to document these is using a "rake setup" task. You can have the task print what the developer needs to do-- in other words, you don't need to automate it all (though that's nice if you're able to do it).
If you want to get fancy, have your files read the secret settings from a shared/ directory, which also enables deployment symlinking. This is described in the Phusion blog too. This is how I build apps that need to be deployed frequently.
basically I have a mongodb instance running and working on ec2. On the side I have a rails 3.2 app with mongoid as orm working on local. What I want to do next is try to connect my rails app to the mongodb instance using mongoid. Also, intending to host my rails app on Dotcloud later
Ran the code rails g mongoid:config to generate the mongoid.yml file with the following code:
development:
host: localhost
database: mongotest_development
test:
host: localhost
database: mongotest_test
set these environment variables on your prod server
production:
host: <%= ENV['MONGOID_HOST'] %>
port: <%= ENV['MONGOID_PORT'] %>
username: <%= ENV['MONGOID_USERNAME'] %>
password: <%= ENV['MONGOID_PASSWORD'] %>
database: <%= ENV['MONGOID_DATABASE'] %>
# slaves:
# - host: slave1.local
# port: 27018
# - host: slave2.local
# port: 27019
From here onwards, I don't think I have a clear picture of how all this is going to work. But I did some trial and error. Firstly I wanted to try connecting to the mongodb instance on development, so I commented out the mongoid.yml defaults and added the following:
development:
host: <public dns of the mongodb instance>
port: 27017
# username:
# password:
database: <I ssh into the instance and created a database>
I commented the username and password out partly because I am not sure what to put, and partly because when I inspect the mongod.conf file on ec2, I saw that by default :auth is false, so I assume authentication is not required. So I ran rails console and got the following error:
Failed to connect to a master node at <public dns of the mongodb instance>:27017 (Mongo::ConnectionFailure)
from /Users/Kinglee/.rvm/gems/ruby-1.9.2-p180#rails3tutorial/gems/mongo-1.6.2/lib/mongo/connection.rb:589:in `setup'
from /Users/Kinglee/.rvm/gems/ruby-1.9.2-p180#rails3tutorial/gems/mongo-1.6.2/lib/mongo/connection.rb:114:in `initialize'
from /Users/Kinglee/.rvm/gems/ruby-1.9.2-p180#rails3tutorial/gems/mongo-1.6.2/lib/mongo/connection.rb:165:in `new'
from /Users/Kinglee/.rvm/gems/ruby-1.9.2-p180#rails3tutorial/gems/mongo-1.6.2/lib/mongo/connection.rb:165:in `from_uri'
from /Users/Kinglee/.rvm/gems/ruby-1.9.2-p180#rails3tutorial/gems/mongoid-2.4.10/lib/mongoid/config/database.rb:86:in `master'
from /Users/Kinglee/.rvm/gems/ruby-1.9.2-p180#rails3tutorial/gems/mongoid-2.4.10/lib/mongoid/config/database.rb:19:in `configure'
from /Users/Kinglee/.rvm/gems/ruby-1.9.2-p180#rails3tutorial/gems/mongoid-2.4.10/lib/mongoid/config.rb:290:in `configure_databases'
....
At this point, I am kind of confused. I kept asking myself, do I need the username and password to connect to mongodb ? I kind of 80% sure that I need them but I am not sure where to find them or rather not sure what am I connecting to, the mongodb ec2 instance or the mongodb database. How should I go about doing that ? Should I open port 27017 and 28017 on the instance ? Do I need to add config to database.yml (I highly doubt I need to since there is already mongoid.yml but just want to confirm)
I have been looking at a list of documentation and tutorial:
http://mongoid.org/docs/installation/configuration.html
http://www.mongodb.org/display/DOCS/Security+and+Authentication#SecurityandAuthentication-AbouttheKeyFile
MongoDB and Mongoid in production - looks like what I looking for, but not sure, going to try it.
http://craiccomputing.blogspot.com/2011/02/authentication-in-mongo-and-mongoid.html
Appreciate any advice from anyone here.
Ok finally found the problem. In the mongodb.conf file, there is a setting which called
bind_ip = 127.0.0.1
I was blind to not notice this, it means that the server can only be access locally and not externally, hence the fail connection error. A quick fix would be to change it to
bind_ip = 0.0.0.0
and it works. But thanks for the advice guys.
It is most likely a firewall issue. Check to see if the security group for your ec2 instance has the default mongodb port 27017 open.
This article will give you the gist of how it works if you haven't done something like that before:
http://cloud-computing.learningtree.com/2010/09/24/understanding-amazon-ec2-security-groups-and-firewalls/