How to set the RAILS_MASTER_KEY to production server?
i am using Capistrano to deploy (to a nginx/passenger) a rails 6 app (ruby 2.7.0). To let the production app access the credentials I am trying to provide it with the master.key
I can get the key the local env (development) master key to the shared/config folder of the server. Still, deploying the app ultimately fails.
To get there :
step 1 in the local environment, I generate a master key for the production environment and add the relevant variables. I have tried with the master key as well.
step 2 I manually add that key to the server shared/config/master.key file
I get the following response from Capistrano deploy command:
ActiveSupport::EncryptedFile::MissingKeyError: Missing encryption key to decrypt file with. Ask your team for your master key and write it to /home/deploy/tribe/releases/20200130135612/config/credentials/production.key or put it in the ENV['RAILS_MASTER_KEY'].
Assuming this might be a timing issue, I am also updating the current/config folder with the same key and receive the same response.
Obviously I do not get the master key where it should.
the Capistrano link file task is the following
append :linked_files, "config/master.key"
set :linked_files, %w{config/master.key}
namespace :deploy do
namespace :check do
before :linked_files, :set_master_key do
on roles(:app), in: :sequence, wait: 10 do
unless test("[ -f #{shared_path}/config/master.key ]")
upload! 'config/master.key', "#{shared_path}/config/master.key"
end
end
end
end
end
what I did is I added linked not just master.key but also production.key because log was complaining about production.key, not master
this is added in my deploy.rb
set :linked_files, %w{config/credentials/production.key}
now capistrano works and has no issues with secrets
I have a Rails 5 app, and I'm trying to store my environment variables on a non Heroku server.
Therefore I created a config/application.yml file with my environment variables on the production server.
However, when I try to deploy my Rails app, the deployment fails because the environment variables can't be accessed.
Am I missing something?
This answer is tested with Capistrano 3.6.1
Well, in that case, you would have to make capistrano upload your application.yml (database.yml if its ignored from git).
# in config/deploy.rb
# Default value for :linked_files is []
append :linked_files, 'config/database.yml', 'config/application.yml'
Now write some code to upload the files to the server
# lib/capistrano/tasks/uploader.rake
namespace :upload do
desc 'Upload shared files to the server'
task :yml_files do
on roles(:web) do |host|
fetch(:linked_files).each do |common_file|
upload! common_file, "#{fetch(:deploy_to)}/shared/#{common_file}"
end
end
end
end
Now you need to trigger the above mentioned capistrano task just before a very specific event.
# config/deploy.rb
before 'deploy:check:linked_files', 'upload:yml_files'
This will make sure the necessary environment variables are set before your deployed tries to boot your application in the production server.
I used to be a .NET guy, and enjoyed using a nightly build system (continuous integration) called CruiseControl.NET to automatically deploy my applications to my staging environment each night.
Now that I've made the jump to Ruby on Rails and GitHub, I've found myself a little confused about how to setup an equivalent automated, nightly build system. I want to do things right, the Rails way, but I could use a push in the right direction.
Here's what I'm using...
Ruby on Rails 3.2.9 (with asset pipeline)
RVM
Apache + Passenger
MySQL
Ubuntu 12.04 (staging server OS)
GitHub (SCM)
I'm looking for a system/solution that fulfills these requirements: (Ideally utilizing Capistrano...)
Deploy the latest commit from my 'master' branch in my GitHub repository to my staging server.
One-click build on-demand: I want to click a button or link to force a deploy (or re-deploy) at any time
Have the capability to run custom commands on the staging server as a part of the deploy (i.e. 'bundle install', restart Apache, etc...)
Automatic deploy daily or after a commit on GitHub (optional)
And because I'd never ask this question without doing some research first, here are some relevant resources I've found, but haven't been able to decipher:
Automatic Deployment via Git
Deploying Rails 3 apps with Capistrano
Suggestions, anyone? Thanks!
David
Well, it turns out there is alot of good information out there regarding how to use Capistrano for this purpose (including Prakash's reference), but none of it seems to be fully comprehensive.
After many hours of picking through guide, after forum, after stack overflow question, I managed to fulfill most of my goals. To save time for others, I will try to provide answers and information that I found in the context of my original question.
UPDATE:
It turns out I was looking for Jenkins all along: it's a perfect (an even an improved) analog to the CruiseControl build server application I had used before. Jenkins is a web application that kicks off builds on a scheduled basis or, with plugins, commit events. It doesn't include functionality to actually deploy my Rails app, so that's where Capistrano steps in. Using Jenkins' "shell execute" build task to trigger Capistrano deploys, I was able to accomplish all of my goals above.
Take a look at this guide for more detail.
ORIGINAL POST:
First off, Capistrano can work for the purpose of build system, however, it is not at all similar to CruiseControl.
CruiseControl is:
A web application, that is run on a machine that serves as a 'build server'
Any user may visit the web app to run builds; the user does not have to have anything setup or configured on their end... it's just a webpage.
The interface also provides functions for logs, blame, scheduled builds, etc...
Capistrano is:
Not an application or service, but a gem that works like rake; it bears similar functionality to the Ant and Nant scripts used by CruiseControl for the purpose of deploying.
Capistrano is run on a user's local machine (typically the Rails app folder), and requires configuration on their end to run
It directly connects from the user's machine to the remote server via SSH to perform deploys (this may not be preferable if you don't want to grant users SSH access to the remote server)
Commands are run through a console i.e. cap deploy (this is as close as it gets to 'one click' deploys.)
It does produce logs, perform rollbacks etc.
It does not kick-off scheduled builds on its own (using cron was suggested)
In regards to my original requirements...
Deploy the latest commit from my 'master' branch in my GitHub repository to my staging server.
Configuring Capistrano's deploy.rb file will accomplish this.
One-click build on-demand: I want to click a button or link to force a deploy (or re-deploy) at any time
All Capistrano deploys are done through 'force': you run 'cap deploy' in your console manually
Have the capability to run custom commands on the staging server as a part of the deploy (i.e. 'bundle install', restart Apache, etc...)
Configuring Capistrano's deploy.rb file will accomplish this. Most of these commands are included out of the box.
Automatic deploy daily or after a commit on GitHub (optional)
I haven't figured this one out yet... a cron job might be the best way to do this.
Setting up Capistrano
Start with this tutorial from GitHub first. In your Rails App folder, you should end up with a Capfile and config/deploy.rb file.
To save you some time, copy and paste these files, and tweak the settings to your needs.
The files below are configured for:
Deployment to a test environment
GitHub
RVM based Rails
Using an explicit version of Ruby and gemset
Including untracked files (i.e. a database.yml file you didn't commit to SCM)
Using Rails asset pipeline
Running migrate and seed with each deploy
Restarting Apache based Passenger with each deploy
Capfile
# Set this if you use a particular version of Ruby or Gemset
set :rvm_ruby_string, 'ruby-1.9.3-p286#global'
#set :rvm_ruby_string, ENV['GEM_HOME'].gsub(/.*\//,"") # Read from local system
require "bundler/capistrano"
# Uncomment this if you're using RVM
require "rvm/capistrano"
load 'deploy'
# Uncomment if you are using Rails' asset pipeline
load 'deploy/assets'
load 'config/deploy' # remove this line to skip loading any of the default tasks
config/deploy.rb
# BEGIN RUBY CONFIG
# You can manually override path variables here
# set :default_environment, {
# 'PATH' => "/usr/local/bin:/bin:/usr/bin:/bin:/<ruby-dir>/bin",
# 'GEM_HOME' => '<ruby-dir>/lib/ruby/gems/1.8',
# 'GEM_PATH' => '<ruby-dir>lib/ruby/gems/1.8',
# 'BUNDLE_PATH' => '<ruby-dir>/lib/ruby/gems/1.8/gems'
# }
# This changes the default RVM bin path
# set :rvm_bin_path, "~/bin"
# If your remote server doesn't have a ~/.rvm directory, but is installed
# at the /usr/local/rvm path instead, use this line
set :rvm_type, :system
# END RUBY CONFIG
default_run_options[:pty] = true # Must be set for the password prompt
# from git to work
# BEGIN MULTIPLE ENVIRONMENT DEPLOYS
# Read the following URL if you need to deploy to different environments (test, development, etc)
# https://github.com/capistrano/capistrano/wiki/2.x-Multistage-Extension
# set :stages, %w(production test)
# set :default_stage, "test"
# require 'capistrano/ext/multistage'
# END MULTIPLE ENVIRONMENT DEPLOYS
# BEGIN APPLICATION VARS
set :application, "yourapp"
set :rails_env, 'test'
# END APPLICATION VARS
# BEGIN PATH DEFINITIONS
set(:releases_path) { File.join(deploy_to, version_dir) }
set(:shared_path) { File.join(deploy_to, shared_dir) }
set(:current_path) { File.join(deploy_to, current_dir) }
set(:release_path) { File.join(releases_path, release_name) }
# END PATH DEFINITIONS
# BEGIN SCM VARS
set :repository, "git#github.com:yourgithubuser/yourrepository.git" # Your clone URL
set :scm, "git"
set :scm_username, "yourgithubuser"
set :scm_password, proc{Capistrano::CLI.password_prompt('GitHub password:')} # The deploy user's password
set :branch, "master"
# END SCM VARS
# BEGIN SERVER VARS
set :user, "ubuntu" # The server's user for deploys
role :web, "dev.#{application}" # The location of your web server i.e. dev.myapp.com
role :app, "dev.#{application}" # The location of your app server i.e. dev.myapp.com
role :db, "dev.#{application}", :primary => true # The location of your DB server i.e. dev.myapp.com
set :deploy_to, "/home/#{user}/www/#{application}"
set :deploy_via, :remote_cache
# Uncomment this if you want to store your Git SSH keys locally and forward
# Else, it uses keys in the remote server's .ssh directory
# ssh_options[:forward_agent] = true
# END SERVER VARS
# BEGIN ADDITIONAL TASKS
before "deploy:start" do
deploy.migrate
deploy.seed
end
before "deploy:restart" do
deploy.migrate
deploy.seed
end
# Some files that the Rails app needs are too sensitive to store in SCM
# Instead, manually upload these files to your <Rails app>/shared folder
# then the following code can be used to generate symbolic links to them,
# so that your rails app may use them.
# In this example below, I link 'shared/config/database.yml' and 'shared/db/seed_data/moderators.csv'
before "deploy:assets:precompile" do
run ["ln -nfs #{shared_path}/config/settings.yml #{release_path}/config/settings.yml",
"ln -nfs #{shared_path}/config/database.yml #{release_path}/config/database.yml",
"mkdir -p #{release_path}/db/seed_data",
"ln -nfs #{shared_path}/db/seed_data/moderators.csv #{release_path}/db/seed_data/moderators.csv",
"ln -fs #{shared_path}/uploads #{release_path}/uploads"
].join(" && ")
end
namespace :deploy do
# Define seed task (call 'cap deploy:seed')
desc "Reload the database with seed data"
task :seed do
run "cd #{current_path}; bundle exec rake db:seed RAILS_ENV=#{rails_env}"
end
# If you are using Passenger mod_rails uncomment this:
task :start do ; end
task :stop do ; end
task :restart, :roles => :app, :except => { :no_release => true } do
run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
end
end
# END ADDITIONAL TASKS
Yes. Capistrano is the tool you are looking for. Using Capistrano with the system cron should enable you to implement a build/deploy system with all four of the mentioned requirements. Rather easily at that.
The best resource to learn about Capistrano is the Deploying Rails book. It has two chapters on Capistrano, the first one covering the basic usage, and the second covering some advanced usage concepts. There is also a Capistrano Case study at the end which explores further config options with a real life deploy script example.
FYI: I finished reading the two chapters yesterday in a couple of hours and tweeted about it ;-)
I'd say go with Capistrano for the deploy part.
The CI part (build when you do a git push) would be best implemented by using a CI server, such as travis (if you're on Github) or Jenkins if you have a private repo.
For the one-click-build, go with either capistrano directly (cap deploy from command line) or a simple CI hook which runs all tests, then runs cap deploy.
I thought I'd do a simple yet potentially very useful thing, and create another symlink called live, that points to an arbitrary release, and leave current at the head where it usually is:
20120519235508
20120521004833
20120521024312 <-- live
20120521025150
20120521030449 <-- current
I then configured www.mysite.com to hit
live/public
and stage.mysite.com to hit
current/public
Unfortunately both hosts seem to run the same app, and not 2 different apps. I've confirmed the httpd.conf has the correct settings and restarted it. However no change, they're both still running the same app, the app referenced by current/public to be exact.
I don't know if I have a setting wrong, or if something else needs to be restarted, or if this simply can't work as I imagined. I'm using passenger.
Can someone shed some light on this subject, because this configuration would be VERY useful to me for many projects.
Instead of creating an other symlink in the releases directory, I suggest to use the multistage extension. With this extension you can define different stages and add custom configuration to them. So instead of using one deployment directory for both staging and production, use a separate one for each other.
Add these lines to deploy.rb:
require "capistrano/ext/multistage"
set :stages, ["staging", "production"]
set :default_stage, "staging"
Remove the deploy_to variable from deploy.rb. Then create a deploy directory inside config which has files with the stage names. In this case: deploy/staging.rb and deploy/production.rb. The content of staging.rb:
set :rails_env, "staging"
set :deploy_to, "staging/capistrano"
And similarly for production.rb:
set :rails_env, "production"
set :deploy_to, "production/capistrano"
Of course change the paths in deploy_to. Then point staging.example.com to staging/capistrano/current/public and www.example.com to production/capistrano/current/public.
To do a staging deploy, execute cap staging deploy or simple cap deploy (remember, staging was set to default in deploy.rb) and cap production deploy to deploy to production.
I have some ENV variables that are sourced for the deploy user. (Similar to what Heroku recommends, but without using Heroku.)
My rails app depends on these for certain functions, for example, in application.rb:
config.action_mailer.default_url_options = { host: ENV['MY_HOST'] }
This is necessary because we have several staging hosts. Each host has MY_HOST defined to its correct hostname in .bashrc like so:
export MY_HOST="staging3.example.com"
This allows us to only use one rails staging environment, but still have each host's correct hostname used for testing, sending email, etc since this can be set on a per-machine basis.
Unfortunately it looks like when I restart Unicorn using USR2 it doesn't pick up changes to those variables. Doing a hard-stop and start will correctly load any changes.
I'm using preload_app = true which may I'm guessing has something to do with it. Any ideas?
In the end I went away from this approach altogether in favor of loading my app config from an app_config.yml file. Ryan Bates covers this approach in Railscast #226.
The only thing I did differently is that I load a shared app_config.yml for each server I use. Since I'm using capistrano, I just symlink the file on deploy.
For example, on staging2 my shared/configs/app_config.yml looks like this:
staging:
host: "staging2.example.com"
... whereas on staging3 it looks like this:
staging:
host: "staging3.example.com"
Now my application.rb has this line instead:
config.action_mailer.default_url_options = { host: APP_CONFIG[:host] }
I removed the actual config/app_config.yml from git so that it's not included on deploy. (I moved it to config/app_config.yml.template.) Then on deploy I use a capistrano task to symlink shared/configs/app_config.yml to config/app_config.yml:
namespace :deploy do
desc "Symlinks the app_config.yml"
task :symlink_app_config, :roles => [:web, :app, :db] do
run "ln -nfs #{deploy_to}/shared/config/app_config.yml #{release_path}/config/app_config.yml"
end
end
This strategy has these benefits over using ENV vars:
Deployed to all nodes via capistrano
We can do conditional hosts by simply changing the file on the appropriate server
Unicorn will get changes with USR2 since everything's done inside of rails
Everything's kept in one place, and the environment isn't affected by some other variables outside of the codebase