My master branch layout is like this:
/ <-- top level
/client <-- desktop client source files
/server <-- Rails app
What I'd like to do is only pull down the /server directory in my deploy.rb, but I can't seem to find any way to do that. The /client directory is huge, so setting up a hook to copy /server to / won't work very well, it needs to only pull down the Rails app.
Without any dirty forking action but even dirtier !
In my config/deploy.rb :
set :deploy_subdir, "project/subdir"
Then I added this new strategy to my Capfile :
require 'capistrano/recipes/deploy/strategy/remote_cache'
class RemoteCacheSubdir < Capistrano::Deploy::Strategy::RemoteCache
private
def repository_cache_subdir
if configuration[:deploy_subdir] then
File.join(repository_cache, configuration[:deploy_subdir])
else
repository_cache
end
end
def copy_repository_cache
logger.trace "copying the cached version to #{configuration[:release_path]}"
if copy_exclude.empty?
run "cp -RPp #{repository_cache_subdir} #{configuration[:release_path]} && #{mark}"
else
exclusions = copy_exclude.map { |e| "--exclude=\"#{e}\"" }.join(' ')
run "rsync -lrpt #{exclusions} #{repository_cache_subdir}/* #{configuration[:release_path]} && #{mark}"
end
end
end
set :strategy, RemoteCacheSubdir.new(self)
For Capistrano 3.0, I use the following:
In my Capfile:
# Define a new SCM strategy, so we can deploy only a subdirectory of our repo.
module RemoteCacheWithProjectRootStrategy
def test
test! " [ -f #{repo_path}/HEAD ] "
end
def check
test! :git, :'ls-remote', repo_url
end
def clone
git :clone, '--mirror', repo_url, repo_path
end
def update
git :remote, :update
end
def release
git :archive, fetch(:branch), fetch(:project_root), '| tar -x -C', release_path, "--strip=#{fetch(:project_root).count('/')+1}"
end
end
And in my deploy.rb:
# Set up a strategy to deploy only a project directory (not the whole repo)
set :git_strategy, RemoteCacheWithProjectRootStrategy
set :project_root, 'relative/path/from/your/repo'
All the important code is in the strategy release method, which uses git archive to archive only a subdirectory of the repo, then uses the --strip argument to tar to extract the archive at the right level.
UPDATE
As of Capistrano 3.3.3, you can now use the :repo_tree configuration variable, which makes this answer obsolete. For example:
set :repo_url, 'https://example.com/your_repo.git'
set :repo_tree, 'relative/path/from/your/repo' # relative path to project root in repo
See http://capistranorb.com/documentation/getting-started/configuration.
We're also doing this with Capistrano by cloning down the full repository, deleting the unused files and folders and move the desired folder up the hierarchy.
deploy.rb
set :repository, "git#github.com:name/project.git"
set :branch, "master"
set :subdir, "server"
after "deploy:update_code", "deploy:checkout_subdir"
namespace :deploy do
desc "Checkout subdirectory and delete all the other stuff"
task :checkout_subdir do
run "mv #{current_release}/#{subdir}/ /tmp && rm -rf #{current_release}/* && mv /tmp/#{subdir}/* #{current_release}"
end
end
As long as the project doesn't get too big this works pretty good for us, but if you can, create an own repository for each component and group them together with git submodules.
You can have two git repositories (client and server) and add them to a "super-project" (app). In this "super-project" you can add the two repositories as submodules (check this tutorial).
Another possible solution (a bit more dirty) is to have separate branches for client and server, and then you can pull from the 'server' branch.
There is a solution. Grab crdlo's patch for capistrano and the capistrano source from github. Remove your existing capistrano gem, appy the patch, setup.rb install, and then you can use his very simple configuration line set :project, "mysubdirectory" to set a subdirectory.
The only gotcha is that apparently github doesn't "support the archive command" ... at least when he wrote it. I'm using my own private git repo over svn and it works fine, I haven't tried it with github but I imagine if enough people complain they'll add that feature.
Also see if you can get capistrano authors to add this feature into cap at the relevant bug.
For Capistrano 3, based on #Thomas Fankhauser answer:
set :repository, "git#github.com:name/project.git"
set :branch, "master"
set :subdir, "relative_path_to_my/subdir"
namespace :deploy do
desc "Checkout subdirectory and delete all the other stuff"
task :checkout_subdir do
subdir = fetch(:subdir)
subdir_last_folder = File.basename(subdir)
release_subdir_path = File.join(release_path, subdir)
tmp_base_folder = File.join("/tmp", "capistrano_subdir_hack")
tmp_destination = File.join(tmp_base_folder, subdir_last_folder)
cmd = []
# Settings for my-zsh
# cmd << "unsetopt nomatch && setopt rmstarsilent"
# create temporary folder
cmd << "mkdir -p #{tmp_base_folder}"
# delete previous temporary files
cmd << "rm -rf #{tmp_base_folder}/*"
# move subdir contents to tmp
cmd << "mv #{release_subdir_path}/ #{tmp_destination}"
# delete contents inside release
cmd << "rm -rf #{release_path}/*"
# move subdir contents to release
cmd << "mv #{tmp_destination}/* #{release_path}"
cmd = cmd.join(" && ")
on roles(:app) do
within release_path do
execute cmd
end
end
end
end
after "deploy:updating", "deploy:checkout_subdir"
Unfortunately, git provides no way to do this. Instead, the 'git way' is to have two repositories -- client and server, and clone the one(s) you need.
I created a snipped that works with Capistrano 3.x based in previous anwers and other information found in github:
# Usage:
# 1. Drop this file into lib/capistrano/remote_cache_with_project_root_strategy.rb
# 2. Add the following to your Capfile:
# require 'capistrano/git'
# require './lib/capistrano/remote_cache_with_project_root_strategy'
# 3. Add the following to your config/deploy.rb
# set :git_strategy, RemoteCacheWithProjectRootStrategy
# set :project_root, 'subdir/path'
# Define a new SCM strategy, so we can deploy only a subdirectory of our repo.
module RemoteCacheWithProjectRootStrategy
include Capistrano::Git::DefaultStrategy
def test
test! " [ -f #{repo_path}/HEAD ] "
end
def check
test! :git, :'ls-remote -h', repo_url
end
def clone
git :clone, '--mirror', repo_url, repo_path
end
def update
git :remote, :update
end
def release
git :archive, fetch(:branch), fetch(:project_root), '| tar -x -C', release_path, "--strip=#{fetch(:project_root).count('/')+1}"
end
end
It's also available as a Gist on Github.
dont know if anyone is still interested on this. but just letting you guys if anyone is looking for an answer.
now we can use: :repo_tree
https://capistranorb.com/documentation/getting-started/configuration/
Looks like it's also not working with codebasehq.com so I ended up making capistrano tasks that cleans the mess :-) Maybe there's actually a less hacky way of doing this by overriding some capistrano tasks...
This has been working for me for a few hours.
# Capistrano assumes that the repository root is Rails.root
namespace :uploads do
# We have the Rails application in a subdirectory rails_app
# Capistrano doesn't provide an elegant way to deal with that
# for the git case. (For subversion it is straightforward.)
task :mv_rails_app_dir, :roles => :app do
run "mv #{release_path}/rails_app/* #{release_path}/ "
end
end
before 'deploy:finalize_update', 'uploads:mv_rails_app_dir'
You might declare a variable for the directory (here rails_app).
Let's see how robust it is. Using "before" is pretty weak.
Related
I've added new HDD to the server and need to use it to save all users info, so I need to move public directory to new HDD.
I don't know what is the best approach in my case, but I decided to change the public directory to be symbolic link pointing to new HDD path.
Based on that I created the below capistrano task to run after deploy:
# I'm using capistrano 3.1
namespace :public_to_symbolic do
desc 'Change public directory to symbolic link pointing to other Hdd'
task :change do
on roles(:app) do
is_directory = File.directory?("#{release_path}/public")
is_symbolic_link = File.symlink?("#{release_path}/public")
is_new_directory_exist = File.directory?('/mnt/newhdd/public')
can_change = is_directory && !is_symbolic_link && is_new_directory_exist
puts "is_directory : #{is_directory}" # prints false
puts "is_symbolic_link : #{is_symbolic_link}" # prints false
puts "is_new_directory_exist : #{is_new_directory_exist}" # prints false
puts "can_change: #{can_change}" # prints false
puts "release_path: #{release_path}" # prints /var/www/myapp/releases/20180305112922
if can_change
puts 'Changing public directory to symbolic link'
execute "mv #{release_path}/public #{release_path}/public_linked"
execute "ln -s /mnt/newhdd/public #{release_path}/public"
execute "rm -rf #{release_path}/public_linked"
end
end
end
end
after 'deploy', 'public_to_symbolic:change'
So, as commented, is_directory, is_symbolic_link and is_new_directory_exist are all returns false, but when I'm doing the same check (i.e File.directory?("#{Rails.root}/public") or File.directory?("/var/www/myapp/current/public")) using Rails console on deployment server, I'm getting true and I'm able to see the public directory!
To avoid this issue I tried also to use current_path instead of release_path but still same results.
I have two questions:
Why is_directory and is_new_directory_exist are always false while task is running?
If there is any better approach to use public directory on different Hdd, please advice.
All suggestions are welcome
All Ruby methods that you use in your Capistrano tasks run on your local machine.
For example:
File.directory?
File.symlink?
These are always evaluated on your local filesystem. Capistrano never runs Ruby code on the remote server. These methods always return false for you because they are trying to find e.g. "#{release_path}/public" on your local computer, which of course does not exist.
To run code on the server, the tools available to you are Capistrano's test and execute methods. These take in a command string that is executed remotely via SSH.
If you want to test if a remote path is a directory, you cannot use Ruby; you have to use something that can be run in a remote shell. Here is one way to test if a path is a directory, for example:
is_directory = test("[ -d #{release_path}/public ]")
Likewise, to test if a path is a symlink:
is_symbolic_link = test("[ -h #{release_path}/public ]")
After a mina deploy, it's hanging on "Updating the /home/x/app/current symlink". No errors. It just sits there.
I have tried removing the app directory from the server and "mina setup", but still encountering the same problem. I had no issues deploying initially, but it seems any attempt to deploy subsequent releases results in this problem.
I initially followed this guide to deploy: https://www.ralfebert.de/tutorials/rails-deployment/
require 'mina/rails'
require 'mina/git'
require 'mina/rvm'
# Basic settings:
# domain - The hostname to SSH to.
# deploy_to - Path to deploy into.
# repository - Git repo to clone from. (needed by mina/git)
# branch - Branch name to deploy. (needed by mina/git)
set :application_name, 'x'
set :domain, 'x'
set :user, fetch(:application_name)
set :deploy_to, "/home/#{fetch(:user)}/app"
set :repository, 'x'
set :branch, 'x'
set :rvm_use_path, '/etc/profile.d/rvm.sh'
# Optional settings:
# set :user, 'foobar' # Username in the server to SSH to.
# set :port, '30000' # SSH port number.
# set :forward_agent, true # SSH forward_agent.
# shared dirs and files will be symlinked into the app-folder by the 'deploy:link_shared_paths' step.
# set :shared_dirs, fetch(:shared_dirs, []).push('somedir')
set :shared_files, fetch(:shared_files, []).push('config/database.yml', 'config/secrets.yml')
# This task is the environment that is loaded for all remote run commands, such as
# `mina deploy` or `mina rake`.
task :environment do
ruby_version = File.read('.ruby-version').strip
raise "Couldn't determine Ruby version: Do you have a file .ruby-version in your project root?" if ruby_version.empty?
invoke :'rvm:use', ruby_version
end
task :setup do
in_path(fetch(:shared_path)) do
command %[mkdir -p config]
# Create database.yml for Postgres if it doesn't exist
path_database_yml = "config/database.yml"
database_yml = %[production:
database: #{fetch(:user)}
adapter: postgresql
pool: 5
timeout: 5000]
command %[test -e #{path_database_yml} || echo "#{database_yml}" > #{path_database_yml}]
# Create secrets.yml if it doesn't exist
path_secrets_yml = "config/secrets.yml"
secrets_yml = %[production:\n secret_key_base:\n #{`rake secret`.strip}]
command %[test -e #{path_secrets_yml} || echo "#{secrets_yml}" > #{path_secrets_yml}]
# Remove others-permission for config directory
command %[chmod -R o-rwx config]
end
end
desc "Deploys the current version to the server."
task :deploy do
# uncomment this line to make sure you pushed your local branch to the remote origin
# invoke :'git:ensure_pushed'
deploy do
# Put things that will set up an empty directory into a fully set-up
# instance of your project.
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
# invoke :'rails:db_migrate'
invoke :'rails:assets_precompile'
invoke :'deploy:cleanup'
on :launch do
command "sudo service #{fetch(:user)} restart"
end
end
# you can use `run :local` to run tasks on local machine before of after the deploy scripts
# run(:local){ say 'done' }
end
# For help in making your deploy script, see the Mina documentation:
#
# - https://github.com/mina-deploy/mina/tree/master/docs
I used the same great tutorial, got the same problem.
You can run mina deploy --verbose to see where it stuck.
For me it was not the symlink updating, but sudo service rails-demo restart command.
I used sudo visudo on the server and put the following line there:
rails-demo ALL=(ALL) NOPASSWD: /usr/sbin/service rails-demo restart
Now it works like a charm.
Good luck!
I want my TODO list which I get as output of 'rake notes' to be under source control (Git).
Is there a way I can configure 'rake notes' to spit out out a text file which can be under source control?
Not completely understanding your question's direction, so I'll just go from the beginning. :)
Create a rake file :)
The most common is under Rails.root / lib, but it's preferable (depending on Elitests or purists) to put it under lib/tasks/output_notes.rake (yes, .rake, not .rb)
Rake scripts have namespaces. "db" is a namespace for running "rake db:migrate" for example. You can see some examples here: https://github.com/rails/rails/tree/master/railties/lib/rails/tasks
Your script might look something like this.
namespace :vasa do
desc "Output notes"
task :output_notes do
output_file_name = "#{DateTime.now.to_i}_notes.txt"
File.open(output_file_name, 'w') { |file| file.write(Note.dump_all) }
end
end
You can then run rake vasa:output_notes. Want to add the file to your repo automagically?
# after File.open ...
`
git add #{output_file_name}
git commit -m "added notes"
`
The backticks will run it as a system command.
I have no experience with Ruby or rake or anything, but I am using slate for API documentation, and it uses Ruby and rake and stuff to build the file. I know nothing at all about these things, but what I do know is this: when I do a rake build it updates a folder (slate/build). I then have to manually copy slate/build to ../app/docs after every single rake build. Is there something I can do that will copy that folder on every rake build automatically for me?
Add to your Rakefile:
ROOT = File.expand_path('..', __FILE__)
task :build_and_move => [:build] do
cp_r(File.join(ROOT, 'slate/build'), File.join(ROOT, '../app/docs'))
# or
# mv(File.join(ROOT, 'slate/build'), File.join(ROOT, '../app/docs'))
end
and then run rake build_and_move.
You can use FileUtils for this.
Docs: http://ruby-doc.org/stdlib-1.9.3/libdoc/fileutils/rdoc/FileUtils.html#method-c-copy
Example from the docs:
Copies src to dest. If src is a directory, this method copies all its contents recursively. If dest is a directory, copies src to dest/src.
FileUtils.cp 'eval.c', 'eval.c.org'
FileUtils.cp %w(cgi.rb complex.rb date.rb), '/usr/lib/ruby/1.6'
FileUtils.cp %w(cgi.rb complex.rb date.rb), '/usr/lib/ruby/1.6', :verbose => true
FileUtils.cp 'symlink', 'dest' # copy content, "dest" is not a symlink
I am trying to use mina to deploy my app to a digital ocean server and have a git repo on bitbucket. I was able to run mina setup' just fine, but when I runmina deploy` I get an error.
my deploy.rb
require 'mina/bundler'
require 'mina/rails'
require 'mina/git'
require 'mina/rbenv' # for rbenv support. (http://rbenv.org)
# require 'mina/rvm' # for rvm support. (http://rvm.io)
# Basic settings:
# domain - The hostname to SSH to.
# deploy_to - Path to deploy into.
# repository - Git repo to clone from. (needed by mina/git)
# branch - Branch name to deploy. (needed by mina/git)
set :rails_env, 'production'
set :domain, 'my.server'
set :deploy_to, '/home/deployer/mysite'
set :repository, 'git#bitbucket.org:me/myproject.git'
set :branch, 'master'
set :user, 'deployer'
set :forward_agent, true
set :port, '22'
# Manually create these paths in shared/ (eg: shared/config/database.yml) in your server.
# They will be linked in the 'deploy:link_shared_paths' step.
set :shared_paths, ['config/database.yml', 'log', 'config/secrets.yml']
# Optional settings:
# set :user, 'foobar' # Username in the server to SSH to.
# set :port, '30000' # SSH port number.
# This task is the environment that is loaded for most commands, such as
# `mina deploy` or `mina rake`.
task :environment do
# If you're using rbenv, use this to load the rbenv environment.
# Be sure to commit your .rbenv-version to your repository.
queue %{
echo "-----> Loading environment"
#{echo_cmd %[source ~/.bashrc]}
}
invoke :'rbenv:load'
# For those using RVM, use this to load an RVM version#gemset.
# invoke :'rvm:use[ruby-1.9.3-p125#default]'
end
# Put any custom mkdir's in here for when `mina setup` is ran.
# For Rails apps, we'll make some of the shared paths that are shared between
# all releases.
task :setup => :environment do
queue! %[mkdir -p "#{deploy_to}/shared/log"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/log"]
queue! %[mkdir -p "#{deploy_to}/shared/config"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/config"]
queue! %[touch "#{deploy_to}/shared/config/database.yml"]
queue %[echo "-----> Be sure to edit 'shared/config/database.yml'."]
queue! %[touch "#{deploy_to}/shared/config/secrets.yml"]
queue %[echo "-----> Be sure to edit 'shared/config/secrets.yml'."]
end
desc "Deploys the current version to the server."
task :deploy => :environment do
deploy do
# Put things that will set up an empty directory into a fully set-up
# instance of your project.
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
invoke :'rails:db_migrate'
invoke :'rails:assets_precompile'
to :launch do
invoke :'passenger:restart'
end
end
end
desc "Restarts the nginx server."
task :restart do
invoke :'passenger:restart'
end
namespace :passenger do
task :restart do
queue "mkdir #{deploy_to}/current/tmp; touch #{deploy_to}/current/tmp/restart.txt"
end
end
# For help in making your deploy script, see the Mina documentation:
#
# - http://nadarei.co/mina
# - http://nadarei.co/mina/tasks
# - http://nadarei.co/mina/settings
# - http://nadarei.co/mina/helpers
When I do "mina deploy", I get this error
-----> Loading environment
-----> Loading rbenv
-----> Creating a temporary build path
-----> Cloning the Git repository
Cloning into bare repository '/home/deployer/mysite/scm'...
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
! ERROR: Deploy failed.
-----> Cleaning up build
Unlinking current
OK
! Command failed.
Failed with status 19
I have ssh keys set up on bitbucket and Im able to push my repo to it just fine from my computer, I also have an ssh key from my server set up on bitbucket (not sure if it is needed, but I thought i would try it). What could be wrong?
add this in your deploy.rb file
set :term_mode, nil
make sure that you have added server SSH public key on github
You can use your local ssh keys for this.
Use the -A setting of SSH. and set it in mina like set :port, '22 -A' this will append the -A to the ssh command issued by Mina.
Basically it forwards the authentication you use on your deployment server. (like set :forward_agent, true does in Capistrano)
I think best practice however, is to generate a ssh key on the server and set it as a deploy key in your hosted SCM solution.
Your server tries to git from git#bitbucket.org:me/myproject.git, but there is host key at your server.
You can copy your id_rsa.pub and known_hosts file through scp command on server and then run it again. It will work.