INFO -- : executing ["/home/shared/bundle/ruby/2.1.0/bin/unicorn", "-c", "/home/current/config/unicorn/production.rb", "-E", "deployment", "-D", {12=>#<Kgio::UNIXServer:fd 12>}] (in /home/releases/20140714144301)
DEBUG[b59ba27c] I, [2014-07-14T14:43:33.495683 #27897] INFO -- : forked child re-executing...
DEBUG[b59ba27c] /home/
DEBUG[b59ba27c] lic/.rvm/gems/ruby-2.1.2#global/gems/bundler-1.6.3/lib/bundler/definition.rb:23:in `build': /home/releases/20140710032913/Gemfile not found (Bundler::GemfileNotFound)
I am pretty sure the problem stems from the fact that with zero-downtime deployment, unicorn is caching this release number: 20140710032913. I have capistrano set to keep 5 releases. After 5 deploys, the old release number is rolled off. How can I force unicorn to use the current Gemfile?
Possible duplicate: Unicorn restart issue with capistrano
In my #{Rails.root}/config/unicorn/#{environment}.rb file I needed:
before_exec do |_server|
ENV['BUNDLE_GEMFILE'] = "#{working_directory}/Gemfile"
end
Related
I set up a clean install of Yosemite and can't get guard to detect whenever a file is changed.
My Environment:
guard (2.6.1)
guard-minitest (2.2.0)
rubygems-bundler (1.4.4)
Ruby 1.9.3 and 2.0.0 both have been testing
Here is the output from debugging:
tom#Toms-MBP:~/Sites/tick (master)$ LISTEN_GEM_DEBUGGING=2 bundle exec guard -d
I, [2014-10-27T15:34:53.417196 #1059] INFO -- : Celluloid loglevel set to: 0
15:34:53 - DEBUG - Command execution: emacsclient --eval '1' 2> /dev/null || echo 'N/A'
15:34:53 - INFO - Guard is using TerminalTitle to send notifications.
15:34:53 - DEBUG - Command execution: hash stty
15:34:53 - DEBUG - Guard starts all plugins
15:34:53 - DEBUG - Hook :start_begin executed for Guard::Minitest
15:34:53 - INFO - Guard::Minitest 2.3.2 is running, with Minitest::Unit 4.7.5!
15:34:53 - DEBUG - Hook :start_end executed for Guard::Minitest
15:34:53 - INFO - Guard is now watching at '/Users/tom/Dropbox/Sites/tick'
15:34:53 - DEBUG - Command execution: sysctl -n hw.ncpu
D, [2014-10-27T15:34:53.517668 #1059] DEBUG -- : Adapter: considering TCP ...
D, [2014-10-27T15:34:53.517749 #1059] DEBUG -- : Adapter: considering polling ...
D, [2014-10-27T15:34:53.517776 #1059] DEBUG -- : Adapter: considering optimized backend...
I, [2014-10-27T15:34:53.587089 #1059] INFO -- : Record.build(): 0.06837701797485352 seconds
15:34:53 - DEBUG - Command execution: stty -g 2>/dev/null
15:34:53 - DEBUG - Start interactor
[1] guard(main)>
I am not sure how to test the underlying listen gem, but seems like it has to be related. I have confirmed that guard works correctly with polling (bundle exec guard start --force-polling).
I was able to resolve my issue by moving the files out of Dropbox. Once I saw this working, I knew that Dropbox was somehow causing the issue with the file system. I resolved it by removing the folder and re-adding it through Dropbox and now everything is working as before.
I had the same problem, and only thanks to a warning issued by the LiveReload application I fixed it.
Because the problem is not restrained to Dropbox and/or ruby on rails, I wanted to share what I did:
Rename problematic folder (in my case containing the Guardfile)
Create new folder with the old folders name
Move files from old to new folder
Other solutions mentioned by LiveReload are:
rebooting the computer
checking the disk and repairing permissions via Disk Utility
adding the folder to Spotlight privacy list (the list of folders to not index), and then removing from it, effectively forcing a
reindexing
renaming the folder, and then possibly renaming it back
re-creating the folder and moving the old contents back into it
Read more: http://feedback.livereload.com/knowledgebase/articles/86239
This not only fixed my Guard problem, but also allowed Compass watch to run again.
Did you add rb-fsevent gem ?
group :development, :test do
gem 'rb-fsevent' if `uname` =~ /Darwin/
end
if you're using OS X, it will be used OS X FSEvents API.
I’m having this peculiar problem when attempting to restart Unicorn using a USR2 signal. On a clean reboot of the VPS, I have no problems with sending a USR2 signal to Unicorn and having it gracefully restart. However, after an hour or so if I try to do it again, I will be left with an old master hanging around preventing the new master from starting. I am then forced to kill the old master so the new master can start. If I reboot the VPS, it fixes it but then after an hour so the problem starts again. I'm on Rails 4, Ruby 2.0.0.
unicorn.log
I, [2014-01-07T15:37:37.118523 #19797] INFO -- : executing ["/srv/rails/current/bin/unicorn", "-c", "/srv/rails/current/config/unicorn.rb", {12=>#<Kgio::UNIXServer:fd 12>}] (in /srv/rails/releases/20140107091945)
I, [2014-01-07T15:37:37.118983 #19797] INFO -- : forked child re-executing...
I, [2014-01-07T15:37:38.998632 #19797] INFO -- : inherited addr=/srv/rails/shared/sockets/unicorn.sock fd=12
I, [2014-01-07T15:37:38.999038 #19797] INFO -- : Refreshing Gem list
I, [2014-01-07T15:37:41.927794 #19967] INFO -- : Refreshing Gem list
/srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:219:in `pid=': Already running on PID:19967 (or pid=/srv/rails/shared/pids/unicorn.pid is stale) (ArgumentError)
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:151:in `start'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/bin/unicorn:126:in `<top (required)>'
from /srv/rails/current/bin/unicorn:16:in `load'
from /srv/rails/current/bin/unicorn:16:in `<main>'
/srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:219:in `pid=': Already running on PID:21250 (or pid=/srv/rails/shared/pids/unicorn.pid is stale) (ArgumentError)
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:151:in `start'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/bin/unicorn:126:in `<top (required)>'
from /srv/rails/current/bin/unicorn:16:in `load'
from /srv/rails/current/bin/unicorn:16:in `<main>'
E, [2014-01-07T15:40:46.720131 #20878] ERROR -- : reaped #<Process::Status: pid 21075 exit 1> exec()-ed
E, [2014-01-07T15:40:46.720870 #20878] ERROR -- : master loop error: Already running on PID:21250 (or pid=/srv/rails/shared/pids/unicorn.pid is stale) (ArgumentError)
E, [2014-01-07T15:40:46.723525 #20878] ERROR -- : /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:219:in `pid='
E, [2014-01-07T15:40:46.723671 #20878] ERROR -- : /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:413:in `reap_all_workers'
E, [2014-01-07T15:40:46.723747 #20878] ERROR -- : /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:292:in `join'
E, [2014-01-07T15:40:46.723815 #20878] ERROR -- : /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/bin/unicorn:126:in `<top (required)>'
E, [2014-01-07T15:40:46.723880 #20878] ERROR -- : /srv/rails/current/bin/unicorn:16:in `load'
E, [2014-01-07T15:40:46.723930 #20878] ERROR -- : /srv/rails/current/bin/unicorn:16:in `<main>'
E, [2014-01-07T15:41:13.704700 #21250] ERROR -- : adding listener failed addr=/srv/rails/shared/sockets/unicorn.sock (in use)
E, [2014-01-07T15:41:13.704901 #21250] ERROR -- : retrying in 0.5 seconds (4 tries left)
E, [2014-01-07T15:41:14.205452 #21250] ERROR -- : adding listener failed addr=/srv/rails/shared/sockets/unicorn.sock (in use)
E, [2014-01-07T15:41:14.205597 #21250] ERROR -- : retrying in 0.5 seconds (3 tries left)
78.40.124.16, 173.245.49.122 - - [07/Jan/2014 15:41:14] "GET / HTTP/1.0" 200 28697 0.8345
E, [2014-01-07T15:41:14.706179 #21250] ERROR -- : adding listener failed addr=/srv/rails/shared/sockets/unicorn.sock (in use)
E, [2014-01-07T15:41:14.706335 #21250] ERROR -- : retrying in 0.5 seconds (2 tries left)
E, [2014-01-07T15:41:15.206834 #21250] ERROR -- : adding listener failed addr=/srv/rails/shared/sockets/unicorn.sock (in use)
E, [2014-01-07T15:41:15.206987 #21250] ERROR -- : retrying in 0.5 seconds (1 tries left)
E, [2014-01-07T15:41:15.707431 #21250] ERROR -- : adding listener failed addr=/srv/rails/shared/sockets/unicorn.sock (in use)
E, [2014-01-07T15:41:15.707563 #21250] ERROR -- : retrying in 0.5 seconds (0 tries left)
78.40.124.16, 149.154.158.74 - - [07/Jan/2014 15:41:15] "GET / HTTP/1.0" 200 32866 0.4528
E, [2014-01-07T15:41:16.208055 #21250] ERROR -- : adding listener failed addr=/srv/rails/shared/sockets/unicorn.sock (in use)
/srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/socket_helper.rb:158:in `initialize': Address already in use - "/srv/rails/shared/sockets/unicorn.sock" (Errno::EADDRINUSE)
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/socket_helper.rb:158:in `new'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/socket_helper.rb:158:in `bind_listen'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:255:in `listen'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:801:in `block in bind_new_listeners!'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:801:in `each'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:801:in `bind_new_listeners!'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:146:in `start'
from /srv/rails/shared/bundle/ruby/2.0.0/gems/unicorn-4.7.0/bin/unicorn:126:in `<top (required)>'
from /srv/rails/current/bin/unicorn:16:in `load'
from /srv/rails/current/bin/unicorn:16:in `<main>'
unicorn.rb
deploy_path = "/srv/rails"
RAILS_ENV = ENV['RAILS_ENV'] || "production"
working_directory "#{deploy_path}/current"
pid "#{deploy_path}/shared/pids/unicorn.pid"
stderr_path "#{deploy_path}/shared/log/unicorn.log"
# Listen on a UNIX data socket
listen "#{deploy_path}/shared/sockets/unicorn.sock"
worker_processes 4
# Preload application before forking worker processes
preload_app true
# Restart any workers that haven't responded in 30 seconds
timeout 30
before_fork do |server, worker|
##
# When sent a USR2, Unicorn will suffix its pidfile with .oldbin and
# immediately start loading up a new version of itself (loaded with a new
# version of our app). When this new Unicorn is completely loaded
# it will begin spawning workers. The first worker spawned will check to
# see if an .oldbin pidfile exists. If so, this means we've just booted up
# a new Unicorn and need to tell the old one that it can now die. To do so
# we send it a QUIT.
#
# Using this method we get 0 downtime deploys.
old_pid = "#{server.config[:pid]}.oldbin"
if File.exists?(old_pid) && server.pid != old_pid
begin
Process.kill("QUIT", File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH => e
log = File.open(Rails.root.join('log/unicorn.log'), "a")
log.puts "Error encountered when killing process:\n"
log.puts "#{e.message}"
log.close
end
end
# the following is recomended for Rails + "preload_app true"
# as there's no need for the master process to hold a connection
if defined?(ActiveRecord::Base)
ActiveRecord::Base.connection.disconnect!
end
end
after_fork do |server, worker|
##
# Unicorn master loads the app then forks off workers - because of the way
# Unix forking works, we need to make sure we aren't using any of the parent's
# sockets, e.g. db connection
ActiveRecord::Base.establish_connection
# Redis and Memcached would go here but their connections are established
# on demand, so the master never opens a socket
##
# Unicorn master is started as root, which is fine, but let's
# drop the workers to deployer
begin
uid, gid = Process.euid, Process.egid
user, group = 'deployer', 'deployer'
target_uid = Etc.getpwnam(user).uid
target_gid = Etc.getgrnam(group).gid
worker.tmp.chown(target_uid, target_gid)
if uid != target_uid || gid != target_gid
Process.initgroups(user, target_gid)
Process::GID.change_privilege(target_gid)
Process::UID.change_privilege(target_uid)
end
rescue => e
if RAILS_ENV == 'development'
STDERR.puts "couldn't change user, oh well"
else
raise e
end
end
end
deploy.rb
require 'bundler/capistrano' # runs a bundle install --deployment
# https://github.com/sstephenson/rbenv/issues/101
set :keep_releases, 10
set :shared_children, shared_children + %w(public/images public/uploads)
# Multistage extension
set :stages, ["production", "staging"]
set :default_stage, "staging"
require 'capistrano/ext/multistage'
require 'underglow/capistrano'
# Whenever crontab updates
set :whenever_environment, defer { stage }
set :whenever_command, "bin/whenever"
require 'whenever/capistrano'
set :application, "rails"
set :user, "deployer"
default_run_options[:pty] = true
default_run_options[:shell] = '/bin/zsh'
set :use_sudo, false
# repository
set :repository, "XXXXXXXXXXXXXXXXX"
set :branch, fetch(:branch, "master") # can specify a branch from `cap -S branch="<branch_name>"`
set :scm, :git
set :scm_verbose, true
set :ssh_options, forward_agent: true
set :deploy_to, "/srv/rails"
set :deploy_via, :remote_cache
# We're using a rbenv user install, setup the PATH we need to access the rbenv shims
set :default_environment, {
'PATH' => "$HOME/.rbenv/shims:$HOME/.rbenv/bin:$PATH"
}
Has anyone seen this?
You should check the unicorn stdout/stderr logs for more evidence about why the old unicorn may be hanging or the new one failing to kill it off properly.
One gotcha is that if the older capistrano release directory has been removed during deployment of the new release, you may have bundler errors during the hot-swap handoff. Folks advise adding the following to bind to the permanent path to the Gemfile vs. the release-specific path:
before_exec do |server|
ENV['BUNDLE_GEMFILE'] = "#{deploy_path}/current/Gemfile"
end
If that is the problem you're having, you should be seeing bundler errors or a failure in the unicorn logs.
This may not help you, but here is what I did to "fix" the problem.
I started getting this problem from the release of Unicorn 4.7.0. In 4.7.0, the behavior of how pid files were written changed, and broke my restart script. The old pre-4.7.0 behavior was: move pid file to oldpid, write new pid, start up workers, shut down master. The last step was in my unicorn.rb file of course. The new behavior was to remove the old pid quickly, and only write the new one after some heavy lifting occurred. This broke my script as it could not trust that things restarted properly. This caused my sh script to attempt to restart it, leading to confusion with the now-freshly started unicorn process and the eh-script started "full start" one. Both lost in various ways, so both exited, leaving a old master still serving requests.
I also had a defect in my unicorn.rb file which did not properly set up bundler, as someone already mentioned.
Upgrading to Unicorn 4.8.1, released recently, fixed this problem as pid files are written as they were in pre-4.7.0 days.
I use rails 3.0.11, ruby 1.9.3-p0, nginx 1.0.4 and unicorn 3.6.2 for my project. And I have got a problem.
I have to do long-term operation on my server. It's about 150 seconds. And it's okay in this case.
I've set up my nginx config in location
proxy_read_timeout 240;
proxy_send_timeout 240;
And set up my unicorn.rb file with command
timeout 240
But I always get 502 bad gateway error.
I think, problem with unicorn. I get this unicorn logs
E, [2012-05-21T11:52:21.052382 #30423] ERROR -- : worker=1 PID:30871 timeout (104.052329915s > 60s), killing
E, [2012-05-21T11:52:21.080378 #30423] ERROR -- : reaped #<Process::Status: pid 30871 SIGKILL (signal 9)> worker=1
I, [2012-05-21T11:52:21.105045 #30423] INFO -- : worker=1 spawning...
I, [2012-05-21T11:52:21.111148 #894] INFO -- : worker=1 spawned pid=894
I, [2012-05-21T11:52:21.111659 #894] INFO -- : Refreshing Gem list
Can you help me? Any help is appreciated. Thank you.
Copying the answer from the comments in order to remove this question from the "Unanswered" filter:
I have never used this gem, but if you're doing this after
'deploy:restart', 'unicorn:reload' you need to restart unicorn, not
only reload it. sudo /etc/init.d/unicorn restart and the timeout will
be set. Reload and restart are two different things in unicorn.
~ answer per Maurício Linhares
After chaning timeout in config/unicron/production.rb
I had to run
cap deploy
and then stop & start unicron master process to pick up new config with:
cap unicorn:stop
cap unicorn:start
I am trying to use foreman to start my rails app. Unfortunately I have difficulties connecting my IDE for debugging.
I read here about using
Debugger.wait_connection = true
Debugger.start_remote
to start a remote debugging session, but that does not really work out.
Question:
Is there a way to debug a rails (3.2) app started by foreman? If so, what is the approach?
If you use several workers with full rails environment you could use the following initializer:
# Enabled debugger with foreman, see https://github.com/ddollar/foreman/issues/58
if Rails.env.development?
require 'debugger'
Debugger.wait_connection = true
def find_available_port
server = TCPServer.new(nil, 0)
server.addr[1]
ensure
server.close if server
end
port = find_available_port
puts "Remote debugger on port #{port}"
Debugger.start_remote(nil, port)
end
And in the foreman's logs you'll be able to find debugger's ports:
$ foreman start
12:48:42 web.1 | started with pid 29916
12:48:42 worker.1 | started with pid 29921
12:48:44 web.1 | I, [2012-10-30T12:48:44.810464 #29916] INFO -- : listening on addr=0.0.0.0:5000 fd=10
12:48:44 web.1 | I, [2012-10-30T12:48:44.810636 #29916] INFO -- : Refreshing Gem list
12:48:47 web.1 | Remote debugger on port 59269
12:48:48 worker.1 | Remote debugger on port 41301
Now run debugger using:
rdebug -c -p [PORT]
One approach is to require debugger normally in your gemfile, and add debugger normally in your code as needed. When the server hits that line, it will stop, but foreman won't be verbose about it. In your foreman console you can blindly type irb, and only then will you see a prompt appear. Bad UX, right?
Another (augmentative) approach is to tail your logs:
tail -f log/development.log
Hope this helps.
I want to do a performance test for my rails 3 app and I did a try according to the rails online guide
rake test:profile
and it gave some output as:
Specify ruby-prof as application's dependency in Gemfile to run benchmarks.
Loaded suite /usr/local/lib/ruby/gems/1.9.1/gems/rake-0.9.2/lib/rake/rake_test_loader
Started
.
Finished in 0.326233 seconds.
1 tests, 0 assertions, 0 failures, 0 errors, 0 skips
But according to the guide, there should be something like:
BrowsingTest#test_homepage (31 ms warmup)
wall_time: 6 ms
memory: 437.27 KB
objects: 5,514
gc_runs: 0
gc_time: 19 ms
and also some log files produced in app's tmp/performance dir, which doesn't exist in my case.
The performance test is the generated sample test, browsing_test.rb, in my app's test\performance dir:
require 'test_helper'
require 'rails/performance_test_help'
# Profiling results for each test method are written to tmp/performance.
class BrowsingTest < ActionDispatch::PerformanceTest
def test_homepage
get '/'
end
end
And my rails version is 3.0.10. Can anyone give me some tips or hints?
Had the exact same issue, and solved it by running
rvm install 1.9.2-p290 --patch gcdata # Make sure you have the same patch version
(note: not sure how much that step helped since I had an error, but since it's part of what I did...)
Adding to my gemfile
gem 'ruby-prof', :git => 'git://github.com/wycats/ruby-prof.git'
gem 'test-unit'
And of course running
bundle install
Some reading that helped me
http://rubylearning.com/blog/2011/08/14/performance-testing-rails-applications-how-to/
https://rails.lighthouseapp.com/projects/8994/tickets/4171-fresh-rails-3-app-cannot-run-performance-tests
http://guides.rubyonrails.org/performance_testing.html#installing-gc-patched-mri