I'm using a before_restart.rb hook in opsworks and I have a problem when it run "rake i18n:js:export". I don't know why is running sidekiq with this rake. it fails only in setup stage of opsworks. When I deploy it the error disappears.
[2015-01-09T18:52:17+00:00] INFO: deploy[/srv/www/XXX] queueing checkdeploy hook /srv/www/XXX/releases/20150109185157/deploy/before_restart.rb
[2015-01-09T18:52:17+00:00] INFO: Processing execute[rake i18n:js:export] action run (/srv/www/XXXX/releases/20150109185157/deploy/before_restart.rb line 3)
Error executing action `run` on resource 'execute[rake i18n:js:export]'
Mixlib::ShellOut::ShellCommandFailed
Expected process to exit with [0], but received '1'
---- Begin output of bundle exec rake i18n:js:export ----
STDOUT: 2015-01-09T18:52:30Z 1808 TID-92c6g INFO: Sidekiq client with redis options {}
STDERR: /home/deploy/.bundler/XXXX/ruby/2.1.0/gems/redis-3.1.0/lib/redis/client.rb:309:in `rescue in establish_connection': Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED) (Redis::CannotConnectError)
Sidekiq client (NOT sidekiq server) is running because it is defined in an initializer. When rake runs, it loads the entire rails app environment. So either allow for an environment variable to disable sidekiq client in config/initializers/sidekiq.rb or make sure redis-server is properly configured on the instance you're running this on.
unless ENV['DISABLE_SIDEKIQ']
# Sidekiq.configure...
end
DISABLE_SIDEKIQ=true bundle exec rake do:stuff
Related
I'm trying to deploy an application using AWS OpsWorks with chef, I had have run the deploy in other times and had never failed but this time I got the next message in the log. I run the command bundle
exec rake assets:precompile] in localhost and everything is alright,
What can be?
[2018-03-01T18:50:54+00:00] INFO: Processing execute[cd
/srv/www/my_project/releases/20180301185045 && RAILS_ENV=production bundle
exec rake assets:precompile] action run
(/opt/aws/opsworks/releases/20160504095744_3437-
20160504095744/vendor/bundle/ruby/2.0.0/gems/chef-
11.10.4/lib/chef/provider/deploy.rb line 63)
Error executing action 'run' on resource 'execute[cd
/srv/www/my_project/releases/20180301185045 && RAILS_ENV=production bundle
exec rake assets:precompile]'
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '137'
---- Begin output of cd /srv/www/my_project/releases/20180301185045 &&
RAILS_ENV=production bundle exec rake assets:precompile ----
STDOUT:
Agreed that Opsworks will always run into out of memory state during deployment especially on instances (micro/small). SSH to the instance and make swap memory (e.g. 2GB / 4GB) will help to reduce the issue a lot.
Sometimes the instances got out of memory.
Only stop and start the instance where you are doing the deploy.
I have a Rails 5 API with Sidekiq and capistrano-sidekiq that has been happily working fine for the last few months.
The other day, Sidekiq stopped processing jobs. Checking the logs, I saw
bundler: failed to load command: sidekiq (/home/user/project/shared/bundle/ruby/2.2.0/bin/sidekiq)
SignalException: SIGHUP
/home/user/project/shared/bundle/ruby/2.2.0/gems/activesupport-5.0.0.1/lib/active_support/core_ext/module/attribute_accessors.rb:119:in `<class:Module>'
/home/user/project/shared/bundle/ruby/2.2.0/gems/activesupport-5.0.0.1/lib/active_support/core_ext/module/attribute_accessors.rb:6:in `<top (required)>'
... (snip)
Whenever I try to start up Sidekiq, the above appears in the log. It was shut down using the quiet command (USR1) and exited properly.
INFO: Received USR1, no longer accepting new work
I'm using Capistrano to deploy, which has worked fine until this happened. This is the command Capistrano used to start Sidekiq:
INFO [2aac3b89] Running $HOME/.rbenv/bin/rbenv exec bundle exec sidekiq --index 0 --pidfile /home/user/project/shared/tmp/pids/sidekiq-0.pid --environment production --logfile /home/user/project/shared/log/sidekiq.log --daemon as user#xxx.xxx.xxx.xxx
DEBUG [2aac3b89] Command: cd /home/user/project/current && ( export RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.2.3" ; $HOME/.rbenv/bin/rbenv exec bundle exec sidekiq --index 0 --pidfile /home/user/project/shared/tmp/pids/sidekiq-0.pid --environment production --logfile /home/user/project/shared/log/sidekiq.log --daemon )
INFO [2aac3b89] Finished in 1.176 seconds with exit status 0 (successful).
What is going on? And how can I ensure it doesn't happen in future?
In capistrano, I had :pty set to true. I guess it was killing the process before it had a chance to start up. I still am not sure why this is an issue now, but setting :pty to false seems to have done the trick.
I have two resque commands that I'd like to implement into capistrano so I can run it successfully on the server. I've checked by running these manually that they both work, however if I'm to keep these continuously running I'll end up with a broken pipe.
I'd like to be able to start resque:
queue=* rake environment resque:work
and start resque-scheduler:
rake environment resque:scheduler
anybody know how I can implement this into my deploy.rb file?
Try the capistrano-resque gem which should do exactly this (it includes support for resque-scheduler).
After setting it up, you'll get these Capistrano tasks:
➔ cap -vT | grep resque
cap resque:status # Check workers status
cap resque:start # Start Resque workers
cap resque:stop # Quit running Resque workers
cap resque:restart # Restart running Resque workers
cap resque:scheduler:restart #
cap resque:scheduler:start # Starts Resque Scheduler with default configs
cap resque:scheduler:stop # Stops Resque Schedule
(I currently help maintain this gem, so if you have any trouble with it just file an issue and I'll take a look).
It's my first attempt to get Redis working on Heroku.
I've added one worker dyno (just today, so didn't pay yet), added RedisToGo Nano add-on, tested background jobs on my local machine, and pushed the app to heroku.
heroku ps
gives
=== web: `bundle exec rails server -p $PORT`
web.1: up 2013/03/03 18:26:09 (~ 37m ago)
=== worker: `bundle exec rake jobs:work`
worker.1: crashed 2013/03/03 19:02:15 (~ 1m ago)
Sidekiq Web Interface says that one job is enqueued, but zero processed or failed.
I'm guessing it's because my worker dyno is crashed.
Are there any noob mistakes that I don't know about?
(e.g. I need to run some command to start listening to background jobs etc)
heroku logs --tail doesn't show any errors, so I don't understand why my worker dyno chashes.
I did some research and fixed it like this:
Under app's root directory I created a file called "Procfile" with this content:
web: bundle exec rails server -p $PORT
worker: bundle exec sidekiq -c 5 -v
Got this idea from here.
After that it worked ok.
Also make sure you setup REDIS_PROVIDER:
heroku config:set REDIS_PROVIDER=REDISTOGO_URL
Sidekiq's GitHub page also has instruction. Click here
I have rescue 1.22.0 installed locally and on a server. To be able to catch MultiJson::DecodeErrors I added the following to my application.rb:
config.middleware.swap ActionDispatch::ParamsParser, ::MyParamsParser
and added the class to my lib folder. In dev mode this works fine, I can rescue from DecodeErrors and I can start a worker using:
QUEUE=* bundle exec rake environment resque:work
In production mode on my server the code itself works as well, but my god process was not able to start workers again. The error that occurs after god starts a worker:
QUEUE=* /usr/local/rvm/rubies/ruby-1.9.2-p320/bin/ruby /usr/local/rvm/gems/ruby-1.9.2-p320#global/bin/bundle exec rake -f /home/deployer/apps/kassomat/current/Rakefile environment resque:work
rake aborted!
No such middleware to insert before: ActionDispatch::ParamsParser
I tried to patch my application.rb
config.middleware.swap ActionDispatch::ParamsParser, ::MyParamsParser if Object.const_defined?('ActionDispatch') && ActionDispatch.const_defined?('ParamsParser')
but that did not work out. I do not understand why this works in development but fails in production.
Can anyone help?
Regards
Felix