I try to move all the logging of my rails (3.2.17) app in production and staging to papertrail. Now I'm trying to set the logger for sidekiq in config/initializers/sidekiq.rb like this:
Sidekiq.configure_server do |config|
Sidekiq::Logging.logger = RemoteSyslogLogger.new('logs.papertrailapp.com', ENV.fetch('PAPERTRAIL_PORT'), program: "sidekiq-#{Rails.env}")
end
When I try to start sidekiq with:
bundle exec sidekiq --index 0 --pidfile <PATH_TO_PID> --environment staging --daemon
I get
You really should set a logfile if you're going to daemonize
...bundle/ruby/2.0.0/gems/sidekiq-2.17.4/lib/sidekiq/cli.rb:141:in `daemonize'
...bundle/ruby/2.0.0/gems/sidekiq-2.17.4/lib/sidekiq/cli.rb:39:in `parse'
...bundle/ruby/2.0.0/gems/sidekiq-2.17.4/bin/sidekiq:7:in `<top (required)>'
...bundle/ruby/2.0.0/bin/sidekiq:23:in `load'
...bundle/ruby/2.0.0/bin/sidekiq:23:in `<main>'
However if I give sidekiq a dummy for logging first:
bundle exec sidekiq --index 0 --pidfile <PATH_TO_PID> --environment staging --daemon --logfile /dev/null
it works like a charm (logs are sent to papertrail), since the initializer seems to override the previous option.
But I find this approach quite ugly. Anybody got a cleaner approach to this?
I discussed it with the owner of sidekiq and he made a good point: from the time the process is started until the time the initializer has run, a lot of things can go wrong. If the logger is only initialized in the initializer, everything that happended before, won't appear in any log.
I probably will go with the appoach recommended by the sidekiq owner:
bundle exec sidekiq | logger -t sidekiq
Related
I have a Rails 5 API with Sidekiq and capistrano-sidekiq that has been happily working fine for the last few months.
The other day, Sidekiq stopped processing jobs. Checking the logs, I saw
bundler: failed to load command: sidekiq (/home/user/project/shared/bundle/ruby/2.2.0/bin/sidekiq)
SignalException: SIGHUP
/home/user/project/shared/bundle/ruby/2.2.0/gems/activesupport-5.0.0.1/lib/active_support/core_ext/module/attribute_accessors.rb:119:in `<class:Module>'
/home/user/project/shared/bundle/ruby/2.2.0/gems/activesupport-5.0.0.1/lib/active_support/core_ext/module/attribute_accessors.rb:6:in `<top (required)>'
... (snip)
Whenever I try to start up Sidekiq, the above appears in the log. It was shut down using the quiet command (USR1) and exited properly.
INFO: Received USR1, no longer accepting new work
I'm using Capistrano to deploy, which has worked fine until this happened. This is the command Capistrano used to start Sidekiq:
INFO [2aac3b89] Running $HOME/.rbenv/bin/rbenv exec bundle exec sidekiq --index 0 --pidfile /home/user/project/shared/tmp/pids/sidekiq-0.pid --environment production --logfile /home/user/project/shared/log/sidekiq.log --daemon as user#xxx.xxx.xxx.xxx
DEBUG [2aac3b89] Command: cd /home/user/project/current && ( export RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.2.3" ; $HOME/.rbenv/bin/rbenv exec bundle exec sidekiq --index 0 --pidfile /home/user/project/shared/tmp/pids/sidekiq-0.pid --environment production --logfile /home/user/project/shared/log/sidekiq.log --daemon )
INFO [2aac3b89] Finished in 1.176 seconds with exit status 0 (successful).
What is going on? And how can I ensure it doesn't happen in future?
In capistrano, I had :pty set to true. I guess it was killing the process before it had a chance to start up. I still am not sure why this is an issue now, but setting :pty to false seems to have done the trick.
I was watching the RailsCast on Sidekiq and had some questions:
1) Sidekiq handles tasks through threads instead of processes. What does this mean? Why does it save on memory?
2) Does the method inside the worker class need to have a "perform" method?
3) On the Sidekiq docs, it says:
Start sidekiq from the root of your Rails application so the jobs
will be processed:
bundle exec sidekiq
So if I'm running this on localhost, I can run bundle exec sidekiq. If I pushed up to Heroku, what do I do now? How do I run Sidekiq on Heroku?
4) I am not sure if my Sidekiq is working. I have this code:
def set_defaults
self.clicks = 0 if clicks.blank?
self.title = TitleWorker.perform_async(orig_url)
end
But TitleWorker.perform_asynch(orig_url) in testing just seems to return a string of numbers. What is going on? How do I fix this?
While I'm not super clear about the first question, I might be able to answer the other questions you have:
Yes. It needs a Perform method to start the worker process.
Heroku is just amazing. You don't have to do any special configuration on Heroku for running Sidekiq. All you have to do is, just make sure RedisToGo is installed, and add this to the Procfile:
worker: bundle exec rake jobs:start
sidekiq: bundle exec sidekiq -c 15 -v
I've been struggling for two days with getting sidekiq work on heroku production environment. I've read all the available documentation about similar issues, and still haven't been able to produce a working solution, I'd really like some help!
After deploying on heroku, my app crashes and I get the following error stack trace:
2014-09-23T23:38:40.905093+00:00 app[worker.1]: No such file or directory # rb_sysopen - /app/tmp/pids/sidekiq.pid
2014-09-23T23:38:40.905122+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/bin/sidekiq:23:in `<main>'
2014-09-23T23:38:40.905119+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/bin/sidekiq:7:in `<top (required)>'
2014-09-23T23:38:40.905117+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/lib/sidekiq/cli.rb:347:in `write_pid'
2014-09-23T23:38:40.905115+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/lib/sidekiq/cli.rb:347:in `open'
2014-09-23T23:38:40.905121+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/bin/sidekiq:23:in `load'
2014-09-23T23:38:40.905118+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/lib/sidekiq/cli.rb:41:in `parse'
2014-09-23T23:38:40.905114+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/lib/sidekiq/cli.rb:347:in `initialize'
2014-09-23T23:38:39.588001+00:00 heroku[worker.1]: State changed from starting to up
First, sidekiq is working correctly on my local machine. I'm using heroku's REDISTOGO for redis, on local production, sidekiq have been pointed to the REDISTOGO correctly and running fine.
Second, according to the stack trace, especially this line No such file or directory # rb_sysopen - /app/tmp/pids/sidekiq.pid; it leads me to think that for some reason sidekiq.pid file is not generated correctly when running on heroku. On local environment, the sidekiq.pid file is generated everytime I start the app in the app/tmp/pids/ directory, and assigns a different pid number every time. I'm guessing that when running on heroku, sidekiq tried to read from this file yet could not find it.
Here is contents in my Procfile:
web: bundle exec rails server
worker: bundle exec sidekiq -C config/sidekiq.yml
Here is contents in my config/sidekiq.yml
---
:verbose: true
:pidfile: ./tmp/pids/sidekiq.pid
:concurrency: 25
# Set timeout to 8 on Heroku, longer if you manage your own systems.
:timeout: 8
:queues:
- carrierwave
Here is contents in my sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => ENV['REDISTOGO_URL'], :namespace => "mynamespece"}
end
Sidekiq.configure_client do |config|
config.redis = { :url => ENV['REDISTOGO_URL'], :namespace => "mynamespece"}
end
Update 1:
I'm using carrierwave and carrierwave-backgrounder in sync with sidekiq.
This issue has been resolved by the following actions:
1) Thanks to #MikePerham for pointing me to the right direction, first I removed this line in my sidekiq.yml file:
:pidfile: ./tmp/pids/sidekiq.pid
2) Then, in my Procfile, I had to use the following line to replace the origin:
web: bundle exec rails server -p $PORT
worker: bundle exec sidekiq -C config/sidekiq.yml
Now sidekiq is working correctly with redistogo on heroku for me.
I found it useful to add the directory to the repository and re-deploy the application.
mkdir -p tmp/pids
touch tmp/pids/.gitkeep
git add -f tmp/pids/.gitkeep
git commit -m 'Keep tmp/pids directory in repo'
Hope this helps.
It was fixed by making pids directory in tmp directory in root of project
I'm having a problem getting Sidekiq up and running on my Heroku deployed Rails app. I have my app working fine in development (and on Heroku without Sidekiq).
I created a Procfile with:
worker: bundle exec sidekiq
If I run heroku ps, the only process I see is web.1.
Should I see one for
Sidekiq?
I do get an error:
Redis::CannotConnectError (Error connecting to Redis on localhost:6379) in my Heroku logs.
UPDATE: Found I probably needed heroku addons:add redistogo. Still not working. I feel I'm missing some basic configuration.
Is there something I need to do to get Redis up and running for my Heroku app?
I've been using Redis/Sidekiq for about a day, so this is new to me.
Thanks!
Greg
No you do not need any config with Heroku for Sidekiq, just add the RedisToGo plugin and you're on.
Do not forget to attribute at least 1 worker to your app in your Heroku config.
Here is my default Procfile:
web: bundle exec thin start -p $PORT
worker: bundle exec sidekiq -c 5 -v
It's worth checking if the sidekiq process is really started with this command:
heroku ps
If there's no worker, then you might need to run this command:
heroku ps:scale worker+1
It turns out that there's a bug in the web UI in that some team members were not allowed to increase the number of workers from 0 to 1, even though the UI seemed to show that!
Starting with sidekiq version 3.0 there is an additional step, run heroku config:set REDIS_PROVIDER=REDISTOGO_URL in the console.
Here is the process I used for Rails 4:
In the console:
heroku addons:create redistogo
heroku config:set REDIS_PROVIDER=REDISTOGO_URL
In my Procfile I added:
worker: bundle exec sidekiq
In my gemfile.rb I added:
gem 'redis'
I added the following file, config/initializers/redis.rb:
uri = ENV["REDISTOGO_URL"] || "redis://localhost:6379/"
REDIS = Redis.new(:url => uri)
Here is the link to the sidekiq docs.
Complementing gdurelle answer:
You do need some config with Heroku for Sidekiq:
1) Have the Sidekiq and Redis gems installed (in gemfile and bundled), in my case:
Gemfile
gem 'redis', '~> 3.1'
gem 'sidekiq', '~> 2.7', '>= 2.7.1'
2) Add a worker, if you don't have any workers created locally I suggest you create at least one, just in case, use this:
rails g sidekiq:worker Hard # will create app/workers/hard_worker.rb
should create this:
app/workers/hard_worker.rb
class HardWorker
include Sidekiq::Worker
def perform(name, count)
# do something
end
end
3) Add the Redis add-on (in my case Heroku Redis):
heroku addons:create heroku-redis:hobby-dev
4) Add your redis.rb file, in my case:
config/initializers/redis.rb
$redis = Redis.new(url: ENV["REDIS_URL"])
5) Add Procfile or config/sidekiq.yml or both, here are mine:
Procfile
worker: bundle exec sidekiq -c 1 -q default -q mailers
which you can create easier by pasting this in your terminal
echo "worker: bundle exec sidekiq -c 1 -q default -q mailers" > Procfile
config/sidekiq.yml
:concurrency: 1
:queues:
- [mailers, 7]
- [default, 5]
6) Most important part go here:
and turn on the switch for your worker, click on the pencil and then turn on the missing switch. Things should be working fine now, have a great day!
What I found out is that you have to scale process manually like so:
heroku ps:scale worker+1
Makes no sense since my Procfile said:
web: bundle exec....
worker: bundle exec sidekiq
...and one would've expectd Heroku to start the worker automatically. In the end I didn't have to scale the web process...
Also, you might have problems with this line:
worker: bundle exec sidekiq
Add flags for concurency:
worker: bundle exec sidekiq -c 5 -v
Sounds like you're almost there. It may be that you just need to set REDISTOGO_URL on the heroku config?
heroku config
should show you the redistogo value?
If you copy that to REDISTOGO_URL then sidekiq should work?
heroku config:add REDISTOGO_URL=<redistogo value>
Edit:
Sidekiq will use any of these currently: https://github.com/mperham/sidekiq/blob/master/lib/sidekiq/redis_connection.rb#L29-L33
Edit2:
Greg is correct in that you don't need to add the config if you're using RedisToGo. But if you're using OpenRedis, or other Redis providers, then you need to add REDISTOGO_URL for Sidekiq
You can of course run sidekiq alongside your current jobs queue.
Our Procfile currently looks like this:
web: bundle exec thin start -R config.ru -e $RAILS_ENV -p $PORT
worker: bundle exec rake jobs:work
sidekiq: bundle exec sidekiq -c 5 -v
I have a locally developed Rails application. It is very simple, with no database but only a controller that accesses Facebook data and renders them in view. By the way, gem RestGraph is used.
Before I push this application to heroku, I used foreman start to test it. Since I was using WEBrick before I need to install 'thin' gem and create a Procfile which reads:
web: bundle exec thin start -R config.ru -e $RACK_ENV -p $PORT
an '.env' file seems also required, which reads:
RACK_ENV=development
PORT=3000
The 'config.ru' file is generate by rails, which reads:
require ::File.expand_path('../config/environment', __FILE__)
run Myapp::Application
Now I entered 'foreman start', but all I get is one line:
15:33:18 web.1 | started with pid 27143
The server will not boot. And if I force terminate it, the error is:
/usr/local/foreman/lib/foreman/engine.rb:141: [BUG] rb_sys_fail() - errno == 0
Which is not very helpful.
It may be that instead of being "stuck," your log is just being buffered: foreman only shows line with “started wit pid #” and nothing else
I solved that issue by adding
$stdout.sync = true
to config.ru
Then you might get more helpful feedback from Foreman.
This seems a little over complex. From looking at a couple of apps I have I have the following in my procfile:
bundle exec rails server thin -p $PORT
I also don't have a .env file in these instances.
One thing worth noting is that Foreman will ignore the port you pass in most of the time, it likes to sit in a port range higher than 5000