I have a rails 4.1 application with Sidekiq 3.3.2 in production mode with default config.
Everything works fine, but sometimes, I see in Sidekiq admin panel that Sidekiq process, not listed in the "busy" in the panel and does not exist in the list of processes.
That seems like a sidekiq's process has been fail.
In the log there is no error entries.
How can I make that stable?
This is known problem. Hard loaded Sidekiq worker is disappear sometimes from busy list but still working fine.
I known issue marked as fixed but seems like it appear again and again. Just don't worry about it and use ps aux | grep sidekiq to check running Sidekiq workers in this case.
Related
I am using this command
bundle exec sidekiq -d
to run sidekiq server on the background. getting this error message
ERROR: Daemonization mode was removed in Sidekiq 6.0, please use a proper process supervisor to start and manage your services.
sidekiq run but not in the background. After closing the console sidekiq automatically close.
You may also think about using process manager like overmind which will help you manage multiple processes (for instance server and sidekiq)
https://github.com/DarthSim/overmind
There are other tools around the web, this is my personal choice.
You need to open another terminal tab, in ubuntu ctrl + shift + T and run command
bundle exec sidekiq start
It is removed from the latest versions of Sidekiq to promote users to learn the newer, better ways. Here is the link to the discussion on the same.
The discussion suggested using a process supervisor like systemd, upstart, foreman, etc. to manage Sidekiq.
So you need to write your own service file to start, stop sidekiq. For the reference, here is the link to example service of sidekiq.
https://github.com/mperham/sidekiq/blob/master/examples/systemd/sidekiq.service
You didn't mention the operating system so I'll just go with ubuntu production VM. You're going to want to setup sidekiq with something like systemd or upstart. Sidekiq has some example configurations to get you started https://github.com/mperham/sidekiq/tree/master/examples.
I haven't done this on a mac before, but a quick google and found this Start sidekiq automatically on OSX.
Using Rails(4.0.1) and delayed_job_active_record(4.0.1) for background tasks in my application. DJ works fine only when I use rake jobs:work in production mode. But I want it has to run daemon process, also this process no more alive after sometime.
If I run the bin/delayed_job start RAILS_ENV=production. I can able to the pid file in tmp/pids/delayed_job.pid. The process is in alive. But nothing is working. Any clue on this issue?
May be delayed_jobs are failing due to some reason/error. Try installing dj_mon (https://github.com/akshayrawat/dj_mon) in rails application and view jobs log.
If delayed jobs services is shutting down due to memory leakage or some unknown reason, you may try to install and configure 'monit' scripts on your server. It will restart delayed jobs service automatically.
I have a Reporter worker for Resque in my Rails 3.2.8 application. I frequently add new reports for users, or fix bugs in existing reports.
Reports are deployed as Ruby modules whose methods are called by the Resque reporter worker.
Every time I deploy new code, I have to restart Resque. During that time, there are often one or more reports out there that are then killed, left with a status of "Running". What I want to find out is, is there a way to get Resque to reload the ruby modules that it uses to run the reports?
Instead of reloading you could stop the resque workers with a kill -s QUIT. That will cause the workers to finish running their reports before shutting down.
More info on using signals with resque is here, https://github.com/defunkt/resque#signals.
In my production server, I'm using ruby foreman to run multiple processes, I just want my application to keep working, even if one of the processes down, I want my processes to keep working even if one down , is there any tricky way to restart the process or even not to stop all the processes in case one went down ? I mean in production level I want the solution to be stable enough, is that possible without Upstart ? thanks in advance
You should not be using foreman itself for production - it is only intended as a development tool. Instead, you can use something like god with my foreman_god gem in production.
Alternatively, you can use foreman to export config files for other process monitoring systems, for example upstart.
You can monitor your foreman process with another program like http://mmonit.com/monit/. But somehow you'll find that monitoring a process which monitor other processes is kinda strange.
I'm having ruby instances from mod_rails go "rogue" -- these processes are no longer listed in passenger-status and utilize 100% cpu.
Other than installing god/monit to kill the instance, can anyone give me some advice on how to prevent this? I haven't been able to find anything in the logs that helps.
If you're using Linux, you can install the "strace" utility to see what the Ruby process is doing that's consuming all the CPU. That will give you a good low-level view. It should be available in your package manager. Then you can:
$ sudo strace -p 22710
Process 22710 attached - interrupt to quit
...lots of stuff...
(press Ctrl+C)
Then, if you want to stop the process in the middle and dump a stack trace, you can follow the guide on using GDB in Ruby at http://eigenclass.org/hiki.rb?ruby+live+process+introspection, specifically doing:
gdb --pid=(ruby process)
session-ruby
stdout_redirect
(in other terminal) tail -f /tmp/ruby_debug.(pid)
eval "caller"
You can also use the ruby-debug Gem to remotely connect to debug sockets you open up, described in http://duckpunching.com/passenger-mod_rails-for-development-now-with-debugger
There also seems to be a project on Github concerned with debugging Passenger instances that looks interesting, but the documentation is lacking:
http://github.com/ddollar/socket-debugger/tree/master
I had a ruby process related to Phusion Passenger, which consumed lots of CPU, even though it should have been idle.
The problem went away after I ran
date -s "`date`"
as suggested in this thread. (That was on Debian Squeeze)
Apparently, the problem was related to a leap second, and could affect many other applications like MySQL, Java, etc. More info in this thread on lklm.
We saw something similar to this with very long running SQL queries.
MySQL would kill the queries because they exceeded the long running limit and the thread never realized that the query was dead.
You may want to check the database logs.
This is a recurring issue with passenger. I've seen this problem many times helping people that ran ruby on rails with passenger. I don't have a fix but you might want to try this http://www.modrails.com/documentation/Users%20guide%20Apache.html#debugging_frozen