I have an app with both sidekiq and delayed job gems installed. When I trigger handle_asynchronously in active record models it appear to be handled by sidekiq while I would like to trigger delayed_job.
Is there a way to desactivate sidekiq for a specific model?
UPDATE:
Sidekiq now provides ways to either disable its delay module completely or alias it as sidekiq_delay. Please check this to see how to do it. https://github.com/mperham/sidekiq/wiki/Delayed-Extensions#disabling-extensions
For older version of sidekiq:
I use this monkey patch to make it so that calling .sidekiq_delay() goes to sidekiq and .delay() is goes to DelayedJob. According the answer by Viren, I think this may also solve your problem.
The patch is less complex (just a bunch of aliases), and gives you the power to consciously decide which delay you are actually calling.
As I mention in the comment In order to get it working you have to redefine/basically monkey patch the handle_asynchronously method something like this
Anywhere you like (but make sure it loaded )
in your config/initializers/patch.rb the code look like this
module Patch
def handle_asynchronously(method, opts = {})
aliased_method, punctuation = method.to_s.sub(/([?!=])$/, ''), $1
with_method, without_method = "#{aliased_method}_with_delay#{punctuation}", "#{aliased_method}_without_delay#{punctuation}"
define_method(with_method) do |*args|
curr_opts = opts.clone
curr_opts.each_key do |key|
if (val = curr_opts[key]).is_a?(Proc)
curr_opts[key] = if val.arity == 1
val.call(self)
else
val.call
end
end
end
## Replace this with other syntax
# delay(curr_opts).__send__(without_method, *args)
__delay__(curr_opts).__send__(without_method, *args)
end
alias_method_chain method, :delay
end
end
Module.send(:include,Patch)
And I believe rest all will follow then they way it should :)
Reason:
Delayed::Job include delay method on Object and Sidekiq include it delay method over ActiveRecord
Hence when the class try to invoke delay it look up it ancestors class (including the Eigen Class)
and it find the method define or included in ActiveRecord::Base class (which is sidekiq delay)
why does __delay__ work because alias define the copy of the existing method which is delay method of DelayedJob , hence when you invoke the __delay__ method it invoke delay method define DelayedJob
include to Object
Note:
Although the solution is bit patch but the it works . Keeping in mind that every direct .delay methid invocation is invoking delay method of the SideKiq and not DelayedJob to invoke the DelayedJob delay method you always has call it this way __delay__
Suggestion :
Monkey Patching is just a bad practice on my personal note I would rather not use 2 entirely different background processing library for a single application to achieve the same task. If the task is process thing in background why cant it be done with a single library either delayed_job or sidekiq (why it is that you required both of them )
So the point and to simply thing make your background processing an ease with respect to future I sincerely advice you take any one of the two library for background processing and I feel that would the valid answer for your question instead of monkey patching an doing other crazy stuff
Hope this help
Related
I have been trying to see if this is possible and so far have found nothing so I will try and ask specifically
Is it possible to have a sidekiq worker which can recive a method as for example a lambda method and pass on arguments to it?
Example case:
I need to make some heavy computation on my server and my options are to either make a specific sidekiq worker for the job which will only be done 1 time ever and will end up cloddering my code base, or make a worker which could lets say accept something like:
lot_of_work.each do |args|
Workers::Tmp::LetsGo.perform_async(args) { |a| a.lets_go }
end
I've tried looking through old stackoverflow posts and documentation for sidekiq.
I've tried the above method which I hoped worked as a normal method but it does not.
I would have liked it to execute the method which was pass to the worker such that I do not need to make workers for 1 time cases and dont have to use single thread computation.
I found a solution to this problem, there are probably better ones but this worked for me.
Make a worker like this:
module Workers
module Default
class TesterWorker
include Sidekiq::Worker
sidekiq_options queue: :default, retry: false
def perform(method_name, method)
eval(method)
send(method_name)
end
end
end
end
After this you simply just have to write your code as a string like this:
methode_name = 'tester'
spec = "def test; puts 1; end"
Workers::Default::TesterWorker.perform_async(methode_name, spec)
And this will execute the for example the puts 1 action on the sidekiq ^^
My counter cache is locking the row under heavy load so I found wanelo/counter-cache gem which seems to be perfect for my problem but I can't set it up and it must be something really simple but I can't see it.
https://github.com/wanelo/counter-cache
I want it to use my already working delayed jobs and Redis.
In my config file
Counter::Cache.configure do |c|
c.default_worker_adapter = here??? DelayedJob ??
c.recalculation_delay = 5.hours
c.redis_pool = Redis.new
c.counting_data_store = Counter::Cache::Redis
end
If I don't put the line c.default_worker_adapter when executing it says
undefined method 'enqueue' for nil:NilClass
Any idea on what's going on? What should I put in the Worker Adapter? Nothing seems to work.
Thank you for your time
default_worker_adapter is the name of the class that will be handling your updates. An example is given on the github page of the gem. For example if you're using sidekiq, you would make sidekiq worker class and name it whatever you want. On the github page, this class is called CounterWorker and you can copy it exactly as its given, though you can use whatever delayed job framework you want. From then on, any counter_cache_on definitions on your models will use that class to make the counter updates.
I am using Rails 4 w/ the impressionist and resque gem.
I am using impressionist to log unique session hits on my article show page. Due to performance issues and no need to display hits to users (it is for admins only), I would like to move logging impressions off into the background.
Normally I would log an impression using impressionist(#article, unique: [:session_hash]) but to move it off into the bg via resque I am now doing something like this...
articles_controller:
def show
.
.
.
Resque.enqueue(ImpressionLogger, #article.id)
end
app/workers/impression_logger.rb:
class ImpressionLogger
#queue = :impression_queue
def self.perform(article_id)
article = Article.find(article_id)
impressionist(article, unique: [:session_hash])
end
end
When I set it up like this, when resque tries to process the job, it is returning undefined method "impressionist" for ImpressionLogger:Class. What do you guys think the best way to go about this is? I am not sure how to include impressionist methods inside of my resque worker.
The issue
Your problem stems from the fact that it looks like Impressionist works on the controller level due to including a module with the impressionist method in an engine initializer on any instances of ActionController:
https://github.com/charlotte-ruby/impressionist/blob/master/lib/impressionist/engine.rb#L11
You're trying to call the impressionist method from a regular class being invoked in a Resque job, so it's not going to have that method defined.
Solution
It's kind of gross, but if you really want to use impressionist, we can delve into this... Looking at the actual implementation of the impressionist method found here, we see the following:
def impressionist(obj,message=nil,opts={})
if should_count_impression?(opts)
if obj.respond_to?("impressionable?")
if unique_instance?(obj, opts[:unique])
obj.impressions.create(associative_create_statement({:message => message}))
end
else
# we could create an impression anyway. for classes, too. why not?
raise "#{obj.class.to_s} is not impressionable!"
end
end
end
Assuming that you'd be calling something like this manually (as you want to from a resque job) the key are these three lines:
if unique_instance?(obj, opts[:unique])
obj.impressions.create(associative_create_statement({:message => message}))
end
The if wrapper only seems to be important if you want to implement this functionality. Which it looks like you do. The call to associative_create_statement seems to be pulling parameters based off of the controller name as well as parameters passed from Rack such as the useragent string and ip address (here). So, you'll have to resolve these values prior to invoking the Resque job.
What I would suggest at this point is implementing a Resque class that takes in two parameters, an article_id and the impression parameters that you want. The resque class would then just directly create the impression on the impressionable object. Your Resque class would become:
class ImpressionLogger
#queue = :impression_queue
def self.perform(article_id, impression_params = {})
article = Article.find(article_id)
article.impressions.create(impression_params)
end
end
And your controller method would look something like this:
def show
.
.
.
Resque.enqueue(ImpressionLogger, #article.id, associative_create_statement({message: nil})) if unique_instance?(#article, [:session_hash])
end
Disclaimer
There's a fairly big disclaimer that comes with doing it this way though... the method associative_create_statement is marked protected and unique_instance? is marked private... so neither of these is part of the impressionist gem's public API, so this code might break between versions of the gem.
Is impressionist installed properly with bundler? If so Rails should be loading it into your environment. I would check whether you can access impressionist functionality elsewhere in your Rails code (i.e. without going through Resque) as the first step to debugging this.
How are you starting your resque workers? If you need your Rails environment loaded, try rake environment resque:work.
https://github.com/resque/resque/wiki/FAQ#how-do-i-ensure-my-rails-classesenvironment-is-loaded
I have a question about delayed_job in Rails that doesn't seem to be mentioned much.
When you run a delayed job, it doesn't seem to load anything from ApplicationController. We have some code in ApplicationController to use a custom logger:
def setup_logger
logfile = File.open("#{RAILS_ROOT}/log/audit.log", 'a')
#audit_log = Logger.new(logfile)
$audit_log = #audit_log
end
We then reference $audit_log all through our code. But because DelayedJob doesn't load the ApplicationController this variable is nil and we get errors.
So Delayed_job is just running the specified method blindly, which could also be dangerous
if you rely on before_filters for checking data or validating things.
How can we fix our problem of getting DelayedJob to know about our global logging variable? We don't want to explicitly define the logger all through our code.
How else are people dealing with this problem, as it seems like it should be common, but its not talked about much.
Thanks
Why you think, a Job must run ApplicationController ?
A Job, precisely the worker running that job, loads the environment, sure, but not a controller...
If you don't want have it in a initializer, why not using a kind of config object for storing such data and reference to it in the job ?
And instead of do it in each job individually, I would suggest set up the logger in the BaseJob, and use it in the inherited jobs for what you need.
And another suggestion, please if you can, don't use delayed job, please use http://mperham.github.io/sidekiq/ or at least https://github.com/resque/resque
I'm working to learn how to user delayed_job on my rails 3 + heroku app.
I currently have the following which emails on a request (not delayed job) but it works!
UserMailer.conversation_notification(record.commentable, participant, record, #comments).deliver
I updated that to this to start using delayed_job:
Delayed::Job.enqueue UserMailer.conversation_notification(record.commentable, participant, record, #comments).deliver
But that error'd with: "ArgumentError (Cannot enqueue items which do not respond to perform):"
I also tried:
UserMailer.delay.conversation_notification(record.commentable, participant, record, #comments)
But that error'd with:
NoMethodError (undefined method `delay' for UserMailer:Class):
Any delayed_job guru's out there? Thanks
From the docs https://github.com/collectiveidea/delayed_job
Your second method was correct which removes the .deliver method:
UserMailer.delay.conversation_notification(record.commentable, participant, record, #comments)
If you are getting an undefined method delay did you add DelayedJob to the Gemfile?
gem "delayed_job"
Since including the delayed_job will add the "delay" method to everything.
I have mixed results with using delay, and I've found it very challenging to debug. So you are not alone! But when you get it working, its worth it.
I've learned to save my object before calling delay on it. Typically I will trigger my job from an after_save call back.
As an experiment, for awhile I was using a different pattern. I'd create a job object for each job that I have. For example, I would call
Delayed::Job.enqueue(PersonJob.new(#person.id))
Elsewhere in my project I would create the job object. In Rails 2, I put these in lib/ if you do that with rails 3, you need to alter the application.rb config.autload_path
class PersonJob < Struct.new(:person_id)
def perform
person = Person.find(person_id)
#do work
end
end
config.autoload_paths += Dir["#{config.root}/lib/**/"]
I just had a look at the documentation, it's been a while since I actually used delayed_job...
Jobs are Ruby objects with a method called perform, so you'd need enqueue an object which does
UserMailer.conversation_notification(record.commentable, participant, record, #comments).deliver
in its perform method.
Alternatively, you can use send_later:
UserMailer.conversation_notification(record.commentable, participant, record, #comments).send_later(:deliver)