I have been trying to see if this is possible and so far have found nothing so I will try and ask specifically
Is it possible to have a sidekiq worker which can recive a method as for example a lambda method and pass on arguments to it?
Example case:
I need to make some heavy computation on my server and my options are to either make a specific sidekiq worker for the job which will only be done 1 time ever and will end up cloddering my code base, or make a worker which could lets say accept something like:
lot_of_work.each do |args|
Workers::Tmp::LetsGo.perform_async(args) { |a| a.lets_go }
end
I've tried looking through old stackoverflow posts and documentation for sidekiq.
I've tried the above method which I hoped worked as a normal method but it does not.
I would have liked it to execute the method which was pass to the worker such that I do not need to make workers for 1 time cases and dont have to use single thread computation.
I found a solution to this problem, there are probably better ones but this worked for me.
Make a worker like this:
module Workers
module Default
class TesterWorker
include Sidekiq::Worker
sidekiq_options queue: :default, retry: false
def perform(method_name, method)
eval(method)
send(method_name)
end
end
end
end
After this you simply just have to write your code as a string like this:
methode_name = 'tester'
spec = "def test; puts 1; end"
Workers::Default::TesterWorker.perform_async(methode_name, spec)
And this will execute the for example the puts 1 action on the sidekiq ^^
Related
I have some methods that works with API of third party app. To do it on button click is no problem, but it should be permanent process.
How to run them background? And how to pause the cycle for make some other works with same API and resume the cycle after the job is done.
Now I read about ActiveJob, but its has time dependences only...
UPDATE
I've tried to make it with whenever and sidekiq, task runs, but it do nothing. Where to look for logs I can't understand.
**schedule.rb**
every 1.minute do
runner "UpdateWorker.perform_async"
end
**update_worker.rb**
class UpdateWorker
include Sidekiq::Worker
include CommonMods
def perform
logger.info "Things are happening."
logger.debug "Here's some info: #{hash.inspect}"
myMethod
end
def myMethod
....
....
....
end
end
It's not exactly what I need, but better then nothing. Can somebody explain me with examples?
UPDATE 2 After manipulating with code it's absolutely necessary to restart sidekiq . With this problem is solved, but I'm not sure that this is the best way.
You can define a job which enqueues itself:
class MyJob < ActiveJob::Base
def perform(*args)
# Do something unless some flag is raised
ensure
self.class.set(wait: 1.hour).perform_later(*args)
end
end
There are several libraries to schedule jobs on a regular basis. For example you could use to sidekiq-cron to run a job every minute.
If you want to pause it for some time, you could set a flag somewhere (Redis/database/file) and skip execution as long it is detected.
On a somewhat related note: don't use sidetiq. It was really great but it's not maintained anymore and has incompatibilities to current Sidekiq versions.
Just enqueue next execution in ensure section after job completes after checking some flag that indicates that it should.
Also i recommend adding some delay there so that you don't end up with dead loop on some error inside job
I dont know ActiveJobs, but I can recommend the whenever gem to create cron (periodic background) jobs. Basically you end up writing a rake tasks. Like this:
desc 'send digest email'
task send_digest_email: :environment do
# ... set options if any
UserMailer.digest_email_update(options).deliver!
end
I never added a rake task to itself but for repeated processing you could do somehow like this (from answers to this specific question)
Rake::Task["send_digest_email"].execute
I have a rails app in which I would like to use the delayed_jobs gem to send texts/emails in background processes at certain times of the day. This is how I have the relevant parts of my app set up right now:
class SomeClass
after_create :send_reminder
def when_to_run
self.date_time - 1.hour
end
def send_reminder
MessageHandler.new().send_message
end
handle_asynchronously :send_reminder, run_at: Proc.new { |i| i.when_to_run }, queue: "Messages"
end
The MessageHandler class is a separate class I've defined which actually houses the methods for sending texts (with Twilio) and emails (with Mailgun).
After starting delayed_job (bin/delayed_job start) and creating an instance of SomeClass, the delayed_job log reads as follows:
Job SomeClass#send_reminder_without_delay (id=686) RUNNING
I'm not sure why it is running send_reminder_without_delay, and it's doing it every time. I've tried using MessageHandler.new().delay.send_message in the send_reminder method instead of MessageHandler.new().send_message, but that hasn't gotten me anywhere either.
I've searched high and low for answers and keep coming up short - any help would be much appreciated!
I think your problem is in run_at: Proc.new { |i| i.when_to_run }.
I'm not entirely sure what the date_time method does but have you checked whether it actually returns time that is more than 1 hour in the future?
Otherwise the run_At will receive a time in the past and run the method.
if the return value of date_time is similar to this:
def when_to_run
Time.now - 1.hour
end
Then the queue will run as soon as you start delayed job.
Check if it your run_at time is in the future.
We were using Rails 4.2 and set the queue adapter. We needed to use this line in our config:
config.active_job.queue_adapter = :delayed_job
I have an app with both sidekiq and delayed job gems installed. When I trigger handle_asynchronously in active record models it appear to be handled by sidekiq while I would like to trigger delayed_job.
Is there a way to desactivate sidekiq for a specific model?
UPDATE:
Sidekiq now provides ways to either disable its delay module completely or alias it as sidekiq_delay. Please check this to see how to do it. https://github.com/mperham/sidekiq/wiki/Delayed-Extensions#disabling-extensions
For older version of sidekiq:
I use this monkey patch to make it so that calling .sidekiq_delay() goes to sidekiq and .delay() is goes to DelayedJob. According the answer by Viren, I think this may also solve your problem.
The patch is less complex (just a bunch of aliases), and gives you the power to consciously decide which delay you are actually calling.
As I mention in the comment In order to get it working you have to redefine/basically monkey patch the handle_asynchronously method something like this
Anywhere you like (but make sure it loaded )
in your config/initializers/patch.rb the code look like this
module Patch
def handle_asynchronously(method, opts = {})
aliased_method, punctuation = method.to_s.sub(/([?!=])$/, ''), $1
with_method, without_method = "#{aliased_method}_with_delay#{punctuation}", "#{aliased_method}_without_delay#{punctuation}"
define_method(with_method) do |*args|
curr_opts = opts.clone
curr_opts.each_key do |key|
if (val = curr_opts[key]).is_a?(Proc)
curr_opts[key] = if val.arity == 1
val.call(self)
else
val.call
end
end
end
## Replace this with other syntax
# delay(curr_opts).__send__(without_method, *args)
__delay__(curr_opts).__send__(without_method, *args)
end
alias_method_chain method, :delay
end
end
Module.send(:include,Patch)
And I believe rest all will follow then they way it should :)
Reason:
Delayed::Job include delay method on Object and Sidekiq include it delay method over ActiveRecord
Hence when the class try to invoke delay it look up it ancestors class (including the Eigen Class)
and it find the method define or included in ActiveRecord::Base class (which is sidekiq delay)
why does __delay__ work because alias define the copy of the existing method which is delay method of DelayedJob , hence when you invoke the __delay__ method it invoke delay method define DelayedJob
include to Object
Note:
Although the solution is bit patch but the it works . Keeping in mind that every direct .delay methid invocation is invoking delay method of the SideKiq and not DelayedJob to invoke the DelayedJob delay method you always has call it this way __delay__
Suggestion :
Monkey Patching is just a bad practice on my personal note I would rather not use 2 entirely different background processing library for a single application to achieve the same task. If the task is process thing in background why cant it be done with a single library either delayed_job or sidekiq (why it is that you required both of them )
So the point and to simply thing make your background processing an ease with respect to future I sincerely advice you take any one of the two library for background processing and I feel that would the valid answer for your question instead of monkey patching an doing other crazy stuff
Hope this help
I've a ruby script that has been implemented as an independent functionality.
Now I would like to execute this script in my rails environament, with the added difficulty of executing it as a background job, because it needs a great amount of time processing.
After adding the delayed_job gem, I've tryied calling the following sentence:
delay.system("ruby my_script.rb")
And this is the error I get:
Completed 500 Internal Server Error in 95ms
TypeError (can't dump anonymous module: #<Module:0x007f8a9ce14dc0>):
app/controllers/components_controller.rb:49:in `create'
Calling the self.delay method from your controller won't work, because DJ will try to serialize your controller into the Job. You'd better create a class to handle your task then flag its method as asynchronous :
class AsyncTask
def run
system('ruby my_script.rb')
end
handle_asynchronously :run
end
In your controller :
def create
...
AsyncTask.new.run
...
end
See the second example in the "Queing Jobs" section of the readme.
Like Jef stated the best solution is to create a custom job.
The problem with Jef's answer is that its syntax (as far as I know) is not correct and that's his job handles a single system command while the following will allow you more customization:
# lib/system_command_job.rb
class SystemCommandJob < Struct.new(:cmd)
def perform
system(cmd)
end
end
Note the cmd argument for the Struct initializer. It allows you to pass arguments to your job so the code would look like:
Delayed::Job.enqueue(SystemCommandJob.new("ruby my_script.rb"))
I am running a delayed job worker. When ever I invoke the foo method, worker prints hello.
class User
def foo
puts "Hello"
end
handle_asynchronously :foo
end
If I make some changes to the foo method, I have to restart the worker for the changes to reflect. In the development mode this can become quite tiresome.
I am trying to find a way to reload the payload class(in this case User class) for every request. I tried monkey patching the DelayedJob library to invoke require_dependency before the payload method invocation.
module Delayed::Backend::Base
def payload_object_with_reload
if Rails.env.development? and #payload_object_with_reload.nil?
require_dependency(File.join(Rails.root, "app", "models", "user.rb"))
end
#payload_object_with_reload ||= payload_object_without_reload
end
alias_method_chain :payload_object, :reload
end
This approach doesn't work as the classes registered using require_dependency needs to be reloaded before the invocation and I haven't figured out how to do it. I spent some time reading the dispatcher code to figure out how Rails reloads the classes for every request. I wasn't able to locate the reload code.
Has anybody tried this before? How would you advise me to proceed? Or do you have any pointers for locating the Rails class reload code?
I managed to find a solution. I used ActiveSupport::Dependencies.clear method to clear the loaded classes.
Add a file called config/initializers/delayed_job.rb
Delayed::Worker.backend = :active_record
if Rails.env.development?
module Delayed::Backend::Base
def payload_object_with_reload
if #payload_object_with_reload.nil?
ActiveSupport::Dependencies.clear
end
#payload_object_with_reload ||= payload_object_without_reload
end
alias_method_chain :payload_object, :reload
end
end
As of version 4.0.6, DelayedJob reloads automatically if Rails.application.config.cache_classes is set to false:
In development mode, if you are using Rails 3.1+, your application code will automatically reload every 100 jobs or when the queue finishes. You no longer need to restart Delayed Job every time you update your code in development.
This looks like it solves your problem without the alias_method hackery:
https://github.com/Viximo/delayed_job-rails_reloader