I've a ruby script that has been implemented as an independent functionality.
Now I would like to execute this script in my rails environament, with the added difficulty of executing it as a background job, because it needs a great amount of time processing.
After adding the delayed_job gem, I've tryied calling the following sentence:
delay.system("ruby my_script.rb")
And this is the error I get:
Completed 500 Internal Server Error in 95ms
TypeError (can't dump anonymous module: #<Module:0x007f8a9ce14dc0>):
app/controllers/components_controller.rb:49:in `create'
Calling the self.delay method from your controller won't work, because DJ will try to serialize your controller into the Job. You'd better create a class to handle your task then flag its method as asynchronous :
class AsyncTask
def run
system('ruby my_script.rb')
end
handle_asynchronously :run
end
In your controller :
def create
...
AsyncTask.new.run
...
end
See the second example in the "Queing Jobs" section of the readme.
Like Jef stated the best solution is to create a custom job.
The problem with Jef's answer is that its syntax (as far as I know) is not correct and that's his job handles a single system command while the following will allow you more customization:
# lib/system_command_job.rb
class SystemCommandJob < Struct.new(:cmd)
def perform
system(cmd)
end
end
Note the cmd argument for the Struct initializer. It allows you to pass arguments to your job so the code would look like:
Delayed::Job.enqueue(SystemCommandJob.new("ruby my_script.rb"))
Related
I have been trying to see if this is possible and so far have found nothing so I will try and ask specifically
Is it possible to have a sidekiq worker which can recive a method as for example a lambda method and pass on arguments to it?
Example case:
I need to make some heavy computation on my server and my options are to either make a specific sidekiq worker for the job which will only be done 1 time ever and will end up cloddering my code base, or make a worker which could lets say accept something like:
lot_of_work.each do |args|
Workers::Tmp::LetsGo.perform_async(args) { |a| a.lets_go }
end
I've tried looking through old stackoverflow posts and documentation for sidekiq.
I've tried the above method which I hoped worked as a normal method but it does not.
I would have liked it to execute the method which was pass to the worker such that I do not need to make workers for 1 time cases and dont have to use single thread computation.
I found a solution to this problem, there are probably better ones but this worked for me.
Make a worker like this:
module Workers
module Default
class TesterWorker
include Sidekiq::Worker
sidekiq_options queue: :default, retry: false
def perform(method_name, method)
eval(method)
send(method_name)
end
end
end
end
After this you simply just have to write your code as a string like this:
methode_name = 'tester'
spec = "def test; puts 1; end"
Workers::Default::TesterWorker.perform_async(methode_name, spec)
And this will execute the for example the puts 1 action on the sidekiq ^^
We are on Rails 5 and have a flow we are trying to automate which involves the creation of several models. We made a custom generator that handles this, among other things.
The first step creates some files that we need, and works splendidly, however we began to chain things and noticed something quite odd:
If we run invoke model some_args more than once, the second run does nothing, and exits with a 0 status code. We are somewhere in the Rack layer when this happens, however it seems that if we change invoke to be another generator it will run fine.
If we wrap rails g model in back-ticks it will run and execute the generator:
`rails g model`
Here is the code:
require "rails/generators"
require_relative "helpers/scaffold_generator_helper"
module ActiveCsv
class ModelScaffoldGenerator < Rails::Generators::NamedBase
desc <<~DESCRIPTION
This Generator takes a file containing a class inheriting from ActiveCsv
and produces 3 models and migrations (SourceModel, DestinationModel, FailedImportModel)
based on the configurations within the object
DESCRIPTION
def load_object_params
#helper = ActiveCSVScaffoldGeneratorHelper.new(name)
#params = #helper.extract_parameters
puts #params
end
def handle_source_model
case self.behavior
when :invoke
args = #helper.generate_source_model_options
created_files = invoke("model", args)
created_class_name = created_files[0][0][0]
created_model_path = created_files[0][2]
inject_into_class(created_model_path, created_class_name) do
" acts_as_copy_target\n"
end
when :revoke
invoke "model", [#helper.source_model_name], behavior: :revoke
end
end
def handle_destination_model
case self.behavior
when :invoke
args = #helper.generate_destination_model_options
created_files = invoke("model", args)
when :revoke
invoke "model", [#helper.destination_model_name], behavior: :revoke
end
end
end
end
Is it not possible to run the same generator multiple times inside of another generator?
Turns out the problem lies in Thor - it will only read the first invoke statement in any generator. Found the solution in this answer to a question about rails 3 generators - turns out it hasn't changed since.
Quoting the original answer:
"There's a catch however: Probably due to Thors dependency management, this only works once per generator you want to call, meaning that a second invoke of the same generator will do nothing. This can be circumvented by using a statement like
Rails::Generators.invoke 'active_record:model', '...', behavior: behavior"
So replacing invoke with Rails::Generators.invoke totally did the trick.
I am using Rails 4 w/ the impressionist and resque gem.
I am using impressionist to log unique session hits on my article show page. Due to performance issues and no need to display hits to users (it is for admins only), I would like to move logging impressions off into the background.
Normally I would log an impression using impressionist(#article, unique: [:session_hash]) but to move it off into the bg via resque I am now doing something like this...
articles_controller:
def show
.
.
.
Resque.enqueue(ImpressionLogger, #article.id)
end
app/workers/impression_logger.rb:
class ImpressionLogger
#queue = :impression_queue
def self.perform(article_id)
article = Article.find(article_id)
impressionist(article, unique: [:session_hash])
end
end
When I set it up like this, when resque tries to process the job, it is returning undefined method "impressionist" for ImpressionLogger:Class. What do you guys think the best way to go about this is? I am not sure how to include impressionist methods inside of my resque worker.
The issue
Your problem stems from the fact that it looks like Impressionist works on the controller level due to including a module with the impressionist method in an engine initializer on any instances of ActionController:
https://github.com/charlotte-ruby/impressionist/blob/master/lib/impressionist/engine.rb#L11
You're trying to call the impressionist method from a regular class being invoked in a Resque job, so it's not going to have that method defined.
Solution
It's kind of gross, but if you really want to use impressionist, we can delve into this... Looking at the actual implementation of the impressionist method found here, we see the following:
def impressionist(obj,message=nil,opts={})
if should_count_impression?(opts)
if obj.respond_to?("impressionable?")
if unique_instance?(obj, opts[:unique])
obj.impressions.create(associative_create_statement({:message => message}))
end
else
# we could create an impression anyway. for classes, too. why not?
raise "#{obj.class.to_s} is not impressionable!"
end
end
end
Assuming that you'd be calling something like this manually (as you want to from a resque job) the key are these three lines:
if unique_instance?(obj, opts[:unique])
obj.impressions.create(associative_create_statement({:message => message}))
end
The if wrapper only seems to be important if you want to implement this functionality. Which it looks like you do. The call to associative_create_statement seems to be pulling parameters based off of the controller name as well as parameters passed from Rack such as the useragent string and ip address (here). So, you'll have to resolve these values prior to invoking the Resque job.
What I would suggest at this point is implementing a Resque class that takes in two parameters, an article_id and the impression parameters that you want. The resque class would then just directly create the impression on the impressionable object. Your Resque class would become:
class ImpressionLogger
#queue = :impression_queue
def self.perform(article_id, impression_params = {})
article = Article.find(article_id)
article.impressions.create(impression_params)
end
end
And your controller method would look something like this:
def show
.
.
.
Resque.enqueue(ImpressionLogger, #article.id, associative_create_statement({message: nil})) if unique_instance?(#article, [:session_hash])
end
Disclaimer
There's a fairly big disclaimer that comes with doing it this way though... the method associative_create_statement is marked protected and unique_instance? is marked private... so neither of these is part of the impressionist gem's public API, so this code might break between versions of the gem.
Is impressionist installed properly with bundler? If so Rails should be loading it into your environment. I would check whether you can access impressionist functionality elsewhere in your Rails code (i.e. without going through Resque) as the first step to debugging this.
How are you starting your resque workers? If you need your Rails environment loaded, try rake environment resque:work.
https://github.com/resque/resque/wiki/FAQ#how-do-i-ensure-my-rails-classesenvironment-is-loaded
I have an app with both sidekiq and delayed job gems installed. When I trigger handle_asynchronously in active record models it appear to be handled by sidekiq while I would like to trigger delayed_job.
Is there a way to desactivate sidekiq for a specific model?
UPDATE:
Sidekiq now provides ways to either disable its delay module completely or alias it as sidekiq_delay. Please check this to see how to do it. https://github.com/mperham/sidekiq/wiki/Delayed-Extensions#disabling-extensions
For older version of sidekiq:
I use this monkey patch to make it so that calling .sidekiq_delay() goes to sidekiq and .delay() is goes to DelayedJob. According the answer by Viren, I think this may also solve your problem.
The patch is less complex (just a bunch of aliases), and gives you the power to consciously decide which delay you are actually calling.
As I mention in the comment In order to get it working you have to redefine/basically monkey patch the handle_asynchronously method something like this
Anywhere you like (but make sure it loaded )
in your config/initializers/patch.rb the code look like this
module Patch
def handle_asynchronously(method, opts = {})
aliased_method, punctuation = method.to_s.sub(/([?!=])$/, ''), $1
with_method, without_method = "#{aliased_method}_with_delay#{punctuation}", "#{aliased_method}_without_delay#{punctuation}"
define_method(with_method) do |*args|
curr_opts = opts.clone
curr_opts.each_key do |key|
if (val = curr_opts[key]).is_a?(Proc)
curr_opts[key] = if val.arity == 1
val.call(self)
else
val.call
end
end
end
## Replace this with other syntax
# delay(curr_opts).__send__(without_method, *args)
__delay__(curr_opts).__send__(without_method, *args)
end
alias_method_chain method, :delay
end
end
Module.send(:include,Patch)
And I believe rest all will follow then they way it should :)
Reason:
Delayed::Job include delay method on Object and Sidekiq include it delay method over ActiveRecord
Hence when the class try to invoke delay it look up it ancestors class (including the Eigen Class)
and it find the method define or included in ActiveRecord::Base class (which is sidekiq delay)
why does __delay__ work because alias define the copy of the existing method which is delay method of DelayedJob , hence when you invoke the __delay__ method it invoke delay method define DelayedJob
include to Object
Note:
Although the solution is bit patch but the it works . Keeping in mind that every direct .delay methid invocation is invoking delay method of the SideKiq and not DelayedJob to invoke the DelayedJob delay method you always has call it this way __delay__
Suggestion :
Monkey Patching is just a bad practice on my personal note I would rather not use 2 entirely different background processing library for a single application to achieve the same task. If the task is process thing in background why cant it be done with a single library either delayed_job or sidekiq (why it is that you required both of them )
So the point and to simply thing make your background processing an ease with respect to future I sincerely advice you take any one of the two library for background processing and I feel that would the valid answer for your question instead of monkey patching an doing other crazy stuff
Hope this help
Can I do the following?
def ModelObserver < ActiveRecord
def after_save
Rake::Task[name].invoke
end
end
At the moment, this returns the following error:
Don't know how to build task 'name'
Any idea?
Use the system command :
def ModelObserver < ActiveRecord
def after_save
system "rake #{name}"
end
end
Consider using delayed job or similar plugin to handle background execution. In observer (or controller) just notify background job daemon, that it should take care of some action, instead of running this task directly.
In Rails3 if you still want to call rake task like this:
Rake::Task[name].invoke
you have to put
[Application].load_tasks
before invoke command, where application is your application name. For example I had to put
Ead::Application.load_tasks
I suppose you have to load the Rake environment first, and the Rakefile. I would not try to invoke the fullblown command line to do that. You probably need to use "import" as can be found in the Rake API