I need to add a job to the Sidekiq queue when my Rails app starts, to update some data, but I don't know where is the best place to do it.
Right now, I've wrote this on my application.rb:
class Application < Rails::Application
config.after_initialize do
MyWorker.perform_async
end
end
But the problem is that when I run the sidekiq command it will also load the Rails stack, so I'll end up with 2 jobs on the queue.
Is there any other way of doing that? This is my first big Rails app and my first time with Sidekiq, so I don't know if I'm not understanding things correctly. That might not be the right way of doing that.
Thanks!
A better solution would be to create an initializer config/initializers/sidekiq.rb
Sidekiq.configure_client do |config|
Rails.application.config.after_initialize do
# You code goes here
end
end
We were having issues w/ Redis connections and multiple jobs being launched.
I ended up using this and it seems to be working well:
if defined?(Sidekiq)
Sidekiq.configure_server do |config|
config.on(:startup) do
already_scheduled = Sidekiq::ScheduledSet.new.any? {|job| job.klass == "MyWorker" }
MyWorker.perform_async unless already_scheduled
end
end
end
Probably foreman suits for your purposes.
I know this is old, but none of this worked for me - would still start the job several times. I came up with the following solution:
I have a Class to do the actual Job:
class InitScheduling
include Sidekiq::Worker
def perform
# your code here
end
end
And I have an inizializer, which would normally start 3 Times, every time, something loads the Rails environment. So I use the Job as a state variable that this job is already scheduled:
# code in inizilizer/your_inizilizer.rb
Rails.application.config.after_initialize do
all_jobs = Sidekiq::ScheduledSet.new
# If this is True InitScheduling is already scheduled - don't start it again
unless all_jobs.map(&:klass).include?("InitScheduling")
puts "########### InitScheduling ##############"
# give rails time to build before running this job & keeps this initialization from re-running
InitScheduling.perform_in(5.minutes)
# your code here
end
end
Related
I have an API which uses a Service, in which I have used Ruby thread to reduce the response time of the API. I have tried to share the context using the following example. It was working fine with Rails 4, ruby 2.2.1
Now, we have upgraded rails to 5.2.3 and ruby 2.6.5. After which service has stopped working. I can call the service from Console, it works fine. But with API call, service becomes unresponsive once it reaches CurrencyConverter.new. Any Idea what can be the issue?
class ParallelTest
def initialize
puts "Initialized"
end
def perform
# Our sample set of currencies
currencies = ['ARS','AUD','CAD','CNY','DEM','EUR','GBP','HKD','ILS','INR','USD','XAG','XAU']
# Create an array to keep track of threads
threads = []
currencies.each do |currency|
# Keep track of the child processes as you spawn them
threads << Thread.new do
puts currency
CurrencyConverter.new(currency).print
end
end
# Join on the child processes to allow them to finish
threads.each do |thread|
thread.join
end
{ success: true }
end
end
class CurrencyConverter
def initialize(params)
#curr = params
end
def print
puts #curr
end
end
If I remove the CurrencyConverter.new(currency), then everything works fine. CurrencyConverter is a service object that I have.
Found the Issue
Thanks to #anothermh for this link
https://guides.rubyonrails.org/threading_and_code_execution.html#wrapping-application-code
https://guides.rubyonrails.org/threading_and_code_execution.html#load-interlock
As per the blog, When one thread is performing an autoload by evaluating the class definition from the appropriate file, it is important no other thread encounters a reference to the partially-defined constant.
Only one thread may load or unload at a time, and to do either, it must wait until no other threads are running application code. If a thread is waiting to perform a load, it doesn't prevent other threads from loading (in fact, they'll cooperate, and each perform their queued load in turn, before all resuming running together).
This can be resolved by permitting concurrent loads.
https://guides.rubyonrails.org/threading_and_code_execution.html#permit-concurrent-loads
Rails.application.executor.wrap do
urls.each do |currency|
threads << Thread.new do
CurrencyConverter.new(currency)
puts currency
end
ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
threads.map(&:join)
end
end
end
Thank you everybody for your time, I appreciate.
Don't re-invent the wheel and use Sidekiq instead. 😉
From the project's page:
Simple, efficient background processing for Ruby.
Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails to make background processing dead simple.
With 400+ contributors, and 10k+ starts on Github, they have build a solid parallel job execution process that is production ready, and easy to setup.
Have a look at their Getting Started to see it by yourself.
I'm working on a product that calls of perform_later jobs. This works for our product in production because we have a series of workers who will run all the jobs.
But, when I'm using the app locally, I don't have access to these workers, and I'd like to change all the perform_laters into perform_nows only when I use the app locally.
What's the best way to do this? One idea I had was to add something in my env file that would add a variable to make all perform_laters into perform_nows -- but I'm not sure what a flag or variable like that would look like.
Ideas?
The clean solution is to change the adapter in development environment.
In your /config/environments/development.rb you need to add:
Rails.application.configure do
config.active_job.queue_adapter = :inline
end
"When enqueueing jobs with the Inline adapter the job will be executed immediately."
In your app you can have:
/my_app/config/initializers/jobs_initializer.rb
module JobsExt
extend ActiveSupport::Concern
class_methods do
def perform_later(*args)
puts "I'm on #{Rails.env} envirnoment. So, I'll run right now"
perform_now(*args)
end
end
end
if Rails.env != "production"
puts "including mixin"
ActiveJob::Base.send(:include, JobsExt)
end
This mixin will be included on test and development environments only.
Then, if you have the job in:
/my_app/app/jobs/my_job.rb
class MyJob < ActiveJob::Base
def perform(param)
"I'm a #{param}!"
end
end
You can execute:
MyJob.perform_later("job")
And get:
#=> "I'm a job!"
Instead of the job instance:
#<MyJob:0x007ff197cd1938 #arguments=["job"], #job_id="aab4dbfb-3d57-4f6d-8994-065a178dc09a", #queue_name="default">
Remember: Doing this, ALL your Jobs will be executed right now on test and dev environments. If you want to enable this functionality for a single job, you will need to include the JobsExt mixin in that job only.
We solved this by calling an intermediate method which then called perform_later or perform_now depending on Rails config:
def self.perform(*args)
if Rails.application.config.perform_later
perform_later(*args)
else
perform_now(*args)
end
end
And simply updated environments configs accordingly
I have a setup command that I want executed every time I start the rails console -
MyClass.some_method()
I get tired of retyping it each time I fire up rails c - is there a way to have it automatically get run every time a new console is started?
Thanks!
We do this in order to ask for the tenant every time the console starts. It took a bit of investigation, but we got it working fairly elegantly. Note that this works with Rails 5.2 but it has worked mostly the same way since Rails 4.
Another thing to note is that this is written specifically because we wanted to be able to run the method once on start and then be able to run it again while using the console, say if we wanted to switch the tenant during a session.
The first step is to create a set of modules and classes in a lib file. Here is an example extracted from ours:
# lib/console_extension.rb
module ConsoleExtension
# This module provides methods that are only available in the console
module ConsoleHelpers
def do_someting
puts "doing something"
end
end
# This is a simple class that allows us access to the ConsoleHelpers before
# we get into the console
class ConsoleRunner
include ConsoleExtension::ConsoleHelpers
end
# This is specifically to patch into the startup behavior for the console.
#
# In the console_command.rb file, it does this right before start:
#
# if defined?(console::ExtendCommandBundle)
# console::ExtendCommandBundle.include(Rails::ConsoleMethods)
# end
#
# This is a little tricky. We're defining an included method on this module
# so that the Rails::ConsoleMethods module gets a self.included method.
#
# This causes the Rails::ConsoleMethods to run this code when it's included
# in the console::ExtendCommandBundle at the last step before the console
# starts, instead of during the earlier load_console stage.
module ConsoleMethods
def included(_klass)
ConsoleExtension::ConsoleRunner.new.do_someting
end
end
end
The next step is to add the following into your application.rb file:
module MyApp
class Application < Rails::Application
...
console do
require 'console_extension' # lib/console_extension.rb
Rails::ConsoleMethods.send :include, ConsoleExtension::ConsoleHelpers
Rails::ConsoleMethods.send :extend, ConsoleExtension::ConsoleMethods
end
end
end
Now, every time you run rails console, it will do something:
If you're just looking to run something once every time the console starts, this is more complicated than it needs to be. Instead, you can just use the console() method in MyApp::Application and it will run whatever code you want as part of the load_console step.
module MyApp
class Application < Rails::Application
...
console do
puts "do something"
end
end
end
One issue we had with this was that it runs the code before it prints out the environment, so if you're doing any printing or interaction it feels a bit weird:
You may not be as picky as we are though. Do whatever makes you and your team the happiest.
I dont know if its a good practice, but you can check if server is running on Console, like Aditya awnsered
if defined?(Rails::Console)
MyClass.some_method()
end
Note that this won't work during Rails initialization when running Spring like Swartz said.
I would try creating a Rake task for it and invoke it with after_initialize:
config.after_initialize do
IndividualProject::Application.load_tasks
Rake::Task[ 'foo:bar' ].invoke
end
I have a rails app in which I would like to use the delayed_jobs gem to send texts/emails in background processes at certain times of the day. This is how I have the relevant parts of my app set up right now:
class SomeClass
after_create :send_reminder
def when_to_run
self.date_time - 1.hour
end
def send_reminder
MessageHandler.new().send_message
end
handle_asynchronously :send_reminder, run_at: Proc.new { |i| i.when_to_run }, queue: "Messages"
end
The MessageHandler class is a separate class I've defined which actually houses the methods for sending texts (with Twilio) and emails (with Mailgun).
After starting delayed_job (bin/delayed_job start) and creating an instance of SomeClass, the delayed_job log reads as follows:
Job SomeClass#send_reminder_without_delay (id=686) RUNNING
I'm not sure why it is running send_reminder_without_delay, and it's doing it every time. I've tried using MessageHandler.new().delay.send_message in the send_reminder method instead of MessageHandler.new().send_message, but that hasn't gotten me anywhere either.
I've searched high and low for answers and keep coming up short - any help would be much appreciated!
I think your problem is in run_at: Proc.new { |i| i.when_to_run }.
I'm not entirely sure what the date_time method does but have you checked whether it actually returns time that is more than 1 hour in the future?
Otherwise the run_At will receive a time in the past and run the method.
if the return value of date_time is similar to this:
def when_to_run
Time.now - 1.hour
end
Then the queue will run as soon as you start delayed job.
Check if it your run_at time is in the future.
We were using Rails 4.2 and set the queue adapter. We needed to use this line in our config:
config.active_job.queue_adapter = :delayed_job
I am running a delayed job worker. When ever I invoke the foo method, worker prints hello.
class User
def foo
puts "Hello"
end
handle_asynchronously :foo
end
If I make some changes to the foo method, I have to restart the worker for the changes to reflect. In the development mode this can become quite tiresome.
I am trying to find a way to reload the payload class(in this case User class) for every request. I tried monkey patching the DelayedJob library to invoke require_dependency before the payload method invocation.
module Delayed::Backend::Base
def payload_object_with_reload
if Rails.env.development? and #payload_object_with_reload.nil?
require_dependency(File.join(Rails.root, "app", "models", "user.rb"))
end
#payload_object_with_reload ||= payload_object_without_reload
end
alias_method_chain :payload_object, :reload
end
This approach doesn't work as the classes registered using require_dependency needs to be reloaded before the invocation and I haven't figured out how to do it. I spent some time reading the dispatcher code to figure out how Rails reloads the classes for every request. I wasn't able to locate the reload code.
Has anybody tried this before? How would you advise me to proceed? Or do you have any pointers for locating the Rails class reload code?
I managed to find a solution. I used ActiveSupport::Dependencies.clear method to clear the loaded classes.
Add a file called config/initializers/delayed_job.rb
Delayed::Worker.backend = :active_record
if Rails.env.development?
module Delayed::Backend::Base
def payload_object_with_reload
if #payload_object_with_reload.nil?
ActiveSupport::Dependencies.clear
end
#payload_object_with_reload ||= payload_object_without_reload
end
alias_method_chain :payload_object, :reload
end
end
As of version 4.0.6, DelayedJob reloads automatically if Rails.application.config.cache_classes is set to false:
In development mode, if you are using Rails 3.1+, your application code will automatically reload every 100 jobs or when the queue finishes. You no longer need to restart Delayed Job every time you update your code in development.
This looks like it solves your problem without the alias_method hackery:
https://github.com/Viximo/delayed_job-rails_reloader