Rails 5 - Sidekiq worker shows job done but nothing happens - ruby-on-rails

I'm using Sidekiq for delayed jobs with sidekiq-status and sidekiq-ent gems. I've created a worker which is reponsible to update minor status to false when user is adult and has minor: true. This worker should be fired every day at midnight ET. Like below:
#initializers/sidekiq.rb
config.periodic do |mgr|
# every day between midnight 0 5 * * *
mgr.register("0 5 * * *", MinorWorker)
end
#app/workers/minor_worker.rb
class MinorWorker
include Sidekiq::Worker
def perform
User.adults.where(minor: true).remove_minor_status
rescue => e
Rails.logger.error("Unable to update minor field. Exception: #{e.message} : #{e.backtrace.join('\n')}")
end
end
#models/user.rb
class User < ApplicationRecord
scope :adults, -> { where('date_of_birth <= ?', 18.years.ago) }
def self.remove_minor_status
update(minor: false)
end
end
No I want to check this on my local machine - to do so I'm using gem 'timecop' to timetravel:
#application.rb
config.time_zone = 'Eastern Time (US & Canada)'
#config/environments/development.rb
config.after_initialize do
t = Time.local(2021, 12, 21, 23, 59, 0)
Timecop.travel(t)
end
After firing up sidekiq by bundle exec sidekiq and bundle exec rails s I'm waiting a minute and I see that worker shows up:
2021-12-21T22:59:00.130Z 25711 TID-ovvzr9828 INFO: Managing 3 periodic jobs
2021-12-21T23:00:00.009Z 25711 TID-ovw69k4ao INFO: Enqueued periodic job SettlementWorker with JID ddab15264f81e0b417e7dd83 for 2021-12-22 00:00:00 +0100
2021-12-21T23:00:00.011Z 25711 TID-ovw69k4ao INFO: Enqueued periodic job MinorWorker with JID 0bcd6b76d6ee4ff9e7850b35 for 2021-12-22 00:00:00 +0100
But it didn't do anything, the user's minor status is still set to minor: true:
2.4.5 :002 > User.last.date_of_birth
=> Mon, 22 Dec 2003
2.4.5 :001 > User.last.minor
=> true
Did I miss something?
EDIT
I have to add that when I'm trying to call this worker on rails c everything works well. I've got even a RSpec test which also passes:
RSpec.describe MinorWorker, type: :worker do
subject(:perform) { described_class.new.perform }
context 'when User has minor status' do
let(:user1) { create(:user, minor: true) }
it 'removes minor status' do
expect { perform }.to change { user1.reload.minor }.from(true).to(false)
end
context 'when user is adult' do
let(:registrant2) { create(:registrant) }
it 'not change minor status' do
expect(registrant2.reload.minor).to eq(false)
end
end
end
end

Since this is the class method update won't work
def self.remove_minor_status
update(minor: false)
end
Make use of #update_all
def self.remove_minor_status
update_all(minor: false)
end
Also, I think it's best practice to have some test cases to ensure the working of the methods.
As of now you can try this method from rails console and verify if they actually work
test "update minor status" do
user = User.create(date_of_birth: 19.years.ago, minor: true)
User.adults.where(minor: true).remove_minor_status
assert_equal user.reload.minor, false
end

I think you need to either do update_all or update each record by itself, like this:
User.adults.where(minor: true).update_all(minor: false)
or
class MinorWorker
include Sidekiq::Worker
def perform
users = User.adults.where(minor: true)
users.each { |user| user.remove_minor_status }
rescue => e
Rails.logger.error("Unable to update minor field. Exception: #{e.message} : #{e.backtrace.join('\n')}")
end
end
You may also want to consider changing update to update! so it throws an error if failing to be caught by your rescue in the job:
def self.remove_minor_status
update!(minor: false)
end

Related

How to call controller action from _Sidekiq Worker in Rails 5.2.4?

My RoR application triggers data treatment scripts (SAS technology) from Linux operating system. An execute method in the Scheduler::ProductionExecutionsController drives the interactions with the scripts.
Scheduler::ProductionExecutionsController
# POST /production_executions/1/execute
def execute
#production_execution = ProductionExecution.find(params[:id])
#production_execution.update_attributes(started_at: Time.now,
status_id: statuses.find { |x| x["code"] == "RUNNING" }.id)
--- Scripts interaction ---
#production_execution.update_attributes(ended_at: Time.now,
status_id: statuses.find { |x| x["code"] == "FINISHED" }.id,
source_records_count: global_input_count,
processed_count: global_output_count,
error_message: nil
)
respond_to do |format|
format.html { redirect_back fallback_location: #production_execution, notice: #msg }
format.js
end
This method is called using this syntax: link_to "Go!", execute_scheduler_production_execution_path(execution). The route is defined and it works as expected.
As some scripts can take over a minute to execute, I need to execute the scripts in dedicated jobs, and sometimes schedule them. So I installed Sidekiq and defined a Scheduler::ScriptWorker :
class Scheduler::ScriptWorker
include Sidekiq::Worker
sidekiq_options queue: :default, tags: ['script']
def perform(execution_id)
puts "Sidekiq job start"
puts execution_id
redirect_to execute_scheduler_production_execution_path(execution_id) and return
end
end
The executions are queued with Scheduler::ScriptWorker.perform_async(#execution.id)
Sidekiq operates correctly, but each time the worker is invoked, it raise the following error:
WARN: NoMethodError: undefined method `execute_scheduler_production_execution_path' for #<Scheduler::ScriptWorker:0x000000000b15ded8>
Is it the right way to do this, and how can I solve this issue?
Thanks for you help!
The short answer is: you can't
You can't perform a redirect from an asynchronous Sidekiq worker. When the user clicks "Go!", an HTTP request is made to your Rails web application. Rails hands the request off to a controller action, which in your case spawns an async sidekiq job. The controller action logic continues and Rails completes the HTTP request with an HTTP response, even if your sidekiq worker is still running. You cannot broadcast a message from a sidekiq worker back to the user who initiated the task
I'm sorry this does not resolve your issue. You'll need to take a different approach.
As stated by Sean Huber, converting a front-end job into a backend job requires a touch of architecture. Through refactoring, the experimental execute method was moved to a helper and renamed execute_ssh. A new run_once method in the controller fires the helper's execute_ssh method and reloads the page.
script_worker.rb
class Scheduler::ScriptWorker
include Sidekiq::Worker
include SchedulerExecutionHelper
include ParametersHelper
sidekiq_options queue: :default, tags: ['script'], retry: false
def perform(execution_id)
puts "Sidekiq job start"
puts execution_id
execute_ssh execution_id
end
end
scheduler_execution_helper.rb
def execute_ssh(execution_id)
puts "--- Switched to helper"
#production_execution = ProductionExecution.find(execution_id)
puts #production_execution.id
#production_execution.update_attributes(started_at: Time.now, status_id: statuses.find { |x| x["code"] == "RUNNING" }.id)
--- Scripts interaction ---
#production_execution.update_attributes(ended_at: Time.now,
status_id: statuses.find { |x| x["code"] == "FINISHED" }.id,
source_records_count: global_input_count,
processed_count: global_output_count,
error_message: nil
)
end
production_schedules_controller
def run_once
#job = #production_schedule.parent
#execution = #job.production_executions.build(playground_id: #job.playground_id,
production_job_id: #job.id,
environment_id: #production_schedule.environment_id,
owner_id: current_user.id,
status_id: options_for('Statuses', 'Scheduler').find { |x| x["code"] == "READY" }.id || 0)
if #execution.save
#job.production_events.where(production_execution_id: nil).each do |event|
execution_event = event.dup
execution_event.production_execution_id = #execution.id
execution_event.return_value = 0
execution_event.status_id = statuses.find { |x| x["code"] == "READY" }.id
execution_event.save
end
Scheduler::ScriptWorker.perform_async(#execution.id)
redirect_to scheduler_production_job_path(#job)
else
end
end
This way, the controller remains thin, the worker can be easily reused, and the logic is in the helper module.

How to override Date.today when running rails tests?

I have some tests that were failing on specific dates, because someone wrote them to use Date.today. I want to reproduce the failures on previous select dates.
Is there a way to run rake test with an ENV variable that will override the system clock? So that calls to Date.today, Time.now, and 1.day.ago and 1.day.from_now will use the date I set?
Like, for example:
> DATE_TODAY='2017-01-04' rake test
For testing you can use timecop gem.
It offers you two useful methods Timecop.freeze and Timecop.travel.
For example, you can use freeze to statically set time in before hooks:
describe 'your tests' do
before do
Timecop.freeze(Time.new(2017, 1, 4))
end
after do
Timecop.return
end
it 'should do something' do
sleep(10)
Time.now # => 2017-01-04 00:00:00
end
end
Or in a block:
Timecop.freeze(Time.now - 3.months) do
assert product.expired?
end
While with the travel method, you change the starting moment, but time is still passing by.
Timecop.travel(Time.new(2017, 1, 4))
sleep(10)
Time.now # => 2017-01-04 00:00:10
As of Rails 4.1 you can do
travel_to Time.new(2004, 11, 24, 01, 04, 44)
The full API docs are here: http://api.rubyonrails.org/classes/ActiveSupport/Testing/TimeHelpers.html

model method trigger boolean to true?

How can I trigger method accomplished_challenge upon days_left_challenged == 0?
challege.rb
before_save :days_left_challenged_sets_deadline
# makes ongoing_challenge become past (before_save) THIS WORKS
def days_left_challenged_sets_deadline
if self.date_started != nil
if days_left_challenged <= 0
self.accomplished = true
self.deadline = self.date_started
end
end
end
# makes ongoing_challenge become past (whenever gem) THIS DOESN'T
def self.accomplished_challenge
self.all.each do |challenge|
if challenge.days_left_challenged <= 0
challenge.accomplished = true
challenge.deadline = self.date_started
end
end
end
# Counts down how many days left in days_challenged using committed
def days_left_challenged
self.days_challenged - ((date_started.to_date)..Date.yesterday).count do |date|
committed_wdays.include? date.wday
end + self.missed_days
end
Challenge.last
id: 1,
action: "Run",
committed: ["sun", "mon", "tue", "wed", "thu", "fri", "sat", ""],
date_started: Sat, 06 Feb 2016 00:00:00 EST -05:00,
deadline: nil,
accomplished: nil,
days_challenged: 10,
missed_days: 0,
I can't trigger it with a callback or validation I don't think since days_left_challenged can turn to 0 at any point in the life of a challenge.
I suggest you use a gem like Whenever to setup a cron job to run every day or so and do that checking for all Challenges. It would be something like:
every 1.day, :at => '0:01 am' do
runner "Challenge.accomplished_challenge"
end
And your accomplished_challenge must be a class method that checks all (or the one you choose using a filter) Challenges:
def self.accomplished_challenge
self.all.each do |challenge|
if challenge.days_left_challenged == 0
challenge.update_attributes(deadline: self.date_started, accomplished: true)
end
end
end
---- EDIT to work on Heroku ----
Create a task on /lib/tasks/scheduler.rake:
# /lib/tasks/scheduler.rake
desc "This task is called by the Heroku scheduler add-on"
task :check_accomplished_challenges => :environment do
puts "Checking accomplished challenges..."
Challenge.accomplished_challenge
puts "done."
end
Go to your heroku app Resources page and add 'Heroku Scheduler'. Open the scheduler and add the task:
rake check_accomplished_challenges
Set it to run every day.
More details: https://devcenter.heroku.com/articles/scheduler

Get error message out of Sidekiq job

I want to get exception error message out of the sidekiq job. when I set back_trace option to true it retries my job but I want to exit from job when error raises and get error message.
if I find that process ended successful or fail is enough.
def perform(text)
begin
fail StandardError, 'Error!'
rescue
fail 'EEE' # I want to get this error when call job
end
end
# call
NormalJob.perform_async('test')
# I want to get error here after call
If I were you I would try gem sidekiq-status. It has several options, which can be helpful in such situations:
You can retrieve status of your worker:
job_id = MyJob.perform_async(*args)
# :queued, :working, :complete or :failed , nil after expiry (30 minutes)
status = Sidekiq::Status::status(job_id)
Sidekiq::Status::queued? job_id
Sidekiq::Status::working? job_id
Sidekiq::Status::complete? job_id
Sidekiq::Status::failed? job_id
Also you have options for Tracking progress, saving and retrieveing data associated with job
class MyJob
include Sidekiq::Worker
include Sidekiq::Status::Worker # Important!
def perform(*args)
# your code goes here
# the common idiom to track progress of your task
total 100 # by default
at 5, "Almost done"
# a way to associate data with your job
store vino: 'veritas'
# a way of retrieving said data
# remember that retrieved data is always is String|nil
vino = retrieve :vino
end
end
job_id = MyJob.perform_async(*args)
data = Sidekiq::Status::get_all job_id
data # => {status: 'complete', update_time: 1360006573, vino: 'veritas'}
Sidekiq::Status::get job_id, :vino #=> 'veritas'
Sidekiq::Status::at job_id #=> 5
Sidekiq::Status::total job_id #=> 100
Sidekiq::Status::message job_id #=> "Almost done"
Sidekiq::Status::pct_complete job_id #=> 5
Another option is to use sidekiq batches status
This is what batches allow you to do!
batch = Sidekiq::Batch.new
batch.description = "Batch description (this is optional)"
batch.notify(:email, :to => 'me#example.org')
batch.jobs do
rows.each { |row| RowWorker.perform_async(row) }
end
puts "Just started Batch #{batch.bid}"
b = Sidekiq::Batch.new(bid) # bid is a method on Sidekiq::Worker that gives access to the Batch ID associated to the job.
b.jobs do
SomeWorker.perform_async(1)
sleep 1
# Uh oh, Sidekiq has finished all outstanding batch jobs
# and fires the complete message!
SomeWorker.perform_async(2)
end
status = Sidekiq::Batch::Status.new(bid)
status.total # jobs in the batch => 98
status.failures # failed jobs so far => 5
status.pending # jobs which have not succeeded yet => 17
status.created_at # => 2012-09-04 21:15:05 -0700
status.complete? # if all jobs have executed at least once => false
status.join # blocks until the batch is considered complete, note that some jobs might have failed
status.failure_info # an array of failed jobs
status.data # a hash of data about the batch which can easily be converted to JSON for javascript usage
It can be used out of the box

How do I reschedule a failed Rufus every job?

I have a Rufus "every job" that runs periodically, but sometimes it may fail to perform it's task.
On failure, I would like to reschedule a retry soon rather than wait until the next cycle.
class PollProducts
def initialize()
end
def call(job)
puts "Updating products"
begin
# Do something that might fail
raise if rand(3) == 1
rescue Exception => e
puts "Request failed - recheduling: #{e}"
# job.in("5s") <-- What can I do?
end
end
end
scheduler.every '6h', PollProducts.new, :first_in => '5s', :blocking => true
Is this possible?
Ok this worked for me:
job.scheduler.in '5s', self

Resources