Rails 5 delayed job, how to check the job status - ruby-on-rails

Question

I am using a delayed job on a Rails 5 project with the 'delayed_job_active_record' gem.
I can't figure out how to check on the status of the job.
I managed to get delayed job to run. The jobs are failing due to an issue with processing uploaded files (that's another issue).
In my controller:
def create
#upload = Upload.create(status: "in progress")
#upload.delay.create_ln(lecture_note_params, current_user, #upload.id)
render json: {"status": #upload.status, "id": #upload.id}, status: 200
end
Here's my model code for the upload model:
class Upload < ApplicationRecord
def create_ln(lecture_note_params, current_user, upload_id)
#lecture_note = LectureNote.new(lecture_note_params.merge(user: current_user))
#lecture_note.upload_id = upload_id
if #lecture_note.save
#nothing
else
logger.info #lecture_note.errors.full_messages.to_sentence
#upload.update(error: #lecture_note.errors.full_messages.to_sentence, status: "error")
end
end
end
The lecture notes don't save. They fail for the reason mentioned above. I need the upload model to update with the error status. How can I do that?
I can see the delayed job in the console :
irb(main):002:0> ap Delayed::Job.last
D, [2019-12-12T06:12:08.014249 #4] DEBUG -- : Delayed::Backend::ActiveRecord::Job Load (6.3ms) SELECT `delayed_jobs`.* FROM `delayed_jobs` ORDER BY `delayed_jobs`.`id` DESC LIMIT 1
#<Delayed::Backend::ActiveRecord::Job:0x0000561edb710f70> {
:id => 16,
:priority => 0,
:attempts => 1,
:handler => "### ALL KINDS OF STUFF - REDACTED ###",
:run_at => Thu, 12 Dec 2019 06:10:32 UTC +00:00,
:locked_at => nil,
:failed_at => Thu, 12 Dec 2019 06:11:17 UTC +00:00,
:locked_by => nil,
:queue => nil,
:created_at => Thu, 12 Dec 2019 06:10:32 UTC +00:00,
:updated_at => Thu, 12 Dec 2019 06:11:17 UTC +00:00
}
After the job fails, the "failed_at" field is updated. How can I access that and update my upload model?
Here is the log file while a file is uploading:
D, [2019-12-12T06:11:17.846638 #4] DEBUG -- : Upload Load (1.1ms) SELECT `uploads`.* FROM `uploads` WHERE `uploads`.`id` = 31 LIMIT 1
D, [2019-12-12T06:11:17.852741 #4] DEBUG -- : User Load (1.4ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 1 LIMIT 1
[Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] Job Upload#create_ln (id=16) RUNNING
I, [2019-12-12T06:11:17.853180 #4] INFO -- : 2019-12-12T06:11:17+0000: [Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] Job Upload#create_ln (id=16) RUNNING
D, [2019-12-12T06:11:17.855002 #4] DEBUG -- : (1.2ms) BEGIN
I, [2019-12-12T06:11:17.855200 #4] INFO -- : pdf to html running..............
I, [2019-12-12T06:11:17.855268 #4] INFO -- : step 0 ..............
I, [2019-12-12T06:11:17.855326 #4] INFO -- : step 1 ..............
I, [2019-12-12T06:11:17.855388 #4] INFO -- : step 2 ..............
D, [2019-12-12T06:11:17.856673 #4] DEBUG -- : (1.1ms) ROLLBACK
[Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] Job Upload#create_ln (id=16) FAILED (0 prior attempts) with IOError: uninitialized stream
E, [2019-12-12T06:11:17.857261 #4] ERROR -- : 2019-12-12T06:11:17+0000: [Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] Job Upload#create_ln (id=16) FAILED (0 prior attempts) with IOError: uninitialized stream
[Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] Job Upload#create_ln (id=16) FAILED permanently because of 1 consecutive failures
E, [2019-12-12T06:11:17.857416 #4] ERROR -- : 2019-12-12T06:11:17+0000: [Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] Job Upload#create_ln (id=16) FAILED permanently because of 1 consecutive failures
D, [2019-12-12T06:11:17.858576 #4] DEBUG -- : (0.9ms) BEGIN
D, [2019-12-12T06:11:17.861961 #4] DEBUG -- : SQL (1.6ms) UPDATE `delayed_jobs` SET `last_error` = 'uninitialized stream\n/app/vendor/ruby-2.6.3/lib/ruby/2.6.0/tempfile.rb:221:in `path\'\n/app/vendor/ruby-2.6.3/lib/ruby/2.6.0/tempfile.rb:221:in `path\'\n/app/app/models/lecture_note.rb:60:in `pdf_to_html\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:382:in `block in make_lambda\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:150:in `block (2 levels) in halting_and_conditional\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:770:in `block (2 levels) in deprecated_false_terminator\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:769:in `catch\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:769:in `block in deprecated_false_terminator\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:151:in `block in halting_and_conditional\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:454:in `block in call\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:454:in `each\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:454:in `call\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:101:in `__run_callbacks__\'\n/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.0.7.1/lib/active_support/callbacks.rb:750:in `_run_validation_callbacks\'\n/app/vendor/bundle/ruby/2.6.0/gems/activemodel-5.0.7.1/lib/active_model/validations/callbacks.rb:113:in `run_validations!\'\n/app/vendor/bundle/ruby/2.6.0/gems/activemodel-5.0.7.1/lib/active_model/validations.rb:338:in `valid?\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/validations.rb:65:in `valid?\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/validations.rb:82:in `perform_validations\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/validations.rb:44:in `save\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/attribute_methods/dirty.rb:22:in `save\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/transactions.rb:319:in `block (2 levels) in save\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/transactions.rb:395:in `block in with_transaction_returning_status\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/connection_adapters/abstract/database_statements.rb:232:in `block in transaction\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/connection_adapters/abstract/transaction.rb:189:in `within_new_transaction\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/connection_adapters/abstract/database_statements.rb:232:in `transaction\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/transactions.rb:211:in `transaction\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/transactions.rb:392:in `with_transaction_returning_status\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/transactions.rb:319:in `block in save\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/transactions.rb:334:in `rollback_active_record_state!\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/transactions.rb:318:in `save\'\n/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.0.7.1/lib/active_record/suppressor.rb:41:in `save\'\n/app/app/models/upload.rb:7:in `create_ln\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/performable_method.rb:26:in `perform\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/backend/base.rb:81:in `block in invoke_job\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:61:in `block in initialize\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:66:in `execute\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:40:in `run_callbacks\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/backend/base.rb:78:in `invoke_job\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:230:in `block (2 levels) in run\'\n/app/vendor/ruby-2.6.3/lib/ruby/2.6.0/timeout.rb:93:in `block in timeout\'\n/app/vendor/ruby-2.6.3/lib/ruby/2.6.0/timeout.rb:103:in `timeout\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:230:in `block in run\'\n/app/vendor/ruby-2.6.3/lib/ruby/2.6.0/benchmark.rb:308:in `realtime\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:229:in `run\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:312:in `block in reserve_and_run_one_job\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:61:in `block in initialize\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:66:in `execute\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:40:in `run_callbacks\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:312:in `reserve_and_run_one_job\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:213:in `block in work_off\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:212:in `times\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:212:in `work_off\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:175:in `block (4 levels) in start\'\n/app/vendor/ruby-2.6.3/lib/ruby/2.6.0/benchmark.rb:308:in `realtime\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:174:in `block (3 levels) in start\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:61:in `block in initialize\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:66:in `execute\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:40:in `run_callbacks\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:173:in `block (2 levels) in start\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:172:in `loop\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:172:in `block in start\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/plugins/clear_locks.rb:7:in `block (2 levels) in <class:ClearLocks>\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:79:in `block (2 levels) in add\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:61:in `block in initialize\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:79:in `block in add\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:66:in `execute\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/lifecycle.rb:40:in `run_callbacks\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/worker.rb:171:in `start\'\n/app/vendor/bundle/ruby/2.6.0/gems/delayed_job-4.1.8/lib/delayed/tasks.rb:9:in `block (2 levels) in <top (required)>\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `block in execute\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `each\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `execute\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:214:in `block in invoke_with_call_chain\'\n/app/vendor/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:183:in `invoke\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:160:in `invoke_task\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `block (2 levels) in top_level\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `each\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `block in top_level\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:125:in `run_with_threads\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:110:in `top_level\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:83:in `block in run\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:186:in `standard_exception_handling\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:80:in `run\'\n/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>\'\n/app/vendor/bundle/ruby/2.6.0/bin/rake:23:in `load\'\n/app/vendor/bundle/ruby/2.6.0/bin/rake:23:in `<top (required)>\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `load\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `kernel_load\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:28:in `run\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:463:in `exec\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:27:in `dispatch\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:18:in `start\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/exe/bundle:30:in `block in <top (required)>\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors\'\n/app/vendor/bundle/ruby/2.6.0/gems/bundler-1.17.3/exe/bundle:22:in `<top (required)>\'\n/app/bin/bundle:3:in `load\'\n/app/bin/bundle:3:in `<main>\'', `attempts` = 1, `failed_at` = '2019-12-12 06:11:17', `updated_at` = '2019-12-12 06:11:17' WHERE `delayed_jobs`.`id` = 16
D, [2019-12-12T06:11:17.867680 #4] DEBUG -- : (5.1ms) COMMIT
D, [2019-12-12T06:11:17.870699 #4] DEBUG -- : SQL (1.8ms) UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_at` = '2019-12-12 06:11:17', `delayed_jobs`.`locked_by` = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4' WHERE ((run_at <= '2019-12-12 06:11:17.867786' AND (locked_at IS NULL OR locked_at < '2019-12-12 02:11:17.867819') OR locked_by = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1
[Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] 8 jobs processed at 13.8130 j/s, 8 failed
I, [2019-12-12T06:11:17.870885 #4] INFO -- : 2019-12-12T06:11:17+0000: [Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] 8 jobs processed at 13.8130 j/s, 8 failed
D, [2019-12-12T06:11:17.873258 #4] DEBUG -- : SQL (1.2ms) UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_at` = '2019-12-12 06:11:17', `delayed_jobs`.`locked_by` = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4' WHERE ((run_at <= '2019-12-12 06:11:17.870973' AND (locked_at IS NULL OR locked_at < '2019-12-12 02:11:17.871010') OR locked_by = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1
D, [2019-12-12T06:11:22.878755 #4] DEBUG -- : SQL (1.5ms) UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_at` = '2019-12-12 06:11:22', `delayed_jobs`.`locked_by` = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4' WHERE ((run_at <= '2019-12-12 06:11:22.876103' AND (locked_at IS NULL OR locked_at < '2019-12-12 02:11:22.876164') OR locked_by = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1
D, [2019-12-12T06:11:27.882776 #4] DEBUG -- : SQL (1.6ms) UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_at` = '2019-12-12 06:11:27', `delayed_jobs`.`locked_by` = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4' WHERE ((run_at <= '2019-12-12 06:11:27.880101' AND (locked_at IS NULL OR locked_at < '2019-12-12 02:11:27.880160') OR locked_by = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1
D, [2019-12-12T06:11:32.890712 #4] DEBUG -- : SQL (1.6ms) UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_at` = '2019-12-12 06:11:32', `delayed_jobs`.`locked_by` = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4' WHERE ((run_at <= '2019-12-12 06:11:32.887163' AND (locked_at IS NULL OR locked_at < '2019-12-12 02:11:32.887243') OR locked_by = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1
D, [2019-12-12T06:11:37.894751 #4] DEBUG -- : SQL (1.5ms) UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_at` = '2019-12-12 06:11:37', `delayed_jobs`.`locked_by` = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4' WHERE ((run_at <= '2019-12-12 06:11:37.892094' AND (locked_at IS NULL OR locked_at < '2019-12-12 02:11:37.892153') OR locked_by = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1
^C[Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] Exiting...
I, [2019-12-12T06:11:39.976787 #4] INFO -- : 2019-12-12T06:11:39+0000: [Worker(host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4)] Exiting...
D, [2019-12-12T06:11:42.902214 #4] DEBUG -- : SQL (5.4ms) UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_by` = NULL, `delayed_jobs`.`locked_at` = NULL WHERE `delayed_jobs`.`locked_by` = 'host:575e618d-28f3-49a1-8545-88afa11b60d1 pid:4'

Answer 1

OK I found the solution. Tun's answer pointed me in the general direction (very general direction) but led me to it.
delayed_job has callbacks but I didn't know where to put them, or how to use them. They simply go in the model where the job method takes place. That wasn't obvious in the available documentation (to me anyway).
I was looking at this the wrong way. I already have the upload class to manage the creation of the lecture notes. So using the callbacks I can have the job update the class with the status. That's what we needed in the end.
The def failure(job) method did the trick!
I don't 100% understand how these callbacks actually work behind the scenes but they do work.
class Upload < ApplicationRecord
def create_ln(lecture_note_params, current_user, upload_id)
#lecture_note = LectureNote.new(lecture_note_params.merge(user: current_user))
#lecture_note.upload_id = upload_id
# if #lecture_note.save
# #nothing
# else
# logger.info #lecture_note.errors.full_messages.to_sentence
# #upload.update(error: #lecture_note.errors.full_messages.to_sentence, status: "error")
# end
begin
#lecture_note.save
rescue Exception => e
puts #lecture_note.errors.full_messages.to_sentence
self.update(error: #lecture_note.errors.full_messages.to_sentence, status: "error")
end
end
def success(job)
puts "Job complete ------->"
end
def error(job, exception)
puts "Job failed ERROR ------->"
puts #lecture_note.errors.full_messages.to_sentence
self.update(error: #lecture_note.errors.full_messages.to_sentence, status: "error")
end
# def after(job)
# # Do something useful here
# end
def failure(job)
puts "Job failed FAILURE ------->"
puts #lecture_note.errors.full_messages.to_sentence
self.update(error: #lecture_note.errors.full_messages.to_sentence, status: "error")
end
end

Answer 2

Generally, we should not check the job. The job should report back. (Observer pattern - I think)
I usually do one or combination of the followings
ActionCable to inform progress of the job (exception in your case)
Catch Exception in the job and update Upload with status
Use third party gem (such as activity_notification for notification.
DelayedJob proposed Global Hooks but I think it is not implemented yet.