Rails Runner or Rake task can't trigger Active Job job - ruby-on-rails

Question

I have setup a Rails 7 project running in Docker. Using the whenever gem (using cron) I tried to execute either
a rails runner task
a rake task
Both shall do the same thing: call a class method WebpageChangeCheck.check_all which itself calls a method of a model. Which then creates an Active Job job. But both fail in the final step to create the job:
app/cron_jobs/webpage_change_check.rb:
class WebpageChangeCheck
def self.check_all
Webpage.all.each do |page|
if page.checking_active
page.check_for_change
end
end
end
end
app/models/webpage.rb:
def check_for_change
self.update(counter: self.counter += 1)
UpdateOffersHashJob.perform_later(self)
end
update_offers_hash_job.rb:
class UpdateOffersHashJob < ApplicationJob
queue_as :default
require 'nokogiri'
require 'open-uri'
require 'net/http'
after_perform do |job|
compare_hashes(job.arguments.first)
end
def perform(page)
page.update(offers_hash_old: page.offers_hash_new)
all_offers = ""
doc = Nokogiri::HTML(URI.open(page.url))
doc.css(page.selector).each do |offer|
all_offers += offer.to_s
end
page.update(offers_hash_new: all_offers.delete(" \t\r\n\ "))
end
private
def compare_hashes(page)
...
end
end
What works:
calling the same class method from rails console creates the Active Job as expected. I get the following output:
irb(main):002:0> WebpageChangeCheck.check_all
Webpage Load (1.2ms) SELECT "webpages".* FROM "webpages"
Webpage Update All (3.9ms) UPDATE "webpages" SET "counter" = COALESCE("counter", 0) + $1 WHERE "webpages"."id" = $2 [["counter", 1], ["id", 1]]
Enqueued UpdateOffersHashJob (Job ID: 707e164d-b8b9-407b-aa35-4b23c37b4f07) to Async(default) with arguments: #<GlobalID:0x00007f2e0b7c4878 #uri=#<URI::GID gid://my_rails_app/Webpage/1>>
=>
[#<Webpage:0x00007f2e0b7bded8
id: 1,
title: "example.com",
url: "https://www.example.com",
user_id: 1,
created_at: Wed, 18 Jan 2023 14:22:51.904097000 CET +01:00,
updated_at: Fri, 20 Jan 2023 00:12:02.749748000 CET +01:00,
interval: 1,
checking_active: true,
selector: ".headline_content",
counter: 386>]
Webpage Load (0.6ms) SELECT "webpages".* FROM "webpages" WHERE "webpages"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Performing UpdateOffersHashJob (Job ID: 707e164d-b8b9-407b-aa35-4b23c37b4f07) from Async(default) enqueued at 2023-01-20T07:26:02Z with arguments: #<GlobalID:0x00007f2e092e1088 #uri=#<URI::GID gid://my_rails_app/Webpage/1>>
TRANSACTION (0.2ms) BEGIN
User Load (0.8ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
irb(main):003:0> TRANSACTION (1.0ms) COMMIT
TRANSACTION (0.4ms) BEGIN
Webpage Load (0.6ms) SELECT "webpages".* FROM "webpages" WHERE "webpages"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Sim Create (1.4ms) INSERT INTO "sims" ("to", "time", "api_response", "success", "webpage_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id" [["to", 49123456789], ["time", nil], ["api_response", nil], ["success", nil], ["webpage_id", 1], ["created_at", "2023-01-20 07:26:03.108189"], ["updated_at", "2023-01-20 07:26:03.108189"]]
TRANSACTION (3.5ms) COMMIT
TRANSACTION (0.4ms) BEGIN
Sim Update (1.0ms) UPDATE "sims" SET "time" = $1, "api_response" = $2, "success" = $3, "updated_at" = $4 WHERE "sims"."id" = $5 [["time", "Fri, 20.01.23 - 08h26 03s"], ["api_response", "100\nVerbucht: 0\nPreis: 0.075\nGuthaben: 0.35\nText: ALERT\nSMS-Typ: direct\nFlash SMS: false\nEncoding: gsm\nGSM0338: true\nDebug: true"], ["success", true], ["updated_at", "2023-01-20 07:26:03.295980"], ["id", 22]]
TRANSACTION (3.8ms) COMMIT
Performed UpdateOffersHashJob (Job ID: 707e164d-b8b9-407b-aa35-4b23c37b4f07) from Async(default) in 585.89ms
What I see from bash (from rake or rails runner) the Active Job gets created but it will not perform:
root#f77855c949a8:/opt/app# rake debug check_all_pages
Webpage Load (1.5ms) SELECT "webpages".* FROM "webpages"
↳ app/cron_jobs/webpage_change_check.rb:4:in `check_all'
Webpage Update All (2.9ms) UPDATE "webpages" SET "counter" = COALESCE("counter", 0) + $1 WHERE "webpages"."id" = $2 [["counter", 1], ["id", 1]]
↳ app/models/webpage.rb:9:in `check_for_change'
[ActiveJob] Enqueued UpdateOffersHashJob (Job ID: f4495cb8-868f-4ed5-9f03-7f4407b5efa4) to Async(default) with arguments: #<GlobalID:0x00007fd55771ece0 #uri=#<URI::GID gid://my_rails_app/Webpage/1>>
root#f77855c949a8:/opt/app#
here you also go with the rake task:
my_rails_app/lib/tasks/checker_task.rake:
desc "checks all Webpages for changes. Called from cronjob."
task check_all_pages: :environment do
WebpageChangeCheck.check_all
end
Conclusion:
it seems like the environment loaded properly, including environment variables. I have access to my classes and models.
nevertheless there seems to be a difference which prevents the Job from being performed.
part of the problem might be there is not enough logging to debug it, so this could be improved as well!
Any ideas?
Thanks a lot!

Answer 1

You need to process your job queue in another process, e.g.
rake jobs:work

Answer 2

So, thanks to the help in the comments I realized I overlooked to install the delayed_job backend/processing queue incl. db table for Active Job cause I thought this comes with it. It is needed to enqueue and work off the jobs. Not suitable for large amounts of jobs but sufficient for my purpose.
Here is a complete guide:
https://axiomq.com/blog/deal-with-long-running-rails-tasks-with-delayed-job/
You can also check these docs:
https://guides.rubyonrails.org/active_job_basics.html
https://github.com/collectiveidea/delayed_job#active-job
Short version:
I added the gem 'delayed_job_active_record'
Add and execute
bundle install
config/application.rb:
config.active_job.queue_adapter = :delayed_job
rails generate delayed_job:active_record
rails db:migrate
rails jobs:work
As mentioned, this is a seperate process that has to be run.
Question remains why the job performs when being run through rails console. Maybe it won't queue then but execute rightaway ..