I am trying to implement an API endpoint that would queue a request and return immediately.
I am using the gem https://rubygems.org/gems/activejob/versions/5.2.0 (I am on an old version for historical reasons).
I have defined a job that looks something like:
class Service::ExportBooks::Job < ActiveJob::Base
def perform
## ... Do the job
rescue StandardError
binding.pry
raise
end
end
In the controller, I am calling:
Service::ExportBooks::Job.perform_later
The job gets called synchronously and the controller gets even any errors raised by the job.
I've also tried other options such as:
job = Service::ExportBooks::Job.new
job.enqueue(wait: 5.seconds)
but it does the same, the job is not enqueued, is immediately executed.
UPDATE:
It looks like the method Resque.inline? returns true and so the execution is inline and not async. How can I make sure that it's async? I tried to set Resque.inline = false manually and the job was queued but it wasn't executed...
I have started a worker using the command:
QUEUE=* PIDFILE=./tmp/resque.pid bundle exec rake environment resque:work
Two things to do here.
Make sure Resque.inline = false.
Start up the resque workers in another process. See here.
This will get the job enqueued and run on the worker process.
Related
I wish I could decide on which queue to go to work.
This is because if the job is scheduled by the server (cronjob) it must be run on a slow queue, if instead it is run by the user it will go on a fast queue.
How can I run this in Resque?
Controller
MyJob.perform_later(id, :fast)
Rake task
MyJob.perform_later(id, :slow)
Job
class MyJob < ApplicationJob
queue_as :default #<-- This has to be dynamic
def perform(item_id, queue_name)
....
end
I see you are using ActiveJob, you can set the queue by using set method:
Controller
MyJob.set(queue: :fast).perform_later(id)
Rake task
MyJob.set(queue: :slow).perform_later(id)
set method allows you to set more thing than just queue, you can also set eg priority or when should job be performed. See the documentation https://api.rubyonrails.org/v5.2.3/classes/ActiveJob/Core/ClassMethods.html#method-i-set
Note: I presume, you already have slow and fast Resque queues in place and running and only want to use them
I'm wondering what is most efficient to send a bunch of emails.
should I put the loop on the rake tasks with delayed job only doing the sending?
task :publish => :environment do
# insert loop here do
# insert delayed job here
end
end
should I put the loop inside the delayed job?
task :publish => :environment do
# insert delayed job here
end
# and on the job:
def perform
# insert loop here
end
It depends on how many background workers you have. If you have more than one worker, then the first option (creating each job separately, with the loop inside the rake task) is far better, as it allows those tasks to be run in parallel.
It also makes it easier to write your worker method, as you don't need to worry about rerunning the worker over the entire list if it happens to fall over or be terminated. (although it's still good practice to ensure that your workers are idempotent where practical!)
I have some methods that works with API of third party app. To do it on button click is no problem, but it should be permanent process.
How to run them background? And how to pause the cycle for make some other works with same API and resume the cycle after the job is done.
Now I read about ActiveJob, but its has time dependences only...
UPDATE
I've tried to make it with whenever and sidekiq, task runs, but it do nothing. Where to look for logs I can't understand.
**schedule.rb**
every 1.minute do
runner "UpdateWorker.perform_async"
end
**update_worker.rb**
class UpdateWorker
include Sidekiq::Worker
include CommonMods
def perform
logger.info "Things are happening."
logger.debug "Here's some info: #{hash.inspect}"
myMethod
end
def myMethod
....
....
....
end
end
It's not exactly what I need, but better then nothing. Can somebody explain me with examples?
UPDATE 2 After manipulating with code it's absolutely necessary to restart sidekiq . With this problem is solved, but I'm not sure that this is the best way.
You can define a job which enqueues itself:
class MyJob < ActiveJob::Base
def perform(*args)
# Do something unless some flag is raised
ensure
self.class.set(wait: 1.hour).perform_later(*args)
end
end
There are several libraries to schedule jobs on a regular basis. For example you could use to sidekiq-cron to run a job every minute.
If you want to pause it for some time, you could set a flag somewhere (Redis/database/file) and skip execution as long it is detected.
On a somewhat related note: don't use sidetiq. It was really great but it's not maintained anymore and has incompatibilities to current Sidekiq versions.
Just enqueue next execution in ensure section after job completes after checking some flag that indicates that it should.
Also i recommend adding some delay there so that you don't end up with dead loop on some error inside job
I dont know ActiveJobs, but I can recommend the whenever gem to create cron (periodic background) jobs. Basically you end up writing a rake tasks. Like this:
desc 'send digest email'
task send_digest_email: :environment do
# ... set options if any
UserMailer.digest_email_update(options).deliver!
end
I never added a rake task to itself but for repeated processing you could do somehow like this (from answers to this specific question)
Rake::Task["send_digest_email"].execute
I'm using rufus-scheduler to schedule jobs at a certain date through the following code:
job = Rufus::Scheduler.singleton.schedule_at #post.read_attribute(:parse_time).to_s do
end
I then save the id of that job in my post class
#post.update_attribute(:job_id, job.id)
However, if I try and access that job again by calling:
Rufus::Scheduler.singleton.job(#post.read_attribute(:job_id)).unschedule
I get an error because the job is nil. If I try and look at the jobs of the Scheduler by calling:
Rufus::Scheduler.singleton.jobs
I get a blank array. Can anyone explain why my jobs aren't saving properly kept / being tracked?
Here's my initialization file for the scheduler. Do I have to do anything to enable singleton though? Or does it come with rails automatically:
require 'rufus-scheduler'
# Create singleton rufus scheduler
s = Rufus::Scheduler.singleton
rufus-scheduler doesn't keep triggered jobs around.
Your job has probably triggered and is gone.
I am building an application where at some point I need to sync a bunch of data from fb with my database, so I am (attemtping) to use Delayed Job to push this into the background. Here is what part of my Delayed Job class looks like.
class FbSyncJob < Struct.new(:user_id)
require 'RsvpHelper'
def perform
user = User.find(user_id)
FbSyncJob.sync_user(user)
end
def FbSyncJob.sync_user(user)
friends = HTTParty.get(
"https://graph.facebook.com/me/friends?access_token=#{user.fb['token']}"
)
friends_list = friends["data"].map { |friend| friend["id"] }
user.fb["friends"] = friends_list
user.fb["sync"]["friends"] = Time.now
user.save!
FbSyncJob.friend_crawl(user)
end
end
With the RsvpHelper class living in lib/RsvpHelper.rb. So at some point in my application I call Delayed::Job.enqueue(FbSyncJob.new(user.id)) with a known valid user. The worker I set up even tells me that the job has been completed successfully:
1 jobs processed at 37.1777 j/s, 0 failed
However when I check the user in the database he has not had his friends list updated. Am I doing something wrong or what? Thanks so much for the help this has been driving me crazy.
Delayed::Job.enqueue will put a record in the delayed job table, but you need to run a seperate process to execute the job code (perform method)
typically in development this would be bundle exec rake jobs:work (NOTE: you must restart this rake task anytime you make code changes, it will not auto load changes)
see https://github.com/collectiveidea/delayed_job#running-jobs
I usually put the following into my delayed configuration while in development - this never puts a record in the delayed job table and runs all background code synchronously (in development) and by default rails will reload changes to your code
Delayed::Worker.delay_jobs = !(Rails.env.test? || Rails.env.development?)
https://github.com/collectiveidea/delayed_job#gory-details (see config/initializers/delayed_job_config.rb example section)