I am using this plugin for scheduled job.But it is not working.
I am confused about some points,Should I need to create the Job class and set their name in to schedule file?When testing it then,Should I run the rescue scheduler and Resque worker both or only one of them.
Thanks in advance.
My Resque Scheduler config... you will mostly need all these pieces:
YML file (config/resque_scheduler.yml):
every_1_minute:
cron: "* * * * *"
class: EveryMinute
queue: some_queue
description: Tasks to perform every minute
config/initializers/resque.rb:
require 'resque_scheduler'
Resque.schedule = YAML.load_file(File.join(Rails.root, 'config/resque_scheduler.yml'))
Ruby class (lib/every_minute.rb or somewhere in the load path):
class EveryMinute
def self.perform
puts "Hello every minute!"
end
end
You need to run
rake resque:scheduler
rake resque:work
The resque:scheduler process periodically queues up jobs, hence the scheduling. And the workers will just do the jobs blindly. This is why you need BOTH to successfully schedule and run jobs periodically.
Related
I am trying to add couple of scheduled workers to my rails application. These workers will be crawling different sites in given intervals.
I want to test these workers but not able to do it. I am starting redis and my application. What should I do to see whether my scheduled jobs are working or not?
Here is my crawler class:
class AyedasCrawler
include Sidekiq::Worker
and my sidekiq.yml is:
:schedule:
ayedas_crawler:
cron: '0 * * * * *' # Runs once per minute
class: AyedasCrawler
start the sidekiq worker and the scheduler processes by running
bundle exec sidekiq or sidekiq from your app root in the command-line.
sidekiq-scheduler provides an extension to the Sidekiq web interface that adds a Recurring Jobs page.
There are two ways to do this:
In your routes.rb file, just below the require 'sidekiq/web', add require 'sidekiq-scheduler/web'
In your config.ru, just below the require 'sidekiq/web', add
require 'sidekiq-scheduler/web'
run Sidekiq::Web
On the browser, goto ==> http://localhost:{port}/sidekiq/recurring-jobs. where {port} is the port your application is running in.
You will see the list of scheduled jobs for your application and some other details about it.
Read more in the official documentation
You need to run Sidekiq process as well.
bundle exec sidekiq
It will start both worker/s and the scheduler
If you wish to test it using rspec, you can to the following:
it 'spawns scheduled workers' do
Sidekiq::Cron::Job.load_from_hash YAML.load_file('config/sidekiq.yml')[: schedule]
Sidekiq::Cron::Job.all.each(&:enque!)
expect(AyedasCrawler.jobs.size).to be(1)
end
It loads the YAML configuration, enqueues all the jobs, and asserts if the job has been enqued.
Using this method you can validate if your schedule YAML is correct. It will NOT test CRON syntax and scheduled intervals.
I'm also using https://github.com/philostler/rspec-sidekiq to allow sidekiq testing without jobs actually being executed.
Task:
Delete an item in Async manner [HomeWork]
I have already configured Active jobs with delayed_jobs in my rails application but I am still confused about performing Async task in rails project.
Let's take an example:
I have some item to delete from the database but I want to do it in Async manner. I also read about perform_later or perform_now method in delayed_job blogs. Here is my code which is working fine:
Controller class
def destroy
PostJob.perform_now(params[:id])
respond_to do |format|
format.xml { head :ok }
format.js { render 'posts.js.erb' }
end
end
Job class
class PostJob < ActiveJob::Base
queue_as :default
def perform(id)
#post = Post.find(id)
#post.destroy
end
end
According to official doc of delayed_jobs I can add handle_asynchronously in the end of method to run in async manner. How can I implement in this case?
My Question:
When I am looking at destroy method it is not deleting the element in Async way. However every steps written in destroy method is in Synchronous. Am I wrong?
If it's not then How can I implement destroy method to delete post in async manner?
Backgrounding task and cron job are same thing?
Edit -1
After giving suggestion by A Fader Darkly, I changed perform_now to perform_later which is working perfectly for Async process but it is not deleting the entry from table (code is fine because it works when i user perform_now).
Also when I am running job manually by following command, Everything works fine:
rake jobs:work
Is there any way to execute delay_job task as soon as the queue get some new data?
If you change your destroy method to call:
PostJob.perform_later(params[:id])
it should happen asynchronously. If not, you have some more set-up to do.
For your questions:
Yes you are right, but what you say is a tautology. Everything in that method is synchronous - the job queue isn't used because of the perform_now. Thus destroy isn't deleting in an async way.
See above.
Cron jobs work on the operating system level and are scheduled regularly for particular times. You could have a cron job working every minute, for example, or every day, or week (on a particular day at a particular time). They run from a schedule file called a crontab.
'Backgrounding' a task simply stops it from taking over the IO of your terminal session. So you can carry on using the terminal while the process runs in the background. Generally this is done on an ad-hoc basis, so you don't have to wait for a heavy operation to complete before going on to do different tasks.
EDIT
Based on edits to the question, it sounds like Delayed Job daemon needs to be started. From the instructions:
Note: For Rails 4 replace script/delayed_job with bin/delayed_job
When running a queue locally, omit the 'RAILS_ENV=production' part of commands.
Running Jobs
script/delayed_job can be used to manage a background process which will start working off jobs.
To do so, add gem "daemons" to your Gemfile and make sure you've run rails generate delayed_job.
You can then do the following:
RAILS_ENV=production script/delayed_job start
RAILS_ENV=production script/delayed_job stop
Runs two workers in separate processes.
RAILS_ENV=production script/delayed_job -n 2 start
RAILS_ENV=production script/delayed_job stop
Set the --queue or --queues option to work from a particular queue.
RAILS_ENV=production script/delayed_job --queue=tracking start
RAILS_ENV=production script/delayed_job --queues=mailers,tasks start
Use the --pool option to specify a worker pool. You can use this option multiple times to start different numbers of workers for different queues.
The following command will start 1 worker for the tracking queue,
2 workers for the mailers and tasks queues, and 2 workers for any jobs:
RAILS_ENV=production script/delayed_job --pool=tracking --pool=mailers,tasks:2 --pool=*:2 start
Runs all available jobs and then exits
RAILS_ENV=production script/delayed_job start --exit-on-complete
or to run in the foreground
RAILS_ENV=production script/delayed_job run --exit-on-complete
I am a beginner when it comes to Rails. I am trying to follow this example:
http://ryanselk.com/2014/09/25/using-background-jobs-in-rails-42-with-active-job/
It says:
"Jobs can be added to the job queue from anywhere. We can add a job to the queue by: ResizeImage.perform_later 'http://example.com/ex.png' "
[UPDATE] Sorry, I am stupid. I came up with this task:
namespace :simple do
# call from command line:
# rake simple:resize_images
desc "Resize images"
task resize_images: :environment do
Dir.foreach('storage') do |next_image|
puts next_image
next if next_image == '.' or next_image == '..'
ResizeImage.perform_later next_image
end
end
end
but now I do:
rake simple:resize_images
and I get:
zacek2_phpP9JGif.jpg
rake aborted!
NameError: uninitialized constant ResizeImage
I've tried:
require ResizeImage
but that did not fix the problem.
I am afraid I don't understand how loading works in Rails. How do I load ResizeImage?
Do I set it up as a cron job?
No.
From the rails guides:
Active Job is a framework for declaring jobs and making them run on a variety of queueing backends.
Active Job is an interface to queueing backends such as sidekiq, delayed_job or resque. It's simply a way for you to write background jobs where you don't have to care about which of the queueing backends will be used.
How do I start ActiveJob?
So ActiveJob doesn't run background jobs on it's own. You're still missing one of the backends. Say you have decided to use delayed_job: Get it installed and then start it via:
script/delayed_job start
I don't understand where "anywhere" is.
That means anywhere in your code, you could write something like:
user.rb
def send_registration_email
UserRegistraionMailJob.perform_later self
end
I have 5 Resque workers setup like this:
QUEUE=* rake environment resque:work
QUEUE=* rake environment resque:work
QUEUE=* rake environment resque:work
when I run a heavy job like this:
100.times do
Resque.enqueue(DoTheJob)
end
First worker gets about 80 of the job and other workers share the rest...
In my case, I may have 40 concurrent and really heavy jobs -video transcoding-. They will be triggered consecutively and I want jobs to be seperated equally or at least fairly to my existing workers (they can be up to 30).
Is there an option or something like that?
How can I achieve this?
Thank you
Well, when I try the code above with a simplest Resque worker class structure like this:
class MyWorker
#queue=:test
def self.perform(data)
puts "Testing...."
end
end
100.times do
Resque.enqueue(MyWorker)
end
All the jobs are queued to the same worker.
BUT.
When I put a few seconds sleep to the worker, I see that jobs are distrubeted pretty fairly.
class MyWorker #queue=:test
def self.perform(data)
puts "Testing...."
sleep 3
end
end
I don't know what is Resque's algorythm for this purpose but looks like there's no problem with it.
How do I create a delayed job for a rake task that should run every 15 minutes?
You can give it a try: https://github.com/defunkt/resque
I am using Resque + Redis with Heroku. Delayed job is also very much supported on their cloud service.
In lib/tasks/cron.rb
desc "This task is called by the Heroku cron add-on"
task :cron => :environment do
def resubmit_pending_jobs
Resque.enqueue(SomeJob, job.id)
end
end
One way I can think of is by using the cron addon offered by Heroku which does it every hour (not 15 mins). Perhaps the above code block can assist you in finding a similar implementation for Delayed Job.
In the case you are interested in getting Resque setup with RedisToGo and Heroku, please consult this guide.
Hope that helps!
Take a look at SimpleWorker. It's a cloud-based background processing / worker queue for Ruby apps. It's an add-on for Heroku.
You create worker classes in your code and the queue up jobs to run right away or run later -- one time or on a recurring schedule.
worker = SomeWorker.new
# Set attributes for worker to use here
worker.schedule(:start_at => 1.minute, :run_every => 900)