I am trying to understand how to execute custom code with clockwork. This is the example lib/clock.rb file that Heroku uses in its devcenter document.
require File.expand_path('../../config/boot', __FILE__)
require File.expand_path('../../config/environment', __FILE__)
require 'clockwork'
include Clockwork
every(4.minutes, 'Queueing interval job') { Delayed::Job.enqueue IntervalJob.new }
every(1.day, 'Queueing scheduled job', :at => '14:17') { Delayed::Job.enqueue ScheduledJob.new }
What is IntervalJob and ScheduledJob? Where are these files supposed to be located? I want to run my own custom job that has access to my database records.
EDIT
This is my /lib/clock.rb
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job|
puts "Running #{job}"
end
every(2.minutes, 'Filtering Streams') { Delayed::Job.enqueue FilterJob.new}
end
This is my /lib/filter_job.rb
class FilterJob
def perform
#streams = Stream.all
#streams.each do |stream|
# manipulating stream properties
end
end
end
I get the error:
uninitialized constant Clockwork::FilterJob (NameError)
/app/lib/clock.rb:11:in `block in <module:Clockwork>'
You need to do the following:
Firstly install the clockwork gem.
In your lib folder create a clock.rb
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job|
puts "Running #{job}"
end
every(1.day, 'Creating Cycle', :at => '22:00') { Delayed::Job.enqueue CyclePlannerJob.new}
end
In the example your provided IntervalJob and ScheduledJob, are delayed jobs. Clockwork triggers them on the time specified. I am calling the CyclePlannerJob, this is what my file looks like.
lib/cycle_planner_job.rb
class CyclePlannerJob
def perform
CyclePlanner.all.each do |planner|
if Time.now.in_time_zone("Eastern Time (US & Canada)").to_date.send("#{planner.start_day.downcase}?")
planner.create_cycle
end
end
end
end
In my example everyday at 10pm, I am running the CyclePlanner job, which runs the delayed job I have setup. Similar to the Heroku example.
Bare in mind to use this you need to setup the clock work and delayed jobs on your Heroku app in the dashboard.
Also your Procfile should look like this.
worker: bundle exec rake jobs:work
clock: bundle exec clockwork lib/clock.rb
Let me know if you have any questions, I can go into more detail if needed.
Looks like name space issue. Move your filter_job.rb to models directory and try.
Related
I'm trying to create background jobs for email notification and scraper.
I use resque-scheduler (4.0.0), resque (1.25.2) and rails 4.2.1.
My config.ru file:
# This file is used by Rack-based servers to start the application.
require ::File.expand_path('../config/environment', __FILE__)
run Rails.application
require 'resque/server'
run Rack::URLMap.new "/" => AppName::Application, "/resque" => Resque::Server.new
My /lib/tasks/resque.rake:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
end
end
My /config/resque_scheduler.yml:
CheckFsUpdatesJob:
queue: fs_updates
every:
- '1h'
- :first_in: '10s'
class: CheckFsUpdatesJob
args:
description: scrape page
My /config/initializer/active_job.rb
ActiveJob::Base.queue_adapter = :resque
My /config/initializer/resque.rb:
#config/initializers/resque.rb
require 'resque-scheduler'
require 'resque/scheduler/server'
uri = URI.parse("redis://localhost:6379/")
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Resque.schedule = YAML.load_file(Rails.root.join('config', 'resque_schedule.yml'))
Resque::Server.use(Rack::Auth::Basic) do |user, password|
user = 'admin'
password = 'admin'
end
My first job for emails notifications:
class EmailNotificationJob < ActiveJob::Base
queue_as :email_notifications
def perform(episode_id, email)
NotificationMailer.new_record_appears(record_id, email).deliver_now
end
end
My second job for scheduled runs:
class CheckFsUpdatesJob < ActiveJob::Base
queue_as :fs_updates
def perform()
FsStrategy.new.check_for_updates
end
end
So I have to jobs:
1. emails notifications - should sends email when new record in DB appears
2. scrape a page - should runs every hour
How I run it:
redis-server
rake environment resque:work QUEUE=fs_updates
rake environment resque:work QUEUE=email_notifications
rake environment resque:scheduler
rails s
After running these commands I see in Resque Dashboard two workers and two queues, as it is expected.
But!
After clicking on 'queue now' button at resque Schedule tab, I see that task was created and wroted to "fs_updates" queue. But it's not running and in a few second it dissapears.
When I run a job for emails sending from rails console - it does not work at all.
Please, help me to fix my configurations.
Thanks kindly!
As I understood: rails and active_job developers is not responsible for resque plugins. Maybe this problem will be fixed in new gem versions, but now it does not work (active_job does not work fine with resque-scheduler).
Currently I use gem 'active_scheduler' to fix current problem.
I had the same issue trying to configure Sucker Punch on rails 4.2.1 In the end I moved my custom initialiser logic into application.rb, not great but it got me up and running.
Not sure if there is an issue with the initialisers in this release. Try moving your code from active_job.rb and resque.rb into application.rb or the appropriate environment file. Obviously this isn't a long term solution but it will at least help you you identify whether it's an initialiser issue or Resque config problem.
Running Rails 4.2.0 so I am using the ActiveJob w/ Sidekiq backend. I need to invoke regularly scheduled background jobs so I am hoping to use Clockwork, but I haven't seen any examples of how to use it with ActiveJob.
Here is my lib/clock.rb based off the Clockwork Github example:
require 'activejob'
module Clockwork
handler do |job, time|
puts "Running #{job}, at #{time}"
ProductUpdateJob.perform_later
end
every(30.seconds, 'ProductUpdateJob.perform_later')
end
Update:
I was able to get it to work for my situation, but I'm not entirely satisfied with the solution since I am having to load the entire environment. I would only like to require the bare minimum to run the ActiveJobs.
Dir["./jobs/*.rb"].each {|file| require file }
require 'clockwork'
require './config/boot'
require './config/environment'
#require 'active_job'
#require 'active_support'
module Clockwork
handler do |job, time|
puts "Running #{job}, at #{time}"
#send("#{job}")
#puts "#{job}".constantize
"#{job}".constantize.perform_later
end
every(10.seconds, 'ProductUpdateJob')
end
Here is a working version
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job, time|
puts "Running #{job}, at #{time}"
"#{job}".constantize.perform_async
end
every(10.seconds, 'ProductUpdateJob')
end
It is possible, but it will have no context of your redis/job backend configuration. You could always require a file in config/initializers that configures sidekiq and redis. This is the minimum you will need:
require 'clockwork'
require 'active_job'
ActiveJob::Base.queue_adapter = :sidekiq
Dir.glob(File.join(File.expand_path('.'), 'app', 'jobs', '**')).each { |f| require f }
module Clockwork
every(1.minute, 'Do something') { SomeJob.perform_later }
end
I'm creating a DripEmail campaign for my app and using resque scheduler to schedule the tasks.
I've sceduled a static job, which runs every day at a specific time and collect the user's list based on the drip settings and sends out an emailer to them.
This is my resque job user_follow_up.rb
class UserFollowUp
#queue = :user_follow_up
def self.perform
User.each do |u|
# Send the emailers to only those who are not converted
if !user.is_converted and Date.today <= user.next_email
stage(u)
end
end
end
end
This is my scheduler.yml
UserFollowUp:
cron: "0 16 * * *"
I have 2 resque workers, one has my default set of tasks and the other for scheduler.
rake environment resque:work QUEUE=publish_story,accept_story,image_queue,Mango_mailer
and
rake environment resque:scheduler QUEUE=user_follow_up
When I open the resque admin interface, I'm able to see my static job detected in the list. I clicked the Queue Now button to test it. It properly enqueues the task to the queue, but doesn't execute. It keeps these tasks in the pending queue forever.
This is my resque.rake, it's required
require 'resque/tasks'
require 'resque/scheduler/tasks'
require 'resque/scheduler/server'
require 'active_record'
require 'mongoid'
require 'action_controller/railtie'
require 'active_support/buffered_logger'
# load the Rails app all the time
namespace :resque do
puts 'Loading Rails environment for Resque'
task :setup => :environment do
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file("#{Rails.root}/config/scheduler.yml")
Resque::Scheduler.dynamic = true
Resque.logger.info 'Resque Scheduler Initialized!'
Resque.before_first_fork do
# Open the new separate log file
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
# Activate file synchronization
logfile.sync = true
# Create a new buffered logger
Resque.logger = ActiveSupport::BufferedLogger.new(logfile)
Resque.logger.level = Logger::INFO
Resque.logger.info 'Resque Logger Initialized!'
puts 'Resque Logger Initialized!'
end
end
task 'resque:pool:setup' do
Resque::Pool.after_prefork do |job|
Resque.redis.client.reconnect
end
end
end
And this is my Gemfile
gem 'resque', github: 'resque/resque' , branch: '1-x-stable'
gem 'resque_mailer', github: 'zapnap/resque_mailer'
gem 'resque-scheduler'
I'm not sure what is the issue here. Please help me with this fix.
I am trying to run a background job on Heroku using Resque/Clockwork.
Locally, jobs are queued by Clockwork and executed by Resque as expected. On Heroku, sometimes it works... but most of the time Clockwork queues the job as expected, and when executed by Resque I get a failed job with the error 'No job defined for class'.
The exact error that shows up through the Resque admin panel is:
Worker 960f8a1b-cce9-497a-a7ab-9b40c166a600:2+1 on FEATURED_CATALOG_ITEMS_CACHE at 27 minutes ago Retry or Remove
Class nil
Arguments nil
Exception Error
Error No job defined for class 'Workers::FeaturedCatalogItemsCacheWorker'
My code looks like:
/app/models/workers/featured_catalog_items_cache_worker.rb
class Workers::FeaturedCatalogItemsCacheWorker
#queue = :featured_catalog_items_cache
def self.perform
p 'do work' # there is more code here but edited for brevity
end
end
/lib/clock.rb
require File.expand_path('../../config/boot', __FILE__)
require File.expand_path('../../config/environment', __FILE__)
require 'clockwork'
module Clockwork
every(2.minutes, 'Queueing FeaturedCatalogItemsCacheWorker job') { Resque.enqueue(Workers::FeaturedCatalogItemsCacheWorker) }
end
/lib/tasks/resque.rake
require 'resque/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
end
Procfile
web: rackup -p $PORT
resque: env TERM_CHILD=1 bundle exec rake resque:work
clock: bundle exec clockwork lib/clock.rb
Am I missing something?
maybe in /lib/clock.rb
require 'Workers::FeaturedCatalogItemsCacheWorker'
Solution :
require './config/boot'
require './config/environment'
require 'clockwork'
require 'resque'
module Clockwork
handler do |job|
puts "Running job : #{job}"
Resque.enqueue(job)
end
every(15.minutes, NewsRssReader )
end
I don't understand why my rake task is not running from within a resque worker. Running
rake :send_this_email
from the console works fine, I just want to run it as a cron job (as follows) but something is not working proplerly while invoking the rake task from within the worker.
My rescue_schedule.yml
send_this_email:
cron: "*/2 * * * *"
class: SendThisEmailWorker
args:
description: "Send email when condition defined in rake task is met"
My send_this_email_worker.rb in workers directory, where the problem must be if I can manually call the rake task myself from the console?
require 'rake'
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
Rake::Task["send_this_email"].invoke
end
end
When I start my dev server this send_this_email rake task should run every 2 minutes correct? It's not and the resque admin panel shows it as a job in the queue. What am I missing here?
Thanks for your attention.
UPDATED from gerep comment
require 'rake'
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
puts "Hi from the console, I'm started"
Rake::Task["send_this_email"].invoke
end
end
Only require 'rake' is not enough. For example if you do
Rake::Task.tasks #list down all task
You will get []
You need to tell your worker class to load tasks.
Try this
require 'rake'
Rake::Task.clear # necessary to avoid tasks being loaded several times in dev mode
YOUR_APP_NAME::Application.load_tasks
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
puts "Hi from the console, I'm started"
Rake::Task["send_this_email"].invoke
end
end
YOUR_APP_NAME is the name of your app and can be found at config/application.rb