I use http caching on Heroku:
def homepage
response.headers['Cache-Control'] = 'public, max-age=86340'
...
end
I also have added Heroku's free cron addon:
desc "This task is called by the Heroku cron add-on"
task :cron => :environment do
if Time.now.hour == 0 # run at midnight
# I want to request a page here
end
end
Could you tell me what should I put inside this file in order to request a page?
A similar question has been asked at How to force fragment cache on rails from cron schedule?
They don't provide an example in the answer.
require 'uri'
require 'net/http'
desc "This task is called by the Heroku cron add-on"
task :cron => :environment do
if Time.now.hour == 0 # run at midnight
uri = URI.parse('http://my-app.heroku.com/page')
Net::HTTP.get(uri)
end
end
That should do it.
Related
I'm creating a DripEmail campaign for my app and using resque scheduler to schedule the tasks.
I've sceduled a static job, which runs every day at a specific time and collect the user's list based on the drip settings and sends out an emailer to them.
This is my resque job user_follow_up.rb
class UserFollowUp
#queue = :user_follow_up
def self.perform
User.each do |u|
# Send the emailers to only those who are not converted
if !user.is_converted and Date.today <= user.next_email
stage(u)
end
end
end
end
This is my scheduler.yml
UserFollowUp:
cron: "0 16 * * *"
I have 2 resque workers, one has my default set of tasks and the other for scheduler.
rake environment resque:work QUEUE=publish_story,accept_story,image_queue,Mango_mailer
and
rake environment resque:scheduler QUEUE=user_follow_up
When I open the resque admin interface, I'm able to see my static job detected in the list. I clicked the Queue Now button to test it. It properly enqueues the task to the queue, but doesn't execute. It keeps these tasks in the pending queue forever.
This is my resque.rake, it's required
require 'resque/tasks'
require 'resque/scheduler/tasks'
require 'resque/scheduler/server'
require 'active_record'
require 'mongoid'
require 'action_controller/railtie'
require 'active_support/buffered_logger'
# load the Rails app all the time
namespace :resque do
puts 'Loading Rails environment for Resque'
task :setup => :environment do
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file("#{Rails.root}/config/scheduler.yml")
Resque::Scheduler.dynamic = true
Resque.logger.info 'Resque Scheduler Initialized!'
Resque.before_first_fork do
# Open the new separate log file
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
# Activate file synchronization
logfile.sync = true
# Create a new buffered logger
Resque.logger = ActiveSupport::BufferedLogger.new(logfile)
Resque.logger.level = Logger::INFO
Resque.logger.info 'Resque Logger Initialized!'
puts 'Resque Logger Initialized!'
end
end
task 'resque:pool:setup' do
Resque::Pool.after_prefork do |job|
Resque.redis.client.reconnect
end
end
end
And this is my Gemfile
gem 'resque', github: 'resque/resque' , branch: '1-x-stable'
gem 'resque_mailer', github: 'zapnap/resque_mailer'
gem 'resque-scheduler'
I'm not sure what is the issue here. Please help me with this fix.
Have just installed whenever gem https://github.com/javan/whenever to run my rake tasks, which are nokogiri / feedzilla dependent scraping tasks.
eg my tasks are called grab_bbc, grab_guardian etc
My question - as I update my site, I keep add more tasks to scheduler.rake.
What should I write in my config/schedule.rb to make all rake tasks run, no matter what they are called?
Would something like this work?
every 12.hours do
rake:task.each do |task|
runner task
end
end
Am new to Cron, using RoR 4.
namespace :sc do
desc 'All'
task all: [:create_categories, :create_subcategories]
desc 'Create categories'
task create_categories: :environment do
# your code
end
desc 'Create subcategories'
task create_subcategories: :environment do
# your code
end
end
in console write $ rake sc:all
write separate rake tasks for each scraping tasks. then write a aggregated task to run all those scraping rake tasks.
desc "scrape nytimes"
task :scrape_nytimes do
# scraping method
end
desc "scrape guardian"
task :scrape_guardian do
# scraping method
end
desc "perform all scraping"
task :scrape do
Rake::Task[:scrape_nytimes].execute
Rake::Task[:scrape_guardian].execute
end
then call the rake task as
rake scrape
Make sure you have a unique namespace with all the tasks in it, like:
namespace :scrapers do
desc "Scraper Number 1"
task :scrape_me do
# Your code here
end
desc "Scraper Number 2"
task :scrape_it do
# Your code here
end
end
You could then run all tasks of that namespace with a task outside of that namespace:
task :run_all_scrapers do
Rake.application.tasks.each do |task|
task.invoke if task.name.starts_with?("scrapers:")
end
end
That said, I'm pretty sure that this is not how you should run a set of scrapers. If for any reason the if part should return true you might unintenionally run tasks like rake db:drop
Either "manually" maintaining schedule.rb or a master task seems like a better option to me.
The aggregated task can be concise:
namespace :scrape do
desc "scrape nytimes"
task :nytimes do
# scraping method
end
desc "scrape guardian"
task :guardian do
# scraping method
end
end
desc "perform all scraping"
task scrape: ['scrape:nytimes', 'scrape:guardian']
Namespaces are also a good practice.
Use namespace and in_namespace to run all tasks dynamically.
I prefer this method because it keeps things clean and precludes you from having to remember to update your "parent" task if any of our namespace tasks change.
Note, the example was borrowed from Dmitry Shvetsov's excellent answer.
namespace :scrape do
desc "scrape nytimes"
task :nytimes do
# scraping method
end
desc "scrape guardian"
task :guardian do
# scraping method
end
end
desc "perform all scraping"
task :scrape do
Rake.application.in_namespace( :scrape ){ |namespace| namespace.tasks.each( &:invoke ) }
end
I'm trying to create a Rake task that invokes two other rake tasks. I've found people with related questions here and here, but it hasn't been very useful. This is what I've cobbled together so far. Any idea what I'm doing wrong?
task :cron => :environment do
#if Time.now.hour % 2 == 0
Rake::Task["robots:update_robots"].reenable
Rake::Task["robots:update_robots"].invoke
#end
end
As you can see, it's a cron job that's meant for Heroku to do. But I've commented out what I don't need so I can test that it's working.
I keep getting this error:
Don't know how to build task 'robots:update_robots'
But I have no idea why.
UPDATE: So I it turns out I wasn't able to run the original task that was being called by my cron rake task. I had it running ok for a while, buy somewhere along the line, I deleted the "d" in "update". So this command
Rake::Task["robots:upate_robots"].execute
didn't because the robots rake task was "upate", not "update".
Tl;dr: typos.
In general, your solution should work:
require 'rake'
task :environment do
puts 'task environment'
end
namespace :robots do
task :update_robots do
puts "task robots:update_robots"
end
end
task cron: :environment do
puts 'task cron'
Rake::Task['robots:update_robots'].reenable
Rake::Task['robots:update_robots'].invoke
end
Rake::Task['robots:update_robots'].invoke
Rake::Task[:cron].invoke
# >> robots:update_robots was invoked
# >> task robots:update_robots
# >> task environment
# >> task cron
# >> task robots:update_robots
My first thought is that you must have the rake task wrong (are you sure it's "robots:update_robots" ?)
It's unusual to me that you need to reenable it, this implies that what you want is not Rake, but just plain old Ruby. Move the contents of the update_robots task out to a method which you can then invoke directly instead of trying to treat tasks like methods (tasks are for handling dependencies, they only invoke once on purpose, and your need to bend them around that implies you're using the wrong tool for the job). Then, both your code and the robots:update_robots can just call the same method:
require 'rake'
def update_robots
puts "method update_robots"
end
task :environment do
puts 'task environment'
end
namespace :robots do
task :update_robots do
update_robots
puts "task robots:update_robots"
end
end
task cron: :environment do
puts 'task cron'
update_robots
end
Rake::Task['robots:update_robots'].invoke
Rake::Task[:cron].invoke
# >> method update_robots
# >> task robots:update_robots
# >> task environment
# >> task cron
# >> method update_robots
I followed the railscast http://railscasts.com/episodes/164-cron-in-ruby but I cant seem to make it worked.
I have schedule.rb in my config.
I wished to refresh my Database.count everyday in my homepage.
I cannot find the deploy.rb in my folder. Where is it?
For testing purpose, I changed it to every 2 seconds.
[schedule.rb]
every '2 * * * *' do
rake "pages_controller:home"
end
[pages_controller.rb]
class PagesController < ApplicationController
def home
#title = "Home"
#companies = Company.find(:all, :limit => 20)
#count = Company.count
end
I have put
gem 'whenever', :require => false
in my gemfile. What have gone missing?
I have used cron job but i run it as rake task please you can try it
every 3.minutes do
set :environment, 'development'
rake "daily",:output => {:error => 'error.log', :standard => 'cron.log'}
end
And my task is like
require 'rubygems'
task :daily => :environment do
puts "i am fine"
# do your code
end
If cron job run fine then there will be nothing in cron.log.Otherwise it will show you if
any error occurs and this file will be generate in your app root directory.
try it..
So... Couple things.
You need a Capfile and a config/deploy.rb to deploy your code using Capistrano and Whenver. You get this by running capify . ... You should most likely watch the railscast on capistrano deployment
Whenever is using configured like:
schedule.rb
every 1.day, :at => '4:30 am' do
runner 'Rails.cache.clear'
end
I really don't think rake "pages_controller:home is going to work unless this is something you've already created elsewhere. I'm further assuming you are caching this page, and that's why you need to refresh the cache.
Finally, you're setting your environment to development, which makes me think you are not deploying this, but instead just wanting to reset the cached home page... So just run rake cache:clear
I have two questions:
How can I add a heroku worker just before running a delayed job and remove it after it finishes?
Is my cron.rake ok?
cron.rake:
desc "This task is called by the Heroku cron add-on"
task :cron => :environment do
puts "requesting homepage to refresh cache"
uri = URI.parse('http://something.com')
Net::HTTP.get(uri)
puts "end requesting homepage"
puts "start sending daily mail"
User.notified_today.each do |user|
Delayed::Job.enqueue UserMailer.daily_mail(user).deliver
end
puts "end sending daily mail"
end
I use collectiveidea delayed_job.
I've had good success with HireFire.
Easy setup:
Add gem 'hirefire' to your Gemfile
Create Rails.root/config/initializers/hirefire.rb with the config information.
To add remove/remove workers, hook into your ORM's after :create / after :destroy
With DataMapper on Heroku, I did it like this (You must set the ENV vars yourself)
MAX_CONCURRENT_WORKERS = 5
if ENV["HEROKU_APP"]
Delayed::Job.after :create do
workers_needed = [Delayed::Job.count, MAX_CONCURRENT_WORKERS].min
client = Heroku::Client.new(ENV['HEROKU_USERNAME'], ENV['HEROKU_PASSWORD'])
client.set_workers(ENV['HEROKU_APP'], workers_needed)
puts "- Initialized Heroku workers for ZipDecoder"
end
Delayed::Job.after :destroy do
workers_needed = [Delayed::Job.count, MAX_CONCURRENT_WORKERS].min
client = Heroku::Client.new(ENV['HEROKU_USERNAME'], ENV['HEROKU_PASSWORD'])
client.set_workers(ENV['HEROKU_APP'], workers_needed)
puts "- Cleaned Up a Delayed Job for ZipDecoder ---------------------------------"
end
end
You maybe can use an "autoscale" plugin like workless or heroku-autoscale.
About the cron I don't see any problem on it...