I'm working with delayed job for active record gem https://github.com/collectiveidea/delayed_job I'm trying to set up a job to run five minutes after an event occurs in my application. After five minutes passes, I need to make some database updates. I've tried rake jobs:work and RAILS_ENV=development script/delayed_job start. Prior to this all, I have run bundle install, rails generate delayed_job:active_record, and rake db:migrate. I have a lottery website that needs to check winners every five minutes and update tokens for winning players.
I wait five minutes, but no updates are made in my local application.
Here's what I have so far:
Gem File:
gem 'delayed_job_active_record'
gem "daemons"
Job (located in lib)
class WinnersJob < Struct.new(:blast_id)
def perform
...
end
Controller
require 'winners'
Delayed::Job.enqueue(WinnersJob.new(blast.id), 1, 5.minutes.from_now)
end
I think you have to launch the background workers locally using foreman. The following worked on my Mac.
From Heroku docs:
You then need to tell your application to process jobs put into your job queue, you can do that by adding this to your Procfile:
worker: bundle exec rake jobs:work
Now when you start your application using Foreman it will start processing your job queue.
foreman start
Having said all that, unless you are deploying on a Mac, it doesn't really matter if they run locally. (I noticed this after I got it working.) It only matters if it works on your servers. If you are deploying on Heroku, then Delayed Job works well.
Reference:
https://devcenter.heroku.com/articles/delayed-job
https://devcenter.heroku.com/articles/procfile
Related
There's a straight forward explanation of why cron jobs and tasked scheduled with whenever gem won't work on heroku
Is there any way to schedule a rake task to run more frequently than every 10 minutes (the minimum frequency heroku scheduler offers), for example every 1 minute?
You can use a combination of Clockwork gem and Heroku Procfile.
in your lib/clock.rb, you can do something like:
every(1.minute, 'Run task') do
Rake::Task['namespace:task'].invoke
end
then add clock: bundle exec clockwork lib/clock.rb to your Heroku Procfile
Clockwork supports a lot of time variations.
I think #ollaollu's method is probably superior, but for future reference what worked for me was to use a Procfile to run an infinite loop on boot, and that effectively runs a process every 5 seconds and even works after heroku restart.
A known problem with running Rails Rake tasks on Heroku is that they don't submit their logs to Papertrail since the one-off dynos push their output to the console by default. This is solved by running your dyno in "detached" mode by using heroku run:detached rake your:task. Unfortunately, the Heroku Scheduler appears to automatically run tasks as normal instead of in detached mode so these logs are lost.
How can you make the scheduler run a task in "detached" mode so these weekly/daily/hourly tasks get their logs captured by Papertrail as expected?
You can use sidekiq, this gem will help you run any processes with schedule, and inside in your sidekiq you can run rake tasks!
https://github.com/mperham/sidekiq
Example:
class MySidekiqTask
include Sidekiq::Worker
def perform
application_name = Rails.application.class.parent_name
application = Object.const_get(application_name)
application::Application.load_tasks
Rake::Task['db:migrate'].invoke
end
end
Good instruction how setup Sidekiq in Heroku server
https://itnext.io/sidekiq-overview-and-how-to-deploy-it-to-heroku-b8811fea9347
New to Rails and very new to Delayed Jobs.
Got one that's supposed to be triggered after 5 minutes. I finally got it to work so that if I run
rake jobs:work
in my terminal, the job starts up and works correctly. If I CTRL-C and exit that action in my terminal, the delayed job stops working correctly. This is one thing on my local server and another on Heroku, where I have to start up the delayed job using
heroku run rake jobs:work
I looked into the new Heroku toolbelt and downloaded the gem they suggest for worker maintenance, foreman, but when I run "foreman start", I get this error
ERROR: procfile does not exist
I don't know what a procfile is, I'm afraid of breaking things after spending pretty much a day debugging my delayed_jobs actions, and I want to do this right to make sure it works instead of figuring out some hacky fix that breaks down later -- so I figured I should ask this question, however obnoxiously vague it may be.
Should I be using foreman for this? Or workless? (Saw that in another SO question). Where's my procfile? Should I do anything with it?
Thanks,
Sasha
You should be using a procfile to set up your Heroku processes, this is the standard method that Heroku uses to define and control the processes.
If you haven't utilised a procfile to this point everything will probably still work as Heroku adds some default processes when you push a Rails app, including both the web and worker processes. The default worker process is set to delayed job.
Foreman was developed in order to set up your local machine to use the same approach but, unlike the Heroku service, Foreman actually requires a procfile to be present to control the services that are started when Foreman is started as it doesn't know how to setup defaults.
I would suggest creating a procfile, placed in the root directory of your project, to ensure that your processes are set up and operating in the same manner on your local machine as on Heroku. If you want to mimic what Heroku sets up automatically you add the following to the procfile depending on whether you are using the Thin web server (which Heroku recommends) or not.
With Thin in your gemfile:
web: bundle exec thin start -R config.ru -e $RACK_ENV -p $PORT
worker: bundle exec rake jobs:work
Without a special web server (eg you are using webrick, the rails default):
web: bundle exec rails server -p $PORT
worker: bundle exec rake jobs:work
Once this file is in place you can run foreman on your local machine and it will start your web server and delayed_job workers automatically.
Running through this process will only impact starting delayed_job on the local machine. As you are running the exact same command bundle exec rake jobs:work as you are currently using there should be no impact on your dj actions in either locally or on Heroku. Obviously some testing is required to make suer this is actually the case.
Workless is designed to scale workers on Heroku so that you don't have to pay for them when there is no work available. It has no bearing on the procfile or defining how to actually start a dj worker process.
as far as I know, there are 2 version of delayed_job:
original(tobi's) https://github.com/tobi/delayed_job
collectiveidea's fork: https://github.com/collectiveidea/delayed_job
when using the collectiveidea version, you should start it as below:
# Runs two workers in separate processes.
$ RAILS_ENV=production script/delayed_job -n 2 start
I am not familiar with delayed_job on Heroku, please follow its instructions.
I've added gem 'delayed_job' to my gem file and ran a bundle install.
After that I ran rails generate delayed_job
I've created a controller named Online with a method online.
In turn after the method declaration I added the following line:
handle_asynchronously :online
I start up my app, but the code in that method does not run.
What am I doing wrong?
I'd guess that you haven't done rake jobs:work anywhere. From the fine manual:
Running the jobs
You can invoke rake jobs:work which will start working off jobs. You can cancel the rake task with CTRL-C.
You might want to set up Foreman to start the Rails server and the Rake task at the same time in your development environment; there's even a Railscast about it:
http://railscasts.com/episodes/281-foreman
I have an app on Heroku running delayed jobs. However at the moment I have to start the job queue running with the terminal command:
heroku rake jobs:work
...but this means when I shut down my terminal the app's delayed job queue shuts down too.
Is there a way I can get Heroku to just always start and run delayed job in the background when the app starts up? Without having to run the command each time and without having it directly linked to my terminal shell?
Thanks very much.
Edit:
It's on the bamboo stack. Upping workers or running rake jobs:work , the delayed jobs runs for a while, but then the queue seems to just stop getting processed. There are no errors in the delayed jobs queue, the workers just stop processing the jobs. It has to explicitly restarted every 5 or 10 mins.
From the docs:
On Heroku's Aspen or Bamboo stack, use heroku workers 1
On the Cedar stack, you put this line in your Procfile:
worker: bundle exec rake jobs:work
And then do heroku scale worker=1.
we use the workless gem with our heroku stack. it starts worker when the delayed_job queue > 0 and quits the worker when delayed_job queue goes to 0.
It turns out that I was using the wrong rake gem.
The following was causing an issues with rails 3 on Heroku:
gem 'rake', '0.9.2'
Updating the gem fixed the issues, even though there were not errors in the log:
gem "rake", "0.8.7"