I'm using Heroku and would like to have a script (on another server) called via cron that periodically requests an action on my Heroku app. This action will in turn run some processing that may take 1 or 2 minutes to complete. I believe Heroku has 30 second request limit, I was thinking could call a Rake task from my controller action instead.
Would this work? I'm curious if anyone has tried this yet.
Thanks.
The rake task would work as long as you don't use a HTTP request as proxy to initiate the task. In fact, if the task is forked from a HTTP Request, the timeout will be the same of the HTTP request.
You should a different method to start the task. Either a crontab (on Heroku side) or a Worker as good solutions.
I'd recommend using a background job on a worker for this. Your periodic process would then just have to start the worker and it wouldn't matter how long the process took.
I've just created a gem to solve exactly this problem. It allows you to queue up any rake task as a delayed_job e.g.
rake delay:db:seed
which will execute
rake db:seed
as a delayed_job. You can find it at http://rubygems.org/gems/delayed_task or http://blog.opsb.co.uk/long-running-rake-tasks-on-heroku.
Related
I'm trying to use cron in my application to send mails every week but I think it doesn't work on Windows.
Does anybody knows any equivalent to cron solution that works on Windows?
Windows equivalent of Unix's cron is a "Task Scheduler". You can configure your periodical task there.
Purely Ruby solution
If you want a purely Ruby solution look into:
rufus-scheduler - it's Windows cron gem.
crono - it's a in-Rails cron scheduler, so it should work anywhere.
Web services - there are plenty of free online services that would make a request to a given URL in specific time periods. This is basically a poor man's cronjob.
I recommend taking a look at Resque and the extension Resque-scheduler gems. You will need to have a resque scheduler process running with bundle exec rake resque:scheduler and at least one worker process running with QUEUE=* bundle exec rake resque:work.
If you want these services to run in the background as a windows service, you can do it with srvany.exe as described in this SO question.
The above assumes you are ok with installing Redis - a key-value store that is very popular among the Rails community as it can be easily used to support other Rails components such as caching and ActionCable, and it is awesome by itself for many multi-process use cases.
Resque is a queue system on top of Redis that allows you to define jobs that can be executed asynchronously in the background. When you run QUEUE=* bundle exec rake resque:work, a worker process runs constantly and polls the queue. Once a job is enqueued, an available worker pops it from the queue and starts working on it. This architecture is quite scalable, as you can have multiple workers listening to the queues if you'd like.
To define a job, you do this:
class MyWeeklyEmailSenderJob
def self.perform
# Your code to send weekly emails
end
end
While you can enqueue this job to the queue yourself from anywhere (e.g. from a controller as a response to an action), in your case you want it to automatically be placed into the queue once a week. This is what Resque-scheduler is for. It allows you to configure a file such as app/config/resque_schedule.yml in which you can define which jobs should be enqueued in which time interval. For example:
send_weekly_emails:
cron: 0 8 * * Mon
class: MyWeeklyEmailSenderJob
queue: email_sender_queue
description: "Send weekly emails"
Remember that a scheduling process has to run in order for this to work with bundle exec rake resque:scheduler.
thanks guys , actually i tried rufus scheduler gem and it worked for me , i guess it's the best and easier solution
I am working on rails 2 application where i need to send mail in background job.
I have Notifier model and from controller I call a model method to send mail.
I call that method Notifier.deliver_method_name(params) before using delayed job and after creating I use it Notifier.delay.deliver_method_name(params).
Now my question arise
1) Should I need to start the rake jobs:work to send mail in background in development and if yes should I run it every time I start server?
2) what do I need to do on production should I run rake jobs:work for the first time ?
3) Is that right if any error occurs then only there is a entry in delayed job if it is successful then there is no entry in delayed_job table.
I follow the below link
https://github.com/collectiveidea/delayed_job/tree/v2.0
Thanks :)
1) Should I need to start the rake jobs:work to send mail in background in development and if yes should I run it every time I start server?
Yes you have to start rake task separately.
Workers can be running on any computer, as long as they have access to the database and their clock is in sync. Keep in mind that each worker will check the database at least every 5 seconds.
2) what do I need to do on production should I run rake jobs:work for the first time ?
Yes you have to with RAILS_ENV=production script/delayed_job start But I suggest you stop and start this at every deployment to take changes done.
3) Is that right if any error occurs then only there is a entry in delayed job if it is successful then there is no entry in delayed_job table.
No it's completely depends on your configuration
# config/initializers/delayed_job_config.rb
Delayed::Worker.destroy_failed_jobs = false
Delayed::Worker.sleep_delay = 60
Delayed::Worker.max_attempts = 3
Delayed::Worker.max_run_time = 5.minutes
I usually just run "heroku run rake jobs:work" from the command line. The great thing about this apporach is that I get intimidate feedback on whether a job failed or not and what jobs are currently processing.
However, now I need to run "heroku ps:scale worker=1"
Is there a way to see what the worker is processing just like with the rake task via the command line?
You can use the heroku command like to access the log of the worker.
heroku logs -t --ps worker will show you what is currently being executed on your worker.
I also recommend to use gem workless which scales your worker up only when needed. This can save you a lot of money
You didn't say specifically what queue you are running, but it sounds like delayed_job. If so, you can install the delayed_job_web gem. Basic setup is as simple as adding it you your Gemfile and adding a route for it. Then you can browser to your site /delayed_job and manage the jobs.
I have a task that takes over 45 minutes. It runs successfully with
$ heroku run rake:sales
I also doublechecked my settings in scheduler based on this question. Everything looks fine.
In order to prevent run-away jobs, jobs that run longer than their frequency will be terminated. For example, a job that runs every 10 minutes will be terminated after running for 10 minutes.
What's happening is that your rake task is running for the first 10 minutes, but Heroku aborts it after that elapses. They suggest using a background job queue for long-running tasks.
Source:
https://devcenter.heroku.com/articles/scheduler#long-running-jobs
Apart from the long-running issue that #KKobayashi has alluded to, you may not have the correct rake file created for the scheduler to run:
Heroku Scheduler:
For Rails, the convention is to set up rake tasks. To create your
scheduled tasks in Rails, copy the code below into
lib/tasks/scheduler.rake and customize it to fit your needs
Have you tried putting your tasks into a a scheduler.rake file?
It could be that you're scheduling the task for an app other than the one you intend to schedule it for.
To check, open the scheduler (heroku addons:open scheduler) and check the url. If you see another app's name in the url, you need to add the scheduler addon again i.e.:
heroku addons:create scheduler:standard
Now open it again (heroku addons:open scheduler)
The documentation states that a delayed job worker can be invoked using a rake task like so: rake jobs:work, or QUEUE=queue1 rake jobs:work if you want it to work on a specific queue.
I have a couple of questions about this way to invoke jobs:
Is there a way to pass other parameters like sleep-delay or read-ahead (like you would do if you start the worker using the script: delayed_job start --sleep-delay 30 --read-ahead 500 --queue=queue1)?
Is there any gain in processing speed if you launch 2 workers on the same queue using the rake task?
In answer to 1. - yes you can set sleep delay and read ahead from the command line. You do it via environment variables:
QUEUE=queue1 SLEEP_DELAY=1 rake jobs:work
for example. See this commit.
rake jobs:work is just a means to an end to put up another worker, for development purposes or to work off a big queue (you have rake jobs:workoff for this though) so all benefits and disclaimers of multiple workers apply,
two jobs process in parallel so if you've got the cpu power your queue will be worked quicker
I don't know about the question #1 though, it's possible rake jobs wasn't intended to be used outside of development