I'd like to make an email notification if SomeModel has not been updated for 2 hours.
What is the best way to implement it?
After a model has been saved, queue up a background job to run 2 hours from that time to send the email. When a new job is enqueued, remove any still-unrun jobs that are still on the queue.
resque-scheduler providers a pretty simple way of doing this, assuming you have redis up and running.
Personally I find the solution that #x1a4 proposes to be somewhat overkill. Given the relatively large window of 2 hours, I would just run a job periodically (say, once every 10-15 minutes), then search all Models for updated_at <= 2.hours.ago and send out the emails.
As for scheduling that job to run every 15 minutes, there are several options. You may use resque-scheduler, if you are using Resque. You may also use the standard system cron, but will incur some fairly substantial overhead starting Rails each time the job runs. I also have written a distributed scheduler gem (i.e. cron that can run on multiple machines, but act like it's only running on one), which uses Redis under the hood.
Related
I'm migrating from Delayed_jobs to Resque and I have difficulties finding the best way to handle those cases:
A user can NOT add twice the same command to the list of jobs (e.g. "export all my data"). Only one export command at a time. For other it's fine to have many (e.g. send emails)
Some jobs should not run for more than 5 minutes, while other are allowed to run for 30 minutes. In both cases, I'd like to have a time-out in case process is blocked or is not completed on time.
Can add jobs to start in a few days
Inform the user on all their current & future jobs.
Can cancel some jobs (current and future) for the user
Keep ability to have different lists (mostly for priorities / slow and fast tasks)
I looked at resque-status and it seems like it provides the low level query, but I would still need to do my per user job management.
Suggestions on best way to handle this?
I need to implement user faced scheduler, like users have reports and might choose schedule when they want those reports being sent to them.
Requirements are quote complex, like there should be schedules like each 12 hours, each 30 minutes, each second day, at Fridays at 1am, last Sunday of the months etc.
Is there Rails solution for that our should I create it from the ground?
Thanks!
Most schedulers for rails and ruby depend on a static file. You can use a queuing system like Delayed Job and make every job enqueue itself for next time after success. Or you can do a basic SheduledJob model which relates to the user, and stores the periodicity, next execution and last execution. And use a normal (frequent) scheduled task engine like clockwork to check for pending jobs.
I have an application where I want to automatically deactivate a user 72 hours after they have been activated. I have set this up with Delayed Job, but am now wondering if that is the best option.
My question is, if I set a task for 72 hours in the future, will a worker be active for that entire 72 hours? (I'm concerned about this as Heroku charges by the hour)
I'm open to suggestion here as far as better ways of doing this goes. One idea I had was to set this up using an exp_date column and check against that at sign in there by eliminating the need for DJ completely.
My question is, if I set a task for 72 hours in the future, will a worker be active for that entire 72 hours? (I'm concerned about this as Heroku charges by the hour)
Yes, it will be up all time. Delayed job continuously pings the database to see if there any job in its queue.
And, regarding the best option i think i rather put one column knows as valid_upto and put the date till will be active. I only signins (or whatever) to only those user which has created_at dates less then or equal to valid_upto date. And, periodically may be once in month i will run one cron job to remove invalid users.
And, like #leesungchul suggested, you can use that, that looks cool.
You can use the workless gem which is an addon for delayed jobs so you don't leave your worker running constantly on heroku.
https://github.com/lostboy/workless
I need to build a background job that goes through a list of RSS feeds and analyze them say every 10 minutes.
I have been using delayed_job for handling background jobs and I liked it a lot. I believe though that it's not built for recurring background jobs. I guess I can auto-schedule background job at the end of everyone (maybe with begin..rescue just to ensure it gets executes). Or preschedule say a month of advance worth of jobs and have another one that reschedule the every month..etc
This raised some concerned to me as I started asking myself: what if the server goes down in the middle of execution and the jobs didn't get scheduled?
I have also looked at Daemons gems which seemed the like it runs simple Ruby scripts with start/stop commands. I like the way delayed_job schedules and handles retries.
What do you recommend using in this case? What do you think the best way to design such a system with recurring background jobs? Also do you know a way I can monitor that background process and get notified if it stops?
I just implemented delayed_job for a similar task (using :run_at => 2.days.from_now) and found it to be a perfect fit. The easiest way to handle your concern about a process failing is to make the first step of the job to create the next job. Also, you can create a has_many relationship to the delayed_job model which would allow you to access the :last_error. Or, look at the "Hooks" section of readme and it has a perfect example for failure.
I think that this was a similar question: A cron job for rails: best practices? - not only are there answers, but also links to railscasts about background jobs in rails.
I used cron + delayed_job, but scheduled tasks were supposed to run few times a day, mostly just once.
Take a look at SimpleWorker. It's an elastic scheduling and background processing worker queue. It's cloud based and has persistence and redundancy so you don't need to worry if your servers go down or are restarted.
Very flexible in terms of scheduling, provides great introspection of jobs in the queue as well as notifications on status and errors.
Full disclosure: I work at SimpleWorker.
I have a ruby on rails app that uses Heroku. I have the need to run things like import/export tasks on our db that lock up the whole system since they are so heavy on the DB. Is there a way to tell the system to only run these tasks when the database is not being used at that second?
There is no built-in way to schedule a job like this. There are a few things you can do, though.
Schedule the jobs to run during the least busy hours of the day. That will depend on your business, customer base and so on, but hopefully there is a window that is more suitable than others.
You could write your batch job to run for a longer time, doing small units of work. Between each unit of work, sleep for a few seconds, or take a look at the current load average and decide what to do based on that. This should lower the impact of the batch jobs.
Have the website update a "lock" somewhere, either in the database or in a memcached or something. If your normal website usage updates the database, you could look at the existing updated_at. Then only do batch work when there hasn't been any activity for a while. This doesn't guarantee that a new user won't pop in at the same time your batch job runs, of course, but could be a way to find a window where the site is less used.
Have you looked into using Background Jobs / Workers on Heroku? It's also worth reading about Heroku's Delayed Job queuing system