If I need to create recurring tasks in delayed jobs, what is a clean solution? I have an import task that I want to run every 5 minutes, but I don't want to fire up rails/rake in order to tell it to create a Delayed job that can be picked up. If Rails is already running on a system, perhaps I can just create an HTTP request that will make the rails app fire off a DJ? I could put that cron task in a ruby script which runs every 5 minutes, making requests to a server, but without firing up rails. What do you think?
This fork of delayed_job has recurrence built right in (and there may be other forks that do the same):
http://github.com/andrewroth/delayed_job
The cron approach still seems to be quite clean. It need only involve running a rake invocation, not a full Rails server. Rake doesn't need Rails to be running to work.
However if you really don't like that approach then you could arrange for the recurring jobs to be re-queue themselves when they are being processed setting a run_at time to 5 minutes (or whatever) in the future. Obviously you'd need to prime the queue the first time and make sure the delayed_job server stays running
About that... I have the same desire,
I'm planning to have a cron to fire up a curl request at a specific route at my site every 5 minutes, so it runs a action with the result, and I'm gonna be pretty sure it only ran once.
My purposes includes: Awarding Badges and compiling some Averages
Related
I'm trying to short poll an external API which doesn't support websockets, so I need to constantly make API requests every few seconds or maybe multiple times per second.
I also need to be in control of calling the next poll run. For example I wouldn’t want to run the poll again while my request is pending or I might want to increase the timeout if I get a 500 error.
Currently, considering doing this in a separate node process and only notify the rails server when there's new data. But I'd rather just do everything in the rails codebase.
I don't think ActiveJobs is built for this purpose but I could be wrong. I think what I really need is a separate entry point in the rails app repository that loads all the models but doesn't start the server and then write the loop for short polling, But not sure if that's best practice or trivial to do with rails.
So should I proceed with the node approach or is there an easy Rails solution I'm missing? Any suggestion or guidance is appreciated.
Maybe you can try whenever.
It help you run a method or a rake task with crontab.
# schedule.rb
every 1.minute do
runner "YourClass.your_method"
end
every 1.minute do
rake "polling:task"
end
After finish schedule.rb file, you'll need to execute whenever --update-crontab in you deploy pipiline in order to update crontab.
I am trying to find out the best way to run scripts in the background. I have been looking around and found plenty of options, but many/most seem to have become inactive in the past few years. Let me describe my needs.
The rails app is basically a front-end to configure when and how these scripts will be run. The scripts run and generate reports and send email alerts. So the user must be able to configure the start times and how often these scripts will run dynamically. The scripts themselves should have access to the rails environment in order to save the resulting reports in the DB.
Just trying to figure out the best method from the myriad of options.
I think you're looking for a background job queuing system.
For that, you're either looking for resque or delayed_job. Both support scheduling tasks at some point in the future -- delayed_job does this natively, whereas resque has a plugin for it called resque_scheduler.
You would enqueue jobs in the background with parameters that you specify, and then at the time you selected they'll be executed. You can set jobs to recur indefinitely or a fixed number of times (at least with resque-scheduler, not sure about delayed_job).
delayed_job is easier to set up since it saves everything in the database. resque is more robust but requires you to have redis in your stack -- but if you do already it's pretty much the ideal solution for your problem.
I recently learned about Sidekiq, and I think it is really great.
There's also a RailsCast about it - Sidekiq.
Take a look at the gem whenever at https://github.com/javan/whenever.
It allows you to schedule tasks like cron jobs.
Works very well under linux, and the last commit was 14 days ago. A friend of mine used it in a project and was pretty satisfied with it.
edit: take a look at the gem delayed_job as well, it is good for executing long tasks in the background. Useful when creating a cron job only to start other tasks.
With a Rails stack, how can I create a background process that handles events by spawning threads that are worked in real time?
The workers on Heroku pick up jobs every 5 seconds. I need real time. Ideally I'd like to get this working on Heroku, but if I need to, I will move away from it.
This has a long list of background workers: Background Job Manager for Rails 3 but it is not clear if your question heroku specific or not
I think you are looking for something like "run_later" which instead of queueing a job actually returns the request and runs a block in a separate process.
Here is a link to the Rails 3+ version, you can follow the fork network to find many other implementations:
https://github.com/Zelnox/run_later
(I don't use Heroku so I don't know if it runs on it)
Heroku runs rake jobs:work, so you can replace that with your own rake task, either running delayed_job with a shorter than 5 second timeout, or just performing your own task. Probably a good idea to keep a sleep statement in there.
The new cedar stack will run anything you want, so it might be worth checking that out.
With regards to run_later, or spawning from the current request, this does work but if the background process doesn't complete within the 30 second request timeout then heroku will kill it.
I think you need delay_job. please checkout this gem
I have the scheduled job which run every 5 seconds. I'm now using backgroudRB for scheduling. But sometimes, the job may take more than 5 seconds and i don't want more than one job is running at the same time. In .Net, we can use Mutex class, but I'm not sure about what should I use in rails application.
Thanks.
Yuck, I remember using backgrounDRb, it was horrible. I use Resque now, after using delayed_job. Both work well, and you can solve your problem by only running a single worker. You can find both on Github.
I need to perform a task every 5 seconds, but only when users are using the application.
As for now, I use cron that works every minute and activates a task that repeats itself every 5 seconds with sleeps between, for a minute. However, it works also when the application isn't being used.
Is there a gem that will do this kind of thing?
If you're using cron, couldn't you just use things particular to the OS to get it to work? As in, use the underlying filesystem or OS, don't rely on Rails.
For example, your Rails app could touch a file whenever a user hits the site at all, which the cron job would check for existing before running its task. It could also check the ctime/mtime of the file and delete it if the file is too long out of date.
In that case, you wouldn't need a gem... just make your cron task smarter.