I need to perform a task every 5 seconds, but only when users are using the application.
As for now, I use cron that works every minute and activates a task that repeats itself every 5 seconds with sleeps between, for a minute. However, it works also when the application isn't being used.
Is there a gem that will do this kind of thing?
If you're using cron, couldn't you just use things particular to the OS to get it to work? As in, use the underlying filesystem or OS, don't rely on Rails.
For example, your Rails app could touch a file whenever a user hits the site at all, which the cron job would check for existing before running its task. It could also check the ctime/mtime of the file and delete it if the file is too long out of date.
In that case, you wouldn't need a gem... just make your cron task smarter.
Related
I'm trying to short poll an external API which doesn't support websockets, so I need to constantly make API requests every few seconds or maybe multiple times per second.
I also need to be in control of calling the next poll run. For example I wouldn’t want to run the poll again while my request is pending or I might want to increase the timeout if I get a 500 error.
Currently, considering doing this in a separate node process and only notify the rails server when there's new data. But I'd rather just do everything in the rails codebase.
I don't think ActiveJobs is built for this purpose but I could be wrong. I think what I really need is a separate entry point in the rails app repository that loads all the models but doesn't start the server and then write the loop for short polling, But not sure if that's best practice or trivial to do with rails.
So should I proceed with the node approach or is there an easy Rails solution I'm missing? Any suggestion or guidance is appreciated.
Maybe you can try whenever.
It help you run a method or a rake task with crontab.
# schedule.rb
every 1.minute do
runner "YourClass.your_method"
end
every 1.minute do
rake "polling:task"
end
After finish schedule.rb file, you'll need to execute whenever --update-crontab in you deploy pipiline in order to update crontab.
I have a ruby on rails app in which I'm trying to find a way to run some code every few seconds.
I've found lots of info and ideas using cron, or cron-like implementations, but these are only accurate down to the minute, and/or require external tools. I want to kick the task off every 15 seconds or so, and I want it to be entirely self contained within the application (if the app stops, the tasks stop, and no external setup).
This is being used for background generation of cache data. Every few seconds, the task will assemble some data, and then store it in a cache which gets used by all the client requests. The task is pretty slow, so it needs to run in the background and not block client requests.
I'm fairly new to ruby, but have a strong perl background, and the way I'd solve this there would be to create an interval timer & handler which forks, runs the code, and then exits when done.
It might be even nicer to just simulate a client request and have the rails controller fork itself. This way I could kick off the task by hitting the URI for it (though since the task will be running every few seconds, I doubt I'll ever need to, but might have future use). Though it would be trivial to just have the controller call whatever method is being called by the periodic task scheduler (once I have one).
I'd suggest the whenever gem https://github.com/javan/whenever
It allows you to specify a schedule like:
every 15.minutes do
MyClass.do_stuff
end
There's no scheduling cron jobs or monkeying with external services.
Generally speaking, there's no built in way that I know of to create a periodic task within the application. Rails is built on Rack and it expects to receive http requests, do something, and then return. So you just have to manage the periodic task externally yourself.
I think given the frequency that you need to run the task, a decent solution could be just to write yourself a simple rake task that loops forever, and to kick it off at the same time that you start your application, using something like Foreman. Foreman is often used like this to manage starting up/shutting down background workers along with their apps. On production, you may want to use something else to manage the processes, like Monit.
You can either write you own method, something like
class MyWorker
def self.work
#do you work
sleep 15
end
end
run it with rails runner MyWorker.work
There will be a separate process running in the background
Or you can use something like Resque, but that's a different approach. It works like that: something adds a task to the queue, meanwhile a worker is fetching whatever job it is in the queue, and tries to finish it.
So that depends on your own need.
I know it is an old question. But maybe for someone this answer could be helpful. There is a gem called crono.
Crono is a time-based background job scheduler daemon (just like Cron) for Ruby on Rails.
Crono is pure Ruby. It doesn't use Unix Cron and other platform-dependent things. So you can use it on all platforms supported by Ruby. It persists job states to your database using Active Record. You have full control of jobs performing process. It's Ruby, so you can understand and modify it to fit your needs.
The awesome thing with crono is that its code is self explained. In order to do a task periodically you can just do:
Crono.perform(YourJob).every 2.days
Maybe you can also do:
Crono.perform(YourJob).every 30.seconds
Anyway you really can do a lot of things. Another example could be:
Crono.perform(TestJob).every 1.week, on: :monday, at: "15:30"
I suggest this gem instead of whenever because whenever uses Unix Cron table which not always is available.
Throwing out a solution just because it looks somewhat elegant and answers the question without any extra gems. In my scenario I wanted to run some code, but only after all my Sidekiq workers were done doing their thing.
First I defined a method to check if any workers were working...
def workers_working?
workers = Sidekiq::Workers.new.map do |_process_id, _thread_id, work|
work
end
workers.size > 0
end
Then we just call the method with a loop which sleeps between calls.
sleep 5 while workers_working?
Use something like delayed job, and requeue it every so often?
Use thin or other server which uses eventmachine, then just use timers that are part of eventmachine. Example: in config/application.rb
EM.add_periodic_timer(2) do
do_this_every_2_sec
end
If I need to create recurring tasks in delayed jobs, what is a clean solution? I have an import task that I want to run every 5 minutes, but I don't want to fire up rails/rake in order to tell it to create a Delayed job that can be picked up. If Rails is already running on a system, perhaps I can just create an HTTP request that will make the rails app fire off a DJ? I could put that cron task in a ruby script which runs every 5 minutes, making requests to a server, but without firing up rails. What do you think?
This fork of delayed_job has recurrence built right in (and there may be other forks that do the same):
http://github.com/andrewroth/delayed_job
The cron approach still seems to be quite clean. It need only involve running a rake invocation, not a full Rails server. Rake doesn't need Rails to be running to work.
However if you really don't like that approach then you could arrange for the recurring jobs to be re-queue themselves when they are being processed setting a run_at time to 5 minutes (or whatever) in the future. Obviously you'd need to prime the queue the first time and make sure the delayed_job server stays running
About that... I have the same desire,
I'm planning to have a cron to fire up a curl request at a specific route at my site every 5 minutes, so it runs a action with the result, and I'm gonna be pretty sure it only ran once.
My purposes includes: Awarding Badges and compiling some Averages
I have a ruby on rails app that uses Heroku. I have the need to run things like import/export tasks on our db that lock up the whole system since they are so heavy on the DB. Is there a way to tell the system to only run these tasks when the database is not being used at that second?
There is no built-in way to schedule a job like this. There are a few things you can do, though.
Schedule the jobs to run during the least busy hours of the day. That will depend on your business, customer base and so on, but hopefully there is a window that is more suitable than others.
You could write your batch job to run for a longer time, doing small units of work. Between each unit of work, sleep for a few seconds, or take a look at the current load average and decide what to do based on that. This should lower the impact of the batch jobs.
Have the website update a "lock" somewhere, either in the database or in a memcached or something. If your normal website usage updates the database, you could look at the existing updated_at. Then only do batch work when there hasn't been any activity for a while. This doesn't guarantee that a new user won't pop in at the same time your batch job runs, of course, but could be a way to find a window where the site is less used.
Have you looked into using Background Jobs / Workers on Heroku? It's also worth reading about Heroku's Delayed Job queuing system
I've got an app that could benefit from delayed_job and some background processing. The thing is, I don't really need/want delayed_job workers running all the time.
The app runs in a shared hosting environment and in multiple locations (for different users). Plus, the app doesn't get a large amount of usage.
Is there a way to start and stop processing jobs (either with the script or rake task) from my app only after certain actions/events?
You could call out to system:
system "cd #{Rails.root} && rake delayed_job:start RAILS_ENV=production"
You could just change delayed_job to check less often too. Instead of the 5 second default, set it to 15 minutes or something.
Yes, you can, but I'm not sure what the benefit will be. You say you don't want workers running all the time - what are your concerns? Memory usage? Database connections?
To keep the impact of delayed_job low on your system, I'd run only one worker, and configure it to sleep most of the time.
Delayed::Worker::sleep_delay = 60 * 5 # in your initializer.rb
A single worker will only wake up and check the db for new jobs every 5 minutes. Running this way keeps you from 'customizing' too much.
But if you really want to start a Delayed::Worker programatically, look in that class for work_off, and implement your own script/run_jobs_and_exit script. It should probably look much like script/delayed_job does - 3 lines.
I found this because I was looking for a way to run some background jobs without spending all the money to run them all the time when they weren't needed. Someone made a hack using google app engine to run the background jobs:
http://viatropos.com/blog/how-to-run-background-jobs-on-heroku-for-free/
It's a little outdated though. There is an interesting comment in the thread:
"When I need to send an e-mail, copy a file, etc I basically add it to the queue. At the end of every request it checks if there is anything in the queue. If so then it uses the Heroku API to set the worker to 1. At the end of a worker getting a task done it checks to see if there is anything left in the queue. If not then it sets the workers back to 0. The end result is the background worker will just work for a few seconds here and there. I can do all the background processing that I need and the bill at the end of the month rarely ever reaches 1 hour total worth of work. Even if it does no problem, I'll pay $0.05 for background processing. :)"
If you go to stop a worker, you are given the PID. You can simply kill -9 PID if all else fails.