My application (in Rails) should provide two main tasks:
create activity (for example post to Twitter) in given time in the future
periodically crawler some site or download Tweets
I'm thinking about using DelayedJob gem or Whenever gem for cron tasks. Which is better in these situations?
Thanks for any advice.
create activity (for example post to Twitter) in given time in the future
If this is in response to an event/user action, then a background task would be ideal, as it's not a regular schedule. Ruby toolbox seems to be favouring Resque and Sidekiq over delayed job though, so have a look at those before settling.
periodically crawler some site or download Tweets
Converse to the above, if this is a regularly scheduled event then cron jobs are ideal. Nothing wrong with using whenever for this, but make sure you have some form of monitoring on the jobs to alert you if something goes wrong.
For creating the activity, I would definitely use a background job to do that (i.e. PostToTwitterJob). I have a preference for Sidekiq instead DelayedJob, as delayed_job does not seem to be active anymore. If you want the job to be performed at a specific time in the future, check out sidekiq's documentation.
As for the crawling, I would use both. Your crawler would be a background job (i.e. TweetsCrawlerJob), and you could launch it every hour with whenever.
I am using https://github.com/ssoroka/scheduler_daemon for my scheduled jobs, but I would like jobs immediately with a command (rather than waiting for the delay specified in the task).
I've tried using rails runner TaskName.run but the class can never be found (runner.rb:53:in 'eval': uninitialized constant TaskName (NameError)).
How can I run the scheduled tasks immediately?
If I guess correctly, you want to call one of the scheduled task directly.
If I were you, I'd ask to the author directly, via the channel he points at in the readme: https://github.com/ssoroka/scheduler_daemon/issues
If there is a way to do it, I'm sure the author will be glad to explain it in the readme as well.
Corollary question: for a one-time schedule, do you want your direct trigger to cancel the schedule?
You don't need the scheduler to run the task for you, just call the MyTask.new.run directly.
I have a ruby on rails app in which I'm trying to find a way to run some code every few seconds.
I've found lots of info and ideas using cron, or cron-like implementations, but these are only accurate down to the minute, and/or require external tools. I want to kick the task off every 15 seconds or so, and I want it to be entirely self contained within the application (if the app stops, the tasks stop, and no external setup).
This is being used for background generation of cache data. Every few seconds, the task will assemble some data, and then store it in a cache which gets used by all the client requests. The task is pretty slow, so it needs to run in the background and not block client requests.
I'm fairly new to ruby, but have a strong perl background, and the way I'd solve this there would be to create an interval timer & handler which forks, runs the code, and then exits when done.
It might be even nicer to just simulate a client request and have the rails controller fork itself. This way I could kick off the task by hitting the URI for it (though since the task will be running every few seconds, I doubt I'll ever need to, but might have future use). Though it would be trivial to just have the controller call whatever method is being called by the periodic task scheduler (once I have one).
I'd suggest the whenever gem https://github.com/javan/whenever
It allows you to specify a schedule like:
every 15.minutes do
MyClass.do_stuff
end
There's no scheduling cron jobs or monkeying with external services.
Generally speaking, there's no built in way that I know of to create a periodic task within the application. Rails is built on Rack and it expects to receive http requests, do something, and then return. So you just have to manage the periodic task externally yourself.
I think given the frequency that you need to run the task, a decent solution could be just to write yourself a simple rake task that loops forever, and to kick it off at the same time that you start your application, using something like Foreman. Foreman is often used like this to manage starting up/shutting down background workers along with their apps. On production, you may want to use something else to manage the processes, like Monit.
You can either write you own method, something like
class MyWorker
def self.work
#do you work
sleep 15
end
end
run it with rails runner MyWorker.work
There will be a separate process running in the background
Or you can use something like Resque, but that's a different approach. It works like that: something adds a task to the queue, meanwhile a worker is fetching whatever job it is in the queue, and tries to finish it.
So that depends on your own need.
I know it is an old question. But maybe for someone this answer could be helpful. There is a gem called crono.
Crono is a time-based background job scheduler daemon (just like Cron) for Ruby on Rails.
Crono is pure Ruby. It doesn't use Unix Cron and other platform-dependent things. So you can use it on all platforms supported by Ruby. It persists job states to your database using Active Record. You have full control of jobs performing process. It's Ruby, so you can understand and modify it to fit your needs.
The awesome thing with crono is that its code is self explained. In order to do a task periodically you can just do:
Crono.perform(YourJob).every 2.days
Maybe you can also do:
Crono.perform(YourJob).every 30.seconds
Anyway you really can do a lot of things. Another example could be:
Crono.perform(TestJob).every 1.week, on: :monday, at: "15:30"
I suggest this gem instead of whenever because whenever uses Unix Cron table which not always is available.
Throwing out a solution just because it looks somewhat elegant and answers the question without any extra gems. In my scenario I wanted to run some code, but only after all my Sidekiq workers were done doing their thing.
First I defined a method to check if any workers were working...
def workers_working?
workers = Sidekiq::Workers.new.map do |_process_id, _thread_id, work|
work
end
workers.size > 0
end
Then we just call the method with a loop which sleeps between calls.
sleep 5 while workers_working?
Use something like delayed job, and requeue it every so often?
Use thin or other server which uses eventmachine, then just use timers that are part of eventmachine. Example: in config/application.rb
EM.add_periodic_timer(2) do
do_this_every_2_sec
end
I have some rails code that calls model1.func1(). A controller action calls this, where multiple people can be hitting it, as does a scheduled rake task. I want to make sure that model1.func1() cannot be called in parallel. If another thread needs to call at the same time, it should wait for model1.func() to finish. I guess I want to queue these calls. I was going to use sidekiq for this, but with only one worker. I read on a forum that
Sidekiq is not appropriate for the serial job and I don't want to make
it appropriate. Different tools are useful for different reasons,
jack of all trades master of none, etc.
What do you guys recommend instead?
I would consider beanstalkd with one worker process.
I have the scheduled job which run every 5 seconds. I'm now using backgroudRB for scheduling. But sometimes, the job may take more than 5 seconds and i don't want more than one job is running at the same time. In .Net, we can use Mutex class, but I'm not sure about what should I use in rails application.
Thanks.
Yuck, I remember using backgrounDRb, it was horrible. I use Resque now, after using delayed_job. Both work well, and you can solve your problem by only running a single worker. You can find both on Github.