I have rails app deployed over multiple instances and had too many rake tasks to run over different instances so it is hard to manage which rake tasks is already run or which one remaining.
is there any way to manage it from db side, as schema_migrations table managed by migrations. if yes then, i want know how migrations exactly works?.
any suggestions?.
Correct way: use deploy automation. Capistrano is a good choice. Then you'll never need to worry about things like running rake task
I think the rake tasks should have no side effects if you execute it multiple times. If the task is implemented that way, then there's no need to worry about which has been done and which is not.
I think if you want to get a status tracking for the Rake Task, a simple way is to implemented a model to record the execution status of the rake task, and update the model each time rake task is done.
You can use resque-scheduler(https://github.com/resque/resque-scheduler) to manage and track your tasks .
You can use Progress Bar gem to monitor the progress of a particular rake task.
And according to the above suggestion, automated deployment through capistrano is a good option. You can manage the rake tasks running sequence in the cap script.
Related
I have a rails module that processes some active record objects, only about 15-20 at a time, that I need to start off every two minutes.
I have tried to offload it to sidekiq (and sidekiq-cron), which works, but with the concurrency, created many race conditions and duplicate data.
I really just need a simple rake task cron for rails or maybe sinatra (as I would create a new sinatra app just to complete these tasks)
I either need to force sidekiq to process in a single thread or
have a "cron" job run a rake task or even the module directly
def self.process_events
events = StripeEvent.where(processed: false)
events = StripeServices.arrange_processing_order events
events.each do |event_obj|
StripeServices.new(event_obj).process_event_obj
end
end
thanks for any point in the right direction.
edited
sorry I wasnt very clear. pushing my module to sidekiq caused concurrency issues that I wasnt ready for (my bit of code is not threadsafe), and with the restrictions that Heroku places on "crons", whats the best way to run a rake task every 2 min?
If Sinatra can do it, I would prefer it, but I cant find the solution for that same problem.
It's not clear what are you asking. You already tried option 1, you can try option 2 (create the task and cron it, it's pritty easy) and you'll know better than anyone if it's better.
Anyway, I guess that both methods will have concurrency problems if one task takes more than 2 minutes.
You can add extra flags to prevent two task to process the same ServiceEvent (maybe add a boolean "processing" and set it to true when a task takes it).
Or maybe you can have a lock file to prevent a task to run if another one is already running (you create a file with a specific location and name when the task starts and delete it when it finishes processing, you can check if the file exists before starting a new task).
Is there any way to find out whether particular rake task is currently running from Rails controller for example? I have an extensive rake task, which takes 5-6 hours to finish.
I need to see status of that rake task from frontend web interface, like:
Task "some operation" is running...
Also it would be nice to be able to hard stop / run that rake task from within frontend web interface.
If found Railscast devoted to it, but the method described there allows only to run rake task from controller, but not to stop/see its status.
If your job is taking 5-6 hours to complete, you must run it in background. To run long running jobs in background you can use one of these:
Resque
Sidekiq
And for tracking status of your jobs, you can use corresponding status tracking gems:
ResqueStatus
SidekiqStatus
Personally, I prefer sidekiq for its efficiency.
NOTE: All the above mentioned gems has good enough docs. Kindly refer to those.
I have this code in my rake task. It seems overkill, since the rake task is already being run as a cron job. I think I can safely take it out of Resque and run it directly, but not sure if I missed something.
desc "update daily sales"
task :daily_sales => :environment do
Resque.enqueue(DailySaleService.perform)
end
Yes, it's overkill. There is no reason to use background processing for a rake task; you use background processing to remove heavy lifting from the HTTP request/response cycle to provide users with a better front-end experience. It won't provide any value in a rake task.
I want to know if there is a way in RoR to run a rake task, or a ruby code, twice in different times. For example, when a user registers, run the task after three days, and then run the same task one week later, but no more('stop the process'). I was looking at crontab and gems like Resque, Stalker, Starling, etc. but I don't have a clear idea who this gems can help me. I'm thinking in run a daemon for each user and count the task executions.The problem is that the daemon would be active all that time "eating" resources. Is there a way to use as least resources as possible?. I want to run this in Heroku later.
I usually solve this type of thing with date columns on the record (eg. first_reminder_sent_at, user_responed_at, week_reminder_sent_at). In this case, a date for the first item, the user's response, and for the week-later item.
Create a cron task to call a rake task - look at all users that are > 3 days old, < 1 week, user hasn't responded, date not set.
Queue up the Background Job.
The Job will send the mail (or whatever the task is) and then set the date field on the record.
Cron task to call a rake task - looks at all users that are > 1 week old where user hasn't responded and date not set.
Send the reminder and set the date in a Background Job.
Try the whenever cron and create a rake task, which is doing your stuff.. and in the database make some trigger fields when and how the rake task should do his actions..
Posix systems have the at command which is a dead simple way to schedule a job to run once at a later time. Though there's nothing wrong with using an application based queueing mechanism to schedule later work as outlined in the other answers. at is host specific so probably unsuitable if you're running a cluster, especially if the hosts may be transient.
We're hosting our Ruby on Rails application with the EngineYard App Cloud, which has worked really well for us. They provide a series of deploy call backs (before_restart, after_restart, etc.) which are analogous to Capistrano's callbacks.
We have a series of Rake tasks within our application which maintain various parts of the application. If we add a new business rule to the database, we might need to reload the users' associated business rules, etc.
These are tasks that there's no hard or fast schedule for, but also we don't want to run each and every task on every deploy, because they slow down the deploy process.
Are there any systems which would allow us to define a task to be run on the next deploy, sort of like migrations. The ideal system in my mind would work as follows:
We realize that on the next deploy, a task will need to be run
We schedule the task via the system
On the next deploy, the system see the list of post-deploy tasks -- it notices that the most recent one has not been run on the specific server yet (as in how migrations notate the database when they're run so that only the most recent, unrun migrations are triggered) -- the new task is triggered
Any recommendations on best practices for scheduling these post-deploy tasks and have them fire off unless they've already been run on the server?
Thanks!
Try the after_party ruby gem which is modelled on the basic operation of db:migrate but is for post deployment tasks. Post deployment (rake) tasks are created with a name like so
lib/tasks/deployment/20130130215258_task_name.rake
You can of course call any ruby code from within the rake task. The documentation says it supports sync and async tasks (async tasks are long running tasks that you can have going on in the background while your app is starting up)
I've not used it but am about to give it a shot as we have similar requirements as you've described.
Two approaches come to my mind
Quick/dirty solution...could you just use migrations to do this? Create a Rails migration that fires off the tasks when rake db:migrate is run
Take the same approach as migrations. Create a peer table to the schema_migrations table, and then in your before_symlink.rb (or whereever else) run the tasks that have not been executed yet, and then update the table?
You should give rails_tasker a shot. It provides a straight-forward way to automating your post-deploy tasks. Here is an article that describes how to use the gem.