I have a page of a long list of items. Each has a check box next to it. There's a jQuery check-all function, but when I submit all of them at once, the request times out because it's doing a bunch of queries and inserting a bunch of records in the MySQL database for each item. If it were to not timeout, it'd probably take about 20 minutes. Instead, I just submit like 30 at a time.
I want to be able to just check all and submit and then just go on doing other work. My coworker (1) said I should just write a rake task. I did that, but I ended up duplicating code, and I prefer the user interface because what if I want to un-check a few? The rake task just submits them all.
Another coworker (2) recommended I use fork. He said that would spawn a new process that would run on the server but allow the server to respond before it's done. Then, since an item disappears after it's been submitted, I could just refresh the page to check if they're done.
I tried this on my local, however, it still seems that Rails is waiting for the process to finish before it responds to the POST request sent by the HTML form. The code I used looks like this:
def bulk_apply
pid = fork do
params[:ids].each do |id|
Item.find(id).apply # takes a LONG time, esp. x 100
end
end
Process.detach(pid) # reap child process automatically; don't leave running
flash[:notice] = "Applying... Please wait... Then, refresh page. Only submit once. PID: #{pid}"
redirect_to :back
end
Coworker 1 said that generally you don't want to fork Rails because fork creates a child process that is basically a copy of the Rails process. He said if you want to do it through the web GUI, use BackgroundJob (Bj) (because we're already using that in our Rails app). So, I'm looking into BackgroundJob now, but what do you recommend?
I've had good success using background job. If you need rails you will be using script/runner which still starts up a new process with rails. The good thing is that Backround Job will make sure that there is never more than one running at a time.
You can also use script runner directly, or even run a rake task in the background like so:
system " RAILS_ENV=#{RAILS_ENV} ruby #{RAILS_ROOT}/script/runner 'CompositeGrid.calculate_values(#{self.id})' & " unless RAILS_ENV == "test"
The ampersand tells it to start a new process. Be careful because you probably don't want a bunch of these running at the same time. I would definitely take advantage of background job if it is already available.
you should check out IronWorker . It would be super easy to do what you want and it doesn't matter how long it takes.
In your action you'd just instantiate a worker which has the code that's doing all your database queries. Example worker:
Item.find(id).apply # takes a LONG time, esp. x 100
And here's how you'd queue up those jobs to run in parallel:
client = IronWorkerNG::Client.new
ids.each do |id|
client.tasks.create("MyWorker", "id"=>id)
end
That's all you'd need to do and IronWorker takes care of the rest.
Try delayed_job gem. This is a database-based background job gem. We used it in an e-commerce website. For example, sending order confirmation email to the user is an ideal candidate for delayed job.
Additionally you can try multi-threading, which is supported by Ruby. This could make things run faster. Forking an entire process tends to be expensive due to memory usage.
Related
I have the following scenario:
1. Someone creates a task in Redmine
2. Certain things are done (or not done) with this task
3. In case 3 days have passed since task creation and it is not in the right state, send an email to interested persons.
My question is how do I schedule an email (or the task that will check if an email is required) in exactly three days with Rails? The only option I could think of so far is to create some rake task and run it every couple of minutes. Are there better options?
You've got a couple of options. You probably want to look at Active Job (Rails > 4.2). This lets you schedule a job to run after a specified period:
MyJob.set(wait_until: Time.now + 3.days).perform_later(record)
If you're on a version of rails prior to 4.2 then DelayedJob or Sidekiq or Resque could be used. Active Job is essentially a layer over the top of something like this anyway so a later migration to that shouldn't be too painful.
Alternatively if you don't need to check after exactly 3 days then you could sweep for tasks that need to have emails generated using cron (whenever is a good wrapper for this). You can sweep as often as you want, although every few minutes is probably excessive, and it means you won't have to set up a queueing back end on your server.
It does mean that you'll have to find the tasks that need emails generated for them whereas with the queuing system you'll know exactly which task you're dealing with already. However, it seems like plenty of those tasks won't need an email anyway so it might be more efficient to actively look for the ones that do.
As an alternative you can use https://github.com/resque/resque and https://github.com/zapnap/resque_mailer
Hope this helps.
You can use Whenever to schedule jobs/tasks/method calls ..almost anything
JUST add the gem ...run bundle install..add your code ....update the schedule.rb
for example:
##in schedule.rb
every 5.minutes do
p "============Running rake task Only"
rake "events:notify_user"
end
every 2.hours do
p "============Running rake task and local script as well as calling a method from your code"
command "/usr/bin/some_great_command"
runner "MyModel.some_method"
rake "some:great:rake:task"
end
You can also watch the Railscasts
I am working for a project with ruby on rails. There's one page with a list of items you can choose multiple of them and change their information at a time. But when the data are too huge and you choose too many items to get in process at a time. It will run for a few minutes long, then the page will show error message and it is caused by the server timeout...How can I let it run even for ten minutes but not timeout and return an error message?
There is not one way to do this.
Firstly, you can look in your log to check for long sql queries, if you see some very long ones, you can try and optimize them.
You can also use gems like rails-footnotes to see the sql queries in the bottom of each page in dev environment.
If you are doing some expensive tasks, try to process them in the background with tools such as resque or delayed_jobs.
Best user-friendly solution is to use some background tasks...
Like delayed_job, for example :
def my_action
do_some_stuff
end
handle_asynchronously :my_action, :priority => 1
Then, your action will be done asynchronously and your server will continue to work normally.
I have a rails app deployed on Heroku. I want to add a feature that enables users of the app to set a reminder. I need some way for the app to schedule sending an email at the time specified by the user.
I have found numerous posts referring to using delayed_job for this, but none of the write-ups / tutorials / etc. that I have found directly address what I am trying to accomplish (the descriptions I have found seem more geared towards managing long-running jobs that are to be run "whenever").
Am I on the right track looking at delayed_job for this? If so, can somebody point me towards a tutorial that might help me?
If delayed_job is not quite right for the job, does anybody have a suggestion for how I might approach this?
The most typical way of handling this is to use a cron job. You schedule a job to run every 15 minutes or so and deliver any reminders that come up in that time. Unfortunately, heroku only allows cron jobs to run every hour, which usually isn't often enough.
In this case, I'd use delayedjob and trick it into setting up a recurring task that delivers the notifications as often as necessary. For example, you could create a function that begins by rescheduling itself to run in 10 minutes and then goes on to send any reminders that popped up in the previous 10 minutes.
To view delayedjobs send_at syntax to schedule future jobs check here: https://github.com/tobi/delayed_job/wiki
ADDED after comments:
To send the reminder, you would need to create a function that searches for pending reminders and sends them. For example, let's say you have a model called Reminder (rails 3 syntax cause I like it better):
def self.find_and_send_reminders
reminders = Reminder.where("send_at < ? AND sent = ?", Time.now, false).all
reminders.each do |r|
#the following delayed_job syntax is apparently new, and I haven't tried it. came from the collective_idea fork of delayed_job on github
Notifier.delay.deliver_reminder_email(r)
#I'm not checking to make sure that anything actually sent successfully here, just assuming they did. may want to address this better in your real app
r.update_attributes!(:sent => true)
end
#again using the new syntax, untested. heroku may require the old "send_at" and "send_later" syntax
Reminder.delay(:run_at => 15.minutes.from_now).find_and_send_reminders
end
This syntax assumes you decided to use the single reminder entry for every occurence method. If you decide to use a single entry for all recurring reminders, you could create a field like "last_sent" instead of a boolean for "sent" and use that. Keep in mind these are all just ideas, I haven't actually taken the time to implement anything like this yet so I probably haven't considered all the options/problems.
Check out the runt gem, may be useful for you: http://runt.rubyforge.org/
You can use delayed_job's run_at to schedule at a specific time instead of whenever.
If your application allows the users to change the time of the reminders you need to remember the delayed_job to be able to update it or delete it when required.
Here is more details.
It's good to avoid polling if you can. The worker thread will poll at the database level, you don't want to add polling on top of polling.
A certain function in my controller takes a lot of time to process (heavy db work) . So when my user clicks on "submit" on the form he has to wait for the process to complete which is quite long. Is there any way that on "submitting", the user is redirected to the next view without any delay while the processing continues in the back-end without making the user wait ?
Thanks & Cheers !
When the user's request is made, queue up the job and then redirect the request where you want it.
There are two popular Ruby Gems for job processing:
Delayed Job
Resque
Delayed job is probably the easier to setup since it does not require Redis.
For things like this, I usually dump things into a database queue, and then use a cronjob to actually run it.
For instance, say I had to send out an email to all the clients using the software. I'd put the message into a database table, along with some information about who should get it, and then a cron job would actually do the sending.
It sounds to me that you need to fork the process that takes so long.
For example:
fork { "this code is being ran in background" }
The problem is that this code won't work nice with sql since the connection is not persistent. To handle this problem I've been using the spawn plugin for a while with excelent results.
How would I go about sending an email to a user, say, 48 hours after they sign up, in Ruby on Rails? Thanks!
As Joseph Daigle mentioned, you need to obviously record the exact date and time the user registered. After that, you need a cron running every certain number of minutes (every hour, for example) checking to see if there's any new users whose registration time is greater than 48 hours, send a mail to said user and mark that user as already emailed, so you don't email them again.
As per the actual mail sending, check out the following documentation page:
http://wiki.rubyonrails.org/rails/pages/HowToSendEmailsWithActionMailer
It has all you need to know to send mails with RoR.
I recommend that you use the latest version of BackgrounDRb to handle this. You can read about BackgrounDRb here: http://backgroundrb.rubyforge.org/
In order to queue a message for later delivery, the BackgrounDRb client code (in your application model's after_create callback, maybe) could look something like this:
MiddleMan(:email_worker).enq_send_email_task(:message => #message,
:job_key => "notify1",
:scheduled_at => Time.now + 48.hours)
You'd have to build a BackgrounDRb worker to handle sending the email:
# RAILS_ROOT/lib/workers/email_worker.rb
class EmailWorker < BackgrounDRb::MetaWorker
set_worker_name :email_worker
def send_email_task(message)
# ... Code to send the email message
end
end
Note that in order to use BackgrounDRb in this way, you have to use persistent job queues, so make sure you run the migration included with BackgrounDRb to set up the persistence table in your application.
BackgrounDRb is started separately from Rails (mongrel, apache, etc) using 'script/backgroundrb start', so make sure that you add the daemon to whatever process monitoring you're using (god, monit, etc) or that you create an /etc/init.d script for it.
First you're going to need a running daemon or background service which can poll your queue (probably from in a database) every few minutes.
The algorithm is pretty simple. Record the time of the user event in the queue. When the daemon checks that item in the queue, and the time difference is greater than 48 hours, prepare the e-mail to send.
You can queue jobs with a delay using async observer. Ideally, anything you have that isn't known to be instant (or very close to it) all the time should pass through something like that.
I wrote a plugin called acts_as_scheduled that may help you out.
acts_as_scheduled allows you to manage
scheduled events for your models.
A good example of this is scheduling
the update of RSS Feeds in a
background process using Cron or
BackgroundRB.
With acts_as_scheduled your schedule
manager can simply call
"Model.find_next_scheduled()" to grab
the next item from the database.
How I would approach this is by creating a scheduling controller, that will query the database for the next_scheduled and then use a mailer to send the message. The you set up a Cron Job to call the controller periodically using WGET or CURL. The advantage of the Cron/Controller approach is that no further infrastructure or configuration is required on the server and you avoid complicated threading code.
I think I'd be inclined to store the need for the email and the earliest time after which it should be sent, somewhere separate, then have my things-to-do task look at that. That way I only have to process as many records as there are emails to be sent, rather than examine every user every time, which would either get tedious or require an otherwise probably unnecessary index. As a bonus, if I had other tasks to be performed on some sort of a diarised basis, the same construct would be useful with little modification.