After reading Running Rails jobs in background threads I tried to refactor an controller action that is basically rendering an report in a xml format and sends the report file to the user browser. Now the action runs c. 30 sec, so I thought this a good candidate for a background job. Hovewer, I have run into a problem: it seems I can't access the report erb template from within the new thread.
This is my first attempt with threads and I am not sure if I am able to accomplish my refactoring with the approach.
Simplified code of action before refactoring:
#orders = Order.onrange(range).includes(:logo => [:simulations, :pantones])
respond_to do |format|
format.xml {
orders_xml = render_to_string template: 'orders/bom'
send_data orders_xml, filename: "BOM raport - #{range.to_s}.xml"
}
end
I don't know if you already see this guide: http://guides.rubyonrails.org/active_job_basics.html if not, please take a closer look.
From what you describe you can achieve your goal with:
Make a first request, that will call a controller action to produce the report and use the callback 'after_perform' to update the view with the produced report.
Split the reports from "asking the report" and "view the report". The first will enqueue the report generation and once it's ready will be available for user in a reports section.
I would prefer the second option if you plan the report generation time will increase or if it's unacceptable to have the user waiting 30 secs for a report. Other option could be to generate the report after you have the needed data, so when the user ask for it it's already generated. (just guessing because I don't know your application flow)
Related
I have made a rails app. Users can upload images. Once the images are saved into database. An algorithm being called to process those pictures.
For now, it was realized within a controller action like this:
def create
#post = current_user.posts.build(params)
if #post.save
flash[:success]="post created"
redirect_to root_url
image_names = []
#post.picture.each do |imgs|
image_names << imgs.url
end
my_algorithm(image_names)
else
render 'static_pages/home'
end
end
It works correctly. The problem is the page didn't show until the algorithm finishing. And the algorithm took long time. How to fix it. Or maybe call my_algorithm other places? Or delay_job?
I think you should use Active Job for that it will make your job background job
You should try the background jobs to perform that action in particular time.
You may use sidekiq gem to trigger the events in background.
Or You can refer the following URL to, do the active jobs,
http://edgeguides.rubyonrails.org/active_job_basics.html#enqueue-the-job
Refer the above url and configure the app and perform the operations in background.
I have additionally mention the one more link for scheduled jobs,
How to perform a background job now?
First of all, this should be done in a call back, here in the after_create callback of the Post model. And from the callback you can en queue one delayed job or other background job which will fire the algorithm. In front end you can show some messages like under processing in the place where you want to show the processed information.
Thanks
I'm using the axlsx_rails Ruby gem in Rails 4.2.5 to generate an Excel file to let users download their data.
I have this in my index.xlsx.axlsx template:
wb = xlsx_package.workbook
wb.add_worksheet(name: 'Transactions') do |sheet|
sheet.add_row ["Date", "Vendor Name", "Account",
"Transaction Category",
"Amount Spent", "Description"]
#transactions.find_each(batch_size: 100) do |transaction|
sheet.add_row [transaction.transaction_date,
transaction.vendor_name,
transaction.account.account_name,
transaction.transaction_category.name,
transaction.amount,
transaction.description]
end
end
The page times out before returning an Excel file if there's enough data. Is there a way to use HTTP streaming to send results back as it's processing, rather than waiting until the entire transactions.find_each loop has completed?
I saw code here using response.stream.write:
response.headers['Content-Type'] = 'text/event-stream'
10.times {
response.stream.write "This is a test message"
sleep 1
}
response.stream.close
That approach looks promising, but I couldn't figure out how to integrate response.stream.write into an axlsx_rails template. Is there a way?
This is my first Stack Overflow question- apologies for any faux pas and thank you for any ideas you can offer.
Welcome to SO, Joe.
I asked in comment, but perhaps it's better to answer and explain.
The short answer, is yes, you can always stream if you can render (though with sometimes mixed performance results).
It does not, however, work if your referencing a file directly. IE, http://someurl.com/reports/mycustomreport.xlsx
Streaming in rails just isn't built that way by default. But not to worry, you "should" still be able to tackle your issue, providing the time you wish to save is rendering only.
In your controller (* note for future, when you're asking about rendering actions, it helps to provide your controller action code *) you should be able to do something similar to:
def report
#transactions = current_user.transactions.all
respond_to do |format|
format.html { render xlsx: 'report', stream: true}
end
end
Might help to do a sanity check on your loading. In your log as part of the 200 response you should get something like:
Completed 200 OK in 506ms (Views: 494.6ms | ActiveRecord: 2.8ms)
If the active record number is too high, or higher than the view number, this solution might not work for your query, and as suggested, this might need to be threaded or sent to a job.
Even if you can stream, I don't think it will be any faster. The problem is Axlsx is not going to generate your spreadsheet until you are done building it. And axlsx_rails just wraps that process, so it won't help either. So there will be no partial spreadsheet to serve in bits, and the delay will be just as long.
You should bite the bullet and try Sidekiq (which is very fast) or some other job scheduler. Then you can return the request immediately and generate the spreadsheet in the background. You will have to do some kind of monitoring or notification to get the generated report, or a ping back to another url using javascript that forwards to a new page when a flag is set on render complete. Your call there.
Having a job scheduler is also very convenient when you need to fire off an email in response to a request; the response can return immediately and not wait for the email to complete. Once you have a scheduler you will find more uses for it.
If you choose a job scheduler, axlsx_rails will let you use your template to generate the attachment, or you can create your own view context to generate the file. Or for a really bare bones way of rendering the template, see this test.
I'm developing an application, and on one page it requires approximately 12-15 GET requests to be made to an API in the background. My original intent was to make the requests using AJAX from jQuery, but it turns out that it is impossible to do so with the Steam Web API I am using.
Doing this in the Rails controller before the page loads is, for obvious reasons, very slow.
After I get the data from the API, I parse it and send it to the JavaScript using gon. The problem is that I don't know how to get and set the data after the page renders.
Here is what my controller would look like:
def index
#title = "My Stats"
if not session.key?(:current_user) then
redirect_to root_path
else
gon.watch.winlossdata = GetMatchHistoryRawData(session[:current_user][:uid32])
end
end
The function GetMatchHistoryRawData is a helper function that is calling the GET requests.
Using the whenever gem --(possibly, see below)....
Set a value in a queue database table before rendering the page. Using a "cron" task (whenever gem) that monitors the queue table you can make requests to the Steam API and populate a queue result table. On the rendered page you could implement a JavaScript periodic check with AJAX to monitor the queue result table and populate the page once the API returns a result.
Additional Info:
I have not used the whenever gem yet but I did some more reading on it and there might be an issue with the interval not being short enough to make it as close to real time as possible. I am currently doing my job processing with a Java application implementing a timer but have wondered about moving to whenever and CRON. So whenever might not work for you but the idea of an asynchronous processor doing the work of contacting the API is the gist of my answer. If the payload from the Steam API is small and returned fast enough then like what was stated above you could use a direct call via AJAX to the controller and then the Steam API.
Regarding the Rails code it should be pretty much standard.
controller:
def index
# Create a Steam API Queue row in the database and save any pertinent information needed for contacting the Steam API
#unique_id = Model.id # some unique id created for the Steam API queue row
end
# AJAX calls START
def get_api_result
# Check for a result using
params[:unique_id]
# render partial for <div>
end
# AJAX calls end
View: index
# Display your page
# Setup an intermittent AJAX call to "controller#get_api_result" with some unique id #{#unique_id} i.e. params[:unique_id] to identify the Steam API Queue table row, populate the result of the call into a <div>
external_processor_code (Whenever Gem, Java implementation, some Job processor, etc...)
Multiple threads should be available to process the Steam API Queue table and retrieve results every few seconds and populate the result table that will be read by the controller when requested via the AJAX call.
To give a complete example of this type of implementation would take some time so I have briefly, from the conceptual level, outlined it above. There might be some other ways to do this that could be more efficient with the way technology is expanding so please do some investigation.
I hope this is helpful!
(This question is a follow-up to How do I handle long requests for a Rails App so other users are not delayed too much? )
A user submits an answer to my Rails app and it gets checked in the back-end for up to 10 seconds. This would cause delays for all other users, so I'm trying out the delayed_job gem to move the checking to a Worker process. The Worker code returns the results back to the controller. However, the controller doesn't realize it's supposed to wait patiently for the results, so it causes an error.
How do I get the controller to wait for the results and let the rest of the app handle simple requests meanwhile?
In Javascript, one would use callbacks to call the function instead of returning a value. Should I do the same thing in Ruby and call back the controller from the Worker?
Update:
Alternatively, how can I call a controller method from the Worker? Then I could just call the relevant actions when its done.
This is the relevant code:
Controller:
def submit
question = Question.find params[:question]
user_answer = params[:user_answer]
#result, #other_stuff = SubmitWorker.new.check(question, user_answer)
render_ajax
end
submit_worker.rb :
class SubmitWorker
def check
#lots of code...
end
handle_asynchronously :check
end
Using DJ to offload the work is absolutely fine and normal, but making the controller wait for the response rather defeats the point.
You can add some form of callback to the end of your check method so that when the job finishes your user can be notified.
You can find some discussion on performing notifications in this question: push-style notifications simliar to Facebook with Rails and jQuery
Alternatively you can have your browser periodically call a controller action that checks for the results of the job - the results would ideally be an ActiveRecord object. Again you can find discussion on periodic javascript in this question: Rails 3 equivalent for periodically_call_remote
I think what you are trying to do here is little contradicting, because you use delayed_job when do done want to interrupt the control flow (so your users don't want to want until the request completes).
But if you want your controller to want until you get the results, then you don't want to use background processes like delayed_job.
You might want to think of different way of notifying the user, after you have done your checking, while keeping the background process as it is.
I have an action that takes a long time. I want to be able to provide updates during the process so the user is not confused as to whether he lost the connection or something. Can I do something like this:
class HeavyLiftingController < ApplicationController
def data_mine
render_update :js=>"alert('Just starting!')"
# do some complicated find etc.
render_update :js=>"alert('Found the records!')"
# do some processing ...
render_update :js=>"alert('Done processig')"
# send #results to view
end
end
No, you can only issue ONE render within a controller action. The render does NOTHING until the controller terminates. When data_mine terminates, there will be THREE renders, which will result in an error.
UPDATE:
You'll likely have to set up a JavaScript (jquery) timer in the browser, then periodically send an AJAX request to the server to determine the current status of your long running task.
For example the long running task could write a log as it progresses, and the periodic AJAX request would read that log and create some kind of status display, and return that to the browser for display.
It is impossible to handle the request that way. For each request, you have just one answer.
If your action takes a long time, then maybe it should be performed asynchronously. You could send user e-mails during the process to notify him of the progress.
I suggest that you to take a look on DelayedJob gem:
http://rubygems.org/gems/delayed_job
It will handle most difficult parts of dealing with assync stuff for you (serializing / deserializing your objects, storage, so on...).
Hope it helps you!