We have the following situation:
We invoke a url which runs an action in a controller. The action is fairly long running - it builds a big string of XML, generates a PDF and is supposed to redirect when done.
After 60 seconds or so, the browswer gets a 200, but with content type of "application/x-unknown-content-type" no body and no Response Headers (using Tamper to look at headers)
The controller action actually continues to run to completion, producing the PDF
This is happening in our prod environment, in staging the controller action runs to completion, redirecting as expected.
Any suggestions where to look?
We're running Rails 2.2.2 on Apache/Phusion Passenger.
Thanks,
I am not 100% sure, but probably your Apache times out the request to Rails application. Could you try to set Apache's Timeout directive higher? Something like:
Timeout 120
I'd consider bumping this task off to a job queue and returning immediately rather than leaving the user to sit and wait. Otherwise you're heading for a world of problems when lots of people try to use this and you run out of available rails app instances to handle any new connections.
One way to do this easily might be to use an Ajax post to trigger creating the document, drop this into Delayed Job and then run a 10 second periodic check via ajax informing the waiting user of the jobs status. Once delayed_job has finished processing your task in the background and updated something in the database to indicate it is complete, then you can redirect the user via ajax to the newly created document.
Related
I am building a website, and I have an administrator page. The admin will have to run a reporting task, meaning that, the task will iterate all the records fetch information and generate a pdf file. Now this will be heavy on the app and the database.
What is the usual approach for it ? Should I have a button that calls a method of a class or should I have a rake task? I heard that HTTP GET requests have a limit and if the report generation takes more than that then it kills the request.
I would like to use send_data(....) so the user is given a nice download pop up box when the report is done. Will it be better to use a mailer and email it?
Thanks
We have similar functionality in our Rails apps at my job.
We have one URL/action that initiates the request to generate the PDF file, and returns right away saying the request was started successfully.
Then we have another action that we can poll with AJAX that returns whether or not the report is complete, and when it is complete, it gives the user the PDF.
The actual generation is done by a Sidekiq worker which is not subject to the webserver timeout.
I am writing a controller in Grails 2.X which kicks off a longish job. I would like to be able to render something to the web page which states that the process has started, with the id of the job that is in progress, and have that response actually show up in the user's web browser, and then continue processing on the job.
I have tried just using render without returning, and the user's browser just hangs until the entire job has completed, then renders that the job started.
I tried redirecting to a different action that renders my message, but that also hangs the browser until the job is complete.
I have looked into using filters and the afterInterceptor, but as best as I can tell these take effect and do their processing before the final page is sent back to the client. I need to send my final page back to my client and then continue processing.
You will want to kick off a background job. You can use quartz or look at grails 2.3 async items. If it is pretty long using Quartz is probably your best option.
You will want to return something the client can use to query for the state of the job such as a job id or some record you update once processing is finished.
I am designing a web service under Rails 3.2.8 which will execute external program for the users, the expected use case is:
1) User fill a form of parameters and submit the request
2) Rails execute a Matlab program based on user request. It would last a few minutes (less than 3mins) and generate a result file shows that the program is finished
3) During this process, redirecting user to a page shows "loading" status, and monitoring if the result file has been generated
4) Once the result file is generated, reload the page via AJAX and display the results.
Is there a "rails" way to do so? I did this before in JAVA SSH framework but pretty painful. What will be the tools I need? For example, do I need gems like backgroundjob
to manage the task queue? Or are there any "one-stand" gems can handle this? Thanks!
Use one of several job queue providers. Examples include DelayedJob (simplest one ever, no additional pieces required), Resque and Sidekiq (much faster, but you need a small Redis server to use them).
A "job" will be (depending on the provider) a class or instance, which implements the execution. If it implies running something in the shell, you can do it like
%x[matlab command --options > output.ext]
(it's just an example, you can use string interpolation too: %x[#{executable_name}])
I kind figured out what to use during the whole process:
1) User fill a form of parameters and submit the request
Using the method from this rails cast: http://railscasts.com/episodes/219-active-model for server side validation and gem 'client-side-validations' to enable client side check.
2) Rails execute a Matlab program based on user request. It would last a few minutes (less than 3mins) and generate a result file shows that the program is finished
Using gem 'DelayedJob' for task management and 'ChildProcess' to start the Matlab program.
3) During this process, redirecting user to a page shows "loading" status, and monitoring if the result file has been generated
Monitoring child process as well as the existence of the result file to check whether the task is finished.
4) Once the result file is generated, reload the page via AJAX and display the results.
Using polling method from this rails cast: http://railscasts.com/episodes/229-polling-for-changes to update the result page. I am not using pushing although it would be more secure - as polling seems to be a more light-weighted solution and can meet my requirements.
Obviously this is not the best practice, but satisfies my needs and easy to implement. Thanks for all the comments and answers.
My situation is like this:
1. User uploads 150MB zip file, with 600 files inside. It takes 4 minutes or so to upload the file to the server.
2. Server processes the file contents, takes 70 seconds or so.
3. The server responds with Service Unavailable, with a log like, "could not forward the response to the client... stop button was clicked"
4. The Rails application log says, 200 OK response was returned.
So, I am guessing it must be a problem within one of Nginx or Passenger that is causing it return with the error even thought it is going fine inside the Rails app. My suspect is a timeout setting, because I could reproduce it by just putting a sleep of 180 seconds inside the long running method and doing nothing.
I will appreciate if you guys know what specific nginx/passenger config may fix it.
If you're using S3 as your storage you may consider using something like carrierwave_direct to skip passing the file through the web server and instead upload directly to S3.
Like noted above you could incorporate a queueing process like delayed_job.
https://github.com/dwilkie/carrierwave_direct
I presume that nginx is the public-facing server and it proxies requests through to another server running your RoR application for you. If this assumption is correct, you may need to increase the value of your nginx proxy_read_timeout setting for the specific locations that are causing you trouble.
For long run request, I think you should return an 'please wait' page immediately and make the processing background. After the processing is completed, set the task in the database as 'completed'. Within the period, whenever user refresh the page, return 'please wait' immediately. After completed, return the result. You can set an autorefresh timeout in the page to refresh the page after an estimated period.
I'd instantly store the upload somewhere and redirect to a "please wait" page which asks for the status of the background processing and could even display some progress bar then, e.g. using ajax.
For the actual background processing I'd recommend DelayedJob which worked great for us and supports easy job deployment and implementation.
Does Rails provide a way to execute code on the server after the view is rendered and after the response is sent to the browser?
I have an action in my application that performs a lot of database transactions, which results in a slow response time for the user. What I'd like is to (1) perform some computations, (2) send the results of those computations to the browser, and then (3) save the results to the database.
It sounds like you want to implement a background job processor. This allows you to put the job into a queue to be processed asynchronously and for your users to not notice a long page load.
There are many options available. I have used and had no issues with delayed_job. Another popular one lately, which I have not used is resque.