I have a scenario where my rails controller action has to make a API request to a backend business logic server which does a lot of computations and returns me the result.
I'm thinking to show a loading page to the user and make the call asynchronous using Faye or any other option and redirect the user when the call is complete..
But even if I make the call asynchronous, the HTTP request needs to wait for the server to return the data after process, which would take around 20 seconds.
I would like to know what is the best way to make such calls in rails.?
I had faced a similar situation, below is the route that I took:
When the controller action is triggered
a. I fired off a 'async' request to the API using a worker(I used sidekiq)
b. Loaded a 'AJAX' spinner gif on top of a modal
The worker handling the API request runs on another thread which is synchronous and waits for the result from the API
When the processing is done, the worker fires off notification via 'Faye' which removes the modal and populates the data.
Return an HTTP response with status 202 Accepted(for the request that need to take long to process) and start making AJAX requests(to a URL, e.g /jobs/1) to check the status of the background job. Once your job has finished, update it's status so that your Javascript(AJAX) can handle the result of that background job.
Related
I have a rails controller (say in application A) whose response is dependent on data from another application (say application B).
I am using RabbitMq for inter application communication.
I can not render a response from the controller till the time the queue worker gets a response from application B. So currently when I get an HTTP call on application A, I publish to application B through a RabbitMq queue to fetch the required data. I am listening for the response of application B on a queue created by 'sneakers' gem. I want to receive this fetched data from 'sneakers' queue inside the controller of application A.
So the question is how can I wait for the RabbitMq, queue response inside the controller?
And also if I am able to wait for the response inside controller, how will I figure out which queue response is for which HTTP call.
To address the second issue, you can send a randomly generated string along with the request you send to application B. And the application B while responding will also send the same string that it got with the request. So the controller A will know for which request the response is.
Now coming to the first question, I think rabbitmq is not the correct tool to do such a thing. Even if you could wait for the message, It will be a very slow affair. compared to that a better way would be to expose application B as an API. It will increase the speed of the application by many times.
If API is not an option you can look at this link on how to create a consumer.
I have a Rails API which can handle requests from the clients. Clients use that API to perform analysis of their data. Client POSTs the data to API, API checks if that data have been analysed before. If so API just respond with analysis result. If the data haven't been analyzed before API:
Tells client that analysis started.
Establishes the connection with analyzing microservice.
Performs asynchronous (or deferred or i don't know) request to the analyzing microservice and waiting for response. The analysis takes much time so neither the API nor the microservice should be blocked while doing it.
When the response from analyzing microservice is returned API hands it to the client.
The main issue for me is to set up things such way that client could receive somehow the message "Your data had been sent to analysis" right after he performed the request. And then when analysis will be done client could receive its result.
The question is what approach I have to use in that case? Async responses, deferred responses, something else? And what known solutions could help me with that? Any gems?
I'm new to that stuff so I'm really sorry if I ask dumb questions.
If using HTTP you can only have one response to every request. To send multiple responses, i.e. "work in progress", then later the "results", you would need to use a different protocol, e.g. web sockets.
Since HTTP is so very common I'd stick with that in combination with background jobs. There are a couple of options which spring to mind.
Polling: The API kicks off a background jobs (to call the microservice) and responds to the client with a URL which the client can ping periodically for the result. The URL would respond with some kind of "work in progress" status until the result is actually ready). The URL would need to include some kind of id so the API can lookup the background job.
The API would potentially have two URLS; /api/jobs/new and /api/jobs/<ID>. They would, in Rails, map to a controller new and show action.
Webhooks: Have the client include a URL of its own in the request. Once the result is available have the background job hit the given URL with the result.
Either way, if using HTTP, you will not be able to handle the whole thing within a request/response, you will have to use some kind of background processing (so request to the microservice happens in a different process). You could look at Sidekiq, for example.
Here is an example for polling:
URL: example.com/api/jobs/new
web app receives client request
generates a unique id for the request, SecureRandom.uuid.
starts a background job (Sidekiq) passing in the uuid and any other parameters needed
respond with URL such as example.com/api/jobs/
--
background job
sends request to microservice API and waits for response
saves result to database with uuid
--
URL: example.com/api/jobs/UUID
look in database for UUID, if not found respond that job is "in progress". If found return result found in database.
Depending on what kind of API you use. I assume your clients interact via HTTP.
If you want to build an asynchronous API over HTTP the first thing that you should do: accept the request, create a job, handle it in the background and immediately return.
For the client to get the response you have to 2 options:
Implement a status endpoint where clients can periodically poll the status of the job
Implement a callback via webhooks. So the client has to provide a URL which you then call after you're done.
A good start for background processing is the sidekiq gem or more general ActiveJob that ships with Rails.
I have a Rails app that can take a long time to prepare its response to some queries. (Mostly the delay is rendering the dataset into JSON or YAML.) The app sits behind a proxy whose configuration I cannot alter, with the result that these long-running queries tend to get terminated by the proxy as timeouts. Chunking doesn't help because there's nothing to chunk until the render is fully complete.
Is there any supported or already existing way in Rails to set up an asynchronous repeating task that could send back 102 Processing responses to keep the proxy happy until the complete response is ready?
I would really prefer not to have to implement pagination semantics.
I have control over the app and the client; both bits are my code. I don't have control over the proxy, nor the app's server.
Any suggestions are really welcome!
I would likely solve the problem by POSTing the initial request and having the rails app return the appropriate HTTP status code. Then I'd have javascript on the client side that would poll the server at reasonable intervals for the status of the render. The status action could return the 102 response until the processing is complete. Then you could insert a link into the page with the javascript that the user could click to download the finished file.
My task is to deliver request content before the request has been processed.
How can I do that?
public ActionResult UpdateRecord()
{
Response.Write("OK");
Response.End();
// I need to complete the request here and
// perform some action in the work thread
Thread.Sleep(2000);
return Content("Something user never sees");
}
You cannot send content back to the client in the same request after the request has ended. It's just not possible.
It sounds to me like what you actually want is an asynchronous processing request. This is where you start a process in one request, then you continue to poll the server (refresh the page) periodically until the request is done.
This is typically accomplished using a background service of some kind, such as a Message Queue, or Web Service that triggers a Windows Service to process something. In a pinch, you can use the a Scheduled Task, or even a Cache removal callback trick as well.
Using a background thread won't generally work because threads started by an ASP.NET worker process are terminated after the request ends.
Another possibility, if you don't want to use polling is to use a push service like SignalR to signal the page that the job has been finished.
EDIT:
Well, as I said, you can't do it like that. You have to push that work into some kind of background process, however you can't just spin off a thread because threads will get terminated once the request ends.
There are many questions (and answers) here on SO on how to do this, and they all boil down to what I mentioned above. Scheduled tasks, windows service to do the processing, the Cache removal callback trick, etc.. you could also use a WCF one-way call with a WCF service or any of a number of other ways.
One place in my rails app requires loading a number of responses from an external server, which currently looks like this:
User makes an AJAX request to the server. "Loading data..." is displayed.
5-30 seconds later, the rails app sends response (assuming the data has not been cached).
It would be much better if I could keep the user informed during that long waiting period with messages informing them of the progress of the request. Such as:
User makes request (as before).
Message "Retrieving ABC" displayed
Message "Retrieving XYZ" displayed
Message "Processing data" displayed
Full response as normal.
How can I go about doing this? I don't think that sending back multiple JavaScript responses to one request is possible, but have no idea what the correct way of doing this is!
This is tricky but Rails supports the notion of streaming a request.
But you probably have to do a lot of work in your project to make this work.
Tenderlove (Aaron Patterson) posted a intro into how Streaming works in Rails and I believe there is a Railscast on this topic.
Probably a simpler solution would be to split this into multiple requests.
So the main request (assuming it's an ajax request) takes forever to complete.
Meanwhile you poll the status on a different ajax request and the main action updates the database with it's process so the other request can retrieve that status and send back the appropriate response (where in the process the main request currently is)
So I'd assign each request something like a request id and then have a database table for those requests and their statuses (could be as simple as having only id:integer and status:string)
You assign the request id on the client (use some random data to create a hash or something) and start the long request with that Id.
The client then polls another endpoint with that same id to get the status back.
The long running request in the meantime updates the Status table with the id it was given and where it is currently in processing that request.