In Rails, can an internal request be generated that behaves identically to an HTTP request? - ruby-on-rails

Within my Rails application, I'd like to generate requests that behave identically to "genuine" HTTP requests.
For a somewhat contrived example, suppose I were creating a system that could batch incoming HTTP requests for later processing. The interface for it would be something like:
Create a new batch resource via the usual CRUD methodology (POST, receive a location to the newly created resource).
Update the batch resource by sending it URLs, HTTP methods, and data to be added to the collection of requests it's supposed to later perform in bulk.
"Process" the batch resource, wherein it would iterate over its collection of requests (each of which might be represented by a URL, HTTP method, and a set of data), and somehow tell Rails to process those requests in the same way as it would were they coming in as normal, "non-batched" requests.
It seems to me that there are two important pieces of work that need to happen to make this functional:
First, the incoming requests need to be somehow saved for later. This could be simply a case of saving various aspects of the incoming request, such as the path, method, data, headers, etc. that are already exposed as part of the incoming request object within a controller. It would be nice if there was a more "automatic" way of handling this--perhaps something more like object marshaling or serialization--but the brute force approach of recording individual parameters should work as well.
Second, the saved requests need to be able to be re-injected into the rails application at a later time, and go through the same process that a normal HTTP request goes through: routing, controllers, views, etc. I'd like to be able to capture the response in a string, much as the HTTP client would have seen it, and I'd also like to do this using Rails' internal machinery rather than simply using an HTTP library to have the application literally make a new request to itself.
Thoughts?

a straight forward way of storing the arguments should be serializing the request object in your controller - this should contain all important data
to call the requests later on, i would consider using the Dispatcher.dispatch class method, that takes 3 arguments: the cgi request, the session options (CgiRequest::DEFAULT_SESSION_OPTIONS should be ok) and the stream which the output is written to

Rack Middleware
After doing a lot of investigation after I'd initially asked this question, I eventually experimented with and successfully implemented a solution using Rack Middleware.
A Basic Methodology
In the `call' method of the middleware:
Check to see if we're making a request as a nested resource of a
transaction object, or if it's an otherwise ordinary request. If it's
ordinary, proceed as normal through the middleware by making a call to
app.call(env), and return the status, headers, and response.
Unless this is a transaction commit, record the "interesting" parts of the
request's env hash, and save them to the database as an "operation" associated
with this transaction object.
If this is a transaction commit, retrieve all of the relevant operations
for this transaction. Either create a new request environment, or clone the
existing one and populate it with the values saved for the operation. Also
make a copy of the original request environment for later restoration, if
control is meant to pass through the application normally post-commit.
Feed the constructed environment into a call to app.call(env). Repeat for
each operation.
If the original request environment was preserved, restore it and make one
final call to app.call(env), returning from the invocation of `call' in the
middleware the status, headers, and response from this final call to
app.call(env).
A Sample Application
I've implemented an example implementation of the methodology I describe here, which I've made available on GitHub. It also contains an in-depth example describing how the implementation might look from an API perspective. Be warned: it's quite rough, totally undocumented (with the exception of the README), and quite possibly in violation of Rails good coding practices. It can be obtained here:
http://github.com/mcwehner/transact-example
A Plugin/Gem
I'm also beginning work on a plugin or gem that will provide this sort of interface to any Rails application. It's in its formative stages (in fact it's completely devoid of code at the moment), and work on it will likely proceed slowly. Explore it as it develops here:
http://github.com/mcwehner/transact
See also
Railscasts - Rack Middleware
Rails Guides - Rails on Rack

Related

Rails application design: Queueing, Resque, Background Services, and Redis

I am designing a Rails app that takes in requests, uses data within the request to call a 3rd party web service, process the reply and then sends out a response to the original requestor and also issues a PUT request to yet another service.
I am trying to wrap my head around how to design this Rails app as it's different from the canonical Rails structure.
The objects are Lists and Tasks. Each List has many Tasks, and each Task belongs to a List.
The request I would get is something like:
http://myrailsapp.heroku.com/v1/lists?id=1&from=2012-02-12&to=2012-02-14&priority=high
In this example I am requesting tasks from 2/12/2012 to 2/14/2012 with a high priority in List #1
I would then issue a 3rd party web service call like this:
http://thirdpartywebservice.com/v1/lists?id=4128&from=2012-02-12&to=2012-02-14&priority=high
As you can see some processing was done on the data (id was changed in this case)
The results are then sent back to the requestor and to another web service via PUT.
My question is, how do I set up the Rails app to handle these types of behaviors? How does the controller structure change? This looks like a good use case for queues, how do I distribute multiple concurrent requests among queues?
For one thing I don't need data persistence (data can be discarded after the response is sent out) and also data structure design is simplified. (I don't think I need ruby objects, simply dictionaries or hashes representing these would be lighter weight and quicker to implement)
Edit
So I broke down the work flow of the app into these components
Parse incoming request
Construct 3rd part web service request
Send 3rd party request
Enqueue a worker to process the expected response
Process the response once it arrives
Send the parsed result back as a response
Which of the standard ruby controllers handle each of these steps? What are the models needed besides Lists and Tasks?
You should still use a database because passing data to Resque is messy. Rather, you should store it in the database and then pass the id to the workers, fetch the data, commit any new data or delete the record. It's really up to you but this method is cleaner. You can also use a push service like faye to let the user know when the processing is complete.
If you expect to have many concurrent requests, I would recommend Sidekiq as it's less of a memory hog. Having 4-5 resque workers can already suck up about 512 MB. The controller structure should not change. Please comment on anything you need clarified and I'll be happy to update my answer.
EDIT
You would want to use a separate database store, such as Postgres. Not sure if it's important what models you need, but essentially this is what should be happening.
In your controller, create a Request object which contains the query params you want to query this 3rd party service with. Then enqueue a job to be handled by Sidekiq/Resque, let's call this ThirdPartyRequest and pass in the id of the Request object you just created as an argument. Then render a view here showing the Request object. Let's say that Request#response is still empty cause it hasn't been processed yet, so let the user know it's still processing.
A worker then handles your job ThirdPartyRequest. ThirdPartyRequest should then fetch the Request object and obtain the query params needed to contact the third party service. It does that then gets a Request. Update the Request object with this Request then save it.
class ThirdPartyRequest
def self.perform(request_id)
request = Request.find(request_id)
# contact third party service
request.response = ...
request.save
end
end
The user can continually refresh his page to check on his/her Request object. Once it gets updated with the response, they will know its completed. If you want the page to refresh automatically, look into faye/juggernaut/private_pub or a SaaS solution like Pusher.

Rails app with no databse and continually updated models

I'm wondering what the best way to go about developing a rails application with the following features:
All of the data comes from a SOAP request to a 3rd party
A background task will make this soap request every ~10s
The background task will parse the response and then update an ActiveRecord model accordingly
The data isn't written to a database at all, if the app fails, when we start it back up the data will come from the soap request again
Users will make a request to the app which will simply show data in the model (i.e. from the soap request).
The idea is to avoid making the SOAP request for every single user as the data won't change that frequently. Not using a database avoids reading and writing of data that only ever comes from the request anyway.
I imagine that all of this can be completely quite simply with a few gems but I've had a bit of trouble sorting through what meets my requirements and what doesn't.
Thanks
I'm not sure what benefit you're getting from using ActiveRecord in this case.
Perhaps consider some other type of persistance for the SOAP calls?
If the results form the WebService are really not changing, I would recommend the Rails caching mechanism. Wherever in your Rails app, you can do:
Rails.cache.fetch "a_unique_cache_key" do
... do your SOAP request and return the result
end
This will do the work within the block just once and fetch its result from the rails cache store in the future.
The cache store be of various types (one of which is the memcache store). I usually go with the file store for medium traffic sites, but you may choose another:
http://guides.rubyonrails.org/caching_with_rails.html

Syncing multple requests (user actions) with Backbone and Rails

The problem resides on building an architecture with Backbone and Rails
that handles syncing multiple actions to the server.
Assume the model is define on both Rails and Backbone.
I have an update and destroy operations on a model and I need them to synced
with the server on a user action (button click). On another part of the webapp,
these actions on this same model are synced on the moment they
made (easy, just send a restful ajax http request).
But in the first case, I can't really figure out an easy, stateless and atomic/transactional
save of the several actions (requests) the user took.
Sending multiple requests to the server makes the save non-atomic and a bit of non stateless.
Sending one big request with the actions formatted makes parsing on the server necessary.
So, is there another better solution?
If you want multiple updates, on different resources, as one atomic transaction, that is not REST.
So, of course, you will have to orchestrate the parameters and the requests in Rails. (but it's not about parsing, since you'll send JSON, more about creating a format for the aggregated parameters and figuring out what to do on the Rails side).
A nice way to handle multiple requests at once is at https://github.com/railscasts/414-batch-api-requests

Security in RESTful Rails controller actions - Should I always use respond_to format block?

Wondering if I should ALWAYS use the respond_to/format.xxx block in ALL of my actions in ALL of my controllers. This came up because I realized that, for apps using only HTML response format (no respond_to block), I could send some other type of request (say XML) and get a valid response. Is this considered insecure? Feels to me like it is, and it seems that the solution would be to ALWAYS add the respond_to/format.xxx block to EVERY action on EVERY controller even if you are just using HTML only. That way, any non-html requests will get rejected (as they should).
So, is there ever a situation where it would be OK to leave out the respond_to/format.xxx bits for any given action? Even if you were just doing a simple, standard, HTML-only app? It seems like different versions of rails code generators have generated different things regarding this over time. Just wondering what is considered best practice, and when you can get away without using it.
And if the answer is "Yes, you should use it all the time in every action to be secure", then would it not be considered boilerplate at that point? And shouldn't it be pushed down into rails somehow so we don't have to write it out over and over again every single time?
I don't think this has any impact on the security of your application. Presuming that the inbound HTTP request is well-formed and the client is authenticated to make the request, you as the server don't really care what the client receives. In fact, the client could be passing in all sorts of headers and parameters that you totally ignore and it's up to the client to handle unexpected formats or errors. This is integral to the underlying architecture of HTTP.
In any case, you can set the routes up to not accept the .format parameter, in which case a request for .xml will result in a 404 or other type of error.
If you only use one format, don't the respond_to block, it clutters code with unnecessary artifacts.

Rails: getting logic to run at end of request, regardless of filter chain aborts?

Is there a reliable mechanism discussed in rails documentation for calling a function at the end of the request, regardless of filter chain aborts?
It's not after filters, because after filters don't get called if any prior filter redirected or rendered.
For context, I'm trying to put some structured profiling/reporting information into the app log at the end of every request. This information is collected throughought the request lifetime via instance variables wrapped in custom controller accessors, and dumped at the end in a JSON blob for use by a post-processing script.
My end goal is to generate reports about my application's logical query distribution (things that depend on controller logic, not just request URIs and parameters), performance profile (time spent in specific DB queries or blocked on webservices), failure rates (including invalid incoming requests that get rejected by before_filter validation rules), and a slew of other things that cannot really be parsed from the basic information in the application and apache logs.
At a higher level, is there a different "rails way" that solves my app profiling goal?
You need a middleware that can execute code after the application has finished the request. See for more information in the guide: http://guides.rails.info/rails_on_rack.html
Updated link: https://guides.rubyonrails.org/rails_on_rack.html
around filters should solve this problem today: http://apidock.com/rails/ActionController/Filters/ClassMethods
(really old thread - but came across this before I found the answer)

Resources