Is there some way to forward with Rails any incoming HTTP request to other web site as is, regardless of request type and its data?
In theory yes, you can collect a request headers, payload and details, and perform an HTTP request to another target reproducing the same request environment (cookies, headers, etc) effectively proxying the request... but I definitely think Rails is not the best solution for that.
You should definitely use a different approach and technology, for instance HAproxy or Nginx.
Related
I was playing around with GZIP compression recently and the way I understand the following:
Client requests some files or data from a Web Server. Client also sends a header that says "Accept-Encoding,gzip"
Web Server retrieves the files or data, compresses them, and sends them back GZIP compressed to the client. The Web Server also sends a header saying "Content-Encoded,gzip" to note to the Client that the data is compressed.
The Client then de-compresses the data/files and loads them for the user.
I understand that this is common practice, and it makes a ton of sense when you need to load a page that requires a ton of HTML, CSS, and JavaScript, which can be relatively large, and add to your browser's loading time.
However, I was trying to look further into this and why is it not common to GZIP compress a request body when doing a POST call? Is it because usually request bodies are small so the time it takes to decompress the file on the web server is longer than it takes to simply send the request? Is there some sort of document or reference I can have about this?
Thanks!
It's uncommon because in a client - server relationship, the server sends all the data to the client, and as you mentioned, the data coming from the client tends to be small and so compression rarely brings any performance gains.
In a REST API, I would say that big request payloads were common, but apparently Spring Framework, known for their REST tools, disagree - they explicitly say in their docs here that you can set the servlet container to do response compression, with no mention of request compression. As Spring Framework's mode of operation is to provide functionality that they think lots of people will use, they obviously didn't feel it worthwhile to provide a ServletFilter implementation that we users could employ to read compressed request bodies.
It would be interesting to trawl the user mailing lists of tomcat, struts, jackson, gson etc for similar discussions.
If you want to write your own decompression filter, try reading this: How to decode Gzip compressed request body in Spring MVC
Alternatively, put your servlet container behind a web server that offers more functionality. People obviously do need request compression enough that web servers such as Apache offer it - this SO answer summarises it well already: HTTP request compression - you'll find the reference to the HTTP spec there too.
Very old question but I decided to resurrect it because it was my first google result and I feel the currently only answer is incomplete.
HTTP request compression is uncommon because the client can't be sure the server supports it.
When the server sends a response, it can use the Accept-Encoding header from the client's request to see if the client would understand a gzipped response.
When the client sends a request, it can be the first HTTP communication so there is nothing to tell the client that the server would understand a gzipped request. The client can still do so, but it's a gamble.
Although very few modern http servers would not know gzip, the configuration to apply it to request bodies is still very uncommon. At least on nginx, it looks like custom Lua scripting is required to get it working.
Don't do it, for no other reason than security. Firewalls have a hard or impossible time dealing with compressed input data.
I'm using Ruby on Rails. Here is the requirement: the client (a native mobile app developed by me) will send a http post request to my Ruby code, my code will add some extra http headers (based on some business logic), then I need to "forward" or "redirect" this post request to another backend server (which has a REST service) and return its response back to the client.
I have been able to write a rack middleware to intercept the post request and add the extra headers. Originally I thought I could just use http redirect (status code: 307 for post request). But the problem is that the extra headers could NOT be submitted, which is the whole point of my code. So this isn't http redirect or forwarding per se, it's more like transforming a request.
I'm able to make a separate post request from my code using net http. This works. But I have to COPY data from the incoming request to my outgoing request (eg form data, http headers). This copying seems a bit tedious.
I would prefer some kind of simple "repackaging" (which is akin to http redirect or forwarding), that is I copy the whole incoming request to the outgoing request, slap on the extra headers and send it to the destination URL and be done with. I am not sure how to do this, and if doing it this way is even a good idea. For example, HTTP_USER_AGENT shows the OS, browser type of the client, when I'm making a new request, I probably don't need to send this on.
Alternatively, I can copy only the application specific data, because they're all the backend server (the destination of this "redirect") cares about. But I'm averse to hardcoding attributes in my code, causing close-coupling with the client (our native mobile app). Ideally I only copy application-specific data without hardcoding their attribute names. Is this possible? If so, how?
Any advice would be appreciated.
Thank you.
HTTP does not allow redirects for anything other than GET request.
(This is not technically correct but using HTTP 307 is kind of sketchy - see https://softwareengineering.stackexchange.com/questions/99894/why-doesnt-http-have-post-redirect)
If you need too send a POST request to another server for processing then using a proxy as you already seem to be doing is the correct solution.
Recreating the request in the proxy may seem tedious but it actually serves as a guarantee that you are calling the other servers "API" correctly.
While you can simply loop through the request headers:
uri = URI('http://www.example.com/todo.cgi')
req = Net::HTTP::Post.new(uri)
request.headers.each do |key, value|
req[key] = value
end
And pass the request form data:
req.set_form_data = request.request_parameters
You should ask yourself if it really is prudent to proxy everything.
See http://api.rubyonrails.org/classes/ActionDispatch/Request.html
I had 3 servers with balancing: server-1.test.com, server-2.test.com, server3.test.com.
And I used domain https://test.com.
I want to make a POST request such as : curl -X POST https://test.com -d "title=test"
After that, if the post request was redirect by balancing to server-1.test.com. it would be ok. But if not, it would be change to GET request. So I couldn't get data from GET request. I knew that problem from https redirect. Any suggestions for it?
I can't use GET request for replace POST. because it was made by webhook of mailchimp.
For http call you can user http package in meteor.
You can also refer the themeteorchef tutorial for more understanding for http in meteor.
One more thing for http calls meteor uses sync call. So you can also use wrapasync in meteor for http calls for sending response back to client.
Using fiddler I can intercept an ajax post request and with request composer I can resend the same request, causing the server to respond it normally. It doesnt matter if protocol is http or https (fiddler deciphers HTTPS traffic), with tools like Fiddler it is just possible.
On web applications side is it possible to understand or prevent such requests? How?
No. There's no way to prevent this. URLs are meant to be accessed. If it shouldn't be accessed, don't put it online or require some method of authentication. Some clients may pass an identifiable User-Agent header that can be restricted, and Fiddler probably does as well. However, the whole point of tools like Fiddler is to be able to make any type of custom request, which includes this User-Agent string. So, even if you block the default User-Agent, there's nothing stopping the user of Fiddler from changing the User-Agent to something that won't be blocked.
I am trying my hand in server applications using Indy Internet tools.
My client sends Post data (XML) in Unicode format.
Can I convey my preference to client (HTTP Client). I prefer Text. In general can a HTTP server send its preferences to its Clients?
Thanks for any hint or help.
The problem with this is the fact, that with only one POST the server has no way to respond, until the client has already sent the data.
The solution is to make two calls: One where the client asks for the server preferences and another to send the data. The OPTIONS HTTP method can be used for this scenario.
You can handle both requests on the same URL: If the clients makes an OPTIONS request the server responds with the configuration data. (via response headers) Then the client can make a POST request on the same URL and the server handles the data appropriately.
For further information see HTTP methods and HTTP headers, especially the Accept header.