I was wondering if there is a way to know if the http request made to a rails app is in cUrl?
I have a code that does some front-end process and is specific only for http request done through the web browser. Now, I want to be able to differentiate the normal http request to a cUrl request so that I can make another process only for cUrl request.
Thanks in advance.
if request.env["HTTP_USER_AGENT"] =~ /curl/i
in your controller should do the trick. Or you can do this in the routing level with the user agent option:
get '/resource' => 'Controller#curl_logic', constraints: {user_agent: /curl/i}
get '/resource/' => 'Controller#view_logic' # everything else
You can look at the user-agent in the HTTP request but that will only work if the curl client doesn't override it, which is easy to do. If you're only doing this for 'friendly' clients where you can trust the user-agent, it's straightforward. See the first example in one of the better resources for rails routing.
Related
A rails application can use the request object to access user agent and more data about the request.
How to detect browser type and its version
But with curl, a developer can set the header data and more. How to use curl to get a GET request exactly same as using Chrome?
Can a rails application accurately detect when a request is sent by a software like curl versus a browser?
No. cURL can simulate any HTTP request with the correct configuration. There is no way to tell the difference between Chrome and cURL from an HTTP request alone.
If you're trying to make it harder to scrape data from your server, you'll want to use other methods (rate-limiting, authentication, etc.). But there is no perfect solution to prevent a determined scraper.
I'm using Ruby on Rails. Here is the requirement: the client (a native mobile app developed by me) will send a http post request to my Ruby code, my code will add some extra http headers (based on some business logic), then I need to "forward" or "redirect" this post request to another backend server (which has a REST service) and return its response back to the client.
I have been able to write a rack middleware to intercept the post request and add the extra headers. Originally I thought I could just use http redirect (status code: 307 for post request). But the problem is that the extra headers could NOT be submitted, which is the whole point of my code. So this isn't http redirect or forwarding per se, it's more like transforming a request.
I'm able to make a separate post request from my code using net http. This works. But I have to COPY data from the incoming request to my outgoing request (eg form data, http headers). This copying seems a bit tedious.
I would prefer some kind of simple "repackaging" (which is akin to http redirect or forwarding), that is I copy the whole incoming request to the outgoing request, slap on the extra headers and send it to the destination URL and be done with. I am not sure how to do this, and if doing it this way is even a good idea. For example, HTTP_USER_AGENT shows the OS, browser type of the client, when I'm making a new request, I probably don't need to send this on.
Alternatively, I can copy only the application specific data, because they're all the backend server (the destination of this "redirect") cares about. But I'm averse to hardcoding attributes in my code, causing close-coupling with the client (our native mobile app). Ideally I only copy application-specific data without hardcoding their attribute names. Is this possible? If so, how?
Any advice would be appreciated.
Thank you.
HTTP does not allow redirects for anything other than GET request.
(This is not technically correct but using HTTP 307 is kind of sketchy - see https://softwareengineering.stackexchange.com/questions/99894/why-doesnt-http-have-post-redirect)
If you need too send a POST request to another server for processing then using a proxy as you already seem to be doing is the correct solution.
Recreating the request in the proxy may seem tedious but it actually serves as a guarantee that you are calling the other servers "API" correctly.
While you can simply loop through the request headers:
uri = URI('http://www.example.com/todo.cgi')
req = Net::HTTP::Post.new(uri)
request.headers.each do |key, value|
req[key] = value
end
And pass the request form data:
req.set_form_data = request.request_parameters
You should ask yourself if it really is prudent to proxy everything.
See http://api.rubyonrails.org/classes/ActionDispatch/Request.html
I had 3 servers with balancing: server-1.test.com, server-2.test.com, server3.test.com.
And I used domain https://test.com.
I want to make a POST request such as : curl -X POST https://test.com -d "title=test"
After that, if the post request was redirect by balancing to server-1.test.com. it would be ok. But if not, it would be change to GET request. So I couldn't get data from GET request. I knew that problem from https redirect. Any suggestions for it?
I can't use GET request for replace POST. because it was made by webhook of mailchimp.
For http call you can user http package in meteor.
You can also refer the themeteorchef tutorial for more understanding for http in meteor.
One more thing for http calls meteor uses sync call. So you can also use wrapasync in meteor for http calls for sending response back to client.
So I have a controller action that renders json.. I can visit the url in the browser and see the json data, verifying that the route is working properly...
Yet, if I do:
uri = URI("#{request.protocol}#{request.host_with_port}/my_controller/action")
Net::HTTP.get(uri)
I get "Timeout::Error: Timeout::Error"
... ?
You're using a single-threaded HTTP server, i.e. Webrick. This means that it will only be able to serve one request at a time. You're attempting to make a request to the webserver from within the webserver itself. It won't be able to complete this action because of now hopefully obvious reasons.
Use a different web server, such as Thin, that would allow for this, or choose a different way to do this.
Is there a web application for testing post requests? What I imagine it'd be like is you would visit the site and then it would redirect you to a unique URL. You could then send a post request to the URL which would display the request after it was received.
Alternative from Microsoft: WFetch
POST request instruction
This looks like it would be more along the lines of what you're looking for:
http://www.htttools.com
Rest Client is a Firefox Add On that I have used in the past as an Http Post/Get testing tool.
The "net" tab in the Firebug plugin for Firefox will show you the contents of all requests including POSTs. You can also intercept and modify them with TamperData.
Fiddler will do the same for Internet Explorer and other windows programs. Wireshark will also show this information.
There are multiple approaches. If you want to do automated browser-based testing, you could use Selenium/Java or Windmill/Python. Alternatively, if you want to perform white-box testing, you can write scripts that make a http post request to the web application (e.g. using httplib if you are using Python), obtains the response and verifies that the response is as expected.
RequestBin allows you to create a temporary URL and view the last twenty requests.
With PutsReq you can test requests and simulate responses using JavaScript.