A rails application can use the request object to access user agent and more data about the request.
How to detect browser type and its version
But with curl, a developer can set the header data and more. How to use curl to get a GET request exactly same as using Chrome?
Can a rails application accurately detect when a request is sent by a software like curl versus a browser?
No. cURL can simulate any HTTP request with the correct configuration. There is no way to tell the difference between Chrome and cURL from an HTTP request alone.
If you're trying to make it harder to scrape data from your server, you'll want to use other methods (rate-limiting, authentication, etc.). But there is no perfect solution to prevent a determined scraper.
Related
I was reading up on how to create a telegram bot and I see that to set a web hook you use the GET http method. I assume that your link will be stored somewhere on telegram servers.
So now my question is:
Why not use the POST http method? What is the advantage of this design of using GET http method in this case rather than POST http method for something that pushes data?
Bot API supports GET and POST HTTP methods both. It's up to you what to use. (See this or this). I think it makes the API easy and fast to get started with.
For the sake of simplicity one might choose a simple GET request to set a webhook - Put together a Url with some parameters and call it in a web browser, done! Webhook is now set.
Still it is possible to do the same with a POST request (and you can argue it is the preferred way to do so). But it would need tools like Postman, CURL or some browser extensions.
We have existed API like
/api/activiation_code
each time, the activiation_code will be different, then server will create a token for this call and return it, usually each call will have different activiation_code which return different token.
Since this API need server to create something so it is designed as POST.
Can we design this API as HTTP GET ?
What is the pro and cons ?
You could design the API to support GET requests, but I would not recommend this. If your API is accessible via a website, a user could accidentally activate an account multiple times since the URL will be stored in the browser's history. Additionally, web crawlers could potentially supply values to your API through the URL if you support GET requests.
POST requests are much better because the information is included in the body of the request, not the URL. Thus, it is much less likely that something will go wrong accidentally.
I am trying to make a WISPr client. For this, I have read WISPr 1.0, 1.2 and 2.0 specifications.
Considering a 1.0 WISPr client, I have parsed the LoginURL from the hotspot login page, and try to make the authentication request (login = test, password = pass) :
POST : http://hotspot.server.com/login
Parameters : button=Login&UserName=test&Password=pass&FNAME=0&OriginatingServer=http://www.google.com
My credentials are good because I use them via the HTML form of the webpage.
With a http POST request, the authentication always failed with the response :
<AuthenticationReply>
<MessageType>120</MessageType>
<ResponseCode>100</ResponseCode>
<ReplyMessage>Authentication Failure</ReplyMessage>
</AuthenticationReply>
I have tried with differents hotspots of other networks with other good credentials.
Do you have an idea where my error comes in my authentication request ?
You coul try using GET. this worked for me
I've done some WISPr client tools in the past and this seems quite weird, at least from the data provided. Best approach I would suggest is to use Wireshark to capture the traffic of a login session via browser and compare it to the session handled by your code. If the browser also uses POST check for missing headers, malformed data etc. If in fact browser does use GET then you should try to stick to that scenario as suggested. Maybe the hotspot supports only one kind of method, e.g. notification polling. Still, without Wireshark details it is hard to guess. Also remember that there are certain differences between WISPr 1.0 and WISPr 2.0 and you haven't state what is the version of the hotspot you are logging in to.
Using fiddler I can intercept an ajax post request and with request composer I can resend the same request, causing the server to respond it normally. It doesnt matter if protocol is http or https (fiddler deciphers HTTPS traffic), with tools like Fiddler it is just possible.
On web applications side is it possible to understand or prevent such requests? How?
No. There's no way to prevent this. URLs are meant to be accessed. If it shouldn't be accessed, don't put it online or require some method of authentication. Some clients may pass an identifiable User-Agent header that can be restricted, and Fiddler probably does as well. However, the whole point of tools like Fiddler is to be able to make any type of custom request, which includes this User-Agent string. So, even if you block the default User-Agent, there's nothing stopping the user of Fiddler from changing the User-Agent to something that won't be blocked.
Is there a web application for testing post requests? What I imagine it'd be like is you would visit the site and then it would redirect you to a unique URL. You could then send a post request to the URL which would display the request after it was received.
Alternative from Microsoft: WFetch
POST request instruction
This looks like it would be more along the lines of what you're looking for:
http://www.htttools.com
Rest Client is a Firefox Add On that I have used in the past as an Http Post/Get testing tool.
The "net" tab in the Firebug plugin for Firefox will show you the contents of all requests including POSTs. You can also intercept and modify them with TamperData.
Fiddler will do the same for Internet Explorer and other windows programs. Wireshark will also show this information.
There are multiple approaches. If you want to do automated browser-based testing, you could use Selenium/Java or Windmill/Python. Alternatively, if you want to perform white-box testing, you can write scripts that make a http post request to the web application (e.g. using httplib if you are using Python), obtains the response and verifies that the response is as expected.
RequestBin allows you to create a temporary URL and view the last twenty requests.
With PutsReq you can test requests and simulate responses using JavaScript.