i have an HttpWebRequest that do 'POST' to a web server and get an HTML page In response.
I've been asked how is the best practice to know that the response i got has been changed or not?
I can't relay on the web server headers, they don't have to be.
this will increase performance in a way that i wont be need to parse the response over again and will go to the next request in a half of a sec or so.
thank you in advanced
You can tell your webserver about last modification date. see here. If you can't rely on that you have to parse your response anyway. You could do that quickly using md5. So you "md5" your current response and compare it with the previous.
You shouldn't be attempting to rely on the headers for a POST request, as it shouldn't emit any caching headers anyway.
What you need to do instead is perform a hash/checksum (this can either be CRC(32) for absolute performance or a "real" hash such as md5) on the returned content (this means, everything underneath the \r\n\r\n in the headers) and do a comparison that way.
It should be good enough to store the last request's checksum/hash and compare against that.
For example (psuedo):
int lastChecksum = 0;
bool hasChanged() {
performWebRequest();
string content = stripHeaders();
int checksum = crc32string(content);
if(checksum != lastChecksum) {
lastChecksum = checksum;
return true;
}
return false;
}
Related
How I get original bytes from request? Calling request.body.asBytes() I get this message:
asBytes() expected list of bytes, instead got List<_InternalLinkedHashMap<String, dynamic>>
I saw that HTTPRequestBody has the property retainOriginalBytes to use in this case, but where I set it?
Thanks!
Whatever endpoint you're hitting with your request is returning a Map in its body, not a list of Bytes.
I'm not sure if you can control the contents of what that endpoint is returning but if you can that would be the place to change it.
Check out the BytesBuilder class. Also, read the Aqueduct docs for Request and Response Objects. Hopefully this gets you on the right path!
You're on the right track; this will work correctly once you've set retainOriginalBytes to true. This must be done before the body is decoded.
In an HTTPController, the request body is decoded before your method that handles the request is called. Just before the decoding, HTTPController calls its willDecodeRequestBody() method. This method does nothing by default, but you can override it to set retainOriginalBytes:
#override
void willDecodeRequestBody(HTTPRequestBody body) {
body.retainOriginalBytes = true;
}
Here is an example of an application that does this.
In Delphi, I'm using Indy's TIdHTTPWebBrokerBridge coupled with TIdHTTP to send/receive data via HTTP. On the Server, I don't have any fancy handling, I always just respond with a simple content stream. If there's any issues, I only return information about that issue in the response content (such as authentication failed, invalid request, etc.). So, on the client side, can I assume that every successful request I make to this server will always have a response code of 200 (OK)?
I'm wondering because on the client, the requests are wrapped inside functions which return just a boolean for the success of the request.
Inside this function:
IdHTTP.Get(SomeURL, AStream);
Result:= IdHTTP.ResponseCode = 200;
This function handles any and every request which could possibly fetch data. If there were any issues in the request, This function should return False. In my scenario, since I always return some sort of content on the server, would the client always receive a response code of 200 in this function?
I guess the real question is, if I always return some sort of content and handle all exceptions on the server, then will the server always return status code of 200 to each request?
"Does every successful HTTP request always return status code 200?"
See w3.org: HTTP/1.1 Status Code Definitions (RFC 2616)
The answer is No. All 2xx are considered successful.
That may depend on the HTTP method used.
Should your web-server application always return 200 upon success? That may as well depend on the request method and the signal it intends for the client . e.g.
for PUT method (emphasis is mine):
If an existing resource is modified, either the 200 (OK) or 204 (No
Content) response codes SHOULD be sent to indicate successful
completion of the request.
for POST method:
The action performed by the POST method might not result in a resource
that can be identified by a URI. In this case, either 200 (OK) or 204
(No Content) is the appropriate response status, depending on whether
or not the response includes an entity that describes the result.
If a resource has been created on the origin server, the response
SHOULD be 201 (Created) and contain an entity which describes the
status of the request and refers to the new resource, and a Location
header (see section 14.30). Responses to this method are not
cacheable, unless the response includes appropriate Cache-Control or
Expires header fields. However, the 303 (See Other) response can be
used to direct the user agent to retrieve a cacheable resource.
As you can learn from the RCF, every method SHOULD have it's own success status codes, depending on the implementation.
Your other question:
"can I assume that every successful request I make to this server will always have a response code of 200 (OK)?"
You can always expect Status code 200, if your web server always responds with Status 200. Your web server application controls what response it returns to the client.
That said, Status code 200 is the Standard response for successful HTTP requests (The actual response will depend on the request method used), and in the real world of web servers, SHOULD be set as default upon successful request, unless told otherwise (As explained in Remy's answer).
To answer your specific question:
can I assume that every successful request I make to this server will always have a response code of 200 (OK)?
The answer is Yes, because TIdHTTPWebBrokerBridge wraps TIdHTTPServer, which always sets the default response code to 200 for every request, unless you overwrite it with a different value yourself, or have your server do something that implicitly replies with a different response code (like Redirect() which uses 302, or SmartServeFile() which uses 304), or encounter an error that causes TIdHTTPServer to assign a 4xx or 5xx error response code.
However, in general, what others have told you is true. On the client side, you should handle any possible HTTP success response code, not just 200 by itself. Don't make any assumptions about the server implementation.
In fact, TIdHTTP already handles that for you. If TIdHTTP encounters a response code that it considers to be an error code, it will raise an EIdHTTPProtocolException exception into your code. So if you don't get an exception, assume the response is successful. You don't need to check the response code manually.
If there is a particular response code that normally raises an exception but you do not want it to, you can specify that value in the optional AIgnoreReplies parameter of TIdHTTP.Get() or TIdHTTP.DoRequest(). Or, if you are are using an up-to-date Indy 10 SVN revision, a new hoNoProtocolErrorException flag was recently added to the TIdHTTP.HTTPOptions property so the EIdHTTPProtocolException exception is not raised for any response code.
Successful resposes are 2xx List_of_HTTP_status_codes
i did the following. Process straight all 200`s and LOG exceptions. worked, not a single non 200 - except unauthorized and timeouts (password or sometimes unavaliable server). but many/all responses will be considered for a wide range of mainstream apps.
while (iRedo < 3) do begin
s := Self.HTTPComponent.Get( sUrl );
if self.HTTPComponent.ResponseCode = 200 then begin
break;
end;
// IDEIA - log what happend if not 200
logWhatHappend( s, HTTPComponent ); // then log content, headers, etc
inc( iRedo ); sleep( 5 );
end;
I'm creating a pretty simple HTTP service using OpenRasta. For HEAD requests, the HTTP 1.1 spec states that HEAD requests should have the Content-Length set to "the size of the entity-body that would have been sent had the request been a GET" (section 14.13).
However, OpenRasta apparently sees that the response body is empty and automatically sets the Content-Length header to "0".
What is the recommended way to override this behavior?
Thanks-
Faced with exactly this problem my solution was to add an IPipelineContributor to deal with HEAD requests. The contributor was initialized as follows:
public void Initialize(IPipeline pipelineRunner)
{
// We're going to modify the HTTP method, so allow Rasta to have a go first
pipelineRunner.Notify(PreProcessRequest).After<HttpMethodOverriderContributor>();
}
In the pre-process step I swapped the HTTP method from HEAD to GET in order to allow the request to be processed as normal.
static PipelineContinuation PreProcessRequest(ICommunicationContext arg)
{
if (arg.Request.HttpMethod == "HEAD")
{
// Change the method to GET to allow normal processing
arg.Request.HttpMethod = HttpMethod.GET.ToString();
}
return PipelineContinuation.Continue;
}
At the end of the pipeline the response headers get written as expected, but nothing is written to the body.
I'm trying to send an XMLHttpRequest object to my Rails server but the headers are causing my function to stop. Here are my observations:
When I comment out the 3 lines of code that set the headers, then xhr.readyState will eventually equal 4 (alert boxes within the anonymous function fire off).
If any one of the 3 header lines are uncommented, then the xhr object never changes state (none of the alert boxes ever fire off).
function saveUserProfile(){
var user_email = $('#userEmail_box').val();
var xhr = new XMLHttpRequest();
xhr.onreadystatechange=function(){
if (xhr.readyState==4 && xhr.status==200)
{
alert("Yes: " + xhr.readyState);
}
alert("No: " + xhr.readyState);
}
var method = 'POST';
var params = 'userEmail=user_email';
var url = 'http://localhost:3000/xhr_requests.json';
var async = true;
//Need to send proper header information with POST request
xhr.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
xhr.setRequestHeader('Content-length', params.length);
xhr.setRequestHeader('Connection', 'close');
xhr.open(method, url, async);
xhr.send(params);
}
My three questions are:
What do I need to change in the code above in order to send data through the POST method?
I'm under the impression that the POST method requires some headers to be sent but am not clear about which ones though "xhr.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');" seems to be one that is often mentioned in references. Can somebody help me understand a) why headers need to be sent and b) which ones need to be sent?
I'm using a rails server and am developing locally. Ultimately, this code will be executed on the client side of a mobile device which will go to a hosted rails server for passing and receiving data. Are there limitations with using the POST method with a rails server? Keep in mind that I plan to use JSON when sending information to the client from the server.
Thanks!
UPDATE: The headers should come AFTER the opening the xhr request but BEFORE sending it:
xhr.open(method, url, async);
xhr.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
xhr.setRequestHeader('Content-length', params.length);
xhr.setRequestHeader('Connection', 'close');
xhr.send(params);
Hope this post saves somebody else 4 hours.
Does your web page with the JavaScript code also live on localhost:3000? If not, this is considered a cross-domain request, and your server will need to return special headers. So you have two options:
1) Host the web page on the same domain as the server, which will make this a same-domain request.
2) Have your server return the appropriate CORS headers. You can learn more about CORS here: http://www.html5rocks.com/en/tutorials/cors/
I'm implementing a REST API using ASP.NET MVC, and a little stumbling block has come up in the form of the Expect: 100-continue request header for requests with a post body.
RFC 2616 states that:
Upon receiving a request which
includes an Expect request-header
field with the "100-continue" expectation, an origin server MUST
either respond with 100 (Continue) status and continue to read
from the input stream, or respond with a final status code. The
origin server MUST NOT wait for the request body before sending
the 100 (Continue) response. If it responds with a final status
code, it MAY close the transport connection or it MAY continue
to read and discard the rest of the request. It MUST NOT
perform the requested method if it returns a final status code.
This sounds to me like I need to make two responses to the request, i.e. it needs to immediately send a HTTP 100 Continue response, and then continue reading from the original request stream (i.e. HttpContext.Request.InputStream) without ending the request, and then finally sending the resultant status code (for the sake of argument, lets say it's a 204 No Content result).
So, questions are:
Am I reading the specification right, that I need to make two responses to a request?
How can this be done in ASP.NET MVC?
w.r.t. (2) I have tried using the following code before proceeding to read the input stream...
HttpContext.Response.StatusCode = 100;
HttpContext.Response.Flush();
HttpContext.Response.Clear();
...but when I try to set the final 204 status code I get the error:
System.Web.HttpException: Server cannot set status after HTTP headers have been sent.
The .NET framework by default always sends the expect: 100-continue header for every HTTP 1.1 post. This behavior can be programmatically controlled per request via the System.Net.ServicePoint.Expect100Continue property like so:
HttpWebRequest httpReq = GetHttpWebRequestForPost();
httpReq.ServicePoint.Expect100Continue = false;
It can also be globally controlled programmatically:
System.Net.ServicePointManager.Expect100Continue = false;
...or globally through configuration:
<system.net>
<settings>
<servicePointManager expect100Continue="false"/>
</settings>
</system.net>
Thank you Lance Olson and Phil Haack for this info.
100-continue should be handled by IIS. Is there a reason why you want to do this explicitly?
IIS handles the 100.
That said, no it's not two responses. In HTTP, when the Expect: 100-continue comes in as part of the message headers, the client should be waiting until it receives the response before sending the content.
Because of the way asp.net is architected, you have little control over the output stream. Any data that gets written to the stream is automatically put in a 200 response with chunked encoding whenever you flush, be it that you're in buffered mode or not.
Sadly all this stuff is hidden away in internal methods all over the place, and the result is that if you rely on asp.net, as does MVC, you're pretty much unable to bypass it.
Wait till you try and access the input stream in a non-buffered way. A whole load of pain.
Seb