Problem
I have added support for http compression in our self-hosted OWIN/Katana Web API OData 4 service but I do not see how to support compression in the .NET client. I'm using OData libraries v6.5.0 and I need to support compression/decompression in the client (OData v4 Client Code Generator). I am using Deflate encoding for the compression via an ActionFilter. Everything compresses correctly on the server as confirmed via Fiddler but I do not know how to configure the client to support this now that the OData client uses the Request and Response Pipelines instead of the now defunct WritingRequest and RecievingResponse events that once supported this very scenario.
Attempts
By experimentation I found that I can hook into the ReceivingResponse event on my DataServiceContext and then call ReceivingResponseEventArgs.ResponseMessage.GetStream() but I don't know what to do to overwrite the message content correctly. If I CopyTo() on the stream, I get a null reference exception at Microsoft.OData.Core.ODataMessageReader.DetectPayloadKind(). I presume this is because the stream was read to the end and the position needs to be set back to zero but I cannot do that because the stream also throws an exception when setting the position back because it says it does not support seeking. I presume this is simply due to the stream being read-only. Even if I could copy the stream to decompress it successfully, how do I modify the response message content with the decompressed content? I don't see any hooks for this at all in the RequestPipeline or ResponsePipeline. To clarify, I want to decompress the response message content and then set it for the materialization that occurs soon after, how might I do that? Extra credit for how to also send compressed requests to the OData service. Thanks!
OData client use the HTTPWebRequest and HTTPWebReponse, which supports the compression well. Try setting the AutomaticDecompression of HTTPWebRequest to Deflate or GZip, in SendingRequest2 pipeline event, like this:
private void OnSendingRequest_(object sender, SendingRequest2EventArgs args)
{
if (!args.IsBatchPart) // The request message is not HttpWebRequestMessage in batch part.
{
HTTPWebRequest request = ((HttpWebRequestMessage)args.RequestMessage).HttpWebRequest;
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
}
}
Then in response, HTTPWebResponse will decompress the stream automatically, before the materialization work.
Related
Background:
Let's have a WebAssembly (wasm) originating from .net code.
This wasm uses HttpClient and HttpClientHandler to access a backend API at https://api.uri.
The actual backend API location might change in time (like https://api.uri/version-5), but there is still this fixed endpoint, which provides redirection (3xx response) to the current location (which is in the same domain).
The API allows CORS, meaning it sends e.g. Access-Control-Allow-Origin: * headers in the responses.
In the normal (non-wasm) world, one just:
Plainly GETs the https://api.uri with no additional headers (CORS safe).
Retrieve the Location: header (containing e.g. https://api.uri/version-5) from the 3xx response as the final URI.
GETs/POSTs the final URI with additional headers (as needed, e.g. custom, auth, etc.).
Note: In ideal world, the redirection is handled transparently and the first two steps can just be omitted.
Although in the wasm world:
You are not allowed to (let the wasm/browser) send the OPTIONS pre-flight requests to a redirecting endpoint (https://api.uri).
You can't send any non-cors headers, when wanting to prevent pre-flight requests (reason for two stages, plain and full, described above).
You can't see the Location: header value (like https://api.uri/version-5) when trying the manual redirection (HttpClientHandler.AllowAutoRedirect = false), because the response is just artificially crafted with HTTP status code of 0 and ReasonPhrase == "opaqueredirect" - adoption to browser's Fetch API. What a nonsense! #1...
You can't see the auto-followed Location: header value in response.RequestMessage?.RequestUri, when trying the (default) automatic redirection (HttpClientHandler.AllowAutoRedirect = true), because there is still the original URI (https://api.uri) instead of the very expected auto-followed one (https://api.uri/version-5). What a nonsense! #2...
You can't send the full blown request with all the headers and rely on the automatic redirection, because it would trigger pre-flight, which is sill not allowed on redirecting endpoint.
So, the obvious question is:
Is there ANY way, how to handle such simple scenario from the Web Assembly?
(and not crash on CORS)
GET https://api.uri => 3xx, Location: https://api.uri/version-5
GET https://api.uri/version-5, Authorization: Basic BlaBlaBase64= ; Custom: Cool-Value => 200
Note: All this has been discovered within the Uno Platform wasm head, but I believe it applies for any .net wasm.
Note: I also guess "disabled" CORS (on the request side, via Sec-Fetch-Mode: no-cors) wouldn't help either, as then such request is not allowed to have additional headers/methods, right?
Am currently communicating to a Mobile device using Windows Compact Framework 3.5. The message sent to the device is built is as thus,
HttpResponseMessage result;
var response = Encoding.UTF8.GetBytes("<?xml version=\"1.0\" encoding=\"windows-1252\"?><message type=\"response\"><header><datetime>2013-04-03T09:49:35</datetime><sender version=\"1.1.4.1138\"><userid>Connect Server</userid></sender><commandlist><module>ADMIN</module><command1>VALIDATE</command1></commandlist><result type=\"ok\"/></header></message>");
result = Request.CreateResponse(HttpStatusCode.OK, response);
The device then retrieves the message and then uses
Encoding.UTF8.GetString(responseContent);
After decoding the message is:
<base64Binary xmlns="http://schemas.microsoft.com/2003/10/Serialization/">PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0id2luZG93cy0xMjUyIj8+PG1lc3NhZ2UgdHlwZT0icmVzcG9uc2UiPjxoZWFkZXI+PGRhdGV0aW1lPjIwMTMtMDQtMDNUMDk6NDk6MzU8L2RhdGV0aW1lPjxzZW5kZXIgdmVyc2lvbj0iMS4xLjQuMTEzOCI+PHVzZXJpZD5Db25uZWN0IFNlcnZlcjwvdXNlcmlkPjwvc2VuZGVyPjxjb21tYW5kbGlzdD48bW9kdWxlPkFETUlOPC9tb2R1bGU+PGNvbW1hbmQxPlZBTElEQVRFPC9jb21tYW5kMT48L2NvbW1hbmRsaXN0PjxyZXN1bHQgdHlwZT0ib2siLz48L2hlYWRlcj48L21lc3NhZ2U+</base64Binary>
Tried decoding the message on the server before sending it off and it's fine. Unsure what could be going wrong.
Any help would be greatly appreciated.
Request.CreateResponse() uses ObjectContent. For this scenario, you don't want that. You should use either StringContent or StreamContent to return the XML. See this question for details https://stackoverflow.com/a/15372410/6819
You are encoding your XML as binary. You are then returning a byte array. Then your client requests XML in the Accept: application/xml header. The Web API serializes the binary into XML. That's what you're seeing.
Just return the XML as a string and you should have no problems, unless you've tried that already?
See here for similar question.
In Delphi, I'm using Indy's TIdHTTPWebBrokerBridge coupled with TIdHTTP to send/receive data via HTTP. On the Server, I don't have any fancy handling, I always just respond with a simple content stream. If there's any issues, I only return information about that issue in the response content (such as authentication failed, invalid request, etc.). So, on the client side, can I assume that every successful request I make to this server will always have a response code of 200 (OK)?
I'm wondering because on the client, the requests are wrapped inside functions which return just a boolean for the success of the request.
Inside this function:
IdHTTP.Get(SomeURL, AStream);
Result:= IdHTTP.ResponseCode = 200;
This function handles any and every request which could possibly fetch data. If there were any issues in the request, This function should return False. In my scenario, since I always return some sort of content on the server, would the client always receive a response code of 200 in this function?
I guess the real question is, if I always return some sort of content and handle all exceptions on the server, then will the server always return status code of 200 to each request?
"Does every successful HTTP request always return status code 200?"
See w3.org: HTTP/1.1 Status Code Definitions (RFC 2616)
The answer is No. All 2xx are considered successful.
That may depend on the HTTP method used.
Should your web-server application always return 200 upon success? That may as well depend on the request method and the signal it intends for the client . e.g.
for PUT method (emphasis is mine):
If an existing resource is modified, either the 200 (OK) or 204 (No
Content) response codes SHOULD be sent to indicate successful
completion of the request.
for POST method:
The action performed by the POST method might not result in a resource
that can be identified by a URI. In this case, either 200 (OK) or 204
(No Content) is the appropriate response status, depending on whether
or not the response includes an entity that describes the result.
If a resource has been created on the origin server, the response
SHOULD be 201 (Created) and contain an entity which describes the
status of the request and refers to the new resource, and a Location
header (see section 14.30). Responses to this method are not
cacheable, unless the response includes appropriate Cache-Control or
Expires header fields. However, the 303 (See Other) response can be
used to direct the user agent to retrieve a cacheable resource.
As you can learn from the RCF, every method SHOULD have it's own success status codes, depending on the implementation.
Your other question:
"can I assume that every successful request I make to this server will always have a response code of 200 (OK)?"
You can always expect Status code 200, if your web server always responds with Status 200. Your web server application controls what response it returns to the client.
That said, Status code 200 is the Standard response for successful HTTP requests (The actual response will depend on the request method used), and in the real world of web servers, SHOULD be set as default upon successful request, unless told otherwise (As explained in Remy's answer).
To answer your specific question:
can I assume that every successful request I make to this server will always have a response code of 200 (OK)?
The answer is Yes, because TIdHTTPWebBrokerBridge wraps TIdHTTPServer, which always sets the default response code to 200 for every request, unless you overwrite it with a different value yourself, or have your server do something that implicitly replies with a different response code (like Redirect() which uses 302, or SmartServeFile() which uses 304), or encounter an error that causes TIdHTTPServer to assign a 4xx or 5xx error response code.
However, in general, what others have told you is true. On the client side, you should handle any possible HTTP success response code, not just 200 by itself. Don't make any assumptions about the server implementation.
In fact, TIdHTTP already handles that for you. If TIdHTTP encounters a response code that it considers to be an error code, it will raise an EIdHTTPProtocolException exception into your code. So if you don't get an exception, assume the response is successful. You don't need to check the response code manually.
If there is a particular response code that normally raises an exception but you do not want it to, you can specify that value in the optional AIgnoreReplies parameter of TIdHTTP.Get() or TIdHTTP.DoRequest(). Or, if you are are using an up-to-date Indy 10 SVN revision, a new hoNoProtocolErrorException flag was recently added to the TIdHTTP.HTTPOptions property so the EIdHTTPProtocolException exception is not raised for any response code.
Successful resposes are 2xx List_of_HTTP_status_codes
i did the following. Process straight all 200`s and LOG exceptions. worked, not a single non 200 - except unauthorized and timeouts (password or sometimes unavaliable server). but many/all responses will be considered for a wide range of mainstream apps.
while (iRedo < 3) do begin
s := Self.HTTPComponent.Get( sUrl );
if self.HTTPComponent.ResponseCode = 200 then begin
break;
end;
// IDEIA - log what happend if not 200
logWhatHappend( s, HTTPComponent ); // then log content, headers, etc
inc( iRedo ); sleep( 5 );
end;
I have written a web-service using Erlang and Mochiweb. The web service returns a lot of results and takes some time to finish the computation.
I'd like to return results as soon as the program finds it, instead of returning them when it found them all.
edit:
i found that i can use a chunked request to stream result, but seems that i can't find a way to close the connection. so any idea on how to close a mochiweb request?
To stream data of yet unknown size with HTTP 1.1 you can use HTPP chunked transfer encoding. In this encoding each chunk of data prepended by its size in hexadecimal. Last chunk is a zero-length chunk, with the chunk size coded as 0, but without any data.
If client doesn't support HTTP 1.1 server can send data as binary chunks and close connection at the end of the stream.
In MochiWeb it's all works as following:
HTTP response should be started with Response = Request:respond({Code, ResponseHeaders, chunked}) function. (By the way, look at the code comments);
Then chunks can be send to client with Response:write_chunk(Data) function. To indicate client the end of the stream chunk of zero length should be sent: Response:write_chunk(<<>>).
When handling of current request is over MochiWeb decides should connection be closed or can be reused by HTTP persistent connection.
I'm implementing a REST API using ASP.NET MVC, and a little stumbling block has come up in the form of the Expect: 100-continue request header for requests with a post body.
RFC 2616 states that:
Upon receiving a request which
includes an Expect request-header
field with the "100-continue" expectation, an origin server MUST
either respond with 100 (Continue) status and continue to read
from the input stream, or respond with a final status code. The
origin server MUST NOT wait for the request body before sending
the 100 (Continue) response. If it responds with a final status
code, it MAY close the transport connection or it MAY continue
to read and discard the rest of the request. It MUST NOT
perform the requested method if it returns a final status code.
This sounds to me like I need to make two responses to the request, i.e. it needs to immediately send a HTTP 100 Continue response, and then continue reading from the original request stream (i.e. HttpContext.Request.InputStream) without ending the request, and then finally sending the resultant status code (for the sake of argument, lets say it's a 204 No Content result).
So, questions are:
Am I reading the specification right, that I need to make two responses to a request?
How can this be done in ASP.NET MVC?
w.r.t. (2) I have tried using the following code before proceeding to read the input stream...
HttpContext.Response.StatusCode = 100;
HttpContext.Response.Flush();
HttpContext.Response.Clear();
...but when I try to set the final 204 status code I get the error:
System.Web.HttpException: Server cannot set status after HTTP headers have been sent.
The .NET framework by default always sends the expect: 100-continue header for every HTTP 1.1 post. This behavior can be programmatically controlled per request via the System.Net.ServicePoint.Expect100Continue property like so:
HttpWebRequest httpReq = GetHttpWebRequestForPost();
httpReq.ServicePoint.Expect100Continue = false;
It can also be globally controlled programmatically:
System.Net.ServicePointManager.Expect100Continue = false;
...or globally through configuration:
<system.net>
<settings>
<servicePointManager expect100Continue="false"/>
</settings>
</system.net>
Thank you Lance Olson and Phil Haack for this info.
100-continue should be handled by IIS. Is there a reason why you want to do this explicitly?
IIS handles the 100.
That said, no it's not two responses. In HTTP, when the Expect: 100-continue comes in as part of the message headers, the client should be waiting until it receives the response before sending the content.
Because of the way asp.net is architected, you have little control over the output stream. Any data that gets written to the stream is automatically put in a 200 response with chunked encoding whenever you flush, be it that you're in buffered mode or not.
Sadly all this stuff is hidden away in internal methods all over the place, and the result is that if you rely on asp.net, as does MVC, you're pretty much unable to bypass it.
Wait till you try and access the input stream in a non-buffered way. A whole load of pain.
Seb