I want to programmatically retrieve my application’s User-Agent string. (Note: not the UA of a UIWebView in my application; the UA for NSURLConnection-based HTTP requests.)
There’s lots of guides on how to read an application's UIWebView User-Agent, but none on how to get at the <appname>/1.0 CFNetwork/456.23 string that CFNetwork stuffs inside its HTTP requests. There’s API for CFHTTPMessage to copy out all header fields, but I only have an NSURLRequest and an NSURLResponse, and these are unbridged opaque types. Help!
(Some good background reading: Changing the userAgent of NSURLConnection mentions the format that I want to get at. What HTTP User-Agent does my iOS program advertise itself as? does as well.)
Completely twisted: create a local socket on port 80 and make a local request (to yourself), read the HTTP headers.
Related
I'm currently upgrading our former webdav implementation to use IT-HIT.
In the process I noticed that read operation of a File can request the whole file or a part of it. I was wondering if there is a way to force to request always the whole file. Our webdav handles small files and there isn't much need for it.
I'm asking because in the documentation I'm using (Java client version 3.2.2420 ) I think it only specifies it for the write operation.
Thanks for your help.
The read operation is an HTTP GET request, which can contain a Range header. WebDAV clients as well as any other clients, like web browsers, can utilize GET requests to read and download file content. As part of the GET request, they can attach the Range header, specifying which part of the file content they want to get. For example, when you pause and then resume a download or when the download is broken and then restored, the Range request can be specified by the client:
GET https://webdavserv/file.ext
Range: bytes=12345-45678
To test if the server supports the Range header, the client app can send the HEAD request. If the server response contains Accept-Ranges: bytes header, the Range header is supported:
HEAD https://webdavserv/file.ext
...
Accept-Ranges: bytes
So the solution is to remove the Accept-Ranges header from the HEAD response. If the client can properly process the absence of the Accept-Ranges header, it will always request an entire file.
If you can not remove it directly from the code, in many cases you can remove or filter the header from the response before it is sent. The specific header removal code depends on your server (Java, ASP.NET, ASP.NET Core, OWIN, etc). For example for ASP.NET it will look like this:
protected void Application_PreSendRequestHeaders(object sender, EventArgs e)
{
HttpContext.Current.Response.Headers.Remove("Accept-Ranges");
}
For Java, you will need to create a filter: How do delete a HTTP response header?
By using NEPacketTunnelProvider I am able to get the destination host address, but I can not get the complete URL, is there any possibility that I can get the complete URL?
If you are referring to capturing a HTTP request path, you will need to sniff each Data packet by parsing each header until you get to the HTTP layer and determine the request path. It would likely take a bit of work to develop the packet parsing structure in Swift, and wouldn't work with HTTPS.
I was playing around with GZIP compression recently and the way I understand the following:
Client requests some files or data from a Web Server. Client also sends a header that says "Accept-Encoding,gzip"
Web Server retrieves the files or data, compresses them, and sends them back GZIP compressed to the client. The Web Server also sends a header saying "Content-Encoded,gzip" to note to the Client that the data is compressed.
The Client then de-compresses the data/files and loads them for the user.
I understand that this is common practice, and it makes a ton of sense when you need to load a page that requires a ton of HTML, CSS, and JavaScript, which can be relatively large, and add to your browser's loading time.
However, I was trying to look further into this and why is it not common to GZIP compress a request body when doing a POST call? Is it because usually request bodies are small so the time it takes to decompress the file on the web server is longer than it takes to simply send the request? Is there some sort of document or reference I can have about this?
Thanks!
It's uncommon because in a client - server relationship, the server sends all the data to the client, and as you mentioned, the data coming from the client tends to be small and so compression rarely brings any performance gains.
In a REST API, I would say that big request payloads were common, but apparently Spring Framework, known for their REST tools, disagree - they explicitly say in their docs here that you can set the servlet container to do response compression, with no mention of request compression. As Spring Framework's mode of operation is to provide functionality that they think lots of people will use, they obviously didn't feel it worthwhile to provide a ServletFilter implementation that we users could employ to read compressed request bodies.
It would be interesting to trawl the user mailing lists of tomcat, struts, jackson, gson etc for similar discussions.
If you want to write your own decompression filter, try reading this: How to decode Gzip compressed request body in Spring MVC
Alternatively, put your servlet container behind a web server that offers more functionality. People obviously do need request compression enough that web servers such as Apache offer it - this SO answer summarises it well already: HTTP request compression - you'll find the reference to the HTTP spec there too.
Very old question but I decided to resurrect it because it was my first google result and I feel the currently only answer is incomplete.
HTTP request compression is uncommon because the client can't be sure the server supports it.
When the server sends a response, it can use the Accept-Encoding header from the client's request to see if the client would understand a gzipped response.
When the client sends a request, it can be the first HTTP communication so there is nothing to tell the client that the server would understand a gzipped request. The client can still do so, but it's a gamble.
Although very few modern http servers would not know gzip, the configuration to apply it to request bodies is still very uncommon. At least on nginx, it looks like custom Lua scripting is required to get it working.
Don't do it, for no other reason than security. Firewalls have a hard or impossible time dealing with compressed input data.
I would like to know some connections that depart from my application.
So I use a proxy (in particular I'm using Charles for OSX) This works fine.
I have noticed that some of these connections are HTTPS (TLS). This is ok, but for these connections, I can only see the base URL and not the complete URL that is invoked. For example I can read: https://www.thewesite.com:443
I would expect not to see the body of the request but at least the header, and then also the whole URL I would expect to see it.
is that correct? is there a way to display the complete URL?
Since URL and the rest of the HTTP header are inside the encrypted connection you would need to enable SSL Proxying (that is Man-In-The-Middle attack) for the specific hosts. See http://www.charlesproxy.com/documentation/proxying/ssl-proxying/
I have a Delphi 6 application that uses an Indy TIdTCPClient instance to communicate with a web server. The reason I am not using an HTTP client directly is because the the server is an image streaming server that uses the same socket connection for receiving the command to start streaming as it does to start "pushing" images back to you. In other words, after you send it a typical HTTP POST request, it replies with an HTTP response, and immediately after that it starts sending out a stream of JPEG images.
I already know how to craft a proper POST request and send it using the TIdTCPClient WriteBuffer() method and then use the ReadBuffer() method to receive reply data. What I'd like to do instead is to send a POST request and then ask Indy to wait for a typical HTTP response including retrieving all the bytes in the response body if there is a Content-Length header variable. I of course want it to leave the JPEG frames intact that may have piled in after the HTTP response in the receive queue until I start requesting them (that is, I don't want it including any of the JPEG frames in the HTTP response to my streaming request command until I ask for them using a successive read call).
Is there a method that I can call on a TIdTCPClient that will retrieve completely a typical HTTP response with body content, and nothing else? I thought about using SendCmd() and checking the LastCmdResult property (type: TIdRFCReply) for the response, but I can't tell from the Indy documentation if it retrieves the response body content too if there is a Content-Length header variable as part of the response it returns, nor can I tell if it leaves the rest of the receive queue after the response intact.
What is the best way to accomplish this mixed mode interaction with an HTTP web server that pushes out a stream of JPEG frames right after you make the HTTP request to start streaming?
Also, if there is a clever way to have Indy split the frames using the JPEG frame WINBONDBOUDARY delimiting string, rather than accumulating blocks of data and parsing them out myself, please share that technique.
The correct way to read an HTTP response is to first read the CRLF-delimited response headers line-by-line until a blank line is encountered, aka a CRLF+CRLF sequence, then you can use those headers to decide how to read the remaining response data. The headers will tell you not only what kind of stream is being sent (via the Content-Type header), but also how the data is being framed (Content-Length, Transfer-Encoding: chunked, something specific to the particular Content-Type, etc).
To receive the headers, you can use the connection's Capture() method, setting its ADelim parameter to a blank string.
How you read the remaining data afterwards depends on the actual formatting/framing of the stream. Without knowing exactly what kind of stream you are receiving, there is no way to advise you how best to read it, as there are several different types of streaming protocols used by HTTP servers, and most of them are not standardized. Provide that information, then I/we can show you how to implement it with Indy.
You cannot use SendCmd() as the HTTP protocol does not format its responses in a way that is compatible with that method.