I am trying to read the first chunk of each image I am requesting to get its mime type and size which I'm able to do.
However, when I use Connection#reset it doesn't kill the connection and keeps downloading next chunks.
I am just wondering is it possible to close the connection after getting the first chunk?
This is my code right now
streamer = lambda do |chunk, _remaining_bytes, total_bytes|
image_format = MimeMagic.by_magic(chunk)
# other code
#connection.reset
end
Excon.defaults[:chunk_size] = 25
#connection = Excon.new(image_url)
#connection.get(response_block: streamer)
I don't believe there is a way currently to stop before the chunked response concludes. That being said, it might be possible that you could get the data you want from a head request and avoid the need for a get request?
Related
We have a Rails app with an integration with box.com. It happens fairly frequently that a request for a box action to our app results in a Passenger process being tied up for right around 15 minutes, and then we get the following exception:
Errno::ETIMEDOUT: Connection timed out - SSL_connect
Often it's on something that should be fairly quick, such as listing the contents of a small folder, or deleting a single document.
I'm under the impression that these requests never actually got to an open channel, that either at the tcp or ssl levels we got no initial response, or the full handshake/session-setup never completed.
I'd like to get either such condition to timeout quickly, say 15 seconds, but allow for a large file that is successfully transferring to continue.
Is there any way to get TCP or SSL to raise a timeout much sooner when the connection at either of those levels fails to complete setup, but not raise an exception if the session is successfully established and it's just taking a long time to actually transfer the data?
Here is what our current code looks like - we are not tied to doing it this way (and I didn't write this code):
def box_delete(uri)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Delete.new(uri.request_uri)
http.request(request)
end
I want cherrypy to return a xml file in response body in post.
In POST(self), I read a xml file and modify some of the attributes and do these things:
cherrypy.response.headers['Content-Type'] = 'application/soap+xml;charset=UTF-8'
cherrypy.response.headers['Content-Length'] = len(data)
cherrypy.response.body = data
cherrypy.log("response body is: %s" % cherrypy.response.body)
When the client calls, it won't get the body.
curl waits for few seconds and returns this:
curl: (18) transfer closed with 4018 bytes remaining to read
Not sure if I am doing the right thing to send the data back to the client.
I took wireshark trace and I am not seeing any data getting sent out from the server.
Can someone please suggest?
I think I have made this work. Earlier I was calling another function to set above mentioned values. Once I moved them in POST function, things started working. I am not sure what difference do it make. Now, I am setting them this way:
cherrypy.response.headers['Content-Type'] = 'application/soap+xml;charset=UTF-8'
cherrypy.response.headers['Content-Length'] = len(data)
return data
I have written a web-service using Erlang and Mochiweb. The web service returns a lot of results and takes some time to finish the computation.
I'd like to return results as soon as the program finds it, instead of returning them when it found them all.
edit:
i found that i can use a chunked request to stream result, but seems that i can't find a way to close the connection. so any idea on how to close a mochiweb request?
To stream data of yet unknown size with HTTP 1.1 you can use HTPP chunked transfer encoding. In this encoding each chunk of data prepended by its size in hexadecimal. Last chunk is a zero-length chunk, with the chunk size coded as 0, but without any data.
If client doesn't support HTTP 1.1 server can send data as binary chunks and close connection at the end of the stream.
In MochiWeb it's all works as following:
HTTP response should be started with Response = Request:respond({Code, ResponseHeaders, chunked}) function. (By the way, look at the code comments);
Then chunks can be send to client with Response:write_chunk(Data) function. To indicate client the end of the stream chunk of zero length should be sent: Response:write_chunk(<<>>).
When handling of current request is over MochiWeb decides should connection be closed or can be reused by HTTP persistent connection.
Is there any way I could get info such as how long did it take to connect to a remote server, time taken to receive the first byte of response, and the time taken to download the whole file?
I'm trying to create something like what Pingdom does.
(source: pingdom.com)
You can do it with sockets, like this:
require "socket"
# START MEASURING CONNECTION TIME
connection = TCPSocket.open("example.com", 80)
# END MEASURING CONNECTION TIME
connection.print "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n"
# START MEASURING RESPONSE FETCHING TIME
response = connection.read
# END MEASURING RESPONSE FETCHING TIME
connection.close
I'm implementing a REST API using ASP.NET MVC, and a little stumbling block has come up in the form of the Expect: 100-continue request header for requests with a post body.
RFC 2616 states that:
Upon receiving a request which
includes an Expect request-header
field with the "100-continue" expectation, an origin server MUST
either respond with 100 (Continue) status and continue to read
from the input stream, or respond with a final status code. The
origin server MUST NOT wait for the request body before sending
the 100 (Continue) response. If it responds with a final status
code, it MAY close the transport connection or it MAY continue
to read and discard the rest of the request. It MUST NOT
perform the requested method if it returns a final status code.
This sounds to me like I need to make two responses to the request, i.e. it needs to immediately send a HTTP 100 Continue response, and then continue reading from the original request stream (i.e. HttpContext.Request.InputStream) without ending the request, and then finally sending the resultant status code (for the sake of argument, lets say it's a 204 No Content result).
So, questions are:
Am I reading the specification right, that I need to make two responses to a request?
How can this be done in ASP.NET MVC?
w.r.t. (2) I have tried using the following code before proceeding to read the input stream...
HttpContext.Response.StatusCode = 100;
HttpContext.Response.Flush();
HttpContext.Response.Clear();
...but when I try to set the final 204 status code I get the error:
System.Web.HttpException: Server cannot set status after HTTP headers have been sent.
The .NET framework by default always sends the expect: 100-continue header for every HTTP 1.1 post. This behavior can be programmatically controlled per request via the System.Net.ServicePoint.Expect100Continue property like so:
HttpWebRequest httpReq = GetHttpWebRequestForPost();
httpReq.ServicePoint.Expect100Continue = false;
It can also be globally controlled programmatically:
System.Net.ServicePointManager.Expect100Continue = false;
...or globally through configuration:
<system.net>
<settings>
<servicePointManager expect100Continue="false"/>
</settings>
</system.net>
Thank you Lance Olson and Phil Haack for this info.
100-continue should be handled by IIS. Is there a reason why you want to do this explicitly?
IIS handles the 100.
That said, no it's not two responses. In HTTP, when the Expect: 100-continue comes in as part of the message headers, the client should be waiting until it receives the response before sending the content.
Because of the way asp.net is architected, you have little control over the output stream. Any data that gets written to the stream is automatically put in a 200 response with chunked encoding whenever you flush, be it that you're in buffered mode or not.
Sadly all this stuff is hidden away in internal methods all over the place, and the result is that if you rely on asp.net, as does MVC, you're pretty much unable to bypass it.
Wait till you try and access the input stream in a non-buffered way. A whole load of pain.
Seb