I'm trying to send a webm video stream over http from Cloud Run to a browser, but it consistently returns a 503 with the message Service Unavailable. Other requests return normally, so the service isn't actually unavailable. The response comes almost immediately, so it doesn't appear to be a timeout. The response headers don't carry over my content type (video/webm) but rather text/plain. The console logs spit out this error:
The request failed because either the HTTP response was malformed or
connection to the instance had an error.
The server and client behave entirely correctly on localhost. I understand there's a 15 minute maximum on requests with Cloud Run. I also inferred based on this blog post from a few months ago that I'd be able to stream video:
Here are some example use cases for server-side HTTP streaming:
Streaming large files (such as videos) from your serverless applications
Long-running calculations that can report using a progress indicator
Batch jobs that can return intermediate or batched responses
I've tried other video container formats (e.g., flv) as well as setting the Content-Length to maximum (Long.MAX_VALUE) or not setting it at all. However, given it works locally and the error I see in the logs, I'm inclined to believe that Cloud Run (or some layer in between) is balking at the response itself rather than passing it through. Is there something I'm missing about what Cloud Run expects?
Did you try using transfer-encoding: chunked ?
Related
I have a FastAPI service that I deploy with traefik via docker swarm. Everything runs fine until I request an endpoint supposed to return a JSON file just over 10 Ko. The call to this endpoint take several minutes before returning an empty string along with a 200 status code. I've checked all the logs I could find, and it looks like FastAPI completes the request properly and instantly (thus the 200 status code). It should send the response to my traefik reverse proxy hosted on a different node on my swarm, that would forward it to the client. However it looks like the response is lost somewhere on the way, and the client never get the expected JSON file, and gets an empty string instead.
Has this ever happened to you ? Is there a parameter to set for docker swarm networks to be handled this kind of data (it doesnt seem that heavy to me) ? Any help would be greatly appreciated, thanks a lot !
I tried changing the endpoint name, deploying the service with standard docker and traefik (it works perfectly), returning a smaller JSON file (works well with very small JSONs). I'm out of options :)
To answer my own question - If anyone stumbles unto this: this was caused due to the GCP infrastructure hosting my swarm.
The VPC provided by GCP had a MTU (Max transmission Unit) of 1460, where docker network defaults to 1500. The package sent to my reverse proxy was thus dropped as soon as it was bigger than 1460 bytes.
I have built a class which computes the http latency from a given url (note I don't want an ICMP ping, I need the http latency).
My system is based on a NSURLConnection, so when the delegate method connection:willSendRequest:redirectResponse: is called, I initialise a NSTimer (the timer is restarted if the connection is redirected again), and as soon as I get the call connection:didReceiveResponse: I take the time as the latency.
As far as I know, that should give a good approximation of the http latency, however results are quite far away from the results I get running it from the terminal:
curl -I -o /dev/null -s -w %{time_total}\\n http://google.com
Actually, which is even weirder, https latencies to the same server (ie. https://google.com) are even faster than http ones (just few ms, but faster). When comparing https latency results with the results I get running a curl from the terminal, they are quite close, however http latencies don't seem to be working fine.
I am running the code on iOS 9, allowing arbitrary loads, and I wonder if iOS is trying to redirect or adding any overhead to the http request that makes them slower.
Any clue about this would be really useful.
I think that the results are correct as http://google.com redirects to the https version. Your timer gets resetted since you're redirect and doesn't take into account the DNS needed for the DNS query.
Maybe you should try websites that don't redirect to https like 9gag.
Is it possible corba service response one request multiple times?
In my case, corba service collects a bunch of data which takes a long time for request. In order to reduce delay client receives response, we want service responses soon when the size of collected data reach 1024k. For example, the total data sizes is 10M, service responses client 10 times on one connection.
My understanding is that corba server should cache connection between client and server, and deliver new data on this cached one once a new data is available. The client, on other handle, should do while loop for incoming response. Either client or server should not close connection until server says all data is connected. This procedure is similar to that in chunked response in Http protocol.
I appreciate if you can provide some tips or sample links in this area.
The CORBA server side is only able to send the data to the client when the server application code returns the function call. If you have just one operation in IDL that returns 10M, than the ORB can only transmit that data to the client when the operation has finished. In order to allow the ORB to send the data it has you have to modify your IDL and add a way for the client to start the operation and than poll for chunks of data to be available. Each implementation of the poll than returns with one chunk.
Some examples of how to do this are part of TAO. You can find the examples under ACE_wrappers/TAO/examples/Content_Server. It also has an example where the server pushes the data to the client when a chunk is available.
I've built a few native iPhone applications that required uploading large/hirez image files to a remote server. I've found that on 3g networks a request can get blocked/dropped if it uses too much bandwidth in a certain amount of time. I believe that to be about 1mb/min
reference:
Max payload size for http request and response, iphone
How is this being handled in trigger.io's API call: request.ajax(options)?
Along the same lines, I've run into trouble with connections retrying multiple times after failure. By default is there any connection retry going on behind the scenes?... or will the error callback fire on first connection failure?
Also! Is there a way to set the timeout of the request from the client side?
Currently, we don't offer any bandwidth throttling in the request module. The HTTP library we are using doesn't support it (note that the ASIHTTPRequest wrapper is no longer maintained, so we can't use that, unfortunately...).
If we find an alternative HTTP library which does support what we need and throttling, then we'd certainly consider switching to use it!
FWIW, we've not had any customers report problems with app store rejection due to bandwidth throttling (or lack of it).
Any connection or HTTP errors will result in the error callback being called - you have control over whatever retry logic you want.
For timeouts, see http://docs.trigger.io/en/v1.4/modules/request.html#ajax - timeout is a support parameter in the options hash.
When Trying to upload 2GB of stream i got invalid content length error
am running Apache as frontend server to mongrel and chrome as my browser.
One more thing one i do it with mongrel alone am able to upload this 2 GB of stream ,cud anybody tell me whats the problem and how do i configure content length in apache??
I'd imagine the problem is that Apache considers it a denial of service attempt and has a cap to prevent you locking up all the server's resources (very reasonable), whether that's configurable or not I'm not sure - can't find anything but will keep hunting. If it's not, you can always build your own copy of Apache with the limit removed.
Have you considered sending the data in reasonable-sized chunks?
You might also wish to inspect the outgoing request using packet inspection or browser debugging tools. It's possible the content-length is being malformed by the browser (I doubt they have a 2GB test case...)