Not sure exactly since when, but some files that are downloaded from OneDrive by Graph API returns 95-byte contents. And it's all like following.
{"error":{"code":"serviceNotAvailable","message":"Service unavailable","retryAfterSeconds":30}}
More precisely, the ordinary file download for the file-location returned by the following API has the issue.
https://learn.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_get_content?view=odsp-graph-online
It seems the server returns the error as contents, not by returning HTTP errors.
This is related to the 'data-integrity', so I think you should always return HTTP errors.
It turned out that the C# HttpClient throws TaskCanceledException in an AggregateException when timeout happens.
So it's not the API problem itself but needs extra care when dealing with HttpClient.
HttpClient needs to be revised to throw HttpRequestException or some other HTTP related exception, other than throwing TaskCanceledException.
Related
This question is maybe more a tip for people to search a solution if they have the same problem (as I found the solution eventually).
I had an application that does some HTTP requests with a local server (a mix of GET/POST with JSON content in the request/response bodies). The server is a third-party application, and after I upgraded it to a recent version, my Delphi app was no longer working.
It turned out that it was now hanging on the statement:
IdHTTP.Post("URL", "Payload", "BytesStreamResult");
As a manual POSTMAN request was still working, it had to be on the Delphi client side.
Further isolating the issue showed that the HTTP POST request did get an HTTP 200 response with valid HTTP response headers, but then was getting stuck reading the response body. It was hanging on:
IOHandler.ReadLn
When I compared the headers with the POSTMAN response, I noticed that 'Transfer-Encoding: chunked' was missing in the Delphi response.
Finally, I noticed the code related to TIdHTTP's hoKeepOrigProtocol option, which is not set by default.
So, my POST request was "downgraded" to an HTTP 1.0 request, and I guess this now made the (updated) server to respond differently (I'm not an RFC expert, but I guess 'chunked' is maybe an HTTP 1.1 option only).
After setting this option, everything worked like before (and indeed, the response was now read as "chunked" in Delphi).
Summary:
Shouldn't hoKeepOrigProtocol be the default option? (why punish good citizens for those that are not...)
Can we intercept this? Now my POST is assuming upfront a streamed response and thus it hangs because the server doesn't write anything to the buffer.
What would that high-level code look like? As it seems a mix of interpreting the header response headers and then deciding if more response reading is required.
(it didn't do anything specific regarding time-outs, either. I have the impression it hangs forever, or at least > 10 minutes...)
TIdHTTP supports non-chunked responses (which yes, is an HTTP 1.1 feature), so the hanging would have to be caused by the server sending a malformed response (a bug that should be reported to the server author).
When reading a non-chunked and non-MIME response, TIdHTTP does not use IOHandler.ReadLn to read the response's body, as you claim. Only when reading the response's headers.
But, since you did not show what the response actually looks like, nobody can explain for sure exactly why the hang occurs.
Shouldn't hoKeepOrigProtocol be the default option?
At the time the option was first introduced, no. There were plenty of buggy HTTP 1.1 servers around that downgrading to HTTP 1.0 was warranted.
However, that was many years ago. Nowadays, HTTP 1.1 is much more mature, and such buggy servers are rare. So, feel free to submit a change/pull request to Indy's GitHub repo if you feel the default behavior should be changed.
Can we intercept this?
No. The behavior you describe is most likely caused by a bug in the HTTP server. Either it is not sending all of the data it should be, or else the response is likely malformed in a way that makes TIdHTTP expect more data than is actually being sent. Either way, all you can do is assign a non-infinite timeout to TIdHTTP.
it didn't do anything specific regarding time-outs, either. I have the impression it hangs forever, or at least > 10 minutes.
Indy is designed to use infinite timeouts by default. You can assign custom timeouts to TIdHTTP's ConnectTimeout and ReadTimeout properties.
Setting this prevent the HTTP protocol downgrade:
IdHTTP.HTTPOptions := IdHTTP.HTTPOptions + [hoKeepOrigProtocol];
This is, of course, dependant upon how the server processes the protocol specification, and if it results in issues or not.
Currently, I am building a javascript app that provides OneDrive filesystem support for chromeOS. Unfortunately, I encounter an issue when downloading a file via the OneDrive API due to preflights failing.
Deeply buried in the Microsoft API docs there is a mention of a workaround for CORS request for javascript apps, because preflights will fail. In order to circumvent a 302 redirect, a file download has to be executed in two steps: first get the download url, then download the file via that url. This works fine as a simple GET request doesn't require a preflight check.
Back to the filesystem. According to the chrome.fileSystemProvider API docs:
The results must be returned in chunks by calling successCallback several times.
This can be achieved by using the Range header on the GET request to the download-url, which actually is supported by OneDrive API using partial range downloads. However, now the request isn't simple anymore and therefore requires a preflight check, which ultimately fails.
So basically, the workaround failed because the problem the workaround tried to work around is still present in the workaround.
How can I overcome this problem? I obviously don't want to funnel the request through a proxy because of bandwidth and privacy issues. Can I perhaps simulate the Range header by downloading the entire file first and then serving it in byte-size chunks to the fileSystemProvider?
I came across a question asked almost 5 years ago with a very similar issue, but unfortunately it didn't get me any further.
I've been trying to replace source file by these docs - Vimeo API Replace source file.
I'm using Automatic (“pull”) uploads for upload and it works just fine. According to the docs, to replace a file, I should make PUT request to /videos/{id}/files and then proceed with POST to /me/videos, but every time PUT request fails and I get the same:
PHP Fatal error: Uncaught exception 'Vimeo\Exceptions\VimeoRequestException' with message
'Unable to complete request.[Operation timed out after 30000 milliseconds with 0 bytes received]'
in /home/<...>/vendor/vimeo/vimeo-api/src/Vimeo/Vimeo.php:154
POST and PUT request are fed with the same parameters. Maybe I should pass different type for PUT (POST gets 'type' => 'pull')?
Using Vimeo API PHP Lib v. 1.2
What am I missing?
Definitely a bug in the API, it shouldn't timeout. We're on it. For direct support reach out to us at support#vimeo.com.
One thing that can help in the short term is to increase your PHP timeout using the curl option CURLOPT_TIMEOUT (this can be set on the php lib using $lib->setCURLOptions([CURLOPT_TIMEOUT => 60]) for 60 seconds).
Once this bug is fixed, it might not resolve the issue. There's a good chance that Vimeo's request to get metadata about the the pull url is timing out, which could be a slow link, or a problem with the upload servers. In either case I recommend reaching out directly for support.
For anyone in the future, if you see timeouts, feel free to reach out to support#vimeo.com for more direct assistance.
I need to talk to some web service and thus I imported the WSDL. I now try to call it but it reports this exception: No Native to Message converter setVery, very irritating, especially since I have no permission to post code snippets from this service here. Still, have to try... Does anyone have some suggestions about how to fix this error?
The error is generated in rio.pas in the function TRIO.Generic. This line:
if not Assigned(FConverter) then
raise Exception.Create(SNoMessageConverter);
For unknown reasons, FConverter is set to nil, thus the exception is generated. This happens even before the request is sent. Nothing is sent to the service, since Delphi crashes even before it can call the service.
WSDL Import options, checked options:
One Outparam is Return
Unwind literal params
Generate destructors
Warning comments
Map string to widestring
Generate verbose information about types and interfaces
Ignore porttypes with HTTP bindings
Do not emit unused types
Validate enumeration types
Import fault types
Import header types
Process included and imported schemas
Generate class aliases as class types
Process nillable and optional elements
Actually, My system is new, Delphi was installed about 3 days ago and importing this WSDL was the first thing I did, basically using these default settings.
Use SoapUI consume the WSDL and make a mock service. Point your app at your SoapUI mockservice, and you can capture your outbound requests. Now you can turn around and submit those requests to the service and see the response. That should give you an idea of where the message is coming from. i.e. is it coming from Delphi's SOAP library as a result of something that it doesn't understand, or is it coming from the web service itself, as a result of something that IT didn't understand in your request?
Alternately, you can do this in Delphi: Intercept the inbound/outbound XML by leveraging the RIO_BeforeExecute/RIO_AfterExecute events of your HttpRIO object.
If your traffic is http (harder with SSL but possible) you can also intercept with Fiddler2.
Once you have the raw XML, submit requests with SoapUI, and see what you get. You may find that your requests need "tweaking", or if everything looks fine in SoapUI, you may need to tweak the responses before de-serialization.
I'm writing an actionscript library for an api. I use a URLLoader object to load data from the api. The problem I'm having is that whenever the api returns an http status in the 400s, actionscript treats this as an io error. This is all find and good, however, it seems like there is no way to access any data that was returned if this is the case. Consequently, any helpful xml about the cause of the error that gets returned is lost. Is there any way around this? It makes the library kind of a pain, if there can't be any useful information for developers when the api returns an error. Thanks for any help!
You can't get access to the data in an event of a 400. You can get the status code, however, by adding a listener for the HTTP status event.
If you control the back-end code, there are a couple of workarounds:
One option is to have the backend respond with 200s even in error cases when talking to a flash client, but with a special error code so the client knows that the 200 response is actually an error.
Another option is to set a cookie on the client containing the error message. Flash can't natively access cookies, but you can call out to javascript using ExternalInterface to read the cookie, or optionally the client can do another hit to a special back-end controller that reads the cookie and responds with an error message.