Given a known OData endpoint, what's the best way to determine the version of the OData service? The client in this scenario can support any version (1-4) but I need to know how to format the request.
For example, the OData-Version returns "4.0" for a V4 service, but a V3 service wouldn't even have that header.
In addition, querying the service root URL could be quite expensive for a service with a large number of entities. For example, a basic Dynamics 2016 Online service with no custom objects returns 2.7KB of data, when all I really want is the version header.
So what is the lightest-weight solution for getting a reliable version number? It's ok if the solution is "check this or, if missing, then check that". What's the "this" and "that"?
I found one question (How to find OData version from metadata) which seemed to get me partly there, but there are some issues with the answer.
First, it's focused on finding min/max version numbers where I would really prefer the max.
Second, it requires querying metadata, but that's a potentially massive load. /$metadata on Dynamics CRM 2016 Online results in a 3.7MB response (that takes 30 seconds to download on my current connection). I thought about requesting a dummy entity, like /dummy__entity and then examining the headers, but that seems a bit iffy to me because it would unnecessarily trigger error logging on the server and I'm not sure that an error response is always likely to have the headers I'm looking for.
OData 1.0/2.0/3.0
According to MS-ODATA 1.7 Versioning and Capability Negotiation:
The OData protocol that is defined in this document enables limited capability negotiation using the DataServiceVersion (section 2.2.5.3) and MaxDataServiceVersion (section 2.2.5.7) version request headers and the DataServiceVersion (section 2.2.5.3) response header.
When it says "limited", it means limited:
In a response from the server to the client, the DataServiceVersion (section 2.2.5.3) header is specified. The value states the version of the protocol that the server used in the request to generate the response and that is used by the client to determine if it can correctly interpret the response (that is, the value is not larger than the value of the MaxDataServiceVersion (section 2.2.5.7) header sent in the associated request). The value of the header is the lowest version of the protocol the server can use to fulfill the request.
So, basically, a conformant service capable of handling OData versions 1.0 to 3.0 would return "1.0" for features defined by OData 1.0, "2.0" for features defined by OData 2.0 and not present in OData 1.0, etc.
OData 4.0
According to OData Version 4.0 Part 1: Protocol, Section 8.1.5 Header OData-Version
OData services MUST include the OData-Version header on a response to specify the version of the protocol used to generate the response. The client MUST interpret the response according to the rules defined in the specified version of the protocol.
According to What's New in OData Version 4.0, Section 2.1.1 Improved: Protocol Versioning
Services now respond with the maximum protocol version supported by the server and indicated acceptable by the client.
Also "downgrade" to versions prior to 4.0 is not covered, and service publishers are advised to use new service root URLs for 4.0 services.
So for future versions of OData, from 4.0 onward, it seems that I can reliably get the maximum version I was seeking.
Conclusion
There is no apparent way to get the maximum version of OData supported by a pre-4.0 service. The DataServiceVersion response header will contain the lowest possible version number based on the URL features present, the version of OData that the service supports and the client-requested version.
However, beginning with OData 4.0, the OData-Version response header will always contain the maximum version based on what the service supports and what the client requested.
Unfortunately, in every instance I've tried so far, passing a "DataServiceVersion" header to an OData 4.0 service results in a 500 Internal Server Error response (with no OData-Version header). So it would seem that sending both OData-Version and DataServiceVersion headers isn't guaranteed to work.
Best bet seems to be sending OData-Version and then looking for a DataServiceVersion response header (which will likely be "1.0" even for a service that supports 3.0). If that header is present in the response, then send a second request with a DataServiceVersion header of "3.0". If you get a 4xx response, then try "2.0" etc.
Related
I am currently working on an application that makes requests using the .NET SDK for the Microsoft Graph API. Specifically to retrieve information about users and their OneDrives.
Microsoft throttles API requests by returning an HTTP 429 status code, and I have implemented a back-off using the Retry-After header. I have noticed however that I seem to be getting throttled after only a handful of requests.
I have also been using Microsoft Graph Explorer to test some of my API calls and have noticed that I never seem to get a 429 response when accessing the API via that method. After also seeing reports of people having issues with the OneDrive client on Linux that they managed to work around by changing their User-Agent header, I thought maybe I needed to set a User-Agent for my requests.
The result is that it seems that if I set the User-Agent header to be something like Mozilla/5.0 then all the throttling issues seem to disappear. I have searched high and low and so far haven't managed to find any documentation on what a valid User-Agent should be, and I would prefer to avoid making my app impersonate a browser, so I wondered whether there was any guidance or documentation around that I might have missed?
For instance, a User-Agent of Mozilla/5.0 seems to result in no throttling, but MyApp/1.0 results in throttling.
There isn't any guidance on the User-Agent header and to be honest, I'm not sure why this would have any effect on throttling.
Throttling in Microsoft Graph is handled by the underlying service you're interacting with. For example, the /notes/ endpoint throttling is governed by OneNote while /messages is governed by Exchange.
In most cases, OneDrive will throttle by number of concurrent requests, per app, per user. So using Delegated permissions, your app should generally be able to concurrently upload 4 files without a problem. Any more than that and you will start to see 429 responses.
there are some best-practices from Microsoft to handle throttling:
https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online#how-to-decorate-your-http-traffic-to-avoid-throttling
you should decorate your http user-agent as following:
NONISV|CompanyName|AppName/Version
Identify as NONISV and include Company Name, App Name separated by a pipe character and then adding Version number separated with a slash character
or
ISV|CompanyName|AppName/Version
Identify as ISV and include Company Name, App Name separated by a pipe character and then adding Version number separated with a slash character
Does the YouTube Data API Client Library for Java use Etags and/or gzip, as described at Getting started page?
Documentation is short (only find java docs) and don't say anything about it, so i guess is just a wrapper.
Based from this link, Etags are supported by youtube but it depends on what kind of data you are asking.
To use the etag, create a header request and put "If-None-Match" equal to your etag value. Note this should be a request header and not appended to the endpoint call. You can also use "If-Match".
Depending on what kind of API you are using, the way of inserting a new value to the request header may differ slightly. The ETag response-header field provides the current value of the entity tag for the requested variant.
You may also check on this related thread.
I'm trying to use etags in order to reduce both my bandwidth and my quota usage but /search returns an new etag even when nothing changed. It also still sends the content if I specify the previous etag in the header.
Is it supported for that api call or am I probably doing something wrong?
Etags are supported by youtube but it depends on what kind of data you are asking
ETags, a standard part of the HTTP protocol, allow applications to refer to a specific version of a particular API resource. The resource could be an entire feed or an item in that feed. This functionality supports the following use cases:
Caching and conditional retrieval – Your application can cache API
resources and their ETags. Then, when your application requests a
stored resource again, it specifies the ETag associated with that
resource. If the resource has changed, the API returns the modified
resource and the ETag associated with that version of the resource. If
the resource has not changed, the API returns an HTTP 304 response
(Not Modified), which indicates that the resource has not changed.
Your application can reduce latency and bandwidth usage by serving
cached resources in this manner.
The client libraries for Google APIs differ in their support of ETags.
For example, the JavaScript client library supports ETags via a
whitelist for allowed request headers that includes If-Match and
If-None-Match. The whitelist allows normal browser caching to occur so
that if a resource's ETag has not changed, the resource can be served
from the browser cache. The Obj-C client, on the other hand, does not
support ETags. Protecting against inadvertent overwrites of changes –
ETags help to ensure that multiple API clients don't inadvertently
overwrite each other's changes. When updating or deleting a resource,
your application can specify the resource's ETag. If the ETag doesn't
match the most recent version of that resource, then the API request
fails.
Using ETags in your application provides several benefits:
The API responds more quickly to requests for cached but unchanged
resources, yielding lower latency and lower bandwidth usage. Your
application will not inadvertently overwrite changes to a resource
that were made from another API client.
https://developers.google.com/youtube/v3/getting-started#etags
I usually scrape Youtube for Videos searches and I just store the Etag that it return. To use the etag, create a header request and put "If-None-Match" equal to your etag value. Note this should be a request header and not appended to the endpoint call. You can also use "If-Match".
Depending on what kind of API you are using, the way of inserting a new value to the request header may differ slightly
Going through the documentation for the Sharepoint API (located here: http://msdn.microsoft.com/en-us/library/office/dn605900(v=office.15).aspx) I am trying to figure out how to select chunks of documents. I was expecting $skip to work, as it is implied that most OData functionality should be present, but this doesn't work. I also can't figure out whether or not $skiptoken is a valid query param. It's listed in the documentation near the bottom (without further explanation), but I can't coax the proper result out of it so far.
If somebody knows about $skip or $skiptoken or another method available for getting chunked responses back, help with this problem would be great!
EDIT: to clarify, things like API_URL/files?$orderby=url&$top=5 work, but things like API_URL/files?$orderby=url&$skip=50 do not. However, I have just found a 'next' URL in the response which appears to provide server-side paging support. I'll try to figure out the use of $skiptoken from here.
The OData V4 protocol has specified (referencing 11.2.5.7 Server-Driven Paging) that:
OData services may use the reserved system query option $skiptoken when building next links. Its content is opaque, service-specific, and must only follow the rules for URL query parts.
OData clients MUST NOT use the system query option $skiptoken when constructing requests.
Thus, the implementation of the O365 SharePoint API that it publishes $skiptoken as a query option that the client should use for excluding first few items in the queried collection is a violation of the protocol. The client should use $skip for such scenario. But it seems from the O365 spec you attached that $skip is not implemented.
If you query a entity set of the O365 service and the response payload contains a next link (a #odata.nextLink annotation in the response JSON object), then it indicates that the service has server-side paging for the entity set. Typically the next link would be a URL containing the $skiptoken query option such as http://host/service/entityset?$skiptoken=n that the client can use for getting the next page.
The Gmail iOS API under the hood appears to be creating 'POST' requested with a payload that specifies an action. So for example if I want to get a single message by ID, I would expect (at least since the documentation on the API specifies it) that I would create a GET request with the correct URL schema. Instead the Gmail iOS implementation does not and utilizes more of an RPC approach. Unfortunately this means all POST requests have a default timeout of 240 seconds and cannot be changed, apparently an iOS bug/weird implementation. I can create a timer and cancel the request but why not make the implementation RESTful?
Yes, this is a quark with the Google iOS client libraries in general--they do not follow the standard REST semantics of the APIs like all the other client libraries use. I imagine due to some limitation in iOS. In any case, https://code.google.com/p/google-api-objectivec-client/ would be the best place to read more about it, discuss and file bugs. (Not specific to the gmail-api.)
The library was written to create JSON-RPC requests because those have always supported a batch request model. A mechanism for batch REST requests was added much later to Google's JSON API servers.
The timeout issue on POSTs was resolved in iOS 6, according to discussions online like this.
The project site does have links to the library's discussion group and issue tracker.