I'm having problems using the server-side paging, utilizing the #odata.nextlink to fetch the next page of data from Microsoft Graph, based on the information in this page. I'm using raw GETs, with the authorization token set in the header (ie, I'm not using a language API, I'm trying this from Powershell using curl). I've scrubbed sensitive data from the following snippets, replacing them with x's, but hopefully the problematic info comes across.
For the first GET, I query with
https://graph.microsoft.com/beta/drives/b!Gxxxxx-xxxxxxge/root:/ReallyBigFolder:/children?top=200
and I get a response with 200 items, as expected. The #odata.nextlink field in this response is
https://graph.microsoft.com/beta/drives/b!Gxxxxx-xxxxxxge/root/children?top=200&$skiptoken=Paged%3dTRUE%26p_SortBehavior%3d0%26p_FileLeafRef%3d279%252ezip%26p_ID%3d208%26p_FileDirRef%3dMaintenance%2520Department%252fReallyBigFolder%26RootFolder%3dMaintenance%2520Department%252fReallyBigFolder
For the examples in the Microsoft Graph documentation linked above, the $skiptoken=... part has random-looking numbers, but mine has $skiptoken=Paged=TRUE&etc. Perhaps the API has changed the response since the documentation was written, or mine is completely incorrect.
My understanding from the documentation is that I should be able to use this URL as an opaque value, and GET it from the Graph API (with auth token of course) without modification. However, when I do this, the response is
{"#odata.context":"https://graph.microsoft.com/beta/$metadata#drives('b%21Gxxxxx-xxxxxxge')/root/children","value":[]}
Where I'm expecting to get another 200 files listed, there are no files returned at all, and it appears the path is gone, pointing to the root rather than the subfolder like it should have been.
I've also tried this in Graph Explorer with both the /beta and /v1.0 endpoints, and it fails in the same way there as well.
Where am I going wrong?
Edit with details for debugging: Note: Graph Explorer doesn't seem to display the Date field from headers, so I'm using Postman Chrome Plugin for these values.
First GET request is to
beta/drives/b!xxx-xxxge/root:/Really%20Big%20Folder/ReallyBigFolder:/children
With response headers
Cache-Control →private
Content-Encoding →gzip
Content-Type →application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8
Date →Fri, 26 May 2017 19:07:54 GMT
Duration →2033.3889
OData-Version →4.0
Transfer-Encoding →chunked
Vary →Accept-Encoding
client-request-id →6faf5d1d-a291-410a-b269-f4667187d7cb
request-id →6faf5d1d-a291-410a-b269-f4667187d7cb
x-ms-ags-diagnostic →{"ServerInfo":{"DataCenter":"North Central US","Slice":"SliceB","ScaleUnit":"002","Host":"AGSFE_IN_11","ADSiteName":"CHI"}}
and nextLink (obfuscated slightly for security)
https://graph.microsoft.com/beta/drives/b!xxx-xxxge/root/children?$skiptoken=Paged%3dTRUE%26p_SortBehavior%3d0%26p_FileLeafRef%3d279%252ezip%26p_ID%3d208%26p_FileDirRef%3dGSH%2520Test%252fMaintenance%2520Department%252fReally%2520Big%2520Folder%252fReallyBigFolder%26RootFolder%3d%252fGSH%2520Test%252fMaintenance%2520Department%252fReally%2520Big%2520Folder%252fReallyBigFolder
Following the nextLink produces headers (unchanged headers omitted):
Date →Fri, 26 May 2017 19:15:17 GMT
Duration →512.9537
client-request-id →6ba61712-a423-4bc8-9376-cc62bf854329
request-id →6ba61712-a423-4bc8-9376-cc62bf854329
x-ms-ags-diagnostic →{"ServerInfo":{"DataCenter":"North Central US","Slice":"SliceA","ScaleUnit":"001","Host":"AGSFE_IN_7","ADSiteName":"CHI"}}
and resulting body:
{
"#odata.context": "https://graph.microsoft.com/beta/$metadata#drives('b%21xxxx-xxxxge')/root/children",
"value": []
}
You are correct that the nextLink should be an opaque URL that returns you the next set of results. The format of that string may change over time, so you should not try to parse or otherwise interpret the string, but the usage should be the same.
The response that you are getting back is consistent with an empty result -- meaning that there are no additional files to list.
How many results do you have in ReallyBigFolder? What happens if you set top to a different value (say, 5? 1000?)
Note that the #odata.context describes the result, but is not necessarily the same as the request URL. Is the #odata.context that you get back from nextLink different than that you got back from the initial request? It should be the same...
Related
Background:
Let's have a WebAssembly (wasm) originating from .net code.
This wasm uses HttpClient and HttpClientHandler to access a backend API at https://api.uri.
The actual backend API location might change in time (like https://api.uri/version-5), but there is still this fixed endpoint, which provides redirection (3xx response) to the current location (which is in the same domain).
The API allows CORS, meaning it sends e.g. Access-Control-Allow-Origin: * headers in the responses.
In the normal (non-wasm) world, one just:
Plainly GETs the https://api.uri with no additional headers (CORS safe).
Retrieve the Location: header (containing e.g. https://api.uri/version-5) from the 3xx response as the final URI.
GETs/POSTs the final URI with additional headers (as needed, e.g. custom, auth, etc.).
Note: In ideal world, the redirection is handled transparently and the first two steps can just be omitted.
Although in the wasm world:
You are not allowed to (let the wasm/browser) send the OPTIONS pre-flight requests to a redirecting endpoint (https://api.uri).
You can't send any non-cors headers, when wanting to prevent pre-flight requests (reason for two stages, plain and full, described above).
You can't see the Location: header value (like https://api.uri/version-5) when trying the manual redirection (HttpClientHandler.AllowAutoRedirect = false), because the response is just artificially crafted with HTTP status code of 0 and ReasonPhrase == "opaqueredirect" - adoption to browser's Fetch API. What a nonsense! #1...
You can't see the auto-followed Location: header value in response.RequestMessage?.RequestUri, when trying the (default) automatic redirection (HttpClientHandler.AllowAutoRedirect = true), because there is still the original URI (https://api.uri) instead of the very expected auto-followed one (https://api.uri/version-5). What a nonsense! #2...
You can't send the full blown request with all the headers and rely on the automatic redirection, because it would trigger pre-flight, which is sill not allowed on redirecting endpoint.
So, the obvious question is:
Is there ANY way, how to handle such simple scenario from the Web Assembly?
(and not crash on CORS)
GET https://api.uri => 3xx, Location: https://api.uri/version-5
GET https://api.uri/version-5, Authorization: Basic BlaBlaBase64= ; Custom: Cool-Value => 200
Note: All this has been discovered within the Uno Platform wasm head, but I believe it applies for any .net wasm.
Note: I also guess "disabled" CORS (on the request side, via Sec-Fetch-Mode: no-cors) wouldn't help either, as then such request is not allowed to have additional headers/methods, right?
I'm using python to make calls to the Graph API regarding planner and tasks. whenever I use PATCH to try and update the task I get a 204 response back and the task remains unchanged. According to Microsoft's documentation here, this request should always return either a 200, or a 400 level error.
I have tried changing the data that I send to the server, to change the title rather than the dates, however I get the same 204 response no matter what data I send or what field I attempt to change. I have no problem making other graph calls like updating files in One Drive or getting data about a user
def SetDates(task):
'''Update planner to match the start date and due date of the passed in task'''
tid = task["id"]
start = task["startDateTime"]
end = task["dueDateTime"]
newDates = {"dueDateTime": end,"startDateTime": start}
etag = task["#odata.etag"]
session.headers.update({'If-Match':etag})
response = session.patch(f"https://graph.microsoft.com/v1.0/planner/tasks/{tid}", data = newDates)
session.headers.pop('If-Match')
print(task["title"] + " Has been scheduled")
Based on the documentation I expect this to return a status code of 200, and for the response to contain the data of the task that was updated, and for the change to actually be applied to the task.
By default, PATCH requests return an empty response with 204 return code. To get the data updated data back, you should send "Prefer" HTTP header with value "return=representation".
PATCH https://graph.microsoft.com/v1.0/planner/tasks/{task-id}
Content-type: application/json
Content-length: 247
If-Match: W/"JzEtVGFzayAgQEBAQEBAQEBAQEBAQEBAWCc="
Prefer: return=representation
I have finally figured this out.
#Tarken Sevilmis mentioned that in order to get a 200 response from a PATCH request you need to add
Prefer: return=representation
to your request. In my case the reason that my changes weren't being applied was because I hadn't set the content type in the header. The Graph API didn't give an error, but this seems to have been the cause of the issue. Once I set the content type to application/json it gave a proper error indication that the values I gave in the body weren't being read correctly, and I realized that I forgot to parse them to JSON.
Once you set the content headers appropriately and make sure to convert your data to proper JSON everything should work as intended
I'm not sure when this started happening (I believe fairly recently). This is a breaking change if you rely on the steps documented here:
https://msdn.microsoft.com/en-us/office/office365/howto/sync-calendar-view
The issue is that the the Office 365 & Outlook.com calendarview api no longer seems to return a #odata.nextLink when there is more data to be fetched if you specify "odata.track-changes" for the "Prefer" header in your request.
Here is a CURL request to repro the issue... be sure make the request authenticated as a user with at least 50 events during the time frame specified (to trigger paging).
curl -H "Authorization: Bearer <OMITTED>" -H "Accept: application/json; odata.metadata=none" -H "Prefer: odata.track-changes" "https://outlook.office.com/api/v2.0/me/calendarview?startdatetime=2016-06-16T00:00:00Z&enddatetime=2017-06-23T00:00:00Z"
When I make this request, the resulting response has 10 entries (even though there are at least 50 events) and the response does not have an #odata.nextLink. It does have a #odata.deltaLink however.
Is anyone else experiencing this issue?
As far as I know it has always worked this way. The initial sync returns a deltaLink instead of a nextLink. You have to treat that initial sync request specially and go ahead and issue the next request using the deltaToken.
Initial sync request: The very first sync request sets up the sync state.
Initial sync response:
Check for "Preference-Applied: odata.track-changes" in the response header to confirm a successful sync attempt and the resource supports synchronization.
If the sync attempt was successful, the initial response always contains an #odata.deltaLink with a deltaToken value. If the response contains any data, save the deltaToken value for the second request.
If the initial response wasn't successful, or doesn't return any data indicating there are no events in the specified calendar view, this round of sync ends.
Subsequent sync request: Use the deltaToken or skipToken value from the previous request to issue the next request. See the second and third sync requests as examples.
Subsequent sync response:
If the response returns any data, and, there is more data to sync in that time range, the response would include an #odata.nextLink and a skipToken value. Save the skipToken for the next sync request.
Go back to step 3, follow the nextLink, if any, apply the corresponding skipToken value in the next sync request, and follow any subsequent nextLink, until you have synchronized all the data in the time range for that calendar.
Final sync response: When all events in the calendar view are synchronized, the final response in this round would include an #odata.deltaLink and a deltaToken again. Save the deltaToken value for the next round of synchronization.
I'm trying out http requests to download a pdf file from google docs using google document list API and OAuth 1.0. I'm not using any external api for oauth or google docs.
Following the documentation, I obtained download URL for the pdf which works fine when placed in a browser.
According to documentation I should send a request that looks like this:
GET https://doc-04-20-docs.googleusercontent.com/docs/secure/m7an0emtau/WJm12345/YzI2Y2ExYWVm?h=16655626&e=download&gd=true
However, the download URL has something funny going on with the paremeters, it looks like this:
https://doc-00-00-docs.googleusercontent.com/docs/securesc/5ud8e...tMzQ?h=15287211447292764666&\;e=download&\;gd=true
(in the url '&\;' is actually without '\' but I put it here in the post to avoid escaping it as '&').
So what is the case here; do I have 3 parameters h,e,gd or do I have one parameter h with value 15287211447292764666&ae=download&gd=true, or maybe I have the following 3 param-value pairs: h = 15287211447292764666, amp;e = download, amp;gd = true (which I think is the case and it seems like a bug)?
In order to form a proper http request I need to know exectly what are the parameters names and values, however the download URL I have is confusing. Moreover, if the params names are h,amp;e and amp;gd, is the request containing those params valid for obtaining file content (if not it seems like a bug).
I didn't have problems downloading and uploading documents (msword docs) and my scope for downloading a file is correct.
I experimented with different requests a lot. When I treat the 3 parameters (h,e,gd) separetaly I get Unauthorized 401. If I assume that I have only one parameter - h with value 15287211447292764666&ae=download&gd=true I get 500 Internal Server Error (google api states: 'An unexpected error has occurred in the API.','If the problem persists, please post in the forum.').
If I don't put any paremeters at all or I put 3 parameters -h,amp;e,amp;gd, I get 302 Found. I tried following the redirections sending more requests but I still couldn't get the actual pdf content. I also experimented in OAuth Playground and it seems it's not working as it's supposed to neither. Sending get request in OAuth with the download URL responds with 302 Found instead of responding with the PDF content.
What is going on here? How can I obtain the pdf content in a response? Please help.
I have experimented same issue with oAuth2 (error 401).
Solved by inserting the oAuth2 token in request header and not in URL.
I have replaced &access_token=<token> in the URL by setRequestHeader("Authorization", "Bearer <token>" )
I'm implementing a REST API using ASP.NET MVC, and a little stumbling block has come up in the form of the Expect: 100-continue request header for requests with a post body.
RFC 2616 states that:
Upon receiving a request which
includes an Expect request-header
field with the "100-continue" expectation, an origin server MUST
either respond with 100 (Continue) status and continue to read
from the input stream, or respond with a final status code. The
origin server MUST NOT wait for the request body before sending
the 100 (Continue) response. If it responds with a final status
code, it MAY close the transport connection or it MAY continue
to read and discard the rest of the request. It MUST NOT
perform the requested method if it returns a final status code.
This sounds to me like I need to make two responses to the request, i.e. it needs to immediately send a HTTP 100 Continue response, and then continue reading from the original request stream (i.e. HttpContext.Request.InputStream) without ending the request, and then finally sending the resultant status code (for the sake of argument, lets say it's a 204 No Content result).
So, questions are:
Am I reading the specification right, that I need to make two responses to a request?
How can this be done in ASP.NET MVC?
w.r.t. (2) I have tried using the following code before proceeding to read the input stream...
HttpContext.Response.StatusCode = 100;
HttpContext.Response.Flush();
HttpContext.Response.Clear();
...but when I try to set the final 204 status code I get the error:
System.Web.HttpException: Server cannot set status after HTTP headers have been sent.
The .NET framework by default always sends the expect: 100-continue header for every HTTP 1.1 post. This behavior can be programmatically controlled per request via the System.Net.ServicePoint.Expect100Continue property like so:
HttpWebRequest httpReq = GetHttpWebRequestForPost();
httpReq.ServicePoint.Expect100Continue = false;
It can also be globally controlled programmatically:
System.Net.ServicePointManager.Expect100Continue = false;
...or globally through configuration:
<system.net>
<settings>
<servicePointManager expect100Continue="false"/>
</settings>
</system.net>
Thank you Lance Olson and Phil Haack for this info.
100-continue should be handled by IIS. Is there a reason why you want to do this explicitly?
IIS handles the 100.
That said, no it's not two responses. In HTTP, when the Expect: 100-continue comes in as part of the message headers, the client should be waiting until it receives the response before sending the content.
Because of the way asp.net is architected, you have little control over the output stream. Any data that gets written to the stream is automatically put in a 200 response with chunked encoding whenever you flush, be it that you're in buffered mode or not.
Sadly all this stuff is hidden away in internal methods all over the place, and the result is that if you rely on asp.net, as does MVC, you're pretty much unable to bypass it.
Wait till you try and access the input stream in a non-buffered way. A whole load of pain.
Seb