We have a registered AAD application marked as multi-tenant. We are using this App ID to generate a Token for Microsoft Graph.
The first user is a Global Admin in the Tenant where the app is registered.
The second user is part of another Tenant.
When the second user tried to use Microsoft Graph to get information from OneDrive, we sometimes get an HTTP 429 activityLimitReached error.
We read the guide about throttling and it says to repeat the request after the Retry-After value from the response header. But in our case there is no Retry-After field in the response.
We received this error by executing one request per day. Also, after receiving the 429, we can retry and get a successful result (after several attempts). This error appears only in the OneDrive, the other services are OK.
What can we do to avoid 429 error? How can we check the current limit or increase it?
Example of request
GET https://graph.microsoft.com/v1.0/users/:userId/drives
Example of response
HTTP/1.1 429
Cache-Control: private
Transfer-Encoding: chunked
Content-Type: application/json
request-id: 377d2cdf-7be3-4286-819a-46060330365f
client-request-id: 377d2cdf-7be3-4286-819a-46060330365f
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"West Europe","Slice":"SliceA","Ring":"4","ScaleUnit":"000","Host":"AGSFE_IN_13","ADSiteName":"AMS"}}
Duration: 170.5668
Strict-Transport-Security: max-age=31536000
Date: Wed, 23 May 2018 11:39:08 GMT
{
"error": {
"code": "activityLimitReached",
"message": "The request has been throttled",
"innerError": {
"request-id": "377d2cdf-7be3-4286-819a-46060330365f",
"date": "2018-05-23T11:39:09"
}
}
}
What can we do to avoid 429 error? How can we check the current limit or increase it?
To avoid the 429 error, we must control our request, don't do too many request within limited time. The limit issue is known issue we canot increase it now.
Setting and publishing exact throttling limits sounds very straightforward, but in fact, it's not the best way to go. We continually monitor resource usage on SharePoint Online. Depending on usage, we fine-tune thresholds so users can consume the maximum number of resources without degrading the reliability and performance of SharePoint Online.
Above reference is from MS documentation about throttling and OneDrive for Business/SharePoint: https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online
I would suggest going to UserVoice for Graph and suggest an improvement (or upvote an existing one). The feedback helps Product Group prioritize future work based on the interest in those suggested improvements. But based on the above official docs, the best solution is still to contorl our request but not the feature request.
I do not have much experience with the onedrive-api but I have certainly experienced throttling when using the onenote-api. This post on how onenote throttles may be more useful than the generic microsoft graph link in your question. In particular, long calls against the api will result in throttling occurring much sooner than more targetted calls (and you need to let calls finish before issuing new calls, be very careful with queueing multiple curl requests). Once you have been throttled trying the same call over and over will likely increase the length of time throttling occurs (I haven't seen as full day but I have seen several hours before).
I presume (but am not sure) that all microsoftgraph-api calls will internally count towards resource limits, but if you direct onedrive calls against the onedrive-api this will not count against the microsoft-graph limits and may allow the requests you need without throttling.
I would definitely recommend getting a second access token for the direct onedrive api (you can use the same refresh token) and try this approach.
GET https://www.onedrive.com/v1.0/users/:userId/drives
If it is one user in particular that has issues maybe they have exceeded their tenant resources?
Related
I'm starting to use Microsoft Graph threat assessment API to report Phishing Website URL.
(Ref: https://learn.microsoft.com/en-us/graph/api/informationprotection-post-threatassessmentrequests?view=graph-rest-1.0&tabs=http)
My use-case is automatic reporting and manual reporting via Slack Command.
But throttling is very strict, so I get "429" response immediately.
"code": "ActivityLimitReached",
"message": "The client application has been throttled for reaching an activity limit. The request may be repeated after a delay, the length of which may be specified in a 'Retry-After' header.",
Does anyone know a workaround for the throttling?
As far as I confirmed, throttling is 1 request per 15 minutes (Limit per resource) by default.
(150 requests per 15 minutes (Limit per tenant) though)
Ref: https://learn.microsoft.com/en-us/graph/throttling#information-protection
I would try the following best practice to avoid/handle the throttling.
When you implement error handling, use the HTTP error code 429 to detect throttling. The failed response includes the Retry-After response header. Backing off requests using the Retry-After delay is the fastest way to recover from throttling because Microsoft Graph continues to log resource usage while a client is being throttled.
Wait the number of seconds specified in the Retry-After header.
Retry the request.
If the request fails again with a 429 error code, you are still being throttled. Continue to use the recommended Retry-After delay and retry the request until it succeeds.
The guidelines for throttling is already provided by Microsoft Graph team and it's straightforward. Please go through this doc, look out for best practices to avoid throttling, to handle throttling etc and think about throttling/batching to see if it suits your scenario (so you can optimize the calls).
If retry-after header doesnt exists then it would be tough - thats the way to handle throttling, unless if any alternate way exists provided. If you still believe Microsoft to implement this feature, consider creating a new user voice.
Update: #rung created a new uservoice on this.
I am working on the similar use case. I am planning to submit the Msg Id/URL/file to Microsoft for phishing assessment using API . I am stuck with an error that shows " "code": "Unauthorized",
"message": "Required authentication information is either missing or not valid for the resource.",
"
I would highly appreciate your help!
I did a Twitter clone using rails api + react, just for study purposes.
I have quite simple logic of requests: click in a user, load its informations and tweets, requesting for the api. However, If I do this fast like 3 times, I receive the status 429 (too many requests) with the header Retry-After: 5.
There is a way to increase the number of requests in a given time? How would be the correct approach to handle with this in such common situation?
From my understanding, the error information you have shown is correct, It means request cannot be served due to the application's rate limit having been peaked for the resource.
Rate limits are divided into 15 minute intervals. All endpoints
require authentication, so there is no concept of unauthenticated
calls and rate limits.
To overcome this situation, here is an example from the documentation itself.
I am currently working on an application that makes requests using the .NET SDK for the Microsoft Graph API. Specifically to retrieve information about users and their OneDrives.
Microsoft throttles API requests by returning an HTTP 429 status code, and I have implemented a back-off using the Retry-After header. I have noticed however that I seem to be getting throttled after only a handful of requests.
I have also been using Microsoft Graph Explorer to test some of my API calls and have noticed that I never seem to get a 429 response when accessing the API via that method. After also seeing reports of people having issues with the OneDrive client on Linux that they managed to work around by changing their User-Agent header, I thought maybe I needed to set a User-Agent for my requests.
The result is that it seems that if I set the User-Agent header to be something like Mozilla/5.0 then all the throttling issues seem to disappear. I have searched high and low and so far haven't managed to find any documentation on what a valid User-Agent should be, and I would prefer to avoid making my app impersonate a browser, so I wondered whether there was any guidance or documentation around that I might have missed?
For instance, a User-Agent of Mozilla/5.0 seems to result in no throttling, but MyApp/1.0 results in throttling.
There isn't any guidance on the User-Agent header and to be honest, I'm not sure why this would have any effect on throttling.
Throttling in Microsoft Graph is handled by the underlying service you're interacting with. For example, the /notes/ endpoint throttling is governed by OneNote while /messages is governed by Exchange.
In most cases, OneDrive will throttle by number of concurrent requests, per app, per user. So using Delegated permissions, your app should generally be able to concurrently upload 4 files without a problem. Any more than that and you will start to see 429 responses.
there are some best-practices from Microsoft to handle throttling:
https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online#how-to-decorate-your-http-traffic-to-avoid-throttling
you should decorate your http user-agent as following:
NONISV|CompanyName|AppName/Version
Identify as NONISV and include Company Name, App Name separated by a pipe character and then adding Version number separated with a slash character
or
ISV|CompanyName|AppName/Version
Identify as ISV and include Company Name, App Name separated by a pipe character and then adding Version number separated with a slash character
I'm currently running a web application that uses Microsoft Graph's API and we encountered the following message today which severely impacted our application, for a whole day:
"error": {
"code": "ErrorTooManyObjectsOpened",
"message": "Too many concurrent connections opened., The process failed to get the correct properties.",
"innerError": {
"request-id": "removed",
"date": "2017-12-13T17:01:14"
}
}
please note that the request-id was removed
Let me summarize what our web application does.
Basically, we have 2 email folders that we are actively subscribed to, Junk and Folder A.
If anything hits Folder A, we strip the body of the email message and then move the message to Folder B. The subscription on our Junk folder also strips the body and sends them over to Folder B.
Sometimes the webhook subscription service skips messages that may come at the same time, therefore we have 2 cron jobs in our server that run a script and check Junk/Folder A for any messages every 5 minutes, therefore my assumption is that the cron job runs about 288*2 times per day. Not counting our subscription to the folders, we usually get around 200-300 email messages per day.
Unfortunately Microsoft's Graph error codes page does not provide us with any explanation about this code. I would really appreciate if anyone can explain what this means and how to avoid it from happening.
This is occurring because your application is exceeding the throttling thresholds.
There are several different throttling metrics that can affect Microsoft Graph requests. For a high-level overview, see the Microsoft Graph throttling guidance. Since in this case you're hitting Exchange Online via Graph, you can find more specific information from What throttling values do I need to take into consideration? in the Exchange documentation.
Architecturally, you are making a lot of unnecessary calls into the API. Rather than having both a subscription and a scheduled job, you should use just the webhook subscription and the /delta endpoint.
Each call to the /delta endpoint gives you a token that can be used to fetch any changes to a given resource since the token was originally issued. So regardless of if 1 email came in or 1,000, you only get the new emails.
Once you're using the /delta to find your changes, you then use a webhook only as a "trigger". When you receive the webhook, you can ignore the contents and instead issue a request to /delta. This ensures that you capture every incoming email even if you didn't necessarily receive separate webhook notifications.
There is a bug. After making 500 message move requests, a "cannot copy/move error" occurs. Subsequently, a "429: Too many concurrent connections opened" error occurs. Most applications miss the first error because you continually get the 429 error afterwards.
If you let the application "rest" for 30 minutes, the throttle resets itself and you can continue on. I do not think there is a time limit for hitting the 500 moves. My application did 500 moves after 6.5 hours and then we started getting the error.
And, if you keep trying your move call before the 30 min rest period, it never resets. Also, in the response, the retry-after is null... so, that doesn't help you.
If you find a work around, please let me know. We are trying a few things like setting the category, then manually moving the messages. I am also investigating making a rule the moves them for us or some other job. I cannot find a way to execute a rule from the Graph API.
See this link for more information. Also, the more people who report having this issue, hopefully the sooner it can be resolved. Outlook API Throttling documentation #144
I am using Microsoft Graph for OneNote. I have observed few issues with these endpoints. Sometimes when API's are called concurrently a throttling exception is thrown (HTTP 429). And suddenly next API call returns successful response.
As per documentation, a Retry-After field will be returned in the response header along with HTTP 429 when throttling is active but I have never received this field in the response header.
Is there any other way to get the wait time?
How long I should wait to further avoid this issue? This is a blocker issue for me. Kindly help.
The documentation should be more clearer and note that not all endpoints return a Retry-After. This is because not all endpoints throttle using the same rules.
Some endpoints, such as Exchange, throttle based on a formula that looks at the number of requests within a window time. These endpoints respond with Retry-After to alert you how long you have to wait before you have a new window.
Other endpoints simply throttle based on the number concurrent requests. OneNote is one of these endpoints. Rather than looking at requests over time, it simply limits you to a maximum of 5 concurrent calls. Since throttling isn't dependent on time, it would be impossible to provide a Retry-After value.
This is also why you so often see your initial retry succeed, it means some previous call had completed and there were now 5 or less concurrent requests.
https://blogs.msdn.microsoft.com/onenotedev/2016/01/12/onenote-api-throttling-and-how-to-avoid-it/
This blog post contains more information about how to avoid throttling from OneNote endpoints (and how it works).