Need guidance on use of Microsoft Graph SDK's WithShouldRetry() delegate - microsoft-graph-api

i am trying to overcome intermittent 409 error that occurs while uploading/updating metadata of a file in SharePoint's Document library using Microsoft Graph SDK. To retry failed calls SDK provides WithMaxRetry() and WithShouldRetry() options. The MaxRetry works for error codes 429, and i am assuming that ShouldRetry delegate offers us an option to implement our own logic for the retries. Based on this assumption, i have the below code:
_graphServiceClientFactory.GetClient().Drives[driveId].Root.ItemWithPath(path).ListItem.Fields.Request()
.WithShouldRetry((delay, attempt, httpResponse) =>
(attempt <= 5 &&
(httpResponse.StatusCode == HttpStatusCode.Conflict)))
.UpdateAsync(new FieldValueSet { AdditionalData = dataDictionary });
In my test the ShouldRetry delegate was never called/evaluated on failures/otherwise, there is no documentation on the usage of WithShouldRetry(). It would be helpful to get inputs on usage of WithShouldRetry() option.

It appears that the WithShouldRetry() is faulty, i had reported this issue in GitHub (Microsft Graph SDK repo), and they have marked the issue as Bug.
As a workaround one could use Polly for retry as shown below,
var result = await Policy.Handle<ServiceException>(ex =>
ex.StatusCode == HttpStatusCode.Conflict ||
ex.StatusCode == HttpStatusCode.Locked ||
ex.StatusCode == HttpStatusCode.ServiceUnavailable ||
ex.StatusCode == HttpStatusCode.GatewayTimeout ||
ex.StatusCode == HttpStatusCode.TooManyRequests)
.Or<HttpRequestException>()
.WaitAndRetryAsync(3, retryCount => TimeSpan.FromSeconds(Math.Pow(2, retryCount) / 2))
.ExecuteAsync<FieldValueSet>(async () =>
await GetDriveItemWithPath(itemPath, driveId).ListItem.Fields.Request()
.WithMaxRetry(0)
.UpdateAsync(new FieldValueSet { AdditionalData = dataDictionary }));
By default Graph SDK does 3 retries for throttled and gateway timeout errors. In the above code those native retries have been blocked by calling WithMaxRetry(0). The internal retry logic of Graph SDK is made part of the Polly implementation.
Note: This Polly implementation should be a temporary solution, i believe it is best to utilize the WithShouldRetry() once the reported bug is resolved.

Related

Online docs for getTwilioClient() as used in "Functions and Assets"

I have found a few examples of context.getTwilioClient(), but I have not been able to locate any online documentation. It's probably right under my nose, but it is eluding me. In particular, I'm trying to get information about workers in a Task Router queue (i.e., how many workers are in the different statuses and how many workers are available), but having the documentation will help with future projects.
I found some documentation saying that the context.getTwilioClient(), "enables you to invoke the getTwilioClient method on the context object for a fully-initialized Twilio REST API client." (https://support.twilio.com/hc/en-us/articles/115007737928-Getting-Started-with-Twilio-Functions)
It then shows this example, but there is no implementation of "messages" when I attempt to run this code:
var client = context.getTwilioClient();
client.messages.create({
to: '+12025551212',
from: '+12065551212',
body: "hello world!"})
Thanks.
The messages property should be on the client. getTwilioClient returns the Twilio helper library for Node.js.
I just created a Function with your code, and it worked as expected, meaning that I got the SMS, however, the function did time out because the callback was never invoked. To end the function invocation and respond to the caller, make sure you always invoke the callback function, like this:
exports.handler = function(context, event, callback) {
var client = context.getTwilioClient();
client.messages.create({
to: '+1xxxxxxxxxx',
from: '+1xxxxxxxxxxx',
body: "hello world!"})
.then((message) => {
console.log('SMS successfully sent');
console.log(message.sid);
// Make sure to only call `callback` once everything is finished, and to pass
// null as the first parameter to signal successful execution.
return callback(null, `Success! Message SID: ${message.sid}`);
})
.catch((error) => {
console.error(error);
return callback(error);
});
};
You can learn more about the callback function here.
If you still encounter this issue, can you tell use what Node Version you're using and which module dependencies and their versions?
You can find these details at Settings & More > Dependencies in your Twilio Functions Service.

1-3 'SyntaxError: Unexpected end of JSON input' errors occurring per minute when streaming tweets in Node

Using express and Node.js, I'm using the twitter streaming API and the needle npm package for accessing APIs to pull tweets related to keywords. The streaming is functional and I am successfully pulling tweets using the following (simplified) code:
const needle = require('needle');
const TOKEN = // My Token
const streamURL = 'https://api.twitter.com/2/tweets/search/stream';
function streamTweets() {
const stream = needle.get(streamURL, {
headers: {
Authorization: `Bearer ${TOKEN}`
}
});
stream.on('data', (data) => {
try {
const json = JSON.parse(data); // This line appears to be causing my error
const text = json.data.text;
} catch (error) {
console.log("error");
}
});
}
However, no matter which search term I use (and the subsequent large or small volume of tweets coming through), the catch block will consistently log 1-3 errors per minute, which look like this:
SyntaxError: Unexpected end of JSON input
at JSON.parse (<anonymous>)
at PassThrough.<anonymous> (C:\Users\danie\OneDrive\Documents\Personal-Projects\twitter-program\server.js:56:31)
at PassThrough.emit (events.js:315:20)
at addChunk (internal/streams/readable.js:309:12)
at readableAddChunk (internal/streams/readable.js:284:9)
at PassThrough.Readable.push (internal/streams/readable.js:223:10)
at PassThrough.Transform.push (internal/streams/transform.js:166:32)
at PassThrough.afterTransform (internal/streams/transform.js:101:10)
at PassThrough._transform (internal/streams/passthrough.js:46:3)
at PassThrough.Transform._read (internal/streams/transform.js:205:10).
I've seen previous advice which says that data can be fired in multiple chunks, and to push the chunks to an array i.e. something like the following:
let chunks = [];
stream.on('data', (dataChunk) => {
chunks.push(dataChunk);
}).on('end',() => {
// combine chunks to create JSON object
})
But this didn't work either (may have been my implementation but I don't think so) and now I'm wondering if it's perhaps an error with the twitter API, because most of the tweet objects do come through correctly. I should note that the streamTweets() function above is called from an async function, and I am also wondering if that is having something to do with it.
Has anyone else encountered this error? Or does anyone have any idea how I might be fix it? Ideally i'd like 100% of the tweets to stream correctly.
Thanks in advance!
For future readers, this error is triggered by Twitter's heartbeat message that is sent every 20 seconds. Per the documentation:
The endpoint provides a 20-second keep alive heartbeat (it will look like a new line character).
Adding a guard against parsing the empty string will prevent the JSON parsing error.
if (data === "")
return
An empty string is invalid JSON, hence the emitted error.
Now, acknowledging that the heartbeat exists, it may be beneficial to add read_timeout = 20 * 1000 in the needle request to avoiding a stalled program with no data, be that due to a local network outage or DNS miss, etc.

Call Graph API from SharePoint

I need to call Graph API from spfx webpart.
Previously we used the following method:
import { MSGraphClient } from '#microsoft/sp-client-preview';
But later we got to know that MSGraphClient is depreciated now in sp-client-preview.
I checked the following method which is mentioned in Microsoft docs also.
import { MSGraphClient } from '#microsoft/sp-http';
But it is giving an error as following:
Module '"d:/O365/upload-onedrive/node_modules/#microsoft/sp-http/dist/index-internal"' has no exported member 'MSGraphClient'
SPFx version we are using now is 1.6
Is there any way call Graph API from spfx now?
Of course we can use Graph in SPFx.
Graph+adal+SPFx steps:
Create an application in Azure portal. Click the manifest, then change "oauth2AllowImplicitFlow" value to true
Go to Settings->Required Permissions->ADD->Select an API->Microsoft Graph, select the permission and then Grant Permissions.
Build HelloWorld SPFx project : https://learn.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/get-started/build-a-hello-world-web-part
Add and IAdalConfig.ts and WebPartAuthenticationContext.js patch files
Tips: If you have no adal module in node_modules/#types folder, you'd better manually install the module using the command : npm install #types/adal#1.0.29
Add the following code to render()
// Make an AJAX request to the Graph API and print the response as JSON.
var getToken;
var getCurrentUser = function (access_token) {
var xhr = new XMLHttpRequest();
xhr.open('GET', 'https://graph.microsoft.com/v1.0/me', true);
xhr.setRequestHeader('Authorization', 'Bearer ' + access_token);
xhr.onreadystatechange = function () {
if (xhr.readyState === 4 && xhr.status === 200) {
// Do something with the response
getToken=JSON.stringify(JSON.parse(xhr.responseText), null, ' ');
console.log('get Graph APi information=='+getToken);
} else {
// TODO: Do something with the error (or non-200 responses)
// console.log(' error');
}
};
xhr.send();
There is actually no reason to create any applications in the Azure side, it's all automatic and taken care of by SharePoint. See following documentation for details. We did change the API structure slightly between preview and GA, but the basics have remained the same with MSGraphClient usage and no reason for any manual access token handling.
https://learn.microsoft.com/en-us/sharepoint/dev/spfx/use-msgraph

Pull pagespeed in seconds from Chrome User Experience Report into a google sheet

By following the guide Create Your Own Google Pagespeed & Mobile Usability Tracking Google Sheet in 5 Steps I managed to set up mobile pagespeed score for a list of (up to 50) URLs.
However since late 2017 or something there is real data available from the Chrome User Experience Report that displays an average load time in seconds for a page based on chrome user data.
(This data is being used for example when using Pagespeed Insights by google.)
Instead of pulling a page score I as described above I would like to pull the average load time into my google sheet.
Is it possible to adapt the script used from the article above to pull load time in seconds instead of pagescore? Or is there any other way to do this?
Thanks in advance your help is much appreciated.
This is the script I run in script editor to get pagescore into google sheet according to the linked article with function =checkAll(C3):
/**
* Returns Mobile Pagespeed, Mobile Usability, and Desktop Pagespeed values in three adjacent columns
* by Cagri Sarigoz
*/
function checkAll(Url) {
//CHANGE YOUR API KEY WITH YOUR_API_KEY BELOW
var key = "AIzaSyB2SeOumbCd6YNfFWRg5Jo_WpISZi4gCFs";
var serviceUrlMobile = "https://www.googleapis.com/pagespeedonline/v2/runPagespeed?url="+Url+"&strategy=mobile&key="+key;
var serviceUrlDesktop = "https://www.googleapis.com/pagespeedonline/v2/runPagespeed?url="+Url+"&strategy=desktop&key="+key;
var array = [];
if (key == "YOUR_API_KEY")
return "Please enter your API key to the script";
var responseMobile = UrlFetchApp.fetch(serviceUrlMobile);
if(responseMobile.getResponseCode() == 200) {
var contentMobile = JSON.parse(responseMobile.getContentText());
if ( (contentMobile != null) && (contentMobile["ruleGroups"] != null) )
{
if (contentMobile["responseCode"] == 200)
{
var speedScoreMobile = contentMobile["ruleGroups"]["SPEED"]["score"];
var usabilityScoreMobile = contentMobile["ruleGroups"]["USABILITY"]["score"];
}
else
{
array.push(["Not Found!", "Not Found!", "Not Found!"]);
return array;
}
}
}
var responseDesktop = UrlFetchApp.fetch(serviceUrlDesktop);
if(responseDesktop.getResponseCode() == 200) {
var contentDesktop = JSON.parse(responseDesktop.getContentText());
if ( (contentDesktop != null) && (contentDesktop["ruleGroups"] != null) )
var speedScoreDesktop = contentDesktop["ruleGroups"]["SPEED"]["score"];
}
array.push([speedScoreMobile, usabilityScoreMobile, speedScoreDesktop]);
return array;
}
I am the writer of the blog post that you shared. As you said, the Google Apps Script there was using Google Pagespeed API v2. The current API version is v4, and v2 will be depreciated on June 30th.
So I updated the code with v4 on my own copy of the spreadsheet. You can make your own copy from here.
I also wanted to add the mobile-friendly test results but it turned out that Google Search Console's API quota restrictions were too tight, returning error almost all the time. So I commented out that part of the code for the time being.
I didn't have the time to update my blog post yet. You can see the new version of the script here.

OWIN middleware for OpenID Connect - Code flow ( Flow type - AuthorizationCode) documentation?

In my implementation I am using OpenID-Connect Server (Identity Server v3+) to authenticate Asp.net MVC 5 app (with AngularJS front-end)
I am planning to use OID Code flow (with Scope Open_ID) to authenticate the client (RP). For the OpenID connect middle-ware, I am using OWIN (Katana Project) components.
Before the implementation, I want to understand back-channel token request, refresh token request process, etc using OWIN.. But I am unable to find any documentation for this type of implementation (most of the available examples use Implicit flow).
I could find samples for generic Code flow implementation for ID Server v3 here https://github.com/IdentityServer/IdentityServer3.Samples/tree/master/source
I am looking for a similar one using OWIN middleware ? Does anyone have any pointers ?
Edit: good news, code flow and response_mode=query support was finally added to Katana, as part of the 4.1 release (that shipped in November 2019): https://github.com/aspnet/AspNetKatana/wiki/Roadmap#410-release-november-2019.
The OpenID Connect middleware doesn't support the code flow: http://katanaproject.codeplex.com/workitem/247 (it's already fixed in the ASP.NET 5 version, though).
Actually, only the implicit flow (id_token) is officially supported, and you have to use the response_mode=form_post extension. Trying to use the authorization code flow will simply result in an exception being thrown during the callback, because it won't be able to extract the (missing) id_token from the authentication response.
Though not directly supported, you can also use the hybrid flow (code + id_token (+ token)), but it's up to you to implement the token request part. You can see https://github.com/aspnet-contrib/AspNet.Security.OpenIdConnect.Server/blob/dev/samples/Nancy/Nancy.Client/Startup.cs#L82-L115 for an example.
The answer and comment replies by Pinpoint are spot on. Thanks!
But if you are willing to step away from the NuGet package and instead run modified source code for Microsoft.Owin.Security.OpenIdConnect you can get code (code) flow with form_post.
Of course this can be said for all open source project problems but this was an quick solution for a big thing in my case so I thought I'd share that it could be an option.
I downloaded code from https://github.com/aspnet/AspNetKatana, added the csproj to my solution and removed lines from https://github.com/aspnet/AspNetKatana/blob/dev/src/Microsoft.Owin.Security.OpenIdConnect/OpenidConnectAuthenticationHandler.cs in AuthenticateCoreAsync().
You must then combine it with backchannel calls and then create your own new ClaimsIdentity() to set as the notification.AuthenticationTicket.
// Install-Package IdentityModel to handle the backchannel calls in a nicer fashion
AuthorizationCodeReceived = async notification =>
{
var configuration = await notification.Options.ConfigurationManager
.GetConfigurationAsync(notification.Request.CallCancelled);
var tokenClient = new TokenClient(configuration.TokenEndpoint,
notification.Options.ClientId, notification.Options.ClientSecret,
AuthenticationStyle.PostValues);
var tokenResponse = await tokenClient.RequestAuthorizationCodeAsync(
notification.ProtocolMessage.Code,
"http://localhost:53004/signin-oidc",
cancellationToken: notification.Request.CallCancelled);
if (tokenResponse.IsError
|| string.IsNullOrWhiteSpace(tokenResponse.AccessToken)
|| string.IsNullOrWhiteSpace(tokenResponse.RefreshToken))
{
notification.HandleResponse();
notification.Response.Write("Error retrieving tokens.");
return;
}
var userInfoClient = new UserInfoClient(configuration.UserInfoEndpoint);
var userInfoResponse = await userInfoClient.GetAsync(tokenResponse.AccessToken);
if (userInfoResponse.IsError)
{
notification.HandleResponse();
notification.Response.Write("Error retrieving user info.");
return;
}
..

Resources