As there is no available feature in MS Graph API to have access to MS Form responses (LINK), what's the code solution to have access to shared online MS Form responses?
I know the following available approaches:
Manually download the responses by clicking on Responses -> Open in Excel and then upload it to the target location (It's a completely manual process and could not be automated by a code script which is not desired at all in my case)
Share with a group first which gives me the option to have online access to the response file by clicking on Responses -> Open in Excel. This time it automatically creates an excel file in the group's OneDrive instead of downloading the file. Now I could read the excel file via MS Graph API. (This solution works but I need to share the Form with a group first and then manually click Responses -> Open in Excel to create the excel file)
Any thought that assists to have a complete code solution is appreciated
I would stick with the above approaches which you said above. In addition, you can consider using Microsoft Flow or Power automate to perform the above to automate the above approach. AFAIK i failed to see any Graph API exposed on this so far. Being said that i would suggest you can consider filing Microsoft Uservoice - so that they can consider implementing it. You upvoted the the uservoice as well.
I came up with this for what I needed recently. The documentation seems to be deliberately vague, so this may not work in the future. I used unofficial and official info from various sources on stackoverflow and MS documentation. There is an (undocumented) API to pull the responses but it requires an access token. In python you can use Microsoft's azure-identity package to get the token, then use it to request the form responses. Here's the outline:
the scope of authorization for an access token for MS Forms is: https://forms.office.com/.default
the endpoint for fetching the responses is: https://forms.office.com/formapi/api/{tenantid}/users/{userid}/light/forms('{formid}')/responses
where {tenantid} is the Azure Tenant ID, {userid} is the Azure user's Object ID, and {formid} is the id of the form (i.e. the id query parameter at the end of the form's URL). This URL requires an access token that entitles the requester to get the data.
use one of the Credential object types in azure.identity package and call its get_token(scope) method with the scope specified above to receive the token. Depending on which type you choose, you may need to login to Azure using az login, or a browser login, or use the current logged-in user's credentials.
Use the returned token as the Bearer token to authorize access to response endpoint (item 2 above)
Like this:
import os
import requests
from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential, AzureCliCredential
# Use one of these credential objects to get an access token
cred = AzureCliCredential() # i.e. `az login`
# cred = InteractiveBrowserCredential()
# cred = DefaultAzureCredential()
# Request an access token with the following scope
scope = "https://forms.office.com/.default"
tok = cred.get_token(scope)
# print(tok)
# print(type(tok))
formid = os.getenv("FORMS_FORM_ID")
tenantid = os.getenv("FORMS_TENANT_ID")
userid = os.getenv("FORMS_ADMIN_ID")
url = f"https://forms.office.com/formapi/api/{tenantid}/users/{userid}/light/forms('{formid}')/responses?$expand=comments&$top=7&$skip=0"
# print(url)
# Provide the access token in the request header
headers = {"Authorization": f"Bearer {tok.token}"}
r = requests.get(url, headers=headers)
print(r.json())
We need to read out distribution lists from a contact folder of a dedicated exchange/outlook mailbox (O365). The process must run as a service with no user interaction.
Unfortunately the Graph API does not support distribution lists (not even the Graph beta version does). Because of this we have to use another API - I tried using EWS.
I succeeded by granting full_access_as_app permission to our service. However this allows to read and modify ANY data in ANY mailbox which is a security risk. Granting this permission only to read out some distribution lists from one mailbox is not acceptable.
So I tried to use the ROPC flow that should allow authenticating a user and then accessing the mailbox with the permissions of this user. I followed the information here: How to get OAuth2 access token for EWS managed API in service/daemon application
(Btw I found this post linked in the discussion here: https://github.com/microsoftgraph/microsoft-graph-docs/issues/5659 which has some more information about the topic.)
I exactly followed the steps mentioned above but unfortunately this is not working: I always get a “401 Unauthorized” exception when doing the EWS calls (OAuth calls succeed) and no additional information.
According to https://developer.microsoft.com/en-us/graph/blogs/upcoming-changes-to-exchange-web-services-ews-api-for-office-365/ this is no longer working. So how can I read out distribution lists from a specific mailbox without giving full access and without an interactive login?
EDIT
Here as requested the full code:
string[] ewsScopes = { "https://outlook-tdf-2.office.com/EWS.AccessAsUser.All" };
IPublicClientApplication clientApplication = PublicClientApplicationBuilder.Create(appId).WithAuthority(AzureCloudInstance.AzurePublic, tenantId).Build();
NetworkCredential credentials = new NetworkCredential(appUsername, appPassword);
AuthenticationResult authResult = await clientApplication.AcquireTokenByUsernamePassword(ewsScopes, credentials.UserName, credentials.SecurePassword).ExecuteAsync().ConfigureAwait(false);
ExchangeService exchangeService = new ExchangeService
{
Url = new Uri("https://outlook.office365.com/EWS/Exchange.asmx"),
Credentials = new OAuthCredentials(authResult.AccessToken),
};
ItemView view = new ItemView(int.MaxValue)
{
PropertySet = new PropertySet(ItemSchema.Id),
};
SearchFilter.IsEqualTo filter = new SearchFilter.IsEqualTo(ItemSchema.ItemClass, "IPM.Contact");
FindItemsResults<Item> ewsResult = await exchangeService.FindItems(WellKnownFolderName.Contacts, filter, view).ConfigureAwait(false);
I have also tried with other scopes such as "https://outlook.office.com/EWS.AccessAsUser.All" or "https://outlook.office365.com/EWS.AccessAsUser.All" but without success. I feel the problem might be related to the scope? I can see that the Exchange legacy API that was listed in the Azure UI when adding permissions is now gone...?
The Scope in your code is wrong (I'm not sure where you got that from) it should be
string[] ewsScopes = { "https://outlook.office.com/EWS.AccessAsUser.All" };
Using your code with the scope you had gives a 401 error and if you look at the response headers of the EWS response it actually tells you that the scope is the issue eg
2000003;reason="The audience claim value is invalid for current resource. Audience claim is 'https://outlook-tdf-2.office.com/', request url is 'https://outlook.office365.com/EWS/Exchange.asmx' and resource type is 'Exchange'.";error_category="invalid_resource"
Using your code with the correct scope works fine
But then why is the option to add the Exchange Legacy permission (step 6 in the MS docu mentioned above) gone from the Azure UI?
Its hasn't been removed from the AzureUI they have just moved all the Exchange legacy permissions (including the ones used in the Exchange Admin cmdlets which isn't really a legacy API) under the Graph Permissions. Why they did this way and didn't communicate it well (I'll only just saw it today as well) I'm not sure.
Yesterday Nov 19th Microsoft has updated the documentation: https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-authenticate-an-ews-application-by-using-oauth
Following the new documentation it works (again). The main difference is to use the shortened scope "EWS.AccessAsUser.All" and not any of the full scopes found in many examples and posts such as “https://outlook.office.com/EWS.AccessAsUser.All”, “https://outlook.office365.com/EWS.AccessAsUser.All” etc.
Thank you MS for wasting my time.
Distribution groups are only exposed in Exchange PowerShell today, and are not currently supported via the Microsoft Graph API.
Please vote on this feature request on UserVoice:
In our application we have following setup:
Web Application (Angular JS) + Web API
We support external logins Google & Facebook.
We have setup necessary infrastructure with default scopes and wired-up during the Startup. This works fine for login.
For additional features, say user wants to import his/her contacts from Google, we need to get consent again with new scopes. Can someone let me know how to do this?
One way is - to include all necessary scopes during login phase but we want to get consents only for required stuff at required time.
I googled but couldn't get any information on this.
In our specific case of importing contacts from Google a/c, I thought of specifically creating a new controller and start fresh authentication mechanism using GoogleAuthorizationCodeFlow (without OWIN middleware) as explained here https://developers.google.com/api-client-library/dotnet/guide/aaa_oauth but it is in the context of MVC but we need support for Web API. So any pointer is highly appreciated.
Finally got the solution.
Below links helped a lot; my solution is a combination of ideas from below links:
http://www.yogihosting.com/implementing-google-contacts-api-version-3-0-oauth-2-0-in-csharp-and-asp-net/
(note: Google OAuth links mentioned in this article, especially for token generation, have changed. For correct URLs, please see https://developers.google.com/identity/protocols/OAuth2WebServer)
https://www.themarketingtechnologist.co/google-oauth-2-enable-your-application-to-access-data-from-a-google-user/
Above links explain how to get access_token and authorization_code. Once we have those values, we can create UserCredentials that can be used for accessing specific Google APIs as mentioned in the below link:
Upload video to youtube with mvc application (all code behind)
Things to remember:
One ends up contacting Google authorization server twice (once for access token and once for authorization code). In both the cases, redirect_uri must be specified and IT IS IMPORTANT for that redirect_uri to be SAME in both the cases; otherwise it won't work. Also, it is important for that redirect_uri to be mentioned in the Google console where the project/application is defined and registered. I faced issues because of mismatched redirect_uri values and below link helped me in solving it:
Google API token endpoint POST returns Bad Request 400
While accessing Google API we need to include filters. In my case for People API, I needed to get email addresses and without required filters, it was always null/empty. So, below is the code for that:
var peopleService = new PeopleService(new BaseClientService.Initializer()
{
HttpClientInitializer = credentials,
ApplicationName = "MyApp",
});
var connList = peopleService.People.Me.Connections.List();
connList.RequestMaskIncludeField = "person.emailAddresses";
connList.PageSize = 500;
ListConnectionsResponse connectionsResponse = connList.Execute();
IList<Person> connections = connectionsResponse.Connections;
Without connList.RequestMaskIncludeField = "person.emailAddresses"; email address is always empty.
Hope this helps someone.
I'm responsible for the API side of our product. We have several different clients, from browsers to iPads to Chromebooks. Right now, all our authentication is done directly from the client to our API, with username & password.
I've inherited some code that does authentication using OAuth, with the usual username/password setup. So inside my OwinAuthConfig class, I have:
var oAuthAuthorizationOptions = new OAuthAuthorizationServerOptions
{
TokenEndpointPath = new PathString("/Authenticate"),
Provider = new MyAuthorizationProvider(),
AccessTokenExpireTimeSpan = TimeSpan.FromDays(14),
AllowInsecureHttp = true
};
app.UseOAuthAuthorizationServer(oAuthAuthorizationOptions);
Then, through some dark magic, this connects up with my MyAuthorizationProvider class (which inherits OAuthAuthorizationServerProvider), and on login, this invokes the method:
public override Task GrantResourceOwnerCredentials(OAuthGrantResourceOwnerCredentialsContext context)
{ ... }
where context contains the important stuff (Username and Password) which I can then use to authenticate the user, build his claims, create an AuthenticationTicket and this information then magically gets returned to the client with the access token etc.
All well and good.
Now I have a new requirement - to allow 3rd party authentication from Google. In this case, the client app (iOS/Android/whatever) does the authentication with Google, and they should just pass the token (and any other required info) to me on the API side. On my side I then need to re-authenticate the Google token, and get all the user info from Google (email, name, etc.), from which I should then again link that to our User table, build up the claims etc. and return a new token to the client, which will be used in all subsequent calls.
Being kinda new to the whole OWIN pipeline thing, I'm not sure exactly how to go about this. I could write a new GoogleAuthController, that just acts like any other controller, and have an API that accepts the Google token, and returns the new token and other info in the same format that the username/password authentication API does it. But 2 things are nagging at me:
I have this awkward feeling like this is the noobie way of doing things, reinventing the wheel, and really there's a super-cool magical way of hooking things together that I should rather be using; and
In MyAuthorizationProvider.GrantResourceOwnerCredentials(), I've got access to an OAuthGrantResourceOwnerCredentialsContext object, which allows me to validate my new AuthenticationTicket. If I'm doing this inside a plain vanilla controller, I have no idea how I would mark that ticket as validated.
Any clues, please?
EDIT I've seen the Google auth flow as described here. I'm still confused by how best to manage the process from the API side. The client will be obtaining the authorization code, and then calling the API with that auth code. I get that then I've got to take that auth code and convert it to a token by calling the Google API. (Or maybe that should be the client's responsibility?) Either way, I then need to use that token to go back to the Google API and get the user's name, email and avatar image, then I need to match up that email with my own database to identify the user and build up their claims. Then I need to return a new token that the client can use to connect to me going forward.
Let me be more specific about my questions, before my question is closed as "too broad":
When the client has completed authentication with the Google API, it gets back a "code". That code still needs to be converted into a token. Whose responsibility should that be - the client or the API? (I'm leaning towards making it the client's responsibility, if just for the reason of distributing the workload better.)
Whether the client is passing through a code or a token, I need to be able to receive it in the API. Should I just use a plain vanilla Controller to receive it, with an endpoint returning an object of type AuthenticationProperties, or is there some special OWIN way of doing this?
If I'm using a plain vanilla Controller, how do I validate my token? In other words, how do I get access to the OWIN context so that I can mark the AuthenticationTicket as validated?
How do I write an automated test that simulates the client side of the process? AFAICT, the authentication wants to have a user physically click on the "Allow" button to grant my app access to their identity stuff, before it will generate the auth code. In an automated test, I would want to pass username/password etc. all from code. How do you do that?
So I found a solution of my own. It's only slightly kludgy, doesn't require referencing any Google OWIN libraries, and best of all, reuses the code from my username/password authentication.
Firstly, I get the app to call the same Authenticate endpoint as I do for username/password, only with dummy credentials, and add in a "GoogleToken" header, containing the token.
In my authentication code, I check for the GoogleToken header, and if it exists, follow that code path to validate on the Google servers, get an email address, and link to my own User table. Then the rest of the process for building claims and returning a new API token follows the original path.
start here : https://developers.google.com/identity/protocols/OAuth2#basicsteps
This explains how oAuth2 works. So you receive a Google token, now you call Google and request the user's details. you will receive their email which is enough to authenticate them. You could store the token as they are valid for a while and you can keep reusing it for whatever you need until it expires or it is invalidated.
Check this discussion on the same subject :
How can I verify a Google authentication API access token?
if you need more info on how OAuth2 works I can point you to one of my own articles : https://eidand.com/2015/03/28/authorization-system-with-owin-web-api-json-web-tokens/
There's a lot to take in, but it sounds like you need to understand how these things work together. Hope this helps.
Update:
I don't have full access to your setup, but I hope that the following code might help you with using Google as ID provider. Please add the following code to your startup.auth.cs file.
var googleAuthOptions = new GoogleOAuth2AuthenticationOptions
{
ClientId = "ef4ob24ttbgmt2o8eikgg.apps.googleusercontent.com",
ClientSecret = "DAK0qzDasdfasasdfsadwerhNjb-",
Scope = { "openid", "profile", "email" },
Provider = new GoogleOAuth2AuthenticationProvider
{
OnAuthenticated = async ctx =>
{
//You can get the claims like this and add them to authentication
var tokenClaim = new Claim("GoogleAccessToken", ctx.AccessToken);
var emailClaim = new Claim("email", ctx.Email);
var claimsIdentity = new ClaimsIdentity();
claimsIdentity.AddClaim(tokenClaim);
claimsIdentity.AddClaim(emailClaim);
HttpContext.Current
.GetOwinContext()
.Authentication
.SignIn(claimsIdentity);
await Task.CompletedTask;
}
},
AuthenticationType = "Google"
};
app.UseGoogleAuthentication(googleAuthOptions);
This allows the Google to act as ID Provider and the OnAuthenticated gets called when the authentication is successful. You can get the claims out of it and use them to signin. Please let me know if this worked, if not give me more details about your setup (what kind of framework, client setup and may be more details about your setup in startup file).
Thank you.
Please see this link for details on how we can use Google as ID Provider. I am sure you might have looked at this link, but in case you missed it. If none of these links work for you please include specific details on where you are deviating from what is mentioned in the links.
I assume you have a different requirement than what is specified in those links. Hence, I will try to answer your questions individually. Please let me know if you have any further questions.
When the client has completed authentication with the Google API, it gets back a "code". That code still needs to be converted into a token. Whose responsibility should that be - the client or the API? (I'm leaning towards making it the client's responsibility, if just for the reason of distributing the workload better.)
Exchanging the code for access token is definitely the responsibility of the API as the token exchange involves sending the ClientId and Client Secret along with the code. Client secret is supposed to be saved on the server side (API) but not on the client
Whether the client is passing through a code or a token, I need to be able to receive it in the API. Should I just use a plain vanilla Controller to receive it, with an endpoint returning an object of type AuthenticationProperties, or is there some special OWIN way of doing this?
This should work seamlessly if you are using the Google provider as mentioned in the above links. If not, the endpoint should be an anonymous endpoint accepting the code and making a request to Google (may be by using HttpClient) to get the access token along with the profile object for user related information.
If I'm using a plain vanilla Controller, how do I validate my token? In other words, how do I get access to the OWIN context so that I can mark the AuthenticationTicket as validated?
You have to implement OnGrantAuthorizationCode as part of your MyAuthorizationProvider class. This gives access to the context to set validated to true.
How do I write an automated test that simulates the client side of the process? AFAICT, the authentication wants to have a user physically click on the "Allow" button to grant my app access to their identity stuff, before it will generate the auth code. In an automated test, I would want to pass username/password etc. all from code. How do you do that?
This can be achieved partially, but, with that partial test you can be sure of good test coverage against your code. So, you have to mock the call to the Google API and assume that you have retrieved a valid response (hard code the response you received from a valid manual test). Now test your code on how it behaves with the valid response. Mock the Google API cal for an invalid response and do the same. This is how we are testing our API now. This assumes that Google API is working fine and tests my code for both valid/ in-valid responses.
Thank you,
Soma.
Having gone through something like this recently, I'll try to answer at least some of your questions:
The client should be getting a token from Google, which you can pass unaltered through to the API:
function onSignIn(googleUser) {
var profile = googleUser.getBasicProfile();
var idToken = googleUser.getAuthResponse().id_token;
}
A plain vanilla Controller should do it. The client can subsequently post an object in there, containing at least that token plus the client id (might be useful to know where the request comes from) and even the providerUserId;
Unfortunately I'm not that familiar with the Owin stack
Fully end-to-end integration testing might be tricky, although you might achieve something through tools like Selenium, or some mocking tool. The API however should be testable just by posting some fake data to that vanilla controller, although you might have to rely on some sort of mock implementation when you get to validating that token through Google (although you could also validate it manually on the server, provided you get the Google public api key).
I am building an API using the Scala version of the play framework. Some of the endpoints will contain confidential data but I am not sure how exactly to secure this.
Secure social (http://securesocial.ws/guide/configuration.html) is a library that I've been looking at but it seems oriented around websites and logging in with OAuth providers.
In this case it seems like I need to be an OAuth provider. Or is it possible that I can allow users to login with a provider, say Twitter? But then how would that work? The documentation around OAuth seems to be incredibly awful.
There is no built in way to manage tokens. I would recommend building a token system, that would distribute and manage access token use. You can have one per user, or use a different system.
Then for each endpoint, you would have a wrapped action to secure the API.
case class SecuredAPIRequest(request:Request[AnyContent]) extends
WrappedRequest(request)
trait SecuredController{
import play.api.mvc.Results._
//This takes an action for a request, and checks to see if the apiKey equals or API_KEY
def SecuredAPIAction(f:SecuredAPIRequest => Result) = Action{
request =>
request.body.asJson.map{ jsValue =>
(jsValue \ "apiKey").asOpt[String] match{
case Some(key) if validKey(key) => f(SecuredAPIRequest(request)) //We are clear to go, execute our function.
// NOTE: validKey would be a function that checks the key against our DB ensuring that it is valid.
case None => Forbidden
}
}.getOrElse(Forbidden)
}
}
What SecureSocial does is route security request to the authentication service ( Twitter, Facebook, etc ), and uses their token as security. This would not work as an API, because it would be impossible to redirect users for auth.