I have setup a Linked Service in Azure Data Factory to use in a Copy Activity to access Graph APIs, which require use of the OAuth protocol, which works fine. I now want to extend that to use the same Linked Service with a Web Activity. But in tests, it is returning "Empty Token"; Am I missing a (simple) config. option, or am I misunderstanding something?
Earlier, I used explicit Token fetch, which also works fine, but as my queries will - in Production - take longer than the (default) Token TTL, I did not want to have to implement Token refresh, at least not if the Linked Service will handle it for me.
I'm grateful to https://learn.microsoft.com/answers/users/7986441/carlzhao-msft.html for pointing out that I need also to specify, in the Web Services Settings Tab, the Authentication as Managed Identity, with the accompanying Resource as https://graph.microsoft.com/, for my Use Case
Related
An ex-employee planned a Microservice Architecture which is being implemented now. I've few question regarding the design and I'd highly appreciate your feedbacks.
Explanation
Dematerialized UI has a matching dematerialized API.
Dematerailized API validates the user and generates token via SSO Library.
Flight API does the I/O validation & validate the request via validate request microservice
Flight API calls Booking API to get some bookings based on the UserId
Flight API calls Print Booking API to generate Messages using Generate Message Microservice
Print Booking API must call Data Access API to get data and then call Generate PDF microservices.
Data Access API calls the database for data.
My Project Structure
FlightBookingsMicroserice.V1 //solution
ApiGatways //folder
DMZ.API/DMZ.API.csproj //Folder/project
BuildingBlocks
EventBus/EventBus.csproj
EventBus/EventBusRabbitMQ
Services
SSO
SSO.API/SSO.csproj
SSO.UnitTests
Flight
Flight.API/Flight.API.csproj
Flight.UnitTets
//Similar for all
ValidationRequest
Booking
PrintBooking
PrintBooking.API.csproj
DataAccess
DataAccess.API.csproj
GeneratePDF
GenerateMessage
UI
UI
Docker-compose
Questions
Should I be using ocelot in DMZ.API.csproj, Flight API and Print Booking API.
Is my project structure a Microservice way of development
Should I continue to use ASP.NET Core Web API with .NET 6 for Dematerialized API in orange, Function API in blue and Microservice in purple projects.
For validation, since the SSO is passed from Dematerialized UI what if the token expires while CRUD operations
is already performed for some stages [rolling back changes is a hassle].
Should each API access to an identidy server and validate the user passed and generate its own token for its
services in purple.
Thank you in advance.
The core question is if you really need all those services and if you perhaps are making things too complicated. I think the important thing is to really consider and really make sure you justify why you want to go through this route.
If you do synchronous API calls between the services, that creates coupling and in the long run a distributed monolith.
For question #4, you typically use one access token for the user to access the public service, and then you use a different set of internal tokens (machine-to-machine also called client credentials in OpenID Connect parlor) between services that have a totally different lifetime.
q1: ocelot is an API GATEWAY which is the entry point for your requests. so it should be the first layer/service meet by user request in front of your services and it forwards the request to the service according to its configuration. so it is lay in the front for all services you have. some arch provide another api gateway for different reasons like specific api gateway for mobiles request for example.
q2: as looking separate services (i cant understand function api but i assume they are services also ) yes but the microservices development is not just about separating things, its about design and identifying the services from business context (Domain Driven Design).its very challenging to identify services and their size and the way they are communicate to each other (asynchronous communication and synchronous communication).
q3: microservices is not about languages and frameworks.one of benefits of microservices architecture is its not language or framework dependent. the may be multiple languages used in microservices. choosing languages it depends on organization policy or your own reasons. if you are .net developer then go for .net.
q4: all the services are registered with identity server and they validate the given token by it. the identity server generate token (there may be multiple tokens) with scopes . the request from identified users always has the token in the headers and the services validate incoming token by referring identity server. this tokens has lifetime and also identity server generates refresh tokens in case of expiry of current token. please look at Oauth docs and rfc. also this https://www.youtube.com/watch?v=Fhfvbl_KbWo&list=PLOeFnOV9YBa7dnrjpOG6lMpcyd7Wn7E8V may helped. you can skip the basic topics. i learned a lot from this series.
I'm curious what is the "best practice" for authenticating against a restful-api that is protected by Google IAP.
Allow me to break it down step by step to elucidate what I am trying to achieve:
Go to my dev environment's url: dev.blah.com. Here, I am prompted by IAP to log-in. I log in. I now have access to my SPA.
I am browsing my SPA. But! I'm unable to talk to my API, because it is ALSO protected by IAP.
I've read that I can do programmatic authentication in the documentation but I'm unsure if my particular use case is suited for programmatic authentication, or if it will even be SECURE.
So, am I correct in that I will have to implement a client-side programmatic-authentication workflow in order to access my API?
Or is there an alternative way to say "Once I'm authenticated to access THIS IAP-protected resource, i can log into these other resources as well." I cannot simply copy the token because it is http-only and the cookie is restricted to my current domain.
Some additional information:
My services are running in a Google Kubernetes Engine cluster and have automatically created load balancers based off of ingress objects I declare.
Each IAP-protected environment gets it's own OAuth credentials, and I configure the load-balancers to use IAP using: gcloud compute backend-services update [backend-service] --global --iap=enabled,oauth2-client-id=[the_id],oauth2-client-secret=[the_secret]
let me make sure I understand what you're trying to do. It sounds like you want to call your API from JavaScript inside the web browser, is that correct?
If so, and:
All backend services are configured to use the same OAuth client ID.
The API and the SPA are serving off the same domain.
then I think things will just work, because the JavaScript-generated HTTP requests will be sent with the IAP session cookie. ... Which must not be your setup, or things would just be working which clearly they aren't. :>
If you're not able to arrange for your API calls to use the IAP session cookie generated when you authenticate to the SPA, this will be tricky. You wouldn't want to use service account authentication from the browser, since that would give all end-users access to the service account.
Hm, it looks like there is a way to do Google OAuth from JavaScript: https://developers.google.com/api-client-library/javascript/features/authentication -- I haven't tried it, but if you're able to get that working, you ought to be able to use a flow like https://cloud.google.com/iap/docs/authentication-howto#authenticating_from_a_desktop_app . If you want to go that route, let me know how it goes and I'd love to help you get unstuck and then get that approach added to our documentation!
--Matthew, Google Cloud IAP engineering
I'm writing an application that will be the backend for a react website. The website is to be used by our customers, but we will fully control the permissions of the user. We have decided to use Azure AD to secure requests, but will also be exposing the API for end users to use directly if desired.
My understanding is in Azure AD I will have to create an application that will allow web based implicit authentication (for the react site), as well as a native application that will allow a dameon based application to authenticate to the API.
This I believe means I will have two audience ids in my application.
I'm trying to get claims to include groups, and I can see if I edit the meta data of both applicaitons in azure AD to include "groupMembershipClaims": "SecurityGroup" I can get claims with the group IDs in, but no names.
I think I can also use appRoles to set roles the application uses, but I've yet to get that to come through as claims in the JWT, but I'm assuming it can be done, however I'd need to setup the roles on each applicaiton, then add the user twice which isn't really ideal. I also think that because my app is multi-teanated that external users could use this to set their own permissions, which isn't what I want to do.
Sorry I'm just totally lost and the documentation is beyond confusing given how frequently this appears to change!
TLDR: Do I need two applicaitons configured in azure ad, and if so whats the best way to set permissions (claims). Also is oAuth 2 the right choice here, or should I look at open id?
Right away I gotta fix one misunderstanding.
Daemon apps usually have to be registered as Web/API, i.e. publicClient: false.
That's because a native app can't have client secrets.
Of course the daemon can't run on a user's device then.
Since that's what a native app. An app that runs on a user's device.
This I believe means I will have two audience ids in my application.
You will have two applications, at least. If you want, the back-end and React front can share one app (with implicit flow enabled). And the daemon will need another registration.
I'm trying to get claims to include groups, and I can see if I edit the meta data of both applicaitons in azure AD to include "groupMembershipClaims": "SecurityGroup" I can get claims with the group IDs in, but no names.
Yes, ids are included only. If you need names, you go to Graph API to get them. But why do you need them? For display? Otherwise, you need to be using the ids to setup permissions. Names always change and then your code breaks.
I think I can also use appRoles to set roles the application uses, but I've yet to get that to come through as claims in the JWT, but I'm assuming it can be done, however I'd need to setup the roles on each applicaiton, then add the user twice which isn't really ideal. I also think that because my app is multi-teanated that external users could use this to set their own permissions, which isn't what I want to do.
Your thoughts for multi-tenant scenarios are correct. If you did want to implement these though, I made an article on it: https://joonasw.net/view/defining-permissions-and-roles-in-aad.
Why would you need to setup the roles in multiple apps though? Wouldn't they only apply in the web app?
If the native app is a daemon, there is no user.
Overall, I can see your problem. You have people from other orgs, who want access to your app, but you want to control their access rights.
Honestly, the best way might be to make the app single-tenant in some tenant which you control. Then invite the external users there as guests (there's an API for this). Then you can assign them roles by using groups or appRoles.
If I misunderstood something, drop a comment and I'll fix up my answer.
Azure AD is of course a powerful system, though I also find the OAuth aspects confusing since these aspects are very mixed up:
Standards Based OAuth 2.0 and Open Id Connect
Microsoft Vendor Specific Behaviour
ROLE RELATED ANSWERS
This is not an area I know much about - Juunas seems like a great guy to help you with this.
OAUTH STANDARDS AND AZURE
I struggled through this a while back for a tutorial based OAuth blog I'm writing. Maybe some of the stuff I learned and wrote up is useful to you.
AZURE SPA AND API CODE SAMPLE
My sample shows how to use the Implicit Flow in an SPA to log the user in via Azure AD, then how to validate received tokens in a custom API:
Code Sample
Write Up
Not sure how much of this is relevant to your use case, but I hope it helps a little on the tech side of things...
Like SO, I am gonna depend on many Open ID providers to provide user authentication and I will use my own authorization methods. but I'm still in development phase, and don't want to work with real OpenID providers currently, what approach can I use to test my users and their activities in the website (w/o TDD), to emulate real users but not really use Open ID providers.
No need to make auto-transfer of users into real OpenID servers (when moving to production mode) since the current users are just for testing purposes and Unit-test code.
I guess, I need a User Service layer which provides a higly abstracted way to deal with users, so that the move to the real Open ID providers can be smooth in the future and doesn't affect the logic of my already written code.
Using C#.Net 4, ASP.Net MVC 3, Ninject
DotNetOpenAuth provides both server and client portions of OpenID and can be used to run your own OpenID provider for local testing.
Give your site members their own OpenIDs with the provider support included in this library.
Sample relying party and provider web sites show you just how to do it.
I simply register my test id as user with various organisations. I don't see anything wrong with that. I get to see the various responses and their differences.
I found it terribly easy to code for openid consumer. Just need to understand the sequence of responses. Draw the UML sequence diagram to aid your understanding before you start coding. No need to fake openId. Otherwise, whip up an openid server yourself.
Your services shouldn't depend on OpenID. Just have OpenID plug into an authentication module to provide a local user principal. In development, you can have the auth module return a fake user principal with the permissions you desire.
In a beta environment you could turn on OpenID and use test accounts from any OpenID provider. Having to log in during the development phase will just slow down all the developers. Any authentication bug or internet outage will kill everyone's productivity.
I'm now reading some introduction materials about OAuth, having the idea to use it in a free software.
And I read this:
The consumer secret must never be
revealed to anyone. DO NOT include it
in any requests, show it in any code
samples (including open source) or in
any way reveal it.
If I am writing a free client for a specific website using OAuth, then I have to include the consumer secret in the source code, otherwise making from source would make the software unusable. However, as it is said, the secret should not be release along with the source.
I completely understand the security considerations, but, how can I solve this dilemma, and use OAuth in free software?
I thought of using an external website as a proxy for authentication, but this is very much complicated. Do you have better ideas?
Edit:
Some clients like Gwibber also use OAuth, but I haven't checked its code.
I'm not sure I get the problem, can't you develop the code as open source retrieve the customer secret from a configuration file or maybe leave it in a special table in the database? That way the code will not contain the customer secret (and as such will be "shareable" as open source), but the customer secret will still be accessible to the application.
Maybe having some more details on the intended platform would help, as in some (I'm thinking tomcat right now) parameters such as this one can be included in server configuration files.
If it's PHP, I know a case of an open source project (Moodle), that keeps a php (config.php) file containing definitions of all important configurations, and references this file from all pages to get the definition. It is the responsibility of the administrator to complete the contents of this file with the values particular to that installation. In fact, if the application sees that the file is missing (usually on the first access to the site) it will redirect to a wizard where the administrator can fill the contents in a more user friendly way. In this case the customer secret will be one of these configurations, and as such will be present in the "production" code, but not in the "distributable" form of the code.
I personally like the idea of storing that value in the database in a table designed for it and possibly other parameters as the code needs not be changed. Maybe a installation wizard can be presented here ass well in the case the values do not exist.
Does this solve your problem?
If your service provider is a webapp, your server needs consumer signup pages that provides the consumer secret as the user signs up their consumer. This is the same process Twitter applications go through. Try signing up there and look at their workflow, you'll have all the steps.
If your software is peer-to-peer, each application needs to be both a service provider and a consumer. The Jira and Confluence use cases below outline that instance.
In one of my comments, I mention https://twitter.com/apps/new as the location of where Twitter app developers generate a consumer secret. How you would make such a page depends on the system architecture. If all the consumers will be talking to one server, that one server will have to have a page like https://twitter.com/apps/new. If there are multiple servers (i.e. federations of clients), each federation will need one server with this page.
Another example to consider is how Atlassian apps use OAuth. They are peer-to-peer. Setting up Jira and Confluence to talk to one another still has a setup page in each app, but it is nowhere near as complex as https://twitter.com/apps/new. Both apps are consumers and service providers at the same time. Visiting the setup in each app allows that app to be set up as a service provider with a one-way trust on the other app, as consumer. To make a two-way trust, the user must visit both app's service provider setup page and tell it the URL of the other app.