For the past 10+ days I've read an watched ALL the content I could find on understanding OAuth2 and OpenID Connect, only to find that many people disagree on the implementation, which really confuses me.
To my understanding, all the articles and examples I found assume you want access to eg. google calendar, profile info or emails if you eg. login with google, but I do NOT need to access other than my own API's - I only want to use Google, Facebook etc for logging in, and getting an id which I can link to my user in my own database - nothing more than that.
I'll try illustrate my use case and use that as an example.
A note on the diagram: the Authentication service could probably be built into the API Gateway - not that i matters for this example, since this is not about "where to do it", but "how to do it the best way" possible, for an architecture such as mine, where it's used for my own API's / Microservices, and not accessing Google, Facebook etc. external API's
If you can understand what I'm trying to illustrate with this diagram above, please tell me if I've misunderstood this.
The most basic requirements for this architecture you see here are:
Users can login with Google, Facebook, etc.
The same login will be used for all micro-services
OpenId user will have a linked account in the database
User access is defined in my own db, based on groups, roles and permissions
I do not intend to use external API's after the user is authenticated and logged in. No need for ever accessing a users calendar, email etc. so I really just need the authentication part and nothing else (proof of successful login). All user access is defined in my own database.
So a few fundamental questions comes to mind.
First of all, is OpenID Connect even the right tool for the job for authentication only (I'll have no use for authorization, since I will not need read/write access to google / facebook API's other than getting the ID from authenticating)?
People generally do not agree on whether to use the ID or Access token for accessing your own API's. As far as I understand the ID token is for the client (user-agent) only, and the access token is for eg. accessing google calendar, emails etc.... External API's of the OpenID Provider... but since I'll only be accessing my own API's, do I event need the access token or the ID token - what is the correct way to protect your own API's?
If the ID token is really just for the client, so it can show eg. currently logged in user, without going to the DB, I have 0 use for it, since I'll probably query the user from from the db and store it in redux for my react frontend app.
Dilemma: To store user details, groups, roles and permission inside JWT or not for API authorization?
By only storing the user identifier in the token, it means that I always allow authenticated users that has a valid token, to call endpoints BEFORE authorization and first then determine access based on the db query result and the permissions in my own database.
By storing more data about the user inside the JWT, it means that in some cases, I'd be able to do the authorization / access (group, role, permission) check before hitting the API - only possible with user info, groups, roles and permission stored inside a JWT issued upon login. In some cases it would not be possible due to eg. the CMS content access permissions being on a per-node level. But still it would mean a little better performance.
As you can see on the diagram I'm sending all API requests through the gateway, which will (in itself or with an authentication service) translate the opaque access token into some JWT with an identifier, so I can identify the user in the graph database - and then verify if the user has the required groups, roles and permissions - not from an external API, but from my own database like you see on the diagram.
This seems like a lot of work on every request, even if the services can share the JWT in case multiple services should need to cross call each other.
The advantage of always looking up the user, and his permissions in the db, is naturally that the moment the user access levels change, he is denied/granted access immediately and it will always be in sync. If I store the user details, groups, roles and permission inside a JWT and persist that in the client localstorage, I guess it could pose a security issue right, and it would be pretty hard to update the user info, groups, roles and permissions inside that JWT?
One big advantage of storing user access levels and info inside the JWT is of course that in many cases I'd be able to block the user from calling certain API's, instead of having to determine access after a db lookup.
So the whole token translation thing means increased security at the cost of performance, but is is generally recommended and worth it? Or is it safe enough to store user info and groups, roles, permissions inside the JWT?
If yes, do I store all that information from my own DB in the ID Token, Access token or a 3rd token - what token is sent to the API and determines if the user should be granted access to a given resource based on his permissions in the db? Do I really need an access token if I don't need to interact with the ID providers API? Or do I store and append all my groups, roles, permissions inside the ID token (that doesn't seem clean to me) issued by OpenID connect, and call the API and authorize my own API endpoints using that, even if some say you should never use the ID token to access an API? Or do I create a new JWT to store all the info fetched from my database, which is to be used for deciding if the user can access a given resource / API endpoint?
Please do not just link to general specs or general info, since I've already read it all - I just failed to understand how to apply all that info to my actual use case (the diagram above). Try to please be as concrete as possible.
Made another attempt to try and simply the flow:
The following answer does only apply for a OpenID Connect authentication flow with a 3rd party IDP (like Google). It does not apply for an architecture where you host your own IDP.
(There are some API gateways (e.g Tyk or Kong) which support OpenID Connect out of the box.)
You can use JWTs (ID token) to secure your APIs. However, this has one disadvantage. JWTs cannot be revoked easily.
I would not recommend this. Instead you should implement an OAuth2 authorization server which issues access tokens for your API. (In this case, you have two OAuth2 flows. One for authentication and one for authorization. The ID and access token from the IDP are used only for authentication.)
The following picture shows a setup where the API gateway and authentication/authorization server are two separate services. (As mentioned above, the authentication/authorization can also be done by the API gateway.)
The authentication flow (Authorization Code Grant) calls are marked blue. The authorization flow (Implicit Grant) calls are marked green.
1: Your web app is loaded from the app server.
2a: The user clicks on your login button, your web app builds the authorization URL and opens it. (See: Authorization Request)
2b: Because the user hasn't authenticated and has no valid session with your authorization server, the URL he wanted to access is stored and your authorization server responds with a redirect to its login page.
3: The login page is loaded from your authorization server.
4a: The user clicks on "Login with ...".
4b: Your authorization server builds the IDP authorization URL and responds with a redirect to it. (See: Authentication Request)
5a: The IDP authorization URL is opend.
5b: Because the user hasn't authenticated and has no valid session with the IDP, the URL he wanted to access is stored and the IDP responds with a redirect to its login page.
6: The login page is loaded from the IDP.
7a: The user fills in his credentials and clicks on the login button.
7b: The IDP checks the credentials, creates a new session and responds with a redirect to the stored URL.
8a: The IDP authorization URL is opend again.
(The approval steps are ignored here for simplicity.)
8b: The IDP creates an authorization and responds with a redirect to the callback URL of your authorization server. (See: Authentication Response)
9a: The callback URL is opened.
9b: Your authorization server extracts the authorization code from the callback URL.
10a: Your authorization server calls the IDP's token endpoint, gets an ID and access token and validates the data in the ID token. (See: Token Request)
(10b: Your authorization server calls the IDP's user info endpoint if some needed claims aren't available in the ID token.)
11a/b: Your authorization server queries/creates the user in your service/DB, creates a new session and responds with a redirect to the stored URL.
12a: The authorization URL is opend again.
(The approval steps are ignored here for simplicity.)
12b/+13a/b: Your authorization server creates/gets the authorization (creates access token) and responds with a redirect to the callback URL of your web app. (See: Access Token Response)
14a: The callback URL is opened.
14b: Your web app extracts the access token from the callback URL.
15: Your web app makes an API call.
16/17/18: The API gateway checks the access token, exchanges the access token with an JWT (which contains user infos, ...) and forwards the call.
A setup where the authorization server calls the API gateway is also possible. In this case, after the authorization is done, the authorization server passes the access token and JWT to the API gateway. Here, however, everytime the user infos change the authorization server has to "inform" the API gateway.
This is a very long question. But I believe most can be summarised by answering below,
To my understanding, all the articles and examples I found assume you want access to eg. google calendar, profile info or emails if you eg. login with google,
You do not necessarily use Access token (ID token in some occasions) to access the services offered by token issuer.You can consume tokens by your own APIs. What these Identity Providers (synonym to Authorization server, or IDP in shorthand) is to hold identities of end users. For example, typical internet have a Facebook account. With OAuth and OpenID Connect, the same user get the ability to consume your API or any OAuth/OIDC accepted service. This reduce user profile creation for end users.
In corporate domain, OAuth and OIDC serves the same purpose. Having a single Azure AD account lets you to consume MS Word as well as Azure AD's OIDC will issue tokens which can be used to Authorise against an in-house API or an third party ERP product (used in organization) which support OIDC based authentication. Hope it's clear now
A note on the diagram is that the Authentication service could probably be built into the API Gateway - not sure if that would be better?
If you are planning to implement an API gateway, think twice. If things are small scale and if you think you can maintain it, then go ahead. But consider about API managers which could provide most of your required functionalities. I welcome you to read this article about WSO2 API manger and understand its capabilities (No I'm not working for them).
For example, that API manager has built in authentication handling mechanism for OAuth and OIDC. It can handle API authentication with simple set of configurations. With such solution you get rid of the requirement of implement everything.
What if you can't use an API manager and has to do it yourself
OpenID Connect is for authentication. Your application can validate the id token and authenticate end user. To access APIs through API Gateway, I think you should utilise Access token.
To validate the access token, you can use introspection endpoint of the identity provider. And to get user information, you can use user-info endpoint.
Once access token is validated, API gateway could create a session for a limited time (ideally to be less or equal to access token lifetime). Consequent requests should come with this session to accept by API gateway. Alternatively, you can still use validated access token. Since you validated it at the first call, you may cache for a certain time period thus avoiding round trips to validations.
To validate user details, permission and other grants, well you must wither bind user to a session or else associate user to access token from API gateway at token validation. I'm also not super clear about this as I have no idea on how your DB logic works.
First Appreciate your patience in writing a very valuable question in this forum
we too have same situation and problem
I want to go through ,as images are blocked in our company in detail
Was trying to draw paralles to similar one quoted in the book
Advance API In Practise - Prabath Siriwerdena [ page 269]Federating access to API's Chapter. Definitely worth reading of his works
API GW should invoke Token Exchange OAUTH2.0 Profile to IDP [ provided the IDP should support TOken Exchange profile for OAUTH 2.0
The Absence of API Gateway , will result in quite bespoke development
for Access Control for each API Check
you land up in having this check at each of the api or microservice [ either as library which does work for you as a reusable code]
definitely will become a choking point.]
After reading the documents of Google API. I know that if a project we create needs to access private data, we should use oauth. But here is my situation. we will open a business account in Youtube, and we will create a project to push videos to our own account, we don't need to operate other user's' account. Can we use google youtube data api without OAuth?
There are two ways to access private user data with Google APIs.
Strait Oauth2. where you have a consent for asking the owner of the account if you can access it
Service accounts which are technically pre authorized by the developer.
Normally I would say because you are only accessing the one account that you own, use a service account. Unfortunately the YouTube API does not support service account authentication.
Due to the lack of service account support you will have to use Oauth2. I have done this in the past.
Authentication your script once, using a server sided language of some kind. The Authentication server will return to you a Refresh token. Refresh tokens can be used at any time to get a new access token. Access tokens are used to access Google APIs and are only valid for an hour. Save this refresh token someplace. You will then be able to allow access the YouTube account in question when ever you like.
Note: You will have to watch it. Refresh tokens can on rare occasion become invalid. I recommend having a script ready that will allow you to re authenticate the application again storing a new refresh token. Its rare that it happens but it can happen best to be pre-paired.
Oauth Play ground
Part of the point of Oauth is that it identifies your application to Google though the creation of your project on Google developer console. Things like quota and access to which APIs is controlled though that. If you spam the API they will know and shut you down. (never seen this happen)
When you request access of a user it pops up with the name of the project on google developer console. This is identified by the client id and client secrete for that project on google developer console. When I use oauth playground I get asked 'Google OAuth 2.0 Playground would like to ..'
So by using playground you are using Googles client id and client secrete to create a refresh token for yourself. If N other devs are also doing this the quota for YouTube may be used up in the course of a day. Also security wise you are now giving that project access to your data. Ignore that for a second what if google suddenly decides to remove change the client id or generate a new one. Your refresh token will no longer work. What if random dev X is using it as well and he starts spamming everything and the client id gets shut down (Think this happened last year) your going to have to wait for google to upload a new client id for the one that has now been banned.
Google OAuth 2.0 Playground might seam nice but its not for daily use IMO its good for testing nothing more. Create your own project and get your own access its not hard just requires a programing language that can handle a http Post.
My tutorial Google 3 legged oauth2 flow
What exactly does the word "offline" mean with regard to the offline access granted by an OAuth server?
Does it mean that the resource server will return data about the user even when the user is logged out of the third-party application or when the user is logged out of the OAuth resource server such as Facebook or Google or Twitter?
Offline access is IMO a really bad name for it, and I think its a term only
Google uses its not in the RFC for OAuth as far as I remember.
What is Google offline access?
When you request offline access the Google Authentication server returns a
refresh token. Refresh tokens give your application the ability to
request data on behalf of the user when the user is not present and in front of
your application.
Example of an app needing offline access
Let's say I have a Super Awesome app that downloads your Google Analytics Data,
makes it into a nice PDF file and emails it to you every morning with your
stats. For this to work my application needs to have the ability to access
your Google Analytics data when you are not around, to give me permission to do
that. So Super Awesome app would request offline access and the
authentication server would return a refresh token. With that refresh token
Super awesome app can request a new access token whenever it wants and get your
Google Analytics data.
Example of an app not needing offline access
Let's try Less Awesome app that lets you upload files to Google Drive. Less
Awesome app doesn't need to access your Google drive account when you're not
around. It only needs to access it when you are online. So in theory it
wouldn't need offline access. But in practice it does, it still gets a refresh
token so that it won't have to ask you for permission again (this is where I
think the naming is incorrect).
Helpful quote from the OpenStack documentation:
If a refresh token is present in the authorization code exchange, then it
can be used to obtain new access tokens at any time. This is called
offline access, because the user does not have to be present at the browser
when the application obtains a new access token.
The truth about offline access
The thing is that in a lot of cases the authentication server will return the
refresh token to you no matter what: You don't have to actually ask for anything –
it gives it to you. Giving you the ability to access the users data when they
aren't around. Users don't know that you could access their data without them
being there. It's only the JavaScript library and I think the PHP library
that hide the refresh token from you, but it's there.
Example
By just posting (i.e. HTTP POST request):
https://accounts.google.com/o/oauth2/token?code={AuthCode}&
client_id={ClientId}.apps.googleusercontent.com&client_secret={ClientSecret}&
redirect_uri=urn:ietf:wg:oauth:2.0:oob&grant_type=authorization_code
Here is the response:
{
"access_token": "ya29.1.AADtN_VSBMC2Ga2lhxsTKjVQ_ROco8VbD6h01aj4PcKHLm6qvHbNtn-_BIzXMw",
"token_type": "Bearer",
"expires_in": 3600,
"refresh_token": "1/J-3zPA8XR1o_cXebV9sDKn_f5MTqaFhKFxH-3PUPiJ4"
}
I now have offline access to this users data, and I never told them that I
would have it. More details be found in this short article: Google 3 legged
OAuth2 flow.
Useful reading
Using OAuth 2.0 for Web Server Applications
Understanding Refresh Tokens
By design the access tokens returned by the OAuth flow expire after a period of time (1 hour for Google access tokens), as a safety mechanism. This means that any application that wants to work with a user's data needs the user to have recently gone through the OAuth flow, aka be online. Requesting offline access provides the application a refresh token it can use to generate new access tokens, allowing it to access user data long after the data has gone through the OAuth flow, aka when they are offline.
Getting offline access is needed when your application continues to run when the user isn't present. For instance, if there is some nightly batch process, or if your application responds to external events like push notifications. However if you only access user data while the user is actively using your application then there is no need for offline access. Just send the user through the OAuth flow every time you need n access token, and if they've previously granted access to your application the authorization page will instantly close, making the process nearly invisible to the user.
For Google APIs, you can request offline access by including the parameter access_type=offline in the authorization URL you present to your users. Offline access, and hence refresh tokens, is requested automatically when using the Installed Application flow.
I need to interact with an API that only supports OAuth2.
The problem is, I would like to write a purely server side application which should sit there without a GUI polling an API every day.
The API gives me the ability to get the application token programatically, but it looks like I need to implement the entire GUI flow to get the subsequent access token. This is because I need to log in via the application providers web based login screen.
It looks like I then need to get that access token, and copy this out as my server side credential where I recreate it. If that ever expires or goes bad, I'll need to go back via the GUI flow to get my server side access token.
Is my understanding correct here as this feels very clunky?
Specifically:
Can I avoid implementing the process where we link over to the application providers login form?
Is it right that after doing this, I have to unpick an access token and store this within my server side application. I don't appear to have any control over whether that will expire?
I can see that e.g. Facebook specifically support server side and client side flow. I wonder if I'm coming up against limitations in this particular implementation of OAuth 2?
I don't know how you actually want the app to behave, but one thing is certain - you do have to input the user credentials once.
Once you authenticate and authorize (there are a lot of open questions on SO, about automatic authentication), your app will get not only an access token, but also a Refresh Token. A refresh token is just what you need in your use case. You can store it your the server side - A refresh token does not have an expiration time. It lives till the user explicitly revokes permissions.
For any OAuth service provider there is a token exchange endpoint where you can exchange the refresh token for a (refresh token + access token) pair. So, at the backend (your server) you can at any time hit this endpoint - get a short-lived access token and perform the operation that you need to. This saves you all the effort of following the GUI flow every time.
See this link - https://developers.google.com/accounts/docs/OAuth2WebServer#offline
EDIT - Made some changes after reading your comment. You simply need to know how to use refresh tokens in your app.
I've found that the typical solution to the problem I asked in this question is to use XAuth.
Many providers such as Twitter and the application I am currently working against support XAuth to provide a simplified flow without the user interface based authentication.
What is the difference among BasicAuth,OAuth and XAuth?
I'm trying to use OAuth with .NET (DotNetOpenAuth) to send updates to a Twitter account via a web application. I understand the basic workflow of OAuth and Twitter.
Where I'm confused if is it useful in a server web application? I don't want any user interaction.
But how it seems after an application start, the request token needs to be recreated and also an access token. This involves user interaction.
What is the correct workflow for my case?
Storing the request token or access token in config file?
Or the easist way, using HTTP basic authentication?
Thanks
If I understand you correctly your application will not be interacting with Twitter on behalf of your users but will be acting as the Twitter account for your application.
In this case there are 2 main factors to consider.
1) Do you want "from API" attached to each status as will be if you use basic auth or your applications name will happen if you use OAuth.
2) Do you want to put in the extra effort to implement OAuth.
If you decide to go with OAuth you would store your apps consumer key/secret and the accounts access token in configuration just like you would store the accounts screenname/password.
Your "request token needs to be recreated" phrase suggests you might be running into the problem where every time your user visits you need to re-authorize to Twitter, and perhaps you're looking for a way to access the user's Twitter account while he's not at your web site, and how can you do this when their token isn't fresh from being re-authorized. Is that right?
If so, the user isn't supposed to have to re-authorize Twitter every time they visit your site. The token is supposed to last a long time, which would also allow your site to access their Twitter account when they are not directly interacting with your web site. The problem may be that you haven't implemented the IConsumerTokenManager interface, but are instead using the default InMemoryTokenManager, which is for sample use only, since this memory-only token manager loses tokens every time the web app is restarted. Your own implementation of this simple interface should store and read the tokens out of some persistent storage such as a database.