One month ago I developed a windows service which calls some Google anaytics APIs and the code is scheduled to execute each 15 minutes. It has worked properly till 5 days ago when I checked the event viewer I saw that the service raised unauthorized access (401) exception in each connection to Google API. When I restarted the service it again started working properly.
What could be the reason?
Regards
Most likely, you're obtaining an authentication token on startup and using that for all subsequent requests. Authentication tokens do expire eventually; when you get a 401, you should re-authenticate and get a new token.
This answer assumes you are using one of the authentication mechanisms such as AuthSub which require you to "sign" requests. Embedded in this signature is a timestamp. If you computer does not have the correct time it will fail.
With this said, could it be a Daylight Savings issue? Does the computer you are running it on have the "Windows Patch" to address the changes when daylight savings takes place (more info).
I had a problem like this too just this past weekend. I think modifying the "Automatically adjust clock for Daylight Saving Time" setting fixed it.
Maybe when you restarted the Service "corrected" itself for the time.
Related
Im wondering if anyone has any answers to this. If one follows the quickstart here:
https://developers.google.com/calendar/api/quickstart/go
we can get authorization from a user that allows us to do things with their calendar.
It seems the HTTP Client given back from the config handles calling the refresh token and keeping the client up to date.
First, as a side-question, does anyone know how long this is allowed to persist? I read in a doc somewhere that for unpublished apps/projects its 7 days, but it didn't specify a timeline otherwise. Is it indefinite or will this expire at some point (ie the refresh token runs out after 30 days). And is this avoidable at all?
now, the real question - if you take the service down entirely, ie for an update, and restart it, is there any way to opick up the authorization again? In the tutorial its writing a code to disk and reading as needed. In my case I just hold reference to the things I need in the application, so obviously when the service went down id have to reauthorize. I then tried writing the code to disk on a persistent volume and re-reading from that, but it doesnt get authorized - I guess because the new instance of the service has a new instance of the oauth config, and it wont accept the old code perhaps?
In any case, im wondering if anyone knows of a way to make this persist through ssytem restarts, or if the only real option here is to move the calendar service client to a small microservice on the side that we do not restart, and contact it from elsewhere as needed.
Thanks in advance!
The refresh token expiration is documented here. As you've noted, tokens expire in 7 days for projects in "Testing" status. Refresh tokens for projects in production last indefinitely, but there are some scenarios where they may get invalidated (user revokes access, token has not been used in 6 months, the account has over 50 live tokens, etc.). It's not mentioned but if you change your OAuth scopes you'll probably need to reauthorize as well.
This means that as long as you keep your refresh tokens and none of the invalidation conditions are met, you should be able to keep using them to get new access tokens even if your service goes down temporarily. But as mentioned in the documentation, you should anticipate the possibility that the token stops working and redirect the user to authorize again.
My guess is that in your case the refresh token stopped working for some reason (maybe the 7 days limit) and the application just tried to use the same expired token. The Quickstarts in Google's documentation usually handle the creation of the token file and how to reuse it, but they don't include the part where you have an invalid token and need to reauthorize.
References:
Google's OAuth2 Overview
OAuth2 for Web Apps
So, last week my application that connects to Office365 using IMAP suddenly stopped working with Basic Authentication (gee, thanks for the early cut-off with no warning Microsoft). I tried troubleshooting that issue for hours, and could find literally nothing wrong since everything was already configured to work with Basic Auth (and was working). I do know they've been threatening to shut-off Basic Auth for a while now, and with a new looming deadline of October, 2022.
After getting nowhere with why it suddenly stopped working, I finally decided to just bite the bullet and switch to using the XOAUTH2 authentication mechanism. After a day and a half of trial-and-error to put together the right series of incantations to make that work correctly, I got that working. Microsoft's documentation is borderline garbage with conflicting information and confusing examples. That and a complete lack of any reasonable error response from the IMAP server about why authentication failed (just the lovely "NO AUTHENTICATE failed" message).
Note: I am using the Client Credentials flow and the AccessToken is being issued to expire after 1 hour.
So, here's the question:
My app basically loops on IDLE and/or processing new mail. When the AccessToken expires, the server will respond with a "Session invalidated - AccessTokenExpired" and then just close or reset the connection. This does make sense, but I would prefer to keep the connection open and just re-authenticate.
My first idea was just re-issue the "AUTHENTICATE" command 1 minute before the AccessToken expired. Server did not like that and gave me a "Command received in Invalid state." response. Hrmph.
Second idea was 1 minute before the AccessToken expired, issue a "LOGOUT" and then "AUTHENTICATE" again. However, Microsoft's IMAP server responds to the "LOGOUT" with "OK" and then closes the connection... Hrmph.
C: R0001 AUTHENTICATE XOAUTH2
S: +
C: <username:token base64>
S: R0001 OK AUTHENTICATE completed.
C: R0002 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4REV1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS MOVE ID UNSELECT CLIENTACCESSRULES CLIENTNETWORKPRESENCELOCATION BACKENDAUTHENTICATE CHILDREN IDLE NAMESPACE LITERAL+
S: R0002 OK CAPABILITY completed.
C: R0003 SELECT INBOX
... <59 mins later> ...
C: R0021 LOGOUT
S: * BYE Microsoft Exchange Server IMAP4 server signing off.
S: R0021 OK LOGOUT completed.
Connection closed.
I did notice there's a server capability called "BACKENDAUTHENTICATE", but there literally is no documentation that I could find on what the heck that capability is about or used for...
What I would really like to do is just issue something like "AUTHENTICATE" again, but with the new AccessToken and then go back to the IDLE/new mail loop. Or maybe a "REAUTHENTICATE" command (that doesn't currently exist).
And yes, I have read RFC4959, RFC3501, and even RFC2222 (as well as a lot of SO posts).
For now, I have just accepted the fact that the connection is going to close and then I'll turn around and re-open a new one...
I would love to hear if anyone has any other bright ideas.
I am considering re-implementing the application using their Graph API, which might be the slightly better long-term option assuming we stay on Microsoft Office365.
I did like that IMAP was a more generic and broadly accepted mailbox protocol and why I chose it in the first place.
Let me try to answer some of your questions.
I do know they've been threatening to shut-off Basic Auth for a while now, and with a new looming deadline of October, 2022. After getting nowhere with why it suddenly stopped working
Microsoft says that in early 2022, they will pick tenants (using some unpublished criteria) and disable basic authentication for all the chosen protocols except SMTP AUTH for a period of between 12 and 48 hours. SMTP AUTH is excluded because it might affect important operational aspects like multi-functional devices or PowerShell scripts sending updates about a job’s progress. When the period expires, Microsoft will enable basic authentication automatically.
Please refer to this link : https://office365itpros.com/2021/09/24/basic-authentication-exchange/
The Basic Auth has been already disabled for many of my clients, so the above statement is valid.
I am using the Client Credentials flow and the AccessToken is being issued to expire after 1 hour.
Using the Client Credentials flow, you do not get a refresh Token which can be used to get a valid Token without losing/dropping/closing the connection.
Based on the RFC6749 (https://www.rfc-editor.org/rfc/rfc6749#section-4.4.3), a refresh Token should not be included.
As #Max mentioned in his comment, "RFC3501 does not include any ability to reauthenticate an existing connection".
Since you don't want to lose/close the connection, I would recommend to use the
Oauth2 authorization code flow (https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-auth-code-flow), which will give you a refresh Token as well.
Also, I would recommend to use the Microsoft Authentication Library (MSAL) (https://learn.microsoft.com/en-us/azure/active-directory/develop/msal-overview)
The Microsoft Authentication Library (MSAL) enables developers to acquire tokens from the Microsoft identity platform in order to authenticate users and access secured web APIs. It can be used to provide secure access to Microsoft Graph, other Microsoft APIs, third-party web APIs, or your own web API. MSAL supports many different application architectures and platforms including .NET, JavaScript, Java, Python, Android, and iOS.
I've been working to setup Oauth communication for an auto-emailing node.js web app using nodemailer. (I don't wish to use gmail's Less Secure Apps setting).
I've taken steps to get the client id, secret, and refresh token from the oauth playground, and have set up the web app to use a stored refresh token to request new access tokens when it first loads.
It is able to send emails (for about 7 days), then I get error invalid status code 400 on client side, and/or invalid grant on server side.
Going back to google playground and getting another refresh token, then updating it in environment variables, solves this for another week. But I'd like to solve this indefinitely.
I read somewhere "A Google Cloud Platform project with an OAuth consent screen configured for an external user type and a publishing status of 'Testing' is issued a refresh token expiring in 7 days"... so last week I switched the app to "In Production" (at console.cloud.google.com) and tried having it verified with google. This week, the same issue has recurred suggesting that wasn't the right fix, or that it wasn't yet verified with google.
I don't know if this was done correctly, nor do I know if this is the true solution to this expiring/revoked refresh token, or invalid grant.
I've also come across these explanations:
The user has revoked your app's access.
The refresh token has not been used for six months.
The user changed passwords and the refresh token contains Gmail scopes.
The user account has exceeded a maximum number of granted (live) refresh tokens.
The client has reached a limit of 50 refresh tokens per account if it's not a service account.
(I didn't make ANY changes during the week, so...not sure why these would have changed)
Is the issue the refresh token?
Or the status of the application?
Would it be dns/cname/cloudflare server issues?
For those who have the same issue in the future:
It turned out that google verification wasn't necessary.
It seems like the refresh token expiring after a week or 7 days was due to the placement of the oauth2Client.setCredentials() function call and accessToken variable.
Calling setCredentials() and obtaining the access token INSIDE the SendEmail() function (at runtime, just before sending email, rather than at application start/spinup time) seemed like it enabled the code to more dynamically generate the tokens it needed. After 12 days, it still seems like its working so I'd call this a success.
My guess at why it wasn't working before was because setting credentials outside of a function meant that code only ran once on server/application startup. It would then store the obtained access token in a const.
The access token would eventually expire, and even if called again/later inside of a function to obtain a new access token, it would be unable to change the value of a const property/variable, and so the call would inevitably fail after a week when it failed to renew.
Hope this helps anyone else having a similar issue.
My apologies for the run-on sentences.
There are a lot of causes for invalid grant it sounds to me like your refresh token is expiring.
If your project on google developer console is still in testing, has not been moved to published and has not gone though the google application verification process then refresh tokens have a max two week life span after which they will expire which may explain your invalid grant. The thing is there is no official word from google that this is happening its just what a lot of developers are seeing these days.
Another one is with gmail scopes if the user changes their password this will also cause the refresh token to expire.
My Box app (using v2 API) keeps track of when an access token expires, and when the app realizes the access token is about to expire, it request a new access token using the refresh token (the reason is that it simpler to avoid errors than to handle them).
I noticed that lately (this definitely was working properly a few months ago), when my app starts up (after more than 1 hour of non-use), the request for a new access token fails! The interesting thing is that at this time, my app has not attempted to use the now expired access token.
Should my app first do a dummy action with the expired access token maybe as a workaround? Again, all this was working as expected when I converted the app over to V2 API. At that time, the refresh token was valid for 14 days.
Thanks
Peter
I've tried to reproduce it, and I'm not able to. My refresh tokens are all working. It could be that you are getting an error condition on your refresh that you are not catching. There are some cases where your admin can decide that the app you are using is no longer approved for your enterprise, and on refresh, you'll be booted out. Or you may have mis-typed your password enough times that your password is going into "captcha" mode, looking to verify that there's a human, and not a machine on the other end of the wire.
Let me suggest that you log out of the app, and log back in. You're more likely to get presented with the error, since Box will be giving you the auth screen, and Box handles all the weird cases in their OAuth2 screens.
Yet another reason, that for all the pain of implementing Oauth2, it's worth it to get a better experience for your users.
I'm calling into a 3rd party API from my ASP.NET MVC app. This API uses a service/API token that you get by calling a "renew" API. It expires after 5 days.
If you call one of the service's APIs and the token as expired, you get an "api token expired" error and you can call the renew function.
I don't want to do this because it complicates my code by having to always check for the "api token expired" error and have the logic to retry. I'd rather just make sure the renew function gets called "relatively frequently but not on every request".
I do not want to use a timer or chron solution if I can avoid it.
I thought I read somewhere that IIS7+ automatically by default recycles the application instances on a regular basis. If this default is less than 5 days then I can just use Application_Start.
You can just use Application_Start and check that it has been at least x time since the last time you renewed the token, perhaps by storing a DateTime of when the last token was retrieved in a database or file.
You shouldn't have the app automatically starting up just for this.
(my advice would be to use the API in the way it is supposed to be used; you might run into a problem with the API provider if you request it too often, for example.)