I'm building a simple commandline application in Java, that logs into my email box (IMAP) and downloads all attachments. I used basic authentication, but Microsoft is in the process of disabling that so I try to convert my application to use OAuth instead.
After reading on the different OAuth flows, it seems that for my simple standalone commandline application, where there is no problem to simply hardcode a password, the Resource Owner Password Credentials Grand (as described here) would be the best (or a good) choice. I further based myself on the instructions from this source where it is described how to enable OAuth using recent versions of Javamail.
Putting it all together seems a bit harder, and I keep getting AUTHENTICATE Failed errors.
So, what did I try? I first retrieve my authorization token as follows:
public String getAuthToken() {
try {
CloseableHttpClient client = HttpClients.createDefault();
HttpPost loginPost = new HttpPost("https://login.microsoftonline.com/organizations/oauth2/v2.0/token");
String clientId = "some client UUID";
String scopes = "email openid IMAP.AccessAsUser.All offline_access";
String client_secret = "My client secret, not base64 encoded";
String username = "my emailadress";
String password = "my password, not base64 encoded";
String encodedBody = "client_id=" + clientId
+ "&scope=" + scopes
+ "&client_secret=" + client_secret
+ "&username=" + username
+ "&password=" + password
+ "&grant_type=password";
loginPost.setEntity(new StringEntity(encodedBody, ContentType.APPLICATION_FORM_URLENCODED));
loginPost.addHeader(new BasicHeader("cache-control", "no-cache"));
CloseableHttpResponse loginResponse = client.execute(loginPost);
byte[] response = loginResponse.getEntity().getContent().readAllBytes();
ObjectMapper objectMapper = new ObjectMapper();
JavaType type = objectMapper.constructType(objectMapper.getTypeFactory()
.constructParametricType(Map.class, String.class, String.class));
Map<String, String> parsed = new ObjectMapper().readValue(response, type);
return parsed.get("access_token");
} catch (Exception e) {
e.printStackTrace();
return null;
}
The response from the oauth service is actually a json-object which contains following fields:
Obviously the tokens are much longer, but are not shared here. The access_token itself is in the form of three base64 encoded strings seperated by a . The first, when decoded contains
{
"typ": "JWT",
"nonce": "Vobb8bI7E...",
"alg": "RS256",
"x5t": "2ZQpJ3Up...",
"kid": "2ZQpJ3Up..."
}
the second part is a larger object, containing following fields (redacted as well):
{
"aud": "someuuid",
"iss": "https://sts.windows.net/someuuid/",
"iat": 1658397625,
"nbf": 1658397625,
"exp": 1658402597,
"acct": 0,
"acr": "1",
"aio": "ASQ....",
"amr": [
"pwd"
],
"app_displayname": "myapp",
"appid": "some uuid",
"appidacr": "1",
"family_name": "My Last Name",
"given_name": "My First Name",
"idtyp": "user",
"ipaddr": "some.ip.address.here",
"name": "My Full name",
"oid": "someuuid",
"platf": "14",
"puid": "10032...",
"rh": "0.AToA....",
"scp": "email IMAP.AccessAsUser.All openid profile",
"sub": "enaKK...",
"tenant_region_scope": "EU",
"tid": "someuuid",
"unique_name": "my email",
"upn": "my email",
"uti": "1cc...",
"ver": "1.0",
"wids": [
"some uuid",
"some uuid"
],
"xms_st": {
"sub": "02n7h..."
},
"xms_tcdt": 1571393936
}
The last part is just binary data. I currenly simply pass on the entire access_token as I receive it to JavaMail as follows:
String accesstoken = new OauthTokenFetcher().getAuthToken();
imapReader = new ImapMailBoxReader(
"outlook.office365.com",
"my email",
accesstoken);
LocalDate startDate = LocalDate.of(2022,4,1);
LocalDate endDate = LocalDate.of(2022,7,1);
imapReader.processOnMessages("Inbox", startDate, endDate,this::processMessage);
with ImapMailBoxReader as follows:
public class ImapMailBoxReader {
private String host;
private String username;
private String password;
public ImapMailBoxReader(String host, String username, String password) {
this.host = host;
this.username = username;
this.password = password;
}
public void processOnMessages(String folder, LocalDate since, LocalDate until, Consumer<Message> mailconsumer) {
try {
System.out.println("Password:" + password);
Properties prop = new Properties();
MailSSLSocketFactory sf = new MailSSLSocketFactory();
sf.setTrustAllHosts(true);
prop.put("mail.debug.auth", "true");
prop.put("mail.imap.sasl.enable", "true");
prop.put("mail.imap.sasl.mechanisms", "XOAUTH2");
prop.put("mail.imap.auth.login.disable", "true");
prop.put("mail.imap.auth.plain.disable", "true");
prop.put("mail.imap.ssl.enable", "true");
// Create the session
//Connect to the server
Session session = Session.getDefaultInstance(prop, null);
session.setDebug(true);
Store store = session.getStore("imap");
store.connect(host, username, password);
//open the inbox folder
Folder inbox = store.getFolder(folder);
inbox.open(Folder.READ_ONLY);
Message[] messages;
if (since != null) {
Date startDate = Date.from(since.atStartOfDay(ZoneId.systemDefault()).toInstant());
SearchTerm newerThan = new ReceivedDateTerm(ComparisonTerm.GE, startDate);
if (until != null) {
Date endDate = Date.from(until.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant());
SearchTerm olderThan = new ReceivedDateTerm(ComparisonTerm.LT, endDate);
SearchTerm both = new AndTerm(olderThan, newerThan);
messages = inbox.search(both);
} else {
messages = inbox.search(newerThan);
}
} else if (until != null) {
Date endDate = Date.from(until.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant());
SearchTerm olderThan = new ReceivedDateTerm(ComparisonTerm.LT, endDate);
messages = inbox.search(olderThan);
} else {
messages = inbox.getMessages();
}
for (Message m: messages) {
mailconsumer.accept(m);
}
inbox.close(false);
store.close();
} catch (Exception e) {
e.printStackTrace();
}
}
The above statement fails at the store.connect statement with AUTHENTICATE FAILED.
I probably pass on the token incorrectly? The JavaMail documentation above states that I should not Base64 encode the token, but I received it as such. Am I supposed to send only part of it? Which part then?
Any help would be appreciated.
With a tip from a co-worker without a stackoverflow account, I finally got it to work. The key was that the scopes I used for the OAuth token, are apparantly not allowed for an application. This bit of information is hidden at the bottom of this page.
Summarized, the solution is:
You must configure the IMAP.AccessAsApp permission, instead of IMAP.AccessAsUser.All . This permission can not be found in the same place as the AccessAsUser.All permission, but is hidden under "Office 365 Exchange Online" permissions.
Unlike what you would expect, you must use the https://outlook.office365.com/.default scope in the body payload for the access token request.
That did the trick. It is ridiculous how much difficulty I had to find that information in the documentation pages using search engines.
Please check if you have enabled Allow public client flows setting in azure app registrations. This is required for ROPC as description says.
Allow public client flows
This is under App Registrations -> Overview-> redirect URIs.
I had a similar situation as the OP. I had a Java (swing) standalone app that reads and processes specific messages from the inbox. The app has been running for a couple of years using basic auth. Recently (Early October), Microsoft disable basic auth. I had to scramble to figure out OAUTH. This post got me very close; HOWEVER, the final answer did not work for me.
Getting the access token was easy enough, but authentication ALWAYS failed. Obviously, the permissions/grants for my token were not correct.
My solution:
Configure Microsoft Graph permissions IMAP.AccessAsUser.All . (opposite of OP's solution)
Scope - "email openid https://outlook.office.com/IMAP.AccessAsUser.All"
Everything else pretty much exactly followed OP.
Hope this might help another lost soul.
I am using the following code. I have a valid accessToken. How can I set the scope to email,profile on the Credential? I am using OAuth2 for authorization.
static final String APPLICATION_NAME = "calendar-service";
static final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
Credential credential = new
Credential(BearerToken.authorizationHeaderAccessMethod()).setAccessToken(accessToken);
// Build a new authorized API client service.
final NetHttpTransport HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
PeopleService service = new PeopleService.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME)
.build();
ListConnectionsResponse response = service.people().connections()
.list("people/me")
.setPersonFields("names,emailAddresses")
.execute();
List<Person> connections = response.getConnections();
if (connections != null && connections.size() > 0) {
for (Person person : connections) {
List<Name> names = person.getNames();
if (names != null && names.size() > 0) {
System.out.println("Name: " + person.getNames().get(0)
.getDisplayName());
} else {
System.out.println("No names available for connection.");
}
}
} else {
System.out.println("No connections found.");
}
return connections.get(0);
I am getting the following error:
GET https://people.googleapis.com/v1/people/me/connections?personFields=names,emailAddresses
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Insufficient Permission",
"reason" : "insufficientPermissions"
} ],
"message" : "Request had insufficient authentication scopes.",
"status" : "PERMISSION_DENIED"
}] with root cause
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
Here is my application.properties:
spring.security.oauth2.client.registration.google.clientId=XYZ456.apps.googleusercontent.com
spring.security.oauth2.client.registration.google.clientSecret=ABC123
spring.security.oauth2.client.registration.google.redirectUri={baseUrl}/oauth2/callback/{registrationId}
spring.security.oauth2.client.registration.google.scope=email,profile
Please advise.
There is a quickstart for using People API with Java
It contains the following line:
private static final List<String> SCOPES = Arrays.asList(PeopleServiceScopes.CONTACTS_READONLY);
This is the line where you can change your scope to a ddiferent one if desired, or add additional scopes.
To know which scopes you need, go onto the documentation page for the method you use.
IMPORTANT
Each time you change the scopes, you need to delete the token file from your machine, so that new authorization flow willbe triggered.
according to documentation we may use the following endpoints for fetching sensitivity labels:
/me/informationProtection/policy/labels (using delegated permissions)
/informationProtection/policy/labels (using application permission. App should have InformationProtectionPolicy.Read.All permission to use this end point)
The following C# code uses app permissions and it works on tenant1:
static void Main(string[] args)
{
string accessToken = getTokenImpl().Result;
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
client.DefaultRequestHeaders.Add("Accept", "application/json");
client.DefaultRequestHeaders.Add("User-Agent", "PostmanRuntime/7.24.1");
using (var response = client.GetAsync($"https://graph.microsoft.com/beta/informationprotection/policy/labels").Result)
{
using (var content = response.Content)
{
string result = content.ReadAsStringAsync().Result;
if (response.IsSuccessStatusCode)
{
Console.WriteLine(result);
}
}
}
}
}
private static async Task<string> getTokenImpl()
{
string clientId = "...";
string clientSecret = "...";
string tenant = "{...}.onmicrosoft.com";
string authority = string.Format("https://login.microsoftonline.com/{0}", tenant);
var authContext = new AuthenticationContext(authority);
var creds = new ClientCredential(clientId, clientSecret);
var authResult = await authContext.AcquireTokenAsync("https://graph.microsoft.com/", creds);
return authResult.AccessToken;
}
But it doesn't work on another tenant2 - there it always returns 404 "The resource could not be found" with the following inner exception "User not found to have labels, policy is empty". Here is full response:
{
"error": {
"code": "itemNotFound",
"message": "The resource could not be found.",
"innerError": {
"code": "notFound",
"message": "User not found to have labels, policy is empty",
"target": "userId",
"exception": null,
"date": "2020-11-18T09:29:20",
"request-id": "657ad51c-9cab-49f2-a242-50929cdc6950",
"client-request-id": "657ad51c-9cab-49f2-a242-50929cdc6950"
}
}
}
Interesting that attempt to call endpoint /me/informationProtection/policy/labels with delegated permissions on the same tenant2 gives the same error, but on tenant1 it also works. Did anybody face with this problem or have idea why it may happen? Need to mention that on tenant2 earlier we created and published several sensitivity labels for specific user - this user doesn't have neither O365 license nor Azure subscription. I.e. when you try to login to SPO/Azure and create site/group - sensitivity labels were not shown at all for this user. We tried to remove these sensitivity labels and their policies with audience targeting to this user, but both end points still return error.
PS. AAD app is Ok on tenant2 - it has InformationProtectionPolicy.Read.All permission and admin consent is granted:
Update 2020-11-25: behavior has been changed on both tenants without any change from our side: now on both tenants we get 502 Bad Gateway. Does MS rolls out this functionality globally now? Here is response which we get now from /beta/me/informationProtection/policy/labels:
{
"error":{
"code":"UnknownError",
"message":"<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>Microsoft-Azure-Application-Gateway/v2</center>\r\n</body>\r\n</html>\r\n",
"innerError":{
"date":"2020-11-25T12:59:51",
"request-id":"93557ae1-b0d9-44a9-bbea-871f18e379ea",
"client-request-id":"93557ae1-b0d9-44a9-bbea-871f18e379ea"
}
}
}
Update 2020-12-07: it started to work by its own. I.e. MS has fixed that on backend side somehow for the tenant when this issue was reproduced.
I've created a project in Big Query and within API Access pane, I've created Service Account so that i can make Big Query API accessible through windows application on behalf of user interaction. Since i am new to using and accessing Google API, I want to know the basic steps so that i can access Big Query via Windows Service.
There are some .NET code samples here using web-based OAuth flows:
Google BigQuery with .NET documentation/ samples
For an example from another dev using service accounts in .NET, see this:
Google OAuth2 Service Account Access Token Request gives 'Invalid Request' Response
These manually implement service accounts on top of the .NET library. The .NET library has recently added native support for service accounts, though I haven't yet found an official example.
Here's an unofficial sample working with the Analytics API. It should be directly analogous to using service accounts with BigQuery:
How do I use a Service Account to Access the Google Analytics API V3 with .NET C#?
Finally here's the working code for Authentication process to Big Query API and retrieving records from Big Query table:-
I've created a class having a method of return type OAuth2Authenticator<AssertionFlowClient>.
internal class clsGetOAuth2Authentication
{
public OAuth2Authenticator<AssertionFlowClient> objGetOAuth2(string strPrivateFilePath, string strPrivateFilePassword, string strServiceAccEmailId,string strScope)
{
AuthorizationServerDescription objAuthServerDesc;
X509Certificate2 objKey;
AssertionFlowClient objClient;
OAuth2Authenticator<AssertionFlowClient> objAuth = null;
string ScopeUrl = "https://www.googleapis.com/auth/" + strScope;
string strSrvAccEmailId = strServiceAccEmailId;
string strKeyFile = strPrivateFilePath; //KeyFile: This is the physical path to the key file you downloaded when you created your Service Account.
string strKeyPassword = (strPrivateFilePassword != "") ? strPrivateFilePassword : "notasecret"; //key_pass: This is probably the password for all key files, but if you're given a different one, use that.
objAuthServerDesc = GoogleAuthenticationServer.Description; //objAuthServerDesc: Description of the server that will grant Authentiation.
objKey = new X509Certificate2(strKeyFile, strKeyPassword, X509KeyStorageFlags.Exportable); //objkey: Load up and decrypt the key.
objClient = new AssertionFlowClient(objAuthServerDesc, objKey) { ServiceAccountId = strSrvAccEmailId, Scope = ScopeUrl }; //objClient: Using the AssertionFlowClient, because we're logging in with our certificate.
objAuth = new OAuth2Authenticator<AssertionFlowClient>(objClient, AssertionFlowClient.GetState); //objAuth: Requesting Authentication.
return objAuth;
}
}
Now, another class calls the above method:-
clsGetOAuth2Authentication objOAuth2 = new clsGetOAuth2Authentication();
try
{
var objAuth = objOAuth2.objGetOAuth2(strKeyFile, strKeyPassword, strSrvAccEmailId, strScope); //Authentication data returned
objBQServ = new BigqueryService(objAuth); //Instantiate BigQueryService with credentials(objAuth) as its parameter
#region Retrieving Records:-
JobsResource j = objBQServ.Jobs;
QueryRequest qr = new QueryRequest();
qr.Query = strQuery;
DateTime dtmBegin = DateTime.UtcNow;
QueryResponse response = j.Query(qr, strProjId).Fetch();
DateTime dtmEnd = DateTime.UtcNow;
string strColHead = "";
foreach (var colHeaders in response.Schema.Fields)
{
strColHead += colHeaders.Name.ToString() + "\t";
}
Console.WriteLine(strColHead);
int intCount = 0;
foreach (TableRow row in response.Rows)
{
intCount += 1;
List<string> list = new List<string>();
foreach (var field in row.F)
{
list.Add(field.V);
}
Console.WriteLine(String.Join("\t", list));
}
TimeSpan tsElapsed = dtmEnd - dtmBegin;
Console.WriteLine("\n" + "Total no. of records:- " + intCount + ". Time taken:- " + tsElapsed);
#endregion
}
catch (Exception ex)
{
Console.WriteLine("Error Occured!" + "\n\n" + "Statement:- " + ex.Message.ToString() + "\n\n" + "Description:- " + ex.ToString() + "\n\n");
Console.WriteLine("\nPress enter to exit");
Console.ReadLine();
}
i am getting the same issue as described in this post
. we have used almost exactly the same code. i have tried both with Client ID and Email address of the google service account in below mehotd
setServiceAccountId(GOOGLE_SERVICE_CLIENT_EMAIL) OR
setServiceAccountId(GOOGLE_CLIENT_ID)
error changes with the change in a/c id. if i use client id, error is
400 Bad Request { "error" : "invalid_grant" }
and if i use service email id, error is
401 Unauthorized {
"code" : 401, "errors" : [ {
"domain" : "androidpublisher",
"message" : "This developer account does not own the application.",
"reason" : "developerDoesNotOwnApplication" } ], "message" : "This developer account does not own the application." }
any idea?
There appears to be some evidence that Google Play API does not currently work with Service Accounts (madness). There is another thread on the issue here. You can read about the Google Service Accounts here. You can read about authentication for Android Google Play API here.
Once you have done the dance on the Google API Console to get a refresh_token you can get an access token like this:
private String getAccessToken()
{
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("https://accounts.google.com/o/oauth2/token");
try
{
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(4);
nameValuePairs.add(new BasicNameValuePair("grant_type", "refresh_token"));
nameValuePairs.add(new BasicNameValuePair("client_id", "YOUR_CLIENT_ID);
nameValuePairs.add(new BasicNameValuePair("client_secret", "YOUR_CLIENT_SECRET"));
nameValuePairs.add(new BasicNameValuePair("refresh_token", "YOUR_REFRESH_TOKEN"));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = client.execute(post);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer buffer = new StringBuffer();
for (String line = reader.readLine(); line != null; line = reader.readLine())
{
buffer.append(line);
}
JSONObject json = new JSONObject(buffer.toString());
return json.getString("access_token");
}
catch (IOException e) { e.printStackTrace(); }
catch (JSONException e) { e.printStackTrace(); }
return null;
}