JavaMail with Oauth and Office365 - oauth

Question

I'm building a simple commandline application in Java, that logs into my email box (IMAP) and downloads all attachments. I used basic authentication, but Microsoft is in the process of disabling that so I try to convert my application to use OAuth instead.
After reading on the different OAuth flows, it seems that for my simple standalone commandline application, where there is no problem to simply hardcode a password, the Resource Owner Password Credentials Grand (as described here) would be the best (or a good) choice. I further based myself on the instructions from this source where it is described how to enable OAuth using recent versions of Javamail.
Putting it all together seems a bit harder, and I keep getting AUTHENTICATE Failed errors.
So, what did I try? I first retrieve my authorization token as follows:
public String getAuthToken() {
try {
CloseableHttpClient client = HttpClients.createDefault();
HttpPost loginPost = new HttpPost("https://login.microsoftonline.com/organizations/oauth2/v2.0/token");
String clientId = "some client UUID";
String scopes = "email openid IMAP.AccessAsUser.All offline_access";
String client_secret = "My client secret, not base64 encoded";
String username = "my emailadress";
String password = "my password, not base64 encoded";
String encodedBody = "client_id=" + clientId
+ "&scope=" + scopes
+ "&client_secret=" + client_secret
+ "&username=" + username
+ "&password=" + password
+ "&grant_type=password";
loginPost.setEntity(new StringEntity(encodedBody, ContentType.APPLICATION_FORM_URLENCODED));
loginPost.addHeader(new BasicHeader("cache-control", "no-cache"));
CloseableHttpResponse loginResponse = client.execute(loginPost);
byte[] response = loginResponse.getEntity().getContent().readAllBytes();
ObjectMapper objectMapper = new ObjectMapper();
JavaType type = objectMapper.constructType(objectMapper.getTypeFactory()
.constructParametricType(Map.class, String.class, String.class));
Map<String, String> parsed = new ObjectMapper().readValue(response, type);
return parsed.get("access_token");
} catch (Exception e) {
e.printStackTrace();
return null;
}
The response from the oauth service is actually a json-object which contains following fields:
Obviously the tokens are much longer, but are not shared here. The access_token itself is in the form of three base64 encoded strings seperated by a . The first, when decoded contains
{
"typ": "JWT",
"nonce": "Vobb8bI7E...",
"alg": "RS256",
"x5t": "2ZQpJ3Up...",
"kid": "2ZQpJ3Up..."
}
the second part is a larger object, containing following fields (redacted as well):
{
"aud": "someuuid",
"iss": "https://sts.windows.net/someuuid/",
"iat": 1658397625,
"nbf": 1658397625,
"exp": 1658402597,
"acct": 0,
"acr": "1",
"aio": "ASQ....",
"amr": [
"pwd"
],
"app_displayname": "myapp",
"appid": "some uuid",
"appidacr": "1",
"family_name": "My Last Name",
"given_name": "My First Name",
"idtyp": "user",
"ipaddr": "some.ip.address.here",
"name": "My Full name",
"oid": "someuuid",
"platf": "14",
"puid": "10032...",
"rh": "0.AToA....",
"scp": "email IMAP.AccessAsUser.All openid profile",
"sub": "enaKK...",
"tenant_region_scope": "EU",
"tid": "someuuid",
"unique_name": "my email",
"upn": "my email",
"uti": "1cc...",
"ver": "1.0",
"wids": [
"some uuid",
"some uuid"
],
"xms_st": {
"sub": "02n7h..."
},
"xms_tcdt": 1571393936
}
The last part is just binary data. I currenly simply pass on the entire access_token as I receive it to JavaMail as follows:
String accesstoken = new OauthTokenFetcher().getAuthToken();
imapReader = new ImapMailBoxReader(
"outlook.office365.com",
"my email",
accesstoken);
LocalDate startDate = LocalDate.of(2022,4,1);
LocalDate endDate = LocalDate.of(2022,7,1);
imapReader.processOnMessages("Inbox", startDate, endDate,this::processMessage);
with ImapMailBoxReader as follows:
public class ImapMailBoxReader {
private String host;
private String username;
private String password;
public ImapMailBoxReader(String host, String username, String password) {
this.host = host;
this.username = username;
this.password = password;
}
public void processOnMessages(String folder, LocalDate since, LocalDate until, Consumer<Message> mailconsumer) {
try {
System.out.println("Password:" + password);
Properties prop = new Properties();
MailSSLSocketFactory sf = new MailSSLSocketFactory();
sf.setTrustAllHosts(true);
prop.put("mail.debug.auth", "true");
prop.put("mail.imap.sasl.enable", "true");
prop.put("mail.imap.sasl.mechanisms", "XOAUTH2");
prop.put("mail.imap.auth.login.disable", "true");
prop.put("mail.imap.auth.plain.disable", "true");
prop.put("mail.imap.ssl.enable", "true");
// Create the session
//Connect to the server
Session session = Session.getDefaultInstance(prop, null);
session.setDebug(true);
Store store = session.getStore("imap");
store.connect(host, username, password);
//open the inbox folder
Folder inbox = store.getFolder(folder);
inbox.open(Folder.READ_ONLY);
Message[] messages;
if (since != null) {
Date startDate = Date.from(since.atStartOfDay(ZoneId.systemDefault()).toInstant());
SearchTerm newerThan = new ReceivedDateTerm(ComparisonTerm.GE, startDate);
if (until != null) {
Date endDate = Date.from(until.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant());
SearchTerm olderThan = new ReceivedDateTerm(ComparisonTerm.LT, endDate);
SearchTerm both = new AndTerm(olderThan, newerThan);
messages = inbox.search(both);
} else {
messages = inbox.search(newerThan);
}
} else if (until != null) {
Date endDate = Date.from(until.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant());
SearchTerm olderThan = new ReceivedDateTerm(ComparisonTerm.LT, endDate);
messages = inbox.search(olderThan);
} else {
messages = inbox.getMessages();
}
for (Message m: messages) {
mailconsumer.accept(m);
}
inbox.close(false);
store.close();
} catch (Exception e) {
e.printStackTrace();
}
}
The above statement fails at the store.connect statement with AUTHENTICATE FAILED.
I probably pass on the token incorrectly? The JavaMail documentation above states that I should not Base64 encode the token, but I received it as such. Am I supposed to send only part of it? Which part then?
Any help would be appreciated.

Answer 1

With a tip from a co-worker without a stackoverflow account, I finally got it to work. The key was that the scopes I used for the OAuth token, are apparantly not allowed for an application. This bit of information is hidden at the bottom of this page.
Summarized, the solution is:
You must configure the IMAP.AccessAsApp permission, instead of IMAP.AccessAsUser.All . This permission can not be found in the same place as the AccessAsUser.All permission, but is hidden under "Office 365 Exchange Online" permissions.
Unlike what you would expect, you must use the https://outlook.office365.com/.default scope in the body payload for the access token request.
That did the trick. It is ridiculous how much difficulty I had to find that information in the documentation pages using search engines.

Answer 2

Please check if you have enabled Allow public client flows setting in azure app registrations. This is required for ROPC as description says.
Allow public client flows
This is under App Registrations -> Overview-> redirect URIs.

Answer 3

I had a similar situation as the OP. I had a Java (swing) standalone app that reads and processes specific messages from the inbox. The app has been running for a couple of years using basic auth. Recently (Early October), Microsoft disable basic auth. I had to scramble to figure out OAUTH. This post got me very close; HOWEVER, the final answer did not work for me.
Getting the access token was easy enough, but authentication ALWAYS failed. Obviously, the permissions/grants for my token were not correct.
My solution:
Configure Microsoft Graph permissions IMAP.AccessAsUser.All . (opposite of OP's solution)
Scope - "email openid https://outlook.office.com/IMAP.AccessAsUser.All"
Everything else pretty much exactly followed OP.
Hope this might help another lost soul.