How to handle authentication with TIdHTTP - c++builder

Question

I'm using C++Builder XE6 and TIdHTTP to communicate with a REST server in a Windows application.
I need some advice on how to handle authentication.
Given the following code:
#include <IdHTTP.hpp>
#include <IdAuthenticationDigest.hpp>
HTTP = new TIdHTTP(NULL);
HTTP->Request->Username = Username;
HTTP->Request->Password = Password;
HTTP->Request->BasicAuthentication = UseBasicAuthentication;
HTTP->OnAuthorization = AuthRequired;
The variables Username (string), Password (string) and UseBasicAuthentication (bool) are user-configurable parameters.
I don't know in advance if the REST server requires authentication or not, nor which type of authentication it supports (Basic or Digest).
AFAIK, the Username and Password members of TIdHTTPRequest are used in the first try to authenticate with the server.
If the initial authentication fails, the OnAuthorization event handler is triggered to obtain new credentials.
Does TIdHTTP only supply the credentials in response to a 401 response code from the server, or are they always included in the request?
Also, is there a standard dialog box that can be used to prompt the user for credentials?
Update
Also need to set the following:
HTTP->HTTPOptions = HTTP->HTTPOptions << hoInProcessAuth;
Update 2
CredUIPromptForCredentials() can be used to show a credentials dialog box on Windows:
#include <wincred.h>
CREDUI_INFO cui;
cui.cbSize = sizeof(cui);
cui.hwndParent = NULL;
cui.pszMessageText = _T("Your message here");
cui.pszCaptionText = Application->Title.c_str();
cui.hbmBanner = NULL;
TCHAR pszUsername[CREDUI_MAX_USERNAME_LENGTH + 1] = _T("Username");
TCHAR pszPassword[CREDUI_MAX_PASSWORD_LENGTH + 1] = _T("Password");
BOOL fSave = false;
if (CredUIPromptForCredentials(&cui, _T("https://www.stackoverflow.com"), NULL, 0, pszUsername, ARRAYSIZE(pszUsername), pszPassword, ARRAYSIZE(pszPassword), &fSave, CREDUI_FLAGS_DO_NOT_PERSIST | CREDUI_FLAGS_SHOW_SAVE_CHECK_BOX | CREDUI_FLAGS_ALWAYS_SHOW_UI | CREDUI_FLAGS_GENERIC_CREDENTIALS) == NO_ERROR)
...
The pszUsername and pszPassword parameters return the entered credentials as plaintext.
The state of the 'Remember my credentials' checkbox is returned in the fSave parameter.
Update 3
I tried the following to test Basic Authentication:
#include <IdHTTP.hpp>
#include <IdAllAuthentications.hpp>
void __fastcall TForm1::Button1Click(TObject *Sender)
{
TIdHTTP* HTTP = new TIdHTTP(NULL);
try
{
HTTP->HTTPOptions = HTTP->HTTPOptions << hoInProcessAuth << hoNoProtocolErrorException;
HTTP->Request->BasicAuthentication = false;
HTTP->MaxAuthRetries = 10;
HTTP->OnAuthorization = AuthRequired;
HTTP->Get("http://httpbin.org/basic-auth/user/pass");
Memo->Lines->Add(IntToStr(HTTP->ResponseCode));
}
__finally
{
delete HTTP;
}
}
void __fastcall TForm1::AuthRequired (TObject *Sender, TIdAuthentication *Authentication, bool &Handled)
{
TIdHTTP* HTTP = dynamic_cast<TIdHTTP*>(Sender);
String Username = Authentication->Username;
String Password = Authentication->Password;
String Server = HTTP->URL->Protocol + "://" + HTTP->URL->Host;
Handled = PromptForCredentials(Application->Title, "Enter credentials for " + Server + ".", Server, Username, Password);
Authentication->Username = Username;
Authentication->Password = Password;
}
I wrapped the code in Update 2 in a function called PromptForCredentials(), which returns true when the user chose OK.
TIdHTTP triggers the OnAuthorization event only once.
If I enter the correct credentials, I get a 200 response, which is correct.
If I enter incorrect credentials, I get a 401 response without triggering the OnAuthorization event again.
After the HTTP->Get() call, the value of HTTP->Request->BasicAuthentication changed to true, which is correct.
Shouldn't TIdHTTP keep triggering OnAuthorization until either a 200 response, Handled is set to false or HTTP->MaxAuthRetries is reached?
PS: My Indy version is 10.6.0.5122.
Update 4
To test Digest Authentication, I changed the following line in the code listed under Update 3:
HTTP->Get("http://httpbin.org/digest-auth/auth/user/pass");
OnAuthorization is also triggered only once. This time, HTTP->Request->BasicAuthentication remains false, which is correct.
Update 5
I've set HTTP->MaxAuthRetries = 3 and added file logging:
HTTP->Intercept = new TIdLogFile(HTTP);
static_cast<TIdLogFile*>(HTTP->Intercept)->Filename = "Project1.log";
static_cast<TIdLogFile*>(HTTP->Intercept)->Active = true;
If I enter incorrect credentials (Username = test, Password = test) in the dialog box when OnAuthorization is triggered, the logs show that TIdHTTP replies to the 401 response 3 times with the same credentials. The OnAuthorization is not triggered again after the first one, for both Basic Autentication and Digest Authentication.
The log for Basic Authentication:
Stat Connected.
Sent 31-03-2022 08:58:55: GET /basic-auth/user/pass HTTP/1.1<EOL>Host: httpbin.org<EOL>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<EOL>Accept-Encoding: identity<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL><EOL>
Recv 31-03-2022 08:58:55: HTTP/1.1 401 UNAUTHORIZED<EOL>Date: Thu, 31 Mar 2022 06:58:55 GMT<EOL>Content-Length: 0<EOL>Connection: keep-alive<EOL>Server: gunicorn/19.9.0<EOL>WWW-Authenticate: Basic realm="Fake Realm"<EOL>Access-Control-Allow-Origin: *<EOL>Access-Control-Allow-Credentials: true<EOL><EOL>
Sent 31-03-2022 08:59:03: GET /basic-auth/user/pass HTTP/1.1<EOL>Host: httpbin.org<EOL>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<EOL>Accept-Encoding: identity<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL>Authorization: Basic dGVzdDp0ZXN0<EOL><EOL>
Recv 31-03-2022 08:59:03: HTTP/1.1 401 UNAUTHORIZED<EOL>Date: Thu, 31 Mar 2022 06:59:03 GMT<EOL>Content-Length: 0<EOL>Connection: keep-alive<EOL>Server: gunicorn/19.9.0<EOL>WWW-Authenticate: Basic realm="Fake Realm"<EOL>Access-Control-Allow-Origin: *<EOL>Access-Control-Allow-Credentials: true<EOL><EOL>
Sent 31-03-2022 08:59:03: GET /basic-auth/user/pass HTTP/1.1<EOL>Host: httpbin.org<EOL>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<EOL>Accept-Encoding: identity<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL>Authorization: Basic dGVzdDp0ZXN0<EOL><EOL>
Recv 31-03-2022 08:59:03: HTTP/1.1 401 UNAUTHORIZED<EOL>Date: Thu, 31 Mar 2022 06:59:03 GMT<EOL>Content-Length: 0<EOL>Connection: keep-alive<EOL>Server: gunicorn/19.9.0<EOL>WWW-Authenticate: Basic realm="Fake Realm"<EOL>Access-Control-Allow-Origin: *<EOL>Access-Control-Allow-Credentials: true<EOL><EOL>
Sent 31-03-2022 08:59:03: GET /basic-auth/user/pass HTTP/1.1<EOL>Host: httpbin.org<EOL>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<EOL>Accept-Encoding: identity<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL>Authorization: Basic dGVzdDp0ZXN0<EOL><EOL>
Recv 31-03-2022 08:59:03: HTTP/1.1 401 UNAUTHORIZED<EOL>Date: Thu, 31 Mar 2022 06:59:03 GMT<EOL>Content-Length: 0<EOL>Connection: keep-alive<EOL>Server: gunicorn/19.9.0<EOL>WWW-Authenticate: Basic realm="Fake Realm"<EOL>Access-Control-Allow-Origin: *<EOL>Access-Control-Allow-Credentials: true<EOL><EOL>
Stat Disconnected.
Stat Disconnected.
Note that dGVzdDp0ZXN0 (Base64) decodes to test:test.
The log for Digest Authentication:
Stat Connected.
Sent 31-03-2022 09:28:41: GET /digest-auth/auth/user/pass HTTP/1.1<EOL>Host: httpbin.org<EOL>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<EOL>Accept-Encoding: identity<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL><EOL>
Recv 31-03-2022 09:28:41: HTTP/1.1 401 UNAUTHORIZED<EOL>Date: Thu, 31 Mar 2022 07:28:41 GMT<EOL>Content-Type: text/html; charset=utf-8<EOL>Content-Length: 0<EOL>Connection: keep-alive<EOL>Server: gunicorn/19.9.0<EOL>WWW-Authenticate: Digest realm="me#kennethreitz.com", nonce="83f09ed264f5704d1cfc630d90b3091d", qop="auth", opaque="d075bef19d3e723ddf38e585e4c15142", algorithm=MD5, stale=FALSE<EOL>Set-Cookie: stale_after=never; Path=/<EOL>Set-Cookie: fake=fake_value; Path=/<EOL>Access-Control-Allow-Origin: *<EOL>Access-Control-Allow-Credentials: true<EOL><EOL>
Sent 31-03-2022 09:28:46: GET /digest-auth/auth/user/pass HTTP/1.1<EOL>Host: httpbin.org<EOL>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<EOL>Accept-Encoding: identity<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL>Authorization: Digest username="test", realm="me#kennethreitz.com", nonce="83f09ed264f5704d1cfc630d90b3091d", algorithm="MD5", uri="/digest-auth/auth/user/pass", qop="auth", nc=00000001, cnonce="f73bb234744d902a6909b0d511c6eab4", response="a7e13d327c9c13a446413530cfc44fc9", opaque="d075bef19d3e723ddf38e585e4c15142"<EOL>Cookie: fake=fake_value; stale_after=never<EOL><EOL>
Recv 31-03-2022 09:28:47: HTTP/1.1 401 UNAUTHORIZED<EOL>Date: Thu, 31 Mar 2022 07:28:47 GMT<EOL>Content-Type: text/html; charset=utf-8<EOL>Content-Length: 0<EOL>Connection: keep-alive<EOL>Server: gunicorn/19.9.0<EOL>WWW-Authenticate: Digest realm="me#kennethreitz.com", nonce="be0945b8e925b386b3f493058d757aeb", qop="auth", opaque="f0301998fadceaf2d562e3b4934da438", algorithm=MD5, stale=FALSE<EOL>Set-Cookie: stale_after=never; Path=/<EOL>Set-Cookie: last_nonce=83f09ed264f5704d1cfc630d90b3091d; Path=/<EOL>Set-Cookie: fake=fake_value; Path=/<EOL>Access-Control-Allow-Origin: *<EOL>Access-Control-Allow-Credentials: true<EOL><EOL>
Sent 31-03-2022 09:28:47: GET /digest-auth/auth/user/pass HTTP/1.1<EOL>Host: httpbin.org<EOL>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<EOL>Accept-Encoding: identity<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL>Authorization: Digest username="test", realm="me#kennethreitz.com", nonce="be0945b8e925b386b3f493058d757aeb", algorithm="MD5", uri="/digest-auth/auth/user/pass", qop="auth", nc=00000001, cnonce="fe65a7188cc197c3ee68c1e2840bab9a", response="2cfce7fc3c9099430841cab1b10eb706", opaque="f0301998fadceaf2d562e3b4934da438"<EOL>Cookie: fake=fake_value; stale_after=never; last_nonce=83f09ed264f5704d1cfc630d90b3091d<EOL><EOL>
Recv 31-03-2022 09:28:47: HTTP/1.1 401 UNAUTHORIZED<EOL>Date: Thu, 31 Mar 2022 07:28:47 GMT<EOL>Content-Type: text/html; charset=utf-8<EOL>Content-Length: 0<EOL>Connection: keep-alive<EOL>Server: gunicorn/19.9.0<EOL>WWW-Authenticate: Digest realm="me#kennethreitz.com", nonce="6cffc5d099ca89fffc766509726ecd1e", qop="auth", opaque="dea9f785474babae87a69b120be54038", algorithm=MD5, stale=FALSE<EOL>Set-Cookie: stale_after=never; Path=/<EOL>Set-Cookie: last_nonce=be0945b8e925b386b3f493058d757aeb; Path=/<EOL>Set-Cookie: fake=fake_value; Path=/<EOL>Access-Control-Allow-Origin: *<EOL>Access-Control-Allow-Credentials: true<EOL><EOL>
Sent 31-03-2022 09:28:47: GET /digest-auth/auth/user/pass HTTP/1.1<EOL>Host: httpbin.org<EOL>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<EOL>Accept-Encoding: identity<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL>Authorization: Digest username="test", realm="me#kennethreitz.com", nonce="6cffc5d099ca89fffc766509726ecd1e", algorithm="MD5", uri="/digest-auth/auth/user/pass", qop="auth", nc=00000001, cnonce="fe65a7188cc197c3ee68c1e2840bab9a", response="e425bf56ca44430aa8f864fb9e4b10a7", opaque="dea9f785474babae87a69b120be54038"<EOL>Cookie: fake=fake_value; stale_after=never; last_nonce=be0945b8e925b386b3f493058d757aeb<EOL><EOL>
Recv 31-03-2022 09:28:47: HTTP/1.1 401 UNAUTHORIZED<EOL>Date: Thu, 31 Mar 2022 07:28:47 GMT<EOL>Content-Type: text/html; charset=utf-8<EOL>Content-Length: 0<EOL>Connection: keep-alive<EOL>Server: gunicorn/19.9.0<EOL>WWW-Authenticate: Digest realm="me#kennethreitz.com", nonce="b448581a59e0fec2df8f3f9f8340721b", qop="auth", opaque="049d8958d842b80beee17b21d04e5b7e", algorithm=MD5, stale=FALSE<EOL>Set-Cookie: stale_after=never; Path=/<EOL>Set-Cookie: last_nonce=6cffc5d099ca89fffc766509726ecd1e; Path=/<EOL>Set-Cookie: fake=fake_value; Path=/<EOL>Access-Control-Allow-Origin: *<EOL>Access-Control-Allow-Credentials: true<EOL><EOL>
Stat Disconnected.
Stat Disconnected.
If I set HTTP->Request->Username = "test" and HTTP->Request->Password = "test" prior to the call to HTTP->Get(...), the OnAuthorization event handler is never triggered and the logs also show that TIdHTTP retries 3 times with the incorrect credentials.

Answer 1

I don't know in advance if the REST server requires authentication or not, nor which type of authentication it supports (Basic or Digest).
TIdHTTP natively supports Basic when Request->Authentication is NULL and Request->BasicAuthentication is true.
TIdHTTP also supports Digest, however you have to enable it manually first, by adding one of the following statements to your code:
#include <IdAuthenticationDigest.hpp>
#include <IdAllAuthentications.hpp>
(#pragma link is handled for you)
Internally, TIdHTTP will then pick the appropriate TIdAuthentication-derived class based on what the server asks for in a 401 response.
AFAIK, the Username and Password members of TIdHTTPRequest are used in the first try to authenticate with the server. If the initial authentication fails, the OnAuthorization event handler is triggered to obtain new credentials.
Correct.
Does TIdHTTP only supply the credentials in response to a 401 response code from the server, or are they always included in the request?
That depends.
If Request->Authentication is assigned a TIdAuthentication-derived object, it is queried for pending credentials, and if any are given then they are sent to the server. For example, in multi-step authentications like NTLM, etc.
Otherwise, if Request->Authentication is NULL and Request->BasicAuthentication is true, Request->Authentication is set to TIdBasicAuthentication, and then Basic credentials are sent to the server (even if the UserName and Password are blank! See #403 in Indy's issue tracker).
Otherwise, no credentials are sent to the server, unless you provide your own credentials in the Request->CustomHeaders property.
UPDATE
TIdHTTP triggers the OnAuthorization event only once. If I enter the correct credentials, I get a 200 response, which is correct. If I enter incorrect credentials, I get a 401 response without triggering the OnAuthorization event again.
TIdHTTP is supposed to trigger the OnAuthorization event each time it receives a 401 response, unless either:
TIdHTTP.AuthRetries reaches TIdHTTP.MaxAuthRetries
the TIdHTTP.OnSelectAuthorization event sets the AuthenticationClass parameter to nil
However, if neither of those conditions are met, the OnAuthorization event is triggered only when the current Request->Authentication object returns wnAskTheProgram from its Next() method, which is called each time 401 is received (after the above conditions are checked).
In the case of TIdBasicAuthentication, its Next() method returns wnAskTheProgram only when the current Username is blank. This is probably a bug that needs to be fixed (I have opened a ticket in Indy's issue tracker for you). The failed Username/Password should be reset on each 401 response. And there is actually code to that effect in IdHTTP.pas, but it is currently commented out because apparently it breaks multi-step authentications, like SSPI/NTLM.
After the HTTP->Get() call, the value of HTTP->Request->BasicAuthentication changed to true, which is correct.
It really shouldn't. But after OnAuthorization is triggered the 1st time, BasicAuthentication is hard-coded to true. I don't know why that was ever added to TIdHTTP.
Shouldn't TIdHTTP keep triggering OnAuthorization until either a 200 response, Handled is set to false or HTTP->MaxAuthRetries is reached?
It should, yes.
If I enter incorrect credentials (Username = test, Password = test) in the dialog box when OnAuthorization is triggered, the logs show that TIdHTTP replies to the 401 response 3 times with the same credentials. The OnAuthorization is not triggered again after the first one, for both Basic Autentication and Digest Authentication.
They are both affected by the same underlying issue - that the current Username is not being cleared after each failed attempt.