Authorization failure TIdHTTP over HTTPS when password is russian - delphi

Question

I try to test my webservice with the TIdHTTP (Indy 10.6.0 and Delphi XE5) by this code:
GIdDefaultTextEncoding := encUTF8;
HTTP.IOHandler.DefStringEncoding := IndyTextEncoding_UTF8;
Http.Request.UserName := AUser;
Http.Request.Password := APass;
Http.Request.Accept := 'text/javascript';
Http.Request.ContentType := 'application/json';
Http.Request.ContentEncoding := 'utf-8';
Http.Request.URL := 'https://sameService';
Http.MaxAuthRetries := 1;
Http.Request.BasicAuthentication := True;
TIdSSLIOHandlerSocketOpenSSL(HTTP.IOHandler).SSLOptions.Method := sslvSSLv3;
HTTP.HandleRedirects := True;
"AUser" and "APass" in UTF-8. When "APass" have same Russian chars I can't login.
By "HTTP Analyze" I see:
...
Authorization: Basic cDh1c2VyOj8/Pz8/PzEyMw==
Decode from Base 64 (base64decode.org) we can see:
p8user:??????123
Why DefStringEncoding not work ?

Answer 1

TIdHTTP's authentication system has no concept of TIdIOHandler or its DefStringEncoding property.
Internally, TIdBasicAuthentication uses TIdEncoderMIME.Encode(), but without specifying any encoding. TIdEncoder.Encode() defaults to 8bit encoding, and thus is not affected by GIdDefaultTextEncoding.
If you need to send a UTF-8 encoded password with BASIC authentication, you will have to encode the UTF-8 data manually and store the resulting octets into a string, then the 8bit encoder can process the octets as-is, eg:
Http.Request.Password := BytesToStringRaw(IndyTextEncoding_UTF8.GetBytes(APass));
On the other hand, Indy's DIGEST authentication, for instance, uses TIdHashMessageDigest5.HashStringAsHex(), and TIdHash.HashString() does not default to any specific encoding, it depends on GIdDefaultTextEncoding.
So, you have to be careful about how you encode passwords, based on which authentications you use. To account for the discrepency, what you could try is not encode TIdHTTP.Request.Password itself, but instead encode the password inside the TIdHTTP.OnAuthorization event instead when BASIC authentication is being used, eg:
Http.Request.Password := APass;
...
procedure TMyForm.HttpAuthorization(Sender: TObject;
Authentication: TIdAuthentication; var Handled: Boolean);
begin
if Authentication is TIdBasicAuthentication then
begin
Authentication.Password := BytesToStringRaw(IndyTextEncoding_UTF8.GetBytes(TheDesiredPasswordHere));
Handled := True;
end;
end;
UPDATE:
Internally, TIdBasicAuthentication uses TIdEncoderMIME.Encode(), but without specifying any encoding.
That last part is no longer true. TIdBasicAuthentication was updated in 2016 to now pass an encoding to TIdEncoderMIME.Encode(). When an HTTP server asks for BASIC authentication, TIdBasicAuthentication now checks if the server's WWW-Authenticate header includes one of the following attributes: charset, accept-charset, encoding, or enc (in that order). If one is found, the specified charset is passed to Encode(), otherwise ISO-8859-1 is used (there is a TODO in the code to use UTF-8 if the username or password contain any characters that do not exist in ISO-8859-1).
If you want to ensure that UTF-8 is used in BASIC authentication, you are better off setting Request.BasicAuthentication to False and using the Request.CustomHeaders to supply your own Authorization header, eg:
Http.Request.BasicAuthentication := False;
Http.Request.CustomHeaders.Values['Authorization'] := 'Basic ' + TIdEncoderMIME.EncodeString(AUser + ':' + APass, IndyTextEncoding_UTF8);
Alternatively, you might be able to just get away with updating the protected TIdBasicAuthentication.FCharset member inside of the TIdHTTP.OnAuthorization event (which is fired after the server's WWW-Authenticate header has been parsed), eg:
Http.Request.Password := APass;
...
type
TIdBasicAuthenticationAccess = class(TIdBasicAuthentication)
end;
procedure TMyForm.HttpAuthorization(Sender: TObject;
Authentication: TIdAuthentication; var Handled: Boolean);
begin
if Authentication is TIdBasicAuthentication then
begin
TIdBasicAuthenticationAccess(Authentication).FCharset := 'utf-8';
Authentication.Password := TheDesiredPasswordHere;
Handled := True;
end;
end;