Delphi XE8 idHttp Erromessage text coding is wrong - delphi

Question

I am using Delphi XE8 and I am sendig a PUT message via idHttp.
Http.Request.CustomHeaders.Clear;
Http.Request.BasicAuthentication := false;
http.Request.Method := 'PUT';
Http.Request.Accept := '*/*';
Http.Request.ContentType := 'application/json';
http.Request.CustomHeaders.AddValue('apiKey','T_API23207_169');
http.Request.CustomHeaders.AddValue('transactionId','20200924_015');
http.Request.CustomHeaders.AddValue('usziID','1');
Http.Request.AcceptEncoding := '*';
http.Request.CharSet := 'utf-8';
kuldes_header.Text := http.Request.CustomHeaders.Text;
http.Intercept := IdLogEvent1;
IdLogEvent1.Active := true;
jsonToSend := TStringStream.create(json_adat.Text,system.sysUtils.TEncoding.UTF8);
kuldes_body.Lines.LoadFromStream(jsonToSend);
try
try
send_text := http.Put('http://10.109.132.24:8090/rest/usziIroda/1',jsonToSend);
resp := http.ResponseText;
code := http.ResponseCode;
jsonToSend.Position := 0;
except
on E: EIdHTTPProtocolException do
begin
code := e.ErrorCode;
error_message := e.ErrorMessage;
end;
end;
hiba_kod.Lines.Add(IntToStr(code));
valasz_uzenet.Text := send_text;
hiba_uzenet.Text := error_message; enter code here
The returned error message has strange characters:
"Megadott tranzakció azonosítóval már történt API hívás"
But it should be like this:
"Megadott tranzakció azonosí­tóval már történt API hí­vás"
How can I convert the returned message to normal string?
Thank you!

Answer 1

The result you showed - Megadott tranzakció azonosítóval már történt API hívás - is the UTF-8 encoded form of Megadott tranzakció azonosí­tóval már történt API hí­vás being misinterpreted in Latin-1/ISO-8859-1. Which most likely means that the response did not specify a UTF-8 charset in its Content-Type header (since you have an Intercept assigned, you can easily verify this for yourself), so Indy would fall back to a default charset instead.
The original UTF-8 bytes have been decoded and lost before you can access the response data in send_text or error_message. However, since ISO-8859-1 basically has a 1:1 relationship between byte values and Unicode codepoint values, what you can try doing in this specific situation is copy the ErrorMessage's Char values as-is to a RawByteString(65001) or UTF8String, and then let the RTL decode that as UTF-8 back into a proper UTF-16 (Unicode)String, eg:
function DecodeISO88591AsUTF8(const S: string): string;
var
utf8: UTF8String;
I: Integer;
begin
SetLength(utf8, Length(S));
for I := Low(S) to High(S) do
utf8[I] := AnsiChar(S[I]);
Result := string(utf8);
end;
...
error_message := e.ErrorMessage;
//if not TextIsSame(http.Response.CharSet, 'utf-8') then
if TextIsSame(http.Response.CharSet, 'ISO-8859-1') then
error_message := DecodeISO88591AsUTF8(error_message);
Alternatively, you can instead call the overloaded version of TIdHTTP.Put() that fills a response TStream instead of returning a decoded String, and then you can decode the original raw bytes however you want. Just be sure to enable the hoNoProtocolErrorException and hoWantProtocolErrorContent flags in the TIdHTTP.HTTPOptions property so that any error response is stored in the TStream, then you don't need a try/except to handle the EIdHTTPProtocolException separately.
http.HTTPOptions := http.HTTPOptions + [hoNoProtocolErrorException, hoWantProtocolErrorContent];
...
RespStrm := TMemoryStream.Create;
try
http.Put('http://10.109.132.24:8090/rest/usziIroda/1', jsonToSend, RespStrm);
resp := http.ResponseText;
code := http.ResponseCode;
jsonToSend.Position := 0;
RespStrm.Position := 0;
if (code div 100) = 2 then
begin
send_text := decode RespStrm as needed...;
end else
begin
error_message := decode RespStrm as needed...;
end;
finally
RespStrm.Free;
end;
...