I'm using Indy to do a Post to an SMS service that will send the SMS, but the SMS text ends up on my phone with %20 instead of spaces, here is the code:
url,text:string;
IdHTTP1: TIdHTTP;
IdSSLIOHandlerSocketOpenSSL2: TIdSSLIOHandlerSocketOpenSSL;
begin
IdSSLIOHandlerSocketOpenSSL2 := TIdSSLIOHandlerSocketOpenSSL.Create;
IdHTTP1 := TIdHTTP.Create;
IdSSLIOHandlerSocketOpenSSL2.SSLOptions.Method := sslvSSLv23;
IdHTTP1.IOHandler := IdSSLIOHandlerSocketOpenSSL2;
IdHTTP1.HandleRedirects := true;
IdHTTP1.ReadTimeout := 5000;
param:=TStringList.create;
param.Clear;
param.Add('action=create');
param.Add('token=' + SMSToken);
param.Add('to=' + Phone);
param.Add('msg=' + MessageText);
url:='https://api.tropo.com/1.0/sessions';
try
text:=IdHTTP1.Post(url, param);
thanks
The TStrings version of TIdHTTP.Post() sends an application/x-www-form-urlencoded request to the server. The posted data is url-encoded by default. The server needs to decode the posted data before processing it. It sounds like the server-side code is not doing that correctly. You can remove the hoForceEncodeParams flag from the TIdHTTP.HTTPOptions property to disable the url-encoding of the posted data, but I would advise you to report the bug to Tropo instead so they can fix their server-side code.
TIdHTTP itself does not apply quoted-printable encoding to posted data, so the data being posted has to be quoted-printable encoded beforehand.
In Indy 10, you can use the TIdFormDataField.Charset property to specify how strings are converted to bytes, and then use the TIdFormDataField.ContentTransfer property to specify how the bytes are encoded. For the ContentTransfer, you can specify '7bit', '8bit', 'binary', 'quoted-printable', 'base64', or a blank string (which is equivilent to '7bit', but without stating as much in the MIME header).
Set the TIdFormDataField.CharSet property to a charset that matches what your OS is using, and then set the TIdFormDataField.ContentTransfer property to '8bit'.
Alternatively, use the TStream overloaded version of TIdMultipartFormDataStream.AddFormField() instead of the String overloaded version, then you can store data in your input TStream any way you wish and it will be encoded as-is based on the value of the TIdFormDataField.ContentTransfer property. This should remove the %20 you are getting.
Related
I'm sending a number of parameters to an API using the TIdHTTP.Get() method.
I pull values for the actual API parameters from string variables or component Text properties (like a ComboBox, for example). Everything is fine until any of those values contains a space.
For example, one of the parameters is a full name field (example: 'John Smith')
Since it contains a space between the first and last name, once I send it to the API using te TIdHTTP.Get() method, it throws a 400 Bad Request error and fails.
If I eliminate the space from the value for that/any particular parameter, it goes through fine.
Code I'm using to test:
httpObject := TIdHTTP.Create;
httpObject.HTTPOptions := [hoForceEncodeParams];
httpobject.MaxAuthRetries := 3;
httpObject.ProtocolVersion := pv1_1;
httpObject.Request.Accept := 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
httpObject.Request.UserAgent := 'Mozilla/3.0 (compatible;Indy Library)';
httpObject.Request.ContentType := 'application/x-www-form-urlencoded; charset=utf-8';
URL := 'url string containing the parameters'; //string variable
httpObject.Get(URL);
API documentation says:
How can I address this?
Using Delphi Community Edition (which is 10.3) and its accompanying Indy components.
You are sending parameters in the URL, not in the request body, so setting the Request.ContentType property and enabling the hoForceEncodeParams option are completely unnecessary and can be omitted.
You need to encode the parameter values when you build up a URL to send a request to. You can use the TIdURI class for that, eg:
uses
..., IdHTTP, IdURI;
URL := 'http://server/shipment?param1='+TIdURI.ParamsEncode(value1)+'¶m2='+TIdURI.ParamsEncode(value2)...;
httpObject.Get(URL);
You have stated that you are submitting the data as Content-Type: application/x-www-form-urlencoded
That format does not allow spaces. You need to properly encode what you are submitting.
Two ways you can do this:
With Indy:
Encoded := TIdURI.URLEncode(str);
With TNetEncoding
Encoded := TNetEncoding.URL.Encode(str);
I am using the latest Delphi 10.4.2 with Indy 10.
In a REST server, JSON commands are received and handled. It works fine except for Unicode.
A simple JSON like this:
{"driverNote": "Test"}
is shown correctly
If I now change to Unicode Russian characters:
{"driverNote": "Статья"}
Not sure where I should begin to track this. I expect ARequestInfo.FormParams to have the same value in debugger as s variable.
If I debug Indy itself, FormParams are set in this code:
if LRequestInfo.PostStream <> nil then
begin
// decoding percent-encoded octets and applying the CharSet is handled by
// DecodeAndSetParams() further below...
EnsureEncoding(LEncoding, enc8Bit);
LRequestInfo.FormParams :=
ReadStringFromStream( LRequestInfo.PostStream,
-1,
LEncoding
{$IFDEF STRING_IS_ANSI}, LEncoding{$ENDIF});
DoneWithPostStream(AContext, LRequestInfo); // don't need the PostStream anymore
end;
It use enc8Bit. But my string has 16-bits characters.
Is this handled incorrect in Indy?
The code snippet you quoted from IdCustomHTTPServer.pas is not what is in Indy's GitHub repo.
In the official code, TIdHTTPServer does not decode the PostStream to FormParams unless the ContentType is 'application/x-www-form-urlencoded':
if LRequestInfo.PostStream <> nil then begin
if TextIsSame(LContentType, ContentTypeFormUrlencoded) then
begin
// decoding percent-encoded octets and applying the CharSet is handled by DecodeAndSetParams() further below...
EnsureEncoding(LEncoding, enc8Bit);
LRequestInfo.FormParams := ReadStringFromStream(LRequestInfo.PostStream, -1, LEncoding{$IFDEF STRING_IS_ANSI}, LEncoding{$ENDIF});
DoneWithPostStream(AContext, LRequestInfo); // don't need the PostStream anymore
end;
end;
That ContentType check was added way back in 2010, so I don't know why it is not present in your version.
In your example, the ContentType is 'application/json', so the raw JSON should be in the PostStream and the FormParams should be blank.
That being said, in your version of Indy, TIdHTTPServer is simply reading the raw bytes from the PostStream and zero-extending each byte to a 16-bit character in the FormParams. To recover the original bytes, simply truncate each Char to an 8-bit Byte. For instance, you can use Indy's ToBytes() function in the IdGlobal unit, specifying enc8Bit/IndyTextEncoding_8Bit as the byte encoding.
JSON is most commonly transmitted as UTF-8 (and that is the case in your example), so when you have access to the raw bytes, in any version, make sure you parse the JSON bytes as UTF-8.
I have upgraded an app from D2007 to XE6. It posts data to a webserver.
I cannot work out what encoding will send the left and right quote characters correctly (code snippet below). I have tried every option I can find, but they get encoded as ? when sent (as far as I can see in WireShark).
D2007 had no problem, but XE6 is all about Unicode, and I am not sure if the problem is encoding or codepages or what.
Params := TIdMultipartFormDataStream.Create;
params.AddFormField('TEST', 'Test ‘n’ Try', 'utf8').ContentTransfer := '8bit';
IdHTTP1.Request.ContentType := 'text/plain';
IdHTTP1.Request.Charset := 'utf-8';
IdHTTP1.Post('http://test.com.au/TestEncoding.php', Params, Stream);
When calling params.AddFormField(), you are setting the charset to 'utf8', which is not a valid charset name. The official charset name is 'utf-8' instead:
params.AddFormField('TEST', 'Test ‘n’ Try', 'utf-8').ContentTransfer := '8bit';
When compiling for Unicode, an invalid charset ends up using Indy's built-in 8bit encoder, which encodes Unicode codepages > U+00FF as byte 0x3F ('?'). The quote characters you are using, ‘ and ’, are codepoints U+2018 and U+2019, respectively.
The reason you do not encounter this issue in D2007 is because the TIdFormDataField.Charset property is ignored for encoding purposes when compiling for Ansi. The TIdFormDataField.FieldValue property is an AnsiString, and its raw bytes get transmitted as-is, so you are required to ensure it is encoded properly before adding it to TIdMultipartFormDataStream, eg:
params.AddFormField('TEST', UTF8Encode('Test ‘n’ Try'), 'utf-8').ContentTransfer := '8bit';
On a side note, you do not need to set the Request.ContentType or Request.Charset properties when posting a TIdMultipartFormDataStream (and especially since 'text/plain' is an invalid content type for a MIME post anyway). This version of Post() will set those properties for you:
Params := TIdMultipartFormDataStream.Create;
params.AddFormField(...);
IdHTTP1.Post('http://test.com.au/TestEncoding.php', Params, Stream);
I notice I have invalid characters for XML files in an application who use Indy Client (I actually use default parameters for IdHttp)
Here is my code :
ts := TStringList.Create;
try
ts.Add('XML=' + AXMLDoc.XML.Text));
HTTPString := IdHTTPClient.Post('http://' + FHost + ':' + IntToStr(FPort) + FHttpRoot, ts);
finally
ts.Free;
end;
My XML file is UTF-8 encoded.
What I have to do get good encoding on my server (I also use Indy for server) ?
UTF-8 is the default charset that TIdHTTP uses for submitting a TStringList object. The real issue is that XML should not be submitted using a TStringList to begin with, even with a proper charset. The reason is because the TIdHTTP.Post(TStrings) method implements the application/x-www-form-urlencoded content type, and thus url-encodes the TStringList content, which can break XML if the receiver is not expecting that. So unless the receiver is actually expecting a real application/x-www-form-urlencoded encoded request, XML should be transmitted using the TIdHTTP.Post(TStream) method instead so the raw XML bytes are preserved as-is.
I have a function that returns an HTML page from Internet, but the Cyrillic symbols are displayed with some others unknown characters.
How can I convert the text and be able to see the normal Cyrillic symbols?
I'm with Delphi 2009 and im using indy to send HTTP request and get back response from the server.
(i think i have indy9)
This is how i take the HTML page
http := TIDHttp.Create(nil);
http.HandleRedirects := true;
http.ReadTimeout := 5000;
http.Request.ContentType:='multipart/form-data';
param:=TIdMultiPartFormDataStream.Create;
param.AddFormField('subcat_id','501');
param.AddFormField('reg_id','1');
text:=http.Post('example.com',param);
I don't know if indy has any functions that gets the page with any unicode.
You have not given enough information, but I will try to suggest this: If possible, load the data in a Stream and then create a StringList and load it like this:
var
MS:TMemoryStream;
SL: TStringList;
(...)
begin
MS:=TMemoryStream.Create;
SL:=TStringList.Create;
// Load your string to MS
SL.LoadFromStream(MS, TEncoding.UTF8);
(...)
MS.Free;
SL.Free;
end;
Comment if there is a problem.
Your question title seems to be out of sync with question body. Assuming you want to decode UTF-8 encoded HTML page, your friend is function UTF8Decode. The opposite operation done by UTF8Encode. These functions were available as early as Delphi 7 (correct me if D6 applies too). Check out "See Also" section of article, there are buffer handling entry-points for more convenience too.
Indy 9 does not support Delphi 2009. Make sure you are using the latest Indy 10 release instead. In Indy 10, the version of TIdHTTP.Post() (and TIdHTTP.Get()) that returns a String will automatically decode the data to Unicode using whatever charset is specified by the server, either in the HTTP Content-Type header, or in a <meta> tag within the HTML itself.