I am using Delphi 10.2.3.
I want to download daily exchange rates from http://www.boi.org.il/currency.xml
My design time component setup:
NetHTTPClient1.AllowCookies := True;
NetHTTPClient1.HandleRedirects := True;
NetHTTPClient1.UserAgent := 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36';
NetHTTPRequest1.MethodString := 'GET';
NetHTTPRequest1.URL := 'http://www.boi.org.il/currency.xml';
My code is very simple:
XML := NetHTTPRequest1.Execute().ContentAsString();
What I get back in XML variable is:
<html><body><script>document.cookie='sssssss=6ed9ca3asssssss_6ed9ca3a; path=/';window.location.href=window.location.href;</script></body></html>
When I try to use a web browser (Opera in my case) I can see correct XML using same URL as above. I could not see what the problem is.
Any help is appreciated.
Edit:
After reading #NineBerry comments, I used Fiddler to watch each and every packet to the site. That showed me that browser is doing a request for three times before it can actually download XML. Second request, browser adding cookie reference in the response to first request. 3rd request is same as 2ns request.
After investigating below is a working code for me and I am not changing any TNetHTTPClient.UserAgent parameter:
function DownloadExchangeRates(const URL: string; out XML: string): Boolean;
var
Cookie: string;
Path: string;
AURI: TURI;
AClient: TNetHTTPClient;
ARequest: TNetHTTPRequest;
begin
AClient := nil;
ARequest := nil;
try
AClient := TNetHTTPClient.Create(nil);
AClient.AllowCookies := True;
AClient.HandleRedirects := True;
ARequest := TNetHTTPRequest.Create(nil);
ARequest.Client := AClient;
ARequest.Asynchronous := False;
ARequest.MethodString := 'GET';
ARequest.URL := URL;
ARequest.CustomHeaders['Pragma'] := 'no-cache';
try
XML := ARequest.Execute().ContentAsString();
if XML.Length > 5 then
begin
if UpperCase(XML.Substring(0, 6)) = '<HTML>' then
begin
Cookie := GetCookie(XML);
AURI := TURI.Create(URL);
Path := AURI.SCHEME_HTTP + '://' + AURI.Host + '/';
AClient.CookieManager.AddServerCookie(Cookie, Path);
AClient.CookieManager.AddServerCookie(Cookie, URL);
ARequest.CustomHeaders['Referer'] := URL;
XML := ARequest.Execute().ContentAsString();
if XML.Length > 5 then
begin
if UpperCase(XML.Substring(0, 6)) = '<HTML>' then
begin
XML := ARequest.Execute().ContentAsString();
end;
end;
end;
end;
except
on E: Exception do
begin
Exit(False);
end;
end;
finally
ARequest.Free();
AClient.Free();
end;
Result := (XML.Length > 2) and (XML[2] = '?');
end;
uses
System.Net.HttpClient;
function TUpdater.DownloadFile(const aURL: string; aStream: TStream): boolean;
var
vHTTP: THTTPClient;
begin
Assert(aStream <> nil);
vHTTP := THTTPClient.Create;
try
Result := vHTTP.Get(aURL, aStream).StatusCode = 200;
finally
vHTTP.Free;
end;
end;
Related
A while ago I wrote a method in Delphi 2010 to get the OAuth2 token using the Indy components (TidHttp). See code below.
I am now doing something new in Delphi 10.4 and would like to use the REST components such as RESTClient, RESTRequest, TOAuth2Authenticator, etc.
Our grant type is Client Credentials but in none of the examples on the net could I find on how to use TOAuth2Authenticator with Client Credentials. Is it even possible?
We have a client id, client secret and token URL. We do not have authorization or redirect endpoints. In Insomnia, the setup will look like this:
Does somebody know how to get the token using TOAuth2Authenticator with grant type = client_credentials?
Here is the Delphi 2010 code:
procedure TfrmToken.btnGetTokenClick(Sender: TObject);
var
IdHTTP: TidHttp;
lsHttpError: string;
loRequest: TStringStream;
loRespJson: TMemoryStream;
liSuper: iSuperObject;
ldtExpiry: TDateTime;
begin
IdHTTP := TIdHTTP.Create();
loRespJson := TMemoryStream.Create();
try
IdHTTP.HandleRedirects := False;
loRequest := TStringStream.Create('grant_type=client_credentials&client_id=' +
edtKey.Text + '&client_secret='+edtSecret.Text);
try
IdHttp.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(IdHttp);
IdHttp.Request.ContentType := 'application/x-www-form-urlencoded';
try
IdHTTP.Post(edtURL.Text, loRequest, loRespJson);
except
on E: EIdHTTPProtocolException do
begin
lsHttpError := E.ErrorMessage;
end;
on E: Exception do
begin
lsHttpError := E.Message;
end;
end;
if idHTTP.ResponseCode = 200 then
begin
liSuper := SO(StreamToString(loRespJSon));
edtToken.Text := liSuper.S['access_token'];
ldtExpiry := IncSecond(Now, liSuper.i['expires_in']);
edtExpiry.Text := 'Expires in ' + liSuper.S['expires_in'] +
' seconds. Time: ' +
FormatDateTime('yyyy/dd/mm hh:nn:ss', ldtExpiry);
end
else
begin
liSuper := SO(lsHttpError);
edtToken.Text := IdHTTP.ResponseText;
edtExpiry.Text := '';
end;
finally
FreeAndNil(loRequest);
end;
finally
FreeAndNil(IdHTTP);
FreeAndNil(loRespJson);
end;
end;
I am using the trial version of DevArt's SecureBridge product. I am trying to process POST, but somehow I could not print the request data.
XML:
<test>
<a>test1</a>
<b>test2</b>
</test>
Delphi:
ScHttpWebRequest1.Method := rmPOST;
ScHttpWebRequest1.ContentType := 'text/xml';
ScHttpWebRequest1.RequestUri := 'https://test.com/api';
ScHttpWebRequest1.KeepAlive := True;
ScHttpWebRequest1.ContentLength := Length(XML);
ScHttpWebRequest1.WriteBuffer(pAnsiChar(XML), 0, Length(XML)); ///I think I'm making a mistake here.
ShowMessage(ScHttpWebRequest1.GetResponse.ReadAsString);
I have reviewed the documents, but there is a feature called RequestStream. This feature was not available in the version I downloaded. I think WriteBuffer is used instead or different. all I want to do is request a POST with XML content on the relevant site. How can I do it?
Thanks.
Here's a chunk of code that has worked for me:
var
Response: TScHttpWebResponse;
ResponseStr: string;
buf: TBytes;
begin
ScHttpWebRequest1.Method := rmPOST;
ScHttpWebRequest1.ContentType := 'text/xml';
ScHttpWebRequest1.RequestUri := 'https://test.com/api';
ScHttpWebRequest1.KeepAlive := True;
buf := TEncoding.UTF8.GetBytes(xml);
ScHttpWebRequest1.ContentLength := Length(buf);
ScHttpWebRequest1.WriteBuffer(buf);
Response:=ScHttpWebRequest1.GetResponse;
ResponseStr:=Response.ReadAsString;
end;
Based on Devart forums information you can post/put stream or strings parameters as below:
var
Request: TScHttpWebRequest;
Response: TScHttpWebResponse;
ResponseStr: string;
Stream: TFileStream;
begin
Request := TScHttpWebRequest.Create(URL);
Stream := TFileStream.Create(FileName, fmOpenRead);
try
Request.Method := rmPut;
Request.ContentType := 'application/pdf';
Request.TransferEncoding := 'binary';
Request.Headers.Add('Content-Disposition', 'form-data; name="FormFile"; filename="Document1.pdf"');
Request.ContentLength := Stream.Size;
Request.SendChunked := True;
Request.RequestStream := Stream;
Response := Request.GetResponse;
ResponseStr := Response.ReadAsString;
Response.Free;
finally
Stream.Free;
Request.Free;
end;
end;
I have Delphi 10.3.2
I do not understand this situations:
1)
Uploading photo about 1M
image1.Bitmap.LoadFromFile('test.jpg');
Then I save the same photo
image1.Bitmap.SaveToFile('test_new.jpg');
and test_new.jpg is about 3M. Why ???
2)
I want to send a photo from the TImage (test1.jpg - 1MB) object using IdHTTP and POST request to server.
I use the function Base64_Encoding_stream to encode image.
Image size (string) after encoding the function is 20 MB! ? Why if the original file has 1MB ?
function Base64_Encoding_stream(_image:Timage): string;
var
base64: TIdEncoderMIME;
output: string;
stream_image : TStream;
begin
try
begin
base64 := TIdEncoderMIME.Create(nil);
stream_image := TMemoryStream.Create;
_image.Bitmap.SaveToStream(stream_image);
stream_image.Position := 0;
output := TIdEncoderMIME.EncodeStream(stream_image);
stream_image.Free;
base64.Free;
if not(output = '') then
begin
Result := output;
end
else
begin
Result := 'Error';
end;
end;
except
begin
Result := 'Error'
end;
end;
end;
....
img_encoded := Base64_Encoding_stream(Image1);
.....
procedure Send(_json:String );
var
lHTTP : TIdHTTP;
PostData : TStringList;
begin
PostData := TStringList.Create;
lHTTP := TIdHTTP.Create(nil);
try
PostData.Add('dane=' + _json );
lHTTP.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0';
lHTTP.Request.Connection := 'keep-alive';
lHTTP.Request.ContentType := 'application/x-www-form-urlencoded';
lHTTP.Request.Charset := 'utf-8';
lHTTP.Request.Method := 'POST';
_dane := lHTTP.Post('http://......./add_photo.php',PostData);
finally
lHTTP.Free;
PostData.Free;
end;
To post your original file using base64 you can basically use your own code. You only need to change the used stream inside your base64 encoding routine like this:
function Base64_Encoding_stream(const filename: string): string;
var
stream_image : TStream;
begin
try
// create read-only stream to access the file data
stream_image := TFileStream.Create(filename, fmOpenRead or fmShareDenyWrite);
// the stream position will be ‘0’, so no need to set that
Try
Result := TIdEncoderMIME.EncodeStream(stream_image);
Finally
stream_image.Free;
End;
if length(result) = 0 then
begin
Result := 'Error';
end;
except
Result := 'Error'
end;
end;
Also, I refactored your code a bit with some try/finally sections, to ensure no memory leaks when errors occur. And I removed the begin/end inside the try/except as those are not needed.
Also removed the local string variable to avoid double string allocation and the unnecessary construction of the TIdEncoderMIME base64 object.
I have the following problem: when I click a button on a site (http://domain-location.com) I receive information back to me and the url is changed to http://domain-location.com?key=value.
I want to create a sample http client to receive information easily. Here is my code:
function DoRequest(aVal: string): string;
const DOMAIN = 'http://domain-location.com';
var
request: TIdHTTP;
responseStream: TMemoryStream;
responseLoader: TStringList;
urlRequest: string;
begin
request := TIdHTTP.Create(nil);
responseStream := TMemoryStream.Create;
responseLoader := TStringList.Create;
try
try
// accept ranges didn't help
// request.Response.AcceptRanges := 'text/json';
// request.Response.AcceptRanges := 'text/xml';
urlRequest := DOMAIN + '?key=' + aVal;
request.Get(urlRequest, responseStream);
responseStream.Position := 0;
responseLoader.LoadFromStream(responseStream);
Result := responseLoader.Text;
except on E: Exception do
Result := e.Message;
end;
finally
responseLoader.Free;
responseStream.Free;
request.Free;
end;
end;
EDIT After the first answer I edited my function (still not working):
function DoRequest(aVal: string): string;
const DOMAIN = 'http://domain-location.com';
var
request: TIdHTTP;
responseStream: TMemoryStream;
responseLoader: TStringList;
urlRequest: string;
uri: TIdURI;
begin
request := TIdHTTP.Create(nil);
responseStream := TMemoryStream.Create;
responseLoader := TStringList.Create;
request.CookieManager := TIdCookieManager.Create(request);
uri := TIdURI.Create(DOMAIN);
try
try
// accept ranges didn't help
// request.Response.AcceptRanges := 'text/json';
// request.Response.AcceptRanges := 'text/xml';
urlRequest := DOMAIN + '?key=' + aVal;
request.CookieManager.AddServerCookie('cookie1', uri);
request.CookieManager.AddServerCookie('cookie2', uri);
request.CookieManager.AddServerCookie('cookie3', uri);
request.Get(urlRequest, responseStream);
responseStream.Position := 0;
responseLoader.LoadFromStream(responseStream);
Result := responseLoader.Text;
except on E: Exception do
Result := e.Message;
end;
finally
responseLoader.Free;
responseStream.Free;
request.Free;
end;
end;
And after I do the request the result is: HTTP1.1 403 Forbidden.
I inspected the page and the button I click is in a form like this:
<form action="http:/domiain.com" method="GET">
<input type="text" name="key">
<input type="submit" value="Click">
</form>
When I type http://domain-location.com?key=value there is no problem.
Any idea how to fix it?
The problem was with the UserAgent:
function DoRequest(aVal: string): string;
const DOMAIN = 'http://domain-location.com';
var
request: TIdHTTP;
urlRequest: string;
begin
request := TIdHTTP.Create(nil);
try
try
request.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36';
urlRequest := DOMAIN + '?key=' + aVal;
Result := request.Get(urlRequest);
except on E: Exception do
Result := e.Message;
end;
finally
request.Free;
end;
end;
If cookies are involved, then you should GET the original HTML page first so the server can send whatever cookies it needs to be posted back when the button is "clicked", then you can GET the next page and let TIdHTTP post whatever cookies it had received.
Try this:
function DoRequest(const aVal: string): string;
const
DOMAIN = 'http://domain-location.com';
var
request: TIdHTTP;
begin
try
request := TIdHTTP.Create(nil);
try
request.Get(DOMAIN, TStream(nil)); // get cookies, discard HTML
Result := request.Get(DOMAIN + '?key=' + TIdURI.ParamsEncode(aVal));
finally
request.Free;
end;
except
on E: Exception do
Result := e.Message;
end;
end;
Check the actual request that is sent by your browser. 403 suggests that there is some sort of authentication going on. This might be a token or a cookie that your browser has, and sends with the request, but your sample client application may not have. Open the browser debugging panel to check the get request made by your browser, and compare it to the one made by your application. I bet there will be some difference.
I'm trying to upload a file to oboom.com i logged in successfully but when try to post the file i get that error
HTTP/1.1500 Internal Server Error.
with this respose text
[500,"illegal post header","Content-Transfer-Encoding"]
procedure TForm1.Button1Click(Sender: TObject);
var
S: tstringlist;
html: string;
clHttpRequest1: tclHttpRequest;
SSLIOHandlerSocketOpenSSL: TIdSSLIOHandlerSocketOpenSSL;
params: TIdMultiPartFormDataStream;
HTTP, HTTP2: tidhttp;
begin
params := TIdMultiPartFormDataStream.Create;
S := tstringlist.Create;
HTTP2 := tidhttp.Create(nil);
try
cookie := tidcookiemanager.Create(nil);
SSLIOHandlerSocketOpenSSL := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
HTTP2.IOHandler := SSLIOHandlerSocketOpenSSL;
HTTP2.HandleRedirects := False;
HTTP2.ConnectTimeout := 10000;
HTTP2.ReadTimeout := 10000;
HTTP2.CookieManager := cookie;
HTTP2.AllowCookies := True;
HTTP2.Request.UserAgent :=
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.154 Safari/537.36';
HTTP2.Request.ContentType := 'application/x-www-form-urlencoded';
HTTP2.Request.ContentEncoding := 'gzip, deflate';
HTTP2.Request.Accept := '*/*';
HTTP2.Request.Connection := 'Keep-Alive';
S.Values['auth'] := 'email#gmail.com';
S.Values['pass'] := 'password';
S.Values['app_id'] := '5hwaJUcDicXprlV3gjaB';
S.Values['app_session'] := '288196272';
html := HTTP2.Post('https://www.oboom.com/1.0/login', S);
finally
HTTP2.Free;
S.Free;
end;
token := ExtractBetween(html, 'session":"', '"');
try
HTTP := tidhttp.Create;
HTTP.HandleRedirects := True;
HTTP.Request.Connection := 'keep-alive';
HTTP.AllowCookies := True;
HTTP.CookieManager := cookie;
HTTP.Request.Referer := 'http://upload.oboom.com';
HTTP.Request.ContentType :=
'multipart/form-data; boundary=----------GI3Ef1cH2GI3gL6ae0Ef1KM7Ef1gL6';
HTTP.Request.Accept := '*/*';
HTTP.Request.AcceptEncoding := 'gzip,deflate';
HTTP.ConnectTimeout := 20000;
HTTP.ReadTimeout := 20000;
HTTP.Request.UserAgent :=
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.154 Safari/537.36';
params.AddFile('file', 'C:\Users\M\Pictures\Martin.jpg', );
HTTP.Post('http://upload.oboom.com/1.0/ul?token=' + token +
'&parent=1&name_policy=rename', params);
finally
HTTP.Free;
params.Free;
cookie.Free;
end;
memo1.Text := html;
end;
i have googled hours for a solution but no luck :/
tried this way :
Params.AddFile('file', 'C:\Users\M\Pictures\Martin.jpg','application/octet-stream');
Params.AddFile('file', 'C:\Users\M\Pictures\Martin.jpg','multipart/form-data');
but same error
i have tried clever internet compenent and succeeded uploading the file but i would like to use indy ..
i use delphi X3
You need to get rid of these lines:
HTTP2.Request.ContentEncoding := 'gzip, deflate';
HTTP.Request.AcceptEncoding := 'gzip,deflate';
TIdHTTP manages those values for you based on whether its Compressor property is assigned and enabled. And, you are not sending a compressed request, so those values should not be used anyway.
Also, you need to get rid of this line:
HTTP.Request.ContentType :=
'multipart/form-data; boundary=----------GI3Ef1cH2GI3gL6ae0Ef1KM7Ef1gL6';
Post() manages that value for you, especially the boundary, which TIdMultipartFormDtaStream generates dynamically.
Update the only other place that Content-Transfer-Encoding is used is on the individual fields of the TIdMultipartFormDataStream. Each TIdFormDataField has a ContentTransfer property. AddFile() initializes it to 'binary', but you can also set it to a blank string to disable the header:
params.AddFile(...).ContentTransfer := '';