I implemented same code (to post a form) using delphi and python. The python code works perfectly, but delphi code fails. In python, I can simply write httplib2.debuglevel=4 to see what content has actually been sent to the server. but I have no idea how to print the content in delphi.
def python_request_data(url, cookie, data):
httplib2.debuglevel = 4
conn = httplib2.Http()
conn.follow_all_redirects = True
headers = {'Cookie': cookie, 'Content-Type': 'application/x-www-form-urlencoded'}
response, contents = conn.request(url, 'POST', data, headers=headers)
procedure DelphiRequestData(const Url, Cookie, Data: string);
var
Client: TIdHttp;
Params: TStringList;
Response: string;
begin
Client := TIdHttp.Create(nil);
try
Client.HTTPOptions := [hoKeepOrigProtocol];
Client.Request.CustomHeaders.AddValue('Cookie', Cookie);
Params := TStringList.Create;
try
Params.QuoteChar := #0;
Params.Delimiter := '&';
Params.DelimiterText := Data;
Client.Request.ContentType := 'application/x-www-form-urlencoded';
Client.Request.ContentLength := Length(Params.DelimitedText);
Response := Client.Post(Url, Params);
finally
Params.Free;
end;
finally
Client.Free;
end;
end;
Any hints are appreciated.
You ca use TIdLogDebug as Intercept of your IdHttp.
The Events OnSend and OnReceive will deliver the desired Informations in a Array or TBytes.
Related
My goal is to be able to convert TMemoryStream to string. I have this code to get the data into the TMemoryStream:
var
idHttp : TIdHTTPEx;
url : string;
slTemp : TStringList;
memoryStream : TMemoryStream;
begin
try
idHttp := TIdHTTPEx.Create(nil);
slTemp := TStringList.Create;
memoryStream := TMemoryStream.Create;
try
url := GetURL;
SetParams(slTemp);
idHttp.Request.Accept := 'application/json, text/javascript, */*; q=0.01';
idHttp.Request.AcceptEncoding := 'gzip, deflate, br';
idHttp.Request.AcceptLanguage := 'en-US,en;q=0.9';
idHttp.Request.CacheControl := 'no-cache';
idHttp.Request.Connection := 'keep-alive';
idHttp.Request.ContentLength := 16;
idHttp.Request.ContentType := 'application/x-www-form-urlencoded; charset=UTF-8';
idHttp.Post(url, slTemp, memoryStream);
Result := MemoryStreamToString(memoryStream);
finally
memoryStream.Free;
slTemp.Free;
idHttp.Free;
end;
except on E : Exception do
begin
Result := 'e:' + E.Message;
end;
end;
end;
And this is my code to convert it to a string:
function MemoryStreamToString(MemoryStream : TMemoryStream): string;
var
StringStream: TStringStream;
begin
Result:='';
StringStream:= TStringStream.Create('', TEncoding.UTF8);
try
MemoryStream.Position := 0;
StringStream.CopyFrom(MemoryStream, MemoryStream.Size);
Result:= StringStream.DataString;
Result := Result;
finally
FreeAndNil(StringStream);
end;
end;
My function works fine in most of the conversion but not this. I checked these links: link1, link2 but they are different than my situation. I tried link3 too but still fail.
Any idea how to solve the problem?
You don't need to decode the raw data manually. Just let TIdHTTP do it for you. The Post() method has an overload that returns a decoded string:
Result := idHttp.Post(url, slTemp);
Also, you need to get rid of this line completely:
idHttp.Request.AcceptEncoding := 'gzip, deflate, br';
Otherwise TIdHTTP will not be able to decode the response correctly if the server decides to send a compressed response. You are manually giving the server permission to do so, but you are not setting up the TIdHTTP.Compressor property so TIdHTTP can handle decompression. Do not set the AcceptEncoding manually unless you are willing and able to manually detect and decode a response that has been actually been encoded in one of the formats you specify. Otherwise, just let TIdHTTP manage the AcceptEncoding property internally based on its actual capabilites.
I am trying to address the OriginStamp.org site to post a file hash and to get its timestamp
curl -X GET -H "Content-Type: application/json" -H "Authorization: xxxxxxxxxxxxxxxxxxx" http://api.originstamp.org/api/yyyyyyyyyyy
I used various formats to access this API but this seemed to be most hopeful but with out success
PROCEDURE TInterNetFm.GetTIMESTAMP;
var
i : integer;
response : string;
JSONToSend : TStringStream;
IndyH : TIdHTTP;
begin
IndyH := TIdHTTP.Create(Application);
JSONToSend := TStringStream.Create('{}');
IndyH.Request.Connection := 'Keep-Alive';
IndyH.Request.CustomHeaders.Clear;
IndyH.Request.CustomHeaders.Values['Content-Type'] := 'application/json';
IndyH.Request.CustomHeaders.Values['Authorization']:='xxxxxxxxxxxxxxx';
IndyH.Request.ContentType := 'application/json';
response:=IdHttp1.post('http://originstamp.org/api/
05c2422f44ddd24ba3f25848773a8fcb48435f8f966381da4732c40a7255780c',
JSONToSend);
JSONToSend.free;
IndyH.free;
end;
This gives a HTTP/1.1 error 403 forbidden.
I have also tried IdHttp1.getand the Delphi REST debugger and various attampts at REST.
Any suggestions about how I am going wrong with this would be grately appreciated.
A 403 error implies that you are likely using an invalid API key in the Authorization header. Also, make sure that Request.BasicAuthentication is set to False so TIdHTTP doesn't try to send its own Authorization header.
That being said, I see several other issues with your code.
You are leaking memory if TIdHTTP.Post() raises an exception on failure.
You are sending 2 Content-Type headers (one from Request.ContentType and the other from CustomerHeaders.Values['Content-Type']). Use Request.ContentType only.
You are sending your request to originstamp.org when you should be sending it to api.originstamp.org instead.
You are creating and configuring a TIdHTTP object named IndyH, but you are performing the actual Post() using another object named IdHttp1 instead.
Try this:
function TInterNetFm.GetTIMESTAMP(const AHash: string): string;
var
IndyH : TIdHTTP;
Response : string;
begin
IndyH := TIdHTTP.Create(nil);
try
IndyH.Request.BasicAuthentication := False;
IndyH.Request.CustomHeaders.Values['Authorization'] := 'YOUR_API_KEY';
Response := IndyH.Get('http://api.originstamp.org/api/' + AHash);
finally
IndyH.Free;
end;
// parse Response JSON as needed...
Result := ...;
end;
function TInterNetFm.GetTIMESTAMP(const AHash, AData: string): string;
var
IndyH : TIdHTTP;
JSONToSend : TStringStream;
Response : string;
begin
IndyH := TIdHTTP.Create(nil);
try
IndyH.Request.BasicAuthentication := False;
IndyH.Request.ContentType := 'application/json';
IndyH.Request.CustomHeaders.Values['Authorization'] := 'YOUR_API_KEY';
JSONToSend := TStringStream.Create(AData, TEncoding.UTF8);
try
Response := IndyH.Post('http://api.originstamp.org/api/' + AHash, JSONToSend);
finally
JSONToSend.Free;
end;
finally
IndyH.Free;
end;
// parse Response JSON as needed...
Result := ...;
end;
I can use the following code to easily get the HTML source from the URL, but how can I get the actual URL itself? Because sometimes the initial URL goes through some redirects and the actual URL is not the same and I would like to capture it for usage. I cannot seem to find any good documentation on the usage of methods or properties for winHTTP in Delphi. Thanks!
var http: variant;
begin
http:=createoleobject('WinHttp.WinHttpRequest.5.1');
http.open('GET', 'http://URLtoWebsite.com', false);
http.send;
showmessage(http.responsetext);
end;
You can use something like this
function GetFinalURL(const AMainURL: string): string;
var
http: Variant;
begin
Result := '';
http := CreateOleObject('WinHttp.WinHttpRequest.5.1');
http.Option(6) := False;
http.open('GET', AMainURL, false);
http.send;
if http.Status = 302 then
Result := http.getResponseHeader('Location')
else
Result := AMainURL;
end;
Another way using Indy
function GetFinalURL(const AMainURL: string): string;
var
idHTTP: TIdHTTP;
begin
Result := '';
idHTTP := TIdHTTP.Create(nil);
try
idHTTP.HandleRedirects := True;
try
idHTTP.Get(AMainURL);
Result := idHTTP.Request.URL;
except
end;
finally
idHTTP.Free;
end;
end;
You can set WinHttpSetStatusCallback with WINHTTP_CALLBACK_FLAG_REDIRECT parameter to receive notifications about every redirect occurred during request.
I have the following problem: when I click a button on a site (http://domain-location.com) I receive information back to me and the url is changed to http://domain-location.com?key=value.
I want to create a sample http client to receive information easily. Here is my code:
function DoRequest(aVal: string): string;
const DOMAIN = 'http://domain-location.com';
var
request: TIdHTTP;
responseStream: TMemoryStream;
responseLoader: TStringList;
urlRequest: string;
begin
request := TIdHTTP.Create(nil);
responseStream := TMemoryStream.Create;
responseLoader := TStringList.Create;
try
try
// accept ranges didn't help
// request.Response.AcceptRanges := 'text/json';
// request.Response.AcceptRanges := 'text/xml';
urlRequest := DOMAIN + '?key=' + aVal;
request.Get(urlRequest, responseStream);
responseStream.Position := 0;
responseLoader.LoadFromStream(responseStream);
Result := responseLoader.Text;
except on E: Exception do
Result := e.Message;
end;
finally
responseLoader.Free;
responseStream.Free;
request.Free;
end;
end;
EDIT After the first answer I edited my function (still not working):
function DoRequest(aVal: string): string;
const DOMAIN = 'http://domain-location.com';
var
request: TIdHTTP;
responseStream: TMemoryStream;
responseLoader: TStringList;
urlRequest: string;
uri: TIdURI;
begin
request := TIdHTTP.Create(nil);
responseStream := TMemoryStream.Create;
responseLoader := TStringList.Create;
request.CookieManager := TIdCookieManager.Create(request);
uri := TIdURI.Create(DOMAIN);
try
try
// accept ranges didn't help
// request.Response.AcceptRanges := 'text/json';
// request.Response.AcceptRanges := 'text/xml';
urlRequest := DOMAIN + '?key=' + aVal;
request.CookieManager.AddServerCookie('cookie1', uri);
request.CookieManager.AddServerCookie('cookie2', uri);
request.CookieManager.AddServerCookie('cookie3', uri);
request.Get(urlRequest, responseStream);
responseStream.Position := 0;
responseLoader.LoadFromStream(responseStream);
Result := responseLoader.Text;
except on E: Exception do
Result := e.Message;
end;
finally
responseLoader.Free;
responseStream.Free;
request.Free;
end;
end;
And after I do the request the result is: HTTP1.1 403 Forbidden.
I inspected the page and the button I click is in a form like this:
<form action="http:/domiain.com" method="GET">
<input type="text" name="key">
<input type="submit" value="Click">
</form>
When I type http://domain-location.com?key=value there is no problem.
Any idea how to fix it?
The problem was with the UserAgent:
function DoRequest(aVal: string): string;
const DOMAIN = 'http://domain-location.com';
var
request: TIdHTTP;
urlRequest: string;
begin
request := TIdHTTP.Create(nil);
try
try
request.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36';
urlRequest := DOMAIN + '?key=' + aVal;
Result := request.Get(urlRequest);
except on E: Exception do
Result := e.Message;
end;
finally
request.Free;
end;
end;
If cookies are involved, then you should GET the original HTML page first so the server can send whatever cookies it needs to be posted back when the button is "clicked", then you can GET the next page and let TIdHTTP post whatever cookies it had received.
Try this:
function DoRequest(const aVal: string): string;
const
DOMAIN = 'http://domain-location.com';
var
request: TIdHTTP;
begin
try
request := TIdHTTP.Create(nil);
try
request.Get(DOMAIN, TStream(nil)); // get cookies, discard HTML
Result := request.Get(DOMAIN + '?key=' + TIdURI.ParamsEncode(aVal));
finally
request.Free;
end;
except
on E: Exception do
Result := e.Message;
end;
end;
Check the actual request that is sent by your browser. 403 suggests that there is some sort of authentication going on. This might be a token or a cookie that your browser has, and sends with the request, but your sample client application may not have. Open the browser debugging panel to check the get request made by your browser, and compare it to the one made by your application. I bet there will be some difference.
I am attempting to convert some code from TWebBrowser to Chromium but am having trouble figuring out how to send post and header data with an HTTP request.
Below is the TWebBrowser functionality I'm trying to implement.
var
VHeader, PostData: OleVariant;
PostData := VarArrayCreate([0, Length(XMLString) - 1], varByte) ;
HeaderData := 'Content-Type: application/x-www-form-urlencoded'+ '\n';
WebBrowser1.Navigate(StrUrl,EmptyParam,EmptyParam,PostData,VHeader);
How do I do the equivalent with Chromium?
Due to a missing documentation for Delphi Chromium Embedded, I'll refer the needed requirements for sending web requests for the C++ version of CEF. So, you need to use the LoadRequest method for sending requests in Chromium. For using it, you need the object instance of the CefRequest request object class along with the HeaderMap and CefPostData objects for request header and data specification.
Expanding on Henri Gourvest's (author of the Delphi CEF wrapper) example from this thread, you can in Delphi try something like the following pseudo-code:
uses
ceflib;
function CreateField(const AValue: AnsiString): ICefPostDataElement;
begin
Result := TCefPostDataElementRef.New;
Result.SetToBytes(Length(AValue), PAnsiChar(AValue));
end;
procedure TForm1.Button1Click(Sender: TObject);
var
Header: ICefStringMultimap;
Data: ICefPostData;
Request: ICefRequest;
begin
Header := TCefStringMultimapOwn.Create;
Header.Append('Content-Type', 'application/x-www-form-urlencoded');
Data := TCefPostDataRef.New;
Data.AddElement(CreateField('Data.id=27'));
Data.AddElement(CreateField('&Data.title=title'));
Data.AddElement(CreateField('&Data.body=body'));
Request := TCefRequestRef.New;
Request.Flags := WUR_FLAG_NONE;
Request.Assign('http://example.com/', 'POST', Data, Header);
Chromium1.Browser.MainFrame.LoadRequest(Request);
end;
The same should do another version of the above code:
procedure TForm1.Button1Click(Sender: TObject);
var
Header: ICefStringMultimap;
Data: ICefPostData;
Request: ICefRequest;
begin
Request := TCefRequestRef.New;
Request.Url := 'http://example.com/';
Request.Method := 'POST';
Request.Flags := WUR_FLAG_NONE;
Header := TCefStringMultimapOwn.Create;
Header.Append('Content-Type', 'application/x-www-form-urlencoded');
Request.SetHeaderMap(Header);
Data := TCefPostDataRef.New;
Data.AddElement(CreateField('Data.id=27'));
Data.AddElement(CreateField('&Data.title=title'));
Data.AddElement(CreateField('&Data.body=body'));
Request.PostData := Data;
Chromium1.Browser.MainFrame.LoadRequest(Request);
end;