How to get all raw request headers? - delphi

I am trying to get all the raw request headers from Asynchronous Pluggable Protocol I've implemented. But I can only get a few basic headers using IHttpNegotiate. Such as Accept-Language, Referer. With a tool called HTTP Analyzer these things can be viewed in more detail.
function RetrieveRequestHeaders(const szUrl: PWideChar; const OIProtSink: IInternetProtocolSink): String;
var
pHttpNeg: IHttpNegotiate;
Headers: PWideChar;
HR: HResult;
begin
Result := '';
HR := IUnknown_QueryService(OIProtSink, IID_IHttpNegotiate, IID_IHttpNegotiate, pHttpNeg);
if Succeeded(HR) then
begin
Headers := nil;
HR := pHttpNeg.BeginningTransaction(szUrl, nil, 0, Headers);
if Succeeded(HR) then
begin
Result := Headers;
CoTaskMemFree(Headers);
end;
end;
end;

IHTTPNegotiate.BeginningTransaction will give you the (additional) headers the browser wants to add to the outgoing request. As the protocol handler you're responsible to create the full outgoing HTTP request header, like you can see with a HTTP analyzer tool.

Related

Convert cURL headers to Delphi TIdHTTP

I am trying to address the OriginStamp.org site to post a file hash and to get its timestamp
curl -X GET -H "Content-Type: application/json" -H "Authorization: xxxxxxxxxxxxxxxxxxx" http://api.originstamp.org/api/yyyyyyyyyyy
I used various formats to access this API but this seemed to be most hopeful but with out success
PROCEDURE TInterNetFm.GetTIMESTAMP;
var
i : integer;
response : string;
JSONToSend : TStringStream;
IndyH : TIdHTTP;
begin
IndyH := TIdHTTP.Create(Application);
JSONToSend := TStringStream.Create('{}');
IndyH.Request.Connection := 'Keep-Alive';
IndyH.Request.CustomHeaders.Clear;
IndyH.Request.CustomHeaders.Values['Content-Type'] := 'application/json';
IndyH.Request.CustomHeaders.Values['Authorization']:='xxxxxxxxxxxxxxx';
IndyH.Request.ContentType := 'application/json';
response:=IdHttp1.post('http://originstamp.org/api/
05c2422f44ddd24ba3f25848773a8fcb48435f8f966381da4732c40a7255780c',
JSONToSend);
JSONToSend.free;
IndyH.free;
end;
This gives a HTTP/1.1 error 403 forbidden.
I have also tried IdHttp1.getand the Delphi REST debugger and various attampts at REST.
Any suggestions about how I am going wrong with this would be grately appreciated.
A 403 error implies that you are likely using an invalid API key in the Authorization header. Also, make sure that Request.BasicAuthentication is set to False so TIdHTTP doesn't try to send its own Authorization header.
That being said, I see several other issues with your code.
You are leaking memory if TIdHTTP.Post() raises an exception on failure.
You are sending 2 Content-Type headers (one from Request.ContentType and the other from CustomerHeaders.Values['Content-Type']). Use Request.ContentType only.
You are sending your request to originstamp.org when you should be sending it to api.originstamp.org instead.
You are creating and configuring a TIdHTTP object named IndyH, but you are performing the actual Post() using another object named IdHttp1 instead.
Try this:
function TInterNetFm.GetTIMESTAMP(const AHash: string): string;
var
IndyH : TIdHTTP;
Response : string;
begin
IndyH := TIdHTTP.Create(nil);
try
IndyH.Request.BasicAuthentication := False;
IndyH.Request.CustomHeaders.Values['Authorization'] := 'YOUR_API_KEY';
Response := IndyH.Get('http://api.originstamp.org/api/' + AHash);
finally
IndyH.Free;
end;
// parse Response JSON as needed...
Result := ...;
end;
function TInterNetFm.GetTIMESTAMP(const AHash, AData: string): string;
var
IndyH : TIdHTTP;
JSONToSend : TStringStream;
Response : string;
begin
IndyH := TIdHTTP.Create(nil);
try
IndyH.Request.BasicAuthentication := False;
IndyH.Request.ContentType := 'application/json';
IndyH.Request.CustomHeaders.Values['Authorization'] := 'YOUR_API_KEY';
JSONToSend := TStringStream.Create(AData, TEncoding.UTF8);
try
Response := IndyH.Post('http://api.originstamp.org/api/' + AHash, JSONToSend);
finally
JSONToSend.Free;
end;
finally
IndyH.Free;
end;
// parse Response JSON as needed...
Result := ...;
end;

Is there way to get the final URL after redirects using WinHTTP in Delphi?

I can use the following code to easily get the HTML source from the URL, but how can I get the actual URL itself? Because sometimes the initial URL goes through some redirects and the actual URL is not the same and I would like to capture it for usage. I cannot seem to find any good documentation on the usage of methods or properties for winHTTP in Delphi. Thanks!
var http: variant;
begin
http:=createoleobject('WinHttp.WinHttpRequest.5.1');
http.open('GET', 'http://URLtoWebsite.com', false);
http.send;
showmessage(http.responsetext);
end;
You can use something like this
function GetFinalURL(const AMainURL: string): string;
var
http: Variant;
begin
Result := '';
http := CreateOleObject('WinHttp.WinHttpRequest.5.1');
http.Option(6) := False;
http.open('GET', AMainURL, false);
http.send;
if http.Status = 302 then
Result := http.getResponseHeader('Location')
else
Result := AMainURL;
end;
Another way using Indy
function GetFinalURL(const AMainURL: string): string;
var
idHTTP: TIdHTTP;
begin
Result := '';
idHTTP := TIdHTTP.Create(nil);
try
idHTTP.HandleRedirects := True;
try
idHTTP.Get(AMainURL);
Result := idHTTP.Request.URL;
except
end;
finally
idHTTP.Free;
end;
end;
You can set WinHttpSetStatusCallback with WINHTTP_CALLBACK_FLAG_REDIRECT parameter to receive notifications about every redirect occurred during request.

How to trace raw request and response content of TIdHttp

I implemented same code (to post a form) using delphi and python. The python code works perfectly, but delphi code fails. In python, I can simply write httplib2.debuglevel=4 to see what content has actually been sent to the server. but I have no idea how to print the content in delphi.
def python_request_data(url, cookie, data):
httplib2.debuglevel = 4
conn = httplib2.Http()
conn.follow_all_redirects = True
headers = {'Cookie': cookie, 'Content-Type': 'application/x-www-form-urlencoded'}
response, contents = conn.request(url, 'POST', data, headers=headers)
procedure DelphiRequestData(const Url, Cookie, Data: string);
var
Client: TIdHttp;
Params: TStringList;
Response: string;
begin
Client := TIdHttp.Create(nil);
try
Client.HTTPOptions := [hoKeepOrigProtocol];
Client.Request.CustomHeaders.AddValue('Cookie', Cookie);
Params := TStringList.Create;
try
Params.QuoteChar := #0;
Params.Delimiter := '&';
Params.DelimiterText := Data;
Client.Request.ContentType := 'application/x-www-form-urlencoded';
Client.Request.ContentLength := Length(Params.DelimitedText);
Response := Client.Post(Url, Params);
finally
Params.Free;
end;
finally
Client.Free;
end;
end;
Any hints are appreciated.
You ca use TIdLogDebug as Intercept of your IdHttp.
The Events OnSend and OnReceive will deliver the desired Informations in a Array or TBytes.

How do I send post and header data with Chromium?

I am attempting to convert some code from TWebBrowser to Chromium but am having trouble figuring out how to send post and header data with an HTTP request.
Below is the TWebBrowser functionality I'm trying to implement.
var
VHeader, PostData: OleVariant;
PostData := VarArrayCreate([0, Length(XMLString) - 1], varByte) ;
HeaderData := 'Content-Type: application/x-www-form-urlencoded'+ '\n';
WebBrowser1.Navigate(StrUrl,EmptyParam,EmptyParam,PostData,VHeader);
How do I do the equivalent with Chromium?
Due to a missing documentation for Delphi Chromium Embedded, I'll refer the needed requirements for sending web requests for the C++ version of CEF. So, you need to use the LoadRequest method for sending requests in Chromium. For using it, you need the object instance of the CefRequest request object class along with the HeaderMap and CefPostData objects for request header and data specification.
Expanding on Henri Gourvest's (author of the Delphi CEF wrapper) example from this thread, you can in Delphi try something like the following pseudo-code:
uses
ceflib;
function CreateField(const AValue: AnsiString): ICefPostDataElement;
begin
Result := TCefPostDataElementRef.New;
Result.SetToBytes(Length(AValue), PAnsiChar(AValue));
end;
procedure TForm1.Button1Click(Sender: TObject);
var
Header: ICefStringMultimap;
Data: ICefPostData;
Request: ICefRequest;
begin
Header := TCefStringMultimapOwn.Create;
Header.Append('Content-Type', 'application/x-www-form-urlencoded');
Data := TCefPostDataRef.New;
Data.AddElement(CreateField('Data.id=27'));
Data.AddElement(CreateField('&Data.title=title'));
Data.AddElement(CreateField('&Data.body=body'));
Request := TCefRequestRef.New;
Request.Flags := WUR_FLAG_NONE;
Request.Assign('http://example.com/', 'POST', Data, Header);
Chromium1.Browser.MainFrame.LoadRequest(Request);
end;
The same should do another version of the above code:
procedure TForm1.Button1Click(Sender: TObject);
var
Header: ICefStringMultimap;
Data: ICefPostData;
Request: ICefRequest;
begin
Request := TCefRequestRef.New;
Request.Url := 'http://example.com/';
Request.Method := 'POST';
Request.Flags := WUR_FLAG_NONE;
Header := TCefStringMultimapOwn.Create;
Header.Append('Content-Type', 'application/x-www-form-urlencoded');
Request.SetHeaderMap(Header);
Data := TCefPostDataRef.New;
Data.AddElement(CreateField('Data.id=27'));
Data.AddElement(CreateField('&Data.title=title'));
Data.AddElement(CreateField('&Data.body=body'));
Request.PostData := Data;
Chromium1.Browser.MainFrame.LoadRequest(Request);
end;

How to send a HTTP POST Request in Delphi using WinInet api

I am trying to make HTTP Requests from Delphi using the WinInet functions.
So far I have:
function request:string;
var
hNet,hURL,hRequest: HINTERNET;
begin
hNet := InternetOpen(PChar('User Agent'),INTERNET_OPEN_TYPE_PRECONFIG or INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
if Assigned(hNet) then
begin
try
hURL := InternetConnect(hNet,PChar('http://example.com'),INTERNET_DEFAULT_HTTP_PORT,nil,nil,INTERNET_SERVICE_HTTP,0,DWORD(0));
if(hURL<>nil) then
hRequest := HttpOpenRequest(hURL, 'POST', PChar('param=value'),'HTTP/1.0',PChar(''), nil, INTERNET_FLAG_RELOAD or INTERNET_FLAG_PRAGMA_NOCACHE,0);
if(hRequest<>nil) then
HttpSendRequest(hRequest, nil, 0, nil, 0);
InternetCloseHandle(hNet);
except
on E : Exception do
ShowMessage(E.ClassName+' error raised, with message : '+E.Message);
end;
end
end;
But this doesn't do anything (I am sniffing network http traffic to see if it works).
I have successfully used InternetOpenURL but I also need to send POST request and that function doesn't do that.
Could someone show me a simple example? The result I want is to get the http response page in a var as string.
I got all the url/filename part messed up with the previous code. I'm using this from Jeff DeVore now and it's working fine:
function request(const AUrl, AData: AnsiString; blnSSL: Boolean = True): AnsiString;
var
aBuffer : Array[0..4096] of Char;
Header : TStringStream;
BufStream : TMemoryStream;
sMethod : AnsiString;
BytesRead : Cardinal;
pSession : HINTERNET;
pConnection : HINTERNET;
pRequest : HINTERNET;
parsedURL : TStringArray;
port : Integer;
flags : DWord;
begin
ParsedUrl := ParseUrl(AUrl);
Result := '';
pSession := InternetOpen(nil, INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
if Assigned(pSession) then
try
if blnSSL then
Port := INTERNET_DEFAULT_HTTPS_PORT
else
Port := INTERNET_DEFAULT_HTTP_PORT;
pConnection := InternetConnect(pSession, PChar(ParsedUrl[0]), port, nil, nil, INTERNET_SERVICE_HTTP, 0, 0);
if Assigned(pConnection) then
try
if (AData = '') then
sMethod := 'GET'
else
sMethod := 'POST';
if blnSSL then
flags := INTERNET_FLAG_SECURE or INTERNET_FLAG_KEEP_CONNECTION
else
flags := INTERNET_SERVICE_HTTP;
pRequest := HTTPOpenRequest(pConnection, PChar(sMethod), PChar(ParsedUrl[1]), nil, nil, nil, flags, 0);
if Assigned(pRequest) then
try
Header := TStringStream.Create('');
try
with Header do
begin
WriteString('Host: ' + ParsedUrl[0] + sLineBreak);
WriteString('User-Agent: Custom program 1.0'+SLineBreak);
WriteString('Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'+SLineBreak);
WriteString('Accept-Language: en-us,en;q=0.5' + SLineBreak);
WriteString('Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7'+SLineBreak);
WriteString('Keep-Alive: 300'+ SLineBreak);
WriteString('Connection: keep-alive'+ SlineBreak+SLineBreak);
end;
HttpAddRequestHeaders(pRequest, PChar(Header.DataString), Length(Header.DataString), HTTP_ADDREQ_FLAG_ADD);
if HTTPSendRequest(pRequest, nil, 0, Pointer(AData), Length(AData)) then
begin
BufStream := TMemoryStream.Create;
try
while InternetReadFile(pRequest, #aBuffer, SizeOf(aBuffer), BytesRead) do
begin
if (BytesRead = 0) then Break;
BufStream.Write(aBuffer, BytesRead);
end;
aBuffer[0] := #0;
BufStream.Write(aBuffer, 1);
Result := PChar(BufStream.Memory);
finally
BufStream.Free;
end;
end;
finally
Header.Free;
end;
finally
InternetCloseHandle(pRequest);
end;
finally
InternetCloseHandle(pConnection);
end;
finally
InternetCloseHandle(pSession);
end;
end;
ParseUrl is a function that splits a URL in "hostname / filename" and TStringArray is an array of strings. I still have to review the code tomorrow but it looks fine and in my sniffer I saw the post data and headers being sent.
Personally I prefer to use the synapse library for all of my TCP/IP work. For example, a simple HTTP post can be coded as:
uses
httpsend;
function testpost;
begin
stm := tStringstream.create('param=value');
try
HttpPostBinary('http://example.com',Stm);
finally
stm.free;
end;
end;
The library is well written and very easy to modify to suit your specific requirements. The latest subversion release works without any problems for both Delphi 2009 and Delphi 2010. This framework is not component based, but rather is a series of classes and procedures which well in a multi-threaded environment.
The third parameter (lpszObjectName) to HttpOpenRequest should be the URL you wish to request. That's why the documentation describes the fifth parameter (lpszReferer) as "a pointer to a null-terminated string that specifies the URL of the document from which the URL in the request (lpszObjectName) was obtained."
The posted data gets sent with HttpSendRequest; the lpOptional parameter is described like this:
Pointer to a buffer containing any optional data to be sent immediately after the request headers. This parameter is generally used for POST and PUT operations. The optional data can be the resource or information being posted to the server. This parameter can be NULL if there is no optional data to send.
The second parameter to InternetOpen should be just the server name; it should not include the protocol. The protocol you specify with the sixth parameter.
After you've sent the request, you can read the response with InternetReadFile and InternetQueryDataAvailable.
Don't just check whether the API functions return zero and then proceed on the next line. If they fail, call GetLastError to find out why. The code you've posted will not raise exceptions, so it's futile to catch any. (And it's foolish to "handle" them the way you're doing so anyway. Don't catch an exception that you don't already know how to fix. Let everything else go up to the caller, or the caller's caller, etc.)

Resources