I'm using Indy TIdHTTP along with TIdCookieManager. I would like to check the current cookies for the request I'm about to send and identify the likelyhood that it will be valid (I know I can't be 100% sure the server will accept my request). If there are no cookies, or if they're expired, I will want to login first and acquire new cookies. Otherwise, just send the request.
How would I go about doing such a check? I believe I have to check the cookie manager before I send a request, but don't know what to check.
Try something like this:
function CheckCookies(Cookies: TIdCookieManager; const TargetURL: String): Boolean;
var
URL: TIdURI;
Headers: TIdHeaderList;
begin
Result := False;
URL := TIdURI.Create(TargetURL);
try
Headers := TIdHeaderList.Create(QuoteHTTP);
try
Cookies.GenerateClientCookies(URL, False, Headers);
Result := Headers.Count > 0;
finally
Headers.Free;
end;
finally
URL.Free;
end;
end;
.
if not CheckCookies(IdHTTP1.CookieManager, 'http://www.someurl.com/') then
begin
// login and get new cookies ...
end;
Like already stated in the comments you cannot perform an actual acceptance check on the client, only the server can do that.
However you can filter out expired or invalid cookies:
function filterInvalidCookies(cookies: TIdCookies; targetURL: TIdURI): Boolean;
var
c: Integer;
begin
Result := False;
c := 0;
while (cookies.Count > c) do
if (not cookies[c].IsExpired and cookies[c].IsAllowed(targetURL, False) and
(cookies[c].CookieName <> '')) then
begin
Result := True;
Inc(c);
end
else
cookies.Delete(c);
end;
The function removes invalid cookies and returns False if there are no valid ones left. Call it before a request like this:
if (Assigned(con.CookieManager)) then
filterInvalidCookies(con.CookieManager.CookieCollection,
TIdURI.Create('http://www.someurl.com/'));
where con is an TIdHTTP object.
You can do additional, maybe target page specific checks of course.
Related
I'm trying to send response back to servers requesting digest access authentication
....
FResponseHeader.Text := FResponseText;// received header.
FResponseHeader.ProcessHeaders;
....
WriteLn(FResponseHeader.WWWAuthenticate); //WWW-Authenticate: Digest realm="xxxx.com", nonce="fq1uvocyzvr17e6a5syproazd5phwdvhvlc5", stale=false, algorithm=MD5, qop="auth"
LIdAuthentication := TIdDigestAuthentication.Create;
try
LIdAuthentication.Username := FUser;
LIdAuthentication.Password := FPass;
LIdAuthentication.Uri := FURI;
LIdAuthentication.Method := GetMsgTypeString(FResponseHeader.RequestMethods);
LIdAuthentication.Params.Values['Authorization'] := FResponseHeader.WWWAuthenticate;
LIdAuthentication.AuthParams.AddValue('Digest', FResponseHeader.WWWAuthenticate);
for I := 0 to LIdAuthentication.Steps do
LIdAuthentication.Next;
Result := LIdAuthentication.Authentication;
finally
LIdAuthentication.Free;
end;
I got 401 from the server.
What is the correct way to create the Authorization Header ?
TIdDigestAuthentication (and other TIdAuthentication-derived classes) are intended to be used with TIdHTTP, not standalone.
If you are using TIdHTTP to communicate with a server, you do not need to manage Digest authentication manually at all. If the server requests Digest in its WWW-Authenticate header, and if IdAuthenticationDigest (or IdAllAuthentications) is in your uses clause, then TIdHTTP will automatically send a Digest response for you. The only thing you have to concern yourself with doing is:
set the TIdHTTP.Request.Username and TIdHTTP.Request.Password properties for the initial authentication attempt.
set a TIdHTTP.OnAuthorization event handler to handle the possibility of the server rejecting the current Username/Password so you can supply new values for retry, optionally after prompting the user.
optionally set a TIdHTTP.OnSelectProxyAuthorization event handler to choose which authentication scheme to use if multiple schemes are requested by the server, and/or if you want to control which scheme takes priority over others.
For example:
uses
..., IdHTTP, IdAuthenticationDigest;
...
IdHTTP1.OnAuthorization := AuthRequested;
IdHTTP1.Request.Username := ...; // initial username
IdHTTP1.Request.Password := ...; // initial password
IdHTTP1.Get(...);
...
procedure TMyClass.AuthRequested(Sender: TObject; Authentication: TIdAuthentication; var Handled: Boolean);
begin
if (new credentials are available) then
begin
Authentication.Username := ...; // new username
Authentication.Password := ...; // new password
Handled := True;
end else
Handled := False;
end;
That being said, if you want to use TIdDigestAuthentication standalone, then you should use it similarly to how TIdHTTP uses it, eg:
LIdAuthentication := TIdDigestAuthentication.Create;
try
LIdAuthentication.SetRequest(FGetMsgTypeString(FResponseHeader.RequestMethods), FURI);
LIdAuthentication.Username := FUser;
LIdAuthentication.Password := FPass;
LIdAuthentication.Params.Values['Authorization'] := LIdAuthentication.Authentication;
LIdAuthentication.AuthParams := FResponseHeader.WWWAuthenticate; // assuming WWWAuthenticate is a TIdHeaderList...
repeat
case LIdAuthentication.Next of
wnAskTheProgram:
begin
// set LIdAuthentication.Username and LIdAuthentication.Password to new credentials to retry...
end;
wnDoRequest:
begin
// send new request with LIdAuthentication.Authentication in the 'Authorization' header...
Result := LIdAuthentication.Authentication;
Exit;
end;
wnFail:
begin
// error handling ...
Result := '';
Exit;
end;
end;
until False;
finally
LIdAuthentication.Free;
end;
I have seen a lot of examples online, but I cannot understand why my code doesn't work.
I have an url that looks like this:
http://www.domain.com/confirm.php?user=USERNAME&id=THEID
confirm.php is a page that does some checks on a MySQL database and then the only output of the page is a 0 or a -1 (true or false):
<?php
//long code...
if ( ... ) {
echo "0"; // success!
die();
} else {
echo "-1"; // fail!
die();
}
?>
My Delphi FireMonkey app has to open the URL above, passing the username and the id, and then read the result of the page. The result is only a -1 or a 0. This is the code.
//I have created a subclass of TThread
procedure TRegister.Execute;
var
conn: TIdHTTP;
res: string;
begin
inherited;
Queue(nil,
procedure
begin
ProgressLabel.Text := 'Connecting...';
end
);
//get the result -1 or 0
try
conn := TIdHTTP.Create(nil);
try
res := conn.Get('http://www.domain.com/confirm.php?user='+FUsername+'&id='+FPId);
finally
conn.Free;
end;
except
res := 'error!!';
end;
Queue(nil,
procedure
begin
ProgressLabel.Text := res;
end
);
end;
The value of res is always error!! and never -1 or 0. Where is my code wrong? The error caught from on E: Exception do is:
HTTP/1.1 406 not acceptable
I have found a solution using System.Net.HttpClient. I can simply use this function
function GetURL(const AURL: string): string;
var
HttpClient: THttpClient;
HttpResponse: IHttpResponse;
begin
HttpClient := THTTPClient.Create;
try
HttpResponse := HttpClient.Get(AURL);
Result := HttpResponse.ContentAsString();
finally
HttpClient.Free;
end;
end;
This works and gives me -1 and 0 as I expected. To get an example of a working code I have tested this:
procedure TForm1.Button1Click(Sender: TObject);
function GetURL(const AURL: string): string;
var
HttpClient: THttpClient;
HttpResponse: IHttpResponse;
begin
HttpClient := THTTPClient.Create;
try
HttpResponse := HttpClient.Get(AURL);
Result := HttpResponse.ContentAsString();
finally
HttpClient.Free;
end;
end;
function GetURLAsString(const aURL: string): string;
var
lHTTP: TIdHTTP;
begin
lHTTP := TIdHTTP.Create;
try
Result := lHTTP.Get(aURL);
finally
lHTTP.Free;
end;
end;
begin
Memo1.Lines.Add(GetURL('http://www.domain.com/confirm.php?user=user&id=theid'));
Memo1.Lines.Add(GetURLAsString('http://www.domain.com/confirm.php?user=user&id=theid'))
end;
end.
The first function works perfectly but Indy raises the exception HTTP/1.1 406 not acceptable. It seems that Indy cannot automatically handle the content type of the page. Here you can see the REST Debugger log:
HTTP Error 406 Not acceptable typically means that the server is not able to respond with the content type the client wanted. Both the Server and Client need to appropriately use the MIME type as you need. In this case, your client's Accept headers should provide the desired type of response, and your server should also be responding with the same. In your case, the Content-Type will most likely be text/plain.
So long story short, your client is expecting a MIME type which the server does not explicitly return in its response. The problem could be on either side, or perhaps both.
Your Client's Accept headers must provide the MIME type(s) you expect and need. Specifically Accept, Accept-Charset, Accept-Language, Accept-Encoding. By default in Indy TIdHTTP, these headers should accept essentially anything, assuming these headers haven't been overwritten. The Accept header is by default set to text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 where the */* opens the door for any MIME type.
Your Server's Response's Content-Type must be one of the provided MIME types, as well as the format of the response as also desired by the client. It is likely that your HTTP server is not providing the appropriate Content-Type in its response. If the server responds with anything in the */* filter (which should mean everything), then the client will accept it (assuming the server responds with text/plain). If the server responds with an invalid content type (such as just text or plain), then it could be rejected.
I am trying to intercept GET/POST that i get from IdHTTPProxyServer and send it to IdHTTP so i can replicate the GET/POST and eventually get cookies so I can login in to any website.
How can this be improved? If i try to execute this code it crashes.
procedure TForm1.IdHTTPProxyServer1HTTPDocument(
AContext: TIdHTTPProxyServerContext; var VStream: TStream);
begin
if AContext.Command='POST' then begin
EmbeddedWB1.LoadFromString(IdHTTP1.Post(AContext.Target,AContext.Headers.Text)); << CRASH
end;
if AContext.Command='GET' then begin
EmbeddedWB1.LoadFromString(IdHTTP1.Get(AContext.Target)); << CRASH
end;
end;
When using the OnHTTPDocument event, you need to look at the TIdHTTPProxyServerContext.TransferSource property to know if the event is being triggered for a client request that contains body data, or is being triggered for the target server's response to the client request. Sending your own GET/POST request only makes sense when processing client requests, however the OnHTTPDocument event is not likely to ever be triggered for a GET request since there is no body data to capture.
You don't need to use TIdHTTP in order to get the server's cookies. Let TIdHTTPProxyServer do its work normally, and then you can extract the cookies from the TIdHTTPProxyServerContext.Headers property in the OnHTTPResponse event, eg:
procedure TForm1.IdHTTPProxyServer1HTTPResponse(AContext: TIdHTTPProxyServerContext);
var
Cookies: TStringList;
begin
Cookies := TStringList.Create;
try
AContext.Headers.Extract('Set-Cookie', Cookies);
// use Cookies as needed, such as:
// URI := TIdURI.Create(AContext.Target);
// try
// CookieManager1.AddServerCookies(Cookies, URI);
// finally
// URI.Free;
// end;
finally
Cookies.Free;
end;
end;
I know there's alot of Indy threads but I can't get one to match my case.
I have been given a URL with a username and password form. this then actions to a URL/reports.php on which there are multiple hyperlinks.
Each of these links will direct to a page with URL variables e.g. reports.php?report=variablename where a download will immediately start.
My thinking so far:
procedure TForm1.PostData(Sender: TObject);
var
paramList:TStringList;
url,text:string;
// IdHTTP1: TIdHTTP;
IdSSLIOHandlerSocket1: TIdSSLIOHandlerSocket;
idLogFile1 : TidLogFile;
begin
idLogFile1 := TidLogFile.Create(nil);
with idLogFile1 do
begin
idLogFile1.Filename := 'C:\HTTPSlogfile.txt';
idLogFile1.active := True;
end;
IdHTTP1 := TIdHTTP.Create(nil);
IdSSLIOHandlerSocket1 := TIdSSLIOHandlerSocket.Create(nil);
IdSSLIOHandlerSocket1.SSLOptions.Method := sslvSSLv23;
IdHTTP1.IOHandler := IdSSLIOHandlerSocket1;
IdHTTP1.HandleRedirects := true;
IdHTTP1.ReadTimeout := 5000;
IdHTTP1.Intercept := idLogFile1;
paramList:=TStringList.create;
paramList.Clear;
paramList.Add('loguser=testuser');
paramList.Add('logpass=duke7aunt');
paramList.Add('logclub=8005');
url := 'https://www.dfcdata.co.uk/integration/reports.php?report=live';
try
IdHTTP1.Post(url,paramList);
except
on E:Exception do
begin
showMessage('failed to post to: '+url);
ShowMessage('Exception message = '+E.Message);
end;
end;
end;
procedure TForm1.Button1Click(Sender: TObject);
var
reportType : String;
begin
PostData(Self);
reportType := 'live';
GetUrlToFile('',reportType+'.csv');
end;
procedure TForm1.GetUrlToFile(AURL, AFile : String);
var
Output : TMemoryStream;
success : Boolean;
begin
success := True;
Output := TMemoryStream.Create;
try
try
IdHTTP1.Get(AURL, Output);
IdHTTP1.Disconnect;
except
on E : Exception do
begin
ShowMessage('Get failed to GET from '+IdHTTP1.GetNamePath +'. Exception message = '+E.Message);
success := False;
end;
end;
if success = True then
begin
showMessage('Filed saved');
Output.SaveToFile(AFile);
end;
finally
Output.Free;
end;
end;
On each try I get "IOHandler is not valid" error. Obviously I'm not posting correctly to the initial page but can anyone advise me on what I'm missing? Also can I simply then hit the download URL after login or will I have to use cookies?
Thanks
There are several bugs in your code:
1) PostData() is requesting an HTTPS URL, but it is not assigning an SSL-enabled IOHandler to the TIdHTTP.IOHandler property. You need to do so.
2) Button1Click() is passing a URL to GetUrlToFile() that does not specify any protocol, so TIdHTTP will end up treating that URL as relative to its existing URL, and thus try to download from https://www.testurl.com/test/testurl.com/test/reports.phpinstead of https://testurl.com/test/reports.php. If you want to request a relative URL, don't include the hostname (or even the path in this case, since you are sending multiple requests to the same path, just different documents).
3) you are leaking the TIdHTTP object.
Issue 1) has now been resolved in another post:
Delphi 5 Indy/ics SSL workaround?
However I would greatly appreciate help on the rest, as follows.
Would I need to make a GET call with the same IdHTTP object and additional URL variable? or should I create a new IdHTTP object?
Would I need to record the session using cookies or can all of this be done with the same call?
Is the GET call above actually what I need to save a csv to file? I may also choose to handle it directly as the data will need importing anyway.
Currently the code gets the error: EIdHTTPProtocolException
This is the continue of my previous question:
Delphi, WebBrowser, Google Login, FusionTable
But the test with WinHTTP also failed as TWebBrowser based test...
And this is one question as you wish... :-)
I have one table what is NOW PUBLIC, but when we will buy non-free account it will be changed to private kind.
I created a simple WinHTTP test, but this also failed.
I can login, I got the "Auth" tag, but the next "private" request returns 401 error.
procedure TForm1.BitBtn1Click(Sender: TObject);
var
WinHttpReq, temp : variant;
URL, s : String;
params : TStringList;
authtoken, query, posts : string;
begin
URL := 'https://www.google.com/accounts/ClientLogin';
WinHttpReq := CreateOleObject('WinHttp.WinHttpRequest.5.1');
params := TStringList.Create;
try
params.Values['accountType'] := 'GOOGLE';
params.Values['Email'] := csEmail;
params.Values['Passwd'] := csPwd;
params.Values['service'] := 'fusiontables';
params.Values['source'] := csSource;
posts := EncodeParamsToURL(params);
finally
params.Free;
end;
URL := URL + '?' + posts;
WinHttpReq.Open('POST', URL, false);
WinHttpReq.Send();
s := WinHttpReq.ResponseText;
Memo1.Lines.Text := s;
params := TStringList.Create;
try
params.Text := s;
authtoken := params.Values['Auth'];
Edit1.Text := authtoken;
finally
params.Free;
end;
//query := URLEncode('SHOW TABLES');
query := URLEncode('select * from 1236965');
url := 'http://www.google.com/fusiontables/api/query?sql=' + query;
WinHttpReq.Open('POST', URL, false);
WinHttpReq.setRequestHeader('Authorization', 'GoogleLogin auth="' + authToken + '"');
WinHttpReq.Send();
s := WinHttpReq.ResponseText;
Memo1.Lines.Text := s;
end;
When I made "select", I got the rows.
But when I want to see the tablenames, I get 401 error...
I'm not sure what cause this error.
a. The free account don't have enough rights to access it privately
b. I set the header wrong
c. I set the csSource wrong (I set it "MyCompanyName-Test-1.0")
d. Other thing I don't know what...
Can anybody help me how to login and access the data successfully?
Change your method to GET:
WinHttpReq.Open('GET', URL, false);
And remove the " around your auth token:
WinHttpReq.setRequestHeader('Authorization', 'GoogleLogin auth=' + authToken);
The first change is to comply with the documentation (although POST also works). The second change fixes the error.