I am trying to intercept GET/POST that i get from IdHTTPProxyServer and send it to IdHTTP so i can replicate the GET/POST and eventually get cookies so I can login in to any website.
How can this be improved? If i try to execute this code it crashes.
procedure TForm1.IdHTTPProxyServer1HTTPDocument(
AContext: TIdHTTPProxyServerContext; var VStream: TStream);
begin
if AContext.Command='POST' then begin
EmbeddedWB1.LoadFromString(IdHTTP1.Post(AContext.Target,AContext.Headers.Text)); << CRASH
end;
if AContext.Command='GET' then begin
EmbeddedWB1.LoadFromString(IdHTTP1.Get(AContext.Target)); << CRASH
end;
end;
When using the OnHTTPDocument event, you need to look at the TIdHTTPProxyServerContext.TransferSource property to know if the event is being triggered for a client request that contains body data, or is being triggered for the target server's response to the client request. Sending your own GET/POST request only makes sense when processing client requests, however the OnHTTPDocument event is not likely to ever be triggered for a GET request since there is no body data to capture.
You don't need to use TIdHTTP in order to get the server's cookies. Let TIdHTTPProxyServer do its work normally, and then you can extract the cookies from the TIdHTTPProxyServerContext.Headers property in the OnHTTPResponse event, eg:
procedure TForm1.IdHTTPProxyServer1HTTPResponse(AContext: TIdHTTPProxyServerContext);
var
Cookies: TStringList;
begin
Cookies := TStringList.Create;
try
AContext.Headers.Extract('Set-Cookie', Cookies);
// use Cookies as needed, such as:
// URI := TIdURI.Create(AContext.Target);
// try
// CookieManager1.AddServerCookies(Cookies, URI);
// finally
// URI.Free;
// end;
finally
Cookies.Free;
end;
end;
Related
I have seen a lot of examples online, but I cannot understand why my code doesn't work.
I have an url that looks like this:
http://www.domain.com/confirm.php?user=USERNAME&id=THEID
confirm.php is a page that does some checks on a MySQL database and then the only output of the page is a 0 or a -1 (true or false):
<?php
//long code...
if ( ... ) {
echo "0"; // success!
die();
} else {
echo "-1"; // fail!
die();
}
?>
My Delphi FireMonkey app has to open the URL above, passing the username and the id, and then read the result of the page. The result is only a -1 or a 0. This is the code.
//I have created a subclass of TThread
procedure TRegister.Execute;
var
conn: TIdHTTP;
res: string;
begin
inherited;
Queue(nil,
procedure
begin
ProgressLabel.Text := 'Connecting...';
end
);
//get the result -1 or 0
try
conn := TIdHTTP.Create(nil);
try
res := conn.Get('http://www.domain.com/confirm.php?user='+FUsername+'&id='+FPId);
finally
conn.Free;
end;
except
res := 'error!!';
end;
Queue(nil,
procedure
begin
ProgressLabel.Text := res;
end
);
end;
The value of res is always error!! and never -1 or 0. Where is my code wrong? The error caught from on E: Exception do is:
HTTP/1.1 406 not acceptable
I have found a solution using System.Net.HttpClient. I can simply use this function
function GetURL(const AURL: string): string;
var
HttpClient: THttpClient;
HttpResponse: IHttpResponse;
begin
HttpClient := THTTPClient.Create;
try
HttpResponse := HttpClient.Get(AURL);
Result := HttpResponse.ContentAsString();
finally
HttpClient.Free;
end;
end;
This works and gives me -1 and 0 as I expected. To get an example of a working code I have tested this:
procedure TForm1.Button1Click(Sender: TObject);
function GetURL(const AURL: string): string;
var
HttpClient: THttpClient;
HttpResponse: IHttpResponse;
begin
HttpClient := THTTPClient.Create;
try
HttpResponse := HttpClient.Get(AURL);
Result := HttpResponse.ContentAsString();
finally
HttpClient.Free;
end;
end;
function GetURLAsString(const aURL: string): string;
var
lHTTP: TIdHTTP;
begin
lHTTP := TIdHTTP.Create;
try
Result := lHTTP.Get(aURL);
finally
lHTTP.Free;
end;
end;
begin
Memo1.Lines.Add(GetURL('http://www.domain.com/confirm.php?user=user&id=theid'));
Memo1.Lines.Add(GetURLAsString('http://www.domain.com/confirm.php?user=user&id=theid'))
end;
end.
The first function works perfectly but Indy raises the exception HTTP/1.1 406 not acceptable. It seems that Indy cannot automatically handle the content type of the page. Here you can see the REST Debugger log:
HTTP Error 406 Not acceptable typically means that the server is not able to respond with the content type the client wanted. Both the Server and Client need to appropriately use the MIME type as you need. In this case, your client's Accept headers should provide the desired type of response, and your server should also be responding with the same. In your case, the Content-Type will most likely be text/plain.
So long story short, your client is expecting a MIME type which the server does not explicitly return in its response. The problem could be on either side, or perhaps both.
Your Client's Accept headers must provide the MIME type(s) you expect and need. Specifically Accept, Accept-Charset, Accept-Language, Accept-Encoding. By default in Indy TIdHTTP, these headers should accept essentially anything, assuming these headers haven't been overwritten. The Accept header is by default set to text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 where the */* opens the door for any MIME type.
Your Server's Response's Content-Type must be one of the provided MIME types, as well as the format of the response as also desired by the client. It is likely that your HTTP server is not providing the appropriate Content-Type in its response. If the server responds with anything in the */* filter (which should mean everything), then the client will accept it (assuming the server responds with text/plain). If the server responds with an invalid content type (such as just text or plain), then it could be rejected.
In my Indy (TIdHTTPServer) based web server application, every HTTP session has a separate TDataModule which will be created 'on demand':
procedure TExampleHTTPServer.OnHandleRequest(Request:
TIdHTTPRequestInfo; Response: TIdHTTPResponseInfo);
const
KEY = 'datamodule';
var
Pos: Integer;
DM: TDemoDataModule;
Tmp: string;
S: string;
begin
Request.Session.Lock;
try
Pos := Request.Session.Content.IndexOf(KEY);
if Pos <> -1 then
begin
DM := TDemoDataModule(Request.Session.Content.Objects[Pos]);
end
else
begin
WriteLn(Format('Create datamodule for session %s',
[Request.Session.SessionID]));
DM := TDemoDataModule.Create(nil);
Request.Session.Content.AddObject(KEY, DM);
end;
finally
Request.Session.Unlock;
end;
... // do something with the datamodule - this is the interesting part
So whenever a request is sent from the same client HTTP session, the server tries to find the datamodule in the session objects, and uses it (to store session specific data, execute database queries etc.). If there is no datamodule instance yet, the server creates it.
This works, however there is a risk of concurrent access: because the session is identified by a session token (stored in a cookie), it may happen that the same client opens two connections at the same time with the same session token, and so the server will receive two requests which both want to access the same datamodule.
For this reason, thread safe access to the datamodule instance is required.
I guess that the simplest solution is a TMonitor, so every access to the datamodule would be wrapped:
TMonitor.Enter(DM);
try
DM.SomeProperty := AValue;
...
finally
TMonitor.Exit(DM);
end;
Or am I missing some other, maybe more elegant, solution?
currently I am able to run a command but i cant figure out how to get the result into a string.
I do a get like so
idhttp1.get('http://codeelf.com/games/the-grid-2/grid/',TStream(nil));
and everything seems to run ok, in wireshark i can see the results from that command. Now if i do
HTML := idhttp1.get('http://codeelf.com/games/the-grid-2/grid/');
it will freeze up the app, in wireshark i can see it sent the GET and got a response, but dont know why it freezes up. HTML is just a string var.
EDIT FULL CODE
BUTTON CLICK
login(EUserName.Text,EPassWord.Text);
procedure TForm5.Login(name: string; Pass: string);
var
Params: TStringList;
html : string;
begin
Params := TStringList.Create;
try
Params.Add('user='+name);
Params.Add('pass='+pass);
Params.Add('sublogin=Login');
//post password/username
IdHTTP1.Post('http://codeelf.com/games/the-grid-2/grid/', Params);
//get the grid source
HTML := idhttp1.Get('http://codeelf.com/games/the-grid-2/grid/');
finally
Params.Free;
end;
llogin.Caption := 'Logged In';
end;
RESPONCE
The responce i get says Transfer-Encoding: chunked\r\n and Content-Type: text/html\r\n dont know if that matters.
Thanks
Indy has support for some types of streamed HTTP responses (see New TIdHTTP hoNoReadMultipartMIME flag), but this will only help if the server uses multipart/* responses. The linked blog article explains the details further and also shows how the Indy HTTP component can feed a MIME decoder with a continuous response stream.
If this is not applicable to your case, a workaround is to go down to the "raw" TCP layer, which means send the HTTP request using a TIdTCPClient component, and then read the response line by line (or byte by byte) from the IOHandler. This gives total control over response handling. Request and Response should be processed in a thread to decouple it from the main thread.
TIdHTTP.Post() returns the response data, you should not be calling TIdHTTP.Get() to retrieve it separately:
procedure TForm5.Login(name: string; Pass: string);
var
Params: TStringList;
html : string;
begin
Params := TStringList.Create;
try
Params.Add('user='+name);
Params.Add('pass='+pass);
Params.Add('sublogin=Login');
//post password/username
HTML := IdHTTP1.Post('http://codeelf.com/games/the-grid-2/grid/', Params);
finally
Params.Free;
end;
llogin.Caption := 'Logged In';
end;
I'm trying to access the URL Shortener ( http://goo.gl/ ) via its API from within Delphi.
However, the only result I get is: HTTP/1.0 400 Bad Request (reason: parseError)
Here is my code (on a form with a Button1, Memo1 and IdHTTP1 that has IdSSLIOHandlerSocketOpenSSL1 as its IOHandler. I got the necessary 32-bit OpenSSL DLLs from http://indy.fulgan.com/SSL/ and put them in the .exe's directory):
procedure TFrmMain.Button1Click(Sender: TObject);
var html, actionurl: String;
makeshort: TStringList;
begin
try
makeshort := TStringList.Create;
actionurl := 'https://www.googleapis.com/urlshortener/v1/url';
makeshort.Add('{"longUrl": "http://slashdot.org/stories"}');
IdHttp1.Request.ContentType := 'application/json';
//IdHTTP1.Request.ContentEncoding := 'UTF-8'; //Using this gives error 415
html := IdHTTP1.Post(actionurl, makeshort);
memo1.lines.add(idHTTP1.response.ResponseText);
except on e: EIdHTTPProtocolException do
begin
memo1.lines.add(idHTTP1.response.ResponseText);
memo1.lines.add(e.ErrorMessage);
end;
end;
memo1.Lines.add(html);
makeshort.Free;
end;
Update: I have left off my API key in this example (should usually work well without one for a few tries), but if you want to try it with your own, you can substitute the actionurl string with
'https://www.googleapis.com/urlshortener/v1/url?key=<yourapikey>';
The ParseError message leads me to believe that there might be something wrong with the encoding of the longurl when it gets posted but I wouldn't know what to change.
I've been fuzzing over this for quite a while now and I'm sure the mistake is right before my eyes - I'm just not seeing it right now.
Any help is therefore greatly appreciated!
Thanks!
As you discovered, the TStrings overloaded version of the TIdHTTP.Post() method is the wrong method to use. It sends an application/x-www-form-urlencoded formatted request, which is not appropriate for a JSON formatted request. You have to use the TStream overloaded version of the TIdHTTP.Post() method instead`, eg:
procedure TFrmMain.Button1Click(Sender: TObject);
var
html, actionurl: String;
makeshort: TMemoryStream;
begin
try
makeshort := TMemoryStream.Create;
try
actionurl := 'https://www.googleapis.com/urlshortener/v1/url';
WriteStringToStream(makeshort, '{"longUrl": "http://slashdot.org/stories"}', IndyUTF8Encoding);
makeshort.Position := 0;
IdHTTP1.Request.ContentType := 'application/json';
IdHTTP1.Request.Charset := 'utf-8';
html := IdHTTP1.Post(actionurl, makeshort);
finally
makeshort.Free;
end;
Memo1.Lines.Add(IdHTTP1.Response.ResponseText);
Memo1.Lines.Add(html);
except
on e: Exception do
begin
Memo1.Lines.Add(e.Message);
if e is EIdHTTPProtocolException then
Memo1.lines.Add(EIdHTTPProtocolException(e).ErrorMessage);
end;
end;
end;
From the URL shortener API docs:
Every request your application sends to the Google URL Shortener API
needs to identify your application to Google. There are two ways to
identify your application: using an OAuth 2.0 token (which also
authorizes the request) and/or using the application's API key.
Your example does not contain code for OAuth or API key authentication.
To authenticate with an API key, the docs are clear:
After you have an API key, your application can append the query
parameter key=yourAPIKey to all request URLs.
I'm using Indy TIdHTTP along with TIdCookieManager. I would like to check the current cookies for the request I'm about to send and identify the likelyhood that it will be valid (I know I can't be 100% sure the server will accept my request). If there are no cookies, or if they're expired, I will want to login first and acquire new cookies. Otherwise, just send the request.
How would I go about doing such a check? I believe I have to check the cookie manager before I send a request, but don't know what to check.
Try something like this:
function CheckCookies(Cookies: TIdCookieManager; const TargetURL: String): Boolean;
var
URL: TIdURI;
Headers: TIdHeaderList;
begin
Result := False;
URL := TIdURI.Create(TargetURL);
try
Headers := TIdHeaderList.Create(QuoteHTTP);
try
Cookies.GenerateClientCookies(URL, False, Headers);
Result := Headers.Count > 0;
finally
Headers.Free;
end;
finally
URL.Free;
end;
end;
.
if not CheckCookies(IdHTTP1.CookieManager, 'http://www.someurl.com/') then
begin
// login and get new cookies ...
end;
Like already stated in the comments you cannot perform an actual acceptance check on the client, only the server can do that.
However you can filter out expired or invalid cookies:
function filterInvalidCookies(cookies: TIdCookies; targetURL: TIdURI): Boolean;
var
c: Integer;
begin
Result := False;
c := 0;
while (cookies.Count > c) do
if (not cookies[c].IsExpired and cookies[c].IsAllowed(targetURL, False) and
(cookies[c].CookieName <> '')) then
begin
Result := True;
Inc(c);
end
else
cookies.Delete(c);
end;
The function removes invalid cookies and returns False if there are no valid ones left. Call it before a request like this:
if (Assigned(con.CookieManager)) then
filterInvalidCookies(con.CookieManager.CookieCollection,
TIdURI.Create('http://www.someurl.com/'));
where con is an TIdHTTP object.
You can do additional, maybe target page specific checks of course.