Trying to inspect the header information being posted from my app but fiddler doesn't seem to pick anything up.
I am also using a web service in my app and when I invoke some of the APIs I can see these requests in Fiddler, however, when doing custom requests using Indy 10 nothing seems to be picked up.
Does Indy use WinInet? If not, that's the problem. Fiddler2 inserts itself as a proxy in your internet settings, but programs like SoapUI that use their own communication stack don't use WinInet, and therefore don't (auto-magically) pass through Fiddler2. So you may need to mess with proxy settings.
I use a construction to handle the requests in Fiddler:
try
// lHTTP.IOHandler := lIOHandler; - even without this line works
lHTTP.ProxyParams.ProxyServer := '127.0.0.1';
lHTTP.ProxyParams.ProxyPort := 8888;
sResponse := lHTTP.Post('<URL>', slRequest);
Memo1.Lines.Text := sResponse;
finally
// lIOHandler.Free;
end;
Related
I call a webapi with a Delphi app, in some pcs, the call timeouts, while in other it works fine.
The request done with Postman works fine.
It is a simple custom ping webservice (URL is in Edit1.Text in the code below), in fact the answer is a textual "Pong".
This is the Delphi code of the call:
errormsg := '';
{
old way of setting custom headers
IdHTTP1.Request.CustomHeaders.AddValue('X-HTTP-Method-Override', 'ForwardCommand');
IdHTTP1.Request.CustomHeaders.AddValue('Connection', 'keep-alive');
IdHTTP1.Request.CustomHeaders.AddValue('Accept', '*/*');
IdHTTP1.Request.CustomHeaders.AddValue('User-Agent', 'QualibusSilent');
IdHTTP1.Request.CustomHeaders.AddValue('Content-Type', 'text/plain');
}
//better way of setting custom headers
IdHTTP1.Request.MethodOverride := 'ForwardCommand';
IdHTTP1.Request.Connection := 'keep-alive';
IdHTTP1.Request.UserAgent := 'myCustomUserAgent';
IdHTTP1.Request.ContentType := 'text/plain';
IdHTTP1.Request.Accept := '*/*';
IdSSLIOHandlerSocketOpenSSL1 := TIdSSLIOHandlerSocketOpenSSL.Create(IdHTTP1);
IdSSLIOHandlerSocketOpenSSL1.SSLOptions.Mode := sslmClient;
IdSSLIOHandlerSocketOpenSSL1.SSLOptions.SSLVersions:=
[sslvTLSv1,sslvTLSv1_1,sslvTLSv1_2];
startTime := GetTickCount;
Try
sHTML := IdHTTP1.Get(Edit1.Text);
Except
On E:Exception do
errormsg := e.Message;
End;
EndTime := GetTickCount;
ShowMessage('Time taken: ' +
IntToStr(endTime-startTime)+#13#10+'Error:'+errormsg);
Basically it is a call where instead of GET I do a custom method (ForwardCommand) that I call with X-HTTP-Method-Override.
In the code above I tried to add many headers so that the call is really as the Postman one.
If the call is done directly to the IP address it works, but if I call the https URL it timeouts, there is no evidence of error in the proxy server.
Checking the logs at the webserver side it seems the call is not performed at all.
And this occurs only from some Windows 10 machines, while in the majority of them the call is performed correctly.
Could you please suggest which could be the cause of the error? What should I try to change in the Delphi code to avoid the timeout so that Delphi behaves like Postman?
Thanks.
As stated in comments:
Why when Tidhttp uses proxyParams timeout does not occur and the call succeeds?
...
I finally got the reason for the Postman vs Indy behavior: Proxy. By passing proxy IP and port to TIdHTTP it works, Postman manages to retrieve the system proxy automatically and therefore it works.
There is no "system proxy" on Windows, however there is a proxy in the WinInet API, which is what Internet Explorer (and Edge?) relies on.
In any case, it sounds like the failing PCs don't have direct access to the Internet to begin with, only through a proxy. Indy has no concept of any "system proxy" on any platform, so you will have to assign the proxy settings to TIdHTTP manually, as you have discovered.
I'm recoding an old Delphi XE program using Delphi 10.3 Rio. It uses the TIdHTTPProxyServer Indy component listening on 127.0.0.1:80.
with IdHTTPProxyServer.Bindings.Add do begin
IP := '127.0.0.1';
Port := 80;
end;
IdHTTPProxyServer.Active := True;
For testing, I added 127.0.0.1 localtest123.com and 127.0.0.1 www.localtest123.com to the hosts file and disabled the DNS cache service. Then in multiple browers I requested http://localtest123.com/ and http://www.localtest123.com/. With OutputDebugString() I can see the connections accepted but then raises an "Unknown Protocol" error.
I debugged the exception in the TIdHTTPProxyServer.CommandPassThrough procedure in IdHTTPProxyServer.pas. It seems LURI.Protocol is an empty string which is why the RSHTTPUnknownProtocol is raised.
LContext := TIdHTTPProxyServerContext(ASender.Context);
LContext.FCommand := ASender.CommandHandler.Command; //<-'GET'
LContext.FTarget := ASender.Params.Strings[0]; //<-'/'
LContext.FOutboundClient := TIdTCPClient.Create(nil);
try
LURI := TIdURI.Create(LContext.Target); //<-'/'
try
TIdTCPClient(LContext.FOutboundClient).Host := LURI.Host; //<-''
if LURI.Port <> '' then begin //<-''
TIdTCPClient(LContext.FOutboundClient).Port := IndyStrToInt(LURI.Port, 80);
end
else if TextIsSame(LURI.Protocol, 'http') then begin //<-'' {do not localize}
TIdTCPClient(LContext.FOutboundClient).Port := IdPORT_HTTP;
end
else if TextIsSame(LURI.Protocol, 'https') then begin //<-'' {do not localize}
TIdTCPClient(LContext.FOutboundClient).Port := IdPORT_https;
end else begin
raise EIdException.Create(RSHTTPUnknownProtocol);
end;
I'm probably missing something but TIdHTTPProxyServer just works without much code so I have to ask for help on this exception. Thanks in advance!
You can't just redirect the domains in your HOSTS file and expect things to magically work. That is not how proxying works.
You must explicitly configure web browsers to make HTTP requests through an HTTP proxy so that they format the proper requests that a proxy would understand. Sending an HTTP request directly to a target web server is handled differently than sending the same HTTP request through a proxy.
You are getting the exception because the browser requests are not targeting your proxy properly.
For example, when a browser sends an HTTP GET request directly to a target web server, it connects directly to that server and then sends a request that looks something like this:
GET /path HTTP/1.1
Host: server.com
But, when it sends the same request through an HTTP proxy, it connects to the proxy and sends a request that looks more like this instead:
GET http://server.com/path HTTP/1.1
That extra path information in the GET line is missing in your browser requests, because you do not have your browsers configured for proxying, thus the exception when TIdHTTPProxyServer is trying to determine the info it needs to make a connection to the target web server and forward the current request to it.
This is fundamentally to how HTTP works, and how TIdHTTPProxyServer is designed to work.
Things are a bit more complicated when HTTPS is involved, but I'm leaving that detail out for now, as it is not relevant to your question about the exception.
UPDATE: in comments, you say:
In the XE version it never raised an exception when checking for the protocol which would still work today because I manually set the host and port in DoHTTPBeforeCommand.
In that old version, there was no exception raised, because TIdHTTPProxyServer did not check the protocol yet to differentiate between HTTP and HTTPS. You were able to manually fill in missing info when a request was received that was not specifically targeting your proxy. That is why things worked for you before.
In a later version, TIdHTTPProxyServer was updated to differentiate between HTTP and HTTPS when no port is explicitly specified in the request, so a default port is set based on protocol requested. That check happens before DoHTTPBeforeCommand() is called.
To get the old behavior back, you would have to alter TIdHTTPProxyServer's source code to delay the raising of the exception until after DoHTTPBeforeCommand() returns, so you have a chance to fill in missing values again.
If you file a feature request for that, I might consider adding it to Indy's official code.
I want send HTTP request (GET) without wait for the server response.
I have tried with IdHTTP by ignoring the response, eg.:
aIdHTTP := TIdHTTP.create;
try
aIdHTTP.Get('https://stackoverflow.com/');
finally
aIdHTTP.free;
free
but it always wait for the server response...
A trick is close the connection by raising an exception in the TIdHTTP.OnWork event when AWorkMode is wmRead, this close the connection sooner... but wait for the "first byte" response.
There is a way for send data without waiting for the response?
The short answer is NO, well not with TIdHTTP, anyway. HTTP is a protocol, and TIdHTTP follows the protocol, which includes reading the server's response.
As you discovered, you could raise an exception in the TIdHTTP.OnWork event, but that will only work once at least 1 byte is received from the server. You could specify a short TIdHTTP.ReadTimeout value to avoid that, though.
Another option is to close the socket connection in the TIdHTTP.OnHeadersAvailable event, but that requires TIdHTTP reading the server's complete response headers.
IF the response is using HTTP 1.1 chunking, you could enable the hoNoReadChunked flag in the TIdHTTP.HTTPOptions property. But that also requires reading the server's complete response header.
Really, the only way to accomplish what you are asking for is to not use TIdHTTP at all. Use TIdTCPClient instead, and manually send your own HTTP request and then close the connection after sending it, eg:
aIdClient := TIdTCPClient.Create;
try
aIdClient.Host := 'stackoverflow.com';
aIdClient.Port := 443;
aIdClient.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(aIdClient);
TIdSSLIOHandlerSocketOpenSSL(aIdClient.IOHandler).PassThrough := False;
aIdClient.Connect;
try
aIdClient.IOHandler.WriteLn('GET / HTTP/1.1');
aIdClient.IOHandler.WriteLn('Host: stackoverflow.com');
aIdClient.IOHandler.WriteLn('Connection: close');
aIdClient.IOHandler.WriteLn;
finally
aIdClient.Disconnect;
end;
finally
aIdClient.free;
end;
But what is the point of requesting a file and not actually downloading it?
If you just want to check that the server can be connected to, you don't have to send anything at all
aIdClient := TIdTCPClient.Create;
try
aIdClient.Host := 'stackoverflow.com';
aIdClient.Port := 443;
aIdClient.Connect;
aIdClient.Disconnect;
finally
aIdClient.free;
end;
If you are just trying to avoid downloading a file's actual content over HTTP, you could use TIdHTTP.Head() instead of TIdHTTP.Get():
aIdHTTP.Head('https://stackoverflow.com/');
I am including these in uses clause
IdAuthentication
,
IdAuthenticationDigest
,
IdAuthenticationNTLM
,
IdAuthenticationSSPI
Currently I have code that does this:
W.Request.BasicAuthentication := True;
W.Request.Username := AOptionsPtr^.AuthUsername;
W.Request.Password := AOptionsPtr^.AuthPassword;
And if I have access to OpenSSL:
TmpOpenSSL := TIdSSLIOHandlerSocketOpenSSL.Create;
TmpOpenSSL.SSLOptions.Method := sslvSSLv23;
TmpOpenSSL.SSLOptions.Mode := sslmClient;
TmpOpenSSL.SSLOptions.VerifyMode := [];
TmpOpenSSL.SSLOptions.VerifyDepth := 0;
//--
W.IOHandler := TmpOpenSSL;
From skimming the documentation for WinINet (yes, I know it is not Indy) it seems persistent connections is also required for authentication. I suppose this also goes for Indy? URL:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa384220(v=vs.85).aspx
I need to get this to work with SharePoint. The problem, however, is hat I have no intranet SharePoint server to test against. Thus I can not step through the code and see what works and what does not. However, I have a potential customer that can test it for me.
What more do I need to do to get above code working with SharePoint using Windows Authentication (NTML? SPPI?)
Will Indy automatically test and use proper auhentication?
do I need to set W.Request.BasicAuthentication := False; for auto authentication/detection to work?
If multiple requests are necessary (with first response being 401) I assume I need to add support for this in my own code when making a GET request? (To set authentication mode and make a new request?)
You can request a persistent connection by setting the Request.Connection property to 'keep-alive'.
TIdHTTP will check the server's WWW-Authorization header and compare it to the TIdAthentication classes you have included in your uses clause. The TIdHTTP.OnSelectAuthorization event will tell you which class was picked, and allow you to override it if needed. The TIdHTTP.OnAuthorization event will be triggered if authentication fails and different credentials are needed.
The BasicAuthrnication property simply allows TIdHTTP to fall back to TIdBasicAuthentication if no other TIdAuthentication class is assigned.
No, you do not need to handle multi-request authentications manually, like NTLM. TIdHTTP and TIdAuthentication handle those details for you.
I tried to employ Indy 10.5.5 (shipped with Delphi 2010) for:
connecting to telnet server
performing username/password authentication (gaining access to the command shell)
executing a command with returning resulting data back to application
and had no success, additionally i'm completely lost in spaghetti logic of Indy's internals and now have no idea why it didnt work or how i supposed to send strings to the server and grab the results. Need some sample code to study.
Formal form of the question: Where can i get 3-rd party contributed demo covering TIdTelnet component? (indyproject.org demos webpage do not have one)
The main problem with Telnet is that it DOES NOT utilize a command/response model like most other Internet protocols do. Either party can send data at any time, and each direction of data is independant from the other direction. This is reflected in TIdTelnet by the fact that it runs an internal reading thread to receive data. Because of this, you cannot simply connect, send a command, and wait for a response in a single block of code like you can with other Indy components. You have to write the command, then wait for the OnDataAvailable event to fire, and then parse the data to determine what it actually is (and be prepared to handle situations where partial data may be received, since that is just how TCP/IP works).
If you are connecting to a server that actually implements a command/response model, then you are better off using TIdTCPClient directly instead of TIdTelnet (and then implement any Telnet sequence decoding manually if the server really is using Telnet, which is rare nowadays but not impossible). For Indy 11, we might refactor TIdTelnet's logic to support a non-threaded version, but that is undecided yet.
done with indy.
no comments.. just som old code :-)
telnet don't like the send string kommand.. use sendch.
telnetdude.Host := 1.1.1.1;
try
telnetdude.connect;
except
on E: Exception do begin
E.CleanupInstance;
end; {except}
if telnetdude.Connected then begin
for i := 1 to length(StringToSend) do telnetdude.sendch(StringToSend[i]);
telnetdude.sendch(#13);
end;
end; {while}
end; {if}
if telnetdude.Connected then telnetdude.Disconnect;
end;
I hope this helps anyone looking for answers to a similar question.
Firstly, It would seem the typical command/response model (as mentioned above, does indeed NOT apply).
So I just got it working for some very simple application (rebooting my router).
Specific additions to above code from Johnny Lanewood (and perhaps some clarification)
a) You have to send #13 to confirm the command
b) I got "hangs" on every command I sent / response I requested UNTIL I enabled ThreadedEvent. (this was my big issue)
c) the OnDataAvailable event tells you when new data is available from the Telnet Server - however there are no guarantees as to what this data is - i.e. it's pretty what you get in the command line / what ever is appended to the previous responses. But is is NOT a specific response line to your command - it's whatever the telnet server returns (could be welcome info, ASCII drawings etc etc.)
Given (c) above, one would rather check the OnDataAvailable event and parse the data (knowing what you'd expect). When the output stops (i.e. you need build a mechanism for this), you can parse the data and determine whether the server is ready for something new from the client. For the purpose of my code below, I set a read timemout and I just used Sleep(2000) - ignorantly expecting no errors and that the server would be ready after the sleep for the next command.
My biggest stumbling block was ThreadedEvent := True (see above in b)
Thus, my working solution (for specific application, and possibly horrible to some).
lIDTelnet := TIdTelnet.Create(nil);
try
lIdTelnet.ReadTimeout := 30000;
lIDTelnet.OnDataAvailable := TDummy.Response;
lIDTelnet.OnStatus := TDummy.Status;
lIdTelnet.ThreadedEvent := True;
try
lIDTelnet.Connect('192.168.0.1', 23);
if not lIDTelnet.Connected then
Raise Exception.Create('192.168.0.1 TELNET Connection Failed');
Sleep(2000);
lIdtelnet.SendString(cst_user + #13);
Sleep(2000);
lIdtelnet.SendString(cst_pass + #13);
Sleep(2000);
lIdtelnet.SendString(cst_reboot + #13);
Sleep(2000);
if lIDTelnet.Connected then
lIDTelnet.Disconnect;
except
//Do some handling
end;
finally
FreeAndNil(lIdTelnet);
end;
and then
class procedure TDummy.Response(Sender: TIdTelnet; const Buffer: TIdBytes);
begin
Write(TDummy.ByteToString(Buffer));
end;
class function TDummy.ByteToString(
const aBytes: TIdBytes): String;
var
i : integer;
begin
result := '';
for i := 0 to Length(aBytes) -1 do
begin
result := result + Char(aBytes[i]);
end;
end;