I call a webapi with a Delphi app, in some pcs, the call timeouts, while in other it works fine.
The request done with Postman works fine.
It is a simple custom ping webservice (URL is in Edit1.Text in the code below), in fact the answer is a textual "Pong".
This is the Delphi code of the call:
errormsg := '';
{
old way of setting custom headers
IdHTTP1.Request.CustomHeaders.AddValue('X-HTTP-Method-Override', 'ForwardCommand');
IdHTTP1.Request.CustomHeaders.AddValue('Connection', 'keep-alive');
IdHTTP1.Request.CustomHeaders.AddValue('Accept', '*/*');
IdHTTP1.Request.CustomHeaders.AddValue('User-Agent', 'QualibusSilent');
IdHTTP1.Request.CustomHeaders.AddValue('Content-Type', 'text/plain');
}
//better way of setting custom headers
IdHTTP1.Request.MethodOverride := 'ForwardCommand';
IdHTTP1.Request.Connection := 'keep-alive';
IdHTTP1.Request.UserAgent := 'myCustomUserAgent';
IdHTTP1.Request.ContentType := 'text/plain';
IdHTTP1.Request.Accept := '*/*';
IdSSLIOHandlerSocketOpenSSL1 := TIdSSLIOHandlerSocketOpenSSL.Create(IdHTTP1);
IdSSLIOHandlerSocketOpenSSL1.SSLOptions.Mode := sslmClient;
IdSSLIOHandlerSocketOpenSSL1.SSLOptions.SSLVersions:=
[sslvTLSv1,sslvTLSv1_1,sslvTLSv1_2];
startTime := GetTickCount;
Try
sHTML := IdHTTP1.Get(Edit1.Text);
Except
On E:Exception do
errormsg := e.Message;
End;
EndTime := GetTickCount;
ShowMessage('Time taken: ' +
IntToStr(endTime-startTime)+#13#10+'Error:'+errormsg);
Basically it is a call where instead of GET I do a custom method (ForwardCommand) that I call with X-HTTP-Method-Override.
In the code above I tried to add many headers so that the call is really as the Postman one.
If the call is done directly to the IP address it works, but if I call the https URL it timeouts, there is no evidence of error in the proxy server.
Checking the logs at the webserver side it seems the call is not performed at all.
And this occurs only from some Windows 10 machines, while in the majority of them the call is performed correctly.
Could you please suggest which could be the cause of the error? What should I try to change in the Delphi code to avoid the timeout so that Delphi behaves like Postman?
Thanks.
As stated in comments:
Why when Tidhttp uses proxyParams timeout does not occur and the call succeeds?
...
I finally got the reason for the Postman vs Indy behavior: Proxy. By passing proxy IP and port to TIdHTTP it works, Postman manages to retrieve the system proxy automatically and therefore it works.
There is no "system proxy" on Windows, however there is a proxy in the WinInet API, which is what Internet Explorer (and Edge?) relies on.
In any case, it sounds like the failing PCs don't have direct access to the Internet to begin with, only through a proxy. Indy has no concept of any "system proxy" on any platform, so you will have to assign the proxy settings to TIdHTTP manually, as you have discovered.
I am in some strange situation. I made DataSnap REST server and client. All REST server's methods are call by client through TRESTClient. My REST Server is Apache Module. Also I used TSQLConnection & TDSClientCallbackChannelManager for Peer-to-Peer callback in cleint. I set TDSServer ChannelResponseTimeout = 0 and TDSHTTPWebDispatcher SessionTimeout = 0. Still my client timed out after few seconds. I set TDSClientCallbackChannelManager CommunicationTimeout=0 and ConnectionTimeout=0. The error I am getting in TWinHTTPClient.DoExecuteRequest method of System.Net.HttpClient.Win. Strange thing is on debug mode I got AV but in exe mode I don't receive any AV but none of my callback is working though the REST methods are executing. I also tried set LifeCyle of TDSServerClass to Session & Invocation, both gives timed out. Below is some code for DSClientCallback & TSQLConnection:
SQLConnection.Params.Values['HostName'] := SERVERIP;
SQLConnection.Params.Values['Port'] := SERVER_PORT.ToString;
SQLConnection.Params.Values['ConnectionTimeout'] := '0';
SQLConnection.Connected := True;
ClientCallbackManager.CommunicationTimeout := '0';
ClientCallbackManager.ConnectionTimeout := '0';
ClientCallbackManager.DSHostname := SERVERIP;
ClientCallbackManager.DSPort := SERVER_PORT.ToString;
fClientCallbackId := TDSTunnelSession.GenerateSessionId;
ClientCallbackManager.DSPath := 'mypath';
ClientCallbackManager.ManagerId := TDSTunnelSession.GenerateSessionId;
fClientId := ClientCallbackManager.ManagerId;
ClientCallbackManager.RegisterCallback(fClientCallbackId,
'mychannel', TServerCallback.Create);
What I am doing wrong or missing? Please help. I also post this to Embarcadero Datasnap Forum without any response https://forums.embarcadero.com/thread.jspa?threadID=229678&tstart=0
I deploy my application in environments heavily stricken with firewalls. Frequently I find myself using Telnet to check if a port is open and accessible in the network.
Now I would like to implement an equivalent functionality of the command, Telnet [domainname or ip] [port], in Delphi.
Is it adequate that I just attempt to open and close a TCP/IP socket without sending or receiving any data?
Is there any risk that I might crash the arbitrary application/service listening on the other end?
Here's my code:
function IsPortActive(AHost : string; APort : Word):boolean;
var IdTCPClient : TIdTCPClient;
begin
IdTCPClient := TIdTCPClient.Create(nil);
try
try
IdTCPClient.Host := AHost;
IdTCPClient.Port := APort;
IdTCPClient.Connect;
except
//Igonre exceptions
end;
finally
result := IdTCPClient.Connected;
IdTCPClient.Disconnect;
FreeAndNil(IdTCPClient);
end;
end;
If you just want to check whether the port is open, then you can use this:
function IsPortActive(AHost : string; APort : Word): boolean;
var
IdTCPClient : TIdTCPClient;
begin
Result := False;
try
IdTCPClient := TIdTCPClient.Create(nil);
try
IdTCPClient.Host := AHost;
IdTCPClient.Port := APort;
IdTCPClient.Connect;
Result := True;
finally
IdTCPClient.Free;
end;
except
//Ignore exceptions
end;
end;
But that only tells you if any server app has opened the port. If you want to make sure that YOUR server app opened the port, then you will have to actually communicate with the server and make sure its responses are what you are expecting. For this reason, many common server protocols provide an initial greeting so clients can identify the type of server they are connected to. You might consider adding a similar greeting to your server, if you are at liberty to make changes to your communication protocol.
Simply opening a connection to the server does not impose any risk of crashing the server, all it does is momentarily occupy a slot in the server's client list. However, if you actually send data to the server, and the server app you are connected to is not your app, then you do run a small risk if the server cannot handle arbitrary data that does not conform it its expected protocol. But that is pretty rare. Sending a small command is not uncommon and usually pretty safe, you will either get back a reply (which may be in a format that does not conform to your protocol, so just assume failure), or you may not get any reply at all (like if the server is waiting for more data, or simply is not designed to return a reply) in which case you can simply time out the reading and assume failure.
I have created a webserver using Delphi2007 and Indy10.
The server runs fine initially, but after time (usually between 8 and 48 hours) the DoMaxConnectionsExceeded method begins to fire; and my web server no longer works properly. Currently I have MaxConnections set to 500. I have experimented in the past with changing this setting. And it does seem the larger the value the longer the web server will live. So it makes me think connections are not being released.
Am I doing something wrong in my instantiation? Why are connections not being released?
Is there a way to get a list of all connections (by doing this I can check ip addresses and see if it might be DOS attack). Also a way to get the url they are trying to hit?
I have also experimented with the KeepAlive property with no change.
Should I set MaxConnections to 0?
Source Code for instantiating the TIdHTTPServer:
IdHTTPServer1 := TIdHTTPServer.Create;
IdHTTPServer1.MaxConnections := 500;
IdHTTPServer1.AutoStartSession := True;
IdHTTPServer1.KeepAlive := FGlobalKeepAlive;
IdHTTPServer1.SessionState := True;
IdHTTPServer1.OnCommandGet := IdHTTPServer1CommandGet;
IdHTTPServer1.onexception := IdHttpServerexception;
IdHTTPServer1.onlistenexception := IdHttpServerlistenexception;
idHttpServer1.ParseParams := True;
idHttpServer1.OnQuerySSLPort := QuerySSLPort;
idHttpServer1.IOHandler := ServerIOHandler;
idHttpServer1.Bindings.Add.Port := 80;
idHttpServer1.Bindings.Add.Port := 443;
IdHTTPServer1.Active := True;
Update - want to add I suspect it might be related to SSL. I have similar Indy based web servers that don't need SSL. While they do periodically fail, they don't fail nearly as often. But with these I am not logging DoMaxConnectionsExceeded. I will add tracking of this event to see if and when they do fail it is because maxconnections is exceeded.
Found out what was wrong. While I had set KeepAlive to false on the idHTTPServer component, in another area I was manually setting it on the ResponseInfo:
Aresponseinfo.Connection:='keep-alive';
In Socket applications programmed by TCPServer/Client components, usually we active server side, then connect client to server, and when we need to get or send data from one side to other, first we send a command from client to server and a communication will starts.
But the problem is that always we need to start conversation from client side!
I want to ask is any idea for start conversation randomly from server side without client side request?
I need this functionality for notify client(s) from server side. for example, when a registered user (client-side) connected to server, other connected users (on other client-sides), a notification must send from server to all users (like Yahoo Messenger).
I'm using TIdCmdTCPServer and TIdTCPClient components
You are using TIdCmdTCPServer. By definition, it sends responses to client-issued commands. For what you are asking, you should use TIdTCPServer instead, then you can do whatever you want in the TIdTCPServer.OnExecute event.
What you ask for is doable, but its implementation depends on your particular needs for your protocol.
If you just want to send unsolicited server-to-client messages, and never responses to client-to-server commands, then the implementation is fairly straight forward. Use TIdContext.Connection.IOHandler when needed. You can loop through existing clients in the TIdTCPServer.Contexts list, such as inside the TIdTCPServer.OnConnect and TIdTCPServer.OnDisconnect events. On the client side, you need a timer or thread to check for server messages periodically. Look at TIdCmdTCPClient and TIdTelnet for examples of that.
But if you need to mix both client-to-server commands and unsolicited server-to-client messages on the same connection, you have to design your protocol to work asynchronously, which makes the implementation more complex. Unsolicited server messages can appear at anytime, even before the response to a client command. Commands need to include a value that is echoed in the response so clients can match up responses, and the packets need to be able to differentiate between a response and an unsolicited message. You also have to give each client its own outbound queue on the server side. You can use the TIdContext.Data property for that. You can then add server messages to the queue when needed, and have the OnExecute event send the queue periodically when it is not doing anything else. You still need a timer/thread on the client side, and it needs to handle both responses to client commands and unsolicited server messages, so you can't use TIdConnection.SendCmd() or related methods, as it won't know what it will end up reading.
I have posted examples of both approaches in the Embarcadero and Indy forums many times before.
Clients initiate communication. That is the definition of a client–the actor that initiates the communication. Once the connection is established though, both sides can send data. So, the clients connect to the server. The server maintains a list of all connected clients. When the server wants to send out communications it just sends the data to all connected clients.
Since clients initiate communication, it follows that, in the event of broken communication, it is the client's job to re-establish connection.
If you want to see working code examples where server sends data, check out Indy IdTelnet: the telnet client uses a thread to listen to server messages. There is only one socket, created by the client, but the server uses the same socket for its messages to the client, at any time.
The client starts the connection, but does not have to start a conversation by saying 'HELLO' or something like that.
Technically, the client only needs to open the socket connection, without sending any additional data. The client can remain quiet as long as he wants, even until the end of the connection.
The server has a socket connection to the client as soon as the client has connected. And over this socket, the server can send data to the client.
Of course, the client has to read from the connection socket to see the server data. This can be done in a loop in a background thread, or even in the main thread (not in a VCL application of course as it would block).
Finally, this is the code that I used to solve my problem:
// Thread at client-side
procedure FNotifRecieverThread.Execute;
var
str: string;
MID: Integer;
TCP1: TIdTCPClient;
begin
if frmRecieverMain.IdTCPClient1.Connected then
begin
TCP1 := TIdTCPClient.Create(nil);
TCP1.Host := frmRecieverMain.IdTCPClient1.Host;
TCP1.Port := frmRecieverMain.IdTCPClient1.Port;
TCP1.ConnectTimeout := 20000;
while True do
begin
try
TCP1.Connect;
while True do
begin
try
str := '';
TCP1.SendCmd('checkmynotif');
TCP1.Socket.WriteLn(IntToStr(frmRecieverMain.UserID));
str := TCP1.Socket.ReadLn;
if Pos('showmessage_', str) = 1 then
begin
MID := StrToInt(Copy(str, Pos('_', str) + 1, 5));
frmRecieverMain.NotifyMessage(MID);
end
else
if str = 'updateusers' then
begin
LoadUsers;
frmRecieverMain.sgMsgInbox.Invalidate;
frmRecieverMain.sgMsgSent.Invalidate;
frmRecieverMain.cbReceipent.Invalidate;
end
else
if str = 'updatemessages' then
begin
LoadMessages;
frmRecieverMain.DisplayMessages;
end;
except
// be quite and try next time :D
end;
Sleep(2000);
end;
finally
TCP1.Disconnect;
TCP1.Free;
end;
Sleep(5000);
end;
end;
end;
// And command handlers at server-side
procedure TfrmServer.cmhCheckMyNotifCommand(ASender: TIdCommand);
var
UserID, i: Integer;
str: string;
begin
str := 'notifnotfound';
UserID := StrToIntDef(ASender.Context.Connection.Socket.ReadLn, -1);
for i := 0 to NotificationStack.Count - 1 do
if NotificationStack.Notifs[i].Active and
(NotificationStack.Notifs[i].UserID = UserID)
then
begin
NotificationStack.Notifs[i].Active := False;
str := NotificationStack.Notifs[i].NotiffText;
Break;
end;
ASender.Context.Connection.Socket.WriteLn(str);
end;
// And when i want to some client notificated from server, I use some methodes like this:
procedure TfrmServer.cmhSetUserOnlineCommand(ASender: TIdCommand);
var
UserID, i: Integer;
begin
UserID := StrToIntDef(ASender.Context.Connection.Socket.ReadLn, -1);
if UserID <> -1 then
begin
for i := 0 to OnLineUsersCount - 1 do // search for duplication...
if OnLineUsers[i].Active and (OnLineUsers[i].UserID = UserID) then
Exit; // duplication rejected!
Inc(OnLineUsersCount);
SetLength(OnLineUsers, OnLineUsersCount);
OnLineUsers[OnLineUsersCount - 1].UserID := UserID;
OnLineUsers[OnLineUsersCount - 1].Context := ASender.Context;
OnLineUsers[OnLineUsersCount - 1].Active := True;
for i := 0 to OnLineUsersCount - 1 do // notify all other users for refresh users list
if OnLineUsers[i].Active and (OnLineUsers[i].UserID <> UserID) then
begin
Inc(NotificationStack.Count);
SetLength(NotificationStack.Notifs, NotificationStack.Count);
NotificationStack.Notifs[NotificationStack.Count - 1].UserID := OnLineUsers[i].UserID;
NotificationStack.Notifs[NotificationStack.Count - 1].NotiffText := 'updateusers';
NotificationStack.Notifs[NotificationStack.Count - 1].Active := True;
end;
end;
end;