I'm using TIdFTP (Indy 10.2) in Delphi XE2; the method 'connect' (to create a connection...) is very slow; it takes 7 - 8 seconds to accomplish task.
I have tried with different configurations: Passive true or false, AutoLogin true or false, but more or less the same situation.
Is there any particular configuration to consider ?
First, 10.2 is a VERY OLD and OUTDATED version. The current version is 10.6. You should seriously consider upgrading.
Second, Connect() does a lot of work. It always sends a SYST command at a minimum, but depending on the values of the following properties, it may send a lot of additional commands as well:
UseHOST=True
HOST
AutoLogin=True
AUTH 1-4 times (only if UseTLS is utUseRequireTLS or utUseExplicitTLS)
XAUT (only if ProxySettings.ProxyType is fpcmNone)
USER
PASS
ACCT
OPEN (only if ProxySettings.ProxyType is fpcmOpen)
EPSV ALL (only if EPSV is supported)
SITE ZONE (only if SITE ZONE is supported)
TYPE
AutoIssueFEAT=True
FEAT
CLNT
So taking 7-8 seconds to complete is not unreasonable depending on how many commands it has to send and how much bandwidth you have available. To make Connect() return as soon as possible, turn off all of the auto features, and then call Login() and IssueFEAT() manually when ready.
Firstly I'd verify if the issue is with Indy, have you tried connecting using a different ftp client (ftp at a command prompt, Filezilla, etc.), as it could be the server that is slow. I remember an issue I had where the FTP server was attempting to do a reverse DNS lookup and stalling the connection.
Related
I developped an application that uses indy component to download updates from a remote server.
The problem is that if the FTP server is down or the IP address is not correct, the idFTP.connect() takes too long to give the result (connection failure).
What is the best way to accelerate the connection answer, or may be checking ip address before connection to idFTP.
Thanks in advance.
You should set ReadTimeout property, by default it is set to one minute.
By default, Indy clients wait as long as it takes for the OS to report whether the connection was successful or not. Yes, that can take a long time, if the OS has to look up the hostname with DNS, do network checks, deal with network latency, etc. If you do not want to wait that long, you can use the Timeout parameter of Connect() in Indy 9 and earlier, or the ConnectTimeout property in Indy 10, to reduce the amount of time waited on. HOWEVER, that only applies to the actual socket connect attempt once the server IP has been determined. If you set the Host property to a non-IP hostname, Indy asks the OS to perform a DNS lookup to get the hostname's IP, and there is no logic available in Connect() to control the time it takes to do that lookup. If you need that much control, then use TIdDNSResolver to get the IP manually and then assign it to the Host property before calling Connect().
Well, native connect() API timeouts are notoriously lengthy by design, (to accommodate high latency links like modems). Artificially shortening the timeout may result in premature failure notification, (though as many developers have never seen a modem, it's not that much of a problem today:).
FTP is a reasonably complex transfer requiring two TCP connections and perhaps a DNS lookup - any of these could conceivably generate long connection delays. TidFTP has an inherited 'ReadTimeout' property and a connect() overload with a timeout parameter, but I'm not sure how effective they are.
Historically, I have always timed out such operations myself using a TTimer or similar - if the FTP thread does not respond with a suitable signal, (eg. TThread.Sychronize or user-defined Windows message SendMessage()'d to the GUI), in time, a 'FTP failed' actions are taken and a flag is set in the FTP thread that tells it to ignore any replies and self-terminate. Don't use PostMessage - if you do, there is a small window of time in which a posted response my be queued up while the TTimer is firing - a race.
Oh - and if you are just plonking a TidFTP onto the form, (or creating one in TForm.FormCreate), and trying to run it from the main GUI thread, (with, or without, TidAntiFreeze), stop doing it and thread off the FTP.
I was given 10 new PCs, all (supposedly) with Windows 7 Pro freshly installed and nothing else done to them.
I have a program, coded in Delphi XE2, using Indy 10 components for the networking. I set the "connect timeout" and "read timeout" properties of my TIdTcpCleint to 500ms, set "resuse socket" to 'o/s dependant'" (I also tried a build with it set to No) and leave "use Nagle" (whatever that is set to True (I also tried with false).
Here's the problem: when I run the same .EXE on these PCs and test the case where I pull the network cable, my debug trace shows the connect attempt / connect timeout happening in the same second or the next second (with a granularity of 1 second) - but on others it is 20 or 21 seconds before I see the conenction timeout.
It would seem some of that the PCs are not totally "fresh install" as claimed, although I see no aps installed. Maybe some one installed somethign then removed it, maybe they tried to tweak performance.
Before I reinstall Windows on 10 PCs, can anyone suggest where to look? Does 20 (or 21) seconds ring a bell with regard to TCP Client connect timeout?
[update] I am attempting to connect directly to a specific IP Address, so I am not sure if #Nikolai suggestion to check DNS is relevant. Sorry for not mentioning this originally.
[upperdate] the program does not attempt to keep the socket open. It connects, sends some data & disconnects - repeatedly, for each new piece of data.
Sadly, this is working as intended. The connect did already timeout. Indy made the determination that the connect would fail in the 500 milliseconds that you asked it to. However, that does not guarantee the function will return.
After the connect times out, Indy spins down the connection to release all of its resources. It does this synchronously. This means that you wind up waiting for the underlying TCP operation to fail. This typically takes 20 seconds.
The solution is to call connect in a thread. Believe it or not, this is what Indy already does to implement the timeout. However, when it times out waiting for the thread, it tries to shut down the connection in the main thread. You need to defer that to a worker thread.
As for why it happens immediately on some systems and in 20 seconds on others, it depends on the precise networking configuration. For example, if IPv6 is enabled, the stack may attempt to use an IPv6-to-IPv4 connection, and that may not report down even if the physical interface is down. Immediate detection of connection impossibility is never guaranteed and you shouldn't rely on it.
I've had same problems with INDY in the past (while using D6, year 1998-2000). I changed the component to IP*Works. At that time it was an external component, but as far as I know it is included in XE2. Ip*Works is a bit hard to understand at the beginning but the way they approach to the communication structure is a lot different.
I think that it would be worth to give it a try.
Given a Delphi and Interbase client-server application, I'd like to license the application by the number of users at the database. How can this be done with commercial licensing software? I don't see any of those listing features that look like they would cover this. Every user initially logs on to the database. The database seems so available that it would be open to any user - or at least administrators. Would I have to also write a Delphi exe or dll to run on the server - perhaps as a function in the database - with the licensing connected to that? Not sure how to proceed.
BTW, Interbase licenses simultaneous users, but I think they wrote that right into the server, but I want something similar.
To control simultaneous client connections you definitively need a server side application.
It can be a simple tcp/ip socket server as a service (daemon on linux) or another (midas?) server layer.
When your client app starts it call a server method for example Session.Connect, here you count active connections and return false (no code) in case of maximum limit reaches.
When application closes you notify server with Session.Disconnect. to decrease connection count.
Also is a good idea to keep a live (permanent) connection between client app and server service (as I sad sockets) to handle application hangups, uncontrolled restarts and process this event for example OnSockedDisconnect on server side, to decrease connection count and handle for disconnect propery, for example write in logs etc...
Of course communication should be crypted (handshaked), to avoid unwanted guests.
You can play also with sim cardreaders etc..
This method will not provide a industrial (nuclear) level of security, but if coded corectly it can take some time even for an expert hacker to broke it.
OR, you may take a look at some ready protection tools like SafeNet (HASP protection).
Also, Firebird (and maybe Interbase) have on DB Connect / Disconnect triggers, where if user have privileges it can read connection count. But these can be easily changed if DB are stored on customer server.
I need to connect to a device on the local network, using a TCP/IP connection.
When I use a TIdTCPClient, all works well, except one thing:
If the connection is not available, it takes about 18-20 seconds before I get a timeout.
The property ConnectTimeout has no effect, no matter what values I set. It always takes the same amount of time before a timeout.
This answer mentions long delay times for a timeout, and I am wondering if that's related to the Indy components?
I have to find out if the connection cannot be established very quickly, let's say within 1 second at most.
Is there a way to do this using Indy, or do I need to use different components / a different approach?
(I'm using the Indy 10 version that shipped with Delphi 2009)
EDIT:
I followed the instructions to upgrade Indy to the latest version in this post.
Still the same, it now consistently takes 22 seconds until TCPClient.Connect returns when there is no connection. ConnectTimeout and/or ReadTimeout seem to have no influence on this.
ConnectTimeout works correctly for 2010 and XE. Perhaps you can update your Indy version to the latest (its free). I have a function that does 'quick check' connects, just to check availability of the device and those timeouts are 5 seconds without problems (in both 2010 and XE).
With a default TCP client connect timeout (not specifically set) and a read timeout of 1 second, using Delphi 2010 and the latest Indy version, a local connection (using localhost as the host name) times out in 1 second. So this is definitely not a Delphi/Indy issue. BTW, this gives me an EIdSocketError ("socket error # 10061 connection refused").
I had the same exact problem. Check out this StackOverflow post.
In short, because Indy threads are blocking, you will need to make a threaded process. Then in the primary application create a timer which will terminate the thread if it has not done what it is supposed to do in the time given.
After I implemented this is worked great.
We have a C/S application all written in Delphi (Client and Server-or middleware if you want)
For the client part we use Indy.
For the server we use DXSock.
Since DXSock is dead for a while we are investigating alternatives for the sever part.
I want to hear some comments about the best Server Socket alternative component for Delphi.
The current system usually have tens of permanent connections working each one on its own thread but could be hundreads in the future (this should be improved to a thread pool if possible)
If you want to have the best possible performance, you'd have to use sockets in non blocking mode, or using completion ports. IPWorks is implemented like that, as well as iocp. As far as I can tell, Indy or Synapse don't implement them (at least officially).
We used completion ports and a thread pool in our open source SynCrtSock unit, used in our Synopse SQLite3 framework.
Here are some benchmarks of this solution, working from Delphi 6 up to Delphi XE. I don't tell this is the "best component", but it's a working and speedy one (every request is about 4 KB of JSON data):
Http client keep alive (i.e. one HTTP/1.1 client connection kept alive during requests):
first in 7.87ms, done in 153.37ms i.e. 6520/s, average 153us
Http client multi connect (i.e. one new HTTP/1.0 client connection created for each request - this one uses completion ports and a thread pool):
first in 151us, done in 305.98ms i.e. 3268/s, average 305us
For speed comparison, here are other communication protocols available in our framework:
Named pipe access:
first in 78.67ms, done in 187.15ms i.e. 5343/s, average 187us
Local window messages:
first in 148us, done in 112.90ms i.e. 8857/s, average 112us
Direct in process access:
first in 44us, done in 41.69ms i.e. 23981/s, average 41us
We use HTTP/1.1 protocol over TCP/IP, because there is very little overhead over plain TCP/IP, and this is a well handled protocol for firewalls and such, and allows our framework to be used by an AJAX application, whereas its main purpose is to serve Delphi clients.
IMHO there is no "best Server Socket alternative component for Delphi", it depends what is the purpose of your server application. The main bottleneck will be in the Windows kernel itself. Perhaps direct access to the HTTP Kernel-Mode Driver (Http.sys) of Windows could help.
Consider using a dedicated optimized Server instead of a Delphi server, like lighttpd or Cherokee using FastCGI to handle the requests via a Free Pascal (or CrossKylix) application, under Linux. I guess this will be the best performance possible.
I use Indy components for commercial server-side work and the component set is pretty solid (9 or 10). My servers have millions of connections per day with no issues.
I used DXSock many moons ago. He was always optimizing, but never seemed to finish it. He does seem to have another version out.
If you want commercial support, then I'd recommend IPWorks from nSoftware.
Actually DXSock is not dead, v6.1 was just released. The web hosting company we used to use in Tennessee lost the domain - so only customers who have kept their subscription renewed annually have received DXSock 5.0, 6.0 and 6.1.
Indy CANNOT support more than 2,000 concurrent connections on 32bit Windows - as Chad and crew use TThread, which implements the defacto 1MB per thread/socket connection - 2000x1MB = >2.5GB of RAM which 32bit OSes do not support. DXSock implements a 0b per connection model (unless you define otherwise) and can handle over 50,000 concurrent on Windows, Linux, Mac, Pi, etc.
Ozz Nixon - ozznixon#bpdx.com if you want more details on 6.1
Author of DXSock
Co-Author of Winshoes which became INDY.