I'm getting a problem using the TidFTP component.
I'm able to connect with the server using a code like this
vFileList := TStringList.Create;
oClientFTP := TidFTP.Create(nil);
oClientFTP.Port := PortFTP;
oClientFTP.Host := IPHost;
oClientFTP.UserName := UserFTP;
oClientFTP.Password := PasswordFTP;
After getting several files from the StringList (this one has exactly 778 elements) when the element no. 137 is retrieved the exception EIdAcceptTimeout is raised with "Accept timed out." message.
The code that I run is like this (runs in a Thread by the way)
procedure TDownloadFTP.Get;
begin
try
for I := 0 to vFileList .Count - 1 do
begin
sFileName:= vFileList [I];
posPoint := LastDelimiter('.', sFileName);
if posPoint = 0 then
ForceDirectories(ExtractFilePath(Application.ExeName) + '/BackUp/' + sFileName)
else
try
oClienteFTP.Get(sFileName,IncludeTrailingPathDelimiter(ExtractFilePath(Application.ExeName) + '/BackUp/') + sFileName, True);
except
on E: EIdReplyRFCError do
begin
end;
on E: Exception do
exceptionList.Add(sFileName);
end;
end;
After the exception, the file is downloaded correctly but the process needs like 25 seconds per file (I'm downloading 2KB png images).
Any idea of the meaning of this Exception?
Thanks
Googling for EIdAcceptTimeout leads to this discussion in the Indy forum:
UseHOST in TIdFTP (client) => EIdAcceptTimeout
Where Remy Lebeau states:
The only time that exception can occur during a data transfer is if
you have the TIdFTP.Passive property set to False, which tells the FTP
server to make an inbound connection to TIdFTP. Those connections are
usually blocked by firewalls/routers that are not FTP-aware. You
usually have to set TIdFTP.Passive=True when you are behind a
firewall/router.
So, the solution could be for you to add a line
oClientFTP.Passive := True;
Btw. In your code snippets you have both oClientFTP and oClienteFTP. Adjust my suggestion if needed.
I would have written this as comments, rather than an answer, but comments are too limited. Please let me know and excuse me if I misbehave.
Looking at your code a second time raises a few questions. I see that the StringList can have both files (posPoint <> 0) and presumably directories (posPoint = 0). Is element 137 a file or directory and if a file, is it the first file after a new directory?
Do the entries in the StringList include the path they ought to have after '\backup\?
Assuming your application is a Windows application (since you don't say otherwise), When you create new paths, why do you use forward slashes (/) instead of backslashes () which is the path delimiter on Windows? Does your code even create subdirectories on Windows? Well, maybe crossplatform Delphi adjusts according to OS.
In the oClienteFTP.Get statement you say IncludeTrailingPathDelimiter even if you already have a slash as the trailing delimiter in '/backup/'.
You should never anymore use 'ExtractFilePath(Application.ExeName)' and subdirectories, as storage for data files.
Related
I'm having a problem downloading a file using the TidFTP component in Delphi XE2. I am able to get a connection to the FTP site, get a list of files and perform the get command. However, when I try to download a file using the get command the file always downloads larger than the source file. Then the subsequent file is corrupt.
Additionally, if I try to download multiple files, the first file downloads (larger than the source) and the remaining files are skipped. No error is thrown from the get command, it just quits. I tried to hook into some of the events on the TidFTP control like AfterGet and OnStatus but everything appears normal.
I tried using a 3rd party FTP client to access the file and download it just to make sure it was not a problem with our FTP server and that download worked as expected. So I'm thinking it might be related to the TidFTP control or perhaps I'm doing something incorrectly.
Here is the routine I'm using to download the file:
var
ftp: TIdFTP;
strDirectory: string;
begin
ftp := TIdFTP.Create(nil);
try
ftp.Host := 'ftp.myftpserver.com'
ftp.Passive := false;
ftp.Username := 'TestUser';
ftp.Password := 'TestPassword';
ftp.ConnectTimeout := 1000;
ftp.Connect();
ftp.BeginWork(wmRead);
ftp.ChangeDir('/TestArea/');
strDirectory := 'c:\test\';
if not DirectoryExists(strDirectory) then
CreateDir(strDirectory);
ftp.Get('Test.zip', strDirectory + '\' + 'Test.zip', true, false);
ftp.Disconnect();
except
on e: exception do
showMessage(e.message);
end;
end;
You need to set the TIdFTP.TransferType. Its default value is Id_TIdFTP_TransferType, which is ftASCII. You need to use ftBinary instead, and set it before executing the Get:
ftp.Connect();
...
ftp.TransferType := ftBinary;
ftp.Get('Test.zip', strDirectory + '\' + 'Test.zip', true, false);
ftp.Disconnect();
The documentation for TIdFTP.TransferType says in one place that it's automatically set to ftBinary when Login is executed or when Connect is called when AutoLogin is set to true, but you're not executing Login in your code and haven't set AutoLogin. The paragraph immediately following the above statement says:
According to #RemyLebeau in the comments below, the documentation quoted is in error, and TransferType was never set to ftBinary in Login. Leaving the stricken content for future reference.
According to the documentation:
The default value for TransferType is Id_TIdFTP_TransferType, as assigned during initialization of the component.
I tried to employ Indy 10.5.5 (shipped with Delphi 2010) for:
connecting to telnet server
performing username/password authentication (gaining access to the command shell)
executing a command with returning resulting data back to application
and had no success, additionally i'm completely lost in spaghetti logic of Indy's internals and now have no idea why it didnt work or how i supposed to send strings to the server and grab the results. Need some sample code to study.
Formal form of the question: Where can i get 3-rd party contributed demo covering TIdTelnet component? (indyproject.org demos webpage do not have one)
The main problem with Telnet is that it DOES NOT utilize a command/response model like most other Internet protocols do. Either party can send data at any time, and each direction of data is independant from the other direction. This is reflected in TIdTelnet by the fact that it runs an internal reading thread to receive data. Because of this, you cannot simply connect, send a command, and wait for a response in a single block of code like you can with other Indy components. You have to write the command, then wait for the OnDataAvailable event to fire, and then parse the data to determine what it actually is (and be prepared to handle situations where partial data may be received, since that is just how TCP/IP works).
If you are connecting to a server that actually implements a command/response model, then you are better off using TIdTCPClient directly instead of TIdTelnet (and then implement any Telnet sequence decoding manually if the server really is using Telnet, which is rare nowadays but not impossible). For Indy 11, we might refactor TIdTelnet's logic to support a non-threaded version, but that is undecided yet.
done with indy.
no comments.. just som old code :-)
telnet don't like the send string kommand.. use sendch.
telnetdude.Host := 1.1.1.1;
try
telnetdude.connect;
except
on E: Exception do begin
E.CleanupInstance;
end; {except}
if telnetdude.Connected then begin
for i := 1 to length(StringToSend) do telnetdude.sendch(StringToSend[i]);
telnetdude.sendch(#13);
end;
end; {while}
end; {if}
if telnetdude.Connected then telnetdude.Disconnect;
end;
I hope this helps anyone looking for answers to a similar question.
Firstly, It would seem the typical command/response model (as mentioned above, does indeed NOT apply).
So I just got it working for some very simple application (rebooting my router).
Specific additions to above code from Johnny Lanewood (and perhaps some clarification)
a) You have to send #13 to confirm the command
b) I got "hangs" on every command I sent / response I requested UNTIL I enabled ThreadedEvent. (this was my big issue)
c) the OnDataAvailable event tells you when new data is available from the Telnet Server - however there are no guarantees as to what this data is - i.e. it's pretty what you get in the command line / what ever is appended to the previous responses. But is is NOT a specific response line to your command - it's whatever the telnet server returns (could be welcome info, ASCII drawings etc etc.)
Given (c) above, one would rather check the OnDataAvailable event and parse the data (knowing what you'd expect). When the output stops (i.e. you need build a mechanism for this), you can parse the data and determine whether the server is ready for something new from the client. For the purpose of my code below, I set a read timemout and I just used Sleep(2000) - ignorantly expecting no errors and that the server would be ready after the sleep for the next command.
My biggest stumbling block was ThreadedEvent := True (see above in b)
Thus, my working solution (for specific application, and possibly horrible to some).
lIDTelnet := TIdTelnet.Create(nil);
try
lIdTelnet.ReadTimeout := 30000;
lIDTelnet.OnDataAvailable := TDummy.Response;
lIDTelnet.OnStatus := TDummy.Status;
lIdTelnet.ThreadedEvent := True;
try
lIDTelnet.Connect('192.168.0.1', 23);
if not lIDTelnet.Connected then
Raise Exception.Create('192.168.0.1 TELNET Connection Failed');
Sleep(2000);
lIdtelnet.SendString(cst_user + #13);
Sleep(2000);
lIdtelnet.SendString(cst_pass + #13);
Sleep(2000);
lIdtelnet.SendString(cst_reboot + #13);
Sleep(2000);
if lIDTelnet.Connected then
lIDTelnet.Disconnect;
except
//Do some handling
end;
finally
FreeAndNil(lIdTelnet);
end;
and then
class procedure TDummy.Response(Sender: TIdTelnet; const Buffer: TIdBytes);
begin
Write(TDummy.ByteToString(Buffer));
end;
class function TDummy.ByteToString(
const aBytes: TIdBytes): String;
var
i : integer;
begin
result := '';
for i := 0 to Length(aBytes) -1 do
begin
result := result + Char(aBytes[i]);
end;
end;
Actually i'm using the TIdHTTP component for download a file from internet. i'm wondering if is possible pause and resume the download using this component o maybe another indy component.
this is my current code, this works ok for download a file (without resume), but . now i want pause the download close my app ,and when my app restart then resume the download from the last position saved.
var
Http: TIdHTTP;
MS : TMemoryStream;
begin
Result:= True;
Http := TIdHTTP.Create(nil);
MS := TMemoryStream.Create;
try
try
Http.OnWork:= HttpWork;//this event give me the actual progress of the download process
Http.Head(Url);
FSize := Http.Response.ContentLength;
AddLog('Downloading File '+GetURLFilename(Url)+' - '+FormatFloat('#,',FSize)+' Bytes');
Http.Get(Url, MS);
MS.SaveToFile(LocalFile);
except
on E : Exception do
Begin
Result:=False;
AddLog(E.Message);
end;
end;
finally
Http.Free;
MS.Free;
end;
end;
the following code worked to me. It downloads the file by chunks:
procedure Download(Url,LocalFile:String;
WorkBegin:TWorkBeginEvent;Work:TWorkEvent;WorkEnd:TWorkEndEvent);
var
Http: TIdHTTP;
quit:Boolean;
FLength,aRangeEnd:Integer;
begin
Http := TIdHTTP.Create(nil);
fFileStream:=nil;
try
try
Http.OnWork:= Work;
Http.OnWorkEnd := WorkEnd;
Http.Head(Url);
FLength := Http.Response.ContentLength;
quit:=false;
repeat
if not FileExists(LocalFile) then begin
fFileStream := TFileStream.Create(LocalFile, fmCreate);
end
else begin
fFileStream := TFileStream.Create(LocalFile, fmOpenReadWrite);
quit:= fFileStream.Size >= FLength;
if not quit then
fFileStream.Seek(Max(0, fFileStream.Size-4096), soFromBeginning);
end;
try
aRangeEnd:=fFileStream.Size + 50000;
if aRangeEnd < fLength then begin
Http.Request.Range := IntToStr(fFileStream.Position) + '-'+ IntToStr(aRangeEnd);
end
else begin
Http.Request.Range := IntToStr(fFileStream.Position) + '-';
quit:=true;
end;
Http.Get(Url, fFileStream);
finally
fFileStream.Free;
end;
until quit;
Http.Disconnect;
except
on E : Exception do
Begin
//Result:=False;
//AddLog(E.Message);
end;
end;
finally
Http.Free;
end;
end;
Maybe the HTTP RANGE header can help you here. Have a look at archive.org's copy of http://www.west-wind.com/Weblog/posts/244.aspx for more info on resuming HTTP downloads:
(2004-02-07) A couple of days ago somebody on the Message Board asked an interesting question about how to provide resumable HTTP downloads. My first response to this question was that this isn't possible since HTTP is a stateless protocol that has no concept of file pointers and thus can't resume an HTTP download.
However it turns out HTTP 1.1 does have the ability to specify ranges in downloads by using the Range: header in the Http header sent form the client. You can do things like:
Range: 0-10000
Range: 100000-
Range: -100000
which download the first 100000 bytes, everything over 100000 bytes or the last 100000 bytes. There are more combinations but the first two are the ones that are of interest for a resumable download.
To demonstrate this feature I used wwHTTP (in Web Connection/VFP) to download a first 400k chunk of a file into a file with HTTPGetEx which is meant to simulate an aborted download. Next I do a second request to pick up the existing file and download the remainder:
#INCLUDE wconnect.h
CLEAR
CLOSE DATA
DO WCONNECT
LOCAL o as wwHTTP
lcDownloadedFile = "d:\temp\wwipstuff.zip"
*** Simulate partial output
lcOutput = ""
Text=""
tnSize = 0
o = CREATEOBJECT("wwHTTP")
o.HttpConnect("www.west-wind.com")
? o.httpgetex("/files/wwipstuff.zip",#Text,#tnSize,"Range: bytes=0-400000"+CRLF,lcDownloadedFile)
o.Httpclose()
lcOutput = Text
? LEN(lcOutput)
*** Figure out how much we downloaded
lnOpenAt = FILESIZE(lcDownloadedFile)
*** Do a partial download starting at this byte count
Text=""
tnSize =0
o = CREATEOBJECT("wwHTTP")
o.HttpConnect("www.west-wind.com")
? o.httpgetex("/files/wwipstuff.zip",#Text,#tnSize,"Range: bytes=" + TRANSFORM(lnOpenAt) + "-" + CRLF)
o.Httpclose()
? LEN(Text)
*** Read the existing partial download and append current download
lcOutput = FILETOSTR(lcDownloadedFile) + TEXT
? LEN(lcOutput)
STRTOFILE(lcOutput,lcDownloadedFile)
RETURN
Note that this approach uses a file on disk, so you have to use HTTPGetEx (with Web Connection). The second download can also be done to disk if you choose, but things will get tricky if you have multiple aborts and you need to piece them together. In that case you might want to try to keep track of each file and add a number to it, then combine the result at the very end.
If you download to memory using WinInet (which is what wwHTTP uses behind the scenes) you can also try to peel out the file from the Temporary Internet Files cache. Although this works I suspect this process will become very convoluted quickly so if you plan on providing the ability to resume I would highly recommend that you write your output to file yourself using the approach above.
Some additional information on WinInet and some of the requirements for this approach to work with it are described here: http://www.clevercomponents.com/articles/article015/resuming.asp.
The same can be done with wwHTTP for .Net by adding the Range header to the wwHTTP:WebRequest.Headers object.
(Randy Pearson) Say you don't know what the file size is at the server. Is there a way to find this out, so you can know how many chunks to request, for example? Would you send a HEAD request first, or does the header of the GET response tell you the total size also?
(Rick Strahl) You have to read the Content-Length: header to get the size of the file downloaded. If you're resuming this shouldn't matter - you just use Range: (existingsize)- to get the rest. For chunky downloads you can read the content length and only download the first x bytes. This gets tricky with wwHTTP - you have to make individual calls with HTTPGetEx and set the tnBufferSize parameter to the chunk size to retrieve to have it stop after the size is reached.
(Randy Pearson) Follow-up: It looks like a compliant server would send you enough to know the size. If it provides chunks it should reply with something like:
Content-Range: 0-10000/85432
so you could (if desired) extract that and use it in a loop to continue with intelligent chunk requests.
Also look here https://forums.embarcadero.com/message.jspa?messageID=219481 for TIdHTTP related discussion on the same topic:
(at least partly as per tfilestream.seek and offset confusion)
if FileExists(dstFile) then
begin
Fs := TFileStream.Create(dstFile, fmOpenReadWrite);
try
Fs.Seek(Max(0, Fs.Size-1024), soFromBeginning);
// alternatively:
// Fs.Seek(-1024, soFromEnd);
Http.Request.Range := IntToStr(Fs.Position) + '-';
Http.Get(Url, Fs);
finally
Fs.Free;
end;
end;
I have a problem adding strings to a TStringList. I've searched other posts but couldn't find an answer to this.
What I'm trying to do is to add a big amount of strings to a TStringList (more than 14000) but somewhere in the process I get an EAccessViolation. Here's the code I'm using:
procedure TForm1.FormCreate(Sender: TObject);
begin
List := TStringList.Create;
List.Duplicates := dupAccept;
end;
procedure TForm1.ButtonStartClick(Sender: TObject);
begin
List.Clear;
List.Add('125-AMPLE');
List.Add('TCUMSON');
List.Add('ATLV 4300');
List.Add('150T-15');
List.Add('TDL-08ZE');
List.Add('RT20L');
List.Add('SIN LINEA');
List.Add('TIARA');
List.Add('FL200ZK1');
List.Add('FL250ZK1');
List.Add('SIN LINEA');
List.Add('CENTAURO-70 S.P.');
List.Add('CORSADO');
{ This list continues to about 14000 strings...}
List.Add('VOSJOD 2');
List.Add('Z 125');
List.Add('ZUMY');
List.Add('NEW AGE 125');
List.Add('SIN LINEA');
end;
procedure TForm1.FormClose(Sender: TObject; var Action: TCloseAction);
begin
FreeAndNil(List);
end;
¿What's wrong with this code? The list contains duplicate strings so I set the Duplicates property to dupAccept. I was able to load the list using LoadFromFile, but I don't want to have a text file outside my application.
I hope you can help me!!! Please tell me if you need any further information.
Thank you very much. I really appreciate your help.
The suggestions for using an external file are on the mark here. However, your post indicates your desire for not having an external file. I would then suggest you link the file to the executable as a resource. You can easily do this following these steps:
Place all the strings into a text file called stringdata.txt (or whatever name you choose). Then create a .rc file of whatever name you choose and put the following into it (STRING_DATA can be any identifier you choose):
STRING_DATA RCDATA "stringdata.txt"
Create a .res file from the .rc:
BRCC32 <name of rc>.rc
Now reference this file from the source code. Place the following someplace in the unit:
{$R <name of res>.res}
Instead of loading from a file stream, load from a resource stream:
StringData := TResourceStream.Create(HInstance, 'STRING_DATA', RT_RCDATA);
try
List.LoadFromStream(StringData);
finally
StringData.Free;
end;
If you do command-line automated builds, I would suggest you keep the .rc file under source control and build the .res during the build process. This way you can also keep the stringdata.txt file under source control and any edits are automatically caught on the next build without having to explicitly build the .res file each time the .txt file changes.
What Delphi version are you using? Some older versions had a bug in the memory manager that can cause an access violation when trying to reallocate an array to a size that's too large.
Try adding FastMM4 to your project to replace the old memory manager and see if that helps.
Also, you're probably better off keeping the list in an external file. Yes, it's another file, but it also means that you can change the list without having to recompile the entire program. This also makes creating (and distributing!) updates easier.
Mason is probably right for the cause of the AV; this is quite a large array to grow.
On a side note, when doing such a long processing on a StringList, it's recommended to surround it by BeginUpdate/EndUpdate to avoid firing any update event.
Even if you don't have any now, they might be added later and you'll get problems.
Set list.capacity to the number of items you plan to add, immediately after you create the list. Alternatively, place the list in an RC file (named other than with the name of your project) and add it to your project. This gets compiled into your application, but does not involve executable code to create the list.
I would also worry about compiler integrity with a 14,000 line procedure. People have found other cases where going beyond anything reasonable breaks the compiler in various ways.
You may also want to try THashedStringList, could see a speed boost (although not in this function), although I'm not sure if the add method is a whole lot different.
try using the following instead of your code to add the strings to the StringList
var
Str: string;
begin
Str := '125-AMPLE' + #13#10;
Str := Str + 'TCUMSON' + #13#10;
Str := Str + 'ATLV 4300' + #13#10;
Str := Str + '150T-15' + #13#10;
................
List.Text := Str;
end;
This question already has answers here:
Delphi fast file copy
(6 answers)
Closed 2 years ago.
I have an application which accesses files on the intranet. The users can copy the files from the server located on the network to their local PCs. I have encoutered a problem with initial connection. i.e. when the application is started. The user searches for the file in the database. When he finds a file he wants to download he clicks on the name and the application downloads it.
The first time download operation takes about 8-12 seconds just to initial the download and see the progress bar. The next downloads are almost instant. The file size varies between
1 MB to 15 MB.
Here is my code:
const
projectFilesURL = '\\IntranetServer\Directory\filesLocation\';
procedure form1.GetSelectedFile(const fileName: string);
var
sourceFile: string;
begin
{ \\IntranetServer\Directory\filesLocation\userSelectedFile.zip}
sourceFile := projectFilesURL + fileName;
if FileExists(sourceFile) then
begin
fileCopy(fileName);
lblSearching.Hide;
AnimSearching.Hide;
end
else
MessageDlg(
'The file was not found on the server'
, mtInformation, [mbCancel], 0);
end;
end;
procedure form1.fileCopy(const sourceFile: string);
var
SourceF, DestF: file;
Buf: array [0 .. 1023] of byte;
NumRead, FSize, BytesCopied: Integer;
destinationPathandFile: string;
begin
destinationPathandFile := ExtractFilePath(ParamStr(0))
+ exportPath + sourceFile;
try
AssignFile(SourceF, projectFilesURL + sourceFile);
AssignFile(DestF, destinationPathandFile);
FileMode := 0;
Reset(SourceF, 1);
Rewrite(DestF, 1);
FSize := FileSize(SourceF);
BytesCopied := 0;
fileCopyProgress.Percent := 0;
while not Eof(SourceF) do
begin
BlockRead(SourceF, Buf, SIZEOF(Buf), NumRead);
BlockWrite(DestF, Buf, NumRead);
Inc(BytesCopied, NumRead);
fileCopyProgress.Percent := (BytesCopied * 100) div FSize;
Application.ProcessMessages;
end;
CloseFile(SourceF);
CloseFile(DestF);
except
on E: Exception do
begin
raise Exception.Create('Error occured while copying a file');
Exit;
end;
end;
I am not sure where could be a choking point...maybe fileExists is not necessary.
Since a file is on the server or not...maybe to throw exception is better...
I appreciate any suggestions.
Thanks,
You say that a second copy is almost instant, so a fundamental problem with your code is almost certainly excluded (and I don't see any fundamental problems in the copy code itself anyways).
Have you tried stepping through the program in the debugger to see on what line it "hangs" or spends abnormal amounts of time?
A problem with comparable symptoms that I've see before is when a Windows client copies off a Linux Samba server, with somewhat older Samba versions : it takes a lot of time to "get connected", after that everything works at normal speeds.
As a more general remark, I'd work towards a better separation of concerns, having a filecopy method in a form object is not really state of the art...
Since this is a network resource, the most likely cause of the problem is the fact that the system must cache the source directory before it can start transferring files. Once the directory is cached your performance improves.
One way you can "reduce" this amount would be to request a file via a thread at program startup to "prime the pump". Just throw away the result. My suggestion would be to create a small "dummy" file that is only a few bytes in size.
Another option would be to write a simple server that you would run on the server to serve files to your program. The advantage of this approach is that you can easily add on the fly compression that might speed up your file transfers (depends on the type of data being transfered).
in GetSelected:
if FileExists(sourceFile) then
What is in sourceFile?
Certainly not (projectFilesURL + fileName) which is what you might want
I'm not sure if this is your problem, but browsing,working with and copying files from mapped (where you have assigned a drive letter to a network drive) is much quicker than using the UNC (\Intranet) notation. If you can map a drive for all you clients this may improve you performance.