I have serious problem and i'm stuck on this :(
I need to transfer from my application big files (sometimes more than 1GB) with process it to another application i.e. TotalCommander or Explorer.
To do it i need to read part of file from source file to stream, process it and save to output stream.
I can't load entire file i need to read about 1-30MB, process it and save.
I'm using Drag Drop component suite for Delphi 7
Sample code (in desperate i try many variations):
Var
F13Stream : TmemoryStream;
Procedure Process_Data;
Var
FileIn : TFileStream;
begin
FileIn := TFileStream.Create('c:\SampleData.dat', fmOpenRead);
Repeat
..read from FileIn, process data
..save to F13Stream
..write F13Stream to output? and free memory to next part data//Sorry i have no idea what to doit :(
Until eof(source_file);
FileIn.Free;
end;
procedure TForm13.OnGetStream(Sender: TFileContentsStreamOnDemandClipboardFormat;
Index: integer; out AStream: IStream);
begin
Form13.F13Stream := TMemorystream.Create;
Process_Data;
try
AStream := nil;
AStream := TFixedStreamAdapter.Create(F13Stream, soOwned);
except
F13Stream.Free;
raise;
end;
end;
procedure TForm13.LMDShellList1MouseDown(Sender: TObject;
Button: TMouseButton; Shift: TShiftState; X, Y: Integer);
begin
TVirtualFileStreamDataFormat(DataFormatAdapterSource.DataFormat).FileNames.Clear;
TVirtualFileStreamDataFormat(DataFormatAdapterSource.DataFormat).FileNames.Add('TestFile');
DropEmptySource1.Execute;
end;
This is example what i try to do. Maybe another component/code will do this, without problems.
I can copy files without size limitations, but i need partially process big files and partially send to another applications as stream/array only by memory not to using temporary disk drive memory.
Thanks for any help.
UPDATE:
procedure TForm13.OnGetStream(Sender: TFileContentsStreamOnDemandClipboardFormat;
Index: integer; out AStream: IStream);
var
plk1 : TFileStream;
buff : array[0..65535] of byte;
read : Longint;
begin
AStream := nil;
Form13.F13Stream := TMemoryStream.Create;
AStream := TFixedStreamAdapter.Create(Form13.F13Stream, soOwned);
plk1 := TFileStream.Create('h:\Sample2GB_file.dat', fmOpenRead);
repeat
read := plk1.Read(buff, 65536);
AStream.Write(#buff, read, PLongint(65536));
AStream.SetSize(0);
until read <> 65536;
plk1.Free;
end;
It also not work. As result i have 0 bytes file output.
UPDATE2:
If i remove line: "AStream.SetSize(0);" then i have "Out of memory" error.
You cannot actually write to AStream in OnGetStream. It is a source IStream that you need to set and the receiver (where the drop occurs) will read your data from it. If you need to process your source file, do it before the drag start and save the result in another file. Then just configure your new file as drag source. You can simply do it in your Mouse Down event, or if you prefer to use the Virtual Method, do it in OnGetStream event:
procedure TForm13.OnGetStream(Sender: TFileContentsStreamOnDemandClipboardFormat;
Index: integer; out AStream: IStream);
var
plk1 : TFileStream;
begin
AStream := nil;
// TODO: Create the file to stream (read your source and save it to new_source.dat after your processing)
plk1 := TFileStream.Create('h:\new_source.dat', fmOpenRead);
AStream := TFixedStreamAdapter.Create(plk1, soOwned);
end;
You can also create your own TStream class to read, process and return your data on the fly in Read method.
Related
Delphi Tokyo - I have a parameter file that I am needing to save (and later load) from disk. The parameters are a series of record objects. There is one HEADER record and then multiple COMMAND records. These are true records (i.e type = records). The HEADER record has String, Boolean, Integer, and TStringList types within it. I save, which appears to work fine, but when I load, whatever is AFTER a TStringList causes a Stream read error. For example...
type tEDP_PROJ = record
Version : Integer;
Name: String;
...
ColList1: TStringList;
ColList2: TStringList;
ReadyToRun : Boolean;
...
end;
When I read ReadyToRun I get a Stream read error. If I move it BEFORE TStringList (on both SAVE and LOAD routines) then ReadyToRun will load properly, but whatever is after the TStringList will cause an error. It is interesting to note that ColList2 loads fine (even though it is NOT the first TStringList).
I am specifying the Encoding method when I save the TStringList.
...
ColList1.SaveToStream(SavingStream, TEncoding.Unicode);
ColList2.SaveToStream(SavingStream, TEncoding.Unicode);
I am using the same encoding when I load from the (file) Stream.
...
ColList1.LoadFromStream(SavingStream, TEncoding.Unicode);
ColList2.LoadFromStream(SavingStream, TEncoding.Unicode);
Note that when I create the StringList, I am just doing the standard create...
ColList1 := TStringList.Create;
When I save and load, I am following the examples Remy gave here...
The TStringList appears to be changing the way that the stream reads non-TStringList types... What do I need to do to fix this?
Why are you using TEncoding.Unicode? TEncoding.UTF8 would have made more sense.
In any case, this is not an encoding issue. What you are attempting to do will simply not work the way you are trying to do it, because TStrings data is variable-length and needs to be handled accordingly. However, TStrings does not save any kind of terminating delimiter or size information to an output stream. When loading in a stream, TStrings.LoadFromStream() simply reads the ENTIRE stream (well, everything between the current Position and the End-Of-Stream, anyway). That is why you are getting streaming errors when trying to read/write any non-TStrings data after any TStrings data.
Just like the earlier code needed to serialize String data and other variable-length data into a flat format to know where one field ends and the next begins, you need to serialize TStrings data as well.
One option is to save a TStrings object to an intermediate TMemoryStream first, then write that stream's Size to your output stream followed by the TMemoryStream's data. When loading back later, first read the Size, then read the specified number of bytes into an intermediate TMemoryStream, and then load that stream into your receiving TStrings object:
procedure WriteInt64ToStream(Stream: TStream; Value: Int64);
begin
Stream.WriteBuffer(Value, Sizeof(Value));
end;
function ReadInt64FromStream(Stream: TStream): Int64;
begin
Stream.ReadBuffer(Result, Sizeof(Result));
end;
procedure WriteStringsToStream(Stream: TStream; Values: TStrings);
var
MS: TMemoryStream;
Size: Int64;
begin
MS := TMemoryStream.Create;
try
Values.SaveToStream(MS, TEncoding.UTF8);
Size := MS.Size;
WriteInt64ToStream(Stream, Size);
if Size > 0 then
begin
MS.Position := 0;
Stream.CopyFrom(MS, Size);
end;
finally
MS.Free;
end;
end;
procedure ReadStringsFromStream(Stream: TStream; Values: TStrings);
var
MS: TMemoryStream;
Size: Int64;
begin
Size := ReadInt64FromStream(Stream);
MS := TMemoryStream.Create;
try
if Size > 0 then
begin
MS.CopyFrom(Stream, Size);
MS.Position := 0;
end;
Values.LoadFromStream(MS, TEncoding.UTF8);
finally
MS.Free;
end;
end;
Another option is to write the number of string elements in the TStrings object to your output stream, and then write the individual strings:
procedure WriteStringsToStream(Stream: TStream; Values: TStrings);
var
Count, I: Integer;
begin
Count := Values.Count;
WriteIntegerToStream(Stream, Count);
for I := 0 to Count-1 do
WriteStringToStream(Stream, Values[I]);
end;
procedure ReadStringsFromStream(Stream: TStream; Values: TStrings);
var
Count, I: Integer;
begin
Count := ReadIntegerFromStream(Stream);
if Count > 0 then
begin
Values.BeginUpdate;
try
for I := 0 to Count-1 do
Values.Add(ReadStringFromStream(Stream));
finally
Values.EndUpdate;
end;
end;
end;
Either way, you can then do this when streaming your individual records:
WriteIntegerToStream(SavingStream, Version);
WriteStringToStream(SavingStream, Name);
...
WriteStringsToStream(SavingStream, ColList1);
WriteStringsToStream(SavingStream, ColList2);
WriteBooleanToStream(SavingStream, ReadyToRun);
Version := ReadIntegerFromStream(SavingStream);
Name := ReadStringFromStream(SavingStream);
...
ReadStringsFromStream(SavingStream, ColList1);
ReadStringsFromStream(SavingStream, ColList2);
ReadyToRun := ReadBooleanFromStream(SavingStream);
I have this code, where I am reading buffers to memory stream and I want to save it to file stream. The problem is that memoryStream.GetBuffer() does not work for my Delphi 7 as it is undeclared identifier.
procedure Dictionary.WriteHeaderObj(filename: String);
var MemStream: TMemoryStream;
i: Integer;
begin
self.fileName := filename;
try
MemStream := TMemoryStream.Create;
try
fsOutput := TFileStream.Create(fileName, fmCreate);
try
MemStream.Write(VAR_META, lengths.VR_META);
for i:=0 to length(buffers)-1 do
MemStream.Write(self.buffers[i].b^,self.buffers[i].l^);
fsOutput.Write(MemStream.GetBuffer(), 0, memoryStream.Position);
finally
MemStream.Free;
end;
finally
fsOutput.Free;
end;
except
on E:Exception do
Writeln(E.Classname, ': ', E.Message);
end;
end;
Your code showed several mistakes common to beginners.
There really is no need for the TMemoryStream. As David said, it hurts performance.
It's also not a good idea to write everything to a single TMemoryStream, because you may ran into "Out of memory" problem.
If the buffer size is really small and using a memory buffer is beneficial, you should do it in two loops - the inner loop writes a certain number of data to the memory stream and the out loop copies the memory stream to the file stream and empty the memory stream.
Unfortunately Capacity (protected) of TMemoryStream is useless because TMemoryStream reallocates memory whenever Size is changed. Using a sufficiently big TBytes avoids memory reallocation but you need to keep track of the length.
Why did you just write the header object to the file? Do you intend to append the body to the file later? Append a file is usually a bad idea.
Some thing trivial: always pass string constants with const.
Some thing trivial: always start a type name with 'T'.
Here is how I would do it:
procedure TDictionary.SaveHeaderToStream(AStream: TStream);
// You want to write to a stream, not a file. This is more reusable.
// You don't want to put the create/free code here,
// this is more flexible as it can also be used by SaveToStream
var
i: Integer;
begin
for i := 0 to length(buffers) - 1 do
AStream.Write(self.buffers[i].b^, self.buffers[i].l^);
end;
procedure TDictionary.SaveHeaderToFile(const Filename: string);
var
fsOutput: TFileStream;
begin
fsOutput := TFileStream.Create(Filename, fmCreate);
try
SaveHeaderToStream(fsOutput); // fsOutput IS TStream
finally
fsOutput.Free;
end;
end;
procedure TDictionary.SaveBodyToStream(AStream: TStream);
begin
// Codes here
end;
procedure TDictionary.SaveToStream(AStream: TStream);
begin
SaveHeaderToStream(AStream);
SaveBodyToStream(AStream);
end;
procedure TDictionary.SaveToFile(const Filename: string);
var
fsOutput: TFileStream;
begin
fsOutput := TFileStream.Create(Filename, fmCreate);
try
SaveToStream(fsOutput);
finally
fsOutput.Free;
end;
end;
I'm modifying a program that is written in Delphi 6.0
I have a table in Oracle with a BLOB column named FILE_CONTENT.
I have already managed to upload an XML File that is about 100 KB. I have verified that the file content was correctly uploaded using SQL Developer.
The problem I have is when I try to download back the file content from DB to a file. This is an example code I'm using to donwload it:
procedure TfrmDownload.Save();
var
fileStream: TFileStream;
bField: TBlobField;
begin
dmDigital.qrGetData.Open;
dmDigital.RequestLive := True;
bField := TBlobField(dmDigital.qrGetData.FieldByName('FILE_CONTENT'));
fileStream := TFileStream.Create('FILE.XML', fmCreate);
bField.SaveToStream(fileStream);
FlushFileBuffers(fileStream.Handle);
fileStream.Free;
dmDigital.qrGetData.Close;
end;
The previous code already downloads the file content to FILE.XML. I'm using RequestLive:=True to be able to download a large BLOB (otherwise the file content is truncated to 32K max)
The resulting file is the same size as the original file. However, when I compare the downloaded file with the original one there are some differences (for example the last character is missing and other characters are also changed), therefore it seems to be a problem while downloading the content.
Do you know what cuould be wrong?
The problem seems to be related to Delphi code because I already tried with C# and the file content is downloaded correctly.
Don't use TBlobField.SaveToStream() directly, use TDataSet.CreateBlobStream() instead (which is what TBlobField.SaveToStream() uses internally anyway):
procedure TfrmDownload.Save;
var
fileStream: TFileStream;
bField: TField;
bStream: TStream;
begin
dmDigital.qrGetData.Open;
try
dmDigital.RequestLive := True;
bField := dmDigital.qrGetData.FieldByName('FILE_CONTENT');
bStream := bField.DataSet.CreateBlobStream(bField, bmRead);
try
fileStream := TFileStream.Create('FILE.XML', fmCreate);
try
fileStream.CopyFrom(bStream, 0);
FlushFileBuffers(fileStream.Handle);
finally
fileStream.Free;
end;
finally
bStream.Free;
end;
finally
dmDigital.qrGetData.Close;
end;
end;
TDataSet.CreateBlobStream() allows the DataSet to decide the best way to access the BLOB data. If the returned TStream is not delivering the data correctly, then either the TStream class implementation that CreateBlobStream() uses is broken, or the underlying DB driver is buggy. Try taking CopyFrom() out of the equation so you can verify the data as it is being retrieved:
procedure TfrmDownload.Save;
const
MaxBufSize = $F000;
var
Buffer: array of Byte;
N: Integer;
fileStream: TFileStream;
bField: TField;
bStream: TStream;
begin
dmDigital.qrGetData.Open;
try
dmDigital.RequestLive := True;
bField := dmDigital.qrGetData.FieldByName('FILE_CONTENT');
bStream := bField.DataSet.CreateBlobStream(bField, bmRead);
try
fileStream := TFileStream.Create('FILE.XML', fmCreate);
try
//fileStream.CopyFrom(bStream, 0);
SetLength(Buffer, MaxBufSize);
repeat
N := bStream.Read(PByte(Buffer)^, MaxBufSize);
if N < 1 then Break;
// verify data here...
fileStream.WriteBuffer(PByte(Buffer)^, N);
until False;
FlushFileBuffers(fileStream.Handle);
finally
fileStream.Free;
end;
finally
bStream.Free;
end;
finally
dmDigital.qrGetData.Close;
end;
end;
I am trying to record the session log from other applications (Proxifier) to a Memo.
I've tried using the command :
procedure TForm1.TimerTimer(Sender: TObject);
begin
Memo1.Lines.LoadFromFile('C:\PMASSH\Proxyfier\Profiles\Log.txt');
end;
but at certain times I get an error
Can you help my problem above ? I would really appreciate of all the answers.
Thanks
The other program has opened the file with a sharing mode that does not allow other processes to read it. Typically this happens when the other application is writing to the file.
There's not a whole lot you can do about this. This is perfectly normal behaviour, and is to be expected. You can try detecting the error, waiting for a short period of time, and re-trying.
Since you are already running this on a timer, the re-try will just happen. So perhaps you just need to suppress those exceptions:
procedure TForm1.TimerTimer(Sender: TObject);
begin
try
Memo1.Lines.LoadFromFile(...);
except
on EFOpenError do
; //swallow this error
end;
end;
Note that detecting EFOpenError is perhaps a little crude. Perhaps there are other failure modes that lead to that error. However, as a first pass, the code above is a decent start.
David's answer is correct. I just want to clarify why this is happening.
The answer lies in the code:
procedure TStrings.LoadFromFile(const FileName: string);
var
Stream: TStream;
begin
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite);
try
LoadFromStream(Stream);
finally
Stream.Free;
end;
end;
as you can see, the file is accessed for sharing but no writing is allowed.
you can solve this by creating the filestream yourself:
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyNone);
and then use the Lines.LoadFromStream() method to load the contents into the memo
Please note that the problem may subsist in cases where the other application has opened the file in exclusive mode (i.e. no sharing), so proper Exception management like in David's answer is still needed.
You can try your luck with ReadFile WinAPI. On a shared read open mode, you'll be able to sneak and read the contents of the file at last file buffer flush. If that another application (Proxifier) opened the file with CreateFile WinAPI with FILE_SHARE_READ share mode then you'll be able to open it for reading, as long as you use ReadFile API. Standart LoadFromFile method won't work here if it still was opened for share, and you'll get the same 'lock' error.
But here's the catch.. You'll have to deal with buffers, sizes and handles... You'll have to assing a handle to file for reading, get the file size with that handle, set an array with that size, do read to that array and assign, add (whatever) that array to the memo.. Pure usage of WinAPI. Some job for a simple task...
Here is a basic example of how to deal with files with WinAPI:
The key assumption of that other application's file open process:
var
Form1: TForm1;
logfile: Textfile;
h: THandle;
implementation
{$R *.dfm}
procedure TForm1.Button1Click(Sender: TObject);
begin
// AssignFile(logfile, 'c:\deneme.txt');
// Rewrite(logfile);
h := CreateFile('C:\deneme.txt', GENERIC_WRITE, FILE_SHARE_READ, nil,
CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
Timer1.enabled := true;
end;
procedure TForm1.Button2Click(Sender: TObject);
begin
Timer1.enabled := false;
// CloseFile(logfile);
CloseHandle(h);
end;
procedure TForm1.Timer1Timer(Sender: TObject);
var
str: AnsiString;
p: pointer;
buf: array of ansichar;
written: cardinal;
begin
// Writeln(logfile, 'denemeStr');
str := 'denemeStr' + #13#10;
p := pansichar(str);
SetLength(buf, length(str));
move(p^, buf[0], length(str));
WriteFile(h, buf[0], length(buf), written, nil);
FlushFileBuffers(h);
end;
And if it's been shared for reading, this is how you can read from it:
procedure TForm1.Button1Click(Sender: TObject);
var
h: THandle;
buf: array of ansichar;
size, read: cardinal;
begin
Memo1.Lines.Clear;
// Memo1.Lines.LoadFromFile('c:\deneme.txt');
h := CreateFile('C:\deneme.txt', GENERIC_READ, FILE_SHARE_READ, nil,
OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
size := GetFileSize(h, nil);
SetLength(buf, size);
ReadFile(h, buf[0], size, read, nil);
CloseHandle(h);
Memo1.Lines.Add(pansichar(buf));
end;
Hope this'd help...
procedure TForm1.TimerTimer(Sender: TObject);
begin
Memo1.Lines.LoadFromFile('C:\PMASSH\Proxyfier\Profiles\Log.txt');//// read file path error if file notfound
// if trying to record
Memo1.Lines.SaveToFile(Path...);
end;
i want to process a text file line by line. In the olden days i loaded the file into a StringList:
slFile := TStringList.Create();
slFile.LoadFromFile(filename);
for i := 0 to slFile.Count-1 do
begin
oneLine := slFile.Strings[i];
//process the line
end;
Problem with that is once the file gets to be a few hundred megabytes, i have to allocate a huge chunk of memory; when really i only need enough memory to hold one line at a time. (Plus, you can't really indicate progress when you the system is locked up loading the file in step 1).
The i tried using the native, and recommended, file I/O routines provided by Delphi:
var
f: TextFile;
begin
Reset(f, filename);
while ReadLn(f, oneLine) do
begin
//process the line
end;
Problem withAssign is that there is no option to read the file without locking (i.e. fmShareDenyNone). The former stringlist example doesn't support no-lock either, unless you change it to LoadFromStream:
slFile := TStringList.Create;
stream := TFileStream.Create(filename, fmOpenRead or fmShareDenyNone);
slFile.LoadFromStream(stream);
stream.Free;
for i := 0 to slFile.Count-1 do
begin
oneLine := slFile.Strings[i];
//process the line
end;
So now even though i've gained no locks being held, i'm back to loading the entire file into memory.
Is there some alternative to Assign/ReadLn, where i can read a file line-by-line, without taking a sharing lock?
i'd rather not get directly into Win32 CreateFile/ReadFile, and having to deal with allocating buffers and detecting CR, LF, CRLF's.
i thought about memory mapped files, but there's the difficulty if the entire file doesn't fit (map) into virtual memory, and having to maps views (pieces) of the file at a time. Starts to get ugly.
i just want Reset with fmShareDenyNone!
With recent Delphi versions, you can use TStreamReader. Construct it with your file stream, and then call its ReadLine method (inherited from TTextReader).
An option for all Delphi versions is to use Peter Below's StreamIO unit, which gives you AssignStream. It works just like AssignFile, but for streams instead of file names. Once you've used that function to associate a stream with a TextFile variable, you can call ReadLn and the other I/O functions on it just like any other file.
You can use this sample code:
TTextStream = class(TObject)
private
FHost: TStream;
FOffset,FSize: Integer;
FBuffer: array[0..1023] of Char;
FEOF: Boolean;
function FillBuffer: Boolean;
protected
property Host: TStream read FHost;
public
constructor Create(AHost: TStream);
destructor Destroy; override;
function ReadLn: string; overload;
function ReadLn(out Data: string): Boolean; overload;
property EOF: Boolean read FEOF;
property HostStream: TStream read FHost;
property Offset: Integer read FOffset write FOffset;
end;
{ TTextStream }
constructor TTextStream.Create(AHost: TStream);
begin
FHost := AHost;
FillBuffer;
end;
destructor TTextStream.Destroy;
begin
FHost.Free;
inherited Destroy;
end;
function TTextStream.FillBuffer: Boolean;
begin
FOffset := 0;
FSize := FHost.Read(FBuffer,SizeOf(FBuffer));
Result := FSize > 0;
FEOF := Result;
end;
function TTextStream.ReadLn(out Data: string): Boolean;
var
Len, Start: Integer;
EOLChar: Char;
begin
Data:='';
Result:=False;
repeat
if FOffset>=FSize then
if not FillBuffer then
Exit; // no more data to read from stream -> exit
Result:=True;
Start:=FOffset;
while (FOffset<FSize) and (not (FBuffer[FOffset] in [#13,#10])) do
Inc(FOffset);
Len:=FOffset-Start;
if Len>0 then begin
SetLength(Data,Length(Data)+Len);
Move(FBuffer[Start],Data[Succ(Length(Data)-Len)],Len);
end else
Data:='';
until FOffset<>FSize; // EOL char found
EOLChar:=FBuffer[FOffset];
Inc(FOffset);
if (FOffset=FSize) then
if not FillBuffer then
Exit;
if FBuffer[FOffset] in ([#13,#10]-[EOLChar]) then begin
Inc(FOffset);
if (FOffset=FSize) then
FillBuffer;
end;
end;
function TTextStream.ReadLn: string;
begin
ReadLn(Result);
end;
Usage:
procedure ReadFileByLine(Filename: string);
var
sLine: string;
tsFile: TTextStream;
begin
tsFile := TTextStream.Create(TFileStream.Create(Filename, fmOpenRead or fmShareDenyWrite));
try
while tsFile.ReadLn(sLine) do
begin
//sLine is your line
end;
finally
tsFile.Free;
end;
end;
If you need support for ansi and Unicode in older Delphis, you can use my GpTextFile or GpTextStream.
As it seems the FileMode variable is not valid for Textfiles, but my tests showed that multiple reading from the file is no problem. You didn't mention it in your question, but if you are not going to write to the textfile while it is read you should be good.
What I do is use a TFileStream but I buffer the input into fairly large blocks (e.g. a few megabytes each) and read and process one block at a time. That way I don't have to load the whole file at once.
It works quite quickly that way, even for large files.
I do have a progress indicator. As I load each block, I increment it by the fraction of the file that has additionally been loaded.
Reading one line at a time, without something to do your buffering, is simply too slow for large files.
I had same problem a few years ago especially the problem of locking the file. What I did was use the low level readfile from the shellapi. I know the question is old since my answer (2 years) but perhaps my contribution could help someone in the future.
const
BUFF_SIZE = $8000;
var
dwread:LongWord;
hFile: THandle;
datafile : array [0..BUFF_SIZE-1] of char;
hFile := createfile(PChar(filename)), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_READONLY, 0);
SetFilePointer(hFile, 0, nil, FILE_BEGIN);
myEOF := false;
try
Readfile(hFile, datafile, BUFF_SIZE, dwread, nil);
while (dwread > 0) and (not myEOF) do
begin
if dwread = BUFF_SIZE then
begin
apos := LastDelimiter(#10#13, datafile);
if apos = BUFF_SIZE then inc(apos);
SetFilePointer(hFile, aPos-BUFF_SIZE, nil, FILE_CURRENT);
end
else myEOF := true;
Readfile(hFile, datafile, BUFF_SIZE, dwread, nil);
end;
finally
closehandle(hFile);
end;
For me the speed improvement appeared to be significant.
Why not simply read the lines of the file directly from the TFileStream itself one at a time ?
i.e. (in pseudocode):
readline:
while NOT EOF and (readchar <> EOL) do
appendchar to result
while NOT EOF do
begin
s := readline
process s
end;
One problem you may find with this is that iirc TFileStream is not buffered so performance over a large file is going to be sub-optimal. However, there are a number of solutions to the problem of non-buffered streams, including this one, that you may wish to investigate if this approach solves your initial problem.