Lazarus/Delphi: UnicodeString in self-allocated record data type causes access violations - dynamic-memory-allocation

I guess, my problem is caused by the concept how UnicodeStrings are implemented, but I cannot solve this problem.
I'm trying to scan a directory tree on disk recursively and build a treeview which should show all files and subfolders. Plus, I want to store additional information for each tree node. The TTreeNode object has only a "Data" property (type Pointer) for this purpose, so I allocate memory manually, store the information and assign the allocated pointer to my data property. Everything seems to work fine unless I include usage of a UnicodeString field within my data record.
So, here is my custom data record definition:
type
TFileInformation = record
AbsoluteFileName: UnicodeString;
FileSize: Int64;
FileAttributes: LongInt;
CreationTime, ModificationTime: TDateTime;
end;
And here is my code for directory recusion:
const NO_ERROR = 0;
procedure ScanDirectory(Folder: UnicodeString; Node: TTreeNode);
var
Details: Pointer;
NewNode: TTreeNode;
SearchAttributes: LongInt;
SearchMask: UnicodeString;
SearchRecord: TUnicodeSearchRec;
begin
if (Folder <> '') and (Folder[Length(Folder)] <> DirectorySeparator) then begin
Folder += DirectorySeparator;
end;
SearchMask := Folder + '*'{$IFDEF WINDOWS} + '.*'{$ENDIF};
SearchAttributes := faReadOnly or faHidden or faSysFile or faDirectory or faArchive or faSymLink;
if FindFirst(SearchMask, SearchAttributes, SearchRecord) = NO_ERROR then begin
repeat
if ((SearchRecord.Attr and faDirectory) <> faDirectory) or
((SearchRecord.Name <> '.') and (SearchRecord.Name <> '..')) then begin
Details := MemAlloc(SizeOf(TFileInformation));
//TFileInformation(Details^).AbsoluteFileName := Folder + SearchRecord.Name;
TFileInformation(Details^).FileAttributes := SearchRecord.Attr;
TFileInformation(Details^).FileSize := SearchRecord.Size;
TFileInformation(Details^).CreationTime := SearchRecord.Time;
//TFileInformation(Details^).ModificationTime := -1;
if Node = nil then begin
NewNode := self.trvOrigin.Items.AddNode(nil, nil, ansistring(SearchRecord.Name), Details, naAdd);
end else begin
NewNode := self.trvOrigin.Items.AddNode(nil, Node, ansistring(SearchRecord.Name), Details, naAddChild);
end;
if (SearchRecord.Attr and (faDirectory or faSymLink)) = faDirectory then begin
// only recurse folders which are NOT SymLink:
ScanDirectory(Folder + SearchRecord.Name, NewNode);
end;
end;
until FindNext(SearchRecord) <> NO_ERROR;
end;
FindClose(SearchRecord);
end;
When I uncomment the line containing .AbsoluteFileName :=, I get an Access Violation (SIGSEGV-Exception in Unix).
I'm currently using Lazarus in objfpc mode on Debian Linux, but I guess it's the same with Delphi on Windows. The Treeview.Data property value is stored in the "Details" variable in my example code, self.trvOrigin is my treeview control.

When you allocate the Details record, the memory is not defined.
AbsoluteFileName is a managed type and must be properly initialized before use.
You need to clear the memory after the allocation:
FillChar(Details^, SizeOf(TFileInformation), #0);
As an alternative, use New(Details) in combination with Dispose(Details).
They will correctly initialize/finalize the record.
Note: Details must be a typed pointer, PFileInformation.

Related

How do I populate a Treeview via FTP

Scenario
I'm trying to duplicate the standard way to fill a Treeview with directories/folders from a folder structure, starting at the root, but using IdFTP to get the structure from a remote server instead of my local hard drive. I'd like the result to look similar to clients like Filezilla.
I used this reasonably standard code from the Swiss Delphi Centre (which works to display my hard drive's structure) and then modified it to use IdFTP.ChangeDir(Directory) and IdFTP.List instead of FindFirst() and FindNext().
Problem
I seem to have got myself in a muddle as it is not correctly 'unwinding' the recursion so that once it traverses down the /cpanel/cashes/config directories on the remote server it doesn't return and traverse all the other directories hanging off the root but exits the procedure without displaying anything else. Also it doesn't seem to show all the top level folders but this could be simply due to the order that IdFTP.List returns them in
Can anyone tell me what I have done wrong here?
If you can also tell me how I should get the root (/) shown as well that would be very helpful
(I've commented out displaying non directories as I only want folders at this stage)
What I expected to see Copied from Filezilla
What I did see Using a Ttreeview in Delphi
My Code
procedure TForm2.Button1Click(Sender: TObject);
var StartingDir : string;
begin
TreeView1.Items.BeginUpdate;
try
StartingDir := '/';
Screen.Cursor := crHourGlass;
TreeView1.Items.Clear;
FTPconnect; //procedure to connect to remote server
GetDirectories(TreeView1, StartingDir, nil, True);
FTPDisconnect; //procedure to disconnect from remote server
finally
TreeView1.Items.EndUpdate;
Screen.Cursor := crDefault;
end;
end;
procedure TForm2.GetDirectories(Tree: TTreeView; Directory: string; Item: TTreeNode; IncludeFiles: Boolean);
var
ItemTemp: TTreeNode;
DirItemType : TIdDirItemType ;
Filename , NewStartingDirectory: string;
i : Integer;
begin
Tree.Items.BeginUpdate;
IdFTP.ChangeDir(Directory);
IdFTP.List; //get directory of remote folder
i:=0;
repeat
DirItemType := IdFTP.DirectoryListing[I].ItemType;
Filename := IdFTP.DirectoryListing[I].FileName;
If (DirItemType = ditDirectory) and (Filename <> '.') and (Filename <> '..')then
begin
if DirItemType = ditDirectory then
Item := Tree.Items.AddChild(Item, Filename);
ItemTemp := Item.Parent;
if Directory = '/' then
NewStartingDirectory := Directory + Filename
else
NewStartingDirectory := Directory + '/' +Filename;
GetDirectories(Tree, NewStartingDirectory, Item, IncludeFiles);
Item := ItemTemp;
end
else
if IncludeFiles then
begin //this bit commented out as we only want to see directories
// if (Filename <> '.') and (Filename <> '..') then
// Tree.Items.AddChild(Item, Filename);
end;
inc(i);
until i = IdFTP.DirectoryListing.Count;
Tree.Items.EndUpdate;
end;
Swiss Delhpi Centre's code (for comparison)
procedure TForm1.Button1Click(Sender: TObject);
var
Node: TTreeNode;
Path: string;
Dir: string;
begin
Dir := 'c:\temp';
Screen.Cursor := crHourGlass;
TreeView1.Items.BeginUpdate;
try
TreeView1.Items.Clear;
GetDirectories(TreeView1, Dir, nil, True);
finally
Screen.Cursor := crDefault;
TreeView1.Items.EndUpdate;
end;
end;
procedure TForm1.GetDirectories(Tree: TTreeView; Directory: string; Item: TTreeNode; IncludeFiles: Boolean);
var
SearchRec: TSearchRec;
ItemTemp: TTreeNode;
begin
Tree.Items.BeginUpdate;
if Directory[Length(Directory)] <> '\' then Directory := Directory + '\';
if FindFirst(Directory + '*.*', faDirectory, SearchRec) = 0 then
begin
repeat
if (SearchRec.Attr and faDirectory = faDirectory) and (SearchRec.Name[1] <> '.') then
begin
if (SearchRec.Attr and faDirectory > 0) then
Item := Tree.Items.AddChild(Item, SearchRec.Name);
ItemTemp := Item.Parent;
GetDirectories(Tree, Directory + SearchRec.Name, Item, IncludeFiles);
Item := ItemTemp;
end
else if IncludeFiles then
if SearchRec.Name[1] <> '.' then
Tree.Items.AddChild(Item, SearchRec.Name);
until FindNext(SearchRec) <> 0;
FindClose(SearchRec);
end;
Tree.Items.EndUpdate;
end;
I've looked on SO here - too complicated and wrong language and here - similar to the Swiss Delphi Centre and here - wrong language and not sure what its doing.
if it's better to use a TlistView, can you please show me the equivalent code to use that instead?
Untested:
I made the TIdFTP variable a parameter, since TTreeView was also one and it should be done consistently, not archaic.
Using for loops instead of repeat until.
Eliminating IncludeFiles when it wasn't used anyway.
Eliminating weird logic to always get the new TreeNode's parent.
Not locking the TreeView anymore - do this once before calling this method and unlock it after calling - otherwise you do that dozens of times in vain.
Basic logic is as I wrote in the comments:
Store all folder strings into your own list and avoid recursion at this point.
Fix the path to be concatenated once, not with every iteration of a loop.
Go through that list to do the recursion - at this point the state of FTP is irrelevant and you won't mess up listings at different levels.
Of course, release the created instance of the StringList.
procedure TForm2.GetFolders
( Ftp: TIdFTP // The source, from which we read the content
; Tree: TTreeView // The destination, which we want to fill
; ParentNode: TTreeNode // Node under which all new child nodes should be created
; Path: String // Starting directory
);
var
NewNode: TTreeNode; // New child in the tree
Filename: String; // Check against unwanted folder entries
i: Integer; // Looping over both lists
sl: TStringList; // Collect folders only
begin
FTP.ChangeDir( Path );
FTP.List; // Entire remote listing
sl:= TStringList.Create; // Collect all entries we're interested in
try
for i:= 0 to FTP.DirectoryListing.Count- 1 do begin // For each entry
Filename:= FTP.DirectoryListing[i].FileName;
if (FTP.DirectoryListing[i].ItemType= ditDirectory) // Only folders
and (Filename<> '.')
and (Filename<> '..') then begin
sl.Add( Filename ); // Only the name, not the full path
end;
end;
// Do this only once
if Path<> '/' then Path:= '/'+ Path+ '/';
for i:= 0 to sl.Count- 1 do begin // All collected folders
NewNode:= Tree.Items.AddChild( ParentNode, sl[i] ); // Populate tree
GetFolders( Ftp, Tree, NewNode, Path+ sl[i] ); // Recursion of folder name + current path
end;
finally
sl.Free;
end;
end;
Untested, but should compile.

Drag and drop from VirtualTreeView to shell (Ole drag and drop)

I am trying to drag and drop from VirtualTreeView to create a file in shell (drag and drop from VirtualTreeView to a folder in File Explorer or desktop folder).
I only found example of doing the opposite (shell to VirtualTreeView), but I cannot find any example for doing that. Help?
Doing any drag-drop operations in Windows involves creating an IDataObject, and giving that object to Windows.
The Virtual Treeview handles a lot of that grunt-work for you, creating an object that implements IDataObject for you. The tree then raises events when you need to help populate it.
When passing "file-like" things through a copy-paste or a drag-drop, you are require to add two clipboard formats to the IDataObject:
CF_FILEDESCRIPTOR, and
CF_FILECONTENTS
In addition to support for formats that the virtualtree itself will add, you can choose to indicate support for more clipboard format.
OnGetUserClipboardFormats Event
This is the event where you are given a chance to add additional clipboard formats to the IDataObject that the tree will be creating:
procedure TForm1.lvAttachmentsGetUserClipboardFormats(Sender: TBaseVirtualTree;
var Formats: TFormatEtcArray);
var
i: Integer;
begin
//Add formats for CF_FILEDESCRIPTOR and CF_FILECONTENTS
i := Length(Formats);
SetLength(Formats, i + 1);
Formats[i].cfFormat := CF_FILEDESCRIPTOR;
Formats[i].ptd := nil;
Formats[i].dwAspect := DVASPECT_CONTENT;
Formats[i].lindex := -1;
Formats[i].tymed := TYMED_HGLOBAL;
i := Length(Formats);
SetLength(Formats, i + 1);
Formats[i].cfFormat := CF_FILECONTENTS;
Formats[i].ptd := nil;
Formats[i].dwAspect := DVASPECT_CONTENT;
Formats[i].lindex := 0;
Formats[i].tymed := TYMED_ISTREAM;
end;
The tree will then given the IDataObject to the shell as part of the drag-drop operation.
Later, an application that the user dropped items onto will enumerate all formats in the IDataObject, e.g.:
CF_HTML ("HTML Format")
CFSTR_FILEDESCRIPTOR ("FileGroupDescriptorW")
CFSTR_FILECONTENTS ("FileContents")
CF_ENHMETAFILE
And it will see that the IDataObject contains FileDescriptor and FileContents.
The receiving application will then ask the IDataObject to actually cough up data. (This "delayed-rendering" is a good thing, it means your source application doesn't actually have to read any content unless it actually gets requested).
OnRenderOleData Event
This is the event where the virtual tree realizes its IDataObject has been asked to render something, and it needs you to finally render that actual content.
The general idea with these two clipboard formats is:
CF_FILEDESCRIPTOR lets you return a record that describes the file-like thing (e.g. filename, file size, created date, last modified date, last accessed date)
CF_FILECONTENTS lets you return an IStream that contains the actual file contents
procedure TForm1.lvAttachmentsRenderOLEData(Sender: TBaseVirtualTree; const FormatEtcIn: tagFORMATETC;
out Medium: tagSTGMEDIUM; ForClipboard: Boolean; var Result: HRESULT);
var
global: HGLOBAL;
stm: IStream;
begin
if FormatEtcIn.cfFormat = CF_FILEDESCRIPTOR then
begin
global := GetAttachmentFileDescriptorsFromListView(lvAttachments, ForClipboard);
if global = 0 then
Exit;
ZeroMemory(#Medium, SizeOf(Medium));
Medium.tymed := TYMED_HGLOBAL;
Medium.hGlobal := global;
Result := S_OK;
end
else if FormatEtcIn.cfFormat = CF_FILECONTENTS then
begin
ZeroMemory(#Medium, SizeOf(Medium));
Medium.tymed := TYMED_ISTREAM;
Result := GetAttachmentStreamFromListView(lvAttachments, ForClipboard, FormatEtcIn.lindex, stm);
if Failed(Result) then
Exit;
Medium.stm := Pointer(stm);
IUnknown(Medium.stm)._AddRef;
Result := S_OK;
end;
end;
The first helper function creates an array of FILE_DESCRIPTOR objects, and copies them to a HGLOBAL allocated memory:
function GetAttachmentFileDescriptorsFromListView(Source: TVirtualStringTree; ForClipboard: Boolean): HGLOBAL;
var
i: Integer;
nCount: Integer;
nodes: TNodeArray;
descriptors: TFileDescriptorDynArray;
data: TAttachment;
begin
Result := 0;
if ForClipboard then
nodes := Source.GetSortedCutCopySet(False)
else
nodes := Source.GetSortedSelection(False);
if Length(nodes) = 0 then
Exit;
nCount := 0;
for i := 0 to Length(nodes) - 1 do
begin
//Get the file thing from this node
data := GetNodeDataFromNode(nodes[i]);
if not Assigned(data) then
Continue;
//Increase the size of our descriptors array by one
Inc(nCount);
SetLength(Descriptors, nCount);
//Fill in the next descriptor
descriptors[nCount-1] := data.ToWindowsFileDescriptor;
end;
Result := FileDescriptorsToHGLOBAL(descriptors);
end;
The second helper copies your file-like thing's binary contents to an IStream:
function GetAttachmentStreamFromListView(Source: TVirtualStringTree; ForClipboard: Boolean; lindex: Integer; var stm: IStream): HResult;
var
nodes: TNodeArray;
data: TAttachment;
begin
Result := E_FAIL;
if ForClipboard then
nodes := Source.GetSortedCutCopySet(False)
else
nodes := Source.GetSortedSelection(False);
if Length(nodes) = 0 then
Exit;
if (lIndex < Low(Nodes)) or (lIndex > High(Nodes)) then
begin
Result := DV_E_LINDEX;
Exit;
end;
//Get the file thing from this node
data := GetNodeDataFromNode(nodes[i]);
if not Assigned(data) then
Continue;
//Fetch the content into a IStream wrapped memory stream
stm := data.GetStream(nil);
Result := S_OK;
end;
Your attachment object, whatever it is has to know:
how to represent itself as a TFileDescriptor
how to return the contents as an IStream

How to read FoxPro Memory Variable Files (.MEM) with Delphi

I'm trying to read a .MEM file using Delphi. It's a FoxPro Memory Variable Files. I've tried to read using TFileStream and load into TStringList. But, it only returns the first word.
F := TFileStream.Create(sFile, fmOpenRead);
L := TStringList.Create;
try
F.Position := 0;
L.LoadFromStream(F);
ShowMessage(L.Text);
finally
F.Free;
L.Free;
end;
The reason is because I want to migrate some useful .MEM values from an old program to my new program. Thanks for any help.
If it's a one-time affair and you have access to a VFP installation - i.e. the IDE, not only the runtime - then David Heffernan's suggestion is certainly the most sensible way. In this case you can load and inspect the .MEM via
release all extended && to clear away all existing memvars
restore from foo && assuming the file in question is named FOO.MEM
activate window Locals && inspect the variables...
list memory to foo && or list them to FOO.TXT
modify file foo.txt
However, LIST MEMORY (and DISPLAY MEMORY) also include all the system variables - the things that start with an underscore - which would need to be parsed off.
If it's an ongoing affair - repeated imports necessary - and you know which variables you need then there are two fairly clean and easy ways.
The first is only valid if a VFP IDE is installed on the computer on which the Delphi program is to be run. In this case you can instantiate VFP from Delphi (leave it invisible), have it read the .MEM and then query individual variables:
procedure fetch_variables_from_MEM (mem_filename: string; var_list: CFoos);
var
fox: Variant;
foo: CFoo;
begin
fox := CreateOleObject('VisualFoxpro.Application.9');
try
fox.DoCmd('release all extended');
fox.DoCmd('restore from ' + mem_filename);
for foo in var_list do
foo.Value := fox.Eval('m.' + foo.Name);
finally
fox.Quit; // AutoQuit not supported
end;
end;
I glossed over some details, like that CoInitialize() needs to be called on the thread somewhere before calling this, and I assumed suitable definitions for the variable list (a list/collection of hypothetical CFoo objects), but the sketched outline works - even in 64-bit Delphi.
The advantage is that things like datetime values arrive as TDateTime by virtue of the COM infrastructure and the use of variants.
The second easy way is applicable if an IDE is not available on the machine where the Delphi program is to be used but you have access to an IDE somewhere, so that you can build a small COM server:
define class FoxWrapper as custom olepublic
function Eval (cExpression as string) as variant
return evaluate(m.cExpression)
procedure DoCmd (cCommand as string)
&cCommand
enddefine
This can then be used instead of "VisualFoxPro.Application.9" in the example above. Note: for 64-bit Delphi you need to build this as an out-of-process server (i.e. an EXE). Also, this may run afoul of the VFP licence conditions.
For accessing the data directly, here's some quick & dirty Delphi code that I modelled after some FoxPro stuff that I coded eons ago and updated for VFP9. This is proof-of-principle code with simplified array handling and other compromises for the sake of exposition; it lacks all the production-quality noise necessitated by Delphi's half-assed language definition and its quarter-assed runtime.
type
TMEMVarHeader = packed record
var_name: array [0..10] of AnsiChar;
mem_type: AnsiChar; // 0ACDHLNOQYacdhlnoqy
big_size: UInt32; // only if mem_type == 'H'
width : Byte; // special meaning if mem_type == 'H'
decimals: Byte;
padding : array [0..13] of Byte; // 0 0 0 0 0 0 0 3 0 0 0 0 0 0
end;
SizeOf_TMEMVarHeader_eq_32 = true .. SizeOf(TMEMVarHeader) = 32;
TMEMVarInfo = record
header: TMEMVarHeader;
null_t: AnsiChar;
name : AnsiString;
value : Variant;
function ReadFromStream (stream: TStream): Boolean; // false if EOF
end;
function TMEMVarInfo.ReadFromStream (stream: TStream): Boolean;
const
DELPHI_EPOCH = 2415019.0;
var
header_bytes_read: Integer;
name_length: UInt16;
text_length: UInt32;
array_dim_1: UInt16;
array_dim_2: UInt16;
d: TDate; // 64-bit double
l: Boolean;
n: Double; // 64-bit double
q: array of Byte;
c: AnsiString;
t: TDateTime; // 64-bit double
y: Int64;
binary: Boolean;
i: Cardinal;
a: array of Variant;
v: TMEMVarInfo;
begin
name := ''; value := Unassigned;
header_bytes_read := stream.Read(header, SizeOf(header));
if header_bytes_read <> Sizeof(header) then begin
if not ((header_bytes_read = 1) and (header.var_name[0] = #26)) then
raise Exception.Create('unexpected MEM file format (problem reading header)');
result := false; // EOF
EXIT;
end;
result := true;
// variable name
if header.var_name[0] = #0 then begin // long variable name
assert(header.mem_type = LoCase(header.mem_type));
stream.ReadBuffer(name_length, Sizeof(name_length));
SetLength(name, name_length);
stream.ReadBuffer(name[1], name_length);
end else begin
assert(header.mem_type = UpCase(header.mem_type));
name := header.var_name;
end;
// variable value
case UpCase(header.mem_type) of
'A':
begin
stream.ReadBuffer(array_dim_1, SizeOf(array_dim_1));
stream.ReadBuffer(array_dim_2, SizeOf(array_dim_2));
if array_dim_2 = 0 then // it's a vector, not an array
array_dim_2 := 1;
SetLength(a, array_dim_1 * array_dim_2);
for i := 0 to array_dim_1 * array_dim_2 - 1 do begin
if not v.ReadFromStream(stream) then
raise Exception.Create('error reading array element');
a[i] := v.value;
end;
value := a;
end;
'0': begin stream.ReadBuffer(null_t, 1); value := Null; end;
'C', 'H', 'Q':
begin
if UpCase(header.mem_type) = 'H' then begin // length > 254
binary := header.width <> 0;
text_length := header.big_size;
end else begin
binary := UpCase(header.mem_type) = 'Q';
text_length := header.width;
end;
if binary then begin
SetLength(q, text_length); stream.ReadBuffer(q[0], text_length); value := q;
end else begin
SetLength(c, text_length); stream.ReadBuffer(c[1], text_length); value := c;
end;
end;
'D': begin stream.ReadBuffer(d, Sizeof(d)); if d > 0 then d := d - DELPHI_EPOCH; VarCast(value, d, varDate); end;
'L': begin stream.ReadBuffer(l, Sizeof(l)); value := l; end;
'N': begin stream.ReadBuffer(n, Sizeof(n)); value := n; end;
'T': begin stream.ReadBuffer(t, Sizeof(t)); if t > 0 then t := t - DELPHI_EPOCH; value := t; end;
'Y': begin stream.ReadBuffer(y, Sizeof(y)); VarCast(value, y / 10000.0, varCurrency); end;
else
raise Exception.Create('unexpected type ''' + header.mem_type + ''' in MEM file');
end;
end;
For reading a .MEM, create a TFileStream and a TMEMVarInfo variable, then read variables one by one until var_info.ReadFromStream(stream) returns false.
Note: the byte at offset 19h (shown as 3 in the structure comment) is a code page identifier. The values are the same as those found in .DBF headers, i.e. 1 for DOS 437, 3 for Windows 1252 and so on. However, even though VFP stores these identifiers when writing a .MEM, all the newer versions of VFP that I tested completely ignore these code page marks when loading a .MEM. A self-written importer could put the code page marks to good use, though.
Reading the binary .mem files is not the correct way to proceed. The correct solution is to get VFP to export the data. It knows how to read it. Get VFP to export to a known format, and read that. This is the standard approach to data migration.

TClientDataSet And Deleting Files In A Collection

I have an experimental app that I am developing that adds image filenames to a collection. I am attempting to find the most efficient way to delete all the files in a collection except
for files that exist in another collection. Files can exist in any collection.
I have a TClientDataSet with the following fields:
ClientDataSet1.FieldDefs.Add('Index', ftInteger);
ClientDataSet1.FieldDefs.Add('Collection', ftString, 50);
ClientDataSet1.FieldDefs.Add('Filename', ftString, 254);
I came up with this which seems to work but seems inefficient:
var
i: Integer;
j: Integer;
iCollectionToDelete: string;
iCollection: string;
iFilename: string;
iFilenameInOtherCollection: string;
iFilesInOtherCollectionsStringList: TStringList;
begin
iCollectionToDelete := ListBox1.Items[ListBox1.ItemIndex];
{ Set filtered to false to see all the records in the database }
ClientDataSet1.Filtered := False;
{ Create a list of files not in the collection to be deleted }
iFilesInOtherCollectionsStringList := TStringList.Create;
try
for i := 0 to ClientDataSet1.RecordCount - 1 do
begin
iCollection := ClientDataSet1.FieldByName('Collection').AsString;
iFilename := ClientDataSet1.FieldByName('Filename').AsString;
if iCollection <> iCollectionToDelete then
begin
iFilenameInOtherCollection := ClientDataSet1.FieldByName('Filename').AsString;
iFilesInOtherCollectionsStringList.Add(iFilename);
end;
ClientDataSet1.Next;
end;
{ Compare the iFilenameInOtherCollection with all the filenames in the
dataset and if the iFilename is not in the other collection then
erase the file }
ClientDataSet1.First;
for i := 0 to ClientDataSet1.RecordCount - 1 do
begin
iFilename := ClientDataSet1.FieldByName('Filename').AsString;
ClientDataSet1.Next;
for j := 0 to iFilesInOtherCollectionsStringList.Count - 1 do
begin
iFilenameInOtherCollection := iFilesInOtherCollectionsStringList[j];
if iFilesInOtherCollectionsStringList.IndexOf(iFilename) = -1 then
if FileExists(iFilename) then
WindowsErase(handle, iFilename, False, False, False, False);
end;
end;
finally
iFilesInOtherCollectionsStringList.Free;
end;
end;
My question is can this be made more efficient or is there a way to do the same thing
using just TClientDataset methods?
Just for amusement, I thought I'd try doing this without using Stringlists at all, and instead use a couple of features of ClientDataSets, namely Filtering and the ability to copy data from one CDS to another in a single statement. It's quite a bit shorter than using stringlists and probably easier to maintain/modify/refactor as a result.
I've not benchmarked it against the Stringlist version but would be surprised if it were any faster,
because it depends on TClientDataSet.Locate, which is not a particularly efficient operation even when working against an indexed field.
The code is below. Hopefully the comments will explain how it works.
procedure TForm1.SetUp;
begin
ClientDataSet1.FieldDefs.Add('Index', ftInteger);
ClientDataSet1.FieldDefs.Add('Collection', ftString, 50);
ClientDataSet1.FieldDefs.Add('Filename', ftString, 254);
ClientDataSet1.CreateDataSet;
// Create some test data
ClientDataSet1.InsertRecord([1, 'C1', 'F1']);
ClientDataSet1.InsertRecord([2, 'C2', 'F1']);
ClientDataSet1.InsertRecord([3, 'C3', 'F1']);
ClientDataSet1.InsertRecord([4, 'C1', 'F2']);
ClientDataSet1.InsertRecord([5, 'C3', 'F3']);
end;
procedure Tform1.ApplyCDSFilter(CDS : TClientDataSet; FilterExpr : String);
// utility routine to filter/unfilter a dataset
begin
CDS.Filtered := False;
CDS.Filter := FilterExpr;
if FilterExpr <> '' then
CDS.Filtered := True;
end;
procedure TForm1.RemoveFilesOnlyInCollection(CollectionName : String);
var
CDS : TClientDataSet;
FilterExpr : String;
AFileName : String;
begin
// In the following, I'm just going to add the names of the files which belong to the
// specified collection as well as to another one
// to a listbox so as to be able to check the results by inspection
Listbox1.Items.Clear;
// next create a temporary CDS
CDS := TClientDataSet.Create(Nil);
// Index it by Filename
CDS.IndexFieldNames := 'Filename';
// Copy the data from ClientDataSet1 into it
CDS.Data := ClientDataSet1.Data;
// Construct a filter expression to select the collection whose members are to be
// retained. NOTE : the QuotedStr is to handle quotes embedded in the collection name.
FilterExpr := '(Collection =' + QuotedStr(CollectionName) + ')';
// Apply the filter to ClientDataSet1, so that only records that contain the CollectionName
// are "visible", temporarily
ApplyCDSFilter(ClientDataSet1, FilterExpr);
// Next, negate the filter expression and apply it to the temporary CDS
FilterExpr := 'not ' + FilterExpr;
ApplyCDSFilter(CDS, FilterExpr);
// Now, we can loop through ClientDataSet1 and test whether the Filename is present
// in the temporary CDS. If it is, that means that the Filename belongs to another
// collection too.
try
ClientDataSet1.DisableControls;
ClientDataSet1.First;
while not ClientDataSet1.Eof do begin
AFileName := ClientDataSet1.FieldByName('Filename').AsString;
if not CDS.Locate('Filename', AFileName, [loCaseInsensitive]) then
Listbox1.Items.Add(AFileName);
ClientDataSet1.Next;
end;
finally
CDS.Free;
ClientDataSet1.EnableControls;
ApplyCDSFilter(ClientDataSet1, '');
end;
end;
Just add iFilesInOtherCollectionsStringList.Sorted := True after you fill it. IndexOf will then use fast binary search instead of extremely slow one-by-one looping. Probably that will be enough for your purposes.
Another option is to prepare list-to-delete first and then launch a worker thread which will execute removal in the background. This will likely help because usually file operations much more slow than memory comparisons. You may check if it is the deletion that slows down your process by commenting out the WindowsErase line.

How to Search a File through all the SubDirectories in Delphi

I have implemented this code in Delphi, it will search for the File or the name given but it omits searching all the subdirectories. How can this be done?
Code:
if FindFirst(filePath,faAnyFile,searchResult)=0 then
try
repeat
lbSearchResult.Items.Append(searchResult.Name);
until FindNext(searchResult)<>0
except
on e:Exception do
ShowMessage(e.Message);
end; //try ends
FindClose(searchResult);
With Delphi XE and up, you can have a look at IOUtils.pas:
TDirectory.GetFiles('C:\', '*.dll', TSearchOption.soAllDirectories);
If you don't need threading, the simplest way is this:
procedure TForm1.AddAllFilesInDir(const Dir: string);
var
SR: TSearchRec;
begin
if FindFirst(IncludeTrailingBackslash(Dir) + '*.*', faAnyFile or faDirectory, SR) = 0 then
try
repeat
if (SR.Attr and faDirectory) = 0 then
ListBox1.Items.Add(SR.Name)
else if (SR.Name <> '.') and (SR.Name <> '..') then
AddAllFilesInDir(IncludeTrailingBackslash(Dir) + SR.Name); // recursive call!
until FindNext(Sr) <> 0;
finally
FindClose(SR);
end;
end;
procedure TForm1.Button1Click(Sender: TObject);
begin
ListBox1.Items.BeginUpdate;
AddAllFilesInDir('C:\Users\Andreas Rejbrand\Documents\Aweb');
ListBox1.Items.EndUpdate;
end;
The simplest way is:
uses
DSiWin32;
DSiEnumFilesToStringList('c:\somefolder\file.name', 0, ListBox1.Items, true, true);
DSiWin32 is a free Delphi library.
When i need to do trickslike override protected methods i tend to use a generic solution to the problem... i do a hack to the class.
Here is how to do it with TDirectoryListbox.
On every Form you need to use this hacked TDirectoryListbox just add unitTDirectoryListbox_WithHiddenAndSystemFolders to interface uses, that way the form will use the hacked TDirectoryListbox.
Create a file called unitTDirectoryListbox_WithHiddenAndSystemFolders.pas on your proyect folder.
Put this text inside that file (i will explain later what i have done):
unit unitTDirectoryListbox_WithHiddenAndSystemFolders;
interface
uses
Windows
,SysUtils
,Classes
,FileCtrl
;
type TDirectoryListbox=class(FileCtrl.TDirectoryListbox)
private
FPreserveCase:Boolean;
FCaseSensitive:Boolean;
protected
function ReadDirectoryNames(const ParentDirectory:String;DirectoryList:TStringList):Integer;
procedure BuildList;override;
public
constructor Create(AOwner:TComponent);override;
destructor Destroy;override;
property PreserveCase:Boolean read FPreserveCase;
property CaseSensitive:Boolean read FCaseSensitive;
end;
implementation
constructor TDirectoryListbox.Create(AOwner:TComponent);
begin
inherited Create(AOwner);
end;
destructor TDirectoryListbox.Destroy;
begin
inherited Destroy;
end;
function TDirectoryListbox.ReadDirectoryNames(const ParentDirectory:String;DirectoryList:TStringList):Integer;
var
TheCount,Status:Integer;
SearchRec:TSearchRec;
begin
TheCount:=0;
Status:=FindFirst(IncludeTrailingPathDelimiter(ParentDirectory)+'*.*',faDirectory or faHidden or faSysFile,SearchRec);
try
while 0=Status
do begin
if faDirectory=(faDirectory and SearchRec.Attr)
then begin
if ('.'<>SearchRec.Name)
and
('..'<>SearchRec.Name)
then begin
DirectoryList.Add(SearchRec.Name);
Inc(TheCount);
end;
end;
Status:=FindNext(SearchRec);
end;
finally
FindClose(SearchRec);
end;
ReadDirectoryNames:=TheCount;
end;
procedure TDirectoryListBox.BuildList;
var
TempPath: string;
DirName: string;
IndentLevel, BackSlashPos: Integer;
VolFlags: DWORD;
I: Integer;
Siblings: TStringList;
NewSelect: Integer;
Root: string;
begin
try
Items.BeginUpdate;
Items.Clear;
IndentLevel := 0;
Root := ExtractFileDrive(Directory)+'\';
GetVolumeInformation(PChar(Root), nil, 0, nil, DWORD(i), VolFlags, nil, 0);
FPreserveCase := VolFlags and (FS_CASE_IS_PRESERVED or FS_CASE_SENSITIVE) <> 0;
FCaseSensitive := (VolFlags and FS_CASE_SENSITIVE) <> 0;
if (Length(Root) >= 2) and (Root[2] = '\') then
begin
Items.AddObject(Root, OpenedBMP);
Inc(IndentLevel);
TempPath := Copy(Directory, Length(Root)+1, Length(Directory));
end
else
TempPath := Directory;
if (Length(TempPath) > 0) then
begin
if AnsiLastChar(TempPath)^ <> '\' then
begin
BackSlashPos := AnsiPos('\', TempPath);
while BackSlashPos <> 0 do
begin
DirName := Copy(TempPath, 1, BackSlashPos - 1);
if IndentLevel = 0 then DirName := DirName + '\';
Delete(TempPath, 1, BackSlashPos);
Items.AddObject(DirName, OpenedBMP);
Inc(IndentLevel);
BackSlashPos := AnsiPos('\', TempPath);
end;
end;
Items.AddObject(TempPath, CurrentBMP);
end;
NewSelect := Items.Count - 1;
Siblings := TStringList.Create;
try
Siblings.Sorted := True;
{ read all the dir names into Siblings }
ReadDirectoryNames(Directory, Siblings);
for i := 0 to Siblings.Count - 1 do
Items.AddObject(Siblings[i], ClosedBMP);
finally
Siblings.Free;
end;
finally
Items.EndUpdate;
end;
if HandleAllocated then
ItemIndex := NewSelect;
end;
end.
Now i explain what i have done:
By adding unitTDirectoryListbox_WithHiddenAndSystemFolders to interface uses i make the form to use the modified (aka, hacked) component.
I started by copying protected method called ReadDirectoryNames (the one that needs a modification), i copy it from unit FileCtrl and then i edit that copy on my own unit to fix the problem (not showing Hidden folders, neither System folders); the trick is to edit the call to FindFirst by adding after faDirectory the part or faHidden or faSysFile, i also change SlashSep to IncludeTrailingPathDelimiter (avoid some extra references, etc) and also do a reformat (indexing, etc) so i can see that method is the one i had modified.
Then i follow things missing... like BuildList, that one i just simply copy it from unit FileCtrl without any modification (if not copied the hack does not work, since the call to ReadDirectoryNames is inside BuildList).
Then i copy the declaration of FPreserveCase and FCaseSensitive and their property declarations (they are used inside BuildList method).
That is it, now the modified TDirectoryListBox will see hidden and system folders
Hope this helps others, this way you can have both TDirectoryListBox (original one and modified one) at same time (but not on same form, sorry) on your project, without modifing VCL at all.
P.D.: Someone with extra knowledge maybe is able to add properties to configure if it must show or not hidden and/or system folders as an improvement, it must not be very difficoult, two private boolean variables and their corresponding property declaration with read and write methods... i did not do it since i would like to add not only such two, also SymLinks, etc (search for faSymLink on unit SysUtils and see how many there are, a lot of work to add them all), sorry for any inconvenience for that.
I posted this solution for a different question a short while ago:
Delphi: Copy Files from folder with Overall progress. CopyFileEx?

Resources