Object Pascal: TClientDataset Deletions - delphi

Question

I'm creating an in-memory dataset using a TClientDataset to use as a receive buffer. Adding data is great, but once I go to process it I want to be able to remove the row from the dataset. Calling delete works - sort of - the row/index is still accessible but now contains no valid information.
This makes things a bit difficult since when I'm processing this buffer it's not guaranteed that entries will be deleted in fact. I'd rather not start scanning the buffer from the first entry and skipping empty items, so is there a better way to permanently "remove" the item from the dataset? My idea was that it should work something like an actual SQL table where deleting a row doesn't leave empty records.
What's the best way to achieve this, or am I using the wrong component entirely?

Answer 1

By default client datasets mantain a "log" of changes because they are also designed to be able to send client side changes to a remote server, even if they were made in a disconnected session ("briefcase model"). Usually this log is "cleared" when you apply the changes to the remote db, and any other changes is merged with your "local" copy.
Set LogChanges to False if you don't need it and wish that changes are made directly.

Answer 2

There's something wrong with your code. I prepared a test application for this case, because I will face TClientDataSet in Multithreading environment in a few days. My test case application is not presenting this problem (Delphi 2010 Update 5)
I'll publish this code also in my own blog in a couple of days... for now I gave it to you now:
DFM file:
object Form2: TForm2
Left = 0
Top = 0
Caption = 'Form2'
ClientHeight = 337
ClientWidth = 635
Color = clBtnFace
Font.Charset = DEFAULT_CHARSET
Font.Color = clWindowText
Font.Height = -11
Font.Name = 'Tahoma'
Font.Style = []
OldCreateOrder = False
OnClose = FormClose
PixelsPerInch = 96
TextHeight = 13
object Memo1: TMemo
Left = 8
Top = 8
Width = 257
Height = 321
Lines.Strings = (
'Memo1')
TabOrder = 0
end
object Button1: TButton
Left = 271
Top = 8
Width = 170
Height = 25
Caption = 'Start'
TabOrder = 1
OnClick = Button1Click
end
object cdsTest: TClientDataSet
Aggregates = <>
Params = <>
Left = 584
Top = 32
object cdsTestNumber: TIntegerField
FieldName = 'Number'
end
end
object tToMemo: TTimer
Enabled = False
Interval = 500
OnTimer = tToMemoTimer
Left = 376
Top = 144
end
end
pas file:
unit Unit2;
interface
uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, StdCtrls, DB, DBClient, SyncObjs, ExtCtrls;
type
TWriterThread = class(TThread)
private
FDataSet: TClientDataSet;
//FWriteLock: TMultiReadExclusiveWriteSynchronizer;
FLock: TCriticalSection;
public
constructor Create(ADataSet: TClientDataSet; ALock: TCriticalSection);
procedure Execute; override;
end;
TDeleterThread = class(TThread)
private
FDataSet: TClientDataSet;
//FWriteLock: TMultiReadExclusiveWriteSynchronizer;
FLock: TCriticalSection;
public
constructor Create(ADataSet: TClientDataSet; ALock: TCriticalSection);
procedure Execute; override;
end;
TForm2 = class(TForm)
cdsTest: TClientDataSet;
Memo1: TMemo;
cdsTestNumber: TIntegerField;
Button1: TButton;
tToMemo: TTimer;
procedure Button1Click(Sender: TObject);
procedure tToMemoTimer(Sender: TObject);
procedure FormClose(Sender: TObject; var Action: TCloseAction);
private
{ Private declarations }
FLock: TCriticalSection;
FWriterThread: TWriterThread;
FDeleterThread: TDeleterThread;
procedure cdsToMemo;
public
{ Public declarations }
end;
var
Form2: TForm2;
implementation
{$R *.dfm}
procedure TForm2.Button1Click(Sender: TObject);
begin
Button1.Enabled := False;
cdsTest.CreateDataSet;
cdsTest.LogChanges := False;
FLock := TCriticalSection.Create;
tToMemo.Enabled := True;
FWriterThread := TWriterThread.Create(cdsTest, FLock);
FDeleterThread := TDeleterThread.Create(cdsTest, FLock);
end;
{ TWriterThread }
constructor TWriterThread.Create(ADataSet: TClientDataSet;
ALock: TCriticalSection);
begin
inherited Create(False);
FDataSet := ADataSet;
FLock := ALock;
end;
procedure TWriterThread.Execute;
var
I: Integer;
begin
inherited;
I := 0;
while not Terminated do
begin
FLock.Enter;
try
Inc(I);
FDataSet.AppendRecord([I]);
finally
FLock.Leave;
end;
Sleep(500); //a new record aproximately each half second
end;
end;
{ TDeleterThread }
constructor TDeleterThread.Create(ADataSet: TClientDataSet;
ALock: TCriticalSection);
begin
inherited Create(False);
FDataSet := ADataSet;
FLock := ALock;
end;
procedure TDeleterThread.Execute;
const
MaxRecords = 100;
var
ProcessedRecords: Integer;
begin
inherited;
while not Terminated do
begin
Sleep(3000); //delete records aproximately every 3 seconds
FLock.Enter;
try
FDataSet.First;
ProcessedRecords := 0;
while (not FDataSet.Eof) and (ProcessedRecords < MaxRecords) do
begin
Inc(ProcessedRecords);
if Odd(FDataSet.Fields[0].AsInteger) then
FDataSet.Delete
else
FDataSet.Next;
end;
finally
FLock.Leave;
end;
end;
end;
procedure TForm2.cdsToMemo;
begin
FLock.Enter;
try
Memo1.Lines.BeginUpdate;
try
Memo1.Lines.Clear;
cdsTest.First;
while not cdsTest.Eof do
begin
Memo1.Lines.Add(cdsTestNumber.AsString);
cdsTest.Next;
end;
finally
Memo1.Lines.EndUpdate;
end;
finally
FLock.Leave;
end;
end;
procedure TForm2.FormClose(Sender: TObject; var Action: TCloseAction);
begin
tToMemo.Enabled := False;
if cdsTest.Active then
begin
FDeleterThread.Terminate;
FDeleterThread.WaitFor;
FWriterThread.Terminate;
FWriterThread.WaitFor;
end;
end;
procedure TForm2.tToMemoTimer(Sender: TObject);
begin
tToMemo.Enabled := False;
cdsToMemo;
tToMemo.Enabled := True;
end;
end.
I'll no post further explanation, because you seems well versed in multi-threading. If you have any doubt, feel free to comment with questions.
Only one thing... I was planning to use TMultiReadExclusiveWriteSynchronizer to allow better concurrence, but I have no experience in promoting ReadAccess to WriteAccess, so I used a CriticalSection to avoid the time needed to investigate right now.

Answer 3

A couple of remarks regarding your code.
You are using an unusual way to loop through your dataset (using a counter and still using next).
My preferred direction when deleting would be from end to beginning.
You do not post your dataset after delete.
My suggestion would be to try something like this:
MyDataSet.RecNo:= 99
while not MyDataSet.Bof do
begin
fD1 := MyDataset.FieldByName('Field1').AsInteger;
fD2 := MyDataset.FieldByName('Field2').AsInteger;
fD3 := MyDataset.FieldByName('Field3').AsInteger;
if someCondition then
MyDataset.Delete;
MyDataSet.Post;
MyDataset.Previous;
end;