Delphi does not seem to like multi-field indexes.
How do I physically sort a a table so that I wind up with a table that has the rows in the desired order?
Example:
mytable.dbf
Field Field-Name Field-Type Size
0 Payer Character 35
1 Payee Character 35
2 PayDate Date
3 Amount Currency
I need to produce a table sorted alphabetically by "Payee"+"Payer"
When I tried using an index of "Payee+Payer", I got an error:
"Field Index out of range"
The index field names need to be separated by semicolons, not plus symbols. Try that and it should work.
Ok, let's try to put some order.
First, isn't advisable to physically sort a table. In fact the most RDBMS even don't provide you this feature. Usually, one, in order to not force a full table scan (it is called sometimes natural scan) creates indexes on the table fields on which he thinks that the table will be sorted / searched.
As you see, the first step in order to sort a table is usually index creation. This is a separate step, it is done once, usually at, let's say, "design time". After this, the DB engine will take care to automatically update the indexes.
The index creation is done by you (the developer) using (usually) not Delphi (or any other development tool) but the admin tool of your RDBMS (the same tool which you used when you created your table).
If your 'DB engine' is, in fact, a Delphi memory dataset (TClientDataSet) then you will go to IndexDefs property, open it, add a new index and set the properties there accordingly. The interesting property in our discussion is Fields. Set it to Payee;Payer. Set also the Name to eg. "idxPayee". If you use other TDataSet descendant, consult the docs of your DB engine or ask another question here on SO.com providing the details.
Now, to use the index. (IOW, to sort the table, as you say). In your program (either at design time either at run time) set in your 'Table' the IndexName to "idxPayee" or any other valid name you gave or set IndexFieldNames to Payee;Payer.
Note once again that the above is an example based on TClientDataSet. What you must retain from the above (if you don't use it) is that you must have an already created index in order to use it.
Also, to answer at your question, yes, there are some 'table' types (TDataSet descendants in Delphi terminology) which support sorting, either via a Sort method (or the like) either via a SortFields property.
But nowadays usually when one works with a SQL backend, the preferred solution is to create the indexes using the corresponding admin tool and then issue (using Delphi) an SELECT * FROM myTable ORDER BY Field1.
HTH
If you're still using BDE you can use the BDE API to physically sort the DBF table:
uses
DbiProcs, DbiTypes, DBIErrs;
procedure SortTable(Table: TTable; const FieldNums: array of Word; CaseInsensitive: Boolean = False; Descending: Boolean = False);
var
DBHandle: hDBIDb;
RecordCount: Integer;
Order: SORTOrder;
begin
if Length(FieldNums) = 0 then
Exit;
Table.Open;
RecordCount := Table.RecordCount;
if RecordCount = 0 then
Exit;
DBHandle := Table.DBHandle;
Table.Close;
if Descending then
Order := sortDESCEND
else
Order := sortASCEND;
Check(DbiSortTable(DBHandle, PAnsiChar(Table.TableName), nil, nil, nil, nil, nil,
Length(FieldNums), #FieldNums[0], #CaseInsensitive, #Order, nil, False, nil, RecordCount));
end;
for example, in your case:
SortTable(Table1, [2, 1]); // sort by Payee, Payer
Cannot check, but try IndexFieldNames = "Payee, Payer".
Sure indexes by these 2 fields should exist.
You can create an index on your table using the TTable.AddIndex method in one call. That will sort your data when you read it, that is if you use the new index by setting the TTable.IndexName property to the new index. Here's an example:
xTable.AddIndex('NewIndex','Field1;Field2',[ixCaseInsensitive]);
xTable.IndexName := 'NewIndex';
// Read the table from top to bottom
xTable.First;
while not xTable.EOF do begin
..
xTable.Next;
end;
Related
Hi I want to post calc field(cemi) to table (sql). when I calc all field the last field doesn't post on sql table. because last field (cemi) type fkcalc how can I post fkcalc type field to sql table Thanks in advance!
procedure TForm1.ADOQuery1CalcFields(DataSet: TDataSet);
begin
ADOQuery1.FieldValues['cemi']:=
((ADOQuery1.FieldValues['boyuk1'] + ADOQuery1.FieldValues['boyuk2'] +
ADOQuery1.FieldValues['boyuk3'])*0.35)+((ADOQuery1.FieldValues['kicik1'] +
ADOQuery1.FieldValues['kicik2'])*0.25) +(ADOQuery1.FieldValues['qara1']*0.30);
end;
I'm not quite sure what you mean by
the last field doesn't post on sql table
If the "last field" you are referring to is your "Cemi" one and that is a column which is in the table on your SQL Server, it will not get posted back there if you have defined it as a calculated field in your AdoQuery1 in the Object Inspector. Fields with a FieldKind of fkCalculated are local to the AdoQuery.
Just assigning a value to the calculated field is sufficient to "post" it locally to the AdoQuery, as I imagine you know. What you want to do to debug your problem (because readers like me cannot debug it fr you) is to more easily see what value, if any, is being assigned to it.
From that point of view, your code is suffering from "premature optimisation" which will make it difficult for you to see what is going wrong. Try this instead:
In your ADOQuery1CalcFields, declare a local variable for each of the fields you are accessing, including the calculated one. Choose the variable types to suit the fields:
var
Boyuk1 : Double; // or Integer, etc
[...]
Cemi : Double;
Assign values to the local variables, using the AsXXXX (type) of the fields:
Cemi := 0;
if not AdoQuery1.FieldByName('Boyuk1').IsNull then
Cemi := Cemi + AdoQuery1.FieldByName('Boyuk1').AsFloat;
[etc]
That way, at least you'll be able to see the point at which the calculation goes wrong (if it does).
I've used FieldByName().AsFloat, rather than FieldValues[], because FieldValues[] is a Variant, which can be Null, and you don't want that when you are assigning values to it which mat themselves be variants.
Also
Check that AutoCalcFields is set to True for AdoQuery1.
Put a debugger breakpoint on the first line of ADOQuery1CalcFields. Compile and run and check that the breakpoint hits - if it doesn't, there's your answer. Single-step the debugger through each line of the procedure, and, after the final line, use Ctrl-F7 to evaluate the value of AdoQuery1.FieldByName('Cemi').AsFloat.
I have a table called Artist which currently contains four records and a TSQLQuery that contains the following statement:
SELECT name
FROM Artist
The table Artist also contains the following records:
id name
1 Artist 1
2 Artist 2
3 Artist 3
4 Artist 4
Current method of record retrieval:
Query1.Open;
for i := 0 to qArtist.FieldCount -1 do
with cbArtist.ListBox.ListItems[i] do
Text := qArtist.Fields[i].AsString;
Previous method of record retrieval:
Data bind Query1 to ComboBox1.
With the "previous" method, ComboBox1 would display all the expected records from the Artist table. However, when I try to use "current" method Query1 is only selecting the very first record from the Artist table despite there being three other existing values. I have tried the "current" method across other queries and they also returned only the first value of the table.
The reason I am going for this new approach is because I feel that I am very limited in what I can do if I continue to the "previous" / data bind method, but that is besides the point.
So how can I fix this problem? i.e. the problem of the query only selecting the very first record from the table.
You must use the Eof and Next methods to iterate over the records.
Query1.Open;
while not Query1.eof do
begin
cbArtist.Items.Add(Query1.FieldByName('Artist').AsString);
Query1.Next;
end;
You code show an interaction over fields, if you need iterate all record then you must use a code like:
Query1.Open;
Query1.first;
while not Query1.eof do
begin
with cbArtist.ListBox.ListItems[i] do
Text := qArtist.Fields[1].AsString; //put here field you want to bind on ListBox.
Query1.next;
end;
I don't think you are navigating your query's dataset correctly. The FieldCount and Fields[i] access the field metadata (columns going across), not the rows. I beleive in Delphi you use While not Eof begin... end.
Navigating Datasets
I would consider altering the data binding fields to suit your needs. Delphi's Databinding is very powerful. Manually iterating the dataset just to populate a control will just be extra code where bugs can hide. Utilize the built-in capabilities of the tools and it will be easier to understand and maintain.
Does anybody know (or care to make a suppostion as to) why TSqlDataset has a commandtext property (string) whereas TSqlQuery has a sql property (tstrings)?
Consider the sql statement
select id, name from
table
order by name
If I use a TSqlQuery, then I can change the table name in the query dynamically by accessing sql[1], but if I am using a TSqlDataset (as I have to if I need a bidrectional dataset, the dataset is connected to a provider and thence to a tclientdataset), I have to set the commandtext string literally. Whilst the above example is trivial, it can be a problem when the sql statement is much more involved.
Update:
Judging by the comments and answers so far, it seems that I was misunderstood. I don't care very much for improving the runtime performance of the components (what does one millisecond matter when the query takes one second) but I do care about the programmer (ie me) and the ability to maintain the program. In real life, I have the following query which is stored in a TSqlQuery:
select dockets.id, dockets.opendate, customers.name, statuses.statname,
dockets.totalcost, dockets.whopays, dockets.expected, dockets.urgent,
(dockets.totalcost - dockets.billed) as openbill,
(dockets.totalcost - dockets.paid) as opencost,
location.name as locname, dockets.attention,
statuses.colour, statuses.disporder, statuses.future, dockets.urgcomment
from location, statuses, dockets left join customers
on dockets.customer = customers.id
where dockets.location = location.id
and dockets.status = statuses.id
I haven't counted the number of characters in the string, but I'm sure that there are more than 255, thus precluding storing the query in a simple string. In certain circumstances, I want to filter the amount of data being displayed by adding the line 'and statuses.id = 3' or 'and customers.id = 249'. If the query were stored as TStrings, then I could add to the basic query the dummy line 'and 1 = 1', and then update this line as needed. But the query is one long string and I can't easily access the end of it.
What I am currently doing (in lieu of a better solution) is creating another TSqlDataSet, and setting its commandtext to the default TSqlDataSet's commandtext whilst appending the extra condition.
1) TSQLQuery is rather for compatibility with BDE TQuery. And BDE TQuery has SQL: TStrings property. TSQLDataSet is what supposed to be used for new applications.
2) Although SQL: TStrings is usefull for some tasks, it is also error prone. Often programmers forget to clear SQL property before filling again. Also if your query is a big one, the filling of SQL may lead to performance degradation. Because on each SQL.Add(...) call dbExpress code parses query when ParamCheck is True. That may be solved by using BeginUpdate / EndUpdate or setting ParamCheck to False. But note, setting ParamCheck to False stops automatic parameters creation.
SQLQuery1.SQL.BeginUpdate;
try
SQLQuery1.SQL.Clear;
SQLQuery1.SQL.Add('SELECT * FROM');
SQLQuery1.SQL.Add('Orders');
finally
SQLQuery1.SQL.EndUpdate;
end;
CommandText does not have such issues.
3) You can use Format function for building a dynamic SQL string:
var
sTableName: String;
...
sTableName := 'Orders';
SQLDataSet1.CommandText := Format('select * from %s', [sTableName]);
4) Other data access libraries, like AnyDAC, have macro variables, simplifying dynamic query text building. For example:
ADQuery1.SQL.Text := 'SELECT * FROM &TabName';
ADQuery1.Macros[0].AsRaw := 'Orders';
ADQuery1.Open;
I would have to say that the TSqlQuery uses TStrings (TWideStrings in Delphi 2010) because it is much more flexible.
Suppose your query was:
Select
Item1,
Item2,
Item3,
Item4
FROM MyTable
It's a lot easier to read
You can copy and paste into an external query tool and it stays formatted
It's easy to comment out sections
Select
Item1,
/*
Item2,
Item3,
*/
Item4
FROM MyTable
You can easily add items
Select
Item1,
Item2,
Item2a,
Item2b,
Item3,
Item3a,
Item3b,
Item4
FROM MyTable
Try doing that to a contiguous set of characters that goes on forever in one long line with no line breaks inside an edit window that is always to small for viewing that doesn't allow for wrapped text etc. etc. etc.
Just $0.02.
What would be the best way, in delphi, to create and store data which will often be searched on and modified?
Basically, I would like to write a function that searches an existing database for telephone numbers and keeps track of how many times each telephone number has been used, the first date used, and the latest date used. The database that is being searched is basically a log of orders placed, containing the telephone number that was used to place the order. It's not an SQL database or anything that can easily be queried for such things (it's an old btrieve database), so I need to create a way of gaining this information (to eventually output to a text file).
I am thinking of creating a record containing the phone number, the two dates, and the number of times used, and then adding a record to a dynamic array for each telephone number. I would then search the array, entry by entry, for each record in the database, to see if the phone number for the current record is already in the array. Then updating or creating a record as necessary.
This seems like it would work, but as there are tens of thousands of entries in the database, it may not be the best way, and a rather slow and inefficient way of doing things. Is there a better way, given the limited actions I can perform on the database?
Someone suggested that rather than using an array, use a MySQL table to keep track of the numbers, and then query each number for every database record. This seems like even more overhead though!
Thanks a lot for your time.
I would register the aggregates in a totally disconnected TClientDataset(cds), and updating the values as you get them from the looping. If the Btrieve could be sorted by telephone number, much better. Then use the data on the cds to generate the report.
(If you go this way, I suggest get Midas SpeedFix from the Andreas Hausladen' blog, along with the other finest stuff you can find there).
Ok, here is a double pass old-school method that works well and should scale well (I used this approach against a multi-million record database once, it took time but gave accurate results).
Download and install Turbo Power
SysTools -- the sort engine
works very well for this process.
create a sort, with a fixed record
of phone number, you will be using
this to sort.
Loop thru your records, at each order, add the
phone number to the sort.
Once the first iteration is done, start
popping the phone numbers from the
sort, increment a counter if the
phone number is the same as the last
one read, otherwise report the
number and clear your counter.
This process can also be done with any SQL Database, but my experience has been that the sort method is faster than managing a temporary table and generates the same results.
EDIT -- You stated that this is a BTrieve database, why not just create a key on the phone number, sort on that key, then apply step 4 over this table (next instead of pop). Either way you will need to touch every record in your database to get counts, the index/sort just makes your decision process easier.
For example, lets say that you have two tables, one the customer table is where the results will be stored, and the other the orders table. Sort both by the same phone number. Then start a cursor at the top of both lists and then apply the following psuedocode:
Count := 0;
While (CustomerTable <> eof) and (OrderTable <> eof) do
begin
comp = comparetext( customer.phone, order.phone );
while (comp = 0) and (not orderTable eof) do
begin
inc( Count );
order.next;
comp = comparetext( customer.phone, order.phone );
end;
if comp < 0 then
begin
Customer.TotalCount = count;
save customer;
count := 0;
Customer.next;
end
else if (Comp > 0) and (not OrderTable EOF) then
begin
Order.Next; // order no customer
end;
end;
// handle case where end of orders reached
if (OrdersTable EOF) and (not CustomersTable EOF) then
begin
Customer.TotalCount = count;
save customer;
end;
This code has the benefit of walking both lists once. There are no lookups necessary since both lists are sorted the same, they can be walked top to bottom taking action only when necessary. The only requirement is that both lists have something in common (in this example phone number) and both lists can be sorted.
I did not handle the case where there is an order and no customer. My assumption was that orders do not exist without customers and would be skipped for counting.
Sorry, couldn't edit my post (wasn't registered at the time). The data will be thrown away once all the records in the database have been iterated through. The function won't be called often. It's basically going to be used as a way of determining how often people have ordered over a period of time from records we already have, so really it's just needed to produce a one off list.
The data will be persistent for the duration of the creation of the list. That is, all telephone numbers will need to be present to be searched on until the very last database record is read.
If you were going to keep it in memory and don't want anything fancy, you'd be better off using a TStringList so you can use the Find function. Find uses Hoare's selection or Quick-select, an O(n) locator. For instance, define a type:
type
TPhoneData = class
private
fPhone:string;
fFirstCalledDate:TDateTime;
fLastCalledDate:TDateTime;
fCallCount:integer;
public
constructor Create(phone:string; firstDate, lastDate:TDateTime);
procedure updateCallData(date:TDateTime);
property phoneNumber:string read fPhone write fPhone;
property firstCalledDate:TDateTime read fFirstCalledDate write fFirstCalledDate;
property lastCalledDate:TDateTime read fLastCalledDate write fLastCalledDate;
property callCount:integer read fCallCount write fCallCount;
end;
{ TPhoneData }
constructor TPhoneData.Create(phone: string; firstDate, lastDate: TDateTime);
begin
fCallCount:=1;
fFirstCalledDate:=firstDate;
fLastCalledDate:=lastDate;
fPhone:=phone;
end;
procedure TPhoneData.updateCallData(date: TDateTime);
begin
inc(fCallCount);
if fFirstCalledDate<date then fFirstCalledDate:=date;
if date>fLastCalledDate then fLastCalledDate:=date;
end;
and then fill it, report on it:
procedure TForm1.btnSortExampleClick(Sender: TObject);
const phoneSeed:array[0..9] of string = ('111-111-1111','222-222-2222','333-333-3333','444-444-4444','555-555-5555','666-666-6666','777-777-7777','888-888-8888','999-999-9999','000-000-0000');
var TSL:TStringList;
TPD:TPhoneData;
i,index:integer;
phone:string;
begin
randseed;
TSL:=TStringList.Create;
TSL.Sorted:=true;
for i := 0 to 100 do
begin
phone:=phoneSeed[random(9)];
if TSL.Find(phone, index) then
TPhoneData(TSL.Objects[index]).updateCallData(now-random(100))
else
TSL.AddObject(phone,TPhoneData.Create(phone,now,now));
end;
for i := 0 to 9 do
begin
if TSL.Find(phoneSeed[i], index) then
begin
TPD:=TPhoneData(TSL.Objects[index]);
ShowMessage(Format('Phone # %s, first called %s, last called %s, num calls %d', [TPD.PhoneNumber, FormatDateTime('mm-dd-yyyy',TPD.firstCalledDate), FormatDateTime('mm-dd-yyyy',TPD.lastCalledDate), TPD.callCount]));
end;
end;
end;
Instead of a TStringList I would recommend using DeCAL's (on sf.net) DMap to store the items in memory. You could specify the phone is the key and store a Record/Class structure containing the rest of the record.
So your Record class will be:
TPhoneData = class
number: string;
access_count: integer;
added: TDateTime.
...
end;
Then in code:
procedure TSomeClass.RegisterPhone(number, phoneData);
begin
//FStore created in Constructor as FStore := DMap.Create;
FStore.putPair([number, phoneData])
end;
...
procedure TSoemClass.GetPhoneAndIncrement(number);
var
Iter: DIterator;
lPhoneData: TPhoneData;
begin
Iter := FStore.locate([number]);
if atEnd(Iter) then
raise Exception.CreateFmt('Number %s not found',[number])
else
begin
lPhoneData := GetObject(Iter) as TPhoneData;
lPhoneData.access_count = lPhoneData.access_count + 1;
//no need to save back to FStore as it holds a pointer to lPhoneData
end;
end;
DMap implements a red/black tree so the data structure sorts the keys for you for free. You can also use a DHashMap for the same affect and (arguably) increased speed.
DeCAL is one of my favourite data structure libraries and would recommend anybody doing in-memory storage operations to have a look.
Hope that helps
I am using JvMemoryData to populate a JvDBUltimGrid. I'm primarily using this JvMemoryData as a data structure, because I am not aware of anything else that meets my needs.
I'm not working with a lot of data, but I do need a way to enumerate the records I am adding to JvMemoryData. Has anyone done this before? Would it be possible to somehow "query" this data using TSQLQuery?
Or, is there a better way to do this? I'm a bit naive when it comes to data structures, so maybe someone can point me in the right direction. What I really need is like a Dictionary/Hash, that allows for 1 key, and many values. Like so:
KEY1: val1;val2;val3;val4;val5;etc...
KEY2: val1;val2;val3;val4;val5;etc...
I considered using THashedStringList in the IniFiles unit, but it still suffers from the same problem in that it allows only 1 key to be associated with a value.
One way would be to create a TStringList, and have each item's object point to another TList (or TStringList) which would contain all of your values. If the topmost string list is sorted, then retrieval is just a binary search away.
To add items to your topmost list, use something like the following (SList = TStringList):
Id := SList.AddObject( Key1, tStringList.Create );
InnerList := tStringList(SList.Objects[id]);
// for each child in list
InnerList.add( value );
When its time to dispose the list, make sure you free each of the inner lists also.
for i := 0 to SList.count-1 do
begin
if Assigned(SList.Objects[i]) then
SList.Objects[i].free;
SList.Objects[i] := nil;
end;
FreeAndNil(SList);
I'm not a Delphi programmer but couldn't you just use a list or array as the value for each hash entry? In Java terminology:
Map<String,List>
You already seem to be using Jedi. Jedi contains classes that allow you to map anything with anything.
Take a look at this related question.
I have been using an array of any arbitrarily complex user defined record types as a cache in conjunction with a TStringList or THashedStringList. I access each record using a key. First I check the string list for a match. If no match, then I get the record from the database and put it in the array. I put its array index into the string list. Using the records I am working with, this is what my code looks like:
function TEmployeeCache.Read(sCode: String): TEmployeeData;
var iRecNo: Integer;
oEmployee: TEmployee;
begin
iRecNo := CInt(CodeList.Values[sCode]);
if iRecNo = 0 then begin
iRecNo := FNextRec;
inc(FNextRec);
if FNextRec > High(Cache) then SetLength(Cache, FNextRec * 2);
oEmployee := TEmployee.Create;
oEmployee.Read(sCode);
Cache[iRecNo] := oEmployee.Data;
oEmployee.Free;
KeyList.Add(Format('%s=%s', [CStr(Cache[iRecNo].hKey), IntToStr(iRecNo)]));
CodeList.Add(Format('%s=%s', [sCode, IntToStr(iRecNo)]));
end;
Result := Cache[iRecNo];
end;
I have been getting seemingly instant access this way.
Jack