I have a TFDMemTable filled with thousands of records. Is there a way to limit result records for only the first 50 ?
I've tried to use:
FDMemTable.FetchOptions.RecsSkip := 0;
FDMemTable.FetchOptions.RecsMax := 50;
FDMemTable.Open;
But it did not work, data remained unchanged.
I expect #Victoria will be able to show you a better and more general
way, but there are at least two ways to do this:
Use FD's FDLocalSQL feature to copy the first X rows of the FDMemTable into, say,
an FDQuery and then copy them back into your FDMemTable.
Apply a filter to the FDMemTable to filter out the other records, use an FDBatchMove
to copy the X records into a second FDMemTable and then copy them back into the original
FDMemTable.
To implement the first of these, add the following components to your form/datamodule:
FDLocalSQL1: TFDLocalSQL;
FDConnection1: TFDConnection;
FDQuery1: TFDQuery;
FDPhysSQLiteDriverLink1: TFDPhysSQLiteDriverLink;
and then execute code like this:
procedure TForm3.CopyData1;
begin
FDConnection1.DriverName := 'SQLite';
FDConnection1.Connected := True;
FDLocalSQL1.Connection := FDConnection1;
FDLocalSQL1.DataSets.Add(FDMemTable1); // this is the source dataset
FDLocalSQL1.Active := True;
FDQuery1.SQL.Text := 'select * from FDMemTable1 order by ID limit 5'; // ID being an Integer field of the FDMemTable
FDQuery1.Active := True;
FDMemTable1.Close;
FDMemTable1.Data := FDQuery1.Data; // Re-opens FDMemTable 1, which now contains only the first X records
end;
FD's LocalSQL uses Sqlite to do its stuff. The functional equivalent in Sqlite's SQL
to "Select Top X ..." is its limit clause.
An advantage of using LocalSQL for your task, of course, is that because LocalSQL
supports order by, you can it to determine which (top) X records are retained.
The batchmove method requires a bit less code but requires you to have a way of identifying
the first X records using a filter expression. An example using an ID field might be
procedure TForm3.CopyData2;
begin
FDMemTable1.Filter := 'ID <=50';
FDMemTable1.Filtered := True;
FDBatchMove1.Execute; // move data from FDMemTable1 to FDMemTable2;
FDMemTable1.Close;
FDMemTable1.Data := FDMemTable2.Data; // Re-opens FDMemTable 1, which now contains only the first X records
end;
Btw, you say
I have a TFDMemTable filled with thousands of records. I
I think the problem with the method you've tried is probably that by the time you have the records in the FDMemTable, it's too late to try and limit the number of them in the way you're attempting. *)
Related
When using the 'while not TADOQuery.Eof' with an microsoft Excel Workbook, it's including rows which are completely empty. Is there a way to stop including any rows that are completely blank as I don't need them?
You could exclude blank lines in the SQL used to open the spreadsheet. If the first row contains column headings like 'Column1', 'Column2', etc then the following SQL will not return rows where the value in the first column is blank
select * from [sheet1$]
where Column1 <> ''
Obviously the SQL could be a bit more specific (in terms of column values) about what you regard as constituting a blank row.
You'll have gathered that there are various ways to deal with variations in the contents of the column headers, but as the other answer shows, these are likely to be far more verbose than simply skipping blank rows inside the body of your main while not EOF loop to read the table contents, so I can't really see any benefit to not doing it by just skipping the blank rows.
Btw, ime the Excel data accessible via SQL behaves as though the query is automatically restricted to the UsedRange range in the Excel COM interface.
Original answer:
If I understand you correctly and you want to exclude empty rows after the query is opened, then next approach may help (but I think, that you should exclude these rows with SQL statement, as in #MartynA's answer). Here, empty rows are all rows, which have Null value for all fields.
procedure TForm1.btnDataClick(Sender: TObject);
var
i: Integer;
empty: Boolean;
begin
qry.First;
while not qry.Eof do begin
// Check for empty row. Row is empty if all fields have NUull value.
empty := True;
for i := 0 to qry.FieldCount - 1 do begin
if not qry.Fields[i].IsNull then begin
empty := False;
Break;
end{if};
end{for};
// Read record data if record is not empty
if not empty then begin
// Your code here ...
end{if};
// Next record
qry.Next;
end{while};
end;
Update:
It's an attempt to improve my answer. If the table structure is not known, you can query the table with always false WHERE clause to get this structure and generate an SQL statement dynamically:
procedure TForm1.btnDataClick(Sender: TObject);
var
i: Integer;
where: string;
begin
// Get column names
qry.Close;
qry.SQL.Clear;
qry.SQL('SELECT * FROM [SheetName] WHERE 1 = 0');
try
qry.Open;
except
ShowMessage('Error');
end{try};
where := '';
for i := 0 to qry.FieldCount - 1 do begin
where := where + '(' + qry.Fields[i].FieldName + ' <> '''') AND ';
end{for};
where := 'WHERE ' + Copy(where, 1, Length(where) - 5);
// Read data without "empty" rows
qry.Close;
qry.SQL.Clear;
qry.SQL('SELECT * FROM [SheetName] ' + where);
try
qry.Open;
except
ShowMessage('Error');
end{try};
end;
I managed to make a "Search" bar through a TEdit that seeks whatever i type from inside
a ListView that gets its information from a DataBase and goes through a filter and updates the ListView's items on the fly after a key is pressed.
Now i am trying to learn how to implement a way of limiting the results i get in my ListView temporarily until i press a Show More button or something like that in order to get some more relevant results.
Since the Database might return over 500 results by the time i press "A" and that would be harsh to a mobile phone's capabilities so i need that feature to make my Search button more efficient.
Could someone give me some pointers on what i can use in order to make something like that?
EDIT.
The current code i am using for searching in the ListView is this...
procedure TContactsForm.Edit1ChangeTracking(Sender: TObject);
var
Lower: string;
i: integer;
begin
Lower:= LowerCase(Edit1.Text.Trim);
if Lower= '' then
begin
if Filtered then
begin
ListView1.Items.Filter := nil;
ListView1.ItemIndex := BindSourceDB1.ComponentIndex;
end;
end
else
begin
ListView1.ItemIndex := -1;
ListView1.Items.Filter :=
function(X: string): Boolean
begin
Result:= (Lower = EmptyStr) or LowerCase(X).Contains(Lower);
end;
end;
end;
function TContactsForm.Filtered: Boolean;
begin
Result := Assigned(ListView1.Items.Filter);
end;
The easiest way is to model your select statement so it only returns a limited about of rows (you can always remove the limitation upon user request).
For SQLite, MySQL and PostgreSQL you'd use a LIMIT clause:
SELECT acolumn FROM atable WHERE afield LIKE :param LIMIT 4;
In SQL Server you'd have to do something like:
SELECT * FROM (
SELECT column, ROW_NUMBER() OVER (ORDER BY name) as row FROM atable
) a WHERE a.row <= 4
This has the added benefit that less data is generated and transmitted by the database.
When doing the full search you simple omit the limit clause.
If you want to keep the results you already have and just add to extra results, use a
LIMIT 20 OFFSET 5 clause (without the offset keyword the operands are reversed LIMIT 5,20).
You always want to limit so as to make the experience snappy.
Every new page, you fetch the next x records.
You can even do this in real time, as the user is scrolling the list down.
Fetch new records as he nears the bottom of the list.
Is it possible with a Delphi TAdoQuery to execute multiple inserts in one go, or do you have to execute each statement separately? What I want to do is this:
AdoQuery.SQL.Clear;
AdoQuery.SQL.Add('INSERT INTO user VALUES (1, "User 1");');
AdoQuery.SQL.Add('INSERT INTO user VALUES (2, "User 2");');
AdoQuery.SQL.Add('INSERT INTO user VALUES (3, "User 3");');
AdoQuery.ExecSQL;
AdoQuery.Close;
Is this possible? I'm getting an error from MySQL when executing this. I also tried adding BEGIN; and END; around the queries, but that didn't work either.
Edit: I want to do this because when I execute the inserts in a for loop it seems like it takes a really long time for > 10 queries. I'm assuming adding them all like above would speed things up. Does anyone know if the AdoQuery.Close call is necessary between inserts?
Try something like this (with an AdoCommand):
sSql := 'INSERT INTO User (FieldName1, FieldName2) values (:Nr, :Strng)';
AdoCmd.Parameters.Clear();
AdoCmd.CommandText := sSql;
AdoCmd.CommandType := cmdText;
AdoCmd.Parameters.ParseSQL( sSql, True );
AdoCmd.Parameters.ParamByName('Nr').DataType := ftInteger
AdoCmd.Parameters.ParamByName('Strng').DataType := ftString;
for i := 1 to 10 do
begin
AdoCmd.Parameters.ParamByName('Nr').Value := i;
AdoCmd.Parameters.ParamByName('Strng').Value := sUserName(i);
AdoCmd.Execute;
end;
You could speed up thing by using .Params(0) and .Params(1) because ParamByName takes up some time.
But the trick here is the ParseSql statement. It keeps your code clear but still only parses the sql string only once.
And you can use transactions if necessary ... by using AdoCmd.Connection.BeginTrans and AdoCmd.Connection.CommitTrans / RollbackTrans.
With MySQL you can use the syntax:
INSERT INTO user VALUES (1, "User 1"), (2, "User 2"), (3, "User 3")
Then you may use parameters:
AdoQuery.SQL.Text := 'INSERT INTO user VALUES (:p11, :p12), (:p21, :p22), (:p31, :p32)';
AdoQuery.Parameters[0].Value := 1;
AdoQuery.Parameters[1].Value := 'User 1';
AdoQuery.Parameters[2].Value := 2;
AdoQuery.Parameters[3].Value := 'User 2';
AdoQuery.Parameters[4].Value := 3;
AdoQuery.Parameters[5].Value := 'User 3';
AdoQuery.ExecSQL;
Proper use of transactions will also speed up your inserts. If each statement needs to be committed, it will take longer to be executed. If you can execute everything within a single transaction and just commit at the end it will be faster.
Don't know MySQL, but some databases also support "array DML", where a single SQL statement is sent to the DB together array of parameters and thereby execute multiple times but with a single communication roundtrip.
Well, this seems a little tricky (if not imposible). I'm trying to make my DBGrid sort its data by clicking on column's title.
The thing is that I'm (sadly) working with Delphi 3, I'm not using ADO DataSets and the query gets a lot of rows, thus I can't reopen my TQuery changing the order by clause on clicks.
Someone has implemented something like this?
This is actually done by sorting the dataset, and then the grid reflects the change. It can be done easily enough by creating an index on the dataset field for that column. Of course, this can only be done on a dataset that supports index sorting, such as TClientDataset.
On the TDBGrid's OnTitleClick method you can do something like...
procedure TfmForm1.DBGrid1TitleClick(Column: TColumn);
var
i: Integer;
begin
// apply grid formatting changes here e.g. title styling
with DBGrid1 do
for i := 0 to Columns.Count - 1 do
Columns[i].Title.Font.Style := Columns[i].Title.Font.Style - [fsBold];
Column.Title.Font.Style := Column.Title.Font.Style + [fsBold];
with nxQuery1 do // the DBGrid's query component (a [NexusDB] TnxQuery)
begin
DisableControls;
if Active then Close;
for i := 0 to SQL.Count - 1 do
if (Pos('order by', LowerCase(SQL[i])) > 0) then
//NOTE: ' desc' The [space] is important
if (Pos(' desc',LowerCase(SQL[i])) > 0) then
SQL[i] := newOrderBySQL
else
SQL[i] := newOrderBySQL +' desc';
// re-add params here if necessary
if not Active then Open;
EnableControls;
end;
end;
There are plenty of ways in which you could optimise this I'm sure however it depends on the capabilities of the components you use. The example above uses a query component though if you used a table component you'd change the index used instead of the 'order by' clause.
The handling of the SQL here is a very basic version. It does not handle things like SQL batch statements, resulting in possible multiple 'order by..' clauses or commented SQL statements i.e. ignoring bracketed comments "{..}" or single line comments "//"
Regards
Delphi 3 have TClientDataset. And TQuery can use explicitly created indexes on the database to order data on the IndexName property.
Here are some examples of how to do this: Sorting records in Delphi DBGrid by Clicking on Column Title .
As already mentioned, sorting is quite easy if you are using a TClientDataSet (cds.IndexFieldNames := Column.FieldName in the OnTitleClick of the TDBGrid). However if you are not able to do this you can either regenerate your query (which you have stated you don't want to do) or obtain a more advanced data grid such as Express Quantum Grid (which I think allows you to sort).
On the TDBGrid's OnTitleClick method you can write this simple code:
procedure TForm1.DBGrid3TitleClick(Column: TColumn);
var
cFieldName:string;
begin
cFieldName:= DBGrid3.SelectedField.FieldName;
AdoDataset1.Sort:=cFieldName;
end;
example: (https://www.thoughtco.com/sort-records-in-delphi-dbgrid-4077301)
procedure TForm1.DBGrid1MouseMove(Sender: TObject; Shift: TShiftState; X,
Y: Integer);
var
pt: TGridcoord;
begin
pt:= DBGrid1.MouseCoord(x, y);
if pt.y=0 then
DBGrid1.Cursor:=crHandPoint
else
DBGrid1.Cursor:=crDefault;
end;
procedure TForm1.DBGrid1TitleClick(Column: TColumn);
var
cFieldName:string;
begin
Adotable1.Sort := Column.Field.FieldName;
end;
If you are using a combination of TFDQuery, TDataSource and TDBGrid you can order with this easy way!
procedure TFrmGer.DBGridTitleClick(Column: TColumn);
begin
OrderByTitle(MyFDQuery, Column);
end;
Put it in a helper file so you can use it latter again.
procedure OrderByTitle(AQuery: TFDQuery; Column: TColumn);
begin
AQuery.IndexFieldNames := Column.DisplayName;
end;
Ascending and Descending Mode
if Pos('DESC',PChar(Q2.Sort))>0 then
Q2.Sort:=Column.FieldName + ' ASC'
else
Q2.Sort:=Column.FieldName + ' DESC';
Since Embarcadero's NNTP server stopped responding since yesterday, I figured I could ask here: I work with a non-DB-aware grid, and I need to loop through a dataset to extract the number of columns, their name, the number of rows and the value of each fields in each row.
I know to read the values for all the fields in each row, but I don't know how to extract column-related information. Does someone have some code handy?
procedure TForm1.FormCreate(Sender: TObject);
var
index : Integer;
begin
With ASQLite3DB1 do begin
DefaultDir := ExtractFileDir(Application.ExeName);
Database := 'test.sqlite';
CharacterEncoding := 'STANDARD';
Open;
end;
With ASQLite3Query1 do begin
ASQLite3Query1.Connection := ASQLite3DB1;
SQL.Text := 'CREATE TABLE IF NOT EXISTS mytable (id INTEGER PRIMARY KEY, label VARCHAR)';
ExecSQL;
SQL.Text := 'INSERT INTO mytable (label) VALUES ("dummy label")';
ExecSQL;
SQL.Text := 'SELECT id AS Identification, label AS Label FROM mytable';
Open;
//How to get column numbers + names to initialized grid object?
for index := 0 to ASQLite3Query1. - 1 do begin
end;
for index := 0 to FieldCount - 1 do begin
ShowMessage(Fields[index].AsString);
end;
end;
end;
Thank you.
Number of fields and their names could be acquired as follows:
procedure TForm1.Button1Click(Sender: TObject);
begin
with Query1 do
begin
ShowMessage(IntToStr(FieldCount));
ShowMessage(Fields[0].FieldName);
end;
end;
You can checkout TFieldDef for more detail info about the field.
dataset.FieldDefs[0] has properties like DataType and Size.
If what you're looking for is a list of field names, try creating a TStringList and passing it to the TDataset.Fields.GetFieldNames procedure.
If you want more information about fields, the TFields object (ASQLite3Query1.Fields) has a default property and a Count property, so you can use it like an array, and an enumerator, both of which can be used to loop over each TField object and retrieve its metadata.