I am developing an application which has a simple database. All of the functions are going well but when a user is editing the database from the program, the other user cannot see the content immediately. The other user needs to close the program and reopen it for the data to appear and its DBGrid be updated with those changes form the other computers. I am using Delphi 7 for this and ZeosLib to access my Firebird database. I tried using the refresh button on the DBNavigator but it doesn't work.
The components I used to connect to the database are:
ZConnection
ZQuery
DataSource
DBGrid
DBNavigator
This is the code for my ZConnection and ZQuery.
object ZConnection1: TZConnection
ControlsCodePage = cGET_ACP
UTF8StringsAsWideField = False
Connected = True
Port = 3051
Database = '192.168.254.254:test'
User = 'test'
Password = 'test'
Protocol = 'firebird-2.5'
Left = 96
Top = 8
end
object ZQuery1: TZQuery
Connection = ZConnection1
Active = True
SQL.Strings = (
'select * from "test"')
Params = <>
Left = 128
Top = 8
object ZQuery1ID: TStringField
FieldName = 'ID'
Required = True
Size = 8
end
Sounds like you're running afoul of ACID. This is a basic guarantee of SQL-style databases, that all database updates will be Atomic, Consistent, Isolated, and Durable, and is accomplished through transactions.
Specifically, you're having trouble with the Consistency and the Isolation, which ensure that an external viewer never sees an update before it's finished, even if that update contains more than one change. (The classic example is a bank transfer, which requires subtracting money from one account and adding it to another. If you only see one of these two actions but not the other one, you have bad data.)
You can think of a transaction as an independent view of the state of the database. Every database connection has its own transaction, and any changes it makes are invisible to anyone else (Isolated) until they Commit (finalize) the transaction. Depending on the transactions' isolation settings, they may remain invisible to other users even after that, if they have an ongoing transaction, until they commit their transaction and begin a new one. It sounds like your code isn't taking this into account.
If you need updates to become visible immediately, you'll want to ensure that the transaction's isolation mode is READ COMMITTED, and set up database events to send notifications to connected clients when various things get updated, so the clients can perform refreshes of their data. You'll also want to ensure that the user updates result in a Commit action right away, so that the isolated data will become available.
Since I don't use ZeosLib, I can't explain all the details of how you'll need to set this all up, but this is enough to get you on the right track.
I suggest that you add a timer to the form which displays the grid. Set the timer so that it fires its OnTimer event once a minute (or longer). In this event, close the query then reopen it. This way, everyone always gets current information (albeit a minute late).
with qWhatever do // this is the query which is connected to the grid
try
disablecontrols;
close;
open
finally
enablecontrols
end;
For a multi-user application, where clients need to receive notifications, one option is to use Firebird events to send a 'broadcast' message for every data change (SQL INSERT, UPDATE or DELETE).
Clients can 'register' (listen) for a specific message type, and whenever the Firebird server sends a message with this type, they will receive it, and run client application code, which in your case would refresh the user interface (grid).
While this can be a sufficient solution in many simple use cases, there are also some restrictions. I recently blogged about this topic here:
Firebird Database Events and Message-oriented Middleware
(I am author of middleware libraries for Delphi and Free Pascal)
I solved this problem by adding this before query.
IBDatabase1.Close;
IBDatabase1.Open;
Related
Is it possible in FireDac get live data from database (SQL Server in my case) without updating DataSources like it was in Paradox. The closest things I could found was Live Data Window of FDTable, but information is very scanty and I don’t even sure that Live Data Window means that data is updated automatically. I need it for some obvious reasons. For example database sends some alert and user sees it without TTimer or constantly updating some specific DataSource.
I never used LiveDataWindows for any of my programs, but I Think, as you mentioned above, you should give FDTable a try.
Here is the Link on how to set it up correctly on MSSSL Server as well.
Embarcadero docwiki
You can build a little trial app using the code from docwiki:
uses
Windows;
...
// Set locale ID to German phone book collation
FDTable1.FormatOptions.SortLocale := MAKELCID(MAKELANGID (LANG_GERMAN, SUBLANG_GERMAN), SORT_DEFAULT);
// Use the the punctuation and other symbols case insensitive sorting
FDTable1.FormatOptions.SortOptions := [soNoSymbols];
FDTable1.IndexFieldNames := 'NAME';
FDTable1.TableName := 'CUSTOMERS';
FDTable1.Open;
i need to refresh data in a TFDQuery which is in cached updates.
to simplify my problem, let's suppose my MsACCESS database is composed of 2 tables that i have to join.
LABTEST(id_test, dat_test, id_client, sample_typ)
SAMPLEType(id, SampleName)
in the Delphi application, i am using TFDConnection and 1 TFDQuery (in cached updates) in which i join the 2 tables which script is:
"SELECT T.id_test, T.dat_test, T.id_client, T.sample_typ, S.SampleName
FROM LABTEST T
left JOIN SAMPLEType S ON T.sample_typ = S.id"
in my application, i also use a DBGrid to show the result of the query.
and a button to edit the field "sample_typ", like this:
qr.Edit;
qr.FieldByName('sample_typ').AsString:=ce2.text;
qr.Post;
the edition of the 'sample_typ' field works fine but the corresponding 'sampleName' field is not changing (in the grid) after an update.
in fact it is not refreshed !
the problem is here: if i do refresh of the query, an exception is raised: "cannot refresh dataset. cached updates must be commited or canceled
and batch mode terminated before refreshing"
if i commit the updates, data will be sent to database and i don't want that, i need to keep the data in cache till the end of the operation.
also if i get out of the cache, data will be refreshed in the grid but will be sent to the database after qr.post and i don't want that.
i need to refresh data in the cache. what is the solution ?
Thanks in advance.
The issue comes down to the fact that you haven't told your UI that there is any dependency on the two fields - it clearly can't know how to do the join itself without resubmitting it so if you don't want to send the updates and reload you will have a problem.
It's not clear exactly what you are trying to do, but these two ideas may help you.
If you are not going to edit the fields in the SAMPLEType tables (S) then load the values from that table into a lookup table. You can load this into a TFDMemTable. You can use an adapter which loads from a query. Your UI controls can then show the value based on the valus looked up in your local TFDMemTable. Dependiong on the UI control this might be a 'LookupField' or some such.
You may also be able to store your main data in a TFDMemTable with an Adapter - you can specify diferent TFDCommands to read the whole recordset, refresh a record, update, insert and delete a record. The TFDCommands can act on multiple tables for joined recordsets like this. That would automatically refresh the individual record for you when you post it.
If I have an ADO Connection with CursorLocationType.adUseClient and I use it to open a Recordset, then does setting the Recordset.CacheSize property have any effect?
From CacheSize Property
Use the CacheSize property to control how many records to retrieve at one time into local memory from the provider.
And from The Significance of Cursor Location:
In ADO, call for a client-side cursor by using the adUseClient CursorLocationEnum. With a non-keyset client-side cursor, the server sends the entire result set across the network to the client computer.
It seems to me that setting CacheSize will have no effect for a client-side cursor, because the records are already in local memory. But I can't find any official documentation that explicitly says this. I can only find blog posts like this one that says:
We will be using ClientSide cursors ... The CacheSize tells ADO to cache N number of rows on the cursor. Since we chose to have 10 records per page, I set the cache size to 10. ADO will grab the first 10 records, and cache those. It won't hit the database until you request any past the cached limit.
but this seems to contradict "the server sends the entire result set across the network to the client computer."
So what actually happens?
I need an alternative way to prevent someone from accessing a particular piece of code.
I'll explain the scenario.
There are two programs.
In the first program an end-user creates a proforma invoice. When he/she then views the details on the invoice. The code displays the details with the main table's record in EXCLUSIVE-LOCK. This is to prevent other end-users from changing anything while the first user is busy viewing the details. So even when the proforma invoice is completed and can no longer be changed. The main table's record is still in EXCLUSIVE-LOCK. which is wrong, but it prevents other users from messing with it while the first user is still busy updating it. However, the people who work in this program leave the program in the detail view. They don't go out.
The problem is when the second program is used to dispatch the items on the proforma invoice. It uses the same main table's record. And therefore can't do anything because the first program still has it in EXCLUSIVE-LOCK.
My question is...
How can I prevent users changing data in the first program as if the main table's record was in EXCLUSIVE-LOCK, but without actually having it in exclusive-lock? Over multiple sessions...
This might be better a comment, but I don't have enough reputation points to make comments. Sorry.
Some notes:
Optimistic locking -- if it is viable for your situation -- is almost certainly the best solution.
If you are going to add an isLocked field to the table, you will probably want several other fields:
Date/Time the record was locked -- or else an expiration timestamp
LockHolder -- so you know whether or not you've got it.
The expiration can be automatic (as with a cron sweep), or can be ignored unless someone else wants the record. The program which sets the lock must also be smart enough to check to see if it still holds it. It gets complicated.
There are times when it is not convenient to make schema changes to a table, or there are too many tables that need changes. In those cases, you can add these fields to a separate LockIt table. One table can handle these locks for all your other tables.
Aside:
We also use our LockIt table for another purpose: to make sure that only one copy of a given program can run at a time. (Usually this is for cron jobs or a batch daemon.) The program exclusive-locks a particular record in the LockIt table (but DOES NOT start a transaction!), and it holds that lock as long as the program is running.
Most likely, your transaction scope is wrong. I'll go ahead and assume you are trying to dispatch with the invoicing program still open. Then you can't, because the record is still locked. Chances are your whole program is a transaction, and the record will keep locked for as long as the screen is running. Try and revisit your updates, enclose your real update operations in a DO TRANSACTION block, put some MESSAGE TRANSACTION statements around in different places and see what the results are. This will help you find the points in which Progress is being led to "believe" the record still needs to be locked.
I assume you're not using appserver since this behaviour most likely wouldn't be a problem in that case.
One solution could be to change into a "optimistic locking" approach. This means that you start out with a "NO-LOCK" and once you need to change the record you upgrade the lock to EXCLUSIVE-LOCK. This approach will work but you will need to make sure that the record still exists and isn't changed by some other user.
Depending on how often your invoices actually change this might (or might not) be a solution. If a mishap happens "once in a while" this might be a viable solution. If it happens often (every day or so) you need to do something else.
Basic pseudo code for an optimistic approach:
FIND FIRST record WHERE somethingsomething NO-LOCK.
/* Here goes code for displaying the record */
/* .... */
/* Here's the updates */
IF userWantsToSave THEN DO:
FIND CURRENT record EXCLUSIVE-LOCK NO-ERROR NO-WAIT.
IF AVAILABLE record THEN DO:
IF CURRENT-CHANGED record THEN DO:
MESSAGE "Changed!" VIEW-AS ALERT-BOX ERROR.
/* Your code goes here */
END.
ELSE DO:
/* Your code for updating goes here */
MESSAGE "Success!" VIEW-AS ALERT-BOX INFORMATION.
END.
END.
ELSE IF NOT AVAILABLE record THEN DO:
IF LOCKED record THEN DO:
MESSAGE "Locked!" VIEW-AS ALERT-BOX ERROR.
/* Your code goes here */
END.
ELSE DO:
MESSAGE "Deleted!" VIEW-AS ALERT-BOX ERROR.
/* Your code goes here */
END.
END.
END.
Here's an example from the knowledgebase that goes more into depth with this.
I am trying to use Macros in FireDAC to Preprocess my SQL Queries. I have a TADQuery object on a Data Module with the SQL set to something like:
Select * from MyTable
join OtherTable on MyTable.Key = OtherTable.Key
&Where
Then in my code I do this:
WhereClause = 'stuff based on my form';
Query.MacroByName('Where').AsRaw := WhereClause;
Query.Open;
This has worked great for complicated queries because it lets me make sure my fields and join conditions are correct using the SQL Property editor.
My problem is when the SQL statements ends up invalid because of my where clause. Is there any way to see the SQL after pre-processing that is going to be executed? Right now I am catching the FireDac errors and showing the SQL that is on EADDBEngineException object. However that is still showing my original SQL with the macros. If I can't get to it after the error happens is there anyway to force the Macro replacement to take place so I can look at the SQL in the debugger to help me see what is wrong.
If it matters I am connecting to a MS Access database with the goal of moving to SQL Server in the near future.
Apart from using Text property, to monitor what SQL is actually going to the database engine, consider using the "FDMonitor" FireDAC utility. According to the DokWiki pages (below):
drop a TFDMoniRemoteClientLink component on your form,
Set its Tracing property to True,
Add the MonitorBy=Xxx connection definition parameter to your existing FDConnection component. You can do this in the IDE object inspector, by selecting your FDConnection component, expanding the Params property, and setting MonitorBy to mbRemote.
Note that the TFDMoniXxxxClientLink should come before TFDConnection in the data module or form creation order, so adjust this by right clicking on the form or data module, then Creation Order, and moving the TFDMoni.. component above the FDConnection.
Also, it's helpful in the options of the TFDMoniXxxxClientLink, to disable most of the events being recorded, otherwise all the data returned is also shown in the FireDAC monitor. Expand the EventKinds property, and turn all the event kinds off, except for perhaps ekConnConnect, ekConnPrepare, and ekCmdExecute.
Then open the FireDAC Monitor from the IDE, (Tools > FireDAC Monitor). Start your app only once the monitor is running. Double click on a trace event (in the Trace Output tab), and you will see the actual SQL sent to the database in the bottom pane.
It also seems likely that adding the EventType of ekConnPrepare as mentioned above, would show you when the query's Prepare is called, but I haven't played enough with it say for sure.
Please see the following pages on the DocWiki for more information:
Overview: FDMonitor
How to: Tracing and Monitoring (FireDAC)
Other FireDAC utilities: Utilities (FireDAC)
(Just to remove this question from list of unanswered questions)
From comments:
Well, I've roughly checked what's happening there and I'm still not
sure if calling Prepare (which is useless for you as I get) is the
minimal requirement to trigger that preprocessing. Though, the
preprocessed SQL, the one which is sent to the DBMS you can access
through the Text property (quite uncommon name for such property). – TLama Feb
21 '14 at 8:18