I have a derby database to store user state to a table. My application updates the record of the user table very frequently resulting into lock wait timeout. And for each user logged in this update query is fired resulting into table locks.
Can i change the table to row-level locking instead of table locking.
Also, will indexing help to update the user rows?
How do i optimize my table to deal with this?
Thanks.
Here is a good place to start in understanding why lock wait timeouts occur: http://wiki.apache.org/db-derby/LockDebugging
Related
I have Delphi 10.1 and Firebird 3 database, I need to check for new record every 2-3 seconds and display on the main screen.
Currently I am using a timer which checks for new records on net database after every 3 seconds and display in the form if new records are available in the database/table.
but this is not the right way to perform the task, what is best practice for continues checking database records.
Thanks
You can write an insertion trigger and post a database event from there. For client implementation with FireDAC see the Database Alerts topic.
I have a very strange problem with transactions in Interbase 7.5 which seem to be stuck.
I can track the problem with IBConsole -> right click DB -> Performance Monitor -> Transactions
Usually this list should show only a few active transaction. But I get several hundred active transactions when I start my application (a web module for an apache webserver using Delphi 7 Interbase components, e.g. IBQuery, IBTransaction, ...)
Transaction type is always listed as snapshot, if this is of relevance.
I have already triple checked all sql statements and cannot find anything that should produce such problems...
Is there any way get the sql statements of a specific transaction?
Any other suggestion how to find such a problem would be very welcome.
Is there any way get the sql statements of a specific transaction?
Yes, you can SELECT from TMP$STATEMENTS WHERE TRANSACTION_ID = .... That's from memory, but should get you started.
In IB Performance Monitor, you can locate the transaction from the statements tab, using the button on the toolbar. Can't remember if you can go the other way in that app. It's been a long time since I wrote it!
Active IBX data-sets require an active transaction all the time. If you don't have active data-sets just don't forget to commit all the active transactions.
If you have active data-sets, you can configure all your components to use the same TIbTransaction object, and you can also configure the unique TIbTransaction to commit or rollback after a idle time-out period via the IdleTimer and DefaultAction properties.
Terminating the transaction (by manually or automatically committing or rolling back) will close all the linked datasets (TIBQuery, TIBTable and the like).
You may be tempted to use the CommitRetaining or RollbackRetaining methods to terminate the transaction without closing the related data-sets, but this may affect the performance of the server, and my advise is to always avoid using it.
If you want to improve your application, you should consider changing your database connection layer or introducing a in-memory capable dataset over IBX, for example, Delphi's TClientDataSet, which allows you to retrieve data and retain it in memory while closing all the underlying datasets (and transactions), while allowing you to use the traditional Insert/Append/Edit/Delete methods to modify the data and then apply that changes to the database in a new short-time transaction.
Each time a user's profile gets displayed we update an integer counter in the Users table.
I started thinking about high-concurrencey situations and wondered what happens if a bunch of people hit a user's profile page at the exact same time: does rails/activerecord magically handle the record locking and semaphore stuff for me?
Or does my app need to explicitly call some sort of mechanism to avoid missing update events when concurrent calls are made to my update method?
def profile_was_hit
self.update_attributes :hitcounter => self.hitcount + 1
end
And along those lines, when should I use something like Users.increment_counter(:hit counter, self.id) instead?
In the default configuration, a single Rails instance is only handling a single request at a time, so you don't have to worry about any concurrency trouble on the application layer.
If you have multiple instances of your application running (which you probably do/will), they will all make requests to the database without talking to one another about it. This is why this is handled at the database layer. MySQL, PostgreSQL, etc. are all going to lock the row on write.
The way you are handling this situation isn't ideal for performance though because your application is reading the value, incrementing it, and then writing it. That lag between read and write does allow for you to miss values. You can avoid this by pushing the increment responsibility to your database (UPDATE hitcounter SET hitcount = hitcount + 1;). I believe ActiveRecord has support for this built in, I'll/you'll have to go dig around for it though. Update: Oh, duh, yes you want to use the increment_counter method to do this. Reference/Documentation.
Once you update your process push incrementing responsibility to the database I wouldn't worry about performance for a while. I once had a PHP app do this once per request and it scaled gloriously for me to 100+ updates/second (mysqld on the same machine, no persistent connections, and always < 10% cpu).
I have a website where the user can upload an excel spreadsheet to load data in a table. There can be a few 100k rows in the excel spreadsheet. When he uploads the file the website needs to insert an equal amount of rows in a database table.
What strategy should i take to do this? I was thinking of displaying a "Please wait page" until the operation is completed but i want him to be able to continue browsing the website. Also, since the database at that time will be kind of busy - wouldn't that stop people from working on the website?
My data access layer is in NHibernate.
Thanks,
Y
Displaying a please wait page would be pretty unfriendly as your user could be wating quite a while and would block threads on your web server.
I would upload the file, store it and create an entry in a queue (you'll need anouther table for this) to indicate that there is a file waiting to be processed. You can then have another process (which could even run on it's own server) which picks up tasks from this queue table and processes the xls file in it's own time.
I would create an upload queue that would submit this request to. Then the user could just check in on the queue every once in a while. You could store the progress of the batch operation in the queue as the rows are processed.
Also, database servers are robust, powerful, multi-tasking systems. Unless you have observed a problem with the website while the inserts are happening don't assume it will stop people from working on the website.
However, as far as insert or concurrent read/write performance goes there are mechanisms to deal with this. You could use the "INSERT LOW PRIORITY" syntax in MySQL or have your application throttle the inserts by sleeping a millisecond between each insert. Also, how you craft your insert statements, wether you use bound parameters or not, and wether you use multi-valued inserts can affect the insert performance and how it affects clients to a large degree.
On Submit you could pass the DB Operation to a asynchronous RequestHandler and set a Session Value when its done.
While the asynch process is in progress you can check the Session Value on each request and if it is set (operation = completed) display a message, eg in a modal or whatever message mechanism you have.
I'm trying to track down a performance issue by looking at the "Active Queries" tab in the Advantage Management Utility.
The documentation for this tab says:
Active: True if the query is being actively processed by the server. A query must be active to be cancelled.
Is a query Active until it completes? Or can it become inactive for another reason, like waiting on a resource (disk IO or a lock)?
I ask because I only 1-2 queries in an "active" state at a given time, but I also have 20+ worker threads running. Which makes little sense to me.
Active means the server is actively looking for rows to populate the cursor with for the request. It will remain active until enough rows have been made to satisfy the request. If the query needs to wait on a lock or disk I\O it will remain active. One caveat to this is live cursors. Live cursors are treated as tables by the client rather than SQL statements. Are the SQL statements that are open, but not active live cursors?
You might try calling the stored procedure sp_mgGetWorkerThreadActivity to see what commands the other threads are doing.