I am writing an iPhone app that makes many calls to a sqlite database. My question is, is it best to leave open and close the database connection for every query, or to leave it open and just close it when the app closes.
There are times in my app where I may loop through many iterations and in each I am adding a row to the db, but as I have it now, for each iteration I open the db, perform the operation, then close the db.
I would leave it open all the time the app is in the foreground and only close it when it goes into the background or when I get a memory warning. It might be expensive to open the connection, so I like to do it as little as possible, however it might consume resources which could be better used elsewhere.
You need to write all your db access code to open the connection if it's not already open; so you can close it whenever you like.
Personally, if you are making lots of calls to update the database, then I would try and keep it open. If there are 'chunks' of logic that you can keep the db open for then that would good, maybe having it open for all the time the app is running might not be the best thing - but this is really dependent on the architecture of your app.
Related
In our application we have been using an encrypted sqlite database in db3 format that is downloaded from server and then again after processing is uploaded. The app is live and is used by several users.
Sometimes, very intermittently in one or two instances, the database gets corrupted. The user has to discard the entire application and reinstall again to work resulting in data loss.
Only once we could detect that one of the tables got missing from the database through no drop table command was written anywhere in the code.
Did anyone face this instance before? Any idea why does this happens?
Please note: The application is iPad application written in objective C.
One of the main reason:
iDevices shut down quite a time before they'd actually run out of power. Before your device shuts down your App will get notified that it's going to background, and then get notified that it's going to quit. If you're handling those two notifications properly (i.e. closing all SQLite connections at one or the other) then you should not be getting database corruption.
I have an app that uses Realm as a staging database. It receives information from a bluetooth device, processes it, and sends the processed result to a server.
The incoming data from bluetooth gets stored in a Realm table (table1). A separate thread reads data from the Realm database, processes it, and stores it into a second table (table2) for uploading to a server. When it pulls this data and successfully processed it, it deletes it from table1.
The third thread pulls data from table2, and when it successfully sends, removes it from table2.
I'm using a database here in case, for whatever reason, the app is killed - data won't be lost... it will just resume where it left off when the app is restarted. But as you can see, the database is not something that hangs around (it's not like an address book or something... it is just temporary staging)
What I notice is that no matter what the heck I do, the realm database file just increases in size over time. I'll end up with a database that if I open it, will have one record in it, but the database file on disk could be 10s of MB in size if the app is running long enough.
Data is being processed on different background queues so as to not block any UX (one of the reasons I'm using Realm instead of CoreData). But I'm using things like autoreleasepools and the invalidate command to avoid objects that are read from having copies made (as suggested by many realm questions/answers)
What gives? I know I don't have a code sample here, but this just seems like a basic garbage collection problem in Realm. I've seen other questions related to this where people are like "why is my database so huge", and the answers suggest doing things like "writeCopyToPath", but that feels like an incredible hack, and regardless, it would be very difficult - this app is meant to be constantly connected and monitoring a bluetooth device, so to do this, it would mean stopping, making sure all threads that might alter the database are quiesced, doing the copy to compact the db, and then starting everything back up again. That just seems nonsensical to me. I might interrupt user operations for example. I don't want a user to not be able to do something because I decided it was time to do database maintenance.
I feel like I'm either missing some incredibly fundamental point in how to make Realm not keep junk around, or Realm is just the completely wrong solution for my problem. I've never seen this problem with databases - adding and deleting lots of records... quickly... seems like something a database should just be able to do without exploding in size.
Are you making sure that the background thread is not holding on to old versions of the Realm, preventing the space from being reused?
Quote from the docs (https://realm.io/docs/swift/latest/#seeing-changes-from-other-threads):
If a thread has no runloop (which is generally the case in a background thread), then Realm.refresh() must be called manually in order to advance the transaction to the most recent state.
Failing to refresh Realms on a regular basis could lead to some transaction versions becoming “pinned”, preventing Realm from reusing the disk space used by that version, leading to larger file sizes.
Since I've started developing my Blackberry app, the biggest problems I've encountered all had to do with SQLite Databases.
Right now I'm putting my app through a stress test, and when problems pop up I address them by printing out statuses to the console and taking care of things line by line. Right now (after mashing buttons on my app) I received a "Database is locked" error and I'm not sure what to do.
It seems that once the database is locked it's locked for good until it is unlocked........ my question is how can I unlock it?? First of all, how can I check to see if it's locked??
I'm sure our users won't be mashing buttons like I did, but you never know. I want to account for every possible scenario.
Thanks
EDIT: This is what happens in my application..... When I launch it starts a thread, this thread performs a cleanup on one of my tables based on how old certain pieces of data are (uses DELETE). The thread then continues to get a USER object from my DB (read only), it then uses this USER object as a parameter to call a web service. The data retrieved from the web service is INSERTED into my database. (It's a little more complex than that as a few read/write operations are performed at this time. After that, the thread fires a callback method to update my UI.
This all works fine. I can exit the app WHILE the thread is running and relaunch and a flag will prevent it from starting a new instance of the same thread (unless the other one is done of course).
Now my problem: My app's home screen is a list of buttons, when the user clicks one of these buttons another, more detailed list is loaded (this requires a READ ONLY call to the database). When I launch the app (firing the web service calling thread) and then click a button on the main screen right away, the table gets locked. (Not always, sometimes it takes 4 or 5 tries, sometimes more, sometimes less). But if I keep doing this it WILL eventually lock making it impossible to make any calls to my DB, hence no more UI (which depends on the DB).
The DB call that populates the UI on the second screen is READ ONLY, can't I have as many of these as I need?? What causes the DB to lock?? What's the difference between a DB lock and File System error (12)??
I seemed to have fixed the problem. I was under the impression that if a read/write connection was open then a read-only connection could be created safely.
This doesn't seem to be the case. If I have a read/write connection open then no other connections can open until that one is finished.
I basically created one read/write connection, set a flag to identify it as open, and during my read connection use the same Database object if the flag is open, or create a read only if it's closed.
So far so good.
Sqlite does not support concurrent modification. In practice on BlackBerry, this means you can only open the database from one part of the code at a time. To maintain this one-at-a-time access, you need to close the database when you are done with it, as #AnkitRox points out.
However you also need to guard against concurrent access. Even if your code properly closes the database, it is possible for two different threads to access the database. In that case, you will need one to wait. In Java-ME this is typically accomplished through the 'synchronized' keyword, and using the same lock object for all database access.
Check properly that, you are opening and closing database before and after execution of query respectively.
Because if Database is going to open without closing it properly, then it gives errors.
He Guys,
Should I open the data connection to Sqlite every time I make a query? or is it possible to open the sqlite connection when the app opens and the connection stays open until the app exists?
Is this even a solution?
You shouldn't really be using SQLite directly at all. At least use FMDB. Given your apparent new-ness to development, I would recommend Core Data as it is extremely well supported in the OS and will lead to the fewest lines of code (when used correctly).
To answer your concrete question; no, generally, you do not open / close a connection with every single query.
I've been reading up on SQLite3 included in the iOS firmware which might serve my needs for the app i'm writiung.
What I can't figure out is if it is persistent or goes away like some objects do.
For example if I do sqlite3_open() which appears to be a C function rather than an Objective-C object, if I open this at the start of my application, will it stay persistent until I close it no matter how many views I push/pop all over the place.
Obviously that would depend on where I put it but if I was doing a universal app and had some central functions for loading / saving data which were common to both iPhone/iPad, if, in my didFinishLoading: I put a call to open the SQLite database and then called various exec's of queries, would it remain persistent throughout the lifecycle of the application.
or
Am I better off opening and closing as needed, i'm coming from a PHP background so i'd normally open a database at the start of the script and then run many queries and then finally close it before browser output.
From the 1,000,000th i've learned over the last few months about iOS programming, I think the latter might be the better way as there's possibility of app exit prematurely or it going to background.
I'd just like a second opinion on my thinking please.
I dont know directly, but I think you are right - you only need to open it once at the start of your app.
Looking at sqlitepersistentobjects, an ORM framework for iOS, it only opens the DB when its first used, and never closes it except when there is a problem opening it :)
Single opened sqlite database used throughout the app from different places in your app is fine.
You are using word "persistent" which is confusing. What you mean is "reuse of single connection, for executing different statements in the app, possibly from different threads". Persistence has completely different meaning in context of databases - it means that the requested modification of data has been safely stored to media (disk, flash drive) and the device can even unexpectedly shut down without affecting written data.
It's recommended to keep running sqlite statements from a single, dedicated thread.
It's not recommended to connect to sqlite database from different processes for and executing parallel modifications.
A good alternative solution is to use sqlite async extension which sends all writes to a dedicated, background thread.
You can check out https://github.com/mirek/CoreSQLite3 framework if you want to use custom built (newer version) of sqlite.