I am using SQLite 3 in my iOS application. It requires lots of read/write operation and many of them attempted concurrently. Most of these functions are running in separate threads, however accessing same database.
Here are few methods I have thought of:
At every database access ( read/write ) I put a flag as database_open=YES, and if it is found true I retry again in few seconds. I can put the function at the database layer itself.
Run all database operations in same thread. However, my use case is such that I need to wait for some HTTP calls to finish and store the retreived data into SQLite. For this method, I will have to make all these server calls as well synchronous. Not the best idea.
Please suggest if I am thinking in wrong direction.
SQLite supports concurrent access from multiple threads. Simply configure the SQLITE_CONFIG_MULTITHREAD mode, and separately open the database from each of the threads.
Related
Am doing preloading the datas from service in my mobile application.
Here I added around 7 service in NSOperationQueue and all these operations result will update the DataBase.
Here while updating the database am getting warning in log like "FMDatabase is currently in use" and datas have not been inserted into DB.
Here how to handle this problem in concurrent execution and updating the database.
I didn't use FMDatabase, but you have to read it's documentation
It is part of the doc:
So don't instantiate a single FMDatabase object and use it across
multiple threads.
Instead, use FMDatabaseQueue. It's your friend and it's here to help.
I am using FMDB to access the standard iOS internal SQLite database, with one db connection shared among multiple threads.
To make it thread safe I'm locking access to the db to one block of code at a time. All works well, although the access to the db is now a bit of a bottleneck, obviously.
My question is: Can I ease this up a bit by allowing simultaneous queries from multiple threads, as long as they are all readonly SELECT statements?
I can't find an answer anywhere.
You cannot use the same connection to execute multiple queries at the same time.
However, for purely read-only accesses, you can use multiple connections.
You can have one FMDatabase object for each thread. You might have to write code to test for genuine busy conditions and handle them properly. For example set busyRetryTimeout appropriate for your situation (e.g. how long do you want it to retry in contention situations). Also gracefully handle if the timeout expires and your database query fails.
Clearly, using a shared FMDatabaseQueue is the easiest way to do database interactions from multiple threads. See the Using FMDatabaseQueue and Thread Safety section of the FMDB README.
I'm trying to determine if my sqlite access to a database is thread-safe on iOS. I'm writing a non App Store app (or possibly a launch daemon), so Apple's approval isn't an issue. The database in question is the built-in sms.db, so for sure the OS is also accessing this database for reading and writing. I only want to be able to safely read it.
I've read this about reading from multiple processes with sqlite:
Multiple processes can have the same database open at the same time.
Multiple processes can be doing a SELECT at the same time. But only
one process can be making changes to the database at any moment in
time, however.
I understand that thread-safety can be compiled out of sqlite, and that sqlite3_threadsafe() can be used to test for this. Running this on iOS 5.0.1
int safe = sqlite3_threadsafe();
yields a result of 2. According to this, that means mutex locking is available. But, that doesn't necessarily mean it's in use.
I'm not entirely clear on whether thread-safety is dynamically enabled on a per connection, per database, or global basis.
I have also read this. It looks like sqlite3_config() can be used to enable safe multi-threading, but of course, I have no control, or visibility into how the OS itself may have used this call (do I?). If I were to make that call again in my app, would it make it safe to read the database, or would it only deconflict concurrent access for multiple threads in my app that used the same sqlite3 database handle?
Anyway, my question is ...
can I safely read this database that's also accessed by iOS, and if so, how?
I've never used SQLite, but I've spent a decent amount of time reading its docs because I plan on using it in the future (and the docs are interesting). I'd say that thread safety is independent of whether multiple processes can access the same database file at once. SQLite, regardless of what threading mode it is in, will lock the database file, so that multiple processes can read from the database at once but only one can write.
Thread safety only affects how your process can use SQLite. Without any thread safety, you can only call SQLite functions from one thread. But it should still, say, take an EXCLUSIVE lock before writing, so that other processes can't corrupt the database file. Thread safety just protects data in your process's memory from getting corrupted if you use multiple threads. So I don't think you ever need to worry about what another process (in this case iOS) is doing with an SQLite database.
Edit: To clarify, any time you write to the database, including a plain INSERT/UPDATE/DELETE, it will automatically take an EXCLUSIVE lock, write to the database, then release the lock. (And it actually takes a SHARED lock, then a RESERVED lock, then a PENDING lock, then an EXCLUSIVE lock before writing.) By default, if the database is already locked (say from another process), then SQLite will return SQLITE_BUSY without waiting. You can call sqlite3_busy_timeout() to tell it to wait longer.
I don't think any of this is news to you, but a few thoughts:
In terms of enabling multi-threading (either serialized or multi-threaded), the general counsel is that one can invoke sqlite3_config() (but you may have to do a shutdown first as suggested in the docs or as discussed on SO here) to enable the sort of multi-threading you want. That may be of diminished usefulness here, though, where you have no control over what sort of access iOS is requesting of sqlite and/or this database.
Thus, I would have thought that, from an academic perspective, it would not be safe to read this system database (because as you say, you have no assurance of what the OS is doing). But I wouldn't be surprised if iOS is opening the database using whatever the default mode is, so from a more pragmatic perspective, you might be fine.
Clearly, for most users concerned about multi-threaded access within a single app, the best counsel would be to bypass the sqlite3_config() silliness and just simply ensure coordinated access through your own GCD serial queue (i.e., have a dedicated queue through which all database interactions go through, gracefully eliminating the multi-thread issue altogether). Sadly, that's not an option here because you're trying to coordinate database interaction with iOS itself.
I've been reading up on SQLite3 included in the iOS firmware which might serve my needs for the app i'm writiung.
What I can't figure out is if it is persistent or goes away like some objects do.
For example if I do sqlite3_open() which appears to be a C function rather than an Objective-C object, if I open this at the start of my application, will it stay persistent until I close it no matter how many views I push/pop all over the place.
Obviously that would depend on where I put it but if I was doing a universal app and had some central functions for loading / saving data which were common to both iPhone/iPad, if, in my didFinishLoading: I put a call to open the SQLite database and then called various exec's of queries, would it remain persistent throughout the lifecycle of the application.
or
Am I better off opening and closing as needed, i'm coming from a PHP background so i'd normally open a database at the start of the script and then run many queries and then finally close it before browser output.
From the 1,000,000th i've learned over the last few months about iOS programming, I think the latter might be the better way as there's possibility of app exit prematurely or it going to background.
I'd just like a second opinion on my thinking please.
I dont know directly, but I think you are right - you only need to open it once at the start of your app.
Looking at sqlitepersistentobjects, an ORM framework for iOS, it only opens the DB when its first used, and never closes it except when there is a problem opening it :)
Single opened sqlite database used throughout the app from different places in your app is fine.
You are using word "persistent" which is confusing. What you mean is "reuse of single connection, for executing different statements in the app, possibly from different threads". Persistence has completely different meaning in context of databases - it means that the requested modification of data has been safely stored to media (disk, flash drive) and the device can even unexpectedly shut down without affecting written data.
It's recommended to keep running sqlite statements from a single, dedicated thread.
It's not recommended to connect to sqlite database from different processes for and executing parallel modifications.
A good alternative solution is to use sqlite async extension which sends all writes to a dedicated, background thread.
You can check out https://github.com/mirek/CoreSQLite3 framework if you want to use custom built (newer version) of sqlite.
I'm designing an application that relies heavily on a database. I need my application to be resilient to short losses of connectivity to the database (network going down for a few seconds for example). What is the usual patterns that people use for these kind of problems. Is there something that I can do on the database access layer to handle gracefully a small glitch in the network connection to the db (i'm using hibernate + oracle jdbc + dbcp pool).
I'll assume you have hidden every database access behind a DAO or something similiar.
Now create wrappers around these DAOs, that try to call them, and in case of an exception wait a second and retry. Of course this will cause 'hanging' of the application during db-outage, but it will come back to live when the database becomes available.
If this is not acceptable you'll have to move the cut up closer to the ui layer. Consider the following approach.
User causes a
request.
wrap all the request information in a message and put it in the queue.
return to the user, telling him that his request will get processed in a short time.
A worker registered on the queue will process the request, retrying when database problems exist.
Note that you are now deep in concurrency land. So you must handle things like requests referencing an entity which already got deleted.
Read up on 'eventual consistency'
Since you are using hibernate, you'll have to deal with lazy loading. An interruption in connectivity will kill your session, so for you it might be best not to use lazy loading at all, but work with detached objects.