I've been reading up on SQLite3 included in the iOS firmware which might serve my needs for the app i'm writiung.
What I can't figure out is if it is persistent or goes away like some objects do.
For example if I do sqlite3_open() which appears to be a C function rather than an Objective-C object, if I open this at the start of my application, will it stay persistent until I close it no matter how many views I push/pop all over the place.
Obviously that would depend on where I put it but if I was doing a universal app and had some central functions for loading / saving data which were common to both iPhone/iPad, if, in my didFinishLoading: I put a call to open the SQLite database and then called various exec's of queries, would it remain persistent throughout the lifecycle of the application.
or
Am I better off opening and closing as needed, i'm coming from a PHP background so i'd normally open a database at the start of the script and then run many queries and then finally close it before browser output.
From the 1,000,000th i've learned over the last few months about iOS programming, I think the latter might be the better way as there's possibility of app exit prematurely or it going to background.
I'd just like a second opinion on my thinking please.
I dont know directly, but I think you are right - you only need to open it once at the start of your app.
Looking at sqlitepersistentobjects, an ORM framework for iOS, it only opens the DB when its first used, and never closes it except when there is a problem opening it :)
Single opened sqlite database used throughout the app from different places in your app is fine.
You are using word "persistent" which is confusing. What you mean is "reuse of single connection, for executing different statements in the app, possibly from different threads". Persistence has completely different meaning in context of databases - it means that the requested modification of data has been safely stored to media (disk, flash drive) and the device can even unexpectedly shut down without affecting written data.
It's recommended to keep running sqlite statements from a single, dedicated thread.
It's not recommended to connect to sqlite database from different processes for and executing parallel modifications.
A good alternative solution is to use sqlite async extension which sends all writes to a dedicated, background thread.
You can check out https://github.com/mirek/CoreSQLite3 framework if you want to use custom built (newer version) of sqlite.
Related
In our application we have been using an encrypted sqlite database in db3 format that is downloaded from server and then again after processing is uploaded. The app is live and is used by several users.
Sometimes, very intermittently in one or two instances, the database gets corrupted. The user has to discard the entire application and reinstall again to work resulting in data loss.
Only once we could detect that one of the tables got missing from the database through no drop table command was written anywhere in the code.
Did anyone face this instance before? Any idea why does this happens?
Please note: The application is iPad application written in objective C.
One of the main reason:
iDevices shut down quite a time before they'd actually run out of power. Before your device shuts down your App will get notified that it's going to background, and then get notified that it's going to quit. If you're handling those two notifications properly (i.e. closing all SQLite connections at one or the other) then you should not be getting database corruption.
I recently modified my iOS app to enable serialized mode for both a database encrypted using SQLCipher and a non-encrypted database (also SQLite). I also maintain a static sqlite3 connection for each database, and each is only opened once (by simply checking for null values) and shared throughout the lifetime of the app.
The app is required to have a sync-like behavior which will download a ton of records from a remote database at regular intervals using a soap request and update the contents of the local encrypted database. Of course, the person using the app may or may not be updating or reading from the database, depending on what they're doing, so I made the changes mentioned in the above paragraph.
When doing short term testing, there doesn't appear to be any issue with how things work, and I've yet experience any problem.
However, some users are reporting that they've lost access to the encrypted database, and I'm trying to figure out why.
My thoughts are as follows: Methods written by another developer declared all sqlite3_stmt's to be static (I believe this code was in the problematic release). In the past I've noticed crashes when two threads using a particular method run simultaneously. One thread finalizes, modifies or replaces a sqlite3_stmt while another thread is using it. A crash doesn't always occur because he has wrapped most of his SQLite code in try/catch blocks. If it's true that SQLite uses prepare and finalize to implement locking, could the orphaning of sqlite3_stmt's which occurs due to their static nature in this context be putting the database into an inoperable state? For example, when a statement acquires an exclusive lock after being stepped is replaced by an assignment in the same method running in another thread?
I realize that this doesn't necessarily mean that the database will become permanently unusable, but, consider this scenario:
At some point during the app's lifetime it will re-key the encrypted database and that key is stored in another database. Suppose that it successfully re-keys the encrypted database, but then the new key is not stored in the other database because of what I mentioned above.
Provided that the database hasn't become corrupted at some point (I'm not really counting on this being the case), this is the only explanation I can come up with for why the user may not be able to use the encrypted database after restarting the iOS app, seeing as the app would be the only one to access the database file.
Being that I can't recreate this issue, I can only speculate about what the reasoning might be. What thoughts do you have? Does this seem like a plausible scenario for something that happens rarely? Do you have another idea of something to look into?
If the database is rekeyed, and the key for the database is not successfully stored in the other database, then it could certainly cause the problem.
Since I've started developing my Blackberry app, the biggest problems I've encountered all had to do with SQLite Databases.
Right now I'm putting my app through a stress test, and when problems pop up I address them by printing out statuses to the console and taking care of things line by line. Right now (after mashing buttons on my app) I received a "Database is locked" error and I'm not sure what to do.
It seems that once the database is locked it's locked for good until it is unlocked........ my question is how can I unlock it?? First of all, how can I check to see if it's locked??
I'm sure our users won't be mashing buttons like I did, but you never know. I want to account for every possible scenario.
Thanks
EDIT: This is what happens in my application..... When I launch it starts a thread, this thread performs a cleanup on one of my tables based on how old certain pieces of data are (uses DELETE). The thread then continues to get a USER object from my DB (read only), it then uses this USER object as a parameter to call a web service. The data retrieved from the web service is INSERTED into my database. (It's a little more complex than that as a few read/write operations are performed at this time. After that, the thread fires a callback method to update my UI.
This all works fine. I can exit the app WHILE the thread is running and relaunch and a flag will prevent it from starting a new instance of the same thread (unless the other one is done of course).
Now my problem: My app's home screen is a list of buttons, when the user clicks one of these buttons another, more detailed list is loaded (this requires a READ ONLY call to the database). When I launch the app (firing the web service calling thread) and then click a button on the main screen right away, the table gets locked. (Not always, sometimes it takes 4 or 5 tries, sometimes more, sometimes less). But if I keep doing this it WILL eventually lock making it impossible to make any calls to my DB, hence no more UI (which depends on the DB).
The DB call that populates the UI on the second screen is READ ONLY, can't I have as many of these as I need?? What causes the DB to lock?? What's the difference between a DB lock and File System error (12)??
I seemed to have fixed the problem. I was under the impression that if a read/write connection was open then a read-only connection could be created safely.
This doesn't seem to be the case. If I have a read/write connection open then no other connections can open until that one is finished.
I basically created one read/write connection, set a flag to identify it as open, and during my read connection use the same Database object if the flag is open, or create a read only if it's closed.
So far so good.
Sqlite does not support concurrent modification. In practice on BlackBerry, this means you can only open the database from one part of the code at a time. To maintain this one-at-a-time access, you need to close the database when you are done with it, as #AnkitRox points out.
However you also need to guard against concurrent access. Even if your code properly closes the database, it is possible for two different threads to access the database. In that case, you will need one to wait. In Java-ME this is typically accomplished through the 'synchronized' keyword, and using the same lock object for all database access.
Check properly that, you are opening and closing database before and after execution of query respectively.
Because if Database is going to open without closing it properly, then it gives errors.
I have a lot of data present in the database present in my webserver. Each time I starts the ipad application after downloadiding, I want all these data to be copied into the sqlite database present in my application. Then using this data, the application should work.
We are now using xml's and sometimes on 3g it takes about 20minutes which is completely unacceptable. After the 1st time it syncs using time log and all. And it works without any problem.
Is there any other way I could get all the data and make it populated into my sqlite db?
If it is a large database it might be worth doingin the background. And even better if it was just over wifi (otherwise you'll be eating up your users data)
What I usually do is have a local copy of the database shipping with the app, so the user can use that, and update it in the background. It might be worth creating some pages where you just present the updated content, download that, then update your database accordingly. Rather than downloading everything all the time.
This would depend entirely on your implementation however.
On initial app launch you could download just enough data to make the app functional. Then download the rest of the data in the background.
This is a common strategy used by Sync Frameworks and works pretty well. I have personally tried with with synchronizing more than a thousand objects using OpenMobster's sync service.
Now what data to download initially is to be decided by the requirements of the App
Thanks
I store some data in my iOS app directly in a local .sqlite file. I chose to do this instead of CoreData because the data will need to be compatible with non-Apple platforms.
Now, I'm trying to come up with the best way to sync this file over iCloud. I know you can't sync it directly, for many reasons. I know CoreData is able to sync its DBs, but even ignoring that using CD would essentially lock this file into Apple platforms (I think? I've only looked into CD a bit), I need the iCloud syncing of this file to work across ALL of iCloud's supported platforms - which is supposed to include Windows. I have to assume that there won't be any compatibility for the CoreData files in the Windows API. Planning out the best way to accomplish this would be a lot easier if Apple would tell us any more than "There will be a Windows API [eventually?]"
In addition, I'll eventually need to implement at least one more sync service to support platforms that iCloud does not. It would be helpful, though not required, if the method I use for iCloud can be mostly reused for future services.
For these reasons, I don't think CoreData can help me with this. Am I correct in thinking this?
Moving on from there, I need to devise an algorithm for this, or find an existing one or an existing 3rd party solution. I haven't stumbled across anything yet. However, I have been mulling over a couple possible methods I could implement:
Method 1:
Do something similar to how CoreData syncs sqlite DBs: send "transaction logs" to iCloud instead and build each local sqlite file off of those.
I'm thinking each device would send a (uniquely named) text file listing all the sql commands that that device executed, with timestamps. The device would store how far along in each list of commands it has executed, and continue from that point each time the file is updated. If it received updates to multiple log files at once, it would execute each command in timestamp order.
Things could get 'interesting' efficiency-wise once these files get large, but it seems like a solvable problem.
Method 2:
Periodically sync a copy of the working database to iCloud. Have a modification timestamp field in every record. When an updated copy of the DB comes through, query all the records with newer timestamps than some reference time and update the record in the local DB from the new data.
I see many potential problems with this method:
-Have to implement something further to recognize record deletion.
-The DB file could get conflicts. It might be possible to deal with them by handling each conflict version in timestamp order.
-Determining the date to check each update from could be tricky, as it depends on which device the update is coming from.
There are a lot of potential problems with method 2, but method 1 seems doable to me...
Does anyone have any suggestions as to what might be the best course of action? Any better ideas than my "Method 1" (or reasons why it wouldn't work)?
Try those two solutions from Ray Wenderlich:
Exporting/Importing data through mail:
http://www.raywenderlich.com/1980/how-to-import-and-export-app-data-via-email-in-your-ios-app
File Sharing with iTunes:
http://www.raywenderlich.com/1948/how-integrate-itunes-file-sharing-with-your-ios-app
I found it quite complex but helped me a lot.
Both method 1 and method 2 seem doable. Perhaps a combination of the two in fact - use iCloud to send a separate database file that is a subset of data - i.e. just changed items. Or maybe another file format instead of sqlite db - XML/JSON/CSV etc.
Another alternative is to do it outside of iCloud - i.e. a simple custom web service for syncing. So each change gets submitted to a central server via JSON/XML over HTTP, and then other devices pull updates from that.
Obviously it depends how much data and how many devices you want to sync across, and whether you have access to an appropriate server and/or budget to cover running such a server. iCloud will do that for "free" but all it really does is transfer files. A custom solution allows you to define your syncing model as you wish, but you have to develop and manage it and pay for it.
I've considered the possibility of transferring a database file through iCloud but I think that I would run into classic problems of timing - slow start for the user - and corrupted databases if the app is run on multiple devices simultaneously. (iPad/iPhone for example).
Sooo. I've had to use the transaction logs method. It really is difficult to implement, but once in place, seems ok.
I am using Apple's SharedCoreData sample as the base for this work. This link requires an Apple Developer Account.
I did find a much much better solution from Tim Roadley however this only works for IOS and I needed both IOS and MacOS.
rant> iCloud development really has to get easier and more stable! /rant