How to optimize SQLcipher performance? - ios

I used SQLCipher to encrypt sqlite database in my app.everything is ok but my app runs slow during fetching database.i changed PRAGMA kdf_iter to 4000 and it's still slow.before encryption i don't have any problem.
-(NSError *) openDatabase {
NSError *error = nil;
NSString *databasePath = [self getDatabasePath];
const char *dbpath = [databasePath UTF8String];
int result = sqlite3_open_v2 (dbpath, &db , SQLITE_OPEN_READWRITE , NULL);
if (result == SQLITE_OK) {
sqlite3_exec(db, [#"PRAGMA kdf_iter = '4000';" UTF8String], NULL, NULL, NULL);
sqlite3_exec(db, [#"PRAGMA key = 'password'" UTF8String], NULL, NULL, NULL);
NSLog(#"Password is correct , Database is Activated");
sqlite3_exec(db, [#"PRAGMA cipher = 'aes-256-cfb';" UTF8String], NULL, NULL, NULL);
}
else {
NSLog(#"Incorrect password!");
}
if (result != SQLITE_OK) {
const char *errorMsg = sqlite3_errmsg(db);
NSString *errorStr = [NSString stringWithFormat:#"The database could not be opened: %#",[NSString stringWithCString:errorMsg encoding:NSUTF8StringEncoding]];
error = [self createDBErrorWithDescription:errorStr andCode:kDBFailAtOpen];
}
return error;
}

Finally i could optimize my SQLCipher performance with Nick Parker useful guidance.
As he said:
There are a few very important guidelines for optimal SQLCipher performance:
Do not repeatedly open and close connections, as key derivation is very expensive, by design. Frequent opening / closing of the database connection (e.g. for every query) is a very common cause of performance issues that can usually be easily resolved using a singleton database connection.
Use transactions to wrap insert / update / delete operations. Unless executed in a transaction scope, every operation will occur within it's own transaction which slows things down by several orders of magnitude
Ensure your data is normalized (i.e., using good practices for separation of data into multiple tables to eliminate redundancy). Unnecessary duplication of data leads to database bloat, which means more pages for SQLCipher to operate on
Ensure that any columns that are used for searches or join conditions are indexed. If you don't, SQLCipher will need to execute full database scans across large numbers of pages
Vacuum periodically to ensure databases are compact if you do large deletes, updates etc.
To diagnose performance problems with specific query statements, it may be helpful to run an explain query plan command against specific queries.
If you are uncertain of what queries are performing poorly, SQLCipher includes a pragma called cipher_profile that allows for profiling queries and their respective execution time in milliseconds.
This is the Reference Link
Many thanks to Nick Parker.
Also this blog was very useful for me.

Why are you using CFB mode (aes-256-cfb) instead of CBC?
I don't believe hardware encryption supports AES CFB so it probably will use software encryption which can easily be 500+ times slower.
From the docs:
SQLCipher uses aes-256-cbc as the default cipher and mode of operation. It is possible to change this, though not generally recommended

Related

iOS - Using Write Ahead Logging on sqlite3 database but still getting database locked errors

I'm using a library that wraps the sqlite3 c library. Internally, when you execute a transaction or a query it opens up the database, does the action, then closes it. As opposed to maintaining a connection for the lifetime of the app, or the lifetime of the higher level database object. So in effect, each query/transaction utilizes its own connection.
I recently moved to having queries done on one thread and writes done on another. This is because the database can be updated via some network calls. I only just noticed that sometimes my writes are failing because the database is locked.
In order to fix that, I went to implementing write ahead logging for the database. However, I'm still getting the database locked error at the end of a transaction.
To enable it, when the database object is created (and before any connections/queries/transactions have been executed), I call the function below. But I'm still getting locked errors. Am I implementing it incorrectly or not thinking about how to enable it correctly?
I had previously tried calling it within the opened connection everytime that an action was called, but that was also failing. From what I read, I only need to enable it onetime and it should persist across all subsequent connections, which led me to the below implementation.
-(BOOL)enableWriteAheadLogging {
BOOL success = TRUE;
sqlite3 *sqlite3Database;
// Set the database file path.
NSString *databasePath = [self.databaseFolderPath stringByAppendingPathComponent:self.databaseFilename];
// Open the database.
BOOL openDatabaseResult = sqlite3_open([databasePath UTF8String], &sqlite3Database);
if (openDatabaseResult == SQLITE_OK) {
char *errorMsg;
if (sqlite3_exec(sqlite3Database, "PRAGMA journal_mode=WAL;", NULL, NULL, &errorMsg) != SQLITE_OK) {
success = FALSE;
NSString *terror = [NSString stringWithFormat:#"%s \n", sqlite3_errmsg(sqlite3Database)];
NSLog(#"DB Error: %#", terror);
}
}
// Close the database.
sqlite3_close(sqlite3Database);
return success;
}

How to use two sqlite database files in single iOS project with simultaneously fetch data from both Database

My requirement is to add two SQLite database files in my project. I know that is possible. But how can I access two database files simultaneously?
One more thing currently I am using FMDB to manipulate database operation.
Note : Both of my DB file sizes are near about 1GB each. so also want to know that if I use 2 DB file so is there any memory issue will occur in my application ?
Thanks.
Generally, there is rare requirement of 2 Database files in a single project specially in iOS where we have used only one DB.
But as per your requirement, we can use multiple Database files in a single project with multiple Database instances.
To Create 2 instance use below code for SQLite Native:
//Create Object of DataBase
static sqlite3 *databaseOne = nil;
static sqlite3 *databaseTwo = nil;
//Open Both Databases while we needed in our project
//Open First Database
sqlite3_open([path UTF8String], &databaseOne);
//Opne Second Database
sqlite3_open([path UTF8String], &databaseTwo);
//prepare for statement execution
sqlite3_prepare_v2(databaseOne, [sql UTF8String], -1, &stmt, NULL)
sqlite3_prepare_v2(databaseTwo, [sql UTF8String], -1, &stmt, NULL)
But make sure you will not mix these 2 objects while executing.
Note: Close both connections while your DB task is done.
To Create 2 instance with FMDB use below code:
//First DB
FMDatabase *dbOne = [FMDatabase databaseWithPath:<your first DB file Path>];
[dbOne open];
//Do Your DB task
[dbOne close];
//Second DB
FMDatabase *dbTwo = [FMDatabase databaseWithPath:<your second DB file Path>];
[dbTwo open];
//Do Your Second DB task
[dbTwo close];
Hope this will help you.

sqlite3_prepare_v2 returns OK on iPhone and iPad mini devices, but with error on normal iPad device

I'm facing a problem in this code: it's working ok on iPhone and iPad Mini, but it's giving the error message on the big iPad. Is there any reason?
Code:
- (sqlite3_stmt *)executeQuery:(char *)aQuery {
NSString *dbPath = [[NSUserDefaults standardUserDefaults] objectForKey:kDBPath];
sqlite3_stmt *statement = NULL;
sqlite3 *database;
if (sqlite3_open((char *)[dbPath UTF8String], &database) == SQLITE_OK) {
if (sqlite3_prepare_v2(database, aQuery, -1, &statement, NULL) != SQLITE_OK) {
//Here is the point: on iPad Mini and iPhone it works fine, but on the normal iPad it can't execute this query
NSLog(#"Error on SQL - (sqlite3_stmt *)executeWithReturn:(char *)aQuery");
statement = NULL;
}
} else {
NSLog(#"Error on Open database");
}
return statement;
}
Prints from LLDB:
(lldb) print (BOOL)[[NSFileManager defaultManager] isWritableFileAtPath:dbPath]
(BOOL) $0 = YES
(lldb) print (char *)aQuery
(char *) $1 = 0x1f0a1000 "select CodAplicacao, CodLocalidade, Nome, Desc, DataCriacao, Editando from Aplicacao_en"
Any ideas?
Regards!
EDIT:
It's getting the error "database disk image is malformed". I couldn't find in the internet how to fix it.
In your revised question, you inform us that sqlite3_errmsg is reporting "database disk image is malformed". This is a sign that your database has been corrupted. This can happen if your app crashed while doing some database operation at some previous date.
You can confirm this by running the following SQL:
PRAGMA integrity_check;
If you want to go through that process, you can copy the database from your device back to your Mac (using Xcode's Organizer and go to the "Devices" section) and then use your Mac SQLite database tool of choice (worst case, the command line sqlite3 program) and then run the previous PRAGMA command. You can even perform this programmatically (it returns a result set like a SELECT statement).
Thing is, we're fairly confident that it's going to report problems. The easy fix is to rebuild the database (e.g. delete the database and if you originally copied it from the bundle, do that again, if you programmatically created it, repeat that process).
If you're determined to try to recover user data from the database, you can contemplate following the process outlined in Sergei Dorogin's technical blog - SQLiteException "database disk image is malformed". He wrote that for a different platform, but the basic process is probably applicable here. Thing is, that unless you absolutely need to recover user data, that process is probably unnecessary.
So, in short, your database on that device is corrupted and you need to rebuild it. Just delete the app (in case your database is in the Documents folder, you need to delete app to empty out that folder) and reinstall the app and you should be good.
Change your log statement from:
NSLog(#"Error on SQL - (sqlite3_stmt *)executeWithReturn:(char *)aQuery");
to
NSLog(#"Error on SQL - %s: %s", __FUNCTION__, sqlite3_errmsg(database));
That should give you an informative error message that tells you what's wrong.
Generally, these sorts of problems result from version of the database on that particular device. It is extremely unlikely to be related to the device itself.
Perhaps your app created a blank database at some point and thus the table is not found. If you use Xcode's Organizer, go to the "Devices" section, and you can copy the app folders to your Mac and then examine the database on your Mac and see if you confirm that the database is as you expected it to be.

Using Sqlite3 VACUUM command on Core Data SQLite Persistent store

In our app, we are implementing sharing of partial Core Data SQLite database through network/email. In order to keep the file size small, I have implemented the below method to shrink the Core Data database.
- (void) shrinkDB
{
sqlite3 * database;
NSString * string = [shareStoreURL path];
const char * filename = [string cStringUsingEncoding:[NSString defaultCStringEncoding]];
char *errMsg;
if (sqlite3_open(filename, &database) == SQLITE_OK)
{
NSLog(#"Shrinking...");
if (sqlite3_exec(database, "VACUUM;", NULL, NULL, &errMsg) != SQLITE_OK)
{
NSLog(#"Failed execute VACUUM");
}
sqlite3_close(database);
}
}
QUESTION: The above code does shrink the database. But Apple says the implementation details of Core Data are subject to change any time. Do you think I would be safe using this method for foreseeable future? Or is there any other better solution?
The proper way to do this is by giving the NSSQLiteManualVacuumOption to the persistent store coordinator.
Snippet from documentation:
NSSQLiteManualVacuumOption
Option key to rebuild the store file,
forcing a database wide defragmentation when the store is added to the
coordinator. This invokes SQLite's VACUUM command. It is ignored by
stores other than the SQLite store. Available in OS X v10.6 and later.
Declared in NSPersistentStoreCoordinator.h.
See this: https://developer.apple.com/library/mac/#documentation/Cocoa/Reference/CoreDataFramework/Classes/NSPersistentStoreCoordinator_Class/NSPersistentStoreCoordinator.html
How Apple structures persistent data in an SQLite database is an implementation detail which is subject to change. However, the method by which SQLite manages deleted records is independent of Apple's implementation.
That being said, the process of vacuuming a SQLite database results in rebuilding the entire database, which may have negative effects if the sqlite file is in use by a CoreData NSPersistentStoreCoordinator.
In your case, it sounds like you want to vacuum after saving changes but before sending it via email. Using the NSSQLiteManualVacuumOption option appears to only vacuum the DB when the SQLite file is initially opened.
I'd either run the above code after the file is no longer associated with a NSPersistentStoreCoordinator or use the NSSQLiteManualVacuumOption then re-open and close the file before sending it via email.
Another option is to use an external SQLite tool, such as Base on OS X, to manually vacuum files. I've also used the Firefox SQLite manager extension in the past.

iOS SQLite: Insert statement fails, can't see why

So I can create the database with no issue using the following statement for the first table:
const char *sqlStatement = "CREATE TABLE IF NOT EXISTS LIBRARY_SEARCH (SEARCH_ID INTEGER PRIMARY KEY AUTOINCREMENT, SEARCH_TERM TEXT)";
But when I go to insert a row it fails. This is how I insert the row.
NSString *insertStatement = [NSString stringWithFormat:#"INSERT INTO LIBRARY_SEARCH(SEARCH_TERM) VALUES (\"%#\")", myQuery] ;
Where myQuery is an NSString that holds the value of the searchBar.text in the app. It's a property and is of course synthesized.
So I then call this:
if ( sqlite3_exec(searchSessionDB, [insertStatement UTF8String], NULL, NULL, &insert_error) == SQLITE_OK)
Which fails. Can anyone see why?
Got it. Made a small typo in my open database statement.
For anyone who looks at this in the future, this is how the opening statement now looks (and it works fine!):
if(sqlite3_open(dbpath, &searchSessionDB) = SQLITE_OK){
Thanks to Robert Ryan for suggesting this NSLog:
NSLog(#"%s", sqlite3_errmsg(searchSessionDB));
And thanks Elpsk and H2CO3 for the wrapper/app bundle information.
I suggest you to use FMDB for query your database.
It's a wrapper to sqlite, easy to use and easy and all errors are explained better.
I'm sure you're trying to update an SQLite database located in your app bundle. You can't: you can't write to your app's own bundle. Copy the database file into the Documents or Library directory off of your app sandbox.
Other: if you're looking for an OO SQLite-wrapper, check out mine.

Resources