I'm trying to track down a strange bug reported by one user of an iOS app. The crash happens here:
if(sqlite3_exec(database, "pragma cache_size=50;", NULL, NULL, NULL) != SQLITE_OK)
{
NSAssert1(0, #"Error: failed to set cache size");
}
The assertion trips for just this one user. However, this call has always returned SQLITE_OK for years and for countless users, in the wild and in testing, without ever tripping the assert.
I can't reproduce this problem myself, even with the same hardware, iOS version and SQLite database, so I'm a bit stumped.
What could cause pragma cache_size = X to fail?
Related
I've got a JSON file which contains more than 2000 arrays, and in each array there may have an array with 5 strings.
I'm running two for() loops in one function, so I think the 2ed one will be run after the 1st one is done?
and it sometimes appears the following(random data error appear):
database is locked
when I'm executing
INSERT INTO problemTags(problem_IDIndex)VALUES('93E')
And I use the code below to execute my SQL statement
+ (void)execSqliteWithSQL:(NSString *)sql{
sqlite3 *sqlite = nil;
int openResult = sqlite3_open([DBPath UTF8String], &sqlite);
if(openResult != SQLITE_OK){
NSLog(#"open DB error");
}
char *error;
int execResult = sqlite3_exec(sqlite, [sql UTF8String], nil, nil, &error);
if(execResult != SQLITE_OK){
NSString *errorString = [NSString stringWithFormat:#"\n execResult error=>\n %s \n SQL:\n%#\n",error,sql];
NSLog(#"%#",errorString);
}
sqlite3_close(sqlite);
}
I'm thinking about the reason to this issue is that
when I'm running the for loop, it sends several SQL statement to the execSqliteWithSQL function and when it can't run fast enough the data base would be closed?
question added below:
without using threads
the two for loops is like below:
-(void)......{
for(){
//sending SQL to execSqliteWithSQL
}
for(){
//sending SQL to execSqliteWithSQL
}
}
A couple of thoughts:
Do you possibly have multiple threads calling this execSqliteWithSQL concurrently? If so, errors like this are very likely.
You could remedy this by setting up a dedicated serial queue to which execSqliteWithSQL would dispatch its calls, thus eliminating attempts to do this concurrently.
Regardless, I would advise against opening and closing the database for each call. That's horribly inefficient.
In fact, if doing a bunch of update/insert statements, you not only don't want to open and close the database for every call, but rather you want to open, call BEGIN TRANSACTION, do your updates, and then COMMIT those changes. The performance difference is startling.
This execSqliteWithSQL clearly presupposes that you're manually building your SQL (presumably with stringWithFormat). That's problematic at best, dangerous at worst. If a quotation mark appears in a string value being inserted into the database, the SQL will fail. If this feed includes any user input, you are susceptible to SQL injection attacks.
Instead, you should use sqlite3_prepare_v2 to prepare a SQL statement with ? placeholders, and then use sqlite3_bind_xxx to bind values to those placeholders.
I am trying to insert dummy data into my sqlite table. How could I insert this properly and what am I doing wrong? Thanks, all help and advice is appreciated
+(void)setAllValues {
static sqlite3_stmt *insertStatement;
const char *sqlInsert = "insert into session_descriptions (sessionID, sessionTitle, sessionDescription) values (1, 'Programming Title' 'Programming description')";
if(sqlite3_prepare_v2([DBHelper getDB], sqlInsert, -1, &insertStatement, NULL) != SQLITE_OK)
{
NSLog(#"Insert statement not working");
}
sqlite3_finalize(insertStatement);
sqlite3_close([DBHelper getDB]);
}
A couple of issues:
You're checking to see if sqlite3_prepare_v2 failed, but not bothering to check why it failed. This is accomplished by logging sqlite3_errmsg:
if (sqlite3_prepare_v2([DBHelper getDB], sqlInsert, -1, &insertStatement, NULL) != SQLITE_OK) {
NSLog(#"prepare failed: %s", sqlite3_errmsg([DBHelper getDB]));
}
It's important to check sqlite3_prepare_v2 for success, and if not, immediately log sqlite3_errmsg, because most SQLite errors are detected during the preparation of the statement (and if you do any further SQLite calls before examining the error message, any meaningful error message might be replaced with some less-illuminating message).
Even if the prepare succeeded, you're not performing the SQL query. So after the above "prepare" statement, you should "step" it, in order to perform it:
if (sqlite3_step(insertStatement) != SQLITE_DONE) {
NSLog(#"step failed: %s", sqlite3_errmsg([DBHelper getDB]));
}
Do this after sqlite3_prepare_v2 but before sqlite3_finalize.
You are closing the database. This is a bit suspect because you generally want to leave the database in whatever state it was when you called this routine. So, I'd might expect the database to not be open, in which case, this routine would open, prepare, step, finalize, and then close. Or I might expect the database to be open already, in which case you'd merely prepare, step, and finalize, but not close the database, leaving the database in the same state as it was when you called this routine.
As an aside, I would advise opening the database once and keeping it open as long as the app is alive, but either approach works. Just make sure your opens and closes are properly balanced.
Your INSERT statement is missing a comma.
I am trying to insert around 500 records into the sqlite database, it is taking 20 sec, which is way too much! Could anyone please tell how to optimize this.
for (int i = 0 ; i< [[[dic objectForKey:#"response" ]objectForKey:#"requete"]count]; i++) {
[sqlManager executeQuery:[[[dic objectForKey:#"response"]objectForKey:#"requetes"]objectAtIndex:i]];
}
with
-(void)executeQuery:(NSString*)_query
{
sqlite3 *database;
if(sqlite3_open([databasePath UTF8String], &database) == SQLITE_OK) {
if(sqlite3_exec(database, [_query UTF8String],NULL, NULL, NULL) == SQLITE_OK){
//NSLog(#"Query Success");
}
else{
// NSLog(#"Query Failed");
}
sqlite3_close(database);
}
sqlite3_close(database);
}
Open once and leave open until the app is closed or you're done with a DB-intense phase. SQLite can easily handle the situation if your app "crashes" with the DB open.
When doing multiple inserts close together, where it's fine if they either all go or none of them go, start an transaction (and commit it when done). When you do not explicitly start a transaction then SQLite does an implicit start/commit around each insert or update, "forcing" the data to "disk". This takes A LOT of time. Doing one start/commit around a bunch of inserts will increase insert performance at least 10-fold.
You are about to Open & closing the database for every Query(insertion) . So it takes time too.
example For inserting a multiple records at once.
I think you must mantein your database open.
What takes more time is to open and close database.
For example, create a method only for that insertion, that opens the database in the firts element and closes it after the last.
I'm using an sqlite database in my iOS app directly, without taking advantage of Core Data. When the app is launched, I need it to insert several sets of table rows of considerable amount of data that will be needed during the usage of the app and that are fixed sets of data (for example, I have a database table for a list of countries the user could choose in the app). I've those data insertions in an .sql file in the app and, since I need this data to be available in the database from the very first time the user installs and launches the app, I wrote a method that firstly copies the database to Documents, and then reads the file and performs the database insertions, and I call this method in application:didFinishLaunchingWithOptions:.
However, I'm finding that performing a lot of insertions is really a slow operation. I proceed this way:
+ (void)insertFileContent:(NSString *)fileContent intoDatabase:(sqlite3 *)database
{
NSArray *lines = [fileContent componentsSeparatedByString:#"\r\n"];
for (NSString *line in lines) {
const char *sqlStatement = [line UTF8String];
sqlite3_stmt *compiledStatement;
if (sqlite3_prepare_v2(database, sqlStatement, -1, &compiledStatement, NULL) == SQLITE_OK) {
if(sqlite3_step(compiledStatement) == SQLITE_DONE) {
// Insertion was ok
}
else {
NSLog(#"%d",sqlite3_step(compiledStatement));
}
sqlite3_finalize(compiledStatement);
}
}
}
Database is opened only once before calling this method, and closed when it has returned. As I said, I have a lot of data I need to insert into database to make the app work properly, and it is taking even minutes to finish. I've been looking for a way to reduce this time significantly, but I didn't succeed.
How could I handle this? Is there a way to make this operations faster, and/or is there a way to insert data into the database at the time the user installs the app, instead of waiting til launch time?
Thanks in advance
You have the transaction overhead once for each statement.
Wrap all statements into a single transaction.
Execute BEGIN/END manually, or just write them into your .sql file.
Recently I have added BEGIN and COMMIT transaction to all the database calls against my sqlcipher sqlite3 database for an iOS application I am in the process of working on.
I am doing this through the following methods:
#define kTRANSACTION_BEGIN #"BEGIN"
#define kTRANSACTION_COMMIT #"COMMIT"
-(void)transactionBegin
{
int status = sqlite3_exec(self.Database, kTRANSACTION_BEGIN.UTF8String, NULL, NULL, NULL);
NSLog(#"BEGIN Status = %i", status);
}
-(void)transactionEnd
{
char* errorMessage;
int status = 0;
status = sqlite3_exec(self.Database, kTRANSACTION_COMMIT.UTF8String, NULL, NULL, &errorMessage);
NSLog(#"COMMIT Status = %i", status);
if (status != SQLITE_OK) {
NSLog(#"ERROR: SQL Error closing transaction (%i): %s\n %s",status, errorMessage ,sqlite3_errmsg(self.Database));
}
}
These methods are called directly after opening (and setting PRAGMAs) and before closing the database connection.
Both of these appear to execute successfully. However, upon inspecting the location of the database, the db.sqlite file remains 0 bytes with a db.sqlite-journal file at 512 bytes. Queries run agains the database continue to execute successfully until the app is closed and then all the data inserted is lost. Why is my COMMIT not persisting my CREATE TABLE or INSERT statements to disk?
After reading the documentation, it would appear that unless explicitly set, the journal_mode of the sqlite database should be set to DELETE. If this were true, I should no longer have a -journal file after the commit has been completed. Would there be any way through the c API that I would be able to test this?
I am also led to believe that the code is in functional order due to how I am using my database wrapper to act upon a few different database files. Upon successful beginning and committing of the transaction, the BEGIN and COMMIT status codes are always 0 which indicates SQLITE_OK. This odd behavior only occurs in situations where I only insert one record before attempting to COMMIT any changes.
The only other option I have changed from default sqlite configuration is the page size (PRAGMA cipher_page_size = 4096). The page size has been adjusted to help increase performance of some of the other larger databases that I also use with this wrapper. Removing my modification to the page size does not resolve the issue.
Lastly, I have noticed that if I remove the transactions all together, I do not experience this issue at all. Preferably, I would like to leave in the transactions for the increase in performance, but having it not persisting my data defeats the purpose of the database.
Would anyone have any ideas of why this might be happening or anything else I can try?
Thanks in advance.