Is this possible and how would one go about it if it is? Info on the topic is a bit sparse given a Google & Stack search, lots on batch inserts, but nothing solid on batch updates.
Yes it is possible if you insert the correct SQL to do it, but your question is a bit vague.
Rather than trying to update multiple records in a query though, why don't you use a transaction queue instead? Pass your queries in as an array to this function. (requires you have setup an FMDatabase dbQueue of course)
-(BOOL) executeQueryArray:(NSMutableArray*)queryArray {
__block BOOL noErrors = YES;
[self.dbQueue inTransaction:^(FMDatabase *db, BOOL *rollback) {
db.logsErrors = YES;
for (NSString* query in queryArray) {
if (![db executeUpdate:query]) noErrors = NO;
}
*rollback = !noErrors;
}];
return noErrors;
}
Related
As the title, how to use Vacuum to shrink a SQLite database using FMDB?
Thanks in advance!
Thanks for all your support
I've found out the answer : [database executeUpdate:#"vacuum"];
1.Database Updating
I have a single database controller — VSDatabaseController in my latest app — that talks to SQLite via FMDB.
FMDB differentiates between updates and queries. To update the database the app calls:
-[VSDatabaseController runDatabaseBlockInTransaction:(VSDatabaseUpdateBlock)databaseBlock]
VSDatabaseUpdateBlock is simple:
typedef void (^VSDatabaseUpdateBlock)(FMDatabase *database);
runDatabaseBlockInTransaction is also simple:
- (void)runDatabaseBlockInTransaction:(VSDatabaseUpdateBlock)databaseBlock {
dispatch_async(self.serialDispatchQueue, ^{
#autoreleasepool {
[self beginTransaction];
databaseBlock(self.database);
[self endTransaction];
}
});
}
Here’s a simple example of a call to update the database:
SELECT ALL
- (void)emptyTagsLookupTableForNote:(VSNote *)note {
NSString *uniqueID = note.uniqueID;
[self runDatabaseBlockInTransaction:^(FMDatabase *database) {
[database executeUpdate:
#"delete from tagsNotesLookup where noteUniqueID = ?;", uniqueID];
}];
}
[self.database executeUpdate:
#"CREATE INDEX if not exists noteUniqueIDIndex on tagsNotesLookup (noteUniqueID);"];
Database Fetching
To fetch objects, the app calls:
SELECT ALL
-[VSDatabaseController runFetchForClass:(Class)databaseObjectClass
fetchBlock:(VSDatabaseFetchBlock)fetchBlock
fetchResultsBlock:(VSDatabaseFetchResultsBlock)fetchResultsBlock];
These two lines do much of the work:
SELECT ALL
FMResultSet *resultSet = fetchBlock(self.database);
NSArray *fetchedObjects = [self databaseObjectsWithResultSet:resultSet
class:databaseObjectClass];
A database fetch using FMDB returns an FMResultSet. With that resultSet you can step through and create model objects.
3.Keeping Objects in Memory
FMResultSet *resultSet = [self.database executeQuery:#"select uniqueID from some_table"];
4.Web APIs
- (void)uploadNote:(VSNote *)note {
VSNoteAPICall *apiCall = [[VSNoteAPICall alloc] initWithNote:[note detachedCopy]];
[self enqueueAPICall:apiCall];
}
5.Handling Web API Return Values
VSNote *cachedNote = [self.mapTable objectForKey:downloadedNote.uniqueID];
6.Database Migration
[self.database executeUpdate:#"CREATE TABLE if not exists tags "
"(uniqueID TEXT UNIQUE, name TEXT, deleted INTEGER, deletedModificationDate DATE);"];
I hope, it will help you.
I'm building a feature where users of my app can find their Facebook friends and add them in the app. There are three steps which I have to do:
Get the currently connected users
Get the Facebook users
Get the Application Users (this is dependent on step 2)
Once all of these complete I then need to combine/reduce the three resulting arrays into a final array.
I have created three functions that all return RACSignal
getUsersWithFacebookIds, getConnectedUsers, and getFacebookUsers
I'm not sure how to wire all of this using ReactiveCocoa.
The Once All Are Done Do Something With All, can be achieve with:
[[RACSignal combineLatest:#[connectUsersSignal,facebookUsersSignal,applicationUsersSignal]] subscribeNext:^(RACTuple *users) {
NSArray *connectedUsers = [users first];
NSArray *facebookUsers = [users second];
NSArray *applicationUsers = [users third];
}];
The other piece missing is how you make the applicationUsersSignal dependent on the facbookUsersSignal. Which could be done like this:
- (RACSignal *)applicationUsersSignalWithFacebookUsersSignal:(RACSignal *)fbSignal
{
return [fbSignal flattenMap:^RACStream *(NSArray *facebookUsers) {
return [RACSignal createSignal:^RACDisposable *(id<RACSubscriber> subscriber) {
// Do what you have to do with the facebookUsers
return nil;
}];
}];
}
Just to add a bit more to the answer. I am assuming those are cold signals, (signals that haven't started yet aka haven't been subscribed). So the idea of using combineLatest: is that you want to capture the point where each and every single signal as send at least one next, after that you subscribe to it so it can begin. Finally you can get their values from a RACTuple.
I re read your question and you want them to come all in a single Array:
[[[RACSignal combineLatest:#[connectUsersSignal,facebookUsersSignal,applicationUsersSignal]] map:^id(RACTuple *allArrays) {
return [allArrays.rac_sequence foldLeftWithStart:[NSMutableArray array] reduce:^id(id accumulator, id value) {
[accumulator addObjectsFromArray:value];
return accumulator;
}];
}] subscribeNext:^(NSArray *allUsers) {
// Do Something
}];
I've got a JSON object containing 200,000 items. I need to iterate through these objects, and determine if they exist or not and perform the relevant action (insert / update / delete). The shell for this is shown below. Granted, it's not actually saving anything yet. It was more to see how long this way would take. This action takes about 8 minutes to process on an iPhone 4, which seems insane, considering there isn't even any changes occurring yet.
Is there a more efficient way to be handling this?
Any advice or pointers would be greatly appreciated.
- (void) progressiveInsert
{
prodAdd = 0;
prodUpdate = 0;
prodDelete = 0;
dispatch_queue_t backgroundDispatchQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0);
dispatch_async(backgroundDispatchQueue,
^{
_productDBCount = 0;
NSLog(#"Background Queue");
NSLog(#"Number of products in jsonArray: %lu", (unsigned long)[_products count]);
NSManagedObjectContext *backgroundThreadContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSConfinementConcurrencyType];
[backgroundThreadContext setPersistentStoreCoordinator:_persistentStoreCoordinator];
[backgroundThreadContext setUndoManager:nil];
[fetchRequest setPredicate:predicate];
[fetchRequest setEntity:[NSEntityDescription entityForName:#"Products" inManagedObjectContext:_managedObjectContext]];
[fetchRequest setIncludesSubentities:NO]; //Omit subentities. Default is YES (i.e. include subentities)
[fetchRequest setFetchLimit:1];
[_products enumerateObjectsUsingBlock:^(id product, NSUInteger idx, BOOL *stop) {
predicate = [NSPredicate predicateWithFormat:#"code == %#", [product valueForKey:#"product_code"]];
[fetchRequest setPredicate:predicate];
NSError *err;
NSArray *fetchedObjects = [_managedObjectContext executeFetchRequest:fetchRequest error:&err];
if (fetchedObjects == nil) {
if ([[product valueForKey:#"delete"] isEqualToNumber:[NSNumber numberWithBool:TRUE]]){
prodDelete += 1;
} else {
prodAdd += 1;
}
} else {
if ([[product valueForKey:#"delete"] isEqualToNumber:[NSNumber numberWithBool:TRUE]]){
prodDelete += 1;
} else {
prodUpdate += 1;
}
}
dispatch_sync(dispatch_get_main_queue(), ^
{
self.productDBCount += 1;
float progress = ((float)self.productDBCount / (float)self.totalCount);
_downloadProgress.progress = progress;
if (_productDBCount == _totalCount){
NSLog(#"Finished processing");
_endProcessing = [NSDate date];
[_btn.titleLabel setText:#"Finish"];
NSLog(#"Processing time: %f", [_endProcessing timeIntervalSinceDate:_startProcessing]);
NSLog(#"Update: %i // Add: %i // Delete: %i", prodUpdate, prodAdd, prodDelete);
[self completeUpdateProcess];
}
});
}];
});
}
Have a look at
Implementing Find-or-Create Efficiently in the "Core Data Programming Guide".
(Update: This chapter does not exist anymore in the current Core Data Programming Guide. An archived version can be found at
http://web.archive.org/web/20150908024050/https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/CoreData/Articles/cdImporting.html.)
One of the key ideas is not to execute one fetch request per product, but execute a
"bulk fetch" with a predicate like
[NSPredicate predicateWithFormat:#"code IN %#", productCodes]
where productCodes is an array of "many" product codes from your JSON data.
Of course you have to find the optimal "batch size".
With that many objects, I think you need to start being very clever about your data and system and to look for other ways to trim your items prior to fetching 200K JSON objects. You say your using Core Data and are on an iPhone, but you don't specify if this is a client/server application (hitting a web server from the phone). I will try to keep my suggestions general.
Really, you should think outside of your current JSON and more about other data/meta-data that can provides hints about what you really need to fetch prior to merge/update. It sounds like you're synchronizing two databases (phone & remote) and using JSON as your means of transfer.
Can you timestamp your data? If you know the last time you updated your phone DB, you need only pull the data changed after that time.
Can you send your data in sections/partitions? Groupings of 1000-10000 might be much more manageable.
Can you partition your data into sections more or less relevant to the user/app? In this way, items that the user touches first are updated first.
If your data is geographic, can you send data close to region of interest first?
If your data is products, can you send data that the user has looked at more recently first?
If your data is hierarchical, can you mark root nodes as changed (or again timestamp) and only update sub-trees that have changed?
I would be hesitant in any system, whether networked or even local DB, to attempt to merge updates from a 200K list of items unless it were a very simple list (like a numeric merge sort). It's a tremendous waste of time and network resources, and it won't make your customers very happy.
Don't work on individual items, batch them. Currently you make lots of fetch requests to the context and these take time (use the Core Data Instruments tool to take a look). If you set the batch size for your processing to 100 initially, then fetch that group of ids and then locally check for existence in the fetch results array.
I am trying to clean up some transient property recalculation events. In development I have just used a very broad stroke of updating on almost all changes. Now I am trying to determine the best practice, checking for relevant keys and updating as little as needed. My application objects are loaded with dozens of calculations using child and parent attributes where a single change can result in many cascading recalculations.
I thought I understood fairly well, referring to Apple Documentation here and I have experimented with patterns seen on some other StackOverflow posts such as this one by #Marcus S. Zarra. I see Marcus also posted an article on the subject here.
What I have currently is this:
#pragma mark _ worksheetTotal (transient)
+(NSSet *)keyPathsForValuesAffectingWorksheetTotal
{
// local, non-transient, dependent keys here
return [NSSet setWithObjects:#"dailySustenance", nil];
}
-(void)updateWorksheetTotal
{
NSDecimalNumber *lineItemTotal = [self.items valueForKeyPath:#"#sum.lineHoursCost"];
NSDecimalNumber *total = [lineItemTotal decimalNumberByAdding:self.dailySustenance];
[self setWorksheetTotal:total];
}
-(void)setWorksheetTotal:(NSDecimalNumber *)newWorksheetTotal
{
if ([self worksheetTotal] != newWorksheetTotal) {
[self willChangeValueForKey:#"worksheetTotal"];
[self setPrimitiveWorksheetTotal:newWorksheetTotal];
[self didChangeValueForKey:#"worksheetTotal"];
}
}
-(NSDecimalNumber *)worksheetTotal
{
[self willAccessValueForKey:#"worksheetTotal"];
NSDecimalNumber *result = [self primitiveWorksheetTotal];
[self didAccessValueForKey:#"worksheetTotal"];
return result;
}
I am thinking this is straight out of the documentation use case but my observers are not getting notified of the the changes to worksheetTotal.
When a change is made to a lineItem, that notification is received and responds updateWorksheetTotal is called here. I expect this would trigger another notification by the context that worksheetTotal has now changed, but it does not. I have tried numerous variations and combinations of code I have seen but nothing I do seems to make the NSManagedObjectContext consider my update to worksheetTotal to be a change worth reporting to observers.
What am I missing here?
Here is the relevant code for listening for the change in Parent object.
- (void) objectContextDidChange: (NSNotification *) notification
{
NSSet *updatedObjects = [[notification userInfo] objectForKey:NSUpdatedObjectsKey];
// Next look for changes to worksheet that may affect our calculated fields
NSPredicate *worksheetPredicate = [NSPredicate predicateWithFormat:#"entity.name == %# && estimate == %#", #"LaborWorksheet", self];
NSPredicate *materialsPredicate = [NSPredicate predicateWithFormat:#"entity.name == %# && estimate == %#", #"MaterialLineItems", self];
NSPredicate *combinedPredicate = [NSCompoundPredicate orPredicateWithSubpredicates:[NSArray arrayWithObjects:worksheetPredicate, materialsPredicate, nil]];
NSSet *myWorksheets = [updatedObjects filteredSetUsingPredicate:combinedPredicate];
// These are the relevant keys in this estimates labor and materials worksheet
NSSet *relevantKeys = [NSSet setWithObjects:#"worksheetTotal", #"totalEquipmentcost", nil];
BOOL clearCache = NO;
if (myWorksheets.count != 0) {
for (NSManagedObject *worksheet in myWorksheets) {
NSLog(#"got some");
// Here is where it fails, worksheetTotal is not in changedValues.allKeys. It is an empty set.
// Not sure how that could be when I got it from updatedObjects set of userInfo. How could it have no changed values?
NSSet *changedKeys = [NSSet setWithArray:worksheet.changedValues.allKeys];
if ([changedKeys intersectsSet:relevantKeys]) {
clearCache = YES;
NSLog(#"found some, set to clear cache");
// no need to continue checking
break;
}
}
}
if (clearCache) {
// I would update dependent keys here but it is never reached
}
}
The documentation for the changedValues method says:
This method only reports changes to properties that are defined as
persistent properties of the receiver, not changes to transient
properties or custom instance variables. This method does not
unnecessarily fire relationship faults.
Since your property is transient, it will not appear there. I would suggest using KVO to monitor the changes in these properties.
You are returning before didAccessValueForKey. Store to a variable and return it after didAccessValueForKey like
-(NSDecimalNumber *)worksheetTotal
{
[self willAccessValueForKey:#"worksheetTotal"];
NSDecimalNumber *yourVariable = [self primitiveWorksheetTotal];
[self didAccessValueForKey:#"worksheetTotal"];
return yourVariable;
}
I hope it works.
Edit:
In
-(void)setWorksheetTotal:(NSDecimalNumber *)newWorksheetTotal
consider changing
if ([self worksheetTotal] != newWorksheetTotal) {
to
if (worksheetTotal != newWorksheetTotal) {
I am developing an app for ipad and i am using sqlite sentences (select, update, insert, delete).
I open (sqlite3_open) the database at the beginning and close (sqlite3_close) at the end of each sentence. But sometimes i´ve got the "database is locked" message.
I don´t know what can i do to solve this.
Thanks and sorry for this little information.
If I'm not mistaken , the problem with sqllite is that you can only access it once at a time.
If you have multiple threads, you can run in this situation. Example:
Run method1 (which accesses the database) on thread t1.
Run method2 (which accesses the database) on thread t2 after x seconds.
If method1 is not finished in those x seconds , both methods will access it at the same time.
And , as I said , I know that sqllite does not support this.
You should try to flag the usage of your database and if you want to access it but it is in use , try again after x seconds. Like this:
- (void) generalMethodThatUsesDatabses
{
if(databaseIsUsed)
{
[self performSelector:#selector(generalMethodThatUsesDatabses) withObject:nil afterDelay:5];
return;
}
databaseIsUsed = TRUE; //global bool variable
//your code here
databaseIsUsed = FALSE;
}
Hope this helps. Cheers!
You probably opened the database before using the same simulator.
To conclude all actions to a database and release all resources you always
have to use both (!) statements:
sqlite3_finalize(statement);
sqlite3_close(database);
A good way to solve this is to wrap this into a C++ library. This way, you can can create the library wrapper on the stack. This means that the moment that the function goes out of scope, you can close the connection in the destructor.
(note that I use reference counting for the Objective-C)
For instance:
NSArray* ScoreBoard::getAllScores()
{
ScoreBoard::ensureExistingTable();
//Stack allocated
SqliteWrapper sqlite("Scores.sqlite");
NSArray* result = sqlite.RunQuery("SELECT * FROM Scores ORDER BY ID DESC");
return result;
//after this, the sqlite destructor is called
}
It is very nice that the Objective-C compiler allows you to merge C++. It can be extremely useful.
Also
void SqliteWrapper::Close()
{
sqlite3_close(db);
}
as Vincent has pointed out, you have to finalize the statement. If you want keep the connection open, use finalize after each statement. Close out the connection the moment you are discarding the connection.
This method works for me.
it is use for three methed
1.isert
2.update
3. delete.
-(NSMutableArray *)resultSet
-(void)insertWithTitle:(NSString *)title Body:(NSString *)body
-(BOOL)updateAtIndex:(int)index Title:(NSString *)title Body:(NSString *)body
NSMutableArray *result = [[[NSMutableArray alloc] initWithCapacity:0] autorelease];
FMResultSet *rs = [db executeQuery:[self SQL:#"SELECT * FROM %#" inTable:TABLE_NAME]];
while ([rs next]) {
Record *tr = [[Record alloc] initWithIndex:[rs intForColumn:#"id"]
Title:[rs stringForColumn:#"title"]
Body:[rs stringForColumn:#"body"]];
[result addObject:tr];
[tr release];
}
[rs close];
2....
return result;
[db executeUpdate:[self SQL:#"INSERT INTO %# (title, body) VALUES (?,?)" inTable:TABLE_NAME], title, body];
if ([db hadError]) {
NSLog(#"Err %d: %#", [db lastErrorCode], [db lastErrorMessage]);
Delete record :
BOOL success = YES;
[db executeUpdate:[self SQL:#"DELETE FROM %# WHERE id = ?" inTable:TABLE_NAME], [NSNumber numberWithInt:index]];
if ([db hadError]) {
NSLog(#"Err %d: %#", [db lastErrorCode], [db lastErrorMessage]);
success = NO;
}
return success;
}