I'm running into an issue where I need to enumerate an RLMResults collection on a relatively large data set (>7,000 items). I get that realm lazily loads its objects as they are accessed, but the issue I run into is that I need to access each of these items in the collection which causes each of the 7,000+ items to be loaded into memory thereby causing an out of memory error. According to the realm documentation they don't support limiting the results of a query.
An example of something I might need to do is to go through and delete files from the file system, and yes I could query using a predicate and only ask for items that are cached, but in a worst case scenario that query could return all items in the library.
RLMResults<DLLibrary *> *allItems = [DLLibrary allObjects];
for( DLLibrary *item in allItems ) {
// My understanding is that once the realm object is ready, it will be
// lazily loaded into memory. If I have many 1,000's of items in my data
// store this quickly becomes a problem memory wise.
if( item.isCached ) {
[[DLCacheService instance] deleteCachedFileWithDocumentId:item.id];
}
}
The easiest way to mitigate this would be the use of an #autoreleasepool brace to explicitly guarantee the object you lazily-loaded is promptly released once you've finished checking its contents. :)
RLMResults<DLLibrary *> *allItems = [DLLibrary allObjects];
for (NSInteger i = 0; i < allItems.count; i++) {
#autoreleasepool {
DLLibrary *item = allItems[i];
if (item.isCached) {
[[DLCacheService instance] deleteCachedFileWithDocumentId:item.id];
}
}
}
Related
I have two very large NSMutableArray of strings containing more than 40k records each. I have to take each element from one array and sort that string into another array then make a new array which conatins only those records that are in both array. I have implemented the following code which take too much time as well as a lot of memory space also (crash in device). Are there any ways to solve this problem in a more efficient manner.
// _perArray and listArray contains more then 30K records each
for(NSString *gak in _perArray){
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"SELF LIKE[c] %#",gak];
NSArray *results = [listArray filteredArrayUsingPredicate:predicate];
if(results.count>0){
[_resultArray addObject:results[0]];
}
}
Use binary search
index sort one array (the one with less records)
this will enable the usage of binary searching
sort the lesser array just to need less memory for index array
loop through the second array
for each record binary search in the first array
if found add record to output array
do not forget to preallocate the output array to avoid reallocation slowdowns
What it means:
Let N,M be the array sizes where N<=M
naive approach is O(N.M)
this approach (depending on used sort) leads to O(N.log(N).log(M))
Sort both arrays and use single pass incremental search
the complexity will lead to something like O((N.log(N))+(M.log(M))+M)
which in therms of complexity turns to O(M.log(M))
So:
index sort booth arrays
loop through M
increment index for array with lesser record
if match found add it to output array
To be more specific bullet 2 will be something like this (if arrays are sorted ascending):
// variables
string m[M],n[N],o[N]; // your arrays any string type with overloaded <,== operators
int M,N,O; // arrays sizes
int ixm[M],ixn[N]; // indexes for index sort
int i,j;
// bullet 2
for (i=0,j=0,O=0;;)
{
if (m[ixm[i]]==n[ixn[j]]) { o[O]=m[ixm[i]]; O++; }
if (m[ixm[i]]< n[ixn[j]]) { if (i<M) i++; else { if (j<N) j++; else break; }}
else { if (j<N) j++; else { if (i<M) i++; else break; }}
}
If you encode the string comparisons right you can do booth if conditions with single comparison
[notes]
if you do not want to use any of these approaches then there is also another way
you can add flag to one array telling you if it is already used
if it is skip the use of it during your comparisons
that will speed up your naive approach about 2 times
from M.N string comparisons you will need to do just M.N/2
If you have too big data chunks to fit in memory
then segmentate both arrays to some size fit to memory/Cache/...
and first index sort all segments
then do one of the above approach on all segments combinations
the only thing you need to add is checking if O[] does not already contain added string
if you arrays does not have multiples of the same string then this is not the case
otherwise keep O[] sorted or index sorted
and check by binary search ...
this segmentation will be speeded up with the used flag significantly
Always balance the performance issues with the frequency this code path is called. Going the database route might introduce a whole new set of issues to deal with while simply performing the sort in the background, and cutting down the size of the array first might be good enough.
Remove the duplicates first using NSMutableSet
Add all the objects, NSString in this case, to an NSMutableSet. This will eliminate the duplicates. Then sort the remaining objects.
NSArray *array1;
NSArray *array2;
NSMutableSet *mutableSet = [[NSMutableSet alloc] initWithArray:array1];
[mutableSet addObjectsFromArray:array2];
NSSortDescriptor *sortDescriptor = nil; // You'll need to create a sort descriptor.
NSArray *result = [mutableSet sortedArrayUsingDescriptors:#[sortDescriptor]];
// Alternative
NSArray *result = [[mutableSet allObjects] sortedArrayUsingSelector:#selector(compare:)];
I wrote a quick Obj-C test you can try at the command-line.
Run it in the background and return when finished.
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^(void) {
// Perform the sorting
dispatch_async(dispatch_get_main_queue(), ^(void) {
// Tell the main thread I'm done.
});
});
I have an iOS app that I load data from a server into an NSArray and then into UITableView.
This works fine. However, when the user pulls down to refresh the data, I make a new call to my server and grab data again. This new data contains objects that my local data already has and any new objects.
So localArray when populated for the first time will have objects, [A,B,C,D]. I then add a new object server side and refresh the data on the iOS app.
serverData will now have objects [A,B,C,D,E,F] - I need to add, E,F to localArray.
I thought a nested for loop would be the answer, something like this:
NSMutableArray *newItems = [NSMutableArray array];
for (BBItem *itemA in serverDataArray){
for (BBItem *itemB in localArray){
if (![itemA.name isEqualToString:itemb.named]){
//add to a newItems array
}
}
}
However I end up with newItems array containing a lot of duplicates of the same item. What is going on here?
Your algorithm will add an item to newItems if it's not equal to one item from localArray. Therefore, you would end up with lots of identical items most of the time.
You cannot add inside the nested loop. You need to go through all localArray, see that the item is not there, and only then add it. Here is how to fix your code:
for (BBItem *itemA in serverDataArray){
BOOL there = NO;
for (BBItem *itemB in localArray){
if ([itemA.name isEqualToString:itemb.named]){
there = YES;
break;
}
}
if (!there) {
//add to a newItems array
}
}
This is inefficient, because the inner loop goes through all items that you currently have. As the number of local items grows, this loop will get slower and slower. You would be better off maintaining an NSMutableSet of names of local items, because you can check it in constant, rather than in linear, time:
NSMutableSet *localNames = [NSMutableSet set];
for (BBItem *itemB in localArray) {
[localNames addObject:itemB.name];
}
for (BBItem *itemA in serverDataArray){
if (![localNames containsObject:itemA.name]) {
//add to a newItems array
}
}
I have an array full of NSObjects I created called "Questions".
One property of each Question is which level it belongs to.
If the user has chosen to play level 2, I want to get all the Questions that have a .level property of 2. Right now I am looping through all the questions to find the matches, but this is taking ~2 seconds on an iPad 3 / new iPad device. Is there a faster way of dealing with a situation like this?
int goThrough = 0;
do {
Question *currentQuestion = [allQs objectAtIndex:(goThrough)];
if (currentQuestion.level == levelChosen) {
[questions addObject:currentQuestion];
}
goThrough++;
} while (goThrough < [allQs count]);
Your help is greatly appreciated!
If you have to organize the questions by level on a regular basis, then why not keep all of the questions organized by level. Create a dictionary of arrays. Each key if the level and each array is the list of questions for that level. You do this once and it becomes trivial to get the questions for a level.
I dont have access to a mac at the moment but you can give a try to this:
[allQs enumerateObjectsWithOptions:NSEnumerationConcurrent usingBlock:^(id obj, NSUInteger index, BOOL *stop) {
Question *currentQuestion = [allQs objectAtIndex:index];
if (currentQuestion.level == levelChosen) {
[questions addObject:currentQuestion];
}
}
This will use all the cores of your device so it can be twice as fast
You could always use fast enumeration (which, unless you intend on mutating the objects is the fastest way to enumerate a collection). Something like this:
for (Question *thisQuestion in allQs) {
if (thisQuestion.level == levelChosen)
[questions addObject:thisQuestion];
}
}
Since you are not mutating the collection you are iterating through (allQs), this would work fine and be faster than using enumerateObjectsUsingBlock. If you need the index of the array you are iterating through (allQs), then use enumerateObjectsUsingBlock.
I would suggest using the NSArray method enumerateObjectsUsingBlock or one of it's variants. There are even variants that will loop through the array elements concurrently. You'd probably need to use a lock to add elements to your questions array however, since I doubt if NSMutableArray's addObject method is thread-safe.
You should probably test a non-concurrent version against a concurrent version with locking to see which is faster. Which approach is faster would depend on how many of the objects in the allQs array belong to the current level. If only a few belong, the code that asserts a lock won't fire very often, and the benefit of concurrency will outweigh the time penalty of asserting a lock. If most of the objects in the allQs array match the chosen level, code will end up spending a lot of time asserting locks, and the concurrent threads will still waiting for other threads to release a lock.
Modified code might look something like this:
single-threaded version:
[allQs enumerateObjectsUsingBlock:
^(Question *currentQuestion, NSUInteger index, BOOL *stop)
{
if (currentQuestion.level == levelChosen)
[questions addObject:currentQuestion];
}
];
Concurrent version:
[allQs enumerateObjectsWithOptions:
NSEnumerationConcurrent
usingBlock:
^(Question *currentQuestion, NSUInteger index, BOOL *stop)
{
if (currentQuestion.level == levelChosen)
#synchronized
{
[questions addObject:currentQuestion];
}
}
];
Actually, now that I think about it, you would likely get still faster performance by first doing a concurrent pass on the array using indexesOfObjectsWithOptions:passingTest. In that pass you'd build an NSIndexSet of all the objects that match the current level. Then in one pass you'd extract those elements into another array:
NSIndexSet *questionIndexes = [allQs indexesOfObjectsWithOptions: NSEnumerationConcurrent
usingBlock:
^(Question *currentQuestion, NSUInteger index, BOOL *stop)
{
return (currentQuestion.level == levelChosen)
}
];
questions = [allQs objectsAtIndexes: questionIndexes];
Another poster pointed out that you are better off breaking up your arrays of questions up by level in advance. If that works with your program flow it's better, since not filtering your array at all will always be faster than the most highly optimized filtering code.
There is a simple answer that seems to be missing. If you want to filter the objects of an array to only have certain ones left, -filteredArrayUsingPredicate: is what you would want. It can be done exceptionally simply.
NSPredicate *p = [NSPredicate predicateWithBlock:^(Question *aQuestion, NSDictionary *bindings){
return (aQuestion.level==2);
}];
NSArray *filteredArray = [originalArray filteredArrayUsingPredicate:p];
This may be out of the scope of what Objective C can do.
It involves doing something to a named variable (e.g. 15 different operations), and then doing those 15 operations on a similarly named variable, and so on for let's say 10 variables.
Suppose I have a set of 10 NSMutableArray;e.g.
NSMutableArray *a =[[NSMutableArray alloc]init];
...
NSMutableArray *j =[[NSMutableArray alloc]init];
and I do 15 operation on them to get 15 different variables (also arrays or more accurately NSMutableArray's). Let's say the first thing I do is to make a computation the is the product of a thru j:
for (int i=0;i<[a count]; i++)
aProduct = /* some operation upon NSMutableArray a*/...;
...
for (int i=0;i<[j count]; i++)
jProduct = /* some operation upon NSMutableArray j*/...;
Let's say the 15th thing I do is to make a computation the is the selection of a thru j:
for (int i=0;i<[a count]; i++)
aSelection = /* some operation upon NSMutableArray a*/...;
...
for (int i=0;i<[j count]; i++)
jSelection = /* some operation upon NSMutableArray j*/...;
Right now I'm using switch / case for the coding, which involves a lot of thinking, initialization, and even caught that I made a typo in one "case" statement (leaving out a break) and when I copied the code, that mistake propagated down.
So is there a way to avoid having 15 sets of "switch / case" code, and doing it instead "dynamically" (if such a thing exists in Xcode and iOS) and avoiding creating 10 times 15 variables?
Or that I create and declare the 150 variables, but do so in some array, perhaps by using the addressing (&a thru &j) of the variable?
This is getting very complex, very quickly, and there are times when some of the 10 arrays have many elements, and other of the same arrays are null (e.g. a may have 20 elements, and j has null elements)
PS: (Postscript at 6:56pm NYC time 19Mar2012) I was actually hoping that I could use some variable name and then prefix or suffix it with the letters a thru j; eg:
ProductA = [[....;
SelectorA=....;
and have the subroutine just take a letter from A to J via enum to translate that to 0 to 9.
I'm not rejecting the solution proposed earlier by Peter Cetinski; I just haven't tried to implement it yet.
If you need to maintain the computed values for each array, then try using array of arrays
NSMutableArray *arrOfArrays = [[NSMutableArray alloc] init];
[arrOfArrays addObject:a];
.....
[arrOfArrays addObject:j];
for ( NSMutableArray *arr in arrOfArrays ) {
for (int i=0;i<[arr count]; i++) {
aProduct = /* some operation upon NSMutableArray arr*/...;
.....
aSelection = /* some operation upon NSMutableArray arr*/...;
//do something with your computed values
}
}
I have a GPSTestViewController class that has a MKMapView with added MKAnnotations (stored in a class called Bases). I'm trying to continuously update the coordinates for the MKAnnotations (with the updateBaseInformation method in Bases) so the bases is moving on the map. The update is invoked from the GPSTestViewController method locationUpdate (since it's called every second):
- (void)locationUpdate:(CLLocation *)location {
NSLog(#"locationUpdate");
self.cachedLocation = location;
[self centerTo:cachedLocation.coordinate];
//Trying to update the coordinates every second
[bases updateBaseInformation]; <--Program received signal: “EXC_BAD_ACCESS”
return;
}
But then I get the following message:
[bases updateBaseInformation]; Program received signal: “EXC_BAD_ACCESS”
Bases.m contains the following code (and even when some code is commented it crashes):
- (void)updateBaseInformation {
NSLog(#"Updating base information");
for(MyAnnotation *a in bases)
{
//CLLocationCoordinate2D c;
if([a.type compare:#"friend"] == 0)
{
//c.latitude = a.coordinate.latitude+0.001;
//c.longitude = a.coordinate.longitude+0.001;
//a.coordinate = c;
}
else if([a.type compare:#"enemy"] == 0)
{
//[a setCoordinate:CLLocationCoordinate2DMake(a.coordinate.latitude+0.002, a.coordinate.longitude+0.0012)];
}
}
}
My guess is that I'm accessing the objects that are already accessed somewhere else and that causes the EXC_BAD_ACCESS. I have spent many hours on searching, but without results. Why do I get this error and how should I do in order to make the Annotations move around on the map?
I have uploaded the complete project to (link removed).
Solution
The problem is now solved. The problem was that the array that holds the Annotations was autoreleased. So I changed the allocation from
bases = [NSMutableArray array];
to
bases = [[NSMutableArray array] retain];
Thanks in advance.
My guess is that I'm accessing the objects that are already accessed
somewhere else
You are probably accessing an object that no longer exists in memory - You need to check you are not deallocating an object which you are subsequently accessing. You should turn on Zombies for your scheme which will allow you to see when you are accessing a deallocated object by effectively keeping an object's memory under watch after the object has been deallocated. You can switch them on here: Scheme/edit scheme/enable zombie objects.