I've just seeded 25k objects into Core Data and therefore will need to perform a comparison of some of the objects before they're fetched; though the Haversine formula. I saw a similar question that advised that all objects must be fetched and then compared, however that will be resource and memory expensive.
Please can you tell me how I can perform this custom comparison through, I assume, a predicate in this case?
Use the predicate to find objects inside a simple lat-long box (min and max values) and then apply any finer grained distance calculation, filter and sorting on the fetch results.
Related
To perform geoqueries in Firebase or Firestore, there are libraries like GeoFire and GeoFirestore. But to sort the results of that geoquery by distance, the entire dataset must be read, correct? If a geoquery produces a large number of results, there is no way to paginate those results (on the backend, not to the user) when sorting by distance, is there?
Yes, in order to sort by distance you must read all results that fall into the Geoquery range.
The reason for this is how such queries work: they return a set of documents that are within a range of geohash values, which is not necessarily the same order as by their distance to the center of the query.
This also means that there is no way to do meaningful pagination in a list of documents that are ordered by their distance, since you need to read all results anyway. The best I can think of is implementing the Geoquery in Cloud Functions, so that you can do the sort/filter there, and only return the page-full of results to the client. While this doesn't save on your cost (as you're still reading all documents in the range), it will save bandwidth in sending documents to the user.
To learn more about how such geoqueries work, which explains why they can't be optimized the way you're looking to do, have a look at the video of my talk here or this article+shorter video on Jeff Delaney's site.
It's been a while since I've touched Core Data, and I'm now trying to move app on to it. The data is downloaded as JSON from a server and is stored as a few flat files.
When a user interacts with the app, I load the entire data set for a particular screen (hundreds of entries) into memory at once. The user can then filter the records (in my case, they're venues) by distance to them. (Location is provided by Core Data.)
The user can also filter by several other properties of the Venue managed objects, as well as enter a search term.
I'm looking to combine all of these filters to allow the user to search, sort, and browse the data. I've written code to do all of these things using an NSArray in memory, but performance is poor.
Do I want to use NSPredicate here? I know I can use it for filtering by properties of the NSManagedObjects, but for more transient attributes, such as venue's distance from the user, I'm not sure how this would work.
What are the general steps to achieve this with Core Data?
As far as distance and core data:
Store lat and long independently, query everything in the "square" that has a center at the user, and a inner radius of the search radius (aka, the square that the circle fits inside)
Throw out results in the corners manually in memory
As far as the total query: some and rules in your predicate will do wonders
I use a quadtree for geographic queries. It is much faster than scanning an array. My code is derived from the Wikipedia entry on Quadtrees. There are several blog posts about using quadtrees on a MKMapView. NSPredicate is not a Core Data specific filter. It can be applied to any NSArray. If your data set is large, several hundred points, it will likely run faster than your loop.
It turns out that I do want NSPredicate for most of my filtering criteria, but any way I slice it, Core Data can't sort by location.
Filtering alphabetically can be done with NSFetchedResultsController, and I can also filter by a "type" property and search term.
As noted elsewhere on the internet, a block based NSSortDescriptor won't do what I want, because Core Data requires sort descriptors that can compile down to SQL statements. Further, Core Data doesn't know how to do SQL location calculations.
I need to count no. of objects from a collection in core data of that satisfy a certain criteria.
(eg. count no. employees with distinct departments).
There are two solutions to my problem:
(1) Fetch the collection in only one request and filter the array locally
for each department using NSPredicate
(2) Execute multiple NSFetchedRequests directly on the data
Question is which solution will be fastest and take up least amount of memory given this is only for instrumentation purpose and is of no importance in the app in terms of behavior/UI.
Counter Question : If it is (1) - which is the best way to filter the array? manual looping and counting or NSPredicate?
P.S:
a. Names of departments are known to me. (its actually an enum)
b. collection is small - will be max 50
1 is fastest and takes most memory.
2 will use the least memory but may take longer.
However, this is not always true. In the event that your number of individual fetch requests will contain many of the same employee data sets that other fetch requests will return too, then it may even be the other way around. But as you are fetching for departments, that will not be the case.
For a small collection it may not be much of a difference anyway.
Counting question: This, too, depends. However, I'd go for the predicate as that is save for future use if the collection grows.
I have records in my Core Data store with latitude and longitude properties. Now I need to find the record that is closest to a given location.
In SQL I would do it like:
select t.lat
,t.lon
,(t.lat-varLat) + (t.lon - varLon) diff
from table t
order by diff ASC.
given varLat and varLon are the coordinates of the location I want the closest record to. However I have no clue how to do this in Core Data with an NSPredicate.
The worst thing I can imagine is looping fetchrequests with a predicate that expands a 'search range' around the given (varLat,varLon), and using
longitude BETWEEN %# && latitude BETWEEN %# with 2 arrays (each having center+ and center- radius as array members) as parameters, each time expanding this range until I get a hit.. however, this is far from optimal.
Thanks for your advice.
First, your query is not correct, with an error of up to app. 40%. Your calculation just sums up the lat and long differences. The actual difference, assuming a 2D surface, is sqrt((lat-varlat)^2 + (lon-varLon)^2). On the actual surface the geometry is more complicated, but the difference negligible in most cases.
In Core Data it is not uncommon to fetch all instances and then iterate through them in a loop. The reason is that Core Data uses a mechanism called faulting that will allow you to retrieve a large number of objects without too much memory overhead.
I only have a limited experience in using NSSortDescriptor.
It was sorting on one key and it worked fine.
But here is what I need now, I have a set of pairs of numbers, for example :
{(2,3), (44,5), (6,17), (33,7) ……(173,21)}
I want to sort the pairs (x,y) according to the value of a given function myfunction(x,y).
There is the trivial idea of making triplets (x,y,z) where z would be the computation of myfunction(x,y) and then sort the set of triplets, but this not what I want.
Is there a proper way to use NSSortDescriptor to do what I need?
Thanks for any information.
Unless you are using this sort descriptor with Core Data using a SQLite store, you could create a transient attribute on the object where the attribute represented z, the computation of your function. You could then sort on that transient attribute and it would produce the results you want, without storing the z values.
However, if you are using Core Data with a store like SQLite where the entire store contents are not read into memory, you cannot use transient attributes to sort on and you would have to either store your z values actually in the managed objects in order to achieve what you are describing, or read all of your managed objects into an array, and then sort that array of objects on the transient property... which would work OK if you only had a limited number of managed objects, but not so well otherwise.