I'm making a statistical analysis app which needs to store large amounts of data locally. To give you an idea of what I'm talking about, I've made this illustration (not exactly the right information, but very similar):
The app will keep track of several thousand destinations, with the destinations population, temperature, number of cars etc. These numbers will be represented in a graph so that you can look at the numbers "development" over time. This will go over a long period of time, in other words: thousand of dates for each data-type for each thousands of cities.
To achieve this I need to save large amounts of data, and it is preferred to be done locally (is that crazy?). I'm stuck between digging deep into the foundation of Core data, or using my already decent skills in SQLite.
If you suggest I should use SQLite, could you refer to how you would implement this into your app (some code perhaps?).
If you suggest I should use core data (mainly for performance), please show me how you would implement this type of data model with entities, attributes, relationships etc. I can imagine using dictionaries saved in the core data would be a good solution?
Thank you in advance.
If you're going with SQLite with Swift - I highly recommend using this project. I am using it in my current project and it is absolutely fantastic and works perfectly (I'm not affliated in any way with the project or author). You actually drag that project into your project and it becomes a subproject, then you just set it up as 1. target dependency, 2. framework to link with, 3. copy framework (build phases), and it just works. Then you can handle your database with brilliantly constructed Swift interfaces rather than ugly cumbersome libsqlite calls.
I have used it for modest amounts of data. A few databases and multiple tables. Clean and intuitive. So far I haven't found a single bug of any kind. And Stephen Celis, the author, was responsive when I asked a question about a feature that wasn't documented (but actually is present and works, it turns out). It's a prodigious effort.
Its so clean and tightly integrated with Swift that, if I didn't know better, I'd think Apple itself added SQLite support to the Swift language.
https://github.com/stephencelis/SQLite.swift
Core Data is an object persistence model-- and there's your answer really, because every object has a little overhead, so having thousands of objects in memory at one time is problematic. If you already know SQL then that's another plus.
(Discussion of overall merits of core data is outside the scope of this question. The "music" app pulls it off using core data with thousands of items, I think, but because it only needs to display a subset of items. Map Kit drops down to C, and quite handles itself impressively with tens of thousands of single-digit byte items, which you can see running instruments with a Map Kit app.)
I've used SQLite in iOS and it's not a problem, being C-based and Objective C being a strict superset of C. There are various "wrappers" but look at these carefully, in case they take you back to every-row-an-object. I didn't use one and found the SQLite C setup to be tricky but fine. EDIT: I used this tutorial, which is dated now for the Objective C but a clear overview of how to integrate SQLite.
The only catch is the bundle/documents distinction in iOS will catch you if you ship with large amounts of data and want to save to that database: you can't modify files in the bundle, so you need to create or copy an SQL database to the documents folder 1 time, maybe on first app launch, to use it. Since you can't delete from the bundle, now you have your database twice. A problem inherent to the way iOS is set up.
Related
I'm trying to write a iOS note taking app that is blazingly fast for a large number of notes and that syncs without ever blocking the UI. (Don't worry, it's just a learning project, I know there are a billion note apps for iOS). I have decided to use Core Data (mostly because of the excellent posts by Brent Simmons about Vesper). I know UIManagedDocument can do async reads and writes and has a lot of functionality built in, so I'm wondering if there is any information on which would be faster for a fairly simple notes app. I can't really find a lot of information about people using UIManagedDocuments for anything other than a centralized, basically singleton, persistent store. Is it suitable for 1000s of documents? Would it be faster or slower than just a database of NSManagedObjects? It seems like most information I can find about Core Data is oriented towards people using NSManagedObject, so any information about UIManagedDocuments being used in production apps would be really helpful. At this point, the only thing I can think of is to just write the whole app both ways, load 10,000 notes into it, and see what happens.
Update
To clarify, I'm not learning iOS development and Objective-C, the "learning project" mostly means that I've never used Core Data and would like to learn how to write a really performant Core Data application.
UIManagedDocument is designed/intended for document based applications. One UIManagedDocument instance per document. If you are not building a document based application then you should not be using UIManagedDocument.
Everything that people like about UIManagedDocument can be accomplished with very little effort using the Core Data stack directly. UIManagedDocument abstracts you away from what your persistence layer is doing. Something you really do not want.
If you want a high performance Core Data application you do not want to be using UIManagedDocument. You will run into issues with it. It will do things at random times and cause performance issues.
You are far, far, better off learning the framework properly.
In the case of Vesper, those are not documents; they are too small. Think of documents as Word files, or Excel files. Large complicated data structures that are 100% isolated from each other.
Also, whether you use a UIManagedDocument or not, you will be using NSManagedObject instances. NSManagedObject, NSManagedObjectContext, NSPersistentStoreCoordinator are all foundational objects in Core Data. UIManagedDocument is just an abstraction layer on top.
Finally, Core Data is not a database. Thinking of it that way will get you into a jam. Core Data is an object model that can persist to disk and one of the persistence formats happens to be SQLite.
Update (Running Into Problems)
UIManagedDocument is an abstraction on top of Core Data. To use UIManagedDocument you actually need to learn more about Core Data than if you just used the primary Core Data stack.
UIManagedDocument uses a parent/child context internally. Don't understand that yet? See the point above. What it also means is that your requests for it to save are "taken under advisement" as opposed to being saved right then and there. This can lead to unexpected results if you don't understand the point of it or don't want it to save when it feels like it.
UIManagedDocument uses asynchronous saves and at most you can request that it save. Doesn't mean it is going to save now, nor does it mean you can easily stop and wait for the save to complete. You need to trust that it will complete it. In addition, it may decide to save at an inopportune moment.
When you start looking for performance gains with Core Data you tend to want to build the stack in a very specific way to maximize the benefit to your application. That is application dependent and with the abstracts in UIManagedDocument you get limited very quickly.
Even in a situation where I was building a document based application I would still not use UIManagedDocument. Just to much behind the curtain.
Performance is likely to come down to how the rest of your code is implemented, and not necessarily the difference between a UIManagedDocument or a NSManagedObject.
Think of UIManagedDocument as a specific niche implementation of CoreData, that already has the parts of CoreData you want baked into it's structure to save you (the developer) a little bit of time writing code. It's been purpose built for handling UIDocuments, and multi-threading.
Under the hood, it's likely that UIManagedDocument is using CoreData as well as you would (assuming you know what you're doing), but it's theoretically possible you could cut some corners by knowing the exact and inane details of your implementation.
If you're new to CoreData, or GCD and NSOperationQueue, then you'll likely save a ton of developer time by leveraging UIManagedDocument instead of rolling your own.
A very apt analogy would be to using a NSFetchedResultsController to run your UITableView, rather then rolling your own CoreData and UITableView implementation.
If you're new to objective-C, I'd recommend you bang out something functional with UIManagedDocument at first. Later you get lost in the weeds of dispatch_asynch() and NSFetchRequest, and pick up a few milliseconds of performance here and there.
Cheers!
Just store a plain text file on the disk for each note.
Keep a separate database (using sqlite or perhaps just -[NSDictionary writeToFile:atomically]) to store metadata, such as the last user modification date and the last server sync date for each note.
Periodically talk to the server, asking it for a list of stuff that has changed since the last time you did a sync, and sending any data on your end that has changed in the same time period.
This should be perfectly fast as long as you have less than a million note files. Do all network and filesystem operations on an NSOperationQueue.
Any of you out there got the answer to what model is more trivial to iOS cloud data driven apps?
Restkit + RESTful API or TouchDB + CouchDB?
There is really hard to answer your question without know what you want to do. You can certainly make both works for different use cases.
There are a couple of things to consider:
If offline mode is important.
TouchDB is a full functional nosql datastore running on the device, and it allows user to read and write data even without connection. Restkit needs a connection to fully work.
Size the dataset to replicate
TouchDB will replicate data to the device, and it is easier if you have relative small dataset. The size is measured by how many documents in your database, and also the size of documents.
Also, most of time device only need to do a full replication when app starts (initial replication), so you can get around this (embedded most of data into the app apk itself for example) and only replicate delta.
By the way, you can certainly use both, and get the benefits of both.
Employing CouchDB+TouchDB completely takes the hassle of sync away from you. You don't need to care about sync, it just works. You get a notification upon sync, update your UI, that's it.
Replacing the Core Data stack with TouchDB is also fairly easy. The model objects basically stay the same, only that they now inherit from CouchModel instead of NSManagedObject. It's nearly trivial.
Querying is slightly different from Core Data. You define a set of views (indexes) that slice and sort your data along different criteria, and then query these indexes with a start and end key. So, there's no explicit query language, but that's no inconvenience, really.
I've moved a Core Data app to TouchDB, and it was completely painless. In about 3 days, I had CRUD and sync up and running.
I have a new iPad application that is data driven (read, delete, create, update) and requires data persistence on iPad devices that are installed the application. There are about 10 tables for that application; those 10 tables are relational in some ways (one to many; many to many; and stand alone). I am new to data driven application with iPad.
I do not know whether I should follow the Core Data approach or SQLite approach. Which approach is the better and real-world oriented? Please advise. Thank you very much.
My advice is try both; sketch out two simple versions of your app, one with each implementation. Of course Core Data uses SQLite as storage behind the scenes, but it is much more than that. In my view it's a very complex, clunky beast to wrangle, but it is about modeling and persisting objects that have attributes, it can help a lot with consistency when you delete an object that's involved in a relationship, and the NSFetchedResultsController is helpful for populating a UITableView in a memory-efficient way. With SQLite it's just a database and that's all it is; you have to do all the work, but there are nice Objective-C front ends, and you are completely in control rather than having to do everything in Core Data's very strange way.
There's a short section of my book that demonstrates a tiny Core Data app, and even though it's tiny you can see I spend a lot of time banging my head against the wall of Core Data's special way of doing things:
http://www.apeth.com/iOSBook/ch36.html#_core_data
The previous section has a few lines showing how simple it is to fetch data from a SQLite database with a nice Objective-C front end:
http://www.apeth.com/iOSBook/ch36.html#_sqlite
I recommend that you read a book about Core Data before you commit yourself to it. There are several good ones. I'm not saying it's bad, but it is not at all a beginner technology and you really need to know what you're in for.
You are on the right track with Core Data in my opinion, what you need to ask yourself is how your data is being used against the benefits of each approach. Core data lets you update and write data using data types like NSDictionaries which do not require any outside of the box techniques to handle and write data like SQLite does. SQLite does work and can work how you want it to, but in my opinion using a third party wrapper is just one more step to take to achieve something that is already native.
I'm a new iPhone dev and I integrated a team of iOS developper, in the domain of education. First, I thank you for your help.
Here is my question : The lead developper told that for the project (iOS 5), they will not use Core Data to deal with the database of the app, but they will use their own SQLite libraries.
I have been told by the lead dev that Core Data will not fit in that project, because of the data they are dealing with... No more details
Since Core Data uses SQLite, (and so will the library they created), do you have any ideas why Core Data could be a problem (in general)
I really thank you if you have any clue.
EDIT :
I've just learnt that the lead is creating an sqlite database and we are going to use it in the app. And the fields contains a lot of html (a lot of special characters like \ or & or <), and we are going to display it in some UIWebViews. Is he right when he says that those special characters can turn wrong with core data ?
The only reason I can think of is cross-platform portability. If you use a manually-generated schema, and perhaps even provide common library code, then that database schema/library can be used on non-Apple platforms.
It's hard to guess at other people's reasons but.... The purpose of Core Data is object persistence. The fact that it can use a database as a storage mechanism is an implementation detail. For problems that are specifically object-oriented in nature, it works very well.
If you are dealing with a problem that is more suited to solutions that are about rows, columns, and tables, it isn't necessarily a good fit.
I like using objects and have a lot of experience with Object/Relational mapping in other systems, so I tend to be pro-Core Data. People with a database-only background probably have a different view.
I've developed quite a few local apps, however this is the first time I'm introducing networking (more specifically posting to, and reading from a database). I am receiving back a JSON object from the database but I am currently using arrays and dictionaries. The objects do have relationships to each other, and I was wondering whether CoreData is the way to go. If so, do I just replicate part of the database I wish to be made viewable in the app and store it in my CoreData model? Are there any tutorials out there for this?
Also, just as a side note, I've also included Facebook integration with which I download the users list of friends. Would CoreData be good for storing this kind of information too? Or would I be better sticking with dictionaries?
Thanks in advance.
Based on my experience (other could say different things) Core Data is the right choice but its adoption could depend on the time you could dedicate to it. At first could be very complicated but I think you could take advantage applying the knowledge in some other projects.
Out of there there are some tutorials or books on Core Data.
First I suggest to read about core-data-on-ios-5-tutorial-getting-started. In the site there are, I think, other tutorials. Then, you could try to read a post on core data I've written some time ago: Mapping Business Objects with Core Data in iOS. Also Apple doc is your friend. So read the Introduction to Core Data Programming Guide to have the details that are going on.
If so, do I just replicate part of the database I wish to be made
viewable in the app and store it in my CoreData model?
Yes, just a part. You can create a minimal model that includes the key parts you need to have in your device. What I want to highlight is that you don't need to take care of normalization concepts when you deal with Core Data. Yes you could, but in CD you deal with objects and it's important to make attention to memory consumption (the framework helps you) and performances.
Would CoreData be good for storing this kind of information too? Or
would I be better sticking with dictionaries?
With CD you could take advantage of NSFetchedResultsController. A NSFetchedResultsController objects is optimized to work with tables and it used to display data also in batches. Through this component you can deal with a lot of elements (say friends in Facebook) without overload the memory. See core-data-tutorial-how-to-use-nsfetchedresultscontroller.
If you want to know something else, let me know.
Hope that helps.