How to load large amount of data into CoreData - ios

I have a CoreData database that gets initialized with a local file.
The CoreData schema that looks like this:
Category -->> Objections -->> Responses -->> Evidence
("-->>" means, has many)
Each entity also has a description that can be anywhere from 2 to thousands of characters long, stored in an NSString.
Question: How can I store this data so that it would be easy for someone to edit without having to know a lot about programming? (But also follow best practices)
Currently, I am thinking of these as possible approaches:
1) Store everything in 1 big plist file. This would be about 25 pages long.
2) Separate each entity into it's own PList file, and relate each the values with an ID#, like a Relational Data Base. This would make the files a more manageable size, but you have to keep track of ID#.
3) Same as above, but with JSON

Create a dead simple desktop application that uses Core Data. Let the people edit the file in that desktop application and keep it stored in Core Data. Then when you ship your application you can embed that SQLite file into your iOS application so there is no start up parsing required.
Standing up an OS X app for this that doesn't need to be pretty is dead simple. Takes an afternoon or two at most and saves you a ton of headaches.

run in simulator, do the parsing of the plist OR CSV (might be easier, since thats an excel compatible format) there (in the simulated app) and then copy the resulting DB to your xcode project and ship it.
=> your 'content provider' can work with excel
=> you don't have to ship a CSV (or whatever file you use) since you have a filled DB after you parsed it in the simulator

There is a nice post from Mattt Thompson about Core Data Libraries & Utilities: http://nshipster.com/core-data-libraries-and-utilities/, maybe something fit for you. =]

Related

Offline app data [duplicate]

I have created an app using ionic and cordova and now I want to remake it on iOS. I am working with iOS for the first time, and I cannot figure out how to store data.
For example: I have a form where user has to input some data, but the inputs are not in one view, there must be several views. I used to create empty array and just put everything step by step, but now i can't use same view controller on multiple views. Tried to do it with core data, but core data cannot store arrays. My object would look something like this:
var sampleArray = (
duration: 13,
dayOfTheWeek: Thursday,
personList: [
(name: Rocky,
age: 26),
(name: Ralph,
age:23)
]
)
The question would be: How could I make an input form which would be on several views and where should I store the data, and later I would be able to store all the data into core data?
You can work with persistent data in several ways on iOS.
User Default
This is a tool that is used to store small amounts of information like user settings, preferences etc. Don't use it for data that will scale with application usage (e.g. like notes in notepad app). Documentation will answer all your questions about User Defaults.
Database
You have Core Data as an out of the box solution which is build on top of the SQLite and takes some time to learn, but from my experience it's worth the effort. You are free to use pure SQLite or other database type, but it requires more code and probably custom frameworks.
Text files
You can use arbitrary XML, JSON or CSV files to store your data. Tooling is rich (e.g. NSXMLParser or SwifyJSON just to name two) and if you look on Github, you will find what you need. You can also use build in combination of NSCoder and NSKeyArchiver / NSKeyUnarchiver which are easy to grasp.
Binary files
Finally, for a local storage you can use binary files i.e. images. This is too advanced topic to cover here, but I want to share an example of Open Raster file format. It is used to save informations for drawing apps (eq. GIMP) and inside, it is basically an XML file and a bunch of images compressed to zip and named as .ora file. Creating your own specification for a hybrid format is not that hard.
Network repository
Just to not overlook other methods, you can use remote database API to store data outside of the device, but of course you need your own host and some backend skills.
I hope I didn't miss something important. I just wanted to sum up this knowledge in one place for future reference.
As the first comment says, your question is quite large.
When you say 'one form on several view', I consider it as 'one form per view'.
Keep It Simple S... ;)
(Except if you use page control for your form.)
Basically, you have three ways to store data :
NSUserDefaults :
Store data in Dictionary for later use
File :
Save data to a File (why not .csv like ?)
CoreData :
You can persist arrays as binary data in Core Data
There are numerous tutorials on these topics.
www.raywenderlich.com site is a good one to begin...

Advise where to store data for iOS app

I have created an app using ionic and cordova and now I want to remake it on iOS. I am working with iOS for the first time, and I cannot figure out how to store data.
For example: I have a form where user has to input some data, but the inputs are not in one view, there must be several views. I used to create empty array and just put everything step by step, but now i can't use same view controller on multiple views. Tried to do it with core data, but core data cannot store arrays. My object would look something like this:
var sampleArray = (
duration: 13,
dayOfTheWeek: Thursday,
personList: [
(name: Rocky,
age: 26),
(name: Ralph,
age:23)
]
)
The question would be: How could I make an input form which would be on several views and where should I store the data, and later I would be able to store all the data into core data?
You can work with persistent data in several ways on iOS.
User Default
This is a tool that is used to store small amounts of information like user settings, preferences etc. Don't use it for data that will scale with application usage (e.g. like notes in notepad app). Documentation will answer all your questions about User Defaults.
Database
You have Core Data as an out of the box solution which is build on top of the SQLite and takes some time to learn, but from my experience it's worth the effort. You are free to use pure SQLite or other database type, but it requires more code and probably custom frameworks.
Text files
You can use arbitrary XML, JSON or CSV files to store your data. Tooling is rich (e.g. NSXMLParser or SwifyJSON just to name two) and if you look on Github, you will find what you need. You can also use build in combination of NSCoder and NSKeyArchiver / NSKeyUnarchiver which are easy to grasp.
Binary files
Finally, for a local storage you can use binary files i.e. images. This is too advanced topic to cover here, but I want to share an example of Open Raster file format. It is used to save informations for drawing apps (eq. GIMP) and inside, it is basically an XML file and a bunch of images compressed to zip and named as .ora file. Creating your own specification for a hybrid format is not that hard.
Network repository
Just to not overlook other methods, you can use remote database API to store data outside of the device, but of course you need your own host and some backend skills.
I hope I didn't miss something important. I just wanted to sum up this knowledge in one place for future reference.
As the first comment says, your question is quite large.
When you say 'one form on several view', I consider it as 'one form per view'.
Keep It Simple S... ;)
(Except if you use page control for your form.)
Basically, you have three ways to store data :
NSUserDefaults :
Store data in Dictionary for later use
File :
Save data to a File (why not .csv like ?)
CoreData :
You can persist arrays as binary data in Core Data
There are numerous tutorials on these topics.
www.raywenderlich.com site is a good one to begin...

Simple read-only data storage in iOS (with easy editing and visualization)

I've used archiving, user defaults, and some Core Data in my apps before, but I'm running into a wall re: the best method for my current scenario. I have an app that needs to instantiate some objects from a resource file each time before it's used. An example of one of the objects could be a "MathQuestion" object that has the properties:
questionID (Int) - 2341
questionText (String) — "What is the square root of _?"
questionVariable (Float) – "4"
correctAnswer (a block/closure that returns a Float) – "{return sqrt(value)}".
Ideally, I'd just have something like a spreadsheet with columns for each of these properties and rows for each of the different questions. That way, I would really be able to visualize all the data and make quick changes during development.
My app uses Parse, which is great for visualization and easy editing of values, but for this case, I'd rather the resources remained on the device and not the Parse server. I've been considering Core Data up till this point, but (this could be my inexperience with C.D.), I'm unaware of any way to manually edit the data—and it seems like it may be overkill for what I'm looking for anyways. (I basically just need a way to upload and parse a CSV!) Any advice would be welcome!
I would still recommend Core Data. It is simply the most efficient and scalable mechanism to store and retrieve data.
When I work in your kind of scenario, what I often do is work with a spreadsheet where I can conveniently edit the data. You could edit a CSV version of it that you have included in your target. (If this does not work for you, you could also copy paste into a separate CSV file right into Xcode.)
On every start, you just trash the data store (using NSFileManager when creating the NSPersistentStoreCoordinator). Then you call a method that reads in the CSV and stores it in Core Data.
Once you are done with development, you simply keep the sqlite file (you can include it in the bundle and copy it over, or re-generate it from your CSV the first time the app runs).

Shipping 1.2 million records with an iPhone app

I have a data set of 1.2 million key-value pairs. The keys are a string (a sequence of numbers up to 22 characters in length), and the values are strings.
What's the best way to ship this so a value can be looked up and retrieved quickly?
I suspect a plist is not the way to go for a data set this size.
I have the data set stored in two ways - a CSV, and a mySQL database table with 2 columns. I
ll go forward using whatever method gets the data into the app the best.
Core Data and SQLite are two good options for dealing with very large data sets in iOS. It's not difficult to create a Core Data model for the kind of data you're talking about. You can then copy that model into a little command line program that you'll write to move the data into a Core Data store. You can then move the resulting data file into your iOS app's resources.
A third option, particularly useful if the data is likely to change often, is to build a web service to provide the data from the service. I don't think this is what you're asking about, but it's something to consider if the data set is very large and/or subject to frequent change.
a collection of text files could work well. you can:
divide them into multiple files (e.g. by leading character range).
order pairs appropriately (e.g. by character number)
and quickly/easily read incrementally/portions as appropriate.
balance resources between file reads and memory usage well.
choosing the right encoding for the strings can also help (i'd start with utf8 if it's mostly in ascii).
if distribution size is also your concern, you could compress/decompress these files.
or you can just take this approach and use a custom serialized class to represent a subsets of the collection if that sounds like too much parse and read implementation.
if you're using objc types for storage and/or parsing, then it would be good to keep those files small. if you use c or c++, then it would help to profile the app.
your data set will require up to 30 MB using an 8 bit per character single byte encoding. one large file (again, ordered) which you mmap would also be worth consideration. see [NSData initWithContentsOfMappedFile:path];.
My personal experience is with a plist file that has only thousands of records and I can say that it isn't it so fast. So my options for this amount of data you have are:
A database.
Or if you have a sorting criteria for those keys and prefer plist
files split it in many files and keep a reference dictionary with the start key of every file. For ex. all keys that begin with 'abc' go in a.plist etc.
(I don't know if it's the case with your app but you can consider moving the data to a server and search via a webservice, specifically if your data will grow.)
A sqlite file is probably your best bet. You can create it on the desktop using either the command-line sqlite3 or any sqlite gui. Make sure you index the key column.
Import the csv file as described here: Importing csv files into sqlite
Then just add the database to your project/target. If you want to modify the database at runtime, however, you'll have to copy it out into your Documents or cache directories.
For an objective-c wrapper around sqlite, I like fmdb

plists vs Core Data for holding parameters

I am writing an iPad app that will be expandable with new items via in-app purchasing. For example, my current plan is to have a jpg pattern and a matching plist file with the parameters I need to expand that pattern into a full picture.
The user will select one jpg/png from a list of small thumbnails - the list is held in Core Data - and the app will find the matching plist for displaying the jpg/png correctly. I'll only have about 10 of these open at one time. But I could end up with storing 1000s of jpgs and plists.
Does storage of lots of small files cause app problems?
I'm going the plist way, rather than storing the parameters in Core Data, so that if I need to add parameters later, I don't have to migrate the database, just change the access in code. (And when I'm creating the patterns, it's easier to concentrate on a plist file rather than a Core Data row.)
The app seems to work really well at the moment, but I'm worried about futures...
My app does also use Core Data for other things, so I could change over if the app will get bogged down with number of files.
Thanks.
Saving a large number of small files is not a problem as long as you have a well thought out means of naming and tracking the files.
Remember that the user does not have the same flexibility and ease of file management on a mobile as they do on non-mobile platforms. Designs that work on non-mobiles are unworkable on a device used on the move with one finger.
However, when you say:
And when I'm creating the patterns,
it's easier to concentrate on a plist
file rather than a Core Data row.
... the use of "row" suggest that you haven't fully grasped Core Data's utility. Core Data doesn't use rows, columns, tables or joins. It's an object graph management system that sometimes uses SQL way behind the scenes.
Core Data is designed to handle data in a way that meshes seamlessly with the rest of the object oriented API for the UI and other services. When you use other data management systems like plist, you will most likely end up manually duplicating a lot of Core Data's functionality anyway.

Resources