iOS - 50MB file with 1 million entries - ios

I have text-delimited file with 1 million entries, e.g.
A;B;C;D;E;F;G......n
I'd like to create an iOS app that will able the user to sort and filter columns.
What is the best way to store this info locally? Will this be too much for iOS to handle?
Would I be better creating a Web service?
Thanks

Use sqlite3. Do proper indexing.
If your data is read-only, performance will be far more manageable. If you're updating the data very frequently, you may run into performance issues.
Make sure to experiment with different generations of hardware and restrict app to only the ones that can support.
It should be simple enough to experiment. I've had no trouble with database sizes ~8Megs on 2nd genration iphones/iPod touch (~10-20K entries). Latest generation hardware and software should be able to support much more. I'm not sure how much more.
Make sure to experiment and report your findings here.

Related

Best practice for storing and updating images in Xcode project

I am building my flash card iOS app for reviewing my learning Japanese using SwiftUI language.
The problem is how to storing and updating my images(>500 images). Please help me, any suggestion is appreciated, thanks for reading my post.
I think you're asking about how to manage 500+ images in an Xcode project. You could just add all the images to your project and load them as you would any image. You could use asset catalogs, which have the advantage that they let you store different versions of a resource for use on different devices, and only the ones needed for the device the app runs on will actually be installed on the device. See How Many Images Can/Should you Store in xcassets in Xcode? and Asset Catalog Format Reference for more information about asset catalogs. But any way you slice it, managing 500+ images is going to be cumbersome. There's probably a better way...
Managing all those images in your app isn't just a problem for you as the developer; building them into the app will also create problems for the user. Even if each image is relatively small, having hundreds of them in the app will probably make the app huge. That means it'll take a long time to install, and the app will use a lot of storage on the device. Every time you release a new version of the app, with more words, or even just to fix a few small bugs, the user will have to download all that data all over again.
Instead, you should consider building an app that can fetch the data it needs from a server. Ideally, you could apply that approach to all your app's data, not just the images. Maybe you'll organize your flash cards into sets of a few dozen, so that you can fetch a set of cards and the associated images pretty quickly, and sets that the user hasn't used for a while can be removed to free up space on the device. You'll be able to update a set of flash cards without having to update the app, and when you do update the app your users won't have to download all the data all over again.
You've said that you're a beginner, so this approach might seems very difficult. That's OK, you can start with a simpler approach and then improve as you go along. For example, you might just put all the images on a server and fetch them one at a time as you need them. Your flash card data file could contain just a dictionary with words and the URLs associated with those words. There are lots of examples of loading an image from an URL here on SO and elsewhere, so I'm not going to provide code for that, but it won't be hard to find. The earlier you start thinking about how to design your app so that it can scale as you add more and more words, the easier it will be to maintain the app later.
500 images can have a huge size. Applications that published on Appstore have size limit and Apple does not recommend to make big apps.
Store them on server and load needed images on fly. Also you will get possibility to update your images, remove add new.
If you don't have a backend, you can use something easy and free (Firebase storage for example) or with minimal code writing on AWS.
If you need to keep them on device - store them as files in the Documents or another apps folder, do not use CoreData for it (you can keep only the list of names/urls in database).
After loading image to be displayed for user, you can prefetch next bunch of images.
Use Alamofire, or SDWebImage to load images from network (I prefer last). These frameworks can do many useful things with images.
To load images:
you can have a list of your images (just list of the names and urls)
or
you can know only path and names pattern and generate links dynamically (like https://myserver/imageXXX.png.

Xcode 8/Swift 3: make API information available offline? [duplicate]

This question already has an answer here:
How to make offline database for my app?
(1 answer)
Closed 6 years ago.
I'm currently working on an application in Xcode 8/Swift 3 which runs through APIs. Essentially, I'm parsing information using SwiftyJSON from my MySQL database which keeps the content current and easily updated.
To keep it so the content is also available offline, I'd like to introduce a facility where the data is downloaded and stored on the phone so it is available in "offline mode".
I know it's a completely open question but can anyone point me in the correct direction of how I could make this JSON information available offline? I've tried searching the net with no success.
I know it's not Swift, but the absolute master of this has recently open sauced his master piece: Dash for iOS.
Reviewing what he's done to get rapid scrolling and searches might give some deep insight into how to best do this as done by someone with (arguably) more experience in this area than anyone other than Apple:
https://github.com/Kapeli/Dash-iOS
I will prefer here 2 option either I will go with 1.SQLite DB or 2. NSURLCache
For SQLite DB you can use FMDB wrapper-https://github.com/ccgus/fmdb
For NSURLCache check this link Best way to Cache JSON from API in SWIFT?
If you just want to save json then go with NSURLCache for offline mode.
Achieving offline for iOS is having two best paths they are CoreData and SQLITE. As per the definition of CoreData suggests it is a Model layer of the project. It comes with less efforts on developer side. Bit contrast SQLITE having the same way but little efforts on it.
In my project we are using the CoreData for offline maintenance. Really we have few concerns on the Relational data fetching, Although there is a Predicates representing CoreData for the same still it is limited to some part. These type of situations SQLITE is really a life saver. We can easily fetch the records with simple JOIN commands.
Conclusion:
If you have more complex data relations it's really better to go with the SQLITE, Apart from CoreData is best choice.

A/B testing(show new feature only for 50% of users)

I'am creating a new feature for my iOS app. After I publish the app I wants to show the new feature only for 50% of the users, so I can do some testing which version makes more orders. I have no idea how to do it without using some third parties like Optimizely.
Also is it possible to do this using Google Tag Manager(GTM).
So can someone please help me to figure this out.
Thank you very much for your time.:)
It’s hard to do it on your own, though not impossible of course: Optimizelys of the world are just programs. You’ll need to solve these problems:
Targeting: Some algorithm that will assign user session to either control or (one of) treatment(s). This has to be random, of course, or you may as well stop there.
Routing: Send sessios to the targeted experience.
Logging: You’ll need to intelligently log events from sessions as they traverse their targeted experience. These may be many, so be careful not to add latency to your app path. Your statistical analysis will be based on these.
Experience stability: how do you ensure (if you do) that a returning user sees the same experience he’s already seen.
Note as well, that Optimizely will only help you if all your changes are on the device and not on the server. If you need to instrument server changes as well, you’ll have to look into Sitespect or Variant.
I finally figured out how to do the A/B testing with 'Google Tag Manager'(GTM).
In GTM you can create a variable called 'Google Analytics Content Experiment'. With this variable you can select how many percentage of users going to see each Variation(your experiments). You can create up to 10 variations for single experiment.
GTM is so cool and powerful. GTM contains so many features that could save lot of time and I totally recommend it for anyone who is going to do A/B testing.

Fastest and light way(Sqlite or core data or plist or csv or flat file)

I have a simple problem to solve but want to know which is better pattern to use and understand the reason for the choice.
Problem :
I want to create a utility which developers can use to check whether the feature should be enabled/disabled depending on the server package version.
eg : server package versions like 10.234, 11.1 etc and client versions 9.3,10.2 etc
Validation on client to see min version on server package for feature to be enabled/disabled.
example data would be like "search lookup feature >= 10.234". And sometimes complex situation need to even check client version whether the feature should be supported.
edit:
Note: Application is very huge and memory is full for most of the time. (thousands of records of organisation data.) So memory is bottle neck.
Just it even flashed to mind to used macro as to do all comparisons and returning value.
I think Plist would be heavy as all the objects would be in NSDictionary and even to access one object, I'll be holding all data.
I want to reduce memory overhead and comparisons too.
For Light data, Use NSUserDefaults or PList. SQLite and Core Data is overkilled.
Updated Answer for Updated Question:-
For server package versions/your app version or other light weight data, you can always use NSUserDefaults or PList.
For records of organisation data, you might want to consider Core Data.
Yes, you can use 2 different types of persistent storage inside your app.
If your data is never going to change, or grow you better use NSUserDefaults, plist or csv file. If you think that user should have the ability to change or add new entries to this data file you better go ahead with CoreData
Plists, csv, and flat files have low overhead for small to medimum amounts of data (and for data where you use all or nearly all) of it.
Systems that only use some of the data on any given run AND have large amounts of data AND have the data structured so SELECT can use an index can be faster (lots and lots faster) with a database (SQLite, or CoreData with the SQL Persistant store).
As a gut feel I expect your problem could be handled well by SQL, but only if you are talking about 1000s of configurations not 10s or even 100s...but you would be better served by benchmarks then my guesses. Go code :-)

Creating PDFs from iOS text fields

I'm working on the requirements & specifications for a new iOS app intended for use by certain professionals working "in the field". All day long for weeks on end, these folks have a sizable reporting burden to their superiors using standardized forms that track all different kinds of information. Traditionally, those forms are in PDF, and are simply printed and filled out in ink and then shared with the dozens to hundreds of others working the same operation. Sometimes they'll use a PDF with form fields so the data can be typed and then printed as part of the form. Either way, given their workflow, time and stress pressures, and other factors, it's not a very productive way to get the standardized reporting forms done.
The app we're spec'ing would offer an iOS (and Android, if possible -- but secondary or even tertiary requirement at this point) user interface for tracking the data they enter in the field, organizing it in a logical manner for each individual user, and with the press of a button, take all that data and automatically create a PDF file of it using the standardized form.
Of course, the forms are STRICTLY and rigidly standardized in this industry, and any deviation in format, structure, or presentation is simply not tolerable.
So I was approaching the project by thinking the app would maintain an internal repository of the original standardized forms from the accrediting organization, with each possible data area defined as a field. The app would:
open the necessary PDF form for the task at hand;
parse its dictionary to identity the specific data fields;
for every single field, identify the relevant data from the iOS app's own user interface and data tables, and assign that data to the corresponding field from the PDF/dictionary
export the PDF to a NEW PDF file, which the app would either email or store through iCloud, Dropbox, or some other form of file sharing.
The catch with #4 is that that PDF file must remain editable by standard PDF applications on Windows and Mac (Acrobat, Preview, etc.), so all the fields need to remain. And the PDF should be viewable just the same on either Windows or Mac.
Now, at NO time will the PDF (neither the original nor the exported final document) EVER need to be displayed inside the iOS app, nor would it make much sense to be able to do so.
I don't know if any of this is possible. This is our first iOS project, and we've been leaning towards building the app using Moai or Corona or some other framework to save development time and make porting across platforms easier. That said, if it cannot be done using Lua and one of these frameworks (I remain skeptical...they seem HIGHLY geared towards games), we're not opposed to doing it directly in Objective C and building an Android version some time down the road.
But either way, I'm at a loss in assessing whether this is even a practical undertaking. Our requirements are clear, and frankly if this can't be done, the project won't be pursued any further. But I could definitely use some help from you folks in identifying what my options are, whether I can do it in Lua, and what SDK(s) would be most useful in accomplishing this.
Based on what you've said, it seems that there is little reason to do the PDF-based part of the work on the mobile device itself since:
you don't need to display it on the ipad
you plan to email it or store it in the cloud
if you write this for iOS you will have to write again for Android as you've mentioned
Can you simplify the mobile part of your requirement by focusing on the data-collection and validation, then firing off to a server to do the document production? That will give you a lot more flexibility in the tools that you can use to merge the data into PDF docs. If so you could look at creating PDFs or populating the fields from code using something like iText (C# or Java). If you don't want to build your own back end server you could try something like Docmosis Cloud - but that might not allow you to get your precise layouts.
Certainly the catch you mentioned - needing to keep the PDFs editable with their fields is a significant gotcha in all cases. If you could convince the stakeholders that it is better to generate the final documents from your system (generate draft, review, update data, generate again etc) - rather than generating editable documents that you then lose control and tracability over, then you will be miles ahead.
Hope that helps.
Did you consider just generating a new pdf using an image of the form as the background to the pdf and just writing the user's data into the required areas over the form image. Would reduce the complexity of trying to parse the original form PDFs.
That's a point of worthwhile discussion, but one we don't have an ideal answer on. I tend to think of that as the almost perfect scenario -- it'd be considerably easier to develop. There are two key issues with this approach that have made us table it except as a very last resort:
The users of this product would be working in the field. That field could be quite literally anywhere--the streets of Manhattan, a disaster-stricken area with infrastructure that's been severely damaged or even destroyed, or the most war-ravaged third world country. If it were the streets of, say, Manhattan, there's no problem--their iOS or Android device will have 3G or Wi-Fi access just about anywhere they go. In the latter two scenarios (which are arguably more common in this industry), that connectivity may be very limited. The concern is whether the end user's ability to be productive or to see and share data with their colleagues will be too greatly restricted if they don't have a decent signal. To be fair though, even today they often aren't even using mobile devices, forcing them to go back to a headquarters type location or use radios to share information, effectively negating my point here. But if we're not going to significantly increase their productivity in the field, it just gives us pause to think through whether or not we have enough of a value proposition to ask them to fairly significantly change their methods of doing things.
To your latter point, no there's no convincing the stakeholders that this new system is the better approach. Even if there were, it would take years to do so. These forms are a part of a well-defined, decades-old standard used by literally thousands of organizations.

Resources