I would like to know if there are better ways to initialize a large collection of same-type instances. This is not a problem only limited to Swift, but I am using Swift in this case.
Take, for example, a large list of API endpoints. Suppose I have 100 endpoints in this API and each of them share some common functionality, such as headers, parameter lists, parsing formats, etc... albeit with different values for each of these "options".
I could think of a few different ways to express 100 endpoints:
Create a resource file with all of the values and read them in from the file on app launch. The problem with this is that it becomes stringly typed and there is potential for typos and/or lots of copy/paste key values. This would include plist files, json files, sqlite tables, csv files, etc. It centralizes and condenses the data, but it doesn't seem maintenance friendly or swiftly. Furthermore, it seems like resource files are harder to obfuscate should the details be somewhat private.
Create a giant enum-ish function with all of the API endpoint instance initialization code blobbed all in the same area/function/file. This would be equivalent of doing a giant switch statement or making a collection literal with all the instantiation happening in one spot. The advantage here is that it can be strongly typed and it is also contained to one area, similar to what a resource file would do. However, it will be a BIG file with lots of scrolling. Maybe too big?
Create a separate file/module/instance/subtype for each endpoint and, more or less, hardcode computed properties inside the instance. This would be maybe creating an extension and/or subclass for each endpoint and putting them in a separate swift file. This limits the visual scope for each endpoint, but it also just turns your project files into the blob of data instead.
I'm wondering if there are philosophical arguments for either of these options. Or, are there other options I have not thought of. Is it preference? Are there best practices when initializing a large collection of what seems like a bunch of complex literals?
If you have lots of this static data, or machine-generated classes, consider the advice in WWDC 2016's Optimizing App Startup Time. It's a great talk. The loader has to initialize and fix up all your static object instances and classes; if you have a lot, your app load time will be adversely affected.
For static data, one piece of advice is to use Swift, which you've already done, as Swift knows to defer the instantiations until run time.
Swift doesn't help with mass-produced classes; though you can switch to structs instead.
Even ignoring the startup time issue, I'd err on the side of being data driven. Option 1. Less code to maintain. IMHO There's nothing wrong with stringly typed here, this code is unlikely to change much; adding endpoints will be trivial. It's cool to see new function when you didn't even write new code!
Related
What is the best way to load huge text file data in delphi? Is there any component that can load text file superfast?
Let's say I have a text file contains database and stored in fix length format.
It contains 150 field with each at least 50 characters.
1. I need to load it into memory
2. I need to parse it and probably store it in a memdataset for processing
My questions:
1. Is it enough if I use TStringList.loadFromFile method?
2. Is there any other better component to manipulate the text file?
3. Should I use low level reading from textfile?
Thank you in advance.
TStringList is never the optimal way of working with lots of text, but it's the simplest. If you've got small files on your hands you can use TStringList without issues. Even if you have large files (not huge files) you might implement a version of you algorithm using TStringList for testing purposes, because it's simple and easy to understand.
If your files are large, as they probably are since you call them "databases", you need to look into alternative technologies that will enable you to read only as much as you need from the database. Look into:
TFileStream
Memory mapped files.
Don't look at the old "file" based API's still available in Delphi, they're plain old.
I'm not going to go into details on how to access text using those methods because we've recently had two similar questions on SO:
How Can I Efficiently Read The FIrst Few Lines of Many Files in Delphi
and
Fast Search to see if a String Exists in Large Files with Delphi
Since you have a fixed length that you're working with, you can build an access class based on TList with a TWriter and TReader that will take your records into account. You'll have none of the overhead of a TStringList (not that it's a bad thing, but if you don't need it, why have it) and you can build in your own access to records into the class.
Ultimately it depends on what you are trying to accomplish with the data once you have it loaded into memory. While TStringlist is easy to use, it isn't as efficient as "rolling your own".
However, efficiency in data manipulation may not be that much of an issue, as you are using text files to hold a database. If you just need to read in and make decisions based on data in the file, the more flexible TList may be overkill.
I recommend to adhere to TStringList if you find it convenient for your problem. Optimization is another thing that should be done later.
As for TStringList the optimization is to declare a descendant class that overrides TStrings.LoadFromStream method - you can make it practically as fast as possible, taking into account the structure of your files.
It is not entirely clear from your question why you need to load the entire file into memory, prior to then going on to create an in-memory data set.... are you conflating the two issues? (i.e. because you need to create an in-memory data set you think you first need to load the source data entirely into memory? Or is there some initial pre-processing of the source file which is only possible with the entire file loaded in memory (this is unlikely and even if this is the case, it isn't necessary with a navigable stream object such as a TFileStream).
But I think the answer you are looking for is right there in the question....
If you are loading this file in order to parse it and populate/initialise a further data structure (the data set) for further processing, then using an existing high level data structure is an unnecessary and potentially costly (in terms of time) step.
Use the lowest level means of access that provides the capabilities you need.
In this case a TFileStream will likely provide the best balance of convenience and ease of use.
I've looked through a lot of answers on stack overflow but haven't found anything that answers this question.
I have an app where you download data in a downloadVC (there are many other VC's). I want to be able to access the currentUser and the downloaded data in the downloadVC whenever I go to the downloadVC without re-downloading the data.
The options I've looked at so far are:
Making a singleton with data that I could access at all points of the app (feels like the easiest way but people have warned me singletons aren't good).
Making a class var of download VC. (This is the solution I understand the least, what is the scope of a class var? Is it just the same as making a singleton?)
Passing it around (This wouldn't work for me as the app is too big to have it passed between every VC)
Saving it to user defaults, (how quick is accessing something user defaults?)
Please tell me if this questions doesn't fit the stack overflow rules?
Singletons are usually implemented using a static variable, so your first and second options are quite similar.
There is a "singletons are evil" sect that's very vocal in the development community lately.
Personally, I think they have their place and can sometimes clean up your design. I recently worked on a project that had been designed by a member of the "Singletons are evil" cult who then went to absurd lengths to pass a data manager object around to every other object in the project, resulting in a lot of overhead and more than a few bugs when the object got dropped.
There is no one answer. You need to weigh the pros and cons of different approaches for your app.
I would caution against over-using UserDefaults though. That's intended for saving small bits of data like user preference settings, not big data objects.
Basically, adding any networking logic inside the view controller is a first big mistake that you could make. So move that to another class that is responsible only for sending network requests and handling the responses.
Then, when you have the data downloaded, you'll probably need something to manage the cache - no matter whether it is a set of big files or a couple of small strings, the logic would be the same - to have all this cache controlled by one object. This cache manager could save the objects to local files, use CoreData or simply keep those objects in memory - this is for you to decide, depending on what kind of app are you creating.
Now, the cache manager could basically be your endpoint for view controllers, since it will either download the data and return it to view controller after the request's success, or return it immediately. The view controllers don't really have to know about any networking at all.
But, how the view controllers will find out about the cache manager?
You could either pass around the reference to it, but you've already said that this is impossible in your app. So basically another solution would be to use the hated singleton pattern. I too personally think this is a bad pattern, however you cannot create any app without it, since you always must have at least one singleton, which is AppDelegate.
Anyway, a good idea might be to create a singleton class (or maybe even use the AppDelegate for that purpose), that would be responsible for handling the dependencies between the classes responsible for networking, the cache manager, and any other classes that you might need to perform some logic behind your app. This is actually a pattern called Dependency Injection Container. It would allow you to easily access your model classes through this container, without neither having a ton of singletons that will eventually make you confused, nor creating some ridiculous logic of passing around the model objects.
The simple answer is, yes, you use a singleton.
That's exactly correct.
Note that whether you use
SQLite.swift,
core data,
realm.io,
write to a text file,
a baas such as Firebase or Back4app (aka Parse),
or literally just "keep it in an array" ("in memory"),
Yes, the answer to your question is you'll have singleton which is where you "keep - access" that stuff. Exactly correct.
That seems to be what you are asking here.
Given that ...
... if you are then additionally asking "what's the easiest / best / most modern way ("in my data singleton") to store data locally in my app", the real-world answer in 2017 is
realm.io
Previously you used Apple's core data. Which is (a) spectacular (b) extremely difficult. Importantly though: realm.io and SQLite are identically available on both Android and iOS; in very many cases, on real world projects today, this eliminates Core Data from consideration.
However. All of this is somewhat moot. Don't forget ...
We now live in a "baas world"
You can't eg. "get a job programming iOS or Android" any more. You get a job because of how good you are / your specific expertise areas in either Firebase, Parse, PubNub, Cloudbase, or whatever. For better or worse, being able to move buttons and tables around in Xcode, Studio, is not really a thing anymore. (There are a minuscule number of hyper-specialities, like GPU programming, dynamic mesh, or the like, that are exceptions to this.) Similarly, say you're a hobbyist making your own app or startup: again in that case, it's just entirely and totally about what backend you work with. (And that applies equally to games or business/social apps.) There are simply no "local, static" apps any more. (Again, applies equally to games or business/social apps.)
(Note indeed that realm.io, which is "the" simple, obvious way to keep data on apps these days - indeed, those guys have/are moving to becoming, indeed, a OCC cloud baas-paas-whatever service themselves. We'll probably all just use realm.io instead of Firebase in a year.)
So in a sense the answer to your question is something like Firebase or back4app. But: then within your app, you centralize that to a singleton, indeed (the first part of your question).
But note....................
it is extremely unlikely, at the beginner level, that any of this will be relevant: Just keep the data in an array! That's all there is to it. OK, once in a billion years when a user happens to literally restart the device or your app, the app will reload data. So what?
Note that a common name for your "get data here" Singleton is
Local
So,
import Foundation
often .. import SQLite, Realm, Firebase or whatever
public let local = Local.shared
open class Local {
static let shared = Local()
fileprivate init() {
print("'Local' singleton running AOK")
}
// could be this simple..
var clients:[String:[Blah]] = [:]
var stock:[String:String] = [:]
func loadStockInfoFromWWW() { ... }
func calculateQuantityInfoOrWhatever() { ... }
// or it could be more like this..
func reloadClientsFromParse() { ... }
func sendNewDataToFirebaseParse() { ... }
.... etc
you then just access it from anywhere in your app like
local.updateFromWeb()
height = local.stock["ladders"][idNumber].size.h
and so on.
That's all there is to it.
(A word on singleton code style.)
It depends on file complexity & size. If you don't need to parse the contents, or they're simple to parse, but it's large in size then I'd recommend using the Documents folder and retrieving it from there, if it's small or involves complex processing then a singleton acting as a manager would be my approach as it feels appropriate.
Not quite sure if this is on topic, so when in doubt feel free to close.
We have a client who is missing tracking data for a large segment of his visitors in his Report Suite. However the complete set of data is available in a data warehouse. We are now investigating if it is possible to import them as a data source. I have only experience with enriching data via classifications, however the goal here is to create views (sessions etc) for a past timeframe etc from scratch.
According to the documentation this should be possible. However there is one caveat specifically mentioned in the FAQ:
"Adobe recommends you select new, unused variables to import data
using Data Sources. If you are uncertain about the configuration of
your data file, or want to better understand the risks of re-using
variables, contact Customer Care.“
I take that to mean that I should not import data to props,Evars,events etc. that have been used when data has been collected via the tracker, which would pretty much defeat our purpose (basically we want to merge the data from the data warehouse with existing data). Since I have to go to some intermediaries to reach customer care and this takes a long time I wonder if somebody here can explain what the dangers in re-using variables are (and maybe even if there is still a way to do this).
DISCLAIMER: I'm not familiar with Adobe Analytics, but the problem here is pretty universal. If someone with actual experience/knowledge specific to the product comes along, pay more attention to them than me :)
As a rule, Variable reuse in any system runs the risk of data corruption. I'm not familiar with Adobe Analytics, but a brief read through some blogs imply that this is what they're worried about in terms of variable reuse - if you have a variable that is being used in one section, and you import data into it in another section when is in the same scope, you overwrite the data that the other section was using.
Now, that same blog states that provided you have your data structure set up in a specific way, it can allow you to reuse variables/properties without issue and in fact encourages it, hence the statement in your quote "If you are uncertain about the configuration of your data file...". They're probably warning you that if you know what you're doing and know that there won't be any overwriting, fine, go ahead and reuse, but if you don't, or you aren't sure whether something else might be using the original content, then it's unsafe.
Regarding your specific case, you want to merge the two piece of data together, not overwrite, so reusing your existing variables would overwrite the existing data. Sounds like you will need to import to a second (new) set of variables, and then compare/merge between them within the system, rather than trying to import and merge in one go.
We have since received an answer from Adobe Customer Care.
According to them the issue is that hits created via data imports are indistinguishable from hits created via server calls, so they cannot be removed or corrected after they have been imported. They recommend to use different variables so that the original data remains recognizable and salvageable in case one does import faulty data via the imports.
This was already mentioned in the online documentation, but apparently Adobe thinks this is important enough to issue the extra warning.
It seems to satisfy the three requirements here: On Design Patterns: When to use the Singleton?
I need it to exist only once.
I need to access it from all over the source base.
It handles concurrent access, i.e. Locks for writes, but can handle concurrent reads.
Hi all,
I've been reading a lot of no doubt intelligent educated and wise gems of advice that Singletons are 'Evil' and singletons are anti patterns or just plain bad news.
With the argument that logging makes sense but not much else.
Just interested to know if the case of essentially a persistent data store context makes sense for a singleton, i.e. Reading/Writing from disk to memory and referencing an object graph.
And if not, how do people normally tackle this issue, I don't currently have any problem with it, I know it's created only once, it's fast and the accessor logic is in one place. Meaning it takes me one line of code to do anything data model related.
Which leaves the only argument that it's bad for testing, in that it's a hard coded production implementation to data, but couldn't I just write a method swizzle through a category or in the test code to create the test version of the singleton?
And the final argument from DI testers, is that it is a hard coded implementation, rather than simply an interface to something, which I do get but I don't really have a major drive to use a DI framework given that I can use protocols for implementation, and use separate init methods for setting up an objects state in testing. There's only ever going to be two types of state for the singleton, or realistically one type...production.
Making it (in my opinion) much easier to read and faster to develop.
Change my view SO?
Yup, for some singletons are Evil. For the new developers who has little MRC knowledge and more ARC it sounds scary because they need to mess with memory,volatile,synchronize etc.
However it is not against to use them, they indeed has their own purpose to use with some are below.
when sharing large data models like arrays and dictionaries etc between multiple screens (VC's) we can't prefer storing them in UserDefaults (because userdefaults is permanent storage and storing such large entries make app start lazily) instead singletons are best since they stay only current app context and restarting app creates new one.
when we need a stable db connection to be accessible allover the app without messing up with connecting and closing in every business classes we can go for it.
when we wanted an ability to app for theming itself dynamically we would need to create a singleton class which holds all the color,image instances etc. and use that instance in application VC/Views etc so no code duplication and re-processing theme occurs in all places.
You dont have to change your view but tweak it a bit to get some positive intention towards singletons.
Hoping this clears it out, thanks
What is the best way to load huge text file data in delphi? Is there any component that can load text file superfast?
Let's say I have a text file contains database and stored in fix length format.
It contains 150 field with each at least 50 characters.
1. I need to load it into memory
2. I need to parse it and probably store it in a memdataset for processing
My questions:
1. Is it enough if I use TStringList.loadFromFile method?
2. Is there any other better component to manipulate the text file?
3. Should I use low level reading from textfile?
Thank you in advance.
TStringList is never the optimal way of working with lots of text, but it's the simplest. If you've got small files on your hands you can use TStringList without issues. Even if you have large files (not huge files) you might implement a version of you algorithm using TStringList for testing purposes, because it's simple and easy to understand.
If your files are large, as they probably are since you call them "databases", you need to look into alternative technologies that will enable you to read only as much as you need from the database. Look into:
TFileStream
Memory mapped files.
Don't look at the old "file" based API's still available in Delphi, they're plain old.
I'm not going to go into details on how to access text using those methods because we've recently had two similar questions on SO:
How Can I Efficiently Read The FIrst Few Lines of Many Files in Delphi
and
Fast Search to see if a String Exists in Large Files with Delphi
Since you have a fixed length that you're working with, you can build an access class based on TList with a TWriter and TReader that will take your records into account. You'll have none of the overhead of a TStringList (not that it's a bad thing, but if you don't need it, why have it) and you can build in your own access to records into the class.
Ultimately it depends on what you are trying to accomplish with the data once you have it loaded into memory. While TStringlist is easy to use, it isn't as efficient as "rolling your own".
However, efficiency in data manipulation may not be that much of an issue, as you are using text files to hold a database. If you just need to read in and make decisions based on data in the file, the more flexible TList may be overkill.
I recommend to adhere to TStringList if you find it convenient for your problem. Optimization is another thing that should be done later.
As for TStringList the optimization is to declare a descendant class that overrides TStrings.LoadFromStream method - you can make it practically as fast as possible, taking into account the structure of your files.
It is not entirely clear from your question why you need to load the entire file into memory, prior to then going on to create an in-memory data set.... are you conflating the two issues? (i.e. because you need to create an in-memory data set you think you first need to load the source data entirely into memory? Or is there some initial pre-processing of the source file which is only possible with the entire file loaded in memory (this is unlikely and even if this is the case, it isn't necessary with a navigable stream object such as a TFileStream).
But I think the answer you are looking for is right there in the question....
If you are loading this file in order to parse it and populate/initialise a further data structure (the data set) for further processing, then using an existing high level data structure is an unnecessary and potentially costly (in terms of time) step.
Use the lowest level means of access that provides the capabilities you need.
In this case a TFileStream will likely provide the best balance of convenience and ease of use.