This is my very first question here so go easy on me!
I'm a newbie coder and I'm currently trying to loop through JSON, parse the data and backup the information to my Firebase server - using Alamofire to request the JSON information.
Swift 4, Alamofire 4.5.1, Firebase 4.2.0
The process works - but not without infinitely increasing device memory usage & up to 200% CPU usage. Through commenting out lines, I singled the memory and CPU usage down to the Firebase upload setValue line in my data pulling function - which iterates through a JSON database of unknown length (by pulling a max of 1000 rows of data at a time - hence the increasing offset values). The database that I'm pulling information from is huge, and with the increasing memory usage, the function grinds to a very slow pace.
The function detects if it's found an empty JSON (end of the results), and then either ends or parses the JSON, uploads the information to Firebase, increases the offset value by 1000 rows, and then repeats itself with the new offset value.
var offset: Int! = 0
var finished: Bool! = false
func pullCities() {
print("step 1")
let call = GET_CITIES + "&offset=\(self.offset!)&rows=1000"
let cityURL = URL(string: call)!
Alamofire.request(cityURL).authenticate(user: USERNAME, password: PASSWORD).responseJSON { response in
let result = response.result
print("step 2")
if let dict = result.value as? [Dictionary<String, Any>] {
print("step 3")
if dict.count == 0 {
self.finished = true
print("CITIES COMPLETE")
} else {
print("step 4")
for item in dict {
if let id = item["city"] as? String {
let country = item["country"] as? String
let ref = DataService.ds.Database.child("countries").child(country!).child("cities").child(id)
ref.setValue(item)
}
}
self.finished = false
print("SUCCESS CITY \(self.offset!)")
self.offset = self.offset! + 1000
}
}
if self.finished == true {
return
} else {
self.pullCities()
}
}
}
It seems to me like the data being uploaded to Firebase is being saved somewhere and not emptied once the upload completes? Although I couldn't find much information on this issue when searching through the web.
Things I've tried:
a repeat, while function (no good as I only want 1 active repetition of each loop - and still had high memory, CPU usage)
performance monitoring (Xcode call tree found that "CFString (immutable)" and "__NSArrayM" were the main reason for the soaring memory usage - both relating to the setValue line above)
memory usage graphing (very clear that memory from this function doesn't get emptied when it loops back round - no decreases in memory at all)
autoreleasepool blocks (as per suggestions, unsuccessful)
Whole Module Optimisation already enabled (as per suggestions, unsuccessful)
Any help would be greatly appreciated!
UPDATE
Pictured below is the Allocations graph after a single run of the loop (1,000 rows of data). It shows that what is likely happening is that Firebase is caching the data for every item in the result dict, but appears to only de-allocate memory as one whole chunk when every single upload has finished?
Ideally, it should be de-allocating after every successful upload and not all at once. If anyone could give some advice on this I would be very grateful!
FINAL UPDATE
If anyone should come across this with the same problem, I didn't find a solution. My requirements changed so I switched the code over to nodejs which works flawlessly. HTTP requests are also very easy to code for on javascript!
I had a similar issue working with data on external websites and the only way I could fix it was to wrap the loop in an autoreleasepool {} block which forced the memory to clear down on each iteration. Given ARC you might think such a structure is not needed in Swift but see this SO discussion:
Is it necessary to use autoreleasepool in a Swift program?
Hope that helps.
sometimes compiler is not able to properly optimise your code unless you enable whole module optimisation in project build settings. this is usually happening when generics is being used.
try to turn it on even for debug env and test.
I am having trouble displaying an Alert in case of an Error properly.
My idea is: Everytime I download data from my backend with an completion block, I present an Alert if an error occurs.
query?.findObjectsInBackground(block: { (objects, error) -> Void in
if error != nil {
createAlert(error)
return
} else if let objects = objects {
}
Since I got more than one call in a ViewController at the same time, it may happen to find myself having more than 2 or 3 Alerts presenting at the same time saying e.g. "No Connection to the Internet".
It will constantly reload the Alert and it is a pain in terms of UI.
What is best practice to solve this issue?
My solution idea would be to put everything in a Singleton pattern and make sure no other other Alert is currently displayed.
Are there any better ways?
Instead of using a singleton pattern, you might prefer having an optional property (var noConnectivityAlert) in the class currently responsible for creating the alert.
Instead of the createAlert() method you would have a informUserAboutConnectivity() method.
func informUserAboutConnectivity() {
// If noConnectivityAlert is nil
// the method creates an alert and shows it.
// If the property is NOT nil
// do nothing (since the user is already informed).
}
When the internet connection would come back and then disappear again, some apps in the App Store would show an alert once again.
In that case, when the internet connection comes back you can directly set noConnectivityAlert = nil so that when the connection is lost, things will be handled nicely (a new alert will be created and shown).
By the way, in the iOS SDK, singletons are not used often. There are mostly used for providing a default and most common use case of a class (think of UserDefaults), or (of course) a shared manager/provider.
Every firebase client example I see in Swift seems to oversimplify properly loading data from Firebase, and I've now looked through all the docs and a ton of code. I do admit that my application may be a bit of an edge case.
I have a situation where every time a view controller is loaded, I want to auto-post a message to the room "hey im here!" and additionally load what's on the server by a typical observation call.
I would think the flow would be:
1. View controller loads
2. Auto-post to room
3. Observe childAdded
Obviously the calls are asynchronous so there's no guarantee the order of things happening. I tried to simplify things by using a complete handler to wait for the autopost to come back but that loads the auto-posted message twice into my tableview.
AutoPoster.sayHi(self.host) { (error) in
let messageQuery = self.messageRef.queryLimited(toLast:25).queryOrdered(byChild: "sentAt")
self.newMessageRefHandle = messageQuery.observe(.childAdded, with: { (snapshot) in
if let dict = snapshot.value as? [String: AnyObject] {
DispatchQueue.main.async {
let m = Message(dict, key: snapshot.key)
if m.mediaType == "text" {
self.messages.append(m)
}
self.collectionView.reloadData()
}
}
})
}
Worth noting that this seems very inefficient for an initial load. I fixed that by using a trick with a timer that will basically only allow the collection view to reload maximum every .25s and will restart the timer every time new data comes in. A bit hacky but I guess the benefits of firebase justify the hack.
I've also tried to observe the value event once for an initial load and then only after that observe childAdded but I think that has issues as well since childAdded is called regardless.
While I'm tempted to post code for all of the loading methods I have tried (and happy to update the question with it), I'd rather not debug what seems to not be working and instead have someone help outline the recommended flow for a situation like this. Again, the goal is simply to auto-post to the room that I joined in the conversation, then load the initial data (my auto-post should be the most recent message), and then listen for incoming new messages.
Instead of
self.newMessageRefHandle = messageQuery.observe(.childAdded, with: { (snapshot) in
try replacing with
let childref = FIRDatabase.database().reference().child("ChildName")
childref.queryOrdered(byChild:"subChildName").observe(.value, with: { snapshot in
I am new in swift3.0 I am implementing a custom search box. I wish to know how can i make a search queue such that on text change in searchbox i need to perform search operation with new text and if there is an existing search operation going on cancel that. I also want to include threshold ontextchanged. So that search operation does not get fired very frequently
Your question is somehow general, but let me tell you how I accomplished this in Swift 3 and AFNetworking (this assumes you wish to search for the data on the server).
I hold a reference of the networking manager in the properties of the view controller:
//The network requests manager. Stored here because this view controller extensively uses AFNetworking to perform live search updates when the input box changes.
var manager = AFHTTPRequestOperationManager()
Afterwards, using UISearchController I check to see if there is any text entered in the search box at all and, if it is, I want to make sure there aren't any other ongoing AFNetworking tasks from now by closing any of them which are still running:
//Called when the something is typed in the search bar.
func updateSearchResults (for searchController: UISearchController) {
if !SCString.isStringValid(searchController.searchBar.text) {
searchController.searchResultsController?.view.isHidden = false
tableView.reloadData()
return
}
data.searchText = searchController.searchBar.text!
/**
Highly important racing issue solution. We cancel any current request going on because we don't want to have the list updated after some time, when we already started another request for a new text. Example:
- Request 1 started at 12:00:01
- We clear the containers because Request 2 has to start
- Request 2 started at 12:00:02
- Request 1 finished at 12:00:04. We update the containers because data arrived
- Request 2 finished at 12:00:05. We update the containers because data arrived
- Now we have data from both 1 and 2, something really not desired.
*/
manager.session.getTasksWithCompletionHandler { (dataTasks, uploadTasks, downloadTasks) in
dataTasks.forEach { $0.cancel() }
}
/**
Reloads the list view because we have to remove the last search results.
*/
reloadListView()
}
In the end, I also check in the failure closure if the code of the error is not NSURLErrorCancelled. Because, if that happened, I don't display any error message or toast.
//The operation might be cancelled by us on purpose. In this case, we don't want to interfere with the ongoing logic flow.
if (operation?.error as! NSError).code == NSURLErrorCancelled {
return
}
self.retrieveResultListFailureNetwork()
Hope it helps!
In my application I need to download a JSON file from the web. I have made a ResourceService class that have a download method as seen below. I use this service in "higher level" services of my app. You can see there are multiple of things that may go wrong during the download. The server could be on fire and not be able to successfully respond at the moment, there could be go something wrong during the moving of the temporary file etc.
Now, there is probably not much a user can do with this other than trying later. However, he/she probably want to know that there was something wrong and that the download or the behaviour of the "higher level" methods could not succeed.
Me as a developer is confused as this point because I don't understand how to deal with errors in Swift. I have a completionHandler that takes an error if there was one, but I don't know what kind of error I should pass back to the caller.
Thoughts:
1) If I pass the error objects I get from the NSFileManager API or the NSURLSession API, I would think that I am "leaking" some of the implementation of download method to the callers. And how would the caller know what kind of errors to expect based on the error? It could be both.
2) If I am supposed to catch and wrap those errors that could happen inside the download method, how would that look like?
3) How do I deal with multiple error sources inside a method, and how would the code that calls the method that may throw/return NSError objects look like?
Should you as a caller start intercepting the errors you get back and then write a lot of code that differentiates the messages/action taken based on the error code? I don't get this error handling stuff at all and how it would look like when there are many things that could go wrong in a single method.
func download(destinationUrl: NSURL, completionHandler: ((error: NSError?) -> Void)) {
let request = NSURLRequest(URL: resourceUrl!)
let task = downloadSession.downloadTaskWithRequest(request) {
(url: NSURL?, response: NSURLResponse?, error: NSError?) in
if error == nil {
do {
try self.fileManager.moveItemAtURL(url!, toURL: destinationUrl)
} catch let e {
print(e)
}
} else {
}
}.resume()
}
First of all this is a great question. Error handling is a specific task that applies to a incredible array of situations with who know's what repercussions with your App's state. The key issue is what is meaningful to your user, app and you the developer.
I like to see this conceptually as how the Responder chain is used to handle events. Like an event traversing the responder chain an error has the possibility of bubbling up your App's levels of abstraction. Depending on the error you might want to do a number of things related to the type of the error. Different components of your app may need to know about error, it maybe an error that depending on the state of the app requires no action.
You as the developer ultimately know where errors effect your app and how. So given that how do we choose to implement a technical solution.
I would suggest using Enumerations and Closures as to build my error handling solution.
Here's a contrived example of an ENUM. As you can see it is represents the core of the error handling solution.
public enum MyAppErrorCode {
case NotStartedCode(Int, String)
case ResponseOkCode
case ServiceInProgressCode(Int, String)
case ServiceCancelledCode(Int, String, NSError)
func handleCode(errorCode: MyAppErrorCode) {
switch(errorCode) {
case NotStartedCode(let code, let message):
print("code: \(code)")
print("message: \(message)")
case ResponseOkCode:
break
case ServiceInProgressCode(let code, let message):
print("code: \(code)")
print("message: \(message)")
case ServiceCancelledCode(let code, let message, let error):
print("code: \(code)")
print("message: \(message)")
print("error: \(error.localizedDescription)")
}
}
}
Next we want to define our completionHandler which will replace ((error: NSError?) -> Void) the closure you have in your download method.
((errorCode: MyAppErrorCode) -> Void)
New Download Function
func download(destinationUrl: NSURL, completionHandler: ((errorCode: MyAppErrorCode) -> Void)) {
let request = NSURLRequest(URL: resourceUrl!)
let task = downloadSession.downloadTaskWithRequest(request) {
(url: NSURL?, response: NSURLResponse?, error: NSError?) in
if error == nil {
do {
try self.fileManager.moveItemAtURL(url!, toURL: destinationUrl)
completionHandler(errorCode: MyAppErrorCode.ResponseOkCode)
} catch let e {
print(e)
completionHandler(errorCode: MyAppErrorCode.MoveItemFailedCode(170, "Text you would like to display to the user..", e))
}
} else {
completionHandler(errorCode: MyAppErrorCode.DownloadFailedCode(404, "Text you would like to display to the user.."))
}
}.resume()
}
In the closure you pass in you could call handleCode(errorCode: MyAppErrorCode) or any other function you have defined on the ENUM.
You have now the components to define your own error handling solution that is easy to tailor to your app and which you can use to map http codes and any other third party error/response codes to something meaningful in your app. You can also choose if it is useful to let the NSError bubble up.
EDIT
Back to our contrivances.
How do we deal with interacting with our view controllers? We can choose to have a centralized mechanism as we have now or we could handle it in the view controller and keep the scope local. For that we would move the logic from the ENUM to the view controller and target the very specific requirements of our view controller's task (downloading in this case), you could also move the ENUM to the view controller's scope. We achieve encapsulation, but will most lightly end up repeating our code elsewhere in the project. Either way your view controller is going to have to do something with the error/result code
An approach I prefer would be to give the view controller a chance to handle specific behavior in the completion handler, or/then pass it to our ENUM for more general behavior such as sending out a notification that the download had finished, updating app state or just throwing up a AlertViewController with a single action for 'OK'.
We do this by adding methods to our view controller that can be passed the MyAppErrorCode ENUM and any related variables (URL, Request...) and add any instance variables to keep track of our task, i.e. a different URL, or the number of attempts before we give up on trying to do the download.
Here is a possible method for handling the download at the view controller:
func didCompleteDownloadWithResult(resultCode: MyAppErrorCode, request: NSURLRequest, url: NSURL) {
switch(resultCode) {
case .ResponseOkCode:
// Made up method as an example
resultCode.postSuccessfulDownloadNotification(url, dictionary: ["request" : request])
case .FailedDownloadCode(let code, let message, let error):
if numberOfAttempts = maximumAttempts {
// Made up method as an example
finishedAttemptingDownload()
} else {
// Made up method as an example
AttemptDownload(numberOfAttempts)
}
default:
break
}
}
Long story short: yes
... and then write a lot of code that differentiates the
messages/action taken based on the error code?
Most code examples leave the programmer alone about how to do any error handling at all, but in order to do it right, your error handling code might be more than the code for successful responses. Especially when it comes to networking and json parsing.
In one of my last projects (a lot of stateful json server communication) I have implemented the following approach: I have asked myself: How should the app possibly react to the user in case of an error (and translate it to be more user friendly)?
ignore it
show a message/ an alert (possibly only one)
retry by itself (how often?)
force the user to start over
assume (i.e. a previously cached response)
To achieve this, I have create a central ErrorHandler class, which does have several enums for the different types of errors (i.e. enum NetworkResponseCode, ServerReturnCode, LocationStatusCode) and one enum for the different ErrorDomains:
enum MyErrorDomain : String {
// if request data has errors (i.e. json not valid)
case NetworkRequestDomain = "NetworkRequest"
// if network response has error (i.e. offline or http status code != 200)
case NetworkResponseDomain = "NetworkResponse"
// server return code in json: value of JSONxxx_JSON_PARAM_xxx_RETURN_CODE
case ServerReturnDomain = "ServerReturnCode"
// server return code in json: value of JSONxxxStatus_xxx_JSON_PARAM_xxx_STATUS_CODE
case ServerStatusDomain = "ServerStatus"
// if CLAuthorizationStatus
case LocationStatusDomain = "LocationStatus"
....
}
Furthermore there exists some helper functions named createError. These methods do some checking of the error condition (i.e. network errors are different if you are offline or if the server response !=200). They are shorter than you would expect.
And to put it all together there is a function which handles the error.
func handleError(error: NSError, msgType: String, shouldSuppressAlert: Bool = false){
...
}
This method started with on switch statement (and needs some refactoring now, so I won't show it as it still is one). In this statement all possible reactions are implemented. You might need a different return type to keep your state correctly in the app.
Lessons learned:
Although I thought that I have started big (different enums, central user alerting), the architecture could have been better (i.e. multiple classes, inheritance, ...).
I needed to keep track of previous errors (as some are follow ups) in order to only show one error message to the user -> state.
There are good reasons to hide errors.
Within the errorObj.userInfo map, it exits a user friendly error message and a technicalErrorMessage (which is send to a tracking provider).
We have introduced numeric error codes (the error domain is prefixed with a letter) which are consistent between client and server. They are also shown to the user. This has really helped to track bugs.
I have implemented a handleSoftwareBug function (which is almost the same as the handleError but much less cases). It is used in a lot of else-blocks which you normally do not bother to write (as you think that this state can never be reached). Surprisingly it can.
ErrorHandler.sharedInstance.handleSoftwareBug("SW bug? Unknown received error code string was code: \(code)")
How does it look like in code: There are a lot of similar backend network requests where a lot of code looks something like the following:
func postAllXXX(completionHandler:(JSON!, NSError!) -> Void) -> RegisteringSessionTask {
log.function()
return postRegistered(jsonDict: self.jsonFactory.allXXX(),
outgoingMsgType: JSONClientMessageToServerAllXXX,
expectedIncomingUserDataType: JSONServerResponseAllXXX,
completionHandler: {(json, error) in
if error != nil {
log.error("error: \(error.localizedDescription)")
ErrorHandler.sharedInstance.handleError(error,
msgType: JSONServerResponseAllXXX, shouldSuppressAlert: true)
dispatch_async(dispatch_get_main_queue(), {
completionHandler(json, error)
})
return
}
// handle request payload
var returnList:[XXX] = []
let xxxList = json[JSONServerResponse_PARAM_XXX][JSONServerResponse_PARAM_YYY].arrayValue
.....
dispatch_async(dispatch_get_main_queue(), {
completionHandler(json, error)
})
})
}
Within the above code you see that I call a completionHandler and give this caller the chance to customize error handling, too. Most of the time, this caller only handles success.
Whenever I have had the need for retries and other and not so common handling, I have also done it on the caller side, i.e.
private func postXXXMessageInternal(completionHandler:(JSON!, NSError!) -> Void) -> NSURLSessionDataTask {
log.function()
return self.networkquery.postServerJsonEphemeral(url, jsonDict: self.jsonFactory.xxxMessage(),
outgoingMsgType: JSONClientMessageToServerXXXMessage,
expectedIncomingUserDataType: JSONServerResponseXXXMessage,
completionHandler: {(json, error) in
if error != nil {
self.xxxMessageErrorWaitingCounter++
log.error("error(\(self.xxxMessageErrorWaitingCounter)): \(error.localizedDescription)")
if (something || somethingelse) &&
self.xxxMessageErrorWaitingCounter >= MAX_ERROR_XXX_MESSAGE_WAITING {
// reset app because of too many errors
xxx.currentState = AppState.yyy
ErrorHandler.sharedInstance.genericError(MAX_ERROR_XXX_MESSAGE_WAITING, shouldSuppressAlert: false)
dispatch_async(dispatch_get_main_queue(), {
completionHandler(json, nil)
})
self.xxxMessageErrorWaitingCounter = 0
return
}
// handle request payload
if let msg = json[JSONServerResponse_PARAM_XXX][JSONServerResponse_PARAM_ZZZ].stringValue {
.....
}
.....
dispatch_async(dispatch_get_main_queue(), {
completionHandler(json, error)
})
})
}
Here is another example where the user is forced to retry
// user did not see a price. should have been fetched earlier (something is wrong), cancel any ongoing requests
ErrorHandler.sharedInstance.handleSoftwareBug("potentially sw bug (or network to slow?): no payment there? user must retry")
if let st = self.sessionTask {
st.cancel()
self.sessionTask = nil
}
// tell user
ErrorHandler.sharedInstance.genericInfo(MESSAGE_XXX_PRICE_REQUIRED)
// send him back
xxx.currentState = AppState.zzz
return
For any request, you get either an error or an http status code. Error means: Your application never managed to talk properly to the server. http status code means: Your application talked to a server. Be aware that if you take your iPhone into the nearest Starbucks, "your application talked to a server" doesn't mean "your application talked to the server it wanted to talk to". It might mean "your application managed to talk to the Starbucks server which asks you to log in and you have no idea how to do that".
I divide the possible errors into categories: "It's a bug in my code". That's where you need to fix your code. "Something went wrong, and the user can do something about it". For example when WiFi is turned off. "Something went wrong, maybe it works later". You can tell the user to try later. "Something went wrong, and the user can't do anything about it". Tough. "I got a reply from the server that I expected. Maybe an error, maybe not, but something that I know how to handle". You handle it.
I also divide calls into categories: Those that should run invisibly in the background, and those that run as a result of a direct user action. Things running invisibly in the background shouldn't give error messages. (Bloody iTunes telling me it cannot connect to the iTunes Store when I had no interest in connecting to the iTunes Store in the first place is an awful example of getting that wrong).
When you show things to the user, remember that the user doesn't care. To the user: Either it worked, or it didn't work. If it didn't work, the user can fix the problem if it is a problem they can fix, they can try again later, or it's just tough luck. In an enterprise app, you might have a message "call your help desk at xxxxxx and tell them yyyyyy".
And when things don't work, don't annoy the user by showing error after error after error. If you send then requests, don't tell the user ten times that the server is on fire.
There are things that you just don't expect to go wrong. If you download a file, and you can't put it where it belongs, well, that's tough. It shouldn't happen. The user can't do anything about it. (Well, maybe they can. If the storage of the device is full then you can tell the user). Apart from that, it's the same category as "Something went wrong, and the user can't do anything about it". You may find out as a developer what the cause is and fix it, but if it happens with an application out in the user's hands, there's nothing reasonable you can do.
Since all such requests should be asynchronous, you will always pass either one or two callback blocks to the call, one for success and one for failure. I have most of the error handling in the download code, so things like asking the user to turn WiFi on happen only once, and calls may even be repeated automatically if such an error condition is fixed by the user. The error callback is mostly used to inform the application that it won't get the data that it wanted; sometimes the fact that there is an error is useful information in itself.
For consistent error handling, I create my own errors representing either errors returned by the session, or html status codes interpreted as errors. Plus two additional errors "user cancelled" and "no user interaction allowed" if either there was a UI involved and the user cancelled the operation, or I wanted to use some user interaction but wasn't allowed to. The last two errors are different - these errors will never be reported to the user.
I would wrap the errors in your own, but pass the underlying error as a property on your error class (ala C#'s InnerException). That way you are giving consumers a consistent interface, but also providing lower level error detail if required. However, the main reason I would do this is for unit testing. It makes it much easier to mock your ResourceService class and test the code paths for the various errors that could occur.
I don't like the thought of passing back an array of errors, as it adds complexity for the consumer. Instead I would provide an array of InnerException instances. If they are instances of your own error class, they would potentially have their own InnerException's with underlying errors. However, this would probably only make sense if you were doing your own validations where multiple errors might make sense. Your download method will probably have to bail out after the first error encountered.