I have a download task that work by first calling a REST API for which the server needs to generate a fairly large file which takes it several minutes to generate, as it is CPU and disk IO intensive. The client waits for the server to give a JSON response with the URL of the file it generated. The file download then starts after it gets the first result.
For the calls that generate a particularly large file, which causes the server to be very slow to respond, I am seeing duplicate requests that my code is not initiating.
Initially the someone who works on the server side told me about the duplicate requests. Then I set up a way to inspect network traffic. This was done by setting up a Mac connected to a wired network and enabling network sharing and using Proxyman to inspect the traffic from the iPhone to the API server. I see multiple instances of the same API request on the network layer but my code was never notified.
Code looks like this
#objc class OfflineMapDownloadManager : NSObject, URLSessionDelegate, URLSessionDownloadDelegate {
#objc func download(){
let config = URLSessionConfiguration.background(withIdentifier: "OfflineMapDownloadSession")
config.timeoutIntervalForRequest = 500
config.shouldUseExtendedBackgroundIdleMode = true
config.sessionSendsLaunchEvents = true
urlSession = URLSession(configuration: config, delegate: self, delegateQueue: nil)
getMapUrlsFromServer(bounds)
}
func getMapUrlsFromServer(){
var urlString = "http://www.fake.com/DoMakeMap.php"
if let url = URL(string: urlString) {
let request = NSMutableURLRequest(url: url)
//...Real code sets up a JSON body in to params...
request.httpBody = params.data(using: .utf8 )
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
request.httpMethod = "POST"
request.timeoutInterval = 500
urlSession?.configuration.timeoutIntervalForRequest = 500
urlSession?.configuration.timeoutIntervalForResource = 500
request.httpShouldUsePipelining = true
let backgroundTask = urlSession?.downloadTask(with: request as URLRequest)
backgroundTask?.countOfBytesClientExpectsToSend = Int64(params.lengthOfBytes(using: .utf8))
backgroundTask?.countOfBytesClientExpectsToReceive = 1000
backgroundTask?.taskDescription = "Map Url Download"
backgroundTask?.resume()
}
}
func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
if (downloadTask.taskDescription == "CTM1 Url Download") {
do {
let data = try Data(contentsOf: location, options: .mappedIfSafe)
let jsonResult = try JSONSerialization.jsonObject(with: data, options: .mutableLeaves)
if let jsonResult = jsonResult as? Dictionary<String, AnyObject> {
if let ctm1Url = jsonResult["CTM1Url"] as? String {
if let filesize = jsonResult["filesize"] as? Int {
currentDownload?.ctm1Url = URL(string: ctm1Url)
currentDownload?.ctm1FileSize = Int32(filesize)
if (Int32(filesize) == 0) {
postDownloadFailed()
} else {
startCtm1FileDownload(ctm1Url,filesize)
}
}
}
}
} catch {
postDownloadFailed()
}
}
}
There is more to this download class as it will download the actual file once the first api call is done. Since the problem happens before that code would be executed, I did not include it in the sample code.
The log from Proxyman shows that the API call went out at (minutes:seconds) 46:06, 47:13, 48:21, 49:30, 50:44, 52:06, 53:45
It looks like the request gets repeated with intervals that are just over 1 minute.
There is an API field where I can put any value and it will be echoed back to me by the server. I put a timestamp there generated with CACurrentMediaTime() and log in Proxyman shows that indeed its the same API call so there is no way my code is getting called multiple times. It seems as though the iOS networking layer is re-issuing the http request because the server is taking a very long time to respond. This ends up causing problems on the server and the API fails.
Any help would be greatly appreciated.
This sounds a lot like TCP retransmission. If the client sends a TCP segment, and the server does not acknowledge receipt within a short span of time, the client assumes the segment didn't make it to the destination, and it sends the segment again. This is a significantly lower-level mechanism than URLSession.
It's possible the HTTP server application this API is using (think Apache, IIS, LigHTTPd, nginx, etc.) is configured to acknowledge with the response data to save packeting and framing overhead. If so, and if the response data takes longer than the client's TCP retransmission timeout, you will get this behavior.
Do you have a packet capture of the connection? If not, try collecting one with tcpdump and reviewing it in Wireshark. If I'm right, you will see multiple requests, and they will all have the same sequence number.
As for how to fix it if that's the problem, I'm not sure. The server should acknowledge requests as soon as they are received.
I think the problem is in using URLSessionConfiguration.background(withIdentifier:) for this api call.
Use this method to initialize a configuration object suitable for transferring data files while the app runs in the background. A session configured with this object hands control of the transfers over to the system, which handles the transfers in a separate process. In iOS, this configuration makes it possible for transfers to continue even when the app itself is suspended or terminated.
So the problem is that the system is retrying your request unnecessarily because of this wrong API usage.
Here's what I recommend -
Use default session configuration (NOT background).
Do this api call that initiates this long job, do NOT have client wait on this job, from server side return a job_id back to client as soon as this job is initiated.
Client can now poll server every X seconds using that job_id value to know about the status of the job, even can show progress on client side if needed.
When job is completed, and client polls next time, it gets the download URL for this big file.
Download the file (using default / background session configuration as you prefer).
Related
I'm currently working on a cross-platform (native Java and Swift/Objective-C) mobile app where the users may frequently have limited access to a network connection, but we want to record and send up statistics when a network becomes available, even if our app has been closed. We’ve already done this on Android fairly easily with their AndroidX.WorkManager library like so:
String URL = "https://myexampleurl.com";
Data inputData = new Data.Builder()
.putString("URL", URL)
.build();
Constraints constraints = new Constraints.Builder().setRequiredNetworkType(NetworkType.CONNECTED).build();
OneTimeWorkRequest workRequest = new OneTimeWorkRequest.Builder(NetworkWorker.class)
.setInputData(inputData)
.setConstraints(constraints)
.build();
WorkManager.getInstance(context).enqueueUniqueWork("com.lkuich.exampleWorker", ExistingWorkPolicy.APPEND_OR_REPLACE, workRequest);
This works great, when our app invokes this method, our request is added to a queue, and even if the user is offline and closes our app, when they reconnect to a network our request still gets made.
I’ve been struggling however to find an iOS equivalent to this API, and I am fairly new to native iOS development. We've enabled the Background Task capability and registered our identifiers in Plist.info in our project.
First we’ve tried the URLSession "background" API like so:
class NetworkController: NSObject, URLSessionTaskDelegate {
func buildTask() -> URLSession {
let config = URLSessionConfiguration.background(withIdentifier: "com.lkuich.exampleWorker")
config.waitsForConnectivity = true
config.shouldUseExtendedBackgroundIdleMode = true
return URLSession(configuration: config, delegate: self, delegateQueue: OperationQueue.main)
}
func makeNetworkRequest(session: URLSession) {
let url = URL(string: "https://myexampleurl.com")!
let task = session.dataTask(with: url)
task.resume()
}
func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) {
print("We're done here")
}
}
This works well when the WiFi is disabled and re-enabled, but if the app is closed the request never gets sent it seems.
So we tried the BGTaskScheduler instead, but I'm having a hard time testing if it's working in the background with the app closed, since it can often take hours to actually run (forcing it to run immediately with a network connection works):
static let bgIdentifier = "com.lkuich.exampleWorker"
// Invoked on app start (didFinishLaunchingWithOptions)
static func registerBackgroundTasks() {
BGTaskScheduler.shared.register(forTaskWithIdentifier: bgIdentifier, using: nil) { (task) in
task.expirationHandler = {
task.setTaskCompleted(success: false)
}
// Make my network request in BG
let url = URL(string: "https://myexampleworker.com")!
let t = URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
// Print the response
print(response)
task.setTaskCompleted(success: true)
})
t.resume()
}
}
static func makeRequest() {
do {
let bgRequest = BGProcessingTaskRequest(identifier: bgIdentifier)
bgRequest.requiresNetworkConnectivity = true
bgRequest.requiresExternalPower = false
try BGTaskScheduler.shared.submit(bgRequest)
print("Submitted task request")
} catch {
print("Failed to submit BGTask")
}
}
Is it possible to schedule a network request to be run in the background when a network is available, even if the invoking app is closed? And if so, how would I pass input data (like URL args for example) to my background task, since my task has to be registered on app start?
There are two possible states a closed app can be in. It's either in the background, or it is killed. App can be killed by the system, for various reasons, or by the user, who can kill it from the app switcher.
This is important because when the app is killed, you can't do anything in the background. Only VoIP apps are exception (using PushKit, which you are now allowed to be using if the app doesn't have VoIP functionality, you won't get pass the AppStore), and watchOS complications. But for all other apps, when the app is killed you have no options at all. That is just Apple's standard, they don't want apps that are not running to be able to run code for security reasons.
You can register some tasks to be done, when the app is still in the background, using the BGTask API you mentioned. I used that same API for one of my apps, and found it extremely volatile. Sometimes a task is done every 30 minutes, and then it won't be executed until tomorrow morning, then at one time it would just stop getting executed at all.
Therefore I think that silent push notifications are the best way out, but they also only work when the app is still in the background, not killed. Unless your app has VoIP functionality, in which case you can use PushKit.
I am using URLSession in my iOS project. (Swift 4). The following code is only for illustration purpose.
class MyTaskManager {
...
func postMyData(...) {
let defaultSession = URLSession(configuration: .default)
dataTask = defaultSession.dataTask(with: url) { data, response, error in
...
}
dataTask.resume()
}
func getMyData(...) {
let defaultSession = URLSession(configuration: .default)
dataTask = defaultSession.dataTask(with: url) { data, response, error in
...
}
dataTask.resume()
}
}
I am trying to understand the best practice of using URLSession in the sense of whether each function call of making HTTP request should create a new URLSession or should I create a global one & all the calls to HTTP requests should use the same URLSession instance?
I have studied on internet, there is an accepted answer which says I should create a new URLSession for each function/request call , there is/are also suggestions that I should reuse the same URLSession. I get confused by those accepted but conflicting answers. Could someone clarify for me the correct answer to this question?
My application doesn't have upload or download tasks, only pure RESTful request with JSON data format. No multiple configurations needed either.
You should create a shared instance of the data session and use the same creating multiple tasks because it's rarely the case that you need to have a different configuration for an api.
I suggest and use the shared instance of data session for getting data from an api.
class MyTaskManager {
static let sessionManager: URLSession = {
let configuration = URLSessionConfiguration.default
configuration.timeoutIntervalForRequest = 30 // seconds
configuration.timeoutIntervalForResource = 30 // seconds
return URLSession(configuration: configuration)
}()
func postMyData(...) {
dataTask = sessionManager.dataTask(with: url) { data, response, error in
...
}
dataTask.resume()
}
func getMyData(...) {
dataTask = sessionManager.dataTask(with: url) { data, response, error in
...
}
dataTask.resume()
}
}
The benefit of this is that I had to create the session only once, so that will save repetition of same code and also the process to initialise the same thing again per api request. This will be more helpful in case you need to more custom configuration of the session.
Most of the time, you should use a single session for all of your work. This allows the session to limit simultaneous requests to a single host (limiting the potential for accidental abuse), and also is significantly more memory efficient than using a new session for each request.
If you have multiple groups of tasks that need to be canceled as a group (for example, uploading all of the images in a new album, downloading all the resources for loading a single web page, etc.), or multiple requests that need a different configuration (e.g. background downloads) then it makes sense to use one session per group or configuration.
Alternatively, if you don't need to make any configuration changes to the default and are happy with just running a block when each request finishes, you can also use the shared session ([NSURLSession sharedSession]) rather than creating a session at all.
-It depends on your usage of the URLSession Object.
-You will create a new one if you need to create your configuration & assign a delegate.
-You will use the shared instance if you won't change neither the configuration nor setting a delegate.
Also this part from the Apple documentation regarding the shared instance limitation:
the shared session has important limitations:
-You can’t obtain data incrementally as it arrives from the server.
-You can’t significantly customize the default connection behavior.
-Your ability to perform authentication is limited.
-You can’t perform background downloads or uploads when your app isn’t running.
I'm fairly new to iOS/Swift development and I'm working on an app that makes several requests to a REST API. Here's a sample of one of those calls which retrieves "messages":
func getMessages() {
let endpoint = "/api/outgoingMessages"
let parameters: [String: Any] = [
"limit" : 100,
"sortOrder" : "ASC"
]
guard let url = createURLWithComponents(endpoint: endpoint, parameters: parameters) else {
print("Failed to create URL!")
return
}
do {
var request = try URLRequest(url: url, method: .get)
let task = URLSession.shared.dataTask(with: request as URLRequest) { (data, response, error) in
if let error = error {
print("Request failed with error: \(error)")
// TODO: retry failed request
} else if let data = data, let response = response as? HTTPURLResponse {
if response.statusCode == 200 {
// process data here
} else {
// TODO: retry failed request
}
}
}
task.resume()
} catch {
print("Failed to construct URL: \(error)")
}
}
Of course, it's possible for this request to fail for a number of different reasons (server is unreachable, request timed out, server returns something other than 200, etc). If my request fails, I'd like to have the ability to retry it, perhaps even with a delay before the next attempt. I didn't see any guidance on this scenario in Apple's documentation but I found a couple of related discussions on SO. Unfortunately, both of those were a few years old and in Objective-C which I've never worked with. Are there any common patterns or implementations for doing something like this in Swift?
This question is airing on the side of opinion-based, and is rather broad, but I bet most are similar, so here goes.
For data updates that trigger UI changes:
(e.g. a table populated with data, or images loading) the general rule of thumb is to notify the user in a non-obstructing way, like so:
And then have a pull-to-refresh control or a refresh button.
For background data updates that don't impact the user's actions or behavior:
You could easily add a retry counter into your request result depending on the code - but I'd be careful with this one and build out some more intelligent logic. For example, given the following status codes, you might want to handle things differently:
5xx: Something is wrong with your server. You may want to delay the retry for 30s or a minute, but if it happens 3 or 4 times, you're going to want to stop hammering your back end.
401: The authenticated user may no longer be authorized to call your API. You're not going to want to retry this at all; instead, you'd probably want to log the user out so the next time they use your app they're prompted to re-authenticate.
Network time-out/lost connection: Retrying is irrelevant until connection is re-established. You could write some logic around your reachability handler to queue background requests for actioning the next time network connectivity is available.
And finally, as we touched on in the comments, you might want to look at notification-driven background app refreshing. This is where instead of polling your server for changes, you can send a notification to tell the app to update itself even when it's not running in the foreground. If you're clever enough, you can have your server repeat notifications to your app until the app has confirmed receipt - this solves for connectivity failures and a myriad of other server response error codes in a consistent way.
I'd categorize three methods for handling retry:
Reachability Retry
Reachability is a fancy way of saying "let me know when network connection has changed". Apple has some snippets for this, but they aren't fun to look at — my recommendation is to use something like Ashley Mill's Reachability replacement.
In addition to Reachability, Apple provides a waitsForConnectivity (iOS 11+) property that you can set on the URLSession configuration. By setting it, you are alerted via the URLSessionDataDelegate when a task is waiting for a network connection. You could use that opportunity to enable an offline mode or display something to the user.
Manual Retry
Let the user decide when to retry the request. I'd say this is most commonly implemented using a "pull to refresh" gesture/UI.
Timed/Auto Retry
Wait for a few second and try again.
Apple's Combine framework provides a convenient way to retry failed network requests. See Processing URL Session Data Task Results with Combine
From Apple Docs: Life Cycle of a URL Session (deprecated)... your app should not retry [a request] immediately, however. Instead, it should use reachability APIs to determine whether the server is reachable, and should make a new request only when it receives a notification that reachability has changed.
I've ran into a bit of a performance issue with my iOS app, this is my first time working with NSURLSession and NSURLRequest, and although I've tried to inform myself as much as I can, I've hit a wall trying to debug a performance issue I'm facing.
So here's what I got: I've got an iOS 9 app written in Swift 2, I'm communicating with a NodeJS/Express server through Get, Post and Put Http requests, utilizing NSURLRequest and NSURLMutableRequest. I'm sending requests to fetch a group of objects (All together no more than 12000 bytes), however the requests are taking a significant amount of time (sometimes up to a minute). I've added logging to the nodeJs server and I can see that the requests take no longer than 30 milliseconds to be processed.
Note: I'm unsure if this is relevant, but I'm using a singleton "helper "class to make all my api requests and parse the results (saving authentication tokens, parsing JSON objects and saving them to Core Data, saving user preferences to NSUserDefaults, etc), I'm using a singleton so i can access it statically and I'm parsing all the data without saving anything in singleton's properties other than the URL of the server and the NSURLSession.
Here's what my code looks like.
//On initialization of the helper class
private let session = NSURLSession.sharedSession()
func getAllObjects() {
let route = "api/someRoute"
let request = getRequest(route)
request.timeoutInterval = httpTimeout
session.dataTaskWithRequest(request, completionHandler: ResultingObjects).resume()
}
The getRequest method returns a formatted NSMutableURLRequest, shown
here:
func getRequest(route: String) -> NSMutableURLRequest {
let request = NSMutableURLRequest()
request.URL = NSURL(string: "\(serverUrl)/\(route)")!
request.HTTPMethod = "GET"
request.addValue("Bearer \(self.AuthenticationToken()!)", forHTTPHeaderField: "Authorization")
return request
}
The completion handler will parse the objects returned and notify
the main thread with the resulting parsed objects, as so:
private func ResultingObjects(data: NSData?, response: NSURLResponse?, error: NSError?) {
if let d = data {
if !isAuthorized(d){
return
}
do {
if let JSON = try NSJSONSerialization.JSONObjectWithData(d, options: []) as? NSDictionary {
if let message = JSON["message"] as? String {
if message == "Empty result" {
//- Return notification to be handled in main thread
notifyMainThread(NoObjectsFetched, payload: nil)
return
}
}
if let objcts = JSON["SomeObjects"] as? NSArray {
if let SomeObjects = parseResultingObjects(objcts) {
//- Return notification to be handled in main thread
notifyMainThread(ObjectsFetched, payload: ["payload": SomeObjects])
}
return
}
}
}
catch {
print("Error getting resulting objects")
}
}
else if let e = error {
print("\(e), could not process GET request")
}
}
I've also tried parsing the resulting objects on the main thread but
that doesn't seem to make a difference.
If you are curious, this is how I'm sending data to the main thread:
private func notifyMainThread(notification: String, payload: AnyObject?) {
dispatch_async(dispatch_get_main_queue(), {
if let p = payload {
NSNotificationCenter.defaultCenter().postNotificationName(notification, object: nil,
userInfo: p as! [String: [MYMODEL]])
}
else {
NSNotificationCenter.defaultCenter().postNotificationName(notification, object: nil)
}
});
}
What I've found out:
Nothing makes sense! I've attempted debugging this but I cant really pin point what the issue is, When the debugger hits my "getAllObjects" method, it can take a good few seconds (up to 45 seconds) before the server logs that it received and processed the request (which it usually takes around 30 milliseconds). As far as I can tell, this happens for all requests types. Also, once the application gets the data back (super fast), it takes a long time (around 4 seconds) to parse it, and its only around 11kbs.
I've also attempted to change the requests cache policy in case the application was checking the validity of the cached records with the server, I used ReloadIgnoringLocalAndRemoteCachedData which didn't work either.
Now, this sounds like a memory leak
If I pause the application at any point (after using it for a few minutes), I can see a concerning number of threads. I'm honestly not too familiar with IOS so I'm unsure if this threads are from the simulator or if they all belong to the app, the app streams video contents (which has no latency issues) usin the AVPlayer class and I believe that many of this threads are related to this, however I'm unsure if this is normal, here's a screenshot of what I mean (Note the scroll bar T_T) Screenshot
Could it be that I've got a memory leak or some zombie threads considerably slowing the performance of my app? The only really noticeable delay happens only on HTTP requests which is quite odd, no other part of my UI lags, and no other feature in my app suffers from performance issues (even streaming video contents from a url).
What would be the best way to profile this performance issues to pin point the source of the problem?
UPDATE 1:
Thanks to the suggestions by Scriptable, I've managed to address the threading issue (caused by multiple AVPlayers doing their thing). The performance of the requests however are not solved.
Its worth pointing out, the server is physically in the same country as where I'm making the requests from, when making requests from the browser or form the Command Line the requests are almost immediate.
Also, when I randomly pause the app (while I wait to the request to happen) I can see a 'mach_msg_trap' in some of the threads, I'm not familiar with this but I believe this might be a race condition? or a deadlock?
I am trying to make queries to get the fuel type and consumption of a specified car (the user enters both make and model) for an iOS app written in Swift.
The app is targeted for Spain, and I have found a website that allows the user to enter make and model, and it returns the details for that car (http://coches.idae.es/portal/BaseDatos/MarcaModelo.aspx). I have seen using the tool WireShark, that the query is based on POST instead of GET. But I am not quite sure how I can make the requests within the app I am developing, or how to handle the info that is sent to me back from the sender.
Is there any way to make those requests to the given website? If so, I would really appreciate some help on the subject, I am new in iOS development and am looking forward to learning as much as possible.
Thanks :)
Many people prefer to use AFNetworking for making HTTP requests. However you don't need to do that. You said that its a POST request. Setting that up is easy even without AFNetworking using NSMutableURLRequest. I'm assuming you have a link to the API and not just to the aspx page. My Spanish is pretty weak so I can't look up the API reference for you but here's how you can make the request and receive data from the server. You will have to put the correct values and parse the responses:
let request = NSMutableURLRequest(URL: NSURL(string: "/* Paste URL here */")!)
request.HTTPMethod = "POST"
// Do this as many times are required for filling in the headers.
request.addValue("/* The value for the HTTP header */", forHTTPHeaderField: "/*The header field like Accept-Type, etc..*/")
// If you need an HTTP body too then make the JSONObj as a dictionary or array or whatever and then
let data = NSJSONSerialization.dataWithJSONObject(JSONObj, options: [])
request.HTTPBody = data // This needs to be NSData.
// Now make the request.
let session = NSURLSession.sharedSession()
let task = session.dataTaskWithRequest(request, { (data, response, error) -> Void in
if error == nil
{
assert(data != nil)
let JSON = NSJSONSerialization.JSONObjectWithData(data!, options: []) as? [NSObject: AnyObject]
// If you are using swift 2 this needs to be in a do try catch statement.
// TODO: Use JSON for whatever.
}
else
{
print(error!.localizedDescription)
}
}
task?.resume()
Let me know if you have any other questions or if the API doesn't use JSON or is completely different.