I recently encounter two data fetching (download) API that performs seemingly the same thing to me. I cannot see when should I use one over the other.
I can use URLSession.shared.dataTask
var tasks: [URLSessionDataTask] = []
func loadItems(tuple : (name : String, imageURL : URL)) {
let task = URLSession.shared.dataTask(with: tuple.imageURL, completionHandler :
{ data, response, error in
guard let data = data, error == nil else { return }
DispatchQueue.main.async() { [weak self] in
self?.displayFlag(data: data, title: tuple.name)
}
})
tasks.append(task)
task.resume()
}
deinit {
tasks.forEach {
$0.cancel()
}
}
Or I can use URLSession.shared.dataTaskPublisher
var cancellables: [AnyCancellable] = []
func loadItems(tuple : (name : String, imageURL : URL)) {
URLSession.shared.dataTaskPublisher(for: tuple.imageURL)
.sink(
receiveCompletion: {
completion in
switch completion {
case .finished:
break
case .failure( _):
return
}},
receiveValue: { data, _ in DispatchQueue.main.async { [weak self] in self?.displayFlag(data: data, title: tuple.name) } })
.store(in: &cancellables)
}
deinit {
cancellables.forEach {
$0.cancel()
}
}
I don't see their distinct differences, as both also can fetch, and both also provide us the ability to cancel the tasks easily. Can someone shed some light on their differences in terms of when to use which?
The first one is the classic. It has been present for quite some time now and most if not all developers are familiar with it.
The second is a wrapper around the first one and allows combining it with other publishers (e.g. Perform some request only when first two requests were performed). Combination of data tasks using the first approach would be far more difficult.
So in a gist: use first one for one-shot requests. Use second one when more logic is needed to combine/pass results with/to other publishers (not only from URLSession). This is, basically, the idea behind Combine framework - you can combine different ways of async mechanisms (datatasks utilising callbacks being one of them).
More info can be found in last year's WWDC video on introducing combine.
Related
I am using the Combine Future to wrap around an async block operation and adding a subscriber to that publisher to receive the values.. I am noticing the future object is not getting deallocated, even after the subscribers are deallocated. The XCode memory graph and instruments leaks graph itself shows no reference to these future objects. I am puzzled why are they still around.
func getUsers(forceRefresh: Bool = false) -> AnyPublisher<[User], Error> {
let future = Future<[User], Error> { [weak self] promise in
guard let params = self?.params else {
promise(.failure(CustomErrors.invalidData))
return
}
self?.restApi.getUsers(params: params, forceRefresh: forceRefresh, success: { (users: [User]?, _) in
guard let users = users else {
return promise(.failure(CustomErrors.invalidData))
}
promise(.success(users))
}) { (error: Error) in
promise(.failure(error))
}
}
return future.eraseToAnyPublisher()
}
Here's how I am adding a subscription:
self.userDataService?.getUsers(forceRefresh: forceRefresh)
.sink(receiveCompletion: { [weak self] completion in
self?.isLoading = false
if case let .failure(error) = completion {
self?.publisher.send(.error(error))
return
}
guard let users = self?.users, !users.isEmpty else {
self?.publisher.send(.empty)
return
}
self?.publisher.send(.data(users))
}) { [weak self] (response: Array<User>) in
self?.users = response
}.store(in: &self.subscribers)
deinit {
self.subscribers.removeAll()
}
This is the screenshot of the leaked memory for the future that got created above.. It's still staying around even after the subscribers are all deleted. Instruments is also showing a similar memory graph. Any thoughts on what could be causing this ??
Future invokes its closure immediately upon creation, which may be impacting this. You might try wrapping the Future in Deferred so that it isn't created until a subscription happens (which may be what you're expecting anyway from scanning the code).
The fact that it's creating one immediately is what (I think) is being reflected in the objects listed when there are no subscribers.
Background
The function below calls two functions, which both access an API, retrieve JSON data, parse through it, etc, and then take that data and populates the values of an object variable in my View Controller class.
func requestWordFromOxfordAPI(word: String, completion: (_ success: Bool) -> Void) {
oxfordAPIManager.fetchDictData(word: word)
oxfordAPIManager.fetchThesData(word: word)
completion(true)
}
Normally, if there was only one function fetching data, and I wanted to call a new function that takes in the data results and does something with them, I would use a delegate method and call it within the closure of the data fetching function.
For Example:
Here, I fetch data from my firebase database and if retrieving the data is succesful, I call self.delegate?.populateWordDataFromFB(result: combinedModel). Since closures occur on separate thread, this ensures that the populateWordDataFromFB function runs only once retrieving the data has finished. Please correct me if I am wrong. I have just recently learned this and am still trying to see the whole picture.
func readData(word: String) {
let docRef = db.collection(K.FBConstants.dictionaryCollectionName).document(word)
docRef.getDocument { (document, error) in
let result = Result {
try document.flatMap {
try $0.data(as: CombinedModel.self)
}
}
switch result {
case .success(let combinedModel):
if let combinedModel = combinedModel {
self.delegate?.populateWordDataFromFB(result: combinedModel)
} else {
self.delegate?.fbDidFailWithError(error: nil, summary: "\(word) not found, requesting from OxfordAPI")
self.delegate?.requestWordFromOxfordAPI(word: word, completion: { (success) in
if success {
self.delegate?.populateWordDataFromOX()
} else {print("error with completion handler")}
})
}
case .failure(let error):
self.delegate?.fbDidFailWithError(error: error, summary: "Error decoding CombinedModel")
}
}
}
Also notice from the above code that if the data is not in firebase, I call the delegate method below, which is where I am running into my issue.
self.delegate?.requestWordFromOxfordAPI(word: word, completion: { (success) in
if success {
self.delegate?.populateWordDataFromOX()
} else {print("error with completion handler")}
})
My Issue
What I am struggling with is the fact that the oxfordAPIManager.fetchDictData(word: word) and oxfordAPIManager.fetchThesData(word: word) functions both have closures.
The body of these functions look like this:
if let url = URL(string: urlString) {
var request = URLRequest(url: url)
request.addValue(K.APISettings.acceptField, forHTTPHeaderField: "Accept")
request.addValue(K.APISettings.paidAppID , forHTTPHeaderField: "app_id")
request.addValue(K.APISettings.paidAppKey, forHTTPHeaderField: "app_key")
let session = URLSession.shared
_ = session.dataTask(with:request) { (data, response, error) in
if error != nil {
self.delegate?.apiDidFailWithError(error: error, summary: "Error performing task:")
return
}
if let safeData = data {
if let thesaurusModel = self.parseThesJSON(safeData) {
self.delegate?.populateThesData(thesModel: thesaurusModel, word: word)
}
}
}
.resume()
} else {print("Error creating thesaurus request")}
I assume both of these functions are running on separate threads in the background. My goal is to call another function once both the oxfordAPIManager.fetchDictData(word: word) and oxfordAPIManager.fetchThesData(word: word) functions run. These two functions will populate the values of an object variable in my view controller which I will use in the new function. I don't want the new function to be called before the object variable in the view controller is populated with the right data so I tried to implement a completion handler. The completion handler function is being called BEFORE the two functions terminate, so when the new function tries to access the object variable in the View Controller, it's empty.
This is my first time trying to implement a completion handler and I tried to follow some other stack overflow posts but was unsuccessful. Also if this is the wrong approach let me know too, please. Sorry for the long explanation and thank you for any input.
Use DispatchGroup for this,
Example:
Create a DispatchGroup,
let group = DispatchGroup()
Modify the requestWordFromOxfordAPI(word: completion:) method to,
func requestWordFromOxfordAPI(word: String, completion: #escaping (_ success: Bool) -> Void) {
fetchDictData(word: "")
fetchThesData(word: "")
group.notify(queue: .main) {
//code after both methods are executed
print("Both methods executed")
completion(true)
}
}
Call enter() and leave() methods of DispatchGroup at the relevant places in fetchDictData(word:) and fetchThesData(word:) methods.
func fetchDictData(word: String) {
group.enter()
URLSession.shared.dataTask(with: url) { (data, response, error) in
//your code
group.leave()
}.resume()
}
func fetchThesData(word: String) {
group.enter()
URLSession.shared.dataTask(with: url) { (data, response, error) in
//your code
group.leave()
}.resume()
}
At last call requestWordFromOxfordAPI(word: completion:)
requestWordFromOxfordAPI(word: "") { (success) in
print(success)
}
I am trying to do the following approach,
let operationQueue = OperationQueue()
operationQueue.maxConcurrentOperationCount = 10
func registerUser(completionHandler: #escaping (Result<Data, Error>) -> Void) -> String {
self.registerClient() { (result) in
switch result {
case .success(let data):
self.downloadUserProfile(data.profiles)
case .failure(let error):
return self.handleError(error)
}
}
}
func downloadUserProfile(urls: [String]) {
for url in urls {
queue.addOperation {
self.client.downloadTask(with: url)
}
}
}
I am checking is there anyway I can get notified when all operations gets completed and then I can call the success handler there.
I tried checking the apple dev documentation which suggests to use
queue.addBarrierBlock {
<#code#>
}
but this is available only from iOS 13.0
Pre iOS 13, we’d use dependencies. Declare a completion operation, and then when you create operations for your network requests, you’d define those operations to be dependencies for your completion operation.
let completionOperation = BlockOperation { ... }
let networkOperation1 = ...
completionOperation.addDependency(networkOperation1)
queue.addOperation(networkOperation1)
let networkOperation2 = ...
completionOperation.addDependency(networkOperation2)
queue.addOperation(networkOperation2)
OperationQueue.main.addOperation(completionOperation)
That having been said, you should be very careful with your operation implementation. Do I correctly infer that downloadTask(with:) returns immediately after the download task has been initiated and doesn’t wait for the request to finish? In that case, neither dependencies nor barriers will work the way you want.
When wrapping network requests in an operation, you’d want to make sure to use an asynchronous Operation subclass (e.g. https://stackoverflow.com/a/32322851/1271826).
The pre-iOS 13 way is to observe the operationCount property of the operation queue
var observation : NSKeyValueObservation?
...
observation = operationQueue.observe(\.operationCount, options: [.new]) { observed, change in
if change.newValue == 0 {
print("operations finished")
}
}
}
I use the following piece of code to generate a cold RxSwift Observable:
func doRequest<T :Mappable>(request:URLRequestConvertible) -> Observable<T> {
let observable = Observable<T>.create { [weak self] observer in
guard let self = self else { return Disposables.create() }
self.session.request(request).validate().responseObject { (response: AFDataResponse<T>) in
switch response.result {
case .success(let obj):
observer.onNext(obj)
observer.onCompleted()
case .failure(let error):
let theError = error as Error
observer.onError(theError)
}
}
return Disposables.create()
}
return observable
}
where Mappable is an ObjectMapper based type, and self.session is an Alamofire's Session object.
I can't find an equivalent to Observable.create {...} in Apple's Combine framework. What I only found is URLSession.shared.dataTaskPublisher(for:) which creates a publisher using Apple's URLSession class.
How can I convert the above observable to an Alamofire Combine's publisher ?
EDIT:
using the solution provided by rob, I ended up with the following:
private let apiQueue = DispatchQueue(label: "API", qos: .default, attributes: .concurrent)
func doRequest<T>(request: URLRequestConvertible) -> AnyPublisher<T, AFError> where T : Mappable {
Deferred { [weak self] () -> Future<T, AFError> in
guard let self = self else {
return Future<T, AFError> { promise in
promise(.failure(.explicitlyCancelled)) }
}
return Future { promise in
self.session
.request(request)
.validate()
.responseObject { (response: AFDataResponse<T>) in
promise(response.result)
}
}
}
.handleEvents(receiveCompletion: { completion in
if case .failure (let error) = completion {
//handle the error
}
})
.receive(on: self.apiQueue)
.eraseToAnyPublisher()
}
EDIT2: I have to remove the private queue since it's not needed, Alamofire does the parsing the decoding on its own, so remove the queue and its usages (.receive(on: self.apiQueue))
You can use Future to connect responseObject's callback to a Combine Publisher. I don't have Alamofire handy for testing, but I think the following should work:
func doRequest<T: Mappable>(request: URLRequestConvertible) -> AnyPublisher<T, AFError> {
return Future { promise in
self.session
.request(request)
.validate()
.responseObject { (response: AFDataResponse<T>) in
promise(response.result)
}
}.eraseToAnyPublisher()
}
Note that this is somewhat simpler than the RxSwift version because promise takes a Result directly, so we don't have to switch over response.result.
A Future is sort of a “lukewarm” publisher. It is like a hot observable because it executes its body immediately and only once, so it starts the Alamofire request immediately. It is also like a cold observable, because every subscriber eventually receives a value or an error (assuming you eventually call promise). The Future only executes its body once, but it caches the Result you pass to promise.
You can create a truly cold publisher by wrapping the Future in a Deferred:
func doRequest<T: Mappable>(request: URLRequestConvertible) -> AnyPublisher<T, AFError> {
return Deferred {
Future { promise in
self.session
.request(request)
.validate()
.responseObject { (response: AFDataResponse<T>) in
promise(response.result) }
}
}.eraseToAnyPublisher()
}
Deferred calls its body to create a new inner Publisher every time you subscribe to it. So each time you subscribe, you'll create a new Future that will immediately start a new Alamofire request. This is useful if you want to use the retry operator, as in this question.
I have 10 urls in an array and when 4 of them downloaded I need to display them. Im using Semaphores and groups to implement . But looks like im hitting deadlock. Not sure how to proceed. Please advice how I can
Simulating same in playground:
PlaygroundPage.current.needsIndefiniteExecution = true
let group = DispatchGroup()
let queue = DispatchQueue.global(qos: .userInteractive)
let semaphore = DispatchSemaphore(value: 4)
var nums: [Int] = []
for i in 1...10 {
group.enter()
semaphore.wait()
queue.async(group: group) {
print("Downloading image \(i)")
// Simulate a network wait
Thread.sleep(forTimeInterval: 3)
nums.append(i)
print("Hola image \(i)")
if nums.count == 4 {
print("4 downloaded")
semaphore.signal()
group.leave()
}
}
if nums.count == 4 {
break
}
}
group.notify(queue: DispatchQueue.main) {
print(nums)
}
I get this in o/p console
> Downloading image 1
> Downloading image 2
> Downloading image 3
> Downloading image 4
Semaphores(41269,0x70000ade5000) malloc: *** error for object 0x1077d4750: pointer being freed was not allocated
Semaphores(41269,0x70000ade5000) malloc: *** set a breakpoint in malloc_error_break to debug
I'm expecting to print [1,2,3,4] in order
I know im trying to access a shared resource in async but not sure how I can fix this. Please advice
Also How can I use this with semaphore's if I want to download 4,4,2 tasks at a time so it display [1,2,3,4,5,6,7,8,9,10] in my ouput
Your title says “Downloading Images in order of url’s”, but your code snippet is not attempting to do that. It appears to be attempting to use semaphores to constrain the download to four images at a time, but it won’t guarantee that they’ll be in order.
It is commendable that this code snippet isn’t attempting to download them in order, sequentially, one after another, because that would impose a huge performance penalty. It is also good that this code snippet is constraining this degree of concurrency to something reasonable, thereby avoiding exhausting worker threads or causing some of the latter requests to timeout. So, the idea of using semaphore to allow concurrent image download, but constrain it to four at a time, is a fine approach; we only need to sort the results at the end if you want them in order.
But before we get to that, let’s tackle a bunch of problems in the supplied code snippet:
You are calling group.enter() and semaphore.wait() for every iteration (which is correct), but group.leave() and semaphore.signal() only when i is 4 (which is not correct). You want to leave and signal for every iteration.
Obviously, that break call is not needed, either.
So, to fix this “do four at a time” process, one can simplify this code:
let group = DispatchGroup()
let queue = DispatchQueue.global(qos: .userInteractive)
let semaphore = DispatchSemaphore(value: 4)
var nums: [Int] = []
for i in 1...10 {
group.enter()
semaphore.wait()
queue.async() { // NB: the `group` parameter is not needed
print("Downloading image \(i)")
// Simulate a network wait
Thread.sleep(forTimeInterval: 3)
nums.append(i)
print("Hola image \(i)")
semaphore.signal()
group.leave()
}
}
group.notify(queue: .main) {
print(nums)
}
That will download four images at a time and will call your group.notify closure when they’re all done.
While the above fixes the semaphore and group logic, there is yet another problem lurking in the above code snippet. It is updating that nums array from multiple background threads, but Array is not thread-safe. So you should synchronize those updates to that array. An easy way to achieve this is to dispatch that update back to the main thread. (Any serial queue would have been fine, but the main thread works fine for this purpose.)
Also, since one should never call wait on the main queue, so I’d suggest that you explicitly dispatch this entire for loop to a background thread:
DispatchQueue.global(qos: .utility).async {
let group = DispatchGroup()
let queue = DispatchQueue.global(qos: .userInteractive)
let semaphore = DispatchSemaphore(value: 4)
var nums: [Int] = []
for i in 1...10 {
group.enter()
semaphore.wait()
queue.async() {
print("Downloading image \(i)")
// Simulate a network wait
Thread.sleep(forTimeInterval: 3)
DispatchQueue.main.async {
nums.append(i)
print("Hola image \(i)")
}
semaphore.signal()
group.leave()
}
}
group.notify(queue: .main) {
print(nums)
}
}
That is now the correct “do four at a time and let me know when it’s done.”
OK, now that we’re downloading all of the images properly, let’s figure out how to sort the results. Frankly, I think it’s easier to follow what’s going on if we imagine that we have some image download method, like so, that downloads a particular image:
func download(_ url: URL, completion: #escaping (Result<UIImage, Error>) -> Void) { ... }
Then the routine to (a) download the images, no more than four at a time; and (b) return the results back in order, might look like:
func downloadAllImages(_ urls: [URL], completion: #escaping ([UIImage]) -> Void) {
DispatchQueue.global(qos: .utility).async {
let group = DispatchGroup()
let semaphore = DispatchSemaphore(value: 4)
var imageDictionary: [URL: UIImage] = [:]
// download the images
for url in urls {
group.enter()
semaphore.wait()
self.download(url) { result in
defer {
semaphore.signal()
group.leave()
}
switch result {
case .failure(let error):
print(error)
case .success(let image):
DispatchQueue.main.async {
imageDictionary[url] = image
}
}
}
}
// now sort the results
group.notify(queue: .main) {
completion(urls.compactMap { imageDictionary[$0] })
}
}
}
And you’d call it like so:
downloadAllImages(urls) { images in
self.images = images
self.updateUI() // do whatever you want to trigger the update of the UI
}
FWIW, the “download single image” routine might look like:
enum DownloadError: Error {
case notImage
case invalidStatusCode(URLResponse)
}
func download(_ url: URL, completion: #escaping (Result<UIImage, Error>) -> Void) {
URLSession.shared.dataTask(with: url) { data, response, error in
guard let data = data, let response = response as? HTTPURLResponse, error == nil else {
completion(.failure(error!))
return
}
guard 200..<300 ~= response.statusCode else {
completion(.failure(DownloadError.invalidStatusCode(response)))
return
}
guard let image = UIImage(data: data) else {
completion(.failure(DownloadError.notImage))
return
}
completion(.success(image))
}
}
And this is using the Swift 5 Result enumeration. If you’re using an earlier version of Swift, you can define a simple rendition of this enum yourself:
enum Result<Success, Failure> {
case success(Success)
case failure(Failure)
}
Finally, it’s worth noting a few other alternatives:
Wrap your network request in asynchronous Operation subclass and add them to an operation queue whose maxConcurrentOperationCount is set to 4. If you’re interested in this approach, I can supply some references.
Use an image downloading library like Kingfisher.
Instead of manual downloading of all the images, use the UIImageView extension (such as provided by Kingfisher) and completely abandon the “download all images” process at all, and move to a pattern where you simply instruct your image views to asynchronously retrieve the images in either a just-in-time manner (or prefetching).