I'd like to handle a series of network calls in my app. Each call is asynchronous and flatMap() seems like the right call. However, flatMap processes all arguments at the same time and I need the calls to be sequential -- the next network call starts only after the previous one is finished. I looked up an RxSwift answer but it requires concatMap operator that Combine does not have. Here is rough outline of what I'm trying to do, but flatMap fires all myCalls at the same time.
Publishers.Sequence(sequence: urls)
.flatMap { url in
Publishers.Future<Result, Error> { callback in
myCall { data, error in
if let data = data {
callback(.success(data))
} else if let error = error {
callback(.failure(error))
}
}
}
}
After experimenting for a while in a playground, I believe I found a solution, but if you have a better idea, please share. The solution is to add maxPublishers parameter to flatMap and set the value to max(1)
Publishers.Sequence(sequence: urls)
.flatMap(maxPublishers: .max(1)) // <<<<--- here
{ url in
Publishers.Future<Result, Error> { callback in
myCall { data, error in
if let data = data {
callback(.success(data))
} else if let error = error {
callback(.failure(error))
}
}
}
}
You can also use prepend(_:) method on observable which creates concatenated sequence which, I suppose is similar to Observable.concat(:) in RxSwift.
Here is a simple example that I tried to simulate your use case, where I have few different sequences which are followed by one another.
func dataTaskPublisher(_ urlString: String) -> AnyPublisher<(data: Data, response: URLResponse), Never> {
let interceptedError = (Data(), URLResponse())
return Publishers.Just(URL(string: urlString)!)
.flatMap {
URLSession.shared
.dataTaskPublisher(for: $0)
.replaceError(with: interceptedError)
}
.eraseToAnyPublisher()
}
let publisher: AnyPublisher<(data: Data, response: URLResponse), Never> = Publishers.Empty().eraseToAnyPublisher()
for urlString in [
"http://ipv4.download.thinkbroadband.com/1MB.zip",
"http://ipv4.download.thinkbroadband.com/50MB.zip",
"http://ipv4.download.thinkbroadband.com/10MB.zip"
] {
publisher = publisher.prepend(dataTaskPublisher(urlString)).eraseToAnyPublisher()
}
publisher.sink(receiveCompletion: { completion in
print("Completed")
}) { response in
print("Data: \(response)")
}
Here, prepend(_:) operator prefixes the sequence and so, prepended sequences starts first, completes and next sequence start.
If you run the code below, you should see that firstly 10 MB file is download, then 50 MB and at last 1 MB, since the last prepended starts first and so on.
There is other variant of prepend(_:) operator which takes array, but that does not seem to work sequentially.
Related
I have two independent observables. I need to perform some operation when both of them are complete and each of them provided an array.
let myObj1Array = myObj1Manager.getMyObj1List()//returns Observable<[MyObj1]>
let myObj2Array = myObj2Manager.getMyObj2List()//returns Observable<[MyObj2]>
Now I need to compare values of myObj1Array and myObj2Array and on the basis of that create another array using values from both arrays. I know how to subscribe 1 variable but not sure how to observe completion of two different arrays.
Edit:
I also tried following but I get values only from first array:
let myObj1Array = myObj1Manager.getMyObj1List()
let myObj2Array = myObj1Array.flatMap { _ in myObj2Manager.getMyObj2List() }
Observable.combineLatest(myObj1Array, myObj2Array)
.subscribe(onNext: { (sss, sds) in
print(sss)
})
.addDisposableTo(disposeBag)
I am actually kind of clueless about how to handle such scenario.
Edit2:
function to get the observables in first array:
func getMyObj1List() -> Observable<[MyObj1]> {
return Observable.create { observer -> Disposable in
self.specialsRest.getMyObj1List { response, error in
if let error = error {
observer.onError(Exception(error))
return
}
guard let saleItems = MyObj1.decode(data: response?.data) else {
observer.onError(Exception("Could not decode specials!"))
return
}
queueBackground.async {
observer.onNext(saleItems)
observer.onCompleted()
}
}
return Disposables.create { self.specialsRest.cancel() }
}
}
DispatchGroup is probably the way to go here.
https://developer.apple.com/documentation/dispatch/dispatchgroup
When all work items finish executing, the group executes its completion handler. You can also wait synchronously for all tasks in the group to finish executing.
var dg:DispatchGroup = DispatchGroup()
//Wherever you start your observables.
//Start Observer1
dg.enter()
//Start Observer2
dg.enter()
...
...
...
//Wherever you retrieve data
SomeAsyncFuncForObserver1 {
//Get Data
dg.leave()
}
SomeAsyncFuncForObserver2 {
//Get Data
dg.leave()
}
dg.notify(queue: .main) {
print("all finished.")
}
I believe you need to use zip instead of combineLatest. From the docs
The CombineLatest operator behaves in a similar way to Zip, but while
Zip emits items only when each of the zipped source Observables have
emitted a previously unzipped item, CombineLatest emits an item
whenever any of the source Observables emits an item (so long as each
of the source Observables has emitted at least one item).
Observable
.zip(myObj1Array, myObj2Array)
.subscribe(onNext: { (sss, sds) in
print(sss)
})
.addDisposableTo(disposeBag)
Top Level Question:
I want to know how, within a retry, I can modify its source observable if it is an observable shared between multiple subscribers (in this case a BehaviorSubject/Relay).
Solution(s) I have considered:
The suggestion of using defer from this post doesn't seem to naturally port over if the source observable needs to be shared.
Use case (to fully elaborate the question)
Say I have a server connection object that, when initialized, connects to an url. Once it is created, I can also use it to get a data stream for a particular input.
class ServerConnection {
var url: URL
init(url: URL)
func getDataStream(input: String) -> Observable<Data> // the observable also errors when the instance is destroyed.
}
However, one particular url or another may be broken or overloaded. So I may want to obtain the address of a mirror and generate a new ServerConnection object. Let's say I have such a function.
// At any point in time, gets the mirror of the url with the lowest load
func getLowestLoadMirror(url: URL) -> URL {}
Ideally, I want this "mirror url" switching should be an implementation detail. The user of my code may only care about the data they receive. So we would want to encapsulate this logic in a new class:
class ServerConnectionWithMirrors {
private var currentConnection: BehaviorRelay<ServerConnection>
init(startingURL: URL)
func dataStream(for inputParams: String) -> Observable<Data>
}
// usage
let connection = ServerConnectionWithMirrors(startingURL: "www.example.com")
connection.dataStream(for: "channel1")
.subscribe { channel1Data in
// do something with channel1Data
}.disposed(by: disposeBag)
connection.dataStream(for: "channel2")
.subscribe { channel2Data in
// do something with channel2Data
}.disposed(by: disposeBag)
How should I write the dataStream() function for ServerConnectionWithMirrors? I should be using retries, but I need to ensure that the retries, when faced with a particular error (ServerOverLoadedError) update the value on the behaviorRelay.
Here is code that I have so far that demonstrates the crux at what I am trying to do. One problem is that multiple subscribers to the behaviorRelay may all update it in rapid succession when they get an error, where only one update would do.
func dataStream(for inputParams: String) -> Observable<Data> {
self.currentConnection.asObservable()
.flatMapLatest { server in
return server.getDataStream(input: inputParams)
}
.retryWhen { errors in
errors.flatMapLatest { error in
if error is ServerOverLoadedError {
self.currentConnection.accept(ServerConnection(url: getLowestLoadURL()))
} else {
return Observable.error(error)
}
}
}
}
The answer to your top level question:
I want to know how, within a retry, I can modify its source observable if it is an observable shared between multiple subscribers (in this case a BehaviorSubject/Relay).
You cannot modify a retry's source observable from within the retry. (full stop) You cannot do this whether it is shared or not. What you can do is make the source observable in such a way that it naturally updates its data for every subscription.
That is what the question you referred to is trying to explain.
func getData(from initialRequest: URLRequest) -> Observable<Data> {
return Observable.deferred {
var correctRequest = initialRequest
let correctURL = getLowestLoadMirror(url: initialRequest.url!)
correctRequest.url = correctURL
return Observable.just(correctRequest)
}
.flatMapLatest {
getDataFromServer(request: $0)
}
.retryWhen { error in
error
.do(onNext: {
guard $0 is ServerOverloadedError else { throw $0 }
})
}
}
With the above code, every time deferred is retried, it will call its closure and every time its closure is called, the URL will the lowest load will be used.
How to convert URLSession.DataTaskPublisher to Future in Combine framework.
In my opinion, the Future publisher is more appropriate here because the call can emit only one response and fails eventually.
In RxSwift there is helper method like asSingle.
I have achieved this transformation using the following approach but have no idea if this is the best method.
return Future<ResponseType, Error>.init { (observer) in
self.urlSession.dataTaskPublisher(for: urlRequest)
.tryMap { (object) -> Data in
//......
}
.receive(on: RunLoop.main)
.sink(receiveCompletion: { (completion) in
if case let .failure(error) = completion {
observer(.failure(error))
}
}) { (response) in
observer(.success(response))
}.store(in: &self.cancellable)
}
}
Is there any easy way to do this?
As I understand it, the reason to use .asSingle in RxSwift is that, when you subscribe, your subscriber receives a SingleEvent which is either a .success(value) or a .error(error). So your subscriber doesn't have to worry about receiving a .completion type of event, because there isn't one.
There is no equivalent to that in Combine. In Combine, from the subscriber's point of view, Future is just another sort of Publisher which can emit output values and a .finished or a .failure(error). The type system doesn't enforce the fact that a Future never emits a .finished.
Because of this, there's no programmatic reason to return a Future specifically. You could argue that returning a Future documents your intent to always return either exactly one output, or a failure. But it doesn't change the way you write your subscriber.
Furthermore, because of Combine's heavy use of generics, as soon as you want to apply any operator to a Future, you don't have a future anymore. If you apply map to some Future<V, E>, you get a Map<Future<V, E>, V2>, and similar for every other operator. The types quickly get out of hand and obscure the fact that there's a Future at the bottom.
If you really want to, you can implement your own operator to convert any Publisher to a Future. But you'll have to decide what to do if the upstream emits .finished, since a Future cannot emit .finished.
extension Publisher {
func asFuture() -> Future<Output, Failure> {
return Future { promise in
var ticket: AnyCancellable? = nil
ticket = self.sink(
receiveCompletion: {
ticket?.cancel()
ticket = nil
switch $0 {
case .failure(let error):
promise(.failure(error))
case .finished:
// WHAT DO WE DO HERE???
fatalError()
}
},
receiveValue: {
ticket?.cancel()
ticket = nil
promise(.success($0))
})
}
}
}
Instead of converting a data task publisher into a Future, convert a data task into a Future. Just wrap a Future around a call to a URLSession's dataTask(...){...}.resume() and the problem is solved. This is exactly what a future is for: to turn any asynchronous operation into a publisher.
How to return a future from a function
Instead of trying to return a 'future' from a function you need to convert your existing publisher to a AnyPublisher<Value, Error>. You do this by using the .eraseToAnyPublisher() operator.
func getUser() -> AnyPublisher<User, Error> {
URLSession.shared.dataTaskPublisher(for: request)
.tryMap { output -> Data in
// handle response
return output.data
}
.decode(type: User.self, decoder: JSONDecoder())
.mapError { error in
// handle error
return error
}
.eraseToAnyPublisher()
}
I am initiating an operation via a REST API in two steps:
Start operation and return a task id
Poll task with the given id and complete the sequence when the operation returns complete.
The polling the task id will return a 202 which indicates that the operation is still in progress and a 200 when it completes. Any other code is an error.
I need to communicate to the subscribers the response of each step.
Previously, I would have the do operator push the response in between steps to a ReplaySubject.
startReboot()
.do(onNext: { response in
operationStatus.next(response)
)
.flatMap({ response in
// If we could not get the task ID from the response we error
guard let taskID = getTaskIDFromJSON(response) else { return Observable.error(API.serverError) }
return Observable.just(taskID)
})
.flatMap({ taskID in
return pollTask(withID: taskID) // internally, it uses retryWhen to check the api again with a five second delay
})
.do(onNext: { response in
operationStatus.next(response)
})
.subscribe(onError: { _ in
showOperationFailedIcon()
}, onCompleted: {
showOperationCompleteIcon()
})
And somewhere else, a subscriber to the subject would do the following:
operationStatus.subscribe(onNext: { response
showResponse(response)
})
So essentially I am showing the progress of the operation and the response we get from each step at the same time.
At the time I was not familiar with Rx to come up with a cleaner solution. But now that I have familiarized my self with it, it seems to me that there should be a solution where we don't use side effects and contain this to a single final observable. Still, I cannot find a way to do it.
I was thinking about something like this:
let opObs = startReboot()
let pollingObs = pollTask(/* where does the id come from? */)
Observable.concat(opObs, pollingObs)
.subscribe(onNext : { response in
showResponse(response)
}, onError: { _ in
showOperationFailedIcon()
}, onCompleted: {
showOperationCompleteIcon()
})
But that would imply that once opObs is done I would need to save the task id in variable outside of the monad and wrap pollingObs to fetch it when it starts - once again introducing side effects.
Is there an operator or combination of operators that I can use to emit the response of each step to a subscriber and also pass it to another observable?
Something like this should work. Note the use of share() to avoid triggering twice the startReboot sequence.
let opObs = startReboot().share()
let pollingObs = opObs.flatMap {
guard let taskID = getTaskIDFromJSON(response) else { return Observable.error(API.serverError)
return pollTask(withID: taskID)
}
Observable.concat(opObs, pollingObs)
.subscribe(onNext : { response in
showResponse(response)
}, onError: { _ in
showOperationFailedIcon()
}, onCompleted: {
showOperationCompleteIcon()
})
I am still a beginner in Reactive programming, and RxSwift in general.
I want to chain two different operation. In my case I simply want to download a zip file from a web server, and then unzip it locally.
I also want, at the same time to show the progress of the downloaded files.
So I started creating the first observable:
class func rx_download(req:URLRequestConvertible, testId:String) -> Observable<Float> {
let destination:Request.DownloadFileDestination = ...
let obs:Observable<Float> = Observable.create { observer in
let request = Alamofire.download(req, destination: destination)
request.progress { _, totalBytesWritten, totalBytesExpectedToWrite in
if totalBytesExpectedToWrite > 0 {
observer.onNext(Float(totalBytesWritten) / Float(totalBytesExpectedToWrite))
}
else {
observer.onNext(0)
}
}
request.response { _, response, _, error in
if let responseURL = response {
if responseURL.statusCode == 200 {
observer.onNext(1.0)
observer.onCompleted()
} else {
let error = NSError(domain: "error", code: responseURL.statusCode, userInfo: nil)
observer.onError(error)
}
} else {
let error = NSError(domain: "error", code: 500, userInfo: nil)
observer.onError(error)
}
}
return AnonymousDisposable () {
request.cancel()
}
}
return obs.retry(3)
}
After that, I create a similar function for the unzip
class func rx_unzip(testId:String) -> Observable<Float> {
return Observable.create { observer in
do {
try Zip.unzipFile(NSURL.archivePath(testId), destination: NSURL.resourceDirectory(testId), overwrite: true, password: nil)
{progress in
observer.onNext(Float(progress))
}
} catch let error {
observer.onError(error)
}
observer.onCompleted()
return NopDisposable.instance
}
}
For now I have this logic on the "View model layer", so I download-> subscribe on completed-> unzip
What I want is to combine the two Observable in one, in order to perform the download first, then on completed unzip the file. Is there any way to do this?
Concat operator requires the same data type
Indeed, the concat operator allows you to enforce the sequence of observables, however an issue you might encounter with using concat is that the concat operator requires that the Observables have the same generic type.
let numbers : Observable<Int> = Observable.from([1,2,3])
let moreNumbers : Observable<Int> = Observable.from([4,5,6])
let names : Observable<String> = Observable.from(["Jose Rizal", "Leonor Rivera"])
// This works
numbers.concat(moreNumbers)
// Compile error
numbers.concat(names)
FlatMap operator allows you to chain a sequence of Observables
Here's an example.
class Tag {
var tag: String = ""
init (tag: String) {
self.tag = tag
}
}
let getRequestReadHTML : Observable<String> = Observable
.just("<HTML><BODY>Hello world</BODY></HTML>")
func getTagsFromHtml(htmlBody: String) -> Observable<Tag> {
return Observable.create { obx in
// do parsing on htmlBody as necessary
obx.onNext(Tag(tag: "<HTML>"))
obx.onNext(Tag(tag: "<BODY>"))
obx.onNext(Tag(tag: "</BODY>"))
obx.onNext(Tag(tag: "</HTML>"))
obx.onCompleted()
return Disposables.create()
}
}
getRequestReadHTML
.flatMap{ getTagsFromHtml(htmlBody: $0) }
.subscribe (onNext: { e in
print(e.tag)
})
Notice how getRequestReadHTML is of type Observable<String> while the function getTagsFromHtml is of type Observable<Tag>.
Using multiple flatMaps can increase emission frequency
Be wary however, because the flatMap operator takes in an array (e.g. [1,2,3]) or a sequence (e.g. an Observable) and will emit all of the elements as an emission. This is why it is known to produce a transformation of 1...n.
If you defined an observable such as a network call and you are sure that there will only be one emission, you will not encounter any problems since its transformation is a 1...1 (i.e. one Observable to one NSData). Great!
However, if your Observable has multiple emissions, be very careful because chained flatMap operators will mean emissions will exponentially(?) increase.
A concrete example would be when the first observable emits 3 emissions, the flatMap operator transforms 1...n where n = 2, which means there are now a total of 6 emissions. Another flatMap operator could again transform 1...n where n = 2, which means there are now a total of 12 emissions. Double check if this is your expected behavior.
You can use concat operator to chain these two Observables. The resulting Observable will send next values from the first one, and when it completes, from the second one.
There is a caveat: you will get progress values ranging from 0.0 to 1.0 from rx_download and then again the progress from rx_unzip will start with 0.0. This might be confusing to the user if you want to show the progress on a single progress view.
A possible approach would be to show a label describing what is happening along with the progress view. You can map each Observable to a tuple containing the progress value and the description text and then use concat. It can look like that:
let mappedDownload = rx_download.map {
return ("Downloading", $0)
}
let mappedUnzip = rx_download.map {
return ("Unzipping", $0)
}
mapped1.concat(mapped2)
.subscribeNext({ (description, progress) in
//set progress and show description
})
Of course, there are many possible solutions, but this is more of a design problem than a coding one.