I need to create dependent API calls where the second one needs a value returned by the first one. First thing that comes to mind is using flatMap
ApiManager.shared
.createReport(report: report)
.flatMap { (report) -> Observable<Report> in
return ApiManager.shared.createReportStep(reportID: report.ID)
}
createReport returns Observable<Report> where after successfull call returns updated Report model(with ID), after that I need to call API to create report step, where report.ID is needed.
Everything looks and works fine with that code, but the problem comes when I need to do something after each of these steps(createReport and createReportStep). I placed code in onNext block, but it is called only once, after both of the steps are completed.
Is there a way to receive onNext signal after both steps? I could use something like this:
ApiManager.shared
.createReport(report: report)
.concat(ApiManager.shared.createReportStep(reportID: report.ID))
Which would emmit two signals like I want, but then again where do I get updated report.ID from to pass to createReportStep?
If you don't mind the time component and only need to have access to both report and what is returned by createReportStep(reportID:), you could go with creating a tuple in flatMap's block
ApiManager.shared
.createReport(report: report)
.flatMap { (report) -> Observable<Report> in
return ApiManager.shared.createReportStep(reportID: report.ID)
.map { (report, $0) }
}
The resulting observable would contain both results in a tuple.
If the time component is important, you could do the following
let report = ApiManager.shared
.createReport(report: report)
.share()
let reportStep = report.map { $0.ID }.flatMap(ApiManager.shared.createReportStep)
Observable.concat([report, reportStep])
Here, the important bit is the share call. It will ensure createReport performs its work only once, but you would have two next events as requested.
Related
I have 2 publishers that return the same type of value. One publisher is meant to emit N/W responses and other is meant to emit cached responses. These publishers will be shared across multiple requests by different consumers.
I am emitting value on them from different code paths. And I am trying to merge these 2 upstream publishers into a single publisher that will return value from either of the 2, to a downstream consumer.
I tried using MergeMany but it did not work for me. Publisher1 was sending value but Publisher2 was not sending values. So the downstream sink did not fire.
I tried using CombineLatest. But that did not work as 1. It returns a tuple like (Response, Response) instead of just Response 2. It again waits for both to return some value. I can't use Zip for the same reasons.
I saw switchToLatest() which sounds promising, but I could not find a way to use it on a list of Publishers.
Edit:
I also tried this and had similar result as MergeMany
return networkPublisher
// .merge(with: cachedPublisher) // If this is uncommented then the network publisher does not process all the network responses for some reason.
.eraseToAnyPublisher()
First, the example in your own answer should actually work without subscribe(on:) and receive(on:). Why this doesn't happen? Because your implementation is most likely not thread safe. I think, that is your main problem. Also please note, that subscribe(on:) set a scheduler for up- AND downstream. This means that receive(on:) is unnecessary in your example.
Below are two variants for merging two publishers:
let subscription = publisher1.merge(with: publisher2)
.sink(receiveValue: { value in
// your logic
})
OR
let subscription = Publishers.Merge(publisher1, publisher2)
.sink(receiveValue: { value in
// your logic
})
Hopefully I could help you.
All I had to add was a receive and subscribe to the chain:
return networkPublisher
.merge(with: cachedPublisher)
.receive(on: dispatchQueue)
.subscribe(on: dispatchQueue)
.eraseToAnyPublisher()
I'm having a bit of a mental block using the iOS Combine framework.
I'm converting some code from "manual" fetching from a remote API to using Combine. Basically, the API is SQL and REST (in actual fact it's Salesforce, but that's irrelevant to the question). What the code used to do is call a REST query method that takes a completion handler. What I'm doing is replacing this everywhere with a Combine Future. So far, so good.
The problem arises when the following scenario happens (and it happens a lot):
We do a REST query and get back an array of "objects".
But these "objects" are not completely populated. Each one of them needs additional data from some related object. So for each "object", we do another REST query using information from that "object", thus giving us another array of "objects".
This might or might not allow us to finish populating the first "objects" — or else, we might have to do another REST query using information from each of the second "object", and so on.
The result was a lot of code structured like this (this is pseudocode):
func fetchObjects(completion: #escaping ([Object] -> Void) {
let restQuery = ...
RESTClient.performQuery(restQuery) { results in
let partialObjects = results.map { ... }
let group = DispatchGroup()
for partialObject in partialObjects {
let restQuery = ... // something based on partialObject
group.enter()
RESTClient.performQuery(restQuery) { results in
group.leave()
let partialObjects2 = results.map { ... }
partialObject.property1 = // something from partialObjects2
partialObject.property2 = // something from partialObjects2
// and we could go down yet _another_ level in some cases
}
}
group.notify {
completion([partialObjects])
}
}
}
Every time I say results in in the pseudocode, that's the completion handler of an asynchronous networking call.
Okay, well, I see well enough how to chain asynchronous calls in Combine, for example by using Futures and flatMap (pseudocode again):
let future1 = Future...
future1.map {
// do something
}.flatMap {
let future2 = Future...
return future2.map {
// do something
}
}
// ...
In that code, the way we form future2 can depend upon the value we received from the execution of future1, and in the map on future2 we can modify what we received from upstream before it gets passed on down the pipeline. No problem. It's all quite beautiful.
But that doesn't give me what I was doing in the pre-Combine code, namely the loop. Here I was, doing multiple asynchronous calls in a loop, held in place by a DispatchGroup before proceeding. The question is:
What is the Combine pattern for doing that?
Remember the situation. I've got an array of some object. I want to loop through that array, doing an asynchronous call for each object in the loop, fetching new info asynchronously and modifying that object on that basis, before proceeding on down the pipeline. And each loop might involve a further nested loop gathering even more information asynchronously:
Fetch info from online database, it's an array
|
V
For each element in the array, fetch _more_ info, _that's_ an array
|
V
For each element in _that_ array, fetch _more_ info
|
V
Loop thru the accumulated info and populate that element of the original array
The old code for doing this was horrible-looking, full of nested completion handlers and loops held in place by DispatchGroup enter/leave/notify. But it worked. I can't get my Combine code to work the same way. How do I do it? Basically my pipeline output is an array of something, I feel like I need to split up that array into individual elements, do something asynchronously to each element, and put the elements back together into an array. How?
The way I've been solving this works, but doesn't scale, especially when an asynchronous call needs information that arrived several steps back in the pipeline chain. I've been doing something like this (I got this idea from https://stackoverflow.com/a/58708381/341994):
An array of objects arrives from upstream.
I enter a flatMap and map the array to an array of publishers, each headed by a Future that fetches further online stuff related to one object, and followed by a pipeline that produces the modified object.
Now I have an array of pipelines, each producing a single object. I merge that array and produce that publisher (a MergeMany) from the flatMap.
I collect the resulting values back into an array.
But this still seems like a lot of work, and even worse, it doesn't scale when each sub-pipeline itself needs to spawn an array of sub-pipelines. It all becomes incomprehensible, and information that used to arrive easily into a completion block (because of Swift's scoping rules) no longer arrives into a subsequent step in the main pipeline (or arrives only with difficulty because I pass bigger and bigger tuples down the pipeline).
There must be some simple Combine pattern for doing this, but I'm completely missing it. Please tell me what it is.
With your latest edit and this comment below:
I literally am asking is there a Combine equivalent of "don't proceed to the next step until this step, involving multiple asynchronous steps, has finished"
I think this pattern can be achieved with .flatMap to an array publisher (Publishers.Sequence), which emits one-by-one and completes, followed by whatever per-element async processing is needed, and finalized with a .collect, which waits for all elements to complete before proceeding
So, in code, assuming we have these functions:
func getFoos() -> AnyPublisher<[Foo], Error>
func getPartials(for: Foo) -> AnyPublisher<[Partial], Error>
func getMoreInfo(for: Partial, of: Foo) -> AnyPublisher<MoreInfo, Error>
We can do the following:
getFoos()
.flatMap { fooArr in
fooArr.publisher.setFailureType(to: Error.self)
}
// per-foo element async processing
.flatMap { foo in
getPartials(for: foo)
.flatMap { partialArr in
partialArr.publisher.setFailureType(to: Error.self)
}
// per-partial of foo async processing
.flatMap { partial in
getMoreInfo(for: partial, of: foo)
// build completed partial with more info
.map { moreInfo in
var newPartial = partial
newPartial.moreInfo = moreInfo
return newPartial
}
}
.collect()
// build completed foo with all partials
.map { partialArr in
var newFoo = foo
newFoo.partials = partialArr
return newFoo
}
}
.collect()
(Deleted the old answer)
Using the accepted answer, I wound up with this structure:
head // [Entity]
.flatMap { entities -> AnyPublisher<Entity, Error> in
Publishers.Sequence(sequence: entities).eraseToAnyPublisher()
}.flatMap { entity -> AnyPublisher<Entity, Error> in
self.makeFuture(for: entity) // [Derivative]
.flatMap { derivatives -> AnyPublisher<Derivative, Error> in
Publishers.Sequence(sequence: derivatives).eraseToAnyPublisher()
}
.flatMap { derivative -> AnyPublisher<Derivative2, Error> in
self.makeFuture(for: derivative).eraseToAnyPublisher() // Derivative2
}.collect().map { derivative2s -> Entity in
self.configuredEntity(entity, from: derivative2s)
}.eraseToAnyPublisher()
}.collect()
That has exactly the elegant tightness I was looking for! So the idea is:
We receive an array of something, and we need to process each element asynchronously. The old way would have been a DispatchGroup and a for...in loop. The Combine equivalent is:
The equivalent of the for...in line is flatMap and Publishers.Sequence.
The equivalent of the DispatchGroup (dealing with asynchronousness) is a further flatMap (on the individual element) and some publisher. In my case I start with a Future based on the individual element we just received.
The equivalent of the right curly brace at the end is collect(), waiting for all elements to be processed and putting the array back together again.
So to sum up, the pattern is:
flatMap the array to a Sequence.
flatMap the individual element to a publisher that launches the asynchronous operation on that element.
Continue the chain from that publisher as needed.
collect back into an array.
By nesting that pattern, we can take advantage of Swift scoping rules to keep the thing we need to process in scope until we have acquired enough information to produce the processed object.
I have a sequence made up of multiple operators. There are total of 7 places where errors can be generated during this sequence processing. I'm running into an issue where the sequence does not behave as I expected and I'm looking for an elegant solution around the problem:
let inputRelay = PublishRelay<Int>()
let outputRelay = PublishRelay<Result<Int>>()
inputRelay
.map{ /*may throw multiple errors*/}
.flatmap{ /*may throw error*/ }
.map{}
.filter{}
.map{ _ -> Result<Int> in ...}
.catchError{}
.bind(to: outputRelay)
I thought that catchError would simply catch the error, allow me to convert it to failure result, but prevent the sequence from being deallocated. However, I see that the first time an error is caught, the entire sequence is deallocated and no more events go through.
Without this behavior, I'm left with a fugly Results<> all over the place, and have to branch my sequence multiple times to direct the Result.failure(Error) to the output. There are non-recoverable errors, so retry(n) is not an option:
let firstOp = inputRelay
.map{ /*may throw multiple errors*/}
.share()
//--Handle first error results--
firstOp
.filter{/*errorResults only*/}
.bind(to: outputRelay)
let secondOp = firstOp
.flatmap{ /*may throw error*/ }
.share()
//--Handle second error results--
secondOp
.filter{/*errorResults only*/}
.bind(to: outputRelay)
secondOp
.map{}
.filter{}
.map{ _ -> Result<Int> in ...}
.catchError{}
.bind(to: outputRelay)
^ Which is very bad, because there are around 7 places where errors can be thrown and I cannot just keep branching the sequence each time.
How can RxSwift operators catch all errors and emit a failure result at the end, but NOT dispose the entire sequence on first error?
The first trick to come to mind is using materialize. This would convert every Observable<T> to Observable<Event<T>>, so an Error would just be a .next(.error(Error)) and won't cause the termination of the sequence.
in this specific case, though, another trick would be needed. Putting your entire "trigger" chain within a flatMap, as well, and materializeing that specific piece. This is needed because a materialized sequence can still complete, which would cause a termination in case of a regular chain, but would not terminate a flatMapped chain (as complete == successfully done, inside a flatMap).
inputRelay
.flatMapLatest { val in
return Observable.just(val)
.map { value -> Int in
if value == 1 { throw SomeError.randomError }
return value + value
}
.flatMap { value in
return Observable<String>.just("hey\(value)")
}
.materialize()
}
.debug("k")
.subscribe()
inputRelay.accept(1)
inputRelay.accept(2)
inputRelay.accept(3)
inputRelay.accept(4)
This will output the following for k :
k -> subscribed
k -> Event next(error(randomError))
k -> Event next(next(hey4))
k -> Event next(completed)
k -> Event next(next(hey6))
k -> Event next(completed)
k -> Event next(next(hey8))
k -> Event next(completed)
Now all you have to do is filter just the "next" events from the materialized sequence.
If you have RxSwiftExt, you can simply use the errors() and elements() operators:
stream.elements()
.debug("elements")
.subscribe()
stream.errors()
.debug("errors")
.subscribe()
This will provide the following output:
errors -> Event next(randomError)
elements -> Event next(hey4)
elements -> Event next(hey6)
elements -> Event next(hey8)
When using this strategy, don't forget adding share() after your flatMap, so many subscriptions don't cause multiple pieces of processing.
You can read more about why you should use share in this situation here: http://adamborek.com/how-to-handle-errors-in-rxswift/
Hope this helps!
Yes, it's a pain. I've thought about the idea of making a new library where the grammar doesn't require the stream to end on an error, but trying to reproduce the entire Rx ecosystem for it seems pointless.
There are reactive libraries that allow you to specify Never as the error type (meaning an error can't be emitted at all,) and in RxCocoa you can use Driver (which can't error) but you are still left with the whole Result dance. "Monads in my Monads!".
To deal with it properly, you need a set of Monad transformers. With these, you can do all the mapping/flatMapping you want and not worry about looking at the errors until the very end.
I have list List<Mono<String>>. Each Mono represents API call where I wait on I/O for result. The problem is that some times some calls return nothing (empty String), and I need repeat them again on that case.
Now it looks like this:
val firstAskForItemsRetrieved = firstAskForItems.map {
it["statistic"] = (it["statistic"] as Mono<Map<Any, Any>>).block()
it
}
I'm waiting for all Monos to finish, then in case of empty body I repeat request
val secondAskForItem = firstAskForItemsRetrieved
.map {
if ((it["statistic"] as Map<Any, Any>).isEmpty()) {
// repeat request
it["statistic"] = getUserItem(userName) // return Mono
} else
it["statistic"] = Mono.just(it["statistic"])
it
}
And then block on each item again
val secondAskForItemsRetrieved = secondAskForItems.map {
it["statistic"] = (it["statistic"] as Mono<Map<Any, Any>>).block()
it
}
I see that looks ugly
Are any other ways to retry call in Mono if it fails, without doing it manually?
Is it block on each item a right way to get them all?
How to make the code better?
Thank you.
There are 2 operators I believe can help your:
For the "wait for all Mono" use case, have a look at the static methods when and zip.
when just cares about completion, so even if the monos are empty it will just signal an onComplete whenever all of the monos have finished. You don't get the data however.
zip cares about the values and expects all Monos to be valued. When all Monos are valued, it combines their values according to the passed Function. Otherwise it just completes empty.
To retry the empty Monos, have a look at repeatWhenEmpty. It resubscribes to an empty Mono, so if that Mono is "cold" it would restart the source (eg. make another HTTP request).
I need to make a series of database queries that each return a stream of results. Once all the information is collected and sent the 'complete' message needs to be send last. In my code 'sendCompleteMessageToClient' gets sent first.
Future.forEach(centerLdapNames.values, (center) {
db
.collection(center)
.find({'date': {'\$gte': json['from'], '\$lt': json['to']}})
.forEach(sendAppointmentToClient);
}).whenComplete(() => sendCompleteMessageToClient("all"));
How do I wait for all 'sendAppointmentToClient' to finish properly?
I guess you just miss the return of the future
Future.forEach(centerLdapNames.values, (center) {
return db // <== always return the returned future from calls to async functions to keep the chain connected
.collection(center)
.find({'date': {'\$gte': json['from'], '\$lt': json['to']}})
.forEach(sendAppointmentToClient);
}).whenComplete(() => sendCompleteMessageToClient("all"));
If you use wait these calls might be executed in parallel instead of one after the other
Future.wait(centerLdapNames.values.map((center) { ...}, eagerError: false)
.whenComplete((...))