Related
I'm making a simple test app using Swift Network framework. One server, one client (both iOS simulators), and successfully established tcp connection between them. I'm trying to send a series of short messages.
Server is sending strings made of natural numbers from 1 to 999. Each number is sent as separate Data, isComplete and contentContext default values are true and .defaultMessage correspondingly.
var count = 0
func send(data: Data) {
self.connection.send(content: data, completion: .contentProcessed( { error in
if let error = error {
self.connectionDidFail(error: error)
return
}
self.count += 1
let newData = "\(self.count)".data(using: .utf8)!
if self.count < 1000 {
self.send(data: newData)
}
print("connection \(self.id) did send, data: \(newData as NSData)")
}))
}
Client is receiving them...
private func setupReceive() {
nwConnection.receive(minimumIncompleteLength: 1, maximumLength: 65536) { (data, contentContext, isComplete, error) in
if let data = data, !data.isEmpty {
print("isComplete: \(isComplete)")
print("isFinal: \(contentContext.isFinal)")
let message = String(data: data, encoding: .utf8)
print("connection did receive, data: \(data as NSData) string: \(message ?? "-" )")
}
if let error = error {
self.connectionDidFail(error: error)
} else {
self.setupReceive()
}
}
}
... but there is something wrong. Some messages look like their bytes are stuck together (for example consecutive messages "2", "3", "4", "5" could be received like a single message "2345")
For all received messages isComplete equals false and contentContext property isFinal equals true, while .defaultMessage.isFinal should be equal to false.
For now, i'm stuck. Am i just using wrong parameters (I've tried various combinations, but none seems working to me)? Is NWConnection hideously changing messages while sending them?
How can one send a series of separate messages?
I am not familiar with this Network framework. But from reading documentation, it seems like you are directly using the transport layer to transmit messages.
Without an application layer protocol, there probably isn't a way for the client to distinguish between different messages. For example, using http as your application protocol has different parameters in the request to identify if its a complete message or not (Content-Length, Content-Encoding, Transfer-Encoding etc...)(Hope an expert can confirm on this)
You may define you own simple protocol so that decoding is possible on client end. For example, you can wrap each message with <Message>your-message</Message> and use it to identify different messages at the client (you will face some drawbacks of something so simple along the way)
There are many things to consider when developing a custom protocol. Better do some reading on this subject if you are serious on it.
Upon further reading, it seems that the following receive is provided:
final func receiveMessage(completion: #escaping (Data?, NWConnection.ContentContext?, Bool, NWError?) -> Void)
... which is able to read complete data. The Discussion section will provide some valuable insight on transport type implications and framing logic required.
I'm working on a function that connects to GetStream in order to get the activities for a chosen feed group. The problem is that GetStream is returning all the activities inside the feed group as opposed to the ones for a specific slug. In MySQL terms it's ignoring the WHERE UUID = uuid :)
Here's the code I'm currently using which returns all activities, I've tried setting the elementFeed to nil before setting it but that doesn't work either.
func getStream(uuid: String, completion: #escaping([StreamActivity], Bool, APPError?) -> Void) {
let group = DispatchGroup()
group.enter()
var activities = [StreamActivity]()
elementFeed = nil
elementFeed = Client.shared.flatFeed(feedSlug: "element", userId: uuid)
elementFeed?.get(typeOf: StreamActivity.self, enrich: false, pagination: .limit(20), ranking: "", includeReactions: [], completion: { results in
if let foundActivities = results.value?.results {
activities.append(contentsOf: foundActivities)
group.leave()
} else {
completion([], false, .GetDiscussions)
group.leave()
}
})
group.notify(queue: .main) {
self.isUserFollowing(uuid: uuid) { (following) in
completion(activities, following, nil)
}
}
}
Here's the code used to create the activity:
func addActivity(type: ActivityType, body: String?, uuid: String, completion: #escaping(Bool, APPError?) -> Void) {
var feed: FeedId?
var originFeed: FeedId?
switch type {
case .comment:
feed = FeedId(feedSlug: "element", userId: uuid)
case .reply:
originFeed = FeedId(feedSlug: "comment", userId: uuid)
default:
break
}
let activity = StreamActivity(actor: StreamUser.current!, verb: type.rawValue, object: "element:\(uuid)", body: body!)
activity.feedIds = [feed!]
elementFeed = Client.shared.flatFeed(feedSlug: "element", userId: uuid)
elementFeed?.add(activity, completion: { (result) in
if result.error == nil {
completion(true, nil)
} else {
completion(false, .PostComment)
}
print("GETSTREAM: Activity added:\(result)")
})
}
Stream Feed API does not allow you to read activities outside a feed. Feeds are materialised so when you perform a call like this
Client.shared.flatFeed(feedSlug: "element", userId: uuid)
You are not performing a query (ie. SELECT * FROM activities WHERE slug = ? AND userId = ?) but you are reading the activities that were added (directly, via to targets or from followed feeds) to the element:$uuid feed.
If a feed contains wrong activities it is because something is going wrong with the logic used to add activities or to create follow relationships.
When you read a feed, Stream adds information about the origin of each activity; such information is stored in a field called "origin". If the origin field is missing / null it means the activity was added to the feed directly; if the activity was added via follow relationship, then the origin field will be set to the feed id from where the activity originated.
For example:
If element:x follows user:z; all activities user Z adds are propagated to element:x and will have origin = "user:z" on that feed.
You can use the origin field to get some more information about why the activity is in a feed; that should be very useful information to find out where the integration problem is.
Using the new Combine framework in iOS 13.
Suppose I have an upstream publisher sending values at a highly irregular rate - sometimes seconds or minutes may go by without any values, and then a stream of values may come through all at once. I'd like to create a custom publisher that subscribes to the upstream values, buffers them and emits them at a regular, known cadence when they come in, but publishes nothing if they've all been exhausted.
For a concrete example:
t = 0 to 5000ms: no upstream values published
t = 5001ms: upstream publishes "a"
t = 5002ms: upstream publishes "b"
t = 5003ms: upstream publishes "c"
t = 5004ms to 10000ms: no upstream values published
t = 10001ms: upstream publishes "d"
My publisher subscribed to the upstream would produce values every 1 second:
t = 0 to 5000ms: no values published
t = 5001ms: publishes "a"
t = 6001ms: publishes "b"
t = 7001ms: publishes "c"
t = 7001ms to 10001ms: no values published
t = 10001ms: publishes "d"
None of the existing publishers or operators in Combine seem to quite do what I want here.
throttle and debounce would simply sample the upstream values at a certain cadence and drop ones that are missing (e.g. would only publish "a" if the cadence was 1000ms)
delay would add the same delay to every value, but not space them out (e.g. if my delay was 1000ms, it would publish "a" at 6001ms, "b" at 6002ms, "c" at 6003ms)
buffer seems promising, but I can't quite figure out how to use it - how to force it to publish a value from the buffer on demand. When I hooked up a sink to buffer it seemed to just instantly publish all the values, not buffering at all.
I thought about using some sort of combining operator like zip or merge or combineLatest and combining it with a Timer publisher, and that's probably the right approach, but I can't figure out exactly how to configure it to give the behavior I want.
Edit
Here's a marble diagram that hopefully illustrates what I'm going for:
Upstream Publisher:
-A-B-C-------------------D-E-F--------|>
My Custom Operator:
-A----B----C-------------D----E----F--|>
Edit 2: Unit Test
Here's a unit test that should pass if modulatedPublisher (my desired buffered publisher) works as desired. It's not perfect, but it stores events (including the time received) as they're received and then compares the time intervals between events, ensuring they are no smaller than the desired interval.
func testCustomPublisher() {
let expectation = XCTestExpectation(description: "async")
var events = [Event]()
let passthroughSubject = PassthroughSubject<Int, Never>()
let cancellable = passthroughSubject
.modulatedPublisher(interval: 1.0)
.sink { value in
events.append(Event(value: value, date: Date()))
print("value received: \(value) at \(self.dateFormatter.string(from:Date()))")
}
// WHEN I send 3 events, wait 6 seconds, and send 3 more events
passthroughSubject.send(1)
passthroughSubject.send(2)
passthroughSubject.send(3)
DispatchQueue.main.asyncAfter(deadline: .now() + .milliseconds(6000)) {
passthroughSubject.send(4)
passthroughSubject.send(5)
passthroughSubject.send(6)
DispatchQueue.main.asyncAfter(deadline: .now() + .milliseconds(4000)) {
// THEN I expect the stored events to be no closer together in time than the interval of 1.0s
for i in 1 ..< events.count {
let interval = events[i].date.timeIntervalSince(events[i-1].date)
print("Interval: \(interval)")
// There's some small error in the interval but it should be about 1 second since I'm using a 1s modulated publisher.
XCTAssertTrue(interval > 0.99)
}
expectation.fulfill()
}
}
wait(for: [expectation], timeout: 15)
}
The closest I've gotten is using zip, like so:
public extension Publisher where Self.Failure == Never {
func modulatedPublisher(interval: TimeInterval) -> AnyPublisher<Output, Never> {
let timerBuffer = Timer
.publish(every: interval, on: .main, in: .common)
.autoconnect()
return timerBuffer
.zip(self, { $1 }) // should emit one input element ($1) every timer tick
.eraseToAnyPublisher()
}
}
This properly attunes the first three events (1, 2, and 3), but not the second three (4, 5, and 6). The output:
value received: 1 at 3:54:07.0007
value received: 2 at 3:54:08.0008
value received: 3 at 3:54:09.0009
value received: 4 at 3:54:12.0012
value received: 5 at 3:54:12.0012
value received: 6 at 3:54:12.0012
I believe this is happening because zip has some internal buffering capacity. The first three upstream events are buffered and emitted on the Timer's cadence, but during the 6 second wait, the Timer's events are buffered - and when the second set ups upstream events are fired, there are already Timer events waiting in the queue, so they're paired up and fired off immediately.
This is an interesting problem. I played with various combinations of Timer.publish, buffer, zip, and throttle, but I couldn't get any combination to work quite the way you want. So let's write a custom subscriber.
What we'd really like is an API where, when we get an input from upstream, we also get the ability to control when the upstream delivers the next input. Something like this:
extension Publisher {
/// Subscribe to me with a stepping function.
/// - parameter stepper: A function I'll call with each of my inputs, and with my completion.
/// Each time I call this function with an input, I also give it a promise function.
/// I won't deliver the next input until the promise is called with a `.more` argument.
/// - returns: An object you can use to cancel the subscription asynchronously.
func step(with stepper: #escaping (StepEvent<Output, Failure>) -> ()) -> AnyCancellable {
???
}
}
enum StepEvent<Input, Failure: Error> {
/// Handle the Input. Call `StepPromise` when you're ready for the next Input,
/// or to cancel the subscription.
case input(Input, StepPromise)
/// Upstream completed the subscription.
case completion(Subscribers.Completion<Failure>)
}
/// The type of callback given to the stepper function to allow it to continue
/// or cancel the stream.
typealias StepPromise = (StepPromiseRequest) -> ()
enum StepPromiseRequest {
// Pass this to the promise to request the next item from upstream.
case more
// Pass this to the promise to cancel the subscription.
case cancel
}
With this step API, we can write a pace operator that does what you want:
extension Publisher {
func pace<Context: Scheduler, MySubject: Subject>(
_ pace: Context.SchedulerTimeType.Stride, scheduler: Context, subject: MySubject)
-> AnyCancellable
where MySubject.Output == Output, MySubject.Failure == Failure
{
return step {
switch $0 {
case .input(let input, let promise):
// Send the input from upstream now.
subject.send(input)
// Wait for the pace interval to elapse before requesting the
// next input from upstream.
scheduler.schedule(after: scheduler.now.advanced(by: pace)) {
promise(.more)
}
case .completion(let completion):
subject.send(completion: completion)
}
}
}
}
This pace operator takes pace (the required interval between outputs), a scheduler on which to schedule events, and a subject on which to republish the inputs from upstream. It handles each input by sending it through subject, and then using the scheduler to wait for the pace interval before asking for the next input from upstream.
Now we just have to implement the step operator. Combine doesn't give us too much help here. It does have a feature called “backpressure”, which means a publisher cannot send an input downstream until the downstream has asked for it by sending a Subscribers.Demand upstream. Usually you see downstreams send an .unlimited demand upstream, but we're not going to. Instead, we're going to take advantage of backpressure. We won't send any demand upstream until the stepper completes a promise, and then we'll only send a demand of .max(1), so we make the upstream operate in lock-step with the stepper. (We also have to send an initial demand of .max(1) to start the whole process.)
Okay, so need to implement a type that takes a stepper function and conforms to Subscriber. It's a good idea to review the Reactive Streams JVM Specification, because Combine is based on that specification.
What makes the implementation difficult is that several things can call into our subscriber asynchronously:
The upstream can call into the subscriber from any thread (but is required to serialize its calls).
After we've given promise functions to the stepper, the stepper can call those promises on any thread.
We want the subscription to be cancellable, and that cancellation can happen on any thread.
All this asynchronicity means we have to protect our internal state with a lock.
We have to be careful not to call out while holding that lock, to avoid deadlock.
We'll also protect the subscriber from shenanigans involving calling a promise repeatedly, or calling outdated promises, by giving each promise a unique id.
Se here's our basic subscriber definition:
import Combine
import Foundation
public class SteppingSubscriber<Input, Failure: Error> {
public init(stepper: #escaping Stepper) {
l_state = .subscribing(stepper)
}
public typealias Stepper = (Event) -> ()
public enum Event {
case input(Input, Promise)
case completion(Completion)
}
public typealias Promise = (Request) -> ()
public enum Request {
case more
case cancel
}
public typealias Completion = Subscribers.Completion<Failure>
private let lock = NSLock()
// The l_ prefix means it must only be accessed while holding the lock.
private var l_state: State
private var l_nextPromiseId: PromiseId = 1
private typealias PromiseId = Int
private var noPromiseId: PromiseId { 0 }
}
Notice that I moved the auxiliary types from earlier (StepEvent, StepPromise, and StepPromiseRequest) into SteppingSubscriber and shortened their names.
Now let's consider l_state's mysterious type, State. What are all the different states our subscriber could be in?
We could be waiting to receive the Subscription object from upstream.
We could have received the Subscription from upstream and be waiting for a signal (an input or completion from upstream, or the completion of a promise from the stepper).
We could be calling out to the stepper, which we want to be careful in case it completes a promise while we're calling out to it.
We could have been cancelled or have received completion from upstream.
So here is our definition of State:
extension SteppingSubscriber {
private enum State {
// Completed or cancelled.
case dead
// Waiting for Subscription from upstream.
case subscribing(Stepper)
// Waiting for a signal from upstream or for the latest promise to be completed.
case subscribed(Subscribed)
// Calling out to the stopper.
case stepping(Stepping)
var subscription: Subscription? {
switch self {
case .dead: return nil
case .subscribing(_): return nil
case .subscribed(let subscribed): return subscribed.subscription
case .stepping(let stepping): return stepping.subscribed.subscription
}
}
struct Subscribed {
var stepper: Stepper
var subscription: Subscription
var validPromiseId: PromiseId
}
struct Stepping {
var subscribed: Subscribed
// If the stepper completes the current promise synchronously with .more,
// I set this to true.
var shouldRequestMore: Bool
}
}
}
Since we're using NSLock (for simplicity), let's define an extension to ensure we always match locking with unlocking:
fileprivate extension NSLock {
#inline(__always)
func sync<Answer>(_ body: () -> Answer) -> Answer {
lock()
defer { unlock() }
return body()
}
}
Now we're ready to handle some events. The easiest event to handle is asynchronous cancellation, which is the Cancellable protocol's only requirement. If we're in any state except .dead, we want to become .dead and, if there's an upstream subscription, cancel it.
extension SteppingSubscriber: Cancellable {
public func cancel() {
let sub: Subscription? = lock.sync {
defer { l_state = .dead }
return l_state.subscription
}
sub?.cancel()
}
}
Notice here that I don't want to call out to the upstream subscription's cancel function while lock is locked, because lock isn't a recursive lock and I don't want to risk deadlock. All use of lock.sync follows the pattern of deferring any call-outs until after the lock is unlocked.
Now let's implement the Subscriber protocol requirements. First, let's handle receiving the Subscription from upstream. The only time this should happen is when we're in the .subscribing state, but .dead is also possible in which case we want to just cancel the upstream subscription.
extension SteppingSubscriber: Subscriber {
public func receive(subscription: Subscription) {
let action: () -> () = lock.sync {
guard case .subscribing(let stepper) = l_state else {
return { subscription.cancel() }
}
l_state = .subscribed(.init(stepper: stepper, subscription: subscription, validPromiseId: noPromiseId))
return { subscription.request(.max(1)) }
}
action()
}
Notice that in this use of lock.sync (and in all later uses), I return an “action” closure so I can perform arbitrary call-outs after the lock has been unlocked.
The next Subscriber protocol requirement we'll tackle is receiving a completion:
public func receive(completion: Subscribers.Completion<Failure>) {
let action: (() -> ())? = lock.sync {
// The only state in which I have to handle this call is .subscribed:
// - If I'm .dead, either upstream already completed (and shouldn't call this again),
// or I've been cancelled.
// - If I'm .subscribing, upstream must send me a Subscription before sending me a completion.
// - If I'm .stepping, upstream is currently signalling me and isn't allowed to signal
// me again concurrently.
guard case .subscribed(let subscribed) = l_state else {
return nil
}
l_state = .dead
return { [stepper = subscribed.stepper] in
stepper(.completion(completion))
}
}
action?()
}
The most complex Subscriber protocol requirement for us is receiving an Input:
We have to create a promise.
We have to pass the promise to the stepper.
The stepper could complete the promise before returning.
After the stepper returns, we have to check whether it completed the promise with .more and, if so, return the appropriate demand upstream.
Since we have to call out to the stepper in the middle of this work, we have some ugly nesting of lock.sync calls.
public func receive(_ input: Input) -> Subscribers.Demand {
let action: (() -> Subscribers.Demand)? = lock.sync {
// The only state in which I have to handle this call is .subscribed:
// - If I'm .dead, either upstream completed and shouldn't call this,
// or I've been cancelled.
// - If I'm .subscribing, upstream must send me a Subscription before sending me Input.
// - If I'm .stepping, upstream is currently signalling me and isn't allowed to
// signal me again concurrently.
guard case .subscribed(var subscribed) = l_state else {
return nil
}
let promiseId = l_nextPromiseId
l_nextPromiseId += 1
let promise: Promise = { request in
self.completePromise(id: promiseId, request: request)
}
subscribed.validPromiseId = promiseId
l_state = .stepping(.init(subscribed: subscribed, shouldRequestMore: false))
return { [stepper = subscribed.stepper] in
stepper(.input(input, promise))
let demand: Subscribers.Demand = self.lock.sync {
// The only possible states now are .stepping and .dead.
guard case .stepping(let stepping) = self.l_state else {
return .none
}
self.l_state = .subscribed(stepping.subscribed)
return stepping.shouldRequestMore ? .max(1) : .none
}
return demand
}
}
return action?() ?? .none
}
} // end of extension SteppingSubscriber: Publisher
The last thing our subscriber needs to handle is the completion of a promise. This is complicated for several reasons:
We want to protect against a promise being completed multiple times.
We want to protect against an older promise being completed.
We can be in any state when a promise is completed.
Thus:
extension SteppingSubscriber {
private func completePromise(id: PromiseId, request: Request) {
let action: (() -> ())? = lock.sync {
switch l_state {
case .dead, .subscribing(_): return nil
case .subscribed(var subscribed) where subscribed.validPromiseId == id && request == .more:
subscribed.validPromiseId = noPromiseId
l_state = .subscribed(subscribed)
return { [sub = subscribed.subscription] in
sub.request(.max(1))
}
case .subscribed(let subscribed) where subscribed.validPromiseId == id && request == .cancel:
l_state = .dead
return { [sub = subscribed.subscription] in
sub.cancel()
}
case .subscribed(_):
// Multiple completion or stale promise.
return nil
case .stepping(var stepping) where stepping.subscribed.validPromiseId == id && request == .more:
stepping.subscribed.validPromiseId = noPromiseId
stepping.shouldRequestMore = true
l_state = .stepping(stepping)
return nil
case .stepping(let stepping) where stepping.subscribed.validPromiseId == id && request == .cancel:
l_state = .dead
return { [sub = stepping.subscribed.subscription] in
sub.cancel()
}
case .stepping(_):
// Multiple completion or stale promise.
return nil
}
}
action?()
}
}
Whew!
With all that done, we can write the real step operator:
extension Publisher {
func step(with stepper: #escaping (SteppingSubscriber<Output, Failure>.Event) -> ()) -> AnyCancellable {
let subscriber = SteppingSubscriber<Output, Failure>(stepper: stepper)
self.subscribe(subscriber)
return .init(subscriber)
}
}
And then we can try out that pace operator from above. Since we don't do any buffering in SteppingSubscriber, and the upstream in general isn't buffered, we'll stick a buffer in between the upstream and our pace operator.
var cans: [AnyCancellable] = []
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
let erratic = Just("A").delay(for: 0.0, tolerance: 0.001, scheduler: DispatchQueue.main).eraseToAnyPublisher()
.merge(with: Just("B").delay(for: 0.3, tolerance: 0.001, scheduler: DispatchQueue.main).eraseToAnyPublisher())
.merge(with: Just("C").delay(for: 0.6, tolerance: 0.001, scheduler: DispatchQueue.main).eraseToAnyPublisher())
.merge(with: Just("D").delay(for: 5.0, tolerance: 0.001, scheduler: DispatchQueue.main).eraseToAnyPublisher())
.merge(with: Just("E").delay(for: 5.3, tolerance: 0.001, scheduler: DispatchQueue.main).eraseToAnyPublisher())
.merge(with: Just("F").delay(for: 5.6, tolerance: 0.001, scheduler: DispatchQueue.main).eraseToAnyPublisher())
.handleEvents(
receiveOutput: { print("erratic: \(Double(DispatchTime.now().rawValue) / 1_000_000_000) \($0)") },
receiveCompletion: { print("erratic: \(Double(DispatchTime.now().rawValue) / 1_000_000_000) \($0)") }
)
.makeConnectable()
let subject = PassthroughSubject<String, Never>()
cans += [erratic
.buffer(size: 1000, prefetch: .byRequest, whenFull: .dropOldest)
.pace(.seconds(1), scheduler: DispatchQueue.main, subject: subject)]
cans += [subject.sink(
receiveCompletion: { print("paced: \(Double(DispatchTime.now().rawValue) / 1_000_000_000) \($0)") },
receiveValue: { print("paced: \(Double(DispatchTime.now().rawValue) / 1_000_000_000) \($0)") }
)]
let c = erratic.connect()
cans += [AnyCancellable { c.cancel() }]
return true
}
And here, at long last, is the output:
erratic: 223394.17115897 A
paced: 223394.171495405 A
erratic: 223394.408086369 B
erratic: 223394.739186984 C
paced: 223395.171615624 B
paced: 223396.27056174 C
erratic: 223399.536717127 D
paced: 223399.536782847 D
erratic: 223399.536834495 E
erratic: 223400.236808469 F
erratic: 223400.236886323 finished
paced: 223400.620542561 E
paced: 223401.703613078 F
paced: 223402.703828512 finished
Timestamps are in units of seconds.
The erratic publisher's timings are, indeed, erratic and sometimes close in time.
The paced timings are always at least one second apart even when the erratic events occur less than one second apart.
When an erratic event occurs more than one second after the prior event, the paced event is sent immediately following the erratic event without further delay.
The paced completion occurs one second after the last paced event, even though the erratic completion occurs immediately after the last erratic event. The buffer doesn't send the completion until it receives another demand after it sends the last event, and that demand is delayed by the pacing timer.
I've put the the entire implementation of the step operator in this gist for easy copy/paste.
EDIT
There's an even simpler approach to the original one outlined below, which doesn't require a pacer, but instead uses back-pressure created by flatMap(maxPublishers: .max(1)).
flatMap sends a demand of 1, until its returned publisher, which we could delay, completes. We'd need a Buffer publisher upstream to buffer the values.
// for demo purposes, this subject sends a Date:
let subject = PassthroughSubject<Date, Never>()
let interval = 1.0
let pub = subject
.buffer(size: .max, prefetch: .byRequest, whenFull: .dropNewest)
.flatMap(maxPublishers: .max(1)) {
Just($0)
.delay(for: .seconds(interval), scheduler: DispatchQueue.main)
}
ORIGINAL
I know this is an old question, but I think there's a much simpler way to implement this, so I thought I'd share.
The idea is similar to a .zip with a Timer, except instead of a Timer, you would .zip with a time-delayed "tick" from a previously sent value, which can be achieved with a CurrentValueSubject. CurrentValueSubject is needed instead of a PassthroughSubject in order to seed the first ever "tick".
// for demo purposes, this subject sends a Date:
let subject = PassthroughSubject<Date, Never>()
let pacer = CurrentValueSubject<Void, Never>(())
let interval = 1.0
let pub = subject.zip(pacer)
.flatMap { v in
Just(v.0) // extract the original value
.delay(for: .seconds(interval), scheduler: DispatchQueue.main)
.handleEvents(receiveOutput: { _ in
pacer.send() // send the pacer "tick" after the interval
})
}
What happens is that the .zip gates on the pacer, which only arrives after a delay from a previously sent value.
If the next value comes earlier than the allowed interval, it waits for the pacer.
If, however, the next value comes later, then the pacer already has a new value to provide instantly, so there would be no delay.
If you used it like in your test case:
let c = pub.sink { print("\($0): \(Date())") }
subject.send(Date())
subject.send(Date())
subject.send(Date())
DispatchQueue.main.asyncAfter(deadline: .now() + 1.0) {
subject.send(Date())
subject.send(Date())
}
DispatchQueue.main.asyncAfter(deadline: .now() + 10.0) {
subject.send(Date())
subject.send(Date())
}
the result would be something like this:
2020-06-23 19:15:21 +0000: 2020-06-23 19:15:21 +0000
2020-06-23 19:15:21 +0000: 2020-06-23 19:15:22 +0000
2020-06-23 19:15:21 +0000: 2020-06-23 19:15:23 +0000
2020-06-23 19:15:22 +0000: 2020-06-23 19:15:24 +0000
2020-06-23 19:15:22 +0000: 2020-06-23 19:15:25 +0000
2020-06-23 19:15:32 +0000: 2020-06-23 19:15:32 +0000
2020-06-23 19:15:32 +0000: 2020-06-23 19:15:33 +0000
Could Publishers.CollectByTime be useful here somewhere?
Publishers.CollectByTime(upstream: upstreamPublisher.share(), strategy: Publishers.TimeGroupingStrategy.byTime(RunLoop.main, .seconds(1)), options: nil)
Just wanted to mention that I adapted Rob's answer from earlier and converted it to a custom Publisher, in order to allow for a single unbroken pipeline (see comments below his solution). My adaptation is below, but all the credit still goes to him. It also still makes use of Rob's step operator and SteppingSubscriber, as this custom Publisher uses those internally.
Edit: updated with buffer as part of the modulated operator, otherwise that would be required to be attached to buffer the upstream events.
public extension Publisher {
func modulated<Context: Scheduler>(_ pace: Context.SchedulerTimeType.Stride, scheduler: Context) -> AnyPublisher<Output, Failure> {
let upstream = buffer(size: 1000, prefetch: .byRequest, whenFull: .dropNewest).eraseToAnyPublisher()
return PacePublisher<Context, AnyPublisher>(pace: pace, scheduler: scheduler, source: upstream).eraseToAnyPublisher()
}
}
final class PacePublisher<Context: Scheduler, Source: Publisher>: Publisher {
typealias Output = Source.Output
typealias Failure = Source.Failure
let subject: PassthroughSubject<Output, Failure>
let scheduler: Context
let pace: Context.SchedulerTimeType.Stride
lazy var internalSubscriber: SteppingSubscriber<Output, Failure> = SteppingSubscriber<Output, Failure>(stepper: stepper)
lazy var stepper: ((SteppingSubscriber<Output, Failure>.Event) -> ()) = {
switch $0 {
case .input(let input, let promise):
// Send the input from upstream now.
self.subject.send(input)
// Wait for the pace interval to elapse before requesting the
// next input from upstream.
self.scheduler.schedule(after: self.scheduler.now.advanced(by: self.pace)) {
promise(.more)
}
case .completion(let completion):
self.subject.send(completion: completion)
}
}
init(pace: Context.SchedulerTimeType.Stride, scheduler: Context, source: Source) {
self.scheduler = scheduler
self.pace = pace
self.subject = PassthroughSubject<Source.Output, Source.Failure>()
source.subscribe(internalSubscriber)
}
public func receive<S>(subscriber: S) where S : Subscriber, Failure == S.Failure, Output == S.Input {
subject.subscribe(subscriber)
subject.send(subscription: PaceSubscription(subscriber: subscriber))
}
}
public class PaceSubscription<S: Subscriber>: Subscription {
private var subscriber: S?
init(subscriber: S) {
self.subscriber = subscriber
}
public func request(_ demand: Subscribers.Demand) {
}
public func cancel() {
subscriber = nil
}
}
I've been trying since the morning but I didnt achieve what I wanted.
I tried DispatchQueue.main.async and completion block but my "Submit" button in the UI still freezes waiting for the data to be returned from the server. This is my code:
func createData(request:Crudpb_CreateRequest, with completion: #escaping (String) -> Void) throws {
DispatchQueue.main.async {
self.response = try! self.client.create(request) // <---- How to handle error for this server call when the server is not available or is down?
completion(self.response.result)
}
}
I just noticed Im calling the 1st method from the following which is a Synchronous Unary which might be the reason behind the problem. But again I dont know how to call the second function in the fallowing:
/// Synchronous. Unary.
internal func create(_ request: Crudpb_CreateRequest, metadata customMetadata: Metadata) throws -> Crudpb_CreateResponse {
return try Crudpb_CrudServiceCreateCallBase(channel)
.run(request: request, metadata: customMetadata)
}
/// Asynchronous. Unary.
#discardableResult
internal func create(_ request: Crudpb_CreateRequest, metadata customMetadata: Metadata, completion: #escaping (Crudpb_CreateResponse?, CallResult) -> Void) throws -> Crudpb_CrudServiceCreateCall {
return try Crudpb_CrudServiceCreateCallBase(channel)
.start(request: request, metadata: customMetadata, completion: completion)
}
Server Side Code:
func (*server) Create(ctx context.Context, req *crudpb.CreateRequest) (*crudpb.CreateResponse, error) {
var result string
firstName := req.GetAccount().GetFirstName()
lastName := req.GetAccount().GetLastName()
// id := gocql.TimeUUID()
fmt.Println("Triggered CREATE function on Go Server " + firstName + " " + lastName + "! Yayy!")
result = fmt.Sprintf("id for %s %s : %s", firstName, lastName, strconv.Itoa(rand.Intn(100)))
return &crudpb.CreateResponse{
Result: result,
}, nil
I need this app / submit button not to freeze while it fetches result from server.
Please help.
You are still performing work on the main thread.. async only makes the createData() method to return before the task is completed.. but the task will still be processed at some time in the main thread and during this time your application will become unresponsive.. try using a global queue instead.. to keep your main thread clean..
Dont forget to perform all your UI work on the main thread after getting your response.
Use the asynchronous function instead and call the completion block inside create function's completion.
func createData(request:Crudpb_CreateRequest, with completion: #escaping (String) -> Void) throws {
try! self.client.create(request) { (response: Crudpb_CreateResponse?, result: CallResult) in
DispatchQueue.main.async {
// This is assuming your completion involves UI operations. Otherwise there is no need for this async call.
let stringOutput = String(data: result.resultData!, encoding: String.Encoding.utf8))
completion(stringOutput)
}
}
}
Remove DispatchQueue.main.async block from the createData method
func createData(request:Crudpb_CreateRequest, with completion: #escaping (String) -> Void) throws {
self.response = try! self.client.create(request)
completion(self.response.result)
}
Use main queue only where you update the UI from the api response
myobj.createData(request: request, with: { string in
print(string)//background thread
DispatchQueue.main.async {
self.label.text = sting//main thread
}
})
The UI freeze because you are doing too much work on the main thread. You should find out what function blocks the main thread.
The instruments time profiler is an easy way to see which function is spending too much time.
I have run into an issue where I have multiple asynchronous requests occuring which grab images and information from the Facebook API and my Firebase database. I want to perform all my asynchronous requests, then store all that data that I grabbed from the Facebook API/Firebase database into one entire object which I can quickly load. I have set up completion handlers for every asynchronous request which I thought forces the program to "wait" until the request is complete and then have the program continue, but that doesn't seem to work for me. Below is my attempt:
func setupEvents(completion: (result: Bool, Event: Event) -> Void){
// Get a reference to Events
eventsReference = Firebase(url:"<DB Name>")
eventAttendeesRef = Firebase(url:"<DB Name>")
//Read the data at our posts reference
println("Event References: \(eventsReference)")
eventsReference.observeEventType(FEventType.ChildAdded, withBlock: { (snapshot) -> Void in
let eventName = snapshot.value["eventName"] as? String
let eventLocation = snapshot.value["eventLocation"] as? String
let eventCreator = snapshot.value["eventCreator"] as? String
var attendees: NSMutableDictionary = [:]
var attendeesImages = [UIImage]()
let attendee: NSMutableDictionary = [:]
let group = dispatch_group_create()
//Get attendees first
dispatch_group_enter(group)
self.getAttendees(snapshot.key as String, completion:{ (result, name, objectID) -> Void in
if(result == true){
println("Finished grabbing \(name!) \(objectID!)")
attendees.addEntriesFromDictionary(attendee as [NSObject : AnyObject])
}
else {
println("False")
}
dispatch_group_leave(group)
})
//Get attendees photos
dispatch_group_enter(group)
self.getAttendeesPictures(attendee, completion: { (result, image) -> Void in
if result == true {
println("Finished getting attendee photos. Now to store into Event object.")
attendeesImages.append(image!)
}
else{
println("false")
}
dispatch_group_leave(group)
})
dispatch_group_notify(group, dispatch_get_main_queue()) {
println("both requests done")
//Maintain array snapshot keys
self.eventIDs.append(snapshot.key)
if snapshot != nil {
let event = Event(eventName: eventName, eventLocation:eventLocation, eventPhoto:eventPhoto, fromDate:fromDate, fromTime:fromTime, toDate:toDate, toTime:toTime, attendees: attendees, attendeesImages:attendeesImages, attendeesImagesTest: attendeesImagesTest, privacy:privacy, eventCreator: eventCreator, eventCreatorID: eventCreatorID)
println("Event: \(event)")
completion(result: true, Event: event)
}
}
}) { (error) -> Void in
println(error.description)
}
}
I know I have my completion handlers set correctly as I have tested in my program. However, what I want is that only after both the getAttendees and getAttendeesPictures function completes, I then want to store all the information I grabbed the snapshot, getAttendees, and getAttendeesPictures function and store them into an event object. Any ideas on how to accomplish this? I've tried to look into dispatch_groups to help me handle this via this link: Checking for multiple asynchronous responses from Alamofire and Swift but my program seems to only execute the getAttendees function but not the getAttendeesPictures function. Below are also the getAttendees and getAttendeesPictures functions:
func getAttendees(child: String, completion: (result: Bool, name: String?, objectID: String?) -> Void){
//Get event attendees of particular event
var attendeesReference = self.eventAttendeesRef.childByAppendingPath(child)
println("Loading event attendees")
//Get all event attendees
attendeesReference.observeEventType(FEventType.ChildAdded, withBlock: { (snapshot) -> Void in
let name = snapshot.value.objectForKey("name") as? String
let objectID = snapshot.value.objectForKey("objectID") as? String
println("Name: \(name) Object ID: \(objectID)")
completion(result: true, name: name, objectID: objectID)
}) { (error) -> Void in
println(error.description)
}
func getAttendeesPictures(attendees: NSMutableDictionary, completion: (result: Bool, image: UIImage?)-> Void){
println("Attendees Count: \(attendees.count)")
for (key, value) in attendees{
let url = NSURL(string: "https://graph.facebook.com/\(key)/picture?type=large")
println("URL: \(url)")
let urlRequest = NSURLRequest(URL: url!)
//Asynchronous request to display image
NSURLConnection.sendAsynchronousRequest(urlRequest, queue: NSOperationQueue.mainQueue()) { (response:NSURLResponse!, data:NSData!, error:NSError!) -> Void in
if error != nil{
println("Error: \(error)")
}
// Display the image
let image = UIImage(data: data)
if(image != nil){
completion(result: true, image: image)
}
}
}
}
For users seeking answer to question in title then use of dispatch_group and GCD outlined here: i.e embedding one group inside the notification method of another dispatch_group is valid. Another way to go at a higher level would be NSOperations and dependencies which would also give further control such as canceling operations.
Outline:
func doStuffonObjectsProcessAndComplete(arrayOfObjectsToProcess: Array) -> Void){
let firstGroup = dispatch_group_create()
for object in arrayOfObjectsToProcess {
dispatch_group_enter(firstGroup)
doStuffToObject(object, completion:{ (success) in
if(success){
// doing stuff success
}
else {
// doing stuff fail
}
// regardless, we leave the group letting GCD know we finished this bit of work
dispatch_group_leave(firstGroup)
})
}
// called once all code blocks entered into group have left
dispatch_group_notify(firstGroup, dispatch_get_main_queue()) {
let processGroup = dispatch_group_create()
for object in arrayOfObjectsToProcess {
dispatch_group_enter(processGroup)
processObject(object, completion:{ (success) in
if(success){
// processing stuff success
}
else {
// processing stuff fail
}
// regardless, we leave the group letting GCD know we finished this bit of work
dispatch_group_leave(processGroup)
})
}
dispatch_group_notify(processGroup, dispatch_get_main_queue()) {
print("All Done and Processed, so load data now")
}
}
}
The remainder of this answer is specific to this codebase.
There seem to be a few problems here:
The getAttendees function takes an event child and returns an objectID and Name which are both Strings? Shouldn't this method return an array of attendees? If not, then what is the objectID that is returned?
Once an array of attendees is returned, then you can process them in a group to get the pictures.
The getAttendeesPictures eventually returns UIImages from Facebook. It's probably best to cache these out to the disk and pass path ref - keeping all these fetched images around is bad for memory, and depending on size and number, may quickly lead to problems.
Some examples:
func getAttendees(child: String, completion: (result: Bool, attendees: Array?) -> Void){
let newArrayOfAttendees = []()
// Get event attendees of particular event
// process attendees and package into an Array (or Dictionary)
// completion
completion(true, attendees: newArrayOfAttendees)
}
func getAttendeesPictures(attendees: Array, completion: (result: Bool, attendees: Array)-> Void){
println("Attendees Count: \(attendees.count)")
let picturesGroup = dispatch_group_create()
for attendee in attendees{
// for each attendee enter group
dispatch_group_enter(picturesGroup)
let key = attendee.objectID
let url = NSURL(string: "https://graph.facebook.com/\(key)/picture?type=large")
let urlRequest = NSURLRequest(URL: url!)
//Asynchronous request to display image
NSURLConnection.sendAsynchronousRequest(urlRequest, queue: NSOperationQueue.mainQueue()) { (response:NSURLResponse!, data:NSData!, error:NSError!) -> Void in
if error != nil{
println("Error: \(error)")
}
// Display the image
let image = UIImage(data: data)
if(image != nil){
attendee.image = image
}
dispatch_group_leave(picturesGroup)
}
}
dispatch_group_notify(picturesGroup, dispatch_get_main_queue()) {
completion(true, attendees: attendees)
}
}
func setupEvents(completion: (result: Bool, Event: Event) -> Void){
// get event info and then for each event...
getAttendees(child:snapshot.key, completion: { (result, attendeesReturned) in
if result {
self.getAttendeesPictures(attendees: attendeesReturned, completion: { (result, attendees) in
// do something with completed array and attendees
}
}
else {
}
})
}
The above code is just an outline, but hopefully points you in the right direction.
The two requests are executing at the same time, so there is no attendees to get pictures from when the second request executes, if the getAttendees completion closure is going to be called multiple times then you can do something like this:
let group = dispatch_group_create()
for key in keys {
dispatch_group_enter(group)
self.getAttendee(key as String, completion:{ (result, attendee) in
if(result == true){
attendees.addEntriesFromDictionary(attendee)
self.getAttendeesPictures(attendee, completion: { (result, image) in
if result == true {
attendeesImages.append(image!)
}
dispatch_group_leave(group)
})
} else {
dispatch_group_leave(group)
}
})
}
dispatch_group_notify(group, dispatch_get_main_queue()) {}
If the result of the first request is the complete set of attendees you don't even need to use GCD, just call getAttendeesPictures inside the completion closure.
This code doesn't exactly uses the same variables and methods of the original code, it only gives the idea.
Hope it helps!
While there is definitely solution with using GCD and stuff around it, synchronization in general is pain and the more your code gets complicated, the more problems it will start showing - but I think there is one-for-all solution to that: Bolts framework from Facebook (both for android na iOS)
Bolts Framework usage
So what is so magical about it? Well, it lets you create "Tasks", and then chain them. The method in particular that you are interested in is taskForCompletionOfAllTasks: , which is made for parallel processing, just what you need. I wrote a little example for you which you can adjust to your needs:
func fetchAllInformation() -> BFTask {
// First, create all tasks (if you need more, than just create more, it is as easy as that
var task1 = BFTaskCompletionSource()
var task2 = BFTaskCompletionSource()
var tasks = [task1, task2]
// What you do, is you set result / error to tasks and the propagate in the chain upwards (it is either result, or error)
// You run task 1 in background
API.instance.fetchFirstDetailsInBackgroundWithBlock {
(object: AnyObject!, error: NSError!) -> Void in
// On error or on success, you assign result to task (whatever you want)
if error == nil {
task1.setResult(object)
} else {
task1.setError(error)
}
}
// You run task 2 in background
API.instance.fetchSecondDetailsInBackgroundWithBlock {
(object: AnyObject!, error: NSError!) -> Void in
// On error or on success, you assign result to task (whatever you want)
if error == nil {
task2.setResult(object)
} else {
task2.setError(error)
}
}
// Now you return new task, which will continue ONLY if all the tasks ended
return BFTask(forCompletionOfAllTasks: tasks)
}
Once you have main method done, you can use bolts chaining magic:
func processFullObject() {
// Once you have main method done, you can use bolts chaining magic
self.fetchAllInformation().continueWithBlock { (task : BFTask!) -> AnyObject! in
// All the information fetched, do something with result and probably with information along the way
self.updateObject()
}
}
The Bolts framework documentation / README covers basically everything there is to know about it and it is quite extensive, so I would suggest you to go through it - it is very easy to use once you get the basics. I personally use it for exactly this, and it is a blast. This answer will hopefully provide you with different solution and approach, possibly a cleaner one.
There is something wrong with this conceptually. It sounds like you want to wait until both of these functions complete before doing something else, but what you haven't explained is that getAttendeesPictures depends on the outcome of getAttendees. That means what you really want to do it execute one asynchronous block, then execute a second asynchronous block with the output of the first, and then execute your final completion block when both are finished.
GCD is not particularly suited for this; you're better of using NSOperationQueue with NSBlockOperations. There are two distinct advantages to this over GCD:
NSOperation uses familiar object-oriented syntax compared to GCD's c-type functions, so it's pretty easy to write and understand.
Operations in the queue can have explicit dependencies on one another, so you can make it clear that e.g. operation B will only be executed after operation A is complete.
There is a great writeup of this by NSHipster which I'd recommend you go read. It's talked about mostly in the abstract, but what you want to do is use NSBlockOperation to create two block operations, one for executing getAttendees and one for executing getAttendeesPictures, and then make it explicit that the second block depends on the first before adding them both to a queue. They will then both execute and you can use a completion block on the second operation to do something once both have completed.
Dave Roberts is right in his response though: an immediate problem with the code is that you don't use the output of the getAttendees function to actually create any attendees. Perhaps this part of the code is missing, but from what I can see the name and objectID are just printed out. If you want to pass something useful into the getAttendeesPictures function you will need to fix this part first.
This is off the top of my head. The idea is to read and handle new asyc data only when all of the nested blocks complete.
We leverage a while loop to handle waiting for a signal to read the next set of data.
The outside while loop continues as long as done equals false. And nothing is really going on, other than consuming cpu cycles while it waits. The if inside the loop will only be trigged (set to true) when all of the attendees have been read.
Meanwhile inside the loop we work through nested blocks, reading in the attendee and then when that completes, read their picture, and when that completes read the firebase data. Finally once we have all data from the prior blocks we stuff the data into an object which is then added to the dictionary. At that time it is determined if we are finished reading attendees and if so, bail completely. If not, we read the next attendee.
(this is conceptual)
done = false
readyToReadNextAttendee = true
while ( done == false )
{
if (readyToReadNextAttendee == true ) {
readyToReadNextAttendee = false
readAttendee
readPicture
readFirebase {
putDataIntoObject
addObjectToDictionary
if finishedReadingAttendees {
done = true
} else {
readyToReadNextAttendee = true
}
}
}
}
If you have the option of reading in all of the attendees first, you could iterate over and array as well, not reading the next index until readyToReadNextAttendee = true
One Idea i have used is to place an if statement check inside the query statement call back and place the query statement call back in a for loop (so you can loop through all of your queries), so the if statement should check if this the last call back expected, then you should execute a return statement or a deferred.resolve statement, the following is a concept code.
var list=fooKeys //list of keys (requests) i want to fetch form firebase
var array=[] // This is the array that will hold the result of all requests
for(i=xyz;loop breaking condition; i++){
Ref = new Firebase("https://yourlink.firebaseio.com/foo/" + fooKeys[i]);
Ref.once("value", function (data) {
array.push(data.val());
if(loop breaking condition == true){
//This mean that we looped over all items
return array; //or deferred.resolve(array);
}
})
}
Putting this code in a function and call it asynchronously will give you the ability to wait for the whole results before proceed in doing other stuff.
Hope you (and the others) find this beneficial.