How is it possible to wait for an #escaping closure to complete inline before proceeding?
I am utilizing the write method from AVSpeechSynthesizer, which uses an #escaping closure, so the initial AVAudioBuffer from the callback will return after createSpeechToBuffer has completed.
func write(_ utterance: AVSpeechUtterance, toBufferCallback bufferCallback: #escaping AVSpeechSynthesizer.BufferCallback)
My method writes speech to a buffer, then resamples and manipulates the output, for a workflow, where speech is done in faster than real-time.
The goal is to perform the task inline, to avoid changing the workflow to standby for the 'didFinish' delegate
speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance)
I believe this question can be generalized to dealing with #escaping closures within a function\method
import Cocoa
import AVFoundation
let _speechSynth = AVSpeechSynthesizer()
func resampleBuffer( inSource: AVAudioPCMBuffer, newSampleRate: Float) -> AVAudioPCMBuffer
{
// simulate resample data here
let testCapacity = 1024
let audioFormat = AVAudioFormat(standardFormatWithSampleRate: Double(newSampleRate), channels: 2)
let simulateResample = AVAudioPCMBuffer(pcmFormat: audioFormat!, frameCapacity: UInt32(testCapacity))
return simulateResample!
}
func createSpeechToBuffer( stringToSpeak: String, sampleRate: Float) -> AVAudioPCMBuffer?
{
var outBuffer : AVAudioPCMBuffer? = nil
let utterance = AVSpeechUtterance(string: stringToSpeak)
var speechIsBusy = true
utterance.voice = AVSpeechSynthesisVoice(language: "en-us")
let semaphore = DispatchSemaphore(value: 0)
_speechSynth.write(utterance) { (buffer: AVAudioBuffer) in
guard let pcmBuffer = buffer as? AVAudioPCMBuffer else {
fatalError("unknown buffer type: \(buffer)")
}
if ( pcmBuffer.frameLength == 0 ) {
print("buffer is empty")
} else {
print("buffer has content \(buffer)")
}
outBuffer = resampleBuffer( inSource: pcmBuffer, newSampleRate: sampleRate)
speechIsBusy = false
// semaphore.signal()
}
// wait for completion of func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance)
// while ( _speechSynth.isSpeaking )
// {
// /* arbitrary task waiting for write to complete */
// }
//
// while ( speechIsBusy )
// {
// /* arbitrary task waiting for write to complete */
// }
// semaphore.wait()
return outBuffer
}
print("SUCCESS is waiting, returning the non-nil output from the resampleBuffer method.")
for indx in 1...10
{
let sentence = "This is sentence number \(indx). [[slnc 3000]] \n"
let outBuffer = createSpeechToBuffer( stringToSpeak: sentence, sampleRate: 48000.0)
print("outBuffer: \(String(describing: outBuffer))")
}
After I wrote the createSpeechToBuffer method and it failed to produce the desired output (inline), I realized that it returns before getting the results of the resampling. The callback is escaping, so the initial AVAudioBuffer from the callback will return after createSpeechToBuffer has completed. The actual resampling does work, however I currently must save the result and continue after being notified by the delegate "didFinish utterance" to proceed.
Attempts at waiting for _speechSynth.isSpeaking, speechIsBusy flag, dispatch queue and semaphore are blocking the write method (using _speechSynth.write) from completing.
How is it possible to wait for the result inline versus recreating a workflow depending on the delegate "didFinish utterance"?
I'm on macOS 11.4 (Big Sur) but I believe this question is applicable to macOS and ios
It looks to me that the commented-out code for DispatchSemaphore would work if the #escaping closure is run concurrently, and I think the problem is that it is run serially, or more accurately, not run at all, because it is scheduled to run serially. I'm not specifically familiar with the AVSpeechSynthesizer API, but from your description, it sounds to me as though it's calling on the main dispatch queue, which is a serial queue. You call wait to block until _speechSynth.write completes, but that's blocking the main thread, which prevents it from ever continuing to the next iteration of the run loop, so the actual work of _speechSynth.write never even starts.
Let's back up. Somewhere behind the scenes your closure is almost certainly called via DispatchQueue.main's async method, either because that's where speechSynth.write does its work then calls your closure synchronously on the current thread at the time, or because it explicitly calls it on the main thread.
A lot of programmers are sometimes confused as to exactly what async does. All async means is "schedule this task and return control to the caller immediately". That's it. It does not mean that the task will be run concurrently, only that it will be run later. Whether it is run concurrently or serially is an attribute of the DispatchQueue whose async method is being called. Concurrent queues spin up threads for their tasks, which either can be run in parallel on different CPU cores (true concurrency), or interleaved with the current thread on the same core (preemptive multitasking). Serial queues on the other hand have a run loop as in NSRunLoop, and run their scheduled tasks synchronously after dequeuing them.
To illustrate what I mean, the main run loop looks vaguely like this, and other run loops are similar:
while !quit
{
if an event is waiting {
dispatch the event <-- Your code is likely blocking in here
}
else if a task is waiting in the queue
{
dequeue the task
execute the task <-- Your closure would be run here
}
else if a timer has expired {
run timer task
}
else if some view needs updating {
call the view's draw(rect:) method
}
else { probably other things I'm forgetting }
}
createSpeechToBuffer is almost certainly being run in response to some event processing, which means that when it blocks, it does not return back to the run loop to continue to the next iteration where it checks for tasks in the queue... which from the behavior you describe, seems to include the work being done by _speechSynth.write... the very thing you're waiting for.
You can try explicitly creating a .concurrent DispatchQueue and using it to wrap the call to _speechSynth.write in an explicit async call, but that probably won't work, and even if it does, it will be fragile to changes Apple might make to AVSpeechSynthesizer's implementation.
The safe way is to not block... but that means re-thinking your work flow a little. Basically whatever code would be called after createSpeechToBuffer returns should be called at the end of your closure. Of course, as currently written createSpeechToBuffer doesn't know what that code is (nor should it). The solution is to inject it as a parameter... meaning createSpeechToBuffer itself would also take an #escaping closure. And of course, that means it can't return the buffer, but instead passes it to the closure.
func createSpeechToBuffer(
stringToSpeak: String,
sampleRate: Float,
onCompletion: #escaping (AVAudioPCMBuffer?) -> Void)
{
let utterance = AVSpeechUtterance(string: stringToSpeak)
utterance.voice = AVSpeechSynthesisVoice(language: "en-us")
let semaphore = DispatchSemaphore(value: 0)
_speechSynth.write(utterance) { (buffer: AVAudioBuffer) in
guard let pcmBuffer = buffer as? AVAudioPCMBuffer else {
fatalError("unknown buffer type: \(buffer)")
}
if ( pcmBuffer.frameLength == 0 ) {
print("buffer is empty")
} else {
print("buffer has content \(buffer)")
}
onCompletion(
resampleBuffer(
inSource: pcmBuffer,
newSampleRate: sampleRate
)
)
}
}
If you really want to maintain the existing API, the other approach is to move the entire workflow itself to a .concurrent DispatchQueue, which you can block to your heart's content without worry that it will block the main thread. AVSpeechSynthesizer could schedule its work wherever it likes without a problem.
If using Swift 5.5 is an option, you might look into its async and await keywords. The compiler enforces a proper async context for them so that you don't block the main thread.
Update to answer how to call my version.
Let's say your code that calls createSpeechToBuffer currently looks like this:
guard let buffer = createSpeechToBuffer(stringToSpeak: "Hello", sampleRate: sampleRate)
else { fatalError("Could not create speechBuffer") }
doSomethingWithSpeechBuffer(buffer)
You'd call the new version like this:
createSpeechToBuffer(stringToSpeak: "Hello", sampleRate: sampleRate)
{
guard let buffer = $0 else {
fatalError("Could not create speechBuffer")
}
doSomethingWithSpeechBuffer(buffer)
}
Related
I'm trying to improve the time it takes for a task to finish by leveraging multithreading/paralleling.
I'm reading CMSampleBuffers from a video track, manipulating them, then storing them back to an array for later use. Because each manipulation is "costly" in terms of RAM and CPU, I'm using DispatchSemaphore to limit the number of parallel tasks (5 in the example).
Basically, I'm trying to make the "system" process more than a single frame in a time, and not above 5 so the device won't crash due to memory issues. In the current implementation below it's for some reason doing it almost serialize and not in parallel.
Any help will be highly appreciated!
I tried taking reference from here: How to limit gcd queue buffer size for the implementation.
Code:
class MyService {
let semaphore = DispatchSemaphore(value: 5)
let processQueue = DispatchQueue(label: "custom.process", attributes: .concurrent)
func startReading() {
for sampleBuffer in sampleBuffers {
// signal wait
semaphore.wait()
// async queue
processQueue.async {
// run taks
self.process(buffer: sampleBuffer) { pixelBuffer in
// singal to semaphore
self.semaphore.signal()
}
}
}
}
func process(buffer: CMSampleBuffer, completion: #escaping (CVPixelBuffer) -> (Void)) {
// run on a background thread to avoid UI freeze
DispatchQueue.global(qos: .userInteractive).async {
// Do something
// Do something
// Do something
// Do something
completion(processedBuffer)
}
}
}
Background
I'm implementing a search. Each search query results in one DispatchWorkItem which is then queued for execution. As the user can trigger a new search faster than the previous one can be completed, I'd like to cancel the previous one as soon as I receive a new one.
This is my current setup:
var currentSearchJob: DispatchWorkItem?
let searchJobQueue = DispatchQueue(label: QUEUE_KEY)
func updateSearchResults(for searchController: UISearchController) {
let queryString = searchController.searchBar.text?.lowercased() ?? ""
// if there is already an (older) search job running, cancel it
currentSearchJob?.cancel()
// create a new search job
currentSearchJob = DispatchWorkItem() {
self.filter(queryString: queryString)
}
// start the new job
searchJobQueue.async(execute: currentSearchJob!)
}
Problem
I understand that dispatchWorkItem.cancel() doesn't kill the running task immediately. Instead, I need to check for dispatchWorkItem.isCancelled manually. But how do I get the right dispatchWorkItemobject in this case?
If I were setting currentSearchJob only once, I could simply access that attribute like done in this case. However, this isn't applicable here, because the attribute will be overriden before the filter() method will be finished. How do I know which instance is actually running the code in which I want to check for dispatchWorkItem.isCancelled?
Ideally, I'd like to provide the newly-created DispatchWorkItem as an additional parameter to the filter() method. But that's not possible, because I'll get a Variable used within its own initial value error.
I'm new to Swift, so I hope I'm just missing something. Any help is appreciated very much!
The trick is how to have a dispatched task check if it has been canceled. I'd actually suggest consider OperationQueue approach, rather than using dispatch queues directly.
There are at least two approaches:
Most elegant, IMHO, is to just subclass Operation, passing whatever you want to it in the init method, and performing the work in the main method:
class SearchOperation: Operation {
private var queryString: String
init(queryString: String) {
self.queryString = queryString
super.init()
}
override func main() {
// do something synchronous, periodically checking `isCancelled`
// e.g., for illustrative purposes
print("starting \(queryString)")
for i in 0 ... 10 {
if isCancelled { print("canceled \(queryString)"); return }
print(" \(queryString): \(i)")
heavyWork()
}
print("finished \(queryString)")
}
func heavyWork() {
Thread.sleep(forTimeInterval: 0.5)
}
}
Because that's in an Operation subclass, isCancelled is implicitly referencing itself rather than some ivar, avoiding any confusion about what it's checking. And your "start a new query" code can just say "cancel anything currently on the the relevant operation queue and add a new operation onto that queue":
private var searchQueue: OperationQueue = {
let queue = OperationQueue()
// queue.maxConcurrentOperationCount = 1 // make it serial if you want
queue.name = Bundle.main.bundleIdentifier! + ".backgroundQueue"
return queue
}()
func performSearch(for queryString: String) {
searchQueue.cancelAllOperations()
let operation = SearchOperation(queryString: queryString)
searchQueue.addOperation(operation)
}
I recommend this approach as you end up with a small cohesive object, the operation, that nicely encapsulates a block of work that you want to do, in the spirit of the Single Responsibility Principle.
While the following is less elegant, technically you can also use BlockOperation, which is block-based, but for which which you can decouple the creation of the operation, and the adding of the closure to the operation. Using this technique, you can actually pass a reference to the operation to its own closure:
private weak var lastOperation: Operation?
func performSearch(for queryString: String) {
lastOperation?.cancel()
let operation = BlockOperation()
operation.addExecutionBlock { [weak operation, weak self] in
print("starting \(identifier)")
for i in 0 ... 10 {
if operation?.isCancelled ?? true { print("canceled \(identifier)"); return }
print(" \(identifier): \(i)")
self?.heavyWork()
}
print("finished \(identifier)")
}
searchQueue.addOperation(operation)
lastOperation = operation
}
func heavyWork() {
Thread.sleep(forTimeInterval: 0.5)
}
I only mention this for the sake of completeness. I think the Operation subclass approach is frequently a better design. I'll use BlockOperation for one-off sort of stuff, but as soon as I want more sophisticated cancelation logic, I think the Operation subclass approach is better.
I should also mention that, in addition to more elegant cancelation capabilities, Operation objects offer all sorts of other sophisticated capabilities (e.g. asynchronously manage queue of tasks that are, themselves, asynchronous; constrain degree of concurrency; etc.). This is all beyond the scope of this question.
you wrote
Ideally, I'd like to provide the newly-created DispatchWorkItem as an
additional parameter
you are wrong, to be able to cancel running task, you need a reference to it, not to the next which is ready to dispatch.
cancel() doesn't cancel running task, it only set internal "isCancel" flag by the thread-safe way, or remove the task from the queue before execution. Once executed, checking isCancel give you a chance to finish the job (early return).
import PlaygroundSupport
import Foundation
PlaygroundPage.current.needsIndefiniteExecution = true
let queue = DispatchQueue.global(qos: .background)
let prq = DispatchQueue(label: "print.queue")
var task: DispatchWorkItem?
func work(task: DispatchWorkItem?) {
sleep(1)
var d = Date()
if task?.isCancelled ?? true {
prq.async {
print("cancelled", d)
}
return
}
sleep(3)
d = Date()
prq.async {
print("finished", d)
}
}
for _ in 0..<3 {
task?.cancel()
let item = DispatchWorkItem {
work(task: task)
}
item.notify(queue: prq) {
print("done")
}
queue.asyncAfter(deadline: .now() + 0.5, execute: item)
task = item
sleep(1) // comment this line
}
in this example, only the very last job is really fully executed
cancelled 2018-12-17 23:49:13 +0000
done
cancelled 2018-12-17 23:49:14 +0000
done
finished 2018-12-17 23:49:18 +0000
done
try to comment the last line and it prints
done
done
finished 2018-12-18 00:07:28 +0000
done
the difference is, that first two execution never happened. (were removed from the dispatch queue before execution)
I have got a concurrent queue with dispatch barrier from Raywenderlich post Example
private let concurrentPhotoQueue = DispatchQueue(label: "com.raywenderlich.GooglyPuff.photoQueue", attributes: .concurrent)
Where write operations is done in
func addPhoto(_ photo: Photo) {
concurrentPhotoQueue.async(flags: .barrier) { [weak self] in
// 1
guard let self = self else {
return
}
// 2
self.unsafePhotos.append(photo)
// 3
DispatchQueue.main.async { [weak self] in
self?.postContentAddedNotification()
}
}
}
While read opeartion is done in
var photos: [Photo] {
var photosCopy: [Photo]!
// 1
concurrentPhotoQueue.sync {
// 2
photosCopy = self.unsafePhotos
}
return photosCopy
}
As this will resolve Race Condition. Here why only Write operation is done with barrier and Read in Sync. Why is Read not done with barrier and write with sync ?. As with Sync Write, it will wait till it reads like a lock and while barrier Read it will only be read operation.
set(10, forKey: "Number")
print(object(forKey: "Number"))
set(20, forKey: "Number")
print(object(forKey: "Number"))
public func set(_ value: Any?, forKey key: String) {
concurrentQueue.sync {
self.dictionary[key] = value
}
}
public func object(forKey key: String) -> Any? {
// returns after concurrentQueue is finished operation
// beacuse concurrentQueue is run synchronously
var result: Any?
concurrentQueue.async(flags: .barrier) {
result = self.dictionary[key]
}
return result
}
With the flip behavior, I am getting nil both times, with barrier on Write it is giving 10 & 20 correct
You ask:
Why is Read not done with barrier ... ?.
In this reader-writer pattern, you don’t use barrier with “read” operations because reads are allowed to happen concurrently with respect to other “reads”, without impacting thread-safety. It’s the whole motivating idea behind reader-writer pattern, to allow concurrent reads.
So, you could use barrier with “reads” (it would still be thread-safe), but it would unnecessarily negatively impact performance if multiple “read” requests happened to be called at the same time. If two “read” operations can happen concurrently with respect to each other, why not let them? Don’t use barriers (reducing performance) unless you absolutely need to.
Bottom line, only “writes” need to happen with barrier (ensuring that they’re not done concurrently with respect to any “reads” or “writes”). But no barrier is needed (or desired) for “reads”.
[Why not] ... write with sync?
You could “write” with sync, but, again, why would you? It would only degrade performance. Let’s imagine that you had some reads that were not yet done and you dispatched a “write” with a barrier. The dispatch queue will ensure for us that a “write” dispatched with a barrier won’t happen concurrently with respect to any other “reads” or “writes”, so why should the code that dispatched that “write” sit there and wait for the “write” to finish?
Using sync for writes would only negatively impact performance, and offers no benefit. The question is not “why not write with sync?” but rather “why would you want to write with sync?” And the answer to that latter question is, you don’t want to wait unnecessarily. Sure, you have to wait for “reads”, but not “writes”.
You mention:
With the flip behavior, I am getting nil ...
Yep, so lets consider your hypothetical “read” operation with async:
public func object(forKey key: String) -> Any? {
var result: Any?
concurrentQueue.async {
result = self.dictionary[key]
}
return result
}
This effective says “set up a variable called result, dispatch task to retrieve it asynchronously, but don’t wait for the read to finish before returning whatever result currently contained (i.e., nil).”
You can see why reads must happen synchronously, because you obviously can’t return a value before you update the variable!
So, reworking your latter example, you read synchronously without barrier, but write asynchronously with barrier:
public func object(forKey key: String) -> Any? {
return concurrentQueue.sync {
self.dictionary[key]
}
}
public func set(_ value: Any?, forKey key: String) {
concurrentQueue.async(flags: .barrier) {
self.dictionary[key] = value
}
}
Note, because sync method in the “read” operation will return whatever the closure returns, you can simplify the code quite a bit, as shown above.
Or, personally, rather than object(forKey:) and set(_:forKey:), I’d just write my own subscript operator:
public subscript(key: String) -> Any? {
get {
concurrentQueue.sync {
dictionary[key]
}
}
set {
concurrentQueue.async(flags: .barrier) {
self.dictionary[key] = newValue
}
}
}
Then you can do things like:
store["Number"] = 10
print(store["Number"])
store["Number"] = 20
print(store["Number"])
Note, if you find this reader-writer pattern too complicated, note that you could just use a serial queue (which is like using a barrier for both “reads” and “writes”). You’d still probably do sync “reads” and async “writes”. That works, too. But in environments with high contention “reads”, it’s just a tad less efficient than the above reader-writer pattern.
I have a AVAudioPlayerNode that I'm scheduling a lot of buffers on using the following:
node.scheduleBuffer(buffer, at: nil, options: .interrupts, completionHandler: completeFunc)
But I'm having a bit of a problem. If I play another buffer interrupting the currently playing buffer, the completion handler for the buffer getting interrupted is still being called. I guess this makes sense, but I can't seem to find a way to check if the file actually COMPLETED playing or if it was interrupted. Is there a way I can do this?
Any answers will help!
You will need an interruption state variable that is referenced in the completion handler and toggled by the interrupting code. Here's some shorthand code that makes some assumptions on how you're looping through buffers:
var node: AVAudioPlayerNode!
var loopingBuffers: [AVAudioPCMBuffer] = [buffer1, buffer2, buffer3]
var interruptBuffers = false // state variable to handle the interruption
func scheduleNextBuffer() {
/* code to find the next buffer in the loopingBuffers queue */
// assuming you have instantiated an AVAudioPlayerNode elsewhere
node.scheduleBuffer(nextBuffer, completionHandler: bufferCompletion)
}
func bufferCompletion() {
// check the state variable to make certain the completionHandler isn't being triggered by a scheduling interruption
guard !interruptBuffers else { return }
scheduleNextBuffer()
}
func interruptBuffers() {
let newBuffers: [AVAudioPCMBuffer] = [buffer4, buffer5, buffer6]
interruptBuffers = true
node.scheduleBuffer(buffer4, at: nil, options: .interrupts, completionHandler: bufferCompletion)
// cleanup your state and queue up your new buffers
interruptBuffers = false
loopingBuffers = newBuffers
}
In Swift (that's Swift) there are a number of ways to handle asynchronous,
Say you have a loop like this - it's calling a parse cloud code call which goes to background anyway.
public func runImages()
{
print("test begins...")
for i in 1...3
{
print("Tick tock tick tock ..\(i)")
PFCloud.callFunctionInBackground("blah", withParameters:["bla":i,"bla":"bla] )
{
(response: AnyObject?, error: NSError?) -> Void in
print(".. done! Now go again...")
if let rr = response as? String { print(rr) }
if let err = error { print(err.domain) }
}
}
}
How to make that wait for the end of each PFCloud call?
Really is just an ordinary flag best, or? (Note that (a) I can't get a flag to work in Swift and (b) as Paul points out you're blocking the UI!!)
What is the "Swift way" in the context you see here? I feel it would be very inelegant to use a recursive pattern here, for example.
If you want the three calls to execute serially then you can use a serial dispatch queue and a dispatch_barrier_async to do something when all three are finished -
let dispatchQueue=dispatch_queue_create("queue", DISPATCH_QUEUE_SERIAL)
for i in 1...3 {
dispatch_async(dispatchQueue, { () -> Void in
print("i=\(i)")
let result = PFCloud.callFunction("blah", withParameters:["bla":i,"bla":"bla] )
})
}
dispatch_barrier_async(dispatchQueue, { () -> Void in
print("really done")
})
print(" done")
In order for this to work with your Parse function you need to use the synchronous cloud code call, not an asynchronous. And if you update UI in the dispatch_barrier_async closure you would need to dispatch that on the main queue.