Dispatching delegate callback to main queue, from another queue, AVFoundation - ios

I am working on a camera that is wrapped up in a base viewController, with a delegate style interface for callbacks, so all I have to do as a client is subclass the camera view controller, implement the delegate methods, and add the UI buttons.
My question is about recording video. Video recording is started on a unique background task to ensure that the recording can be written to a temporary file. This is done on my
private let sessionQueue = DispatchQueue(label: "com.andrewferrarone.sessionQueue") session queue:
public func startRecording()
{
guard let movieFileOutput = self.movieFileOutput else { return }
//get video preview layer's video orientation on main queue
guard let videoPreviewLayerOrientation = self.previewView.videoPreviewLayer?.connection.videoOrientation else {
print("handleRecord: videoPreviewLayer is nil")
return
}
self.sessionQueue.async {
if !movieFileOutput.isRecording {
if UIDevice.current.isMultitaskingSupported {
self.backgroundRecordingID = UIApplication.shared.beginBackgroundTask(expirationHandler: nil)
}
//update orientation on the movie file output video connection before recording
let movieFileOutputConnection = self.movieFileOutput?.connection(withMediaType: AVMediaTypeVideo)
movieFileOutputConnection?.videoOrientation = videoPreviewLayerOrientation
//start recording to a temporary file:
let outputFileName = UUID().uuidString
let outputFilePath = (NSTemporaryDirectory() as NSString).appendingPathComponent((outputFileName as NSString).appendingPathExtension("mov")!)
movieFileOutput.startRecording(toOutputFileURL: URL(fileURLWithPath: outputFilePath), recordingDelegate: self)
}
}
}
so the recording is setup as a background task dispatched to self.sessionQueue. When I stop recording I receive an AVCaptureFileOutputRecordingDelegate method. In this method, I want to callback my delegate with the filepath, and then cleanup. How do I ensure that the delegate can persist the recording from the temporary file path before cleanup happens and the temporary file is removed?
public func capture(_ captureOutput: AVCaptureFileOutput!, didFinishRecordingToOutputFileAt outputFileURL: URL!, fromConnections connections: [Any]!, error: Error!)
{
//cleanup func for later
func cleanup()
{
let path = outputFileURL.path
if FileManager.default.fileExists(atPath: path) {
do {
try FileManager.default.removeItem(atPath: path)
}
catch let error {
print("Could not remove file at url: \(outputFileURL), error: \(error.localizedDescription)")
}
}
if let currentBackgroundRecordingID = self.backgroundRecordingID {
self.backgroundRecordingID = UIBackgroundTaskInvalid
if currentBackgroundRecordingID != UIBackgroundTaskInvalid {
UIApplication.shared.endBackgroundTask(currentBackgroundRecordingID)
}
}
}
var success = true
if error != nil {
print("Movie file finishing error: \(error)")
success = ((error as NSError).userInfo[AVErrorRecordingSuccessfullyFinishedKey] as AnyObject).boolValue
}
DispatchQueue.main.async {
self.delegate?.camera(self, didFinishRecordingToOutputFileAt: outputFileURL, success: success)
}
cleanup()
}
So I called my delegate back on the main queue with the results, but then I need to call cleanup() should I do this on the main queue right after calling back my delegate? is this safe? or if I leave it the way it is now, then we are on self.sessionQueue, and I am unsure if cleanup() will happen before the delegate method implementation has time to persist the recording. If anyone can give me some insight into what is going on and what would be the safest thing to do here, that would be great. According to apple, the documentation says do not assume that AVCaptureFileOutputRecordingDelegate didFinishRecordingToOutputFileAt method is called on a specific thread. Thanks for your time and help!

How about:
DispatchQueue.main.async {
self.delegate?.camera(self, didFinishRecordingToOutputFileAt: outputFileURL, success: success)
self.sessionQueue.async {
cleanup()
}
}
I think that would be the standard way of handling this situation. When the delegate method finishes, you assume that the delegate is done with the file (or copied it somewhere else).

Related

Swift iOS, Play tracks sequentially from array using AVAudioPlayer, with some function in between each track?

I have an array of audio instructions that I would like to randomize, play (one at a time), and do some data recording in between each instruction.
With the below code, I have a button press that triggers beginTest(), but the instruction tracks play simultaneously. It never gets to startRecord().
I've tried placing startRecord() inside audioPlayerDidFinishPlaying; in that case it does get to startRecord() for the first track, but doesn't continue to the next iteration of the loop inside beginTest() where playScript() is called.
func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool)
{
self.player = nil
print("finished playing this file")
}
func playScript(c: String)
{
let path = Bundle.main.path(forResource: c, ofType:"m4a")!
let url = URL(fileURLWithPath: path)
do {
self.player = try AVAudioPlayer(contentsOf: url)
self.player?.delegate = self
self.player?.play()
print(c)
print("playing script for " + c)
} catch {
print("couldn't load script")
}
}
func beginTest()
{
for c in conditions.shuffled()
{
playScript(c: c)
if self.player == nil
{
startRecord()
print("started data recording")
}
}
exportData()
}
I'm guessing that beginTest() keeps moving along the for loop before audioPlayerDidFinishPlaying resets the player to nil, but I don't know how to stall this.
The approach using audioPlayerDidFinishPlaying(_:,successfully:) is correct. In your code when you call playScript(c:) you will iterate through all audio files from conditions array and call startRecord method really quickly, while first audio is playing. All sounds will be placed in a queue and play one by one. I don't know contents of startRecord method, but I don't think that it was supposed to be called like that.
In order to avoid those problems you need to call playScript(c:) and then wait for method startRecord() to finish, and then call playScript(c:) again. So you can try something like this:
import AVFoundation
class SomeClass: NSObject, AVAudioPlayerDelegate {
var player: AVAudioPlayer?
var conditions = [String]()
var instructions: [() -> ()] = []
func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool) {
print("finished playing \(player.url)")
startRecord {
playNextInstruction()
}
}
func playScript(c: String)
{
let path = Bundle.main.path(forResource: c, ofType:"m4a")!
let url = URL(fileURLWithPath: path)
do {
self.player = try AVAudioPlayer(contentsOf: url)
self.player?.delegate = self
self.player?.play()
print(c)
print("playing script for " + c)
} catch {
print("couldn't load script")
}
}
func beginTest() {
instructions = conditions.shuffled().map { [unowned self] condition in
return {
self.playScript(c: condition)
}
}
playNextInstruction()
}
func playNextInstruction() {
if instructions.count != 0 {
instructions.remove(at: 0)()
} else {
exportData()
}
}
func exportData() {
}
func startRecord(completion: (() -> ())) {
}
}
If it won't work then I suggest to try searching problem somewhere else in your code.

CallKit Audio session starts when navigating to the application only

I'm working on a voip app now and want to support holding.
But when a second call comes and I hold my current call. Switching to my first call I hear no sound at all.
The way to hear it is to navigate from callKit native screen to my app and hence I can hear the voice.
func configureAudioSession() {
_ = try? AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord, mode: .videoChat, options: AVAudioSession.CategoryOptions.mixWithOthers)
_ = try? AVAudioSession.sharedInstance().overrideOutputAudioPort(AVAudioSession.PortOverride.none)
_ = try? AVAudioSession.sharedInstance().setMode(AVAudioSession.Mode.voiceChat)
}
func startAudio() {
print("Starting audio")
do {
_ = try AVAudioSession.sharedInstance().setActive(true)
} catch {
}
}
func stopAudio() {
print("Stopping audio")
do {
_ = try AVAudioSession.sharedInstance().setActive(false)
} catch {
}
}
For supporting holding, you dont have to start/stop audio session, instead you can use CXSetHeldCallAction provided by Callkit itself. Here, is the code for hold that i use.
let callKitCallController = CXCallController()
func performHoldAction(isOnHold:Bool, uuid:UUID) {
let holdCallAction = CXSetHeldCallAction(call: uuid, onHold: isOnHold)
let transaction = CXTransaction(action: holdCallAction)
callKitCallController.request(transaction) { error in
if let error = error {
CPrint("holdCallAction transaction request failed: \(error.localizedDescription).")
return
}
CPrint("holdCallAction transaction request successful")
}
}
Once system puts the call on hold(by above method OR due to other incoming call accepting or any other reason), then in CXProviderDelegate, the method func provider(_ provider: CXProvider, perform action: CXSetHeldCallAction) gives you the callback for the detail.
Here, system/callkit itself interacts with audio, you dont have to explicitly start or stop audio for holding.
Note: Make sure that you given supportsHolding to true for the CXCallUpdate that you gave for new call.

AVAudioSession never stopped

I'm trying to set my AVAudioSession to inactive to get back to normal state.
My utterance function:
class SSpeech : NSObject, AVSpeechSynthesizerDelegate {
var group = DispatchGroup();
var queue = DispatchQueue(label: "co.xxxx.speech", attributes: [])
class var sharedInstance: SSpeech {
struct Static {
static var instance: SSpeech?
}
if !(Static.instance != nil) {
Static.instance = SSpeech()
}
return Static.instance!
}
required override init() {
super.init();
self.speechsynt.delegate = self;
}
deinit {
print("deinit SSpeech")
}
let audioSession = AVAudioSession.sharedInstance();
var speechsynt: AVSpeechSynthesizer = AVSpeechSynthesizer()
var queueTalks = SQueue<String>();
func pause() {
speechsynt.pauseSpeaking(at: .word)
}
func talk(_ sentence: String, languageCode code:String = SUtils.selectedLanguage.code, withEndPausing: Bool = false) {
if SUser.sharedInstance.currentUser.value!.speechOn != 1 {
return
}
queue.async{
self.queueTalks.enQueue(sentence)
do {
let category = AVAudioSessionCategoryPlayback;
var categoryOptions = AVAudioSessionCategoryOptions.duckOthers
if #available(iOS 9.0, *) {
categoryOptions.formUnion(AVAudioSessionCategoryOptions.interruptSpokenAudioAndMixWithOthers)
}
try self.audioSession.setCategory(category, with: categoryOptions)
try self.audioSession.setActive(true);
} catch _ {
return;
}
self.utteranceTalk(sentence, initSentence: false, speechsynt: self.speechsynt, languageCode:code, withEndPausing: withEndPausing)
do {
try self.audioSession.setCategory(AVAudioSessionCategoryPlayback, with: AVAudioSessionCategoryOptions.mixWithOthers)
} catch _ {
return;
}
}
}
func utteranceTalk(_ sentence: String, initSentence: Bool, speechsynt: AVSpeechSynthesizer, languageCode:String = "en-US", withEndPausing: Bool = false){
if SUser.sharedInstance.currentUser.value!.speechOn != 1 {
return
}
let nextSpeech:AVSpeechUtterance = AVSpeechUtterance(string: sentence)
nextSpeech.voice = AVSpeechSynthesisVoice(language: languageCode)
if !initSentence {
nextSpeech.rate = 0.4;
}
if(withEndPausing){
nextSpeech.postUtteranceDelay = 0.2;
}
speechsynt.speak(nextSpeech)
}
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance:AVSpeechUtterance) {
print("Speaker has finished to talk")
queue.async {
do {
try self.audioSession.setActive(false, with: AVAudioSessionSetActiveOptions.notifyOthersOnDeactivation)
}
catch {}
}
}
}
}
My method is correctly called, but my audioSession still active when the utterance is finished. i've tried lot of thing but nothing work :(.
I would suggest using an AvAudioPlayer. They have very easy start and stop commands.
first declare the audio player as a variable
var SoundEffect: AVAudioPlayer!
then select the file you need
let path = Bundle.main.path(forResource: "Untitled2.wav", ofType:nil)!
let url = URL(fileURLWithPath: path)
let sound = try AVAudioPlayer(contentsOf: url)
SoundEffect = sound
sound.numberOfLoops = -1
sound.play()
and to stop the audio player
if SoundEffect != nil {
SoundEffect.stop()
SoundEffect = nil
}
You cannot stop or deactive AudioSession, your app gets it upon launching. Documentation:
An audio session is the intermediary between your app and iOS used to configure your app’s audio behavior. Upon launch, your app automatically gets a singleton audio session.
So method -setActive: does not make your AudioSession "active", it just puts its category and mode configuration into action. For getting back to the "normal state", you could set default settings or just call setActive(false, with:.notifyOthersOnDeactivation), that will be enough.
A part from documentation of AVAudioSession:
Discussion
If another active audio session has higher priority than yours (for
example, a phone call), and neither audio session allows mixing,
attempting to activate your audio session fails. Deactivating your
session will fail if any associated audio objects (such as queues,
converters, players, or recorders) are currently running.
My guess is that the failure to deactivate the session is the running process(es) of your queue as I highlighted in the document quote.
Probably you should make the deactivation process synchronous instead of asynchronous OR make sure that all the running actions under your queue has been processed.
Give this a try:
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance:AVSpeechUtterance) {
print("Speaker has finished to talk")
queue.sync { // <---- `async` changed to `sync`
do {
try self.audioSession.setActive(false, with: AVAudioSessionSetActiveOptions.notifyOthersOnDeactivation)
}
catch {}
}
}
}

didActivate is not called back by the CallKit

I am integrating the new CallKit API with my VOIP app.
As shown in the example app: https://developer.apple.com/library/content/samplecode/Speakerbox/Introduction/Intro.html
I am configuring the audio session:
- (void) configureAudioSession
{
// Configure the audio session
AVAudioSession *sessionInstance = [AVAudioSession sharedInstance];
// we are going to play and record so we pick that category
NSError *error = nil;
[sessionInstance setCategory:AVAudioSessionCategoryPlayAndRecord error:&error];
if (error) {
NSLog(#"error setting audio category %#",error);
}
// set the mode to voice chat
[sessionInstance setMode:AVAudioSessionModeVoiceChat error:&error];
if (error) {
NSLog(#"error setting audio mode %#",error);
}
NSLog(#"setupAudioSession");
return;
}
in my CXAnswerCallAction:
func provider(_ provider: CXProvider, perform action: CXAnswerCallAction) {
print("Provider - CXAnswerCallAction")
// get the active call
guard let call = self.softphone.getCallForCallId(self.currentCallId) else {
action.fail()
return
}
/*
Configure the audio session, but do not start call audio here, since it must be done once
the audio session has been activated by the system after having its priority elevated.
*/
self.softphone.configureAudioSession()
// Trigger the call to be answered via the underlying network service.
call.answer()
// Signal to the system that the action has been successfully performed.
action.fulfill()
}
According to the documentation, didActivate should be called back by the callkit:
func provider(_ provider: CXProvider, didActivate audioSession: AVAudioSession) {
print("Provider - Received \(#function)")
// Start call audio media, now that the audio session has been activated after having its priority boosted.
}
For some reasons, it's not called back after the first VOIP call. The subsequent calls seem to receive the callback and they work fine.
How to fix this?
I've fixed this problem by setting call audio first then call "reportNewIncomingCall" method. Sample code is given below:
func reportIncomingCall(uuid: UUID, handle: String, hasVideo: Bool = false, completion: ((NSError?) -> Void)? = nil) {
let update = CXCallUpdate()
update.remoteHandle = CXHandle(type: .phoneNumber, value: handle)
update.hasVideo = hasVideo
DispatchQueue.global().sync {
_ = try? AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord, with: AVAudioSessionCategoryOptions.mixWithOthers)
_ = try? AVAudioSession.sharedInstance().overrideOutputAudioPort(AVAudioSessionPortOverride.none)
if hasVideo == true {
_ = try? AVAudioSession.sharedInstance().setMode(AVAudioSessionModeVideoChat)
} else {
_ = try? AVAudioSession.sharedInstance().setMode(AVAudioSessionModeVoiceChat)
}
do {
_ = try AVAudioSession.sharedInstance().setActive(true)
} catch (let error){
print("audio session error: \(error)")
}
}
provider.reportNewIncomingCall(with: uuid, update: update) { error in
if error == nil {
}
completion?(error as? NSError)
}
}

How to use thread to play next song by MPRemoteCommandCenter?

I want to use MPRemoteCommandCenter to control my music player app. And now it can play and pause music, but can not play next/previous song, only a poor chance can make it.
When user tap next song button in MPRemoteCommandCenter(e.g in the lock screen), it will call startExtendBGJob() function, then I ask for a thread to do the change song job(I think the bug is here, because I'm don't totally understand the background job's anatomy).
func startExtendBGJob(taskBlock: #escaping () -> Void) {
registerBackgroundTask()
DispatchQueue.global(qos: .userInitiated).async {
DLog("APP into BG")
DispatchQueue.global(qos: .default).async {
taskBlock()
}
while self.isPlaying == false || self.tmpPlayer == nil { // waiting for new avplayer been created.
Thread.sleep(forTimeInterval: 1)
}
DispatchQueue.global(qos: .default).asyncAfter(deadline: .now() + 10) {
self.endBackgroundTask()
}
}
}
func registerBackgroundTask() {
bgIdentifier = UIApplication.shared.beginBackgroundTask(expirationHandler: {
[weak self] in
guard let strongSelf = self else { return }
strongSelf.endBackgroundTask()
})
assert(bgIdentifier != UIBackgroundTaskInvalid)
}
func endBackgroundTask() {
UIApplication.shared.endBackgroundTask(bgIdentifier)
bgIdentifier = UIBackgroundTaskInvalid
isExtendingBGJob = false
DLog("App exit BG!")
}
In startNextPlay() function just finding the next song's url, and prepareToPlay() is for creating a new AVPlayer to play next song.
self.tmpPlayer = AVPlayer(url: streamURL)
I'm not english native spearker, thank you so much to read here if you understand what I'm talking about :]. Any help is welcome.
Sorry, It's my fault. It doesn't need any backgournd job.
Just change something like following:
UIApplication.shared.beginReceivingRemoteControlEvents()
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
try AVAudioSession.sharedInstance().setActive(true)
} catch {
DLog("fail to set category: \(error)")
}
Make sure it's AVAudioSessionCategoryPlayback

Resources