I am using Twilio programmable video to connect two users in an audio chat. I want to give the user the option to record their screen during the audio session so I am using Replaykit. Everything works, except the audio on the recording cuts out as soon as Twilio starts.
Is there some conflict between the type of audio Twilio uses and Replaykit audio capture?
I experienced something like this before when trying to add sound while Twilio was active and it would cause Twilio audio to cut out as soon as another sound was played.
Edit: I've tried different ways, so I only have my latest changes, but here is the code I am using for ReplayKit. It's just standard Start, Stop, and Preview recording.
func startRecording() {
guard recorder.isAvailable else {
print("Recording not available")
return
}
recorder.isMicrophoneEnabled = true
recorder.startRecording{ [unowned self] (error) in
guard error == nil else {
print("error starting the recording")
return
}
print("Started Recording Successfully")
self.isRecording = true
}
}
func stopRecording() {
recorder.stopRecording { [unowned self] (preview, error) in
print("Stopped recording")
guard preview != nil else {
print("Preview controller not available")
return
}
let alert = UIAlertController(title: "Recording Finished", message: "Would you like to edit or delete your recording?", preferredStyle: .alert)
let deleteAction = UIAlertAction(title: "Delete", style: .destructive, handler: { (action: UIAlertAction) in
self.recorder.discardRecording(handler: { () -> Void in
print("Recording suffessfully deleted.")
})
})
let editAction = UIAlertAction(title: "Edit", style: .default, handler: { (action: UIAlertAction) -> Void in
preview?.previewControllerDelegate = self
self.present(preview!, animated: true, completion: nil)
})
alert.addAction(editAction)
alert.addAction(deleteAction)
self.present(alert, animated: true, completion: nil)
self.isRecording = false
}
}
func previewControllerDidFinish(_ previewController: RPPreviewViewController) {
dismiss(animated: true)
}
I've also tried using the new recording feature from control panel. I can capture audio from other apps, but when Twilio starts on my app, the audio on the recording goes silent and comes back when Twilio stops. This is what makes me think there is some conflict between Twilio and Replaykit, but maybe there is a way to capture it that I don't know about.
I also tried .startCapture instead of .startRecording, but I don't think I was using it right and I haven't been able to find much documentation on it.
To prevent a new audio source to interrupt the previous one you can set its category to ambient:
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryAmbient)
In your case I think you should apply it to the Twilio Session
Related
Why won't RPScreenRecorder record the mic, even though it is enabled, if the permissions popup doesn't appear? It works when the popup appears but attempts after restarting the app don't record the mic.
here's the very simple app i made just to test this feature for a larger app.
I have tested this exact application on iOS 11 and it works every time. However on iOS 12+ it only works when the permission popup appears and that's every 8 minutes. It should work every time after giving permissions.
import ReplayKit
class ViewController: UIViewController, RPPreviewViewControllerDelegate {
private let recorder = RPScreenRecorder.shared()
private var isRecording = false
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
}
#IBAction func react() {
if !isRecording {
let alert = UIAlertController(title: "Record", message: "Would you like to record a video?", preferredStyle: .alert)
let okay = UIAlertAction(title: "Okay", style: .destructive, handler: { (action: UIAlertAction) in
self.startRecording()
})
alert.addAction(okay)
self.present(alert, animated: true, completion: nil)
} else {
stopRecording()
}
}
private func startRecording() {
guard self.recorder.isAvailable else {
print("Recording is not available at this time.")
return
}
self.recorder.isMicrophoneEnabled = true
self.recorder.startRecording{ [unowned self] (error) in
guard error == nil else {
print("There was an error starting the recording.")
return
}
print("Started Recording Successfully")
self.isRecording = true
}
}
private func stopRecording() {
recorder.stopRecording { [unowned self] (preview, error) in
print("Stopped recording")
guard preview != nil else {
print("Preview controller is not available.")
return
}
let alert = UIAlertController(title: "Recording Finished", message: "Would you like to edit or delete your recording?", preferredStyle: .alert)
let deleteAction = UIAlertAction(title: "Delete", style: .destructive, handler: { (action: UIAlertAction) in
self.recorder.discardRecording(handler: { () -> Void in
print("Recording suffessfully deleted.")
})
})
let editAction = UIAlertAction(title: "Edit", style: .default, handler: { (action: UIAlertAction) -> Void in
preview?.previewControllerDelegate = self
self.present(preview!, animated: true, completion: nil)
})
alert.addAction(editAction)
alert.addAction(deleteAction)
self.present(alert, animated: true, completion: nil)
self.isRecording = false
}
}
func previewControllerDidFinish(_ previewController: RPPreviewViewController) {
dismiss(animated: true)
}
}
I expect that the mic should record every single time after allowing the permissions however it appears to only be recording the mic during the sessions in which it asks for those permissions.
This appears to have been fixed in iOS 13. The OS now asks for permission every time you request to record the screen. I still don't have a fix for iOS 12 however.
I have a really simple request that is apparently beyond me. All I want to do is allow the user to select an image from the camera roll and then store a ‘reference’ to that image in my app. I can then load the image from the camera roll when I need it.
I do not want to copy the image and save it elsewhere because I feel that it would be wasting space on the phone. I realise that the user could delete the image from their camera roll, but in the case of this app it does not matter. I will simply check for nil just before displaying the image. It will not affect the app functionality.
I can present the ImagePickerController and I am happily getting a UIImage by means of:
info[UIImagePickerControllerOriginalImage]
What I would like to do is use:
info[UIImagePickerControllerReferenceURL]
However, this is being removed and is no longer supported. I have therefore lost several hours of my life looking into PHAsset. This seems fine for retrieval, providing I have something to search for. I cannot seem to write a simple app that gets an image via the UIImagePickerController and then allows me to save a reference/URL/uniqueID etc. that I can then use via some other method to get the image back again.
I am more than happy to be laughed at, providing the person laughing shows me how silly I have been…
All help greatly appreciated.
I needed to check the PHPhotoLibrary.authorizationStatus() programatically as the app was silently failing this. Adding the following to AppDelegate.swift resolved the issue (you need to import Photos):
func applicationDidBecomeActive(_ application: UIApplication) {
// Restart any tasks that were paused (or not yet started) while the application was inactive. If the application was previously in the background, optionally refresh the user interface.
photoLibraryAvailabilityCheck()
}
//MARK:- PHOTO LIBRARY ACCESS CHECK
func photoLibraryAvailabilityCheck()
{
if PHPhotoLibrary.authorizationStatus() == PHAuthorizationStatus.authorized
{
}
else
{
PHPhotoLibrary.requestAuthorization(requestAuthorizationHandler)
}
}
func requestAuthorizationHandler(status: PHAuthorizationStatus)
{
if PHPhotoLibrary.authorizationStatus() == PHAuthorizationStatus.authorized
{
}
else
{
alertToEncouragePhotoLibraryAccessWhenApplicationStarts()
}
}
//MARK:- CAMERA & GALLERY NOT ALLOWING ACCESS - ALERT
func alertToEncourageCameraAccessWhenApplicationStarts()
{
//Camera not available - Alert
let internetUnavailableAlertController = UIAlertController (title: "Camera Unavailable", message: "Please check to see if it is disconnected or in use by another application", preferredStyle: .alert)
let settingsAction = UIAlertAction(title: "Settings", style: .destructive) { (_) -> Void in
let settingsUrl = NSURL(string:UIApplicationOpenSettingsURLString)
if let url = settingsUrl {
DispatchQueue.main.async {
UIApplication.shared.open(url as URL, options: [:], completionHandler: nil) //(url as URL)
}
}
}
let cancelAction = UIAlertAction(title: "Okay", style: .default, handler: nil)
internetUnavailableAlertController .addAction(settingsAction)
internetUnavailableAlertController .addAction(cancelAction)
self.window?.rootViewController!.present(internetUnavailableAlertController , animated: true, completion: nil)
}
func alertToEncouragePhotoLibraryAccessWhenApplicationStarts()
{
//Photo Library not available - Alert
let cameraUnavailableAlertController = UIAlertController (title: "Photo Library Unavailable", message: "Please check to see if device settings doesn't allow photo library access", preferredStyle: .alert)
let settingsAction = UIAlertAction(title: "Settings", style: .destructive) { (_) -> Void in
let settingsUrl = NSURL(string:UIApplicationOpenSettingsURLString)
if let url = settingsUrl {
UIApplication.shared.open(url as URL, options: [:], completionHandler: nil)
}
}
let cancelAction = UIAlertAction(title: "Okay", style: .default, handler: nil)
cameraUnavailableAlertController .addAction(settingsAction)
cameraUnavailableAlertController .addAction(cancelAction)
self.window?.rootViewController!.present(cameraUnavailableAlertController , animated: true, completion: nil)
}
I would like to record the screen, audio, and video of my app with target Action Extension.
If I put this code in a normal app, it works, but in an Action Extension doesn't.
#IBAction func recButton(_ sender: Any) {
if recButton.currentTitle == "stop" {
stopRecording()
recButton.setTitle("rec", for: .normal)
}
else {
recButton.setTitle("stop", for: .normal)
RPScreenRecorder.shared().isMicrophoneEnabled = true
RPScreenRecorder.shared().startRecording(handler: {[unowned self] (error) in
//Handler - never called
if let unwrappedError = error {
print(unwrappedError.localizedDescription)
}
})
}
}
func stopRecording() {
RPScreenRecorder.shared().stopRecording(handler: {(previewController, error) -> Void in
//Handler - never called
if previewController != nil {
let alertController = UIAlertController(title: "Recording", message: "Do you want to discard or view your recording?", preferredStyle: .alert)
let discardAction = UIAlertAction(title: "Discard", style: .default) { (action: UIAlertAction) in
RPScreenRecorder.shared().discardRecording(handler: { () -> Void in
// Executed once recording has successfully been discarded
})
}
let viewAction = UIAlertAction(title: "View", style: .default, handler: { (action: UIAlertAction) -> Void in
self.present(previewController!, animated: true, completion: nil)
})
alertController.addAction(discardAction)
alertController.addAction(viewAction)
self.present(alertController, animated: true, completion: nil)
} else {
// Handle error
}
})
}
Is there another method to achieve this goal using AVCaptureSession, or do I need to use something else to achieve this? Thanks.
I'm pretty sure Apple won't let you do that by design. When it comes to extensions, they are generally very strict both in terms of what is allowed api-wise and what will pass app review.
Even if you figure out a hacky solution to overcome the issues with ReplayKit, I guess it would get rejected by app review.
In the general App Review Guidelines, the app extension programming guide is referenced as a defining guideline where for action extension it specifically says:
In iOS, an Action extension:
Helps users view the current document in a different way
Always appears in an action sheet or full-screen modal view
Receives selected content only if explicitly provided by the host app
Not quite sure how a screen recording would fit into that pattern in a way that convinces Apple...
I don't think that this is not possible, as I have seen some apps on App Store, which are recording video and Audio within iMessage extension app. Like : SuperMoji App This app records face expression and audio
and sends as a video message within iPhone's Message app only.
However, I am not sure how to do that in Extension apps. I am working on it and will let you know very soon.
Writing an app that requires the network. Run a method to check the WiFi is on and working, initially in the ViewController. So I got ...
wifiWorks = Reachability.isConnectedToNetwork()
This is defined like this ...
var wifiWorks: Bool = false {
didSet {
if wifiWorks {
print("Wifi On")
NotificationCenter.default.post(name: Notification.Name("WifiBon"), object: nil, userInfo: nil)
} else {
print("WiFi Off")
NotificationCenter.default.post(name: Notification.Name("noWiFi"), object: nil, userInfo: nil)
}
}
}
Now if noWifi gets called it shows an alert that basically goes to the settings like this...
func noWifi(notification:NSNotification) {
DispatchQueue.main.async {
//self.navigation.isHidden = true
let alert = UIAlertController(title: "Stop", message: "Your iPad isn't connected to the WiFi ...", preferredStyle: UIAlertControllerStyle.alert)
let settingsAction = UIAlertAction(title: "Settings", style: .default) { (_) -> Void in
guard let settingsUrl = URL(string: UIApplicationOpenSettingsURLString) else {
return
}
if UIApplication.shared.canOpenURL(settingsUrl) {
UIApplication.shared.open(settingsUrl, completionHandler: { (success) in
print("Settings opened: \(success)") // Prints true
})
}
}
alert.addAction(settingsAction)
let cancelAction = UIAlertAction(title: "Cancel", style: .default, handler: nil)
alert.addAction(cancelAction)
self.present(alert, animated: true, completion: nil)
}
}
At which point, my app isn't running any longer, the user is sitting in settings, and has to switch back to my app having (hopefully) turned the WiFi on. Back in my app I got this in the app delegate.
func applicationWillEnterForeground(_ application: UIApplication) {
print("applicationWillEnterForeground")
wifiWorks = Reachability.isConnectedToNetwork()
}
Which brings me back full circle, so you cannot continue with the app, cause it needs the WiFi.
My question, does this dance make sense; or was/is there a cleaner way to do this?
You can use Reachability's own Notifications, as it's GitHub page suggests:
NotificationCenter.default.addObserver(self, selector: #selector(self.reachabilityChanged),name: ReachabilityChangedNotification,object: reachability)
do{
try reachability.startNotifier()
}catch{
print("could not start reachability notifier")
}
Callback:
func reachabilityChanged(note: NSNotification) {
let reachability = note.object as! Reachability
if !reachability.isReachable {
//you can show alert here
}
}
Also be aware that user can turn on/off wifi without leaving your app, just by switching wifi toggle from Control Center. That's why you should not rely only on applicationWillEnterForeground method.
I have audio playing in my app using MPMusicPlayerController and Im recording the screen using RPScreenRecorder. The problem Im having is that it only records the screen and not the audio in the app. The other problem I have is that when I press the cancel button for the previewController it doesnt dismiss the view for some reason. What am I doing wrong?
#IBAction func stopTheRecordingAction(sender: AnyObject) {
stopTheRecording.hidden = true
recordButton.hidden = false
RPScreenRecorder.sharedRecorder().stopRecordingWithHandler { (previewController: RPPreviewViewController?, error: NSError?) -> Void in
if previewController != nil {
let alertController = UIAlertController(title: "Recording", message: "Do you wish to discard or view your gameplay recording?", preferredStyle: .Alert)
let discardAction = UIAlertAction(title: "Discard", style: .Default) { (action: UIAlertAction) in
RPScreenRecorder.sharedRecorder().discardRecordingWithHandler({ () -> Void in
// Executed once recording has successfully been discarded
})
}
let viewAction = UIAlertAction(title: "View", style: .Default, handler: { (action: UIAlertAction) -> Void in
self.presentViewController(previewController!, animated: true, completion: nil)
})
alertController.addAction(discardAction)
alertController.addAction(viewAction)
self.presentViewController(alertController, animated: true, completion: nil)
} else {
// Handle error
}
}
}
#IBAction func recordScreen(sender: AnyObject) {
recordButton.hidden = true
stopTheRecording.hidden = false
if RPScreenRecorder.sharedRecorder().available {
RPScreenRecorder.sharedRecorder().startRecordingWithMicrophoneEnabled(true, handler: { (error: NSError?) -> Void in
if error == nil { // Recording has started
} else {
// Handle error
}
})
} else {
// Display UI for recording being unavailable
}
}
func previewControllerDidFinish(previewController: RPPreviewViewController) {
previewController.dismissViewControllerAnimated(true, completion: nil)
print("dismiss")
}
Okay I got it work but I had to use AVAudioPlayer instead of MPMusicPlayerController. For some reason replaykit doesnt record the audio using MPMedia.