I'm trying to use ReplayKit to create a live broadcast inside of my app.
Basically I want to share my screen and see the screen of the other user.
To get the buffers, ReplayKit offers the next function:
func startCapture(handler captureHandler: ((CMSampleBuffer, RPSampleBufferType, Error?) -> Void)?, completionHandler: ((Error?) -> Void)? = nil)
So this is my method to startup ReplayKit and get the buffers:
private func startRecording() {
RPScreenRecorder.shared().startCapture(handler: { (sampleBuffer, bufferType, error) in
switch bufferType {
case RPSampleBufferType.video:
// Handle buffer and send it to server
break
case RPSampleBufferType.audioApp:
break
case RPSampleBufferType.audioMic:
break
}
}, completionHandler: nil)
}
This works perfectly, but I'm facing the next issue; if the screen gets changes constantly, like a flashing button, when I send the app to background and come back several times ReplayKit stops calling its capture handler.
Maybe the problem is that the function startCapture is made to record the screen for a limited time, and not for a live broadcast.
I have made an example in Github with a flashing button that shows the problem that I'm having; ReplayKit runs normally until going to background repeatedly; then it stops, and the only way to make it work again is to reboot the device.
Related
I am using ReplayKit to record the screen. However, I have a very strange issue with having video buffers. When user taps record button, the following method is called;
rpScreenRecorder.startCapture { (sampleBuffer, bufferType, error) in
if let error = error {
print(error.localizedDescription)
}
switch bufferType {
case .video:
print("video")
case .audioApp:
print("audioApp")
case .audioMic:
print("audioMic")
#unknown default:
print("default")
}
} completionHandler: { (error) in
if let error = error {
print(error.localizedDescription)
}
}
The problem is, bufferType does not have video. When I print all cases, audioMic and audioApp is returned forever but video is never returned.
I can easily repeat this problem only on the first launch when I remove the app and install it again. It works as expected in other cases.
p.s: I had a look at other issues before asking this question. None of them solved the problem.
After spending days and nights to solve this issue, I finally came up with a solution.
Simply, the reason was about the main window
Main window is important for replay kit because it is recording the main window.
I was using different windows to hide some views from the recorded video. I also changed windowLevel of the main window. Today, I noticed that removing every adjustment related to main window solved the problem. (other windows have no effect.)
When I want to pause the session, this was the only solution for me:
func pauseSession () {
self.sessionQueue.async {
if self.session.isRunning {
self.session.stopRunning()
}
}
}
func resumeSession () {
self.sessionQueue.async {
if !self.session.isRunning {
self.session.startRunning()
}
}
}
This seems to completely stop the session, which is fine, yet looks expensive.
The issue I seem to have is if pause and resume are called near each other in time, the whole app freezes for about 10 seconds, till going back to being responsive. This is mostly due to it still hasn't finished the last process (whether to stop or start).
Is there a solution to this?
The native camera app seems to do this fine. If you open it, open the last photo, you can see the green indicator on the top right going off, meaning the session has paused/stopped. If you swipe down on the photo, the session resumes. If you swipe and let it get canceled, quickly swipe again you can see the session pauses and resumes quickly over and over without any issues.
You may need to change async to sync
self.sessionQueue.sync {
I have a watchOS 4 app which displays SpriteKit animations (SKActions) on top of the UI. Everything works fine in simulator and also on device first couple of times, then after some time when app is in background, and it is started, animations just freeze and completion block for the most long-lasting animation is not called. Any idea what might be the issue?
This is how I run my actions, caller is waiting for completion closure in order to hide the spritekit scene:
private func runActions(with icon: SKShapeNode?, completion: #escaping () -> Void) {
if let icon = icon, let scaleAction = scaleAction, let bg = background {
self.label?.run(fadeInOutAction)
icon.run(scaleAction)
icon.run(fadeInOutAction)
bg.run(backgroundAction, completion: completion)
} else {
completion()
}
}
And yes, I am aware that SKScene is paused when app moves to background. I am doing this in willActivate of my InterfaceController:
if scene.scene?.isPaused == true {
scene.scene?.isPaused = false
}
I want to emphasize that this works first always. It begins to fail after the app has been backgrounded for some time. Especially if I start the app from complication and try to immediately fire these animations, then this freezing happens.
Can I answer my own question? I guess I can? Here goes:
I finally solved this. It turns out that the WKInterfaceScene in WatchKit has ALSO an isPaused property that you need to turn false sometimes. So now in willActivate of my InterfaceController I will also check that and turn it false if it is true. Since I made this change, I haven't seen a single hiccup, freeze or anything weird anymore.
Case closed, I guess. I leave this here for future generations who might face this issue.
I Have an app that uses AVPlayer() to play a music stream from the web. I have everything setup to play in background but when a call comes in (or any other interrupt) the playback will not resume after the interrupt is over. To be specific if the interrupt is dissmised pretty quickly playback will resume but if I have a long phone call for example playback will not resume.
This only happens if my app is in the background as well. If my app is in the foreground when an interrupt comes in everything works.
I have my interrupt notification set up as:
func playInterrupt(notification: NSNotification) {
var info = notification.userInfo!
var intValue: UInt = 0
(info[AVAudioSessionInterruptionTypeKey] as! NSValue).getValue(&intValue)
if let type = AVAudioSessionInterruptionType(rawValue: intValue) {
switch type {
case .began:
print("began")
pause()
case .ended:
print("ended")
play()
}
}
Using
AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, with: AVAudioSessionCategoryOptions.mixWithOthers )
Fixes the problem but then if I start playing from another audio source, they both overlay each other. Ideally I need my app to stop playback in that situation.
Any suggestions on how to achieve this?
Thanks in advance
I want to implement image sync in my ios app. for this what i have done is
take a photo
upload that photo to firebase
get the firebase stored url and send it to an api server so that server stores that info to a database
It works ok as long as the app is running, but when I exit out from the app pressing home button everything pauses.
So how can run a code that will not pause if app goes to home page?
As per this documentation,
For tasks that require more execution time to implement, you must
request specific permissions to run them in the background without
their being suspended. In iOS, only specific app types are allowed to
run in the background:
Apps that play audible content to the user while in the background,
such as a music player app
Apps that record audio content while in the
background Apps that keep users informed of their location at all
times, such as a navigation app
Apps that support Voice over Internet
Protocol (VoIP) Apps that need to download and process new content
regularly
Apps that receive regular updates from external accessories
Apps that implement these services must declare the services they
support and use system frameworks to implement the relevant aspects of
those services
Declaring the services lets the system know which
services you use, but in some cases it is the system frameworks that
actually prevent your application from being suspended.
Link: https://developer.apple.com/library/content/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/BackgroundExecution/BackgroundExecution.html
Here is how to implement (taken from this answer by Ashley Mills):
func doUpdate () {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), {
let taskID = beginBackgroundUpdateTask()
var response: NSURLResponse?, error: NSError?, request: NSURLRequest?
let data = NSURLConnection.sendSynchronousRequest(request, returningResponse: &response, error: &error)
// Do something with the result
endBackgroundUpdateTask(taskID)
})
}
func beginBackgroundUpdateTask() -> UIBackgroundTaskIdentifier {
return UIApplication.sharedApplication().beginBackgroundTaskWithExpirationHandler({})
}
func endBackgroundUpdateTask(taskID: UIBackgroundTaskIdentifier) {
UIApplication.sharedApplication().endBackgroundTask(taskID)
}
[Swift3] This is kind of worked for me
func doUpdate () {
DispatchQueue.global(qos: .background).async {
let taskID = self.beginBackgroundUpdateTask()
DispatchQueue.main.asyncAfter(deadline: .now() + .seconds(5), execute: {
print("printing after 10 min")
})
DispatchQueue.main.async {
self.endBackgroundUpdateTask(taskID: taskID)
}
}
}
func beginBackgroundUpdateTask() -> UIBackgroundTaskIdentifier {
return UIApplication.shared.beginBackgroundTask(expirationHandler: {})
}
func endBackgroundUpdateTask(taskID: UIBackgroundTaskIdentifier) {
UIApplication.shared.endBackgroundTask(taskID)
}
Not sure what is the max time for this to complete since i see there is a expirationHandler