AVPictureInPictureController cannot activate programmatically during a scrolling - ios

I have a video in a cell, if I put it in PIP mode with the button everything works fine but if I do it programmatically when the cell go out of screen doesn't not automatically put in PIP mode, can just activate it via a button?
func scrollViewDidScroll(_ scrollView: UIScrollView) {
guard let cell = self.playerCell, self.playerController != nil else {
return
}
let rect = self.view.convert(cell.frame, from: scrollView)
if rect.origin.y < 0, self.pipController == nil {
self.pipController = self.startPictureInPicture()
}
}
func startPictureInPicture() -> AVPictureInPictureController? {
guard AVPictureInPictureController.isPictureInPictureSupported() else {
return nil
}
let layer = AVPlayerLayer(player: self.playerController.player)
try? AVAudioSession.sharedInstance().setActive(true)
if let pipController = AVPictureInPictureController(playerLayer: layer) {
if pipController.isPictureInPicturePossible {
pipController.startPictureInPicture()
} else {
pipController.addObserver(self, forKeyPath: "isPictureInPicturePossible", options: [.new, .initial], context: nil)
}
}
return nil
}
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
if keyPath == "isPictureInPicturePossible", let pipController = object as? AVPictureInPictureController, pipController.isPictureInPicturePossible {
pipController.startPictureInPicture()
}
}
UPDATE: the debug console always show a warning Unbalanced calls to begin/end appearance transitions for <UIViewController>. but this I have solved with:
DispatchQueue.main.asyncAfter(deadline: .now() + 0.1) {
pipController.startPictureInPicture()
}

As per Apple's guidelines, we should not start PIP mode without user's action.
Only begin Picture in Picture playback in response to user interaction and never programmatically. The App Store review team rejects apps that fail to follow this requirement.
Source: https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_in_a_custom_player
I would suggest you create your own custom PIP view inside the app to avoid rejection.

Related

PayTM All-In-One SDK payment stuck on processing and delegates are not getting called

I was trying to integrate Paytm payment sdk using the All-In-One SDK they provides. I was using the SDK based integration . But after payment is done, it gets stuck on a page with "Processing Payment" text. Even if the payment is done, it is not calling the didFinish delegate function. This issue only occurs when the payment is done using openPaymentWebVC, if we use paytm app directly everything works fine.
self.handler.openPaytm(merchantId: PayTmKeys.mId, orderId: payTmOrderId, txnToken: payTmTxnToken, amount:strPayment, callbackUrl: PayTmKeys.verifyUrl+self.payTmOrderId, delegate: self, environment: .production)
The delegate functions are as follows.
extension CAPaymentGatewayViewController: AIDelegate {
func openPaymentWebVC(_ controller: UIViewController?) {
if let vc = controller {
DispatchQueue.main.async {[weak self] in
self?.present(vc, animated: true, completion: nil)
}
}
}
func didFinish(with status: AIPaymentStatus, response: [String : Any]) {
print("Response: \(response)")
}
}
The callback URL used is
https://securegw.paytm.in/theia/paytmCallback?ORDER_ID=payTmOrderId
Kindly ensure you are sending the call back url in initiate transaction API while generating the txn token.
I have tried a lot to get a proper solution for this, but didnt get any. However I found a workaround solution.
First I looped through the paytm viewcontroller's subview to get the WKWebview and added an observer to know the url change in webview.
func openPaymentWebVC(_ controller: UIViewController?) {
if let vc = controller {
paytmPaymentController = vc
DispatchQueue.main.async {[weak self] in
self?.present(vc, animated: true, completion: {
print(" super view type: \(type(of: vc.view))")
vc.view.loopViewHierarchy { (view, stop) in
if view is WKWebView {
/// use the view
print("This view is WKWebview")
if let wkview = view as? WKWebView {
print("Entered in column")
wkview.addObserver(self!, forKeyPath: "URL", options: .new, context: nil)
}
stop = true
}
}
})
}
}
}
It will get called whenever the url changed in paytm webview. When I get the callback url(processing screen), I dismiss the paytm viewcontroller.
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
if let key = change?[NSKeyValueChangeKey.newKey] as? NSURL {
print(type(of: key))
print("observeValue \(key)") // url value
if key.absoluteString == "https://merchant.com/callback" {
paytmPaymentController?.dismiss(animated: true, completion: nil)
}
}
}
The next step is to verify the payment status using api from your backend using this api

Swift -AVPlayer' KVO in cell's parent vc causing Xcode to freeze

I have a cell that takes up the entire screen so there is only 1 visible cell at a time. Inside the cell I have an AVPlayer. Inside the cell's parent vc I have a KVO observer that listens to the "timeControlStatus". When the player stops playing I call a function stopVideo() inside the cell to stop the player and either show a replay button or a play button etc.
3 problems:
1- When the video stops/reaches end if I don't use DispatchQueue inside the KVO the app crashes when the cell's function is called.
2- When the video stops and I do use DispatchQueue inside the KVO it's observer keeps observing and Xcode freezes (no crash). There is a print statement inside the KVO that prints indefinitely and that's why Xcode freezes. The only way to stop it is to kill Xcode otherwise the beachball of death keeps spinning.
3- In the KVO I tried to use a notification to send to the cell in place of calling the cell.stopVideo() function but the function inside the cell never runs.
How can i fix this issue?
It should be noted that outside of the KVO not working everything else works fine. I have a periodic time observer that runs perfectly for every cell whenever I scroll, the videos load fine, and when I press the cell stop/play the video it all works fine.
cell:
protocol MyCellDelegate: class {
func sendBackPlayerAndIndexPath(_ player: AVPlayer?, currentIndexPath: IndexPath?)
}
var player: AVPlayer?
var indexPath: IndexPath?
var playerItem: AVPlayerItem? {
didSet {
// add playerItem to player
delegate?.sendBackPlayerAndIndexPath(player, indexPath)
}
}
override init(frame: CGRect) {
super.init(frame: frame)
player = AVPlayer()
// set everything else relating to the player
}
// both get initialized in cellForItem
var delegate: MyCellDelegate?
var myModel: MyModel? {
didSet {
let url = URL(string: myModel!.videUrlStr!)
asset = AVAsset(url: url)
playerItem = AVPlayerItem(asset: asset, automaticallyLoadedAssetKeys: ["playable"])
}
}
// tried using this with NotificationCenter but it didn't trigger from parent vc
#objc public func playVideo() {
if !player?.isPlaying {
player?.play()
}
// depending on certain conditions show a mute button, etc
}
// tried using this with NotificationCenter but it didn't trigger from parent vc
#objc public func stopVideo() {
player?.pause()
// depending on certain conditions show a reload button or a play button etc
}
parent vc
MyVC: ViewController, UICollectionViewDataSource, UICollectionViewDelegateFlowLayout {
var player: AVPlayer?
var currentIndexPath: IndexPath?
var isObserving = false
func sendBackPlayerAndIndexPath(_ player: AVPlayer?, currentIndexPath: IndexPath?) {
if isObserving {
self.player?.removeObserver(self, forKeyPath: "status", context: nil)
self.player?.removeObserver(self, forKeyPath: "timeControlStatus", context: nil)
}
guard let p = player, let i = currentIndexPath else { return }
self.player = p
self.currentIndexPath = i
isObserving = true
self.player?.addObserver(self, forKeyPath: "status", options: [.old, .new], context: nil)
self.player?.addObserver(self, forKeyPath: "timeControlStatus", options: [.old, .new], context: nil)
}
// If I don't use DispatchQueue below the app crashes
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
if object as AnyObject? === player {
if keyPath == "status" {
if player?.status == .readyToPlay {
DispatchQueue.main.async { [weak self] in
self?.playVideoInCell()
}
}
}
} else if keyPath == "timeControlStatus" {
if player?.timeControlStatus == .playing {
DispatchQueue.main.async { [weak self] in
self?.playVideoInCell()
}
} else {
print("3. Player is Not Playing *** ONCE STOPPED THIS PRINTS FOREVER and Xcode freezes but doesn't crash.\n")
DispatchQueue.main.async { [weak self] in
self?.stopVideoInCell()
}
}
}
}
}
func playVideoInCell() {
guard let indexPath = currentIndexPath else { return }
guard let cell = collectionView.cellForItem(at: indexPath) as? MyCell else { return }
cell.playVideo()
// also tried sending a NotificationCenter message to the cell but it didn't trigger
}
func stopVideoInCell() {
guard let indexPath = currentIndexPath else { return }
guard let cell = collectionView.cellForItem(at: indexPath) as? MyCell else { return }
cell.stopVideo()
// also tried sending a NotificationCenter message to the cell but it didn't trigger
}
In the comments #matt asked for the crash log (only occurs when not using DispatchQueue inside the KVO). I have zombies enabled and it didn't give me any info. The crash happens instantaneously and then just goes blank. It doesn't give me any info. I had to quickly take a screen shot just to get the pic otherwise it disappears right after the crash.
Inside the KVO, the 3rd one that prints forever, I removed the DispatchQueue and just added the stopVideoInCell() function. When the function calls the cell's stopVideo() function, player?.pause briefly gets a EXC_BAD_ACCESS (code=2, address=0x16b01ff0) crash.
The app then terminates. Inside the console the only thing that is printed is:
Message from debugger: The LLDB RPC server has crashed. The crash log
is located in ~/Library/Logs/DiagnosticReports and has a prefix
'lldb-rpc-server'. Please file a bug and attach the most recent crash
log
When I go to terminal and see what prints out the only thing I get is a bunch of lldb-rpc-server_2020-06-14-155514_myMacName.crash statements from all the days I ran into this crash.
When using DispatchQueue this doesn't occur and the video does stop but of course that print statement inside the KVO runs forever and Xcode freezes.
The problem is that, in your property observer, you are making a change in the property that you are observing. That is a vicious circle, an infinite recursion; Xcode displays this by freezing your app until ultimately you crash with (oh, the irony) a stack overflow.
Let's take a simpler, self-contained example. We have a UISwitch in the interface. It's On. If the user switches it Off, we want to detect that and switch it back to On. The example is a silly way to do this, but it perfectly illustrates the issue you are facing:
class ViewController: UIViewController {
#IBOutlet weak var theSwitch: UISwitch!
class SwitchHelper: NSObject {
#objc dynamic var switchState : Bool = true
}
let switchHelper = SwitchHelper()
var observer: NSKeyValueObservation!
override func viewDidLoad() {
super.viewDidLoad()
self.observer = self.switchHelper.observe(\.switchState, options: .new) {
helper, change in
self.theSwitch.isOn.toggle()
self.theSwitch.sendActions(for: .valueChanged)
}
}
#IBAction func doSwitch(_ sender: Any) {
self.switchHelper.switchState = (sender as! UISwitch).isOn
}
}
What's going to happen? The user turns the switch Off. We are observing that, as switchState; in response, we switch the switch back to On and call sendActions. And sendActions changes switchState. But we are still in the middle of the code that observes switchState! So we do it again and it happens again. And again and it happens again. Infinite loop...
How would you get out of this? You need to break the recursion somehow. I can think of two obvious ways. One is to think to yourself, "Well, I only care about a switch from On to Off. I don't care about the other way." Assuming that's true, you can solve the problem with a simple if, rather like the solution you elected to use:
self.observer = self.switchHelper.observe(\.switchState, options: .new) {
helper, change in
if let val = change.newValue, !val {
self.theSwitch.isOn.toggle()
self.theSwitch.sendActions(for: .valueChanged)
}
}
A more elaborate solution, which I like to use sometimes, is to stop observing when the observer is triggered, make whatever the change is, and then start observing again. You have to plan ahead a little to implement that, but it's sometimes worth it:
var observer: NSKeyValueObservation!
func startObserving() {
self.observer = self.switchHelper.observe(\.switchState, options: .new) {
helper, change in
self.observer?.invalidate()
self.observer = nil
self.theSwitch.isOn.toggle()
self.theSwitch.sendActions(for: .valueChanged)
self.startObserving()
}
}
override func viewDidLoad() {
super.viewDidLoad()
self.startObserving()
}
That looks recursive, because startObserving calls itself, but it isn't really, because what it does when it is called is to configure the observation; the code in the inner curly braces doesn't run until we get an observed change.
(In real life, I would probably make the NSKeyValueObservation a local variable in that configuration. But that's just an additional bit of elegance, not essential to the point of the example.)
I resolved this using a Boolean. It's not the most elegant answer but it works. If someone can come up with a better answer I'll accept it. It doesn't make sense what's going because of what I put under more:
answer:
var isPlayerStopped = false
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
if object as AnyObject? === player {
if keyPath == "status" {
if player?.status == .readyToPlay {
DispatchQueue.main.async { [weak self] in
self?.playVideoInCell()
}
}
}
} else if keyPath == "timeControlStatus" {
if player?.timeControlStatus == .playing {
DispatchQueue.main.async { [weak self] in
self?.playVideoInCell()
}
} else {
if isPlayerStopped { return }
print("3. Player is Not Playing *** NOW THIS ONLY PRINTS ONCE.\n")
DispatchQueue.main.async { [weak self] in
self?.stopVideoInCell()
}
}
}
}
}
func playVideoInCell() {
guard let indexPath = currentIndexPath else { return }
guard let cell = collectionView.cellForItem(at: indexPath) as? MyCell else { return }
isPlayerStopped = false
cell.playVideo()
}
func stopVideoInCell() {
guard let indexPath = currentIndexPath else { return }
guard let cell = collectionView.cellForItem(at: indexPath) as? MyCell else { return }
isPlayerStopped = true
cell.stopVideo()
}
more:
If I completely remove the DispatchQueues and the functions inside of them and use just print statements, the print statement that prints indefinitely print("3. Player is Not Playing... \n") only prints twice, it no longer prints indefinitely so I don't know what's going on with this.
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
if object as AnyObject? === player {
if keyPath == "status" {
if player?.status == .readyToPlay {
print("1. Player is Playing\n")
}
} else if keyPath == "timeControlStatus" {
if player?.timeControlStatus == .playing {
print("2. Player is Playing\n")
} else {
print("3. Player is Not Playing... \n")
}
}
}
}

Why does observer for status change of AVAsset not work?

I implemented the following code from Apple. It is meant to observe for a change in the status of a playerItem. The problem is that for some reason it does not work. The observe function does not run when ready.
All relevant code is below:
func preloadVideo(media: Media) {
//setup code, and then:
media.playerItem1!.addObserver(self,
forKeyPath: #keyPath(AVPlayerItem.status),
options: [.old, .new],
context: &playerItemContext)
}
Observe method:
private var playerItemContext = 0
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
// Only handle observations for the playerItemContext
print("jdslfhjkfdhaldfahjkflhajfldashkjfdshkjlas")
guard context == &playerItemContext else {
super.observeValue(forKeyPath: keyPath, of: object,change: change, context: context)
return
}
if keyPath == #keyPath(AVPlayerItem.status) {
let status: AVPlayerItem.Status
// Get the status change from the change dictionary
if let statusNumber = change?[.newKey] as? NSNumber {
status = AVPlayerItem.Status(rawValue: statusNumber.intValue)!
} else {
status = .unknown
}
print("dsfdgddgdggdgdgdgddsafdsaf434fdfdg433444343")
// Switch over the status
switch status {
case .readyToPlay:
//Stop animation, and go to p3
print("dsfdsafdsaf434433444343")
if goToVideo {
print("Runinfdsh sahkbdsfhbsadfbadsfl")
goToP3()
}
break
// Player item is ready to play.
case .failed: break
// Player item failed. See error.
case .unknown: break
// Player item is not yet ready.
#unknown default: break
//fatal error
}
}
}
This method will run first:
#objc func viewHandleTap(_ sender: UITapGestureRecognizer) {
if selectedPost?.interimMedia.first?.imageURLString != nil || selectedPost?.interimMedia.first!.playerQueue?.status.rawValue == 1 {
//do something
} else {
//status is 0 (video has not loaded yet)//load animation
//this runs...
goToVideo = true
}
}
How can I make it work? Currently nothing in the observe method is printing.
If there is another method to accomplish this go ahead and post as answer.
Try by updating addObserver options to [.old, .new, .initial, .prior].

Key value observing exposureDuration

I'm building a camera app, and I'm trying to expose the current exposure duration to the user. Since this value constantly changes until manually set, I need to use kvo to stream the values to the user. I've successfully done this with the ISO, and can observe changes to the exposureDuration, but cannot coerce the new value to a CMTime object (which is what exposureDuration is). Below is the code I'm using to try and accomplish this:
override init() {
super.init()
captureDevice = self.selectCamera()
captureDevice?.addObserver(self, forKeyPath: "ISO", options: .New, context: &isoContext)
captureDevice?.addObserver(self, forKeyPath: "exposureDuration", options: .New, context: &shutterContext)
}
deinit {
captureDevice?.removeObserver(self, forKeyPath: "ISO")
captureDevice?.removeObserver(self, forKeyPath: "exposureDuration")
}
override func observeValueForKeyPath(keyPath: String?, ofObject object: AnyObject?, change: [String : AnyObject]?, context: UnsafeMutablePointer<Void>) {
let newValue = change?[NSKeyValueChangeNewKey]
if context == &isoContext {
store.iso.value = newValue as! Float
} else if context == &shutterContext {
// The app crashes at this line.
// Thread 1: EXC_BREAKPOINT (code=1, subcode=0x100091670)
// newValue is "AnyObject" in the debug area
store.shutterSpeed.value = newValue as! CMTime
}
}
Am I doing something wrong, or is this a legitimate bug that I need to file with apple?
exposureDuration's newValue is not CMTime, but NSValue.
This is fixed code(swift3).
store.shutterSpeed.value = (newValue as! NSValue).timeValue

iOS Native Camera AVFoundation brightness exposure difference

AVFoundation brightness/exposure is different than the native camera. Is it possible to replicate the native camera settings with the AVFoundation framework?
auto-exposure in Swift 4.2
// MARK: Exposure Methods
#objc
func tapToExpose(recognizer: UIGestureRecognizer){
if activeInput.device.isExposurePointOfInterestSupported{
let point = recognizer.location(in: camPreview)
// The tap location is converted from the `GestureRecognizer` to the preview's coordinates.
let pointOfInterest = previewLayer.captureDevicePointConverted(fromLayerPoint: point)
// Then , the second conversion is made for the coordinate space of the camera.
showMarkerAtPoint(point: point, marker: exposureMarker)
exposeAtPoint(pointOfInterest)
}
}
func exposeAtPoint(_ point: CGPoint){
let device = activeInput.device
if device.isExposurePointOfInterestSupported, device.isFocusModeSupported(.continuousAutoFocus){
do{
try device.lockForConfiguration()
device.exposurePointOfInterest = point
device.exposureMode = .continuousAutoExposure
if device.isFocusModeSupported(.locked){
// Now let us add the illumination for the `observeValueForKeyPath` method,
device.addObserver(self, forKeyPath: kExposure, options: .new, context: &adjustingExposureContext)
device.unlockForConfiguration()
}
}
catch{
print("Error Exposing on POI: \(String(describing: error.localizedDescription))")
}
}
}
// MARK: KVO
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
// First , check to make sure that the context matches the adjusting exposure context,
// Otherwise, pass the observation on to super.
if context == &adjustingExposureContext {
let device = object as! AVCaptureDevice
// then determine if the camera has stopped adjusting exposure , and if the locked mode is supported.
if !device.isAdjustingExposure , device.isExposureModeSupported(.locked){
// remove self from the observer to stop subsequent notification,
device.removeObserver(self, forKeyPath: kExposure, context: &adjustingExposureContext)
DispatchQueue.main.async {
do{
try device.lockForConfiguration()
device.exposureMode = .locked
device.unlockForConfiguration()
}
catch{
print("Error exposing on POI: \(String(describing: error.localizedDescription))")
}
}// DispatchQueue.main.async
}// if !device.isAdjustingExposure , device.isExposureModeSupported(.locked)
}// if context == &adjustingExposureContext {
else{
super.observeValue(forKeyPath: keyPath, of: object, change: change, context: context)
}
}
If you're looking for auto-exposure, set a key-value observer for "adjustingExposure" and set the exposure mode to AVCaptureExposureModeContinuousAutoExposure. If you're trying to implement manual exposure, have a look at this WWDC session on camera controls.

Resources