I have project that is rendering video playback and applying CIFilters to it. I know that I can use video composition to get video with filters, but problems is that filters needs to be swipeable (with preview of next filter so we're using mask for 1st imageview and filtering 2nd one with next filter).
func displayLinkDidRefresh(link: CADisplayLink){
let itemTime = videoOutput.itemTime(forHostTime: CACurrentMediaTime())
if videoOutput.hasNewPixelBuffer(forItemTime: itemTime) {
if let pixelBuffer = videoOutput.copyPixelBuffer(forItemTime: itemTime, itemTimeForDisplay: nil){
unfilteredImage = CIImage(cvImageBuffer: pixelBuffer)
displayFilteredImage(unfilteredImage: unfilteredImage)
}
}
}
This is the code used to create AVPlayer instance and CADisplayLink:
player = AVPlayer(playerItem: item)
player.isMuted = true
displayLink = CADisplayLink(target: self, selector: #selector(displayLinkDidRefresh(link:)))
displayLink!.preferredFramesPerSecond = 24
displayLink!.add(to: RunLoop.main, forMode: RunLoopMode.commonModes)
NotificationCenter.default.addObserver(self, selector: #selector(self.playerItemDidReachEnd(notification:)), name: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: self.player.currentItem)
When running from debugger I experience no stutter (or it is at minimum level), but when running build from archive it is stuttering a lot. What I do to test is deleting app from phone and then instal it on the phone and test, as said debug builds are fine, but archive are creating issues. Any input on this?
EDIT 1:
Managed to get it to work better, but still stuttering after attaching video composition to player item when item and player status is readyToPlay.
After some time playing with this I found the issue, nothing is wrong with the code. Issue was that we have Appsee analytics inside app and we needed to pause it on these screens in order for them to render properly. Reason why we didn't experience these issues in debug mode was the AppDelegate if condition that was preventing Appsee to work in debug environment.
Related
I use SCNRenderer with Vuforia. On iOS 13, video content not showing, sound is heard. I tried to set the video material in the following ways:
material.diffuse.contents = AVPlayer
or
material.diffuse.contents = SKVideoNode
But none of these methods work on iOS 13. Unfortunately, I don’t have the opportunity to use SCNView to render a SCNScene, since it is impossible to use with Vuforia. Maybe someone has already encountered this? Or does someone know how to otherwise render the AVPlayer in the SCNScene?
Root Cause:
In my app, we have multiple types of AR modes.
When we switch between these modes with the UISegmentedControls, I have to “reset” the scene by clearing any AR nodes that should not be there. (e.g: previous mode's AR nodes not interfering with the new nodes)
I did this with the following code:
func resetARScene() {
self.node.geometry?.firstMaterial?.normal.contents = nil
self.node.geometry?.firstMaterial?.diffuse.contents = nil
self.node.removeFromParentNode()
self.node = SCNNode()
DispatchQueue.main.async {
self.sceneView.scene.rootNode.addChildNode(self.node)
}
}
The issue arises when in iOS 12, we could re-add the childNode back to the sceneView. Somehow in iOS 13 it doesn’t add properly.
Solution:
func resetARScene() {
self.node.enumerateChildNodes { (existingNode, _) in
existingNode.removeFromParentNode()
}
}
This function achieves the same purpose without removing the node from the sceneView and the video plays. I hope this helps, although I am not sure if your circumstances are the same as mine.
I am making an iOS Radio app with Swift 3! Now it is working and playing all the url address audio streaming well. But when I am in my workplace the Wifi has restrictions for Play audios streaming, so the app just works with Mobile Data.
When I click on Play button it change to Pause and does not send any sounds, It does not play and the console does not show any connectivity error or something like that. I want to know when the firewall or proxy of some WIFI network is blocking my app to send some UIAlert and change the Pause icon to Play ( if i do not do it the user will be waiting for the music).
I am using Swift 3.0 Xcode 8 , AVPlayer to play the audio. I have tried .addPeriodicTimerObserver, I have validated the url and did a lot of things but I could not get the result that I am looking for. Can some one help me?
var urlStreaming:String = "http://someurl"
playerStreaming = AVPlayer(url: URL(string: url)!)
playerLayer = AVPlayerLayer(player: playerStreaming)
playerLayer.frame = CGRect(x:0, y: 0, width:1, height: 10)
self.view.layer.addsublayer(playerLayer)
playerStreaming?.play()
My playerStreaming?.play() is not sending me nothing when It can not play because it starts trying to play but after some seconds it stops and it does not send anything. When I using WIFI connection of my workplace it happens, but If i change it to mobile data I can listening the audio. If i play from another wifi connection for example my house, park or starbuck coffee wifi I can listening it too without problems.
I know that the Wifi connection in my workplace has restriction (blocked by firewall or proxy actually I am not sure. )for audio streaming.
I have been looking for a lot of options and I tried all of them but i had gotten the result that I am looking for.
Can you help me?
I think i found something to check it. What do you thing #zombie?
I checked if the url is valid or how much it takes to play.
P.S. Maybe I will change the Time Out value , now it is taking to much time.
Console shows:
Error Info: -> Error Domain=NSURLErrorDomain Code= -1001 "the request time out" . i think I can validate if my error is null or not. if it is null I can show a UIAlert
let urltest = URL(string: yourul)
let task = URlSession.shared.dataTask(with: urlTest!){
data, response, error in
if(error != nil){
print("Error info: -> \(error!)")
//I think here I can changes the Image if i found a error
}
else{
print("It is going to be play without problem")
}
}
You can listen for error by
NotificationCenter.default.addObserver(self, selector: #selector(playerItemFailedToPlay(_:)), name: NSNotification.Name.AVPlayerItemFailedToPlayToEndTime, object: nil)
NotificationCenter.default.addObserver(self, selector: #selector(playerItemFailedToPlay(_:)), name: NSNotification.Name.AVPlayerItemPlaybackStalled, object: nil)
func playerItemFailedToPlay(_ notification: Notification) {
let error = notification.userInfo?.first(where: { $0.value is Error }) as? Error
}
don't for get to remove the observer
deinit {
NotificationCenter.default.removeObserver(self)
}
I have a WKWebView which is playing HTML5 video. When I click the home button and the app enters the background I can still hear the audio playing. How can I pause the HTML 5 video to stop the background audio from playing. I tried just loading a blank page when the app enters background which works,
let url = URL(string: "about:blank")
let request = URLRequest(url:url!)
self.webView.load(request)
but I want to be able to start the video at the same point when the user returns from the background so it is not a viable solution since it just loads the blank page when I open the app back up.
I was able to achive this by injecting JavaScript when the app entered the background which would pause any video elements that are playing.
NotificationCenter.default.addObserver(self, selector: #selector(enteredBackground(notification:)), name: UIApplication.didEnterBackgroundNotification, object: nil)
#objc func enteredBackground(notification: Notification) {
let script = "var vids = document.getElementsByTagName('video'); for( var i = 0; i < vids.length; i++ ){vids.item(i).pause()}"
self.webView.evaluateJavaScript(script, completionHandler:nil)
}
I have been streaming music from remote source using AVPlayer. I get URLs, use one to create an AVPlayerItem, which i then associate with my instance of AVPlayer. I add an observer to the item that I associate with the player to observe when the item finishes playing ( AVPlayerItemDidPlayToEndTimeNotification ). When the observer notifies me at the item end, I then create a new AVPlayerItem and do it all over again. This works well in the foreground AND in the background on iOS 9.2.
Problem: Since I have updated to iOS 9.3 this does not work in the background. Here is the relevant code:
var portionToBurffer = Double()
var player = AVPlayer()
func prepareAudioPlayer(songNSURL: NSURL, portionOfSongToBuffer: Double){
self.portionToBuffer = portionOfSongToBuffer
//create AVPlayerItem
let createdItem = AVPlayerItem(URL: songNSURL)
//Associate createdItem with AVPlayer
player = AVPlayer(playerItem: createdItem)
//Add item end observer
NSNotificationCenter.defaultCenter().addObserver(self, selector: "playerItemDidReachEnd:", name: AVPlayerItemDidPlayToEndTimeNotification, object: player.currentItem)
//Use KVO to see how much is loaded
player.currentItem?.addObserver(self, forKeyPath: "loadedTimeRanges", options: .New, context: nil)
}
override func observeValueForKeyPath(keyPath: String?, ofObject object: AnyObject?, change: [String : AnyObject]?, context: UnsafeMutablePointer<Void>) {
if keyPath == "loadedTimeRanges" {
if let loadedRangeAsNSValueArray = player.currentItem?.loadedTimeRanges {
let loadedRangeAsCMTimeRange = loadedRangeAsNSValueArray[0].CMTimeRangeValue
let endPointLoaded = CMTimeRangeGetEnd(loadedRangeAsCMTimeRange)
let secondsLoaded = CMTimeGetSeconds(endPointLoaded)
print("the endPointLoaded is \(secondsLoaded) and the duration is \(CMTimeGetSeconds((player.currentItem?.duration)!))")
if secondsLoaded >= portionToBuffer {
player.currentItem?.removeObserver(self, forKeyPath: "loadedTimeRanges")
player.play()
}
}
}
}
func playerItemDidReachEnd(notification: NSNotification){
recievedItemEndNotification()
}
func recievedItemEndNotification() {
//register background task
bgTasker.registerBackgroundTask()
if session.playlistSongIndex == session.playlistSongTitles.count-1 {
session.playlistSongIndex = 0
} else {
session.playlistSongIndex += 1
}
prepareAudioPlayer(songURL: session.songURLs[session.playlistSongIndex], portionOfSongToBuffer: 30.00)
}
I have set breakpoints to see that player.play() IS being called when in the background. When i print player.rate it reads 0.0. I have checked the property playbackLikelyToKeepUp of the AVPlayerItem and it is true. I have confirmed also that the new URL is successfully used to create the new AVPlayerItem and associated with the AVPlayer when the app is in the background. I have turned audio and airplay background capabilities on and I have even opened up a finite length background task (in code above as bgTasker.registerBackgroundTask). No idea what is going on.
I found THIS but i'm not sure it helps. Any advice would be great, thanks
When the observer notifies me at the item end, I then create a new AVPlayerItem and do it all over again
But the problem is that meanwhile play stops, and the rule is that background playing is permitted only so long as you were playing in the foreground and continue to play in the background.
I would suggest using AVQueuePlayer instead of AVPlayer. This will allow you to enqueue the next item while the current item is still playing — and thus, we may hope, this will count as continuing to play.
I encountered the similar problem, and I searched lots of websites on google, but didn't find the answer.
The Phenomenon
The problem of my app is that when I start play an audio, and turn the app to background, it will finish the playing of the current audio, and when playing the second audio, it will load some data, but then it stopped, if I turn the app to foreground, it will play the audio.
Solution
My solution is to add the following call.
UIApplication.shared.beginReceivingRemoteControlEvents()
So enable background audio capabilities is not enough, we need to begin receiving remote controller events.
Edited to explain why its not a duplicate:
I am trying to use MPMusicPlayerController.systemMusicPlayer() to control music playback from my app, but I want to disable the Command Center previous track button. I also want to override the default Command Center play and next track functions. The code should be simple:
This code is in ViewController.swift - viewDidLoad
let commandCenter = MPRemoteCommandCenter.sharedCommandCenter()
commandCenter.previousTrackCommand.enabled = false
commandCenter.previousTrackCommand.addTargetWithHandler({ (commandEvent: MPRemoteCommandEvent!) -> MPRemoteCommandHandlerStatus in
self.empty()
return MPRemoteCommandHandlerStatus.Success
})
//MPRemoteCommandCenter.sharedCommandCenter().previousTrackCommand.addTarget(self, action: "empty")
commandCenter.nextTrackCommand.enabled = true
commandCenter.nextTrackCommand.addTargetWithHandler { (commandEvent: MPRemoteCommandEvent!) -> MPRemoteCommandHandlerStatus in
self.gameOver()
return MPRemoteCommandHandlerStatus.Success
}
commandCenter.playCommand.enabled = true
commandCenter.playCommand.addTargetWithHandler { (commandEvent: MPRemoteCommandEvent!) -> MPRemoteCommandHandlerStatus in
self.playing()
return MPRemoteCommandHandlerStatus.Success
}
Also, in AppDelegate.swift - application
UIApplication.sharedApplication().beginReceivingRemoteControlEvents()
And in the iOS simulator (both iPad and iPhone), it works correctly, as can be seen in the first screenshot (of the simulator).
However, when deploying the app to my iPad, none of the MPRemoteCommandCenter commands work at all, as can be seen in the second screenshot (of an actual device).
This is different from the "dupliate" question (How Do I Get Audio Controls on Lock Screen/Control Center from AVAudioPlayer in Swift) in the following ways:
I am not using AVAudioSession, I am using MPMusicPlayerController.systemMusicPlayer()
I have already called beginReceivingRemoteControlEvents, so that cant be the issue (unless I have somehow called it incorrectly, in which case, I would love an answer explaining how else it should be called).
Thank you.
,