I am coding a video player with Xamarin, AVPlayer and AvPlayerViewController. My code supports AirPlay and configures AVPlayer according docs but is not working quite well. The following scenario seems to fail:
Play a video, send it to Apple TV.
Exit playback as it goes on the Apple TV
Try to resume playback from the current position. This leads to playback directly starting on the Apple TV but the seek which I do to resume from the last playback position kind of fails. What happens is that AvPlayerViewController UI shows that playback starts from the beginning while the Apple TV is playing back from the resume point. The AvPlayerViewController play/pause button is also wrong most of the time and is not reflecting the right play state.
I do the seek so I resume the video like this: wait ( by KVO ) for the avplayer status to become ready to play, seek and then play the video.
Note ... all this works just fine when player is not in airplay mode. Video resumes properly and UI is looking correct.
Code looks like this:
avItem = new AVPlayerItem(avAsset);
avPlayer = new AVPlayer(avItem)
{
AllowsExternalPlayback = true,
UsesExternalPlaybackWhileExternalScreenIsActive = true
};
avPlayerViewController = new AVPlayerViewController()
{
View =
{
ContentMode = UIViewContentMode.ScaleAspectFill,
AutoresizingMask = UIViewAutoresizing.All,
},
UpdatesNowPlayingInfoCenter = false,
Player = avPlayer
};
initialPlaybackTime = TimeSpan.FromSeconds(bookmarkTime);
AddChildViewController(avPlayerViewController);
View.AddSubview(avPlayerViewController.View);
avPlayerViewController.DidMoveToParentViewController(this);
avItem.AddObserver(this, new NSString("status"), NSKeyValueObservingOptions.Initial, IntPtr.Zero);
public override async void ObserveValue(NSString keyPath, NSObject ofObject, NSDictionary change, IntPtr context)
{
if (Equals(ofObject, avItem) && keyPath.Equals((NSString)"status"))
{
if (!initialPlaybackTimeSet && (avPlayer.Status == AVPlayerStatus.ReadyToPlay))
{
await avPlayer.SeekAsync(CMTime.FromSeconds(initialPlaybackTime.TotalSeconds, avPlayer.CurrentItem.Asset.Duration.TimeScale));
avPlayer.Play();
initialPlaybackTimeSet = true;
}
}
}
Related
I have a Xamarin iOS app that I want to stream videos from an API endpoint that supports HTTP range requests. I've reviewed many similar questions here on SO, but I cannot seem to get the AVPlayer to start playing the video file before it is downloaded fully no matter what I try.
I've tried:
KVO on playbackLikelyToKeepUp, playbackBufferEmpty and playbackBufferEmpty to play the video as soon as it is ReadyToPlay
set AutomaticallyWaitsToMinimizeStalling = false on the AVPlayer
set CanUseNetworkResourcesForLiveStreamingWhilePaused = true, and PreferredForwardBufferDuration = 1 on the AVPlayerItem
called PlayImmediatelyAtRate(1) on the AVPlayer
But still the file is downloaded fully before the video starts to play, which causes a delay for the user.
Is it possible to get AVPlayer to start playing a video file before it has completed downloading it, similar to how the HTML video tag does it?
Here is my current code:
private void SetUpPlayer()
{
if (ViewModel.VideoStreamUrl == null)
{
return;
}
// See https://stackoverflow.com/questions/38865797/how-to-play-video-with-avplayerviewcontroller-avkit-in-xamarin-ios
_aVPlayerItem = new AVPlayerItem(ViewModel.VideoStreamUrl)
{
CanUseNetworkResourcesForLiveStreamingWhilePaused = true,
PreferredForwardBufferDuration = 1,
};
// See https://stackoverflow.com/questions/38867190/how-can-i-check-if-my-avplayer-is-buffering/38867386#38867386
_playbackLikelyToKeepUpObserver?.Dispose();
_playbackLikelyToKeepUpObserver = (NSObject)_aVPlayerItem.AddObserver("playbackLikelyToKeepUp",
NSKeyValueObservingOptions.New,
AVPlayerItem_BufferUpdated);
_playbackBufferEmptyObserver?.Dispose();
_playbackBufferEmptyObserver = (NSObject)_aVPlayerItem.AddObserver("playbackBufferEmpty",
NSKeyValueObservingOptions.New,
AVPlayerItem_BufferUpdated);
_playbackBufferFullObserver?.Dispose();
_playbackBufferFullObserver = (NSObject)_aVPlayerItem.AddObserver("playbackBufferFull",
NSKeyValueObservingOptions.New,
AVPlayerItem_BufferUpdated);
_aVPlayer = new AVPlayer(_aVPlayerItem)
{
AutomaticallyWaitsToMinimizeStalling = false,
};
var playerViewController = new AVPlayerViewController
{
Player = _aVPlayer,
};
AddChildViewController(playerViewController);
View.AddSubview(playerViewController.View);
playerViewController.View.Frame = View.Frame;
playerViewController.ShowsPlaybackControls = true;
_aVPlayer.PlayImmediatelyAtRate(1);
}
private void AVPlayerItem_BufferUpdated(NSObservedChange obj)
{
ReportVideoBuffering();
}
private void ReportVideoBuffering()
{
bool isBufferEmpty = _aVPlayerItem != null && _aVPlayerItem.PlaybackBufferEmpty;
Console.WriteLine($"Buffer empty? {isBufferEmpty}");
Console.WriteLine($"Player status? {_aVPlayer.Status}");
if (_aVPlayer.Status == AVPlayerStatus.ReadyToPlay)
{
Console.WriteLine($"Playing video.");
_aVPlayer.Play();
}
}
The simple answer to the question is no, the AVPlayer doesn't support streaming using http range requests and partial content (206) responses. In our case, we have decided to use Azure Media Services to provide a streaming endpoint which we can then use in the iPad app as well as on the web.
following the instructions from:
https://codingwithjoe.com/playing-audio-from-the-web-and-http/ (using his example and AudioProvider class)
I have been using:
audioplayer: ^0.8.1
audioplayer_web: ^0.7.1
to play audio from https link.
The problem it has some weird inconsistent effect.
- after playing few audio, it keeps on playing the same audio eventhough new url is loaded
- after playing few audio, the sound is weird like some part is cut with other audio.
What is a good audio player that accepts url link for flutter that can produce a consistent result ?
the provided audioplayer from your example works great and has good features. From what you are describing for me it seems like you're not closing the session when you play a sound. It seems like you are stacking the sounds which causes weird sounds.
You have to close the instance then. even though the article is outdated (march 2018) the audioplayer has developed further. check their offical guide here:
https://pub.dev/packages/audioplayer
This is version audioplayer 0.8.1 not 3.0 or something..
Example from docs:
Instantiate an AudioPlayer instance
//...
AudioPlayer audioPlugin = AudioPlayer();
//...
Player controls:
audioPlayer.play(url);
audioPlayer.pause();
audioPlayer.stop();
status and current position:
//...
_positionSubscription = audioPlayer.onAudioPositionChanged.listen(
(p) => setState(() => position = p)
);
_audioPlayerStateSubscription = audioPlayer.onPlayerStateChanged.listen((s) {
if (s == AudioPlayerState.PLAYING) {
setState(() => duration = audioPlayer.duration);
} else if (s == AudioPlayerState.STOPPED) {
onComplete();
setState(() {
position = duration;
});
}
}, onError: (msg) {
setState(() {
playerState = PlayerState.stopped;
duration = new Duration(seconds: 0);
position = new Duration(seconds: 0);
});
});
Like I said most of the audio plugins are running in singleton mode with instances. To provide getting weird effects you have to load the next song in the same instance, don't open another new instance, and you wont get any weird effects.
If you want to switch to a different audio player another great one which I used in an app project is the following:
https://pub.dev/packages/audio_manager#-readme-tab-
Hope it helps.
I am testing this using iOS 10.2 on my actual iPhone 6s device.
I am playing streamed audio and am able to play/pause audio, skip tracks, etc. I also have enabled background modes and the audio plays in the background and continues through a playlist properly. The only issue I am having is getting the lock screen controls to show up. Nothing displays at all...
In viewDidLoad() of my MainViewController, right when my app launches, I call this...
func setupAudioSession(){
UIApplication.shared.beginReceivingRemoteControlEvents()
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, with: AVAudioSessionCategoryOptions.mixWithOthers)
self.becomeFirstResponder()
do {
try AVAudioSession.sharedInstance().setActive(true)
print("AVAudioSession is Active")
} catch let error as NSError {
print(error.localizedDescription)
}
} catch let error as NSError {
print(error.localizedDescription)
}
}
and then in my AudioPlayer class after I begin playing audio I call ...
func setupLockScreen(){
let commandCenter = MPRemoteCommandCenter.shared()
commandCenter.nextTrackCommand.isEnabled = true
commandCenter.nextTrackCommand.addTarget(self, action:#selector(skipTrack))
MPNowPlayingInfoCenter.default().nowPlayingInfo = [MPMediaItemPropertyTitle: "TESTING"]
}
When I lock my iPhone and then tap the power button again to go to the lock screen, the audio controls are not displayed at all. It is as if no audio is playing, I just see my normal background photo. Also no controls are displayed in the control panel (swiping up on home screen and then swiping left to where the music controls should be).
Is the issue because I am not using AVAudioPlayer or AVPlayer? But then how does, for example, Spotify get the lock screen controls to display using their own custom audio player? Thanks for any advice / help
The issue turned out to be this line...
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, with: AVAudioSessionCategoryOptions.duckOthers)
Once I changed it to
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, with: [])
everything worked fine. So it seems that passing in any argument for AVAudioSessionCategoryPlaybackOptions causes the lock screen controls to not display. I also tried passing in .mixWithOthers an that too caused the lock screen controls to not be displayed
In Swift 4. This example is only to show the player on the lock screen and works with iOS 11. To know how to play auidio on the device you can follow this thread https://stackoverflow.com/a/47710809/1283517
import MediaPlayer
import AVFoundation
Declare player
var player : AVPlayer?
Now create a function
func setupLockScreen(){
let commandCenter = MPRemoteCommandCenter.shared()
commandCenter.nextTrackCommand.isEnabled = true
commandCenter.togglePlayPauseCommand.addTarget(self, action: #selector(controlPause))
MPNowPlayingInfoCenter.default().nowPlayingInfo = [MPMediaItemPropertyTitle: currentStation]
}
now create a function for control play and pause event. I have created a BOOL "isPlaying" to determine the status of the player.
#objc func controlPause() {
if isPlaying == true {
player?.pause()
isPlaying = false
} else {
player?.play()
isPlaying = true
}
}
And ready. Now the player will be displayed on the lock screen
Yes, for the lock screen to work you need to use iOS APIs to play audio. Not sure how Spotify does it but they may be using a second audio session in parallel for this purpose and use the controls to control both. Your background handler (the singleton in my case) could start playing the second audio with 0 volume when it goes into background and stop it when in foreground. I haven't tested it myself but an option to try.
I have an app that needs to have:
Background music playing while using the app (eg. spotify)
Background music playing while watching movie from AVPlayer
Stop the music when recording a video
Like Snapchat, the camera-viewcontroller is part of a "swipeview" and therefore always on.
However, when opening and closing the app, the music makes a short "crack" noise/sound that ruins the music.
I recorded it here:
https://soundcloud.com/morten-stulen/hacky-sound-ios
(3 occurrences)
I use these settings for changing the AVAudiosession in the appdelegate didFinishLaunchingWithOptions:
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord,withOptions:
[AVAudioSessionCategoryOptions.MixWithOthers,
AVAudioSessionCategoryOptions.DefaultToSpeaker])
try AVAudioSession.sharedInstance().setActive(true)
} catch {
print("error")
}
I use the LLSimpleCamera control for video recording and I've set the session there to:
_session.automaticallyConfiguresApplicationAudioSession = NO;
It seems others have the same problem with other camera libraries as well:
https://github.com/rFlex/SCRecorder/issues/127
https://github.com/rFlex/SCRecorder/issues/224
This guy removed the audioDeviceInput, but I kinda need that for recording video.
https://github.com/omergul123/LLSimpleCamera/issues/48
I also tried with Apple's code "AvCam", and I still have the same issue. How does Snapchat do this?!
Any help would be greatly appreciated, and I'll gladly provide more info or code!
I do something similar to what you're wanting, but without the camera aspect, but I think this will do what you want. My app allows background audio that will mix with non-fullscreen video/audio. When the user plays an audio file or a full screen video file, I stop the background audio completely.
The reason I do SoloAmbient then Playback is because I allow my audio to be played in the background when the device is locked. Going SoloAmbient will stop all background music playing and then switching to Playback lets my audio play in the app as well as in the background.
This is why you see a call to a method that sets the lock screen information in the Unload method. In this case, it is nulling it out so that there is no lock screen info.
In AppDelegate.swift
//MARK: Audio Session Mixing
func allowBackgroundAudio()
{
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, withOptions: .MixWithOthers)
} catch {
NSLog("AVAudioSession SetCategory - Playback:MixWithOthers failed")
}
}
func preventBackgroundAudio()
{
do {
//Ask for Solo Ambient to prevent any background audio playing, then change to normal Playback so we can play while locked
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategorySoloAmbient)
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
} catch {
NSLog("AVAudioSession SetCategory - SoloAmbient failed")
}
}
When I want to stop background audio, for example when playing an audio track that should be alone, I do the following:
In MyAudioPlayer.swift
func playUrl(url: NSURL?, backgroundImageUrl: NSURL?, title: String, subtitle: String)
{
ForgeHelper.appDelegate().preventBackgroundAudio()
if _mediaPlayer == nil {
self._mediaPlayer = MediaPlayer()
_mediaPlayer!.delegate = self
}
//... Code removed for brevity
}
And when I'm done with my media playing, I do this:
private func unloadMediaPlayer()
{
if _mediaPlayer != nil {
_mediaPlayer!.unload()
self._mediaPlayer = nil
}
_controlView.updateForProgress(0, duration: 0, animate: false)
ForgeHelper.appDelegate().allowBackgroundAudio()
setLockScreenInfo()
}
Hope this helps you out!
I'm using seekToTime for an AVPlayer. It works fine, however, I'd like to be able to hear audio as I scrub through the video, much like how Final Cut or other video editors work. Just looking for ideas or if I've missed something obvious.
The way to do this is to scrub a simultaneous AVplayer asynchronously alongside the video. I did it this way (in Swift 4):
// create the simultaneous player and feed it the same URL:
let videoPlayer2 = AVPlayer(url: sameUrlAsVideoPlayer!)
videoPlayer2.volume = 5.0
//set the simultaneous player at exactly the same point as the video player.
videoPlayer2.seek(to: sameSeekTimeAsVideoPlayer)
// create a variable(letsScrub)that allows you to activate the audio scrubber when the video is being scrubbed:
var letsScrub: Bool?
//when you are scrubbing the video you will set letsScrub to true:
if letsScrub == true {audioScrub()}
//create this function to scrub audio asynchronously:
func audioScrub() {
DispatchQueue.main.async {
//set the variable to false so the scrubbing does not get interrupted:
self.letsScrub = false
//play the simultaneous player
self.videoPlayer2.play()
//make sure that it plays for at least 0.25 of a second before it stops to get that scrubbing effect:
DispatchQueue.main.asyncAfter(deadline: .now() + 0.25) {
//now that 1/4 of a second has passed (you can make it longer or shorter as you please) - pause the simultaneous player
self.videoPlayer2.pause()
//now move the simultaneous player to the same point as the original videoplayer:
self.videoPlayer2.seek(to: self.sameSeekTimeAsVideoPlayer)
//setting this variable back to true allows the process to be repeated as long as video is being scrubbed:
self.letsScrub = true
}
}
}