Unity video player audio skipping [duplicate] - ipad

MovieTexture is finally deprecated after Unity 5.6.0b1 release and new API that plays video on both Desktop and Mobile devices is now released.
VideoPlayer and VideoClip can be used to play video and retrieve texture for each frame if needed.
I've managed to get the video working but coduldn't get the audio to play as-well from the Editor on Windows 10. Anyone know why audio is not playing?
//Raw Image to Show Video Images [Assign from the Editor]
public RawImage image;
//Video To Play [Assign from the Editor]
public VideoClip videoToPlay;
private VideoPlayer videoPlayer;
private VideoSource videoSource;
//Audio
private AudioSource audioSource;
// Use this for initialization
void Start()
{
Application.runInBackground = true;
StartCoroutine(playVideo());
}
IEnumerator playVideo()
{
//Add VideoPlayer to the GameObject
videoPlayer = gameObject.AddComponent<VideoPlayer>();
//Add AudioSource
audioSource = gameObject.AddComponent<AudioSource>();
//Disable Play on Awake for both Video and Audio
videoPlayer.playOnAwake = false;
audioSource.playOnAwake = false;
//We want to play from video clip not from url
videoPlayer.source = VideoSource.VideoClip;
//Set video To Play then prepare Audio to prevent Buffering
videoPlayer.clip = videoToPlay;
videoPlayer.Prepare();
//Wait until video is prepared
while (!videoPlayer.isPrepared)
{
Debug.Log("Preparing Video");
yield return null;
}
Debug.Log("Done Preparing Video");
//Set Audio Output to AudioSource
videoPlayer.audioOutputMode = VideoAudioOutputMode.AudioSource;
//Assign the Audio from Video to AudioSource to be played
videoPlayer.EnableAudioTrack(0, true);
videoPlayer.SetTargetAudioSource(0, audioSource);
//Assign the Texture from Video to RawImage to be displayed
image.texture = videoPlayer.texture;
//Play Video
videoPlayer.Play();
//Play Sound
audioSource.Play();
Debug.Log("Playing Video");
while (videoPlayer.isPlaying)
{
Debug.LogWarning("Video Time: " + Mathf.FloorToInt((float)videoPlayer.time));
yield return null;
}
Debug.Log("Done Playing Video");
}

Found the problem. Below is the FIXED code that plays Video and Audio:
//Raw Image to Show Video Images [Assign from the Editor]
public RawImage image;
//Video To Play [Assign from the Editor]
public VideoClip videoToPlay;
private VideoPlayer videoPlayer;
private VideoSource videoSource;
//Audio
private AudioSource audioSource;
// Use this for initialization
void Start()
{
Application.runInBackground = true;
StartCoroutine(playVideo());
}
IEnumerator playVideo()
{
//Add VideoPlayer to the GameObject
videoPlayer = gameObject.AddComponent<VideoPlayer>();
//Add AudioSource
audioSource = gameObject.AddComponent<AudioSource>();
//Disable Play on Awake for both Video and Audio
videoPlayer.playOnAwake = false;
audioSource.playOnAwake = false;
//We want to play from video clip not from url
videoPlayer.source = VideoSource.VideoClip;
//Set Audio Output to AudioSource
videoPlayer.audioOutputMode = VideoAudioOutputMode.AudioSource;
//Assign the Audio from Video to AudioSource to be played
videoPlayer.EnableAudioTrack(0, true);
videoPlayer.SetTargetAudioSource(0, audioSource);
//Set video To Play then prepare Audio to prevent Buffering
videoPlayer.clip = videoToPlay;
videoPlayer.Prepare();
//Wait until video is prepared
while (!videoPlayer.isPrepared)
{
Debug.Log("Preparing Video");
yield return null;
}
Debug.Log("Done Preparing Video");
//Assign the Texture from Video to RawImage to be displayed
image.texture = videoPlayer.texture;
//Play Video
videoPlayer.Play();
//Play Sound
audioSource.Play();
Debug.Log("Playing Video");
while (videoPlayer.isPlaying)
{
Debug.LogWarning("Video Time: " + Mathf.FloorToInt((float)videoPlayer.time));
yield return null;
}
Debug.Log("Done Playing Video");
}
Why Audio was not playing:
//Set Audio Output to AudioSource
videoPlayer.audioOutputMode = VideoAudioOutputMode.AudioSource;
//Assign the Audio from Video to AudioSource to be played
videoPlayer.EnableAudioTrack(0, true);
videoPlayer.SetTargetAudioSource(0, audioSource);
must be called before videoPlayer.Prepare(); not after it. This is took hours of experiment to find this this was the problem I was having.
Stuck at "Preparing Video"?
Wait 5 seconds after videoPlayer.Prepare(); is called then exit the while loop.
Replace:
while (!videoPlayer.isPrepared)
{
Debug.Log("Preparing Video");
yield return null;
}
with:
//Wait until video is prepared
WaitForSeconds waitTime = new WaitForSeconds(5);
while (!videoPlayer.isPrepared)
{
Debug.Log("Preparing Video");
//Prepare/Wait for 5 sceonds only
yield return waitTime;
//Break out of the while loop after 5 seconds wait
break;
}
This should work but you may experience buffering when the video starts playing. While using this temporary fix, my suggestion is to file for bug with the title of "videoPlayer.isPrepared always true" because this is a bug.
Some people also fixed it by changing:
videoPlayer.playOnAwake = false;
audioSource.playOnAwake = false;
to
videoPlayer.playOnAwake = true;
audioSource.playOnAwake = true;
Play Video From URL:
Replace:
//We want to play from video clip not from url
videoPlayer.source = VideoSource.VideoClip;
with:
//We want to play from url
videoPlayer.source = VideoSource.Url;
videoPlayer.url = "http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4";
then Remove:
public VideoClip videoToPlay; and videoPlayer.clip = videoToPlay; as these are not needed anymore.
Play Video From StreamingAssets folder:
string url = "file://" + Application.streamingAssetsPath + "/" + "VideoName.mp4";
if !UNITY_EDITOR && UNITY_ANDROID
url = Application.streamingAssetsPath + "/" + "VideoName.mp4";
#endif
//We want to play from url
videoPlayer.source = VideoSource.Url;
videoPlayer.url = url;
All supported video formats:
ogv
vp8
webm
mov
dv
mp4
m4v
mpg
mpeg
Extra supported video formats on Windows:
avi
asf
wmf
Some of these formats don't work on some platforms. See this post for more information on supported video formats.

Similar to what the other answers have been saying. You could use callbacks for when preparing and end of video states. Rather than using coroutines and yield return.
videoPlayer.loopPointReached += EndReached;
videoPlayer.prepareCompleted += PrepareCompleted;
void PrepareCompleted(VideoPlayer vp) {
vp.Play();
}
void EndReached(VideoPlayer vp) {
// do something
}

I used #Programmer 's answer to play videos from a URL, but I couldn't get any sound to play. Eventually I found the answer in the comments of a YouTube tutorial.
To get the audio to play for a movie loaded via URL, you need to add the following line before the call to EnableAudioTrack:
videoPlayer.controlledAudioTrackCount = 1;

By now the VideoPlayer should be updated enough you don't need to write code to get to work correctly. Here are the settings I found to have the most desirable effect:
These settings are:
Video Player:
Play On Awake: True
Wait For First Frame: False
Audio Output Mode: None
Audio Source:
Play On Awake: True
Don't forget to have a VideoClip for the VideoPlayer and an AudioClip for the AudioSource. The file formats I found to work the best are .ogv for video and .wav for audio.

Related

how to get .opus audio file duration in swift

I am using 'YbridPlayerSDK' library to play opus audio
let myEndpoint = MediaEndpoint(mediaUri: audioURL.absoluteString)
try AudioPlayer.open(for: myEndpoint, listener: nil) {
(control) in /// called asychronously
control.play()
}
The above code play the audio but I am not able to find out its duration. how to get audio duration

How to dynamically change players in AudioKit output

I'm implementing an app utilizing AudioKit that allows you to play a large number of audio files and switch between them at any time. Additionally we need one audio file to play on loop for ambient noise.
What is the correct way to dynamically change players in AudioKit's output? Or what's the proper way to implement this behavior?
AudioKit requires you set its output which can be an AKPlayer that plays one audio file or an AKMixer given an array of AKPlayers for example. You cannot change the output once AudioKit has been started as I saw previously. So currently my approach is to use AKMixer to play two AKPlayers - one for the current file and one for the ambient noise. When the user taps the 'Next' button I stop() AudioKit, recreate an AKMixer with a new player for the next song's audio file and the ambient noise player, assign that to output, start() AudioKit, and play both players. This results in undesirable behavior because playback of the ambient noise is stopped when switching songs resulting in a brief pause.
If this is the correct approach, how can we update the output without stopping AudioKit? I wondered if you can initialize the mixer with an array of players that's a strongly held property and simply add/remove players in the array. But that doesn't work - starting a new player throws an error player started when in a disconnected state because this node is not attached to the output.
I've created a sample project to demonstrate this behavior. When launched the app plays drums as ambient noise and waves for the current track. When you tap Next it'll switch to the next track (which in this demo code is just the same audio file) and you can hear the drums stop and then resume which is undesirable. I've included the project's code below:
final class Maestro: NSObject {
static let shared = Maestro()
private var audioPlayer: AKPlayer?
private var ambientPlayer: AKPlayer = {
let player = AKPlayer(url: Bundle.main.url(forResource: "drums", withExtension: "wav")!)!
player.isLooping = true
return player
}()
private var mixer: AKMixer?
private let audioFileURL = Bundle.main.url(forResource: "waves", withExtension: "mp3")!
func play() {
playNewPlayer(fileURL: audioFileURL)
}
func next() {
//In the real app we'd play the next audio file in the playlist but for the demo we'll just play the same file
playNewPlayer(fileURL: audioFileURL)
}
private func playNewPlayer(fileURL: URL) {
audioPlayer?.stop()
audioPlayer = nil
do {
try AudioKit.stop()
} catch {
print("Maestro AudioKit.stop error: \(error)")
}
audioPlayer = AKPlayer(url: fileURL)!
mixer = AKMixer([audioPlayer!, ambientPlayer])
AudioKit.output = mixer
do {
try AudioKit.start()
} catch {
print("Maestro AudioKit.start error: \(error)")
}
if ambientPlayer.isPlaying {
//need to resume playback from current position
let pos = ambientPlayer.currentTime
ambientPlayer.stop()
ambientPlayer.play(from: pos)
} else {
ambientPlayer.play()
}
audioPlayer?.play()
}
}

AVPlayer with Visualizer

I need to create a Audio Player for streamed URL (m3u8 format). I have created music player using AVPlayer. But I need to show visualizer for streamed song. I have tried different solution but not found any working example of it.
I have created visualizer using AVAudioPlayer(averagePower) but it won't support streamed URL.
Any help to show visualizer for AVPlayer? Thanks in advance.
I have also tried using MYAudioTapProcessor which most of the people suggested, but for streamed URL, tracks always returns null.
Added the MYAudioTapProcessor.h and MYAudioTapProcessor.m in project
//Initialization of player
let playerItem = AVPlayerItem( url:NSURL( string:"https://bitdash-a.akamaihd.net/content/sintel/hls/playlist.m3u8" ) as! URL )
let audioPlayer: AVPlayer = AVPlayer(playerItem:playerItem)
//Added periodic time observer
audioPlayer!.addPeriodicTimeObserver(forInterval: CMTimeMakeWithSeconds(1, 1), queue: DispatchQueue.main) { (CMTime) -> Void in
if audioPlayer!.currentItem?.status == .readyToPlay
{
if let playerItem: AVPlayerItem = audioPlayer!.currentItem {
print(playerItem.asset.tracks.count)
if (playerItem.asset.tracks) != nil {
self.tapProcessor = MYAudioTapProcessor(avPlayerItem: playerItem)
playerItem.audioMix = self.tapProcessor.audioMix
self.tapProcessor.delegate = self
}
}
}
}
//Delegate callback method for MYAudioTapProcessor
func audioTabProcessor(_ audioTabProcessor: MYAudioTapProcessor!, hasNewLeftChannelValue leftChannelValue: Float, rightChannelValue: Float) {
print("volume: \(leftChannelValue) : \(rightChannelValue)")
volumeSlider.value = leftChannelValue
}
Also tried by adding the "Track" observer.
playerItem.addObserver(self, forKeyPath: "tracks", options: NSKeyValueObservingOptions.new, context: nil);
Now if play mp3 file, the callback method calls but for m3u8 callback method doesn't call. The main reason for failing m3u8 URL is it always show tracks array count zero whereas for mp3 files tracks array has one item.
You cannot get tracks for HLS via AVPLayer. You should use progressive download or local file for getting audio tracks while playing media.

Set timestamp in CMsampleBuffer using AVAssetWriter not working

Hello I'm working in an app that is recording video + audio. The Video source is the camera, and the audio is coming from streaming. My problem happen when the communication with streaming is closed for some reason. Then in that case I switch the audio source to built in mic. The problem is the audio is not synchronised at all. I would like to add a space in my audio and then set the timestamp in realtime according to the current video timestamp. Seems AvassetWritter is adding the frames consecutive from built in mic and it looks like is ignoring the timestamp.
Do you know why avassetwriter is ignoring the timestamp?
EDIT:
This is the code than gets the latest video timestamp
- (void)renderVideoSampleBuffer:(CMSampleBufferRef)sampleBuffer
{
CVPixelBufferRef renderedPixelBuffer = NULL;
CMTime timestamp = CMSampleBufferGetPresentationTimeStamp( sampleBuffer );
self.lastVideoTimestamp = timestamp;
and this is the code that I use to synchronise audio coming from built in mic when the stream is disconnected.
CFRelease(sampleBuffer);
sampleBuffer = [self adjustTime:sampleBuffer by:self.lastVideoTimestamp];
//Adjust CMSampleBufferFunction
- (CMSampleBufferRef) adjustTime:(CMSampleBufferRef) sample by:(CMTime) offset
{
CMItemCount count;
CMSampleBufferGetSampleTimingInfoArray(sample, 0, nil, &count);
CMSampleTimingInfo* pInfo = malloc(sizeof(CMSampleTimingInfo) * count);
CMSampleBufferGetSampleTimingInfoArray(sample, count, pInfo, &count);
for (CMItemCount i = 0; i < count; i++)
{
pInfo[i].decodeTimeStamp = kCMTimeInvalid;//CMTimeSubtract(pInfo[i].decodeTimeStamp, offset);
pInfo[i].presentationTimeStamp = CMTimeSubtract(pInfo[i].presentationTimeStamp, offset);
}
CMSampleBufferRef sout;
CMSampleBufferCreateCopyWithNewTiming(nil, sample, count, pInfo, &sout);
free(pInfo);
return sout;
}
That is what I want to do.
Video
--------------------------------------------------------------------
Stream disconnect stream Built in mic
----------------------------------- -----------------
I would like to get this, as you can see there is a space with no audio, because the audio coming from the stream is disconnected and maybe you didn't receive all of the audio.
What it is currently doing:
Video
--------------------------------------------------------------------
Stream disconnect stream Built in mic
--------------------------------------------------------------------

AVPlayer Go To Live?

I am using an AVPlayer object to play a live stream of an MP3, just using an HTTP link.
As with normal live video playing in iOS, there is a button that you can press to skip to live.
Is it possible to do this with AVPlayer?
E.g I am listening live, pause, then after however long play again. It continues from where I left it. But what if I want to skip to live?
I had the same need and made this extension to AVPlayer:
extension AVPlayer {
func seekToLive() {
if let items = currentItem?.seekableTimeRanges, !items.isEmpty {
let range = items[items.count - 1]
let timeRange = range.timeRangeValue
let startSeconds = CMTimeGetSeconds(timeRange.start)
let durationSeconds = CMTimeGetSeconds(timeRange.duration)
seek(to: CMTimeMakeWithSeconds(startSeconds + durationSeconds, 1))
}
}
}

Resources