I am creating and playing an AVAudioPlayer as the following:
playerOne = try AVAudioPlayer(contentsOf: URL.init(fileURLWithPath: path))
playerOne.numberOfLoops = -1
playerOne.prepareToPlay()
I am playing an AAC file. I am using
playerOne.play(atTime: startTime)
to schedule a play in future and sync multiple AVAudioPlayers.
All works fine but my problem is when sounds loop they go out of sync, this is due to loops not being seamless.
What happens here is that due to aac decoders I believe that there is an exra small silence is added to the decoded audio data which causes the sync between audio players to be lost. I expected this loop to be perfect with 0 gap between looping from end to beginning.
How may I achieve seamless looping with AVAudioPlayer?
Using AVAudioEngine will give you a lot of flexibility but it's overhead if you don't need anything else but sync your tracks.
In this case you can try to use a single player with AVComposition containing all your tracks, something like this:
func generateComposition(urls: [URL]) throws -> AVComposition {
let composition = AVMutableComposition()
let audioTracks = urls
.map(AVAsset.init(url:))
.flatMap { $0.tracks(withMediaType: .audio) }
for audioTrack in audioTracks {
guard
let compositionTrack = composition.addMutableTrack(withMediaType: .audio, preferredTrackID: kCMPersistentTrackID_Invalid)
else { continue }
try compositionTrack.insertTimeRange(
audioTrack.timeRange,
of: audioTrack,
at: .zero
)
}
return composition
}
And play it using AVPlayer:
AVPlayer(playerItem: AVPlayerItem(asset: composition))
due to loops not being seamless.
I'm not sure what you mean by "not seamless", but if the problem is that you are experiencing latency, that's pretty much natural with AVAudioPlayer. If the goal is to loop with minimal latency, use AVAudioEngine.
Related
We have successfully configured Subtitles/Captions in Azure Media Player which plays media on the Web side.
But, how do we configure the same for playing the media managed by AMS in iOS for Native AVPlayer? We know that captions/subtitles can be played in native iOS player with Sidecar WebVTT file, but is the "transcript.vtt" file generated by AMS, the Sidecar WebVTT file ?
If not, how do we generate the sidecar WebVTT file?
We have implemented the code as below with Media file being accessed from AMS link and a locally downloaded transcript.vtt file, but it fails.
[EDITED : 20200413]
However, when we have local media file and local transcript.vtt file, or when we directly access the media file in the media storage account (https://mediastorageaccount.blob.core.windows.net/container/file.mp4) it works fine. But, when we access the encoded file from the link generated by AMS Transform (https://mediaservice-inct.streaming.media.azure.net/788888-6666-4444-aaaa-823422j218/file.ism/manifest(format=m3u8-cmaf)) it fails.
What is wrong here?
func playVideo()
{
let strUrl = "https://mediaservice-inct.streaming.media.azure.net/79510-6eb-340-a90-824218/German-FAST_Lesson-2-Dialog.ism/manifest(format=m3u8-cmaf)"
localVideoAsset = AVURLAsset(url: URL(string: strUrl)!)
//We have to add tracks in AVMutableComposition same like bellow
//First we have to add video track on AVMutableComposition
let videoTrack = videoPlusSubtitles.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)
do{
guard localVideoAsset!.tracks.count > 0 else{
// error msg
return
}
try? videoTrack?.insertTimeRange(CMTimeRangeMake(start: CMTime.zero, duration: localVideoAsset!.duration),
of:localVideoAsset!.tracks(withMediaType: .video)[0],
at: seconds)
}
//After that we have to add subtitle track in AVMutableComposition
if isEnglishSubtitle {
setSubtitleTrack(subtitle: "transcript")
}else{
setSubtitleTrack(subtitle: "transcript_tr")
}
//After set the video track and subtitle track we have to set in the player same like bellow
player = AVPlayer(playerItem: AVPlayerItem(asset: videoPlusSubtitles))
playerLayer.removeFromSuperlayer()
playerLayer = AVPlayerLayer(player: player)
playerLayer.frame = self.videoView.bounds
playerLayer.videoGravity = .resizeAspect
self.videoView.layer.addSublayer(playerLayer)
player.play()
}
func setSubtitleTrack(subtitle : String){
print(subtitle)
print(seconds)
//Here we have to check if any pre track available. If available then we have to remove it same like bellow
if subtitleTrack != nil{
videoPlusSubtitles.removeTrack(subtitleTrack!)
}
//We have to get subtitle file from path same like bellow
let subtitleAsset = AVURLAsset(url: Bundle.main.url(forResource: subtitle, withExtension: ".vtt")!)
// And we have to add new track from here
subtitleTrack = videoPlusSubtitles.addMutableTrack(withMediaType: .text, preferredTrackID: kCMPersistentTrackID_Invalid)
do{
guard subtitleAsset.tracks.count > 0 else{
//error msg
return
}
try? subtitleTrack?.insertTimeRange(CMTimeRangeMake(start: CMTime.zero, duration: localVideoAsset!.duration),
of:subtitleAsset.tracks(withMediaType: .text)[0],
at: seconds)
}
}
I suspect the issue is not caused by the AMS stream. To double check, you may want to try using another stream HLS(e.g. try a HLS provided by Apple). Apple has specific requirements for playing VTT in AVPlayer. I've included an Apple doc link which has a lot of examples on streaming, and other links that may be helpful:
https://developer.apple.com/streaming/examples/
How to add external .vtt subtitle file to AVPlayerViewController in tvOS
AVUrlAsset and WebVTTs
I'm trying to build an iOS application using Swift 4 that would play different sounds. Part of what I need to do is to limit the duration of the sound file played depending on some settings.
What I'd like to know is if it's possible to set the duration of the sound prior to playing it so it can stop after the set duration. If so, how?
I'm currently using Swift's AVAudioPlayer but I don't know if it can do it. My current code is shown below:
// resName set depending on settings
url = Bundle.main.url(forResource: resName, withExtension: "mp3")
do{
audioPlayer = try AVAudioPlayer(contentsOf: url!)
audioPlayer.prepareToPlay()
audioPlayer.currentTime = 0
} catch let error as NSError{
print(error.debugDescription)
}
audioPlayer.play()
Thanks in advance for the help :)
Well, since you are in control of playing the sounds, one way how to deal with it would be running a Timer when you start playing that would after the given time period stop playing:
let timeLimit = 1.6 // get it from settings
player.play()
Timer.scheduledTimer(withTimeInterval: timeLimit, repeats: false) { (timer) in
player.stop()
}
And in case you need to cancel it, you can keep a reference to the timer and cancel it using timer.invalidate().
As an audio engineer, I recommend editing your audio to be exactly how you want it to avoid any unwanted artifacts that can happen from coding.
Here is a link to a GIF of the problem:
https://gifyu.com/images/ScreenRecording2017-01-25at02.20PM.gif
I'm taking a PHAsset from the camera roll, adding it to a mutable composition, adding another video track, manipulating that added track, and then exporting it through AVAssetExportSession. The result is a quicktime file with .mov file extension saved in the NSTemporaryDirectory():
guard let exporter = AVAssetExportSession(asset: mergedComposition, presetName: AVAssetExportPresetHighestQuality) else {
fatalError()
}
exporter.outputURL = temporaryUrl
exporter.outputFileType = AVFileTypeQuickTimeMovie
exporter.shouldOptimizeForNetworkUse = true
exporter.videoComposition = videoContainer
// Export the new video
delegate?.mergeDidStartExport(session: exporter)
exporter.exportAsynchronously() { [weak self] in
DispatchQueue.main.async {
self?.exportDidFinish(session: exporter)
}
}
I then take this exported file and load it into a mapper object that applies 'slow motion' to the clip based on some time mappings given to it. The result here is an AVComposition:
func compose() -> AVComposition {
let composition = AVMutableComposition(urlAssetInitializationOptions: [AVURLAssetPreferPreciseDurationAndTimingKey: true])
let emptyTrack = composition.addMutableTrack(withMediaType: AVMediaTypeVideo, preferredTrackID: kCMPersistentTrackID_Invalid)
let audioTrack = composition.addMutableTrack(withMediaType: AVMediaTypeAudio, preferredTrackID: kCMPersistentTrackID_Invalid)
let asset = AVAsset(url: url)
guard let videoAssetTrack = asset.tracks(withMediaType: AVMediaTypeVideo).first else { return composition }
var segments: [AVCompositionTrackSegment] = []
for map in timeMappings {
let segment = AVCompositionTrackSegment(url: url, trackID: kCMPersistentTrackID_Invalid, sourceTimeRange: map.source, targetTimeRange: map.target)
segments.append(segment)
}
emptyTrack.preferredTransform = videoAssetTrack.preferredTransform
emptyTrack.segments = segments
if let _ = asset.tracks(withMediaType: AVMediaTypeVideo).first {
audioTrack.segments = segments
}
return composition.copy() as! AVComposition
}
Then I load this file as well as the original file which has also been mapped to slowmo into AVPlayerItems to play in a AVPlayers which is connected to a AVPlayerLayers in my app:
let firstItem = AVPlayerItem(asset: originalAsset)
let player1 = AVPlayer(playerItem: firstItem)
firstItem.audioTimePitchAlgorithm = AVAudioTimePitchAlgorithmVarispeed
player1.actionAtItemEnd = .none
firstPlayer.player = player1
// set up player 2
let secondItem = AVPlayerItem(asset: renderedVideo)
secondItem.seekingWaitsForVideoCompositionRendering = true //tried false as well
secondItem.audioTimePitchAlgorithm = AVAudioTimePitchAlgorithmVarispeed
secondItem.videoComposition = nil // tried AVComposition(propertiesOf: renderedVideo) as well
let player2 = AVPlayer(playerItem: secondItem)
player2.actionAtItemEnd = .none
secondPlayer.player = player2
I then have a start and end time to loop through these videos over and over. I don't use PlayerItemDidReachEnd because i'm not interested in the end, I'm interested in the user inputed time. I even use dispatchGroup to ENSURE that both players have finished seeking before trying to replay the video:
func playAllPlayersFromStart() {
let dispatchGroup = DispatchGroup()
dispatchGroup.enter()
firstPlayer.player?.currentItem?.seek(to: startTime, toleranceBefore: kCMTimeZero, toleranceAfter: kCMTimeZero, completionHandler: { _ in
dispatchGroup.leave()
})
DispatchQueue.global().async { [weak self] in
guard let startTime = self?.startTime else { return }
dispatchGroup.wait()
dispatchGroup.enter()
self?.secondPlayer.player?.currentItem?.seek(to: startTime, toleranceBefore: kCMTimeZero, toleranceAfter: kCMTimeZero, completionHandler: { _ in
dispatchGroup.leave()
})
dispatchGroup.wait()
DispatchQueue.main.async { [weak self] in
self?.firstPlayer.player?.play()
self?.secondPlayer.player?.play()
}
}
}
The strange part here is that the original asset, which has also been mapped via my compose() function loops perfectly fine. However, the renderedVideo which has also been run through the compose() function sometimes freezes when seeking during one of the CMTimeMapping segments. The only difference between the file that freezes and the file that doesnt freeze is that one has been exported to the NSTemporaryDirectory via the AVAssetExportSession to combine the two video tracks into one. They're both the same duration. I'm also sure that it's only the video layer that is freezing and not the audio, because if I add BoundaryTimeObservers to the player that freezes it still hits them and loops. Also the audio loops properly.
To me the strangest part is that the video 'resumes' if it makes it past the spot where it paused to start the seek after a 'freeze'. I've been stuck on this for days and would really love some guidance.
Other odd things to note:
- Even though the CMTimeMapping of the original versus the exported asset are the exact same durations, you'll notice that the rendered asset's slow motion ramp is more 'choppy' than the original.
- Audio continues when video freezes.
- video almost only ever freezes during slow motion sections (caused by CMTimeMapping objects based on segments
- rendered video seems to have to play 'catch up' at the beginning. even though i'm calling play after both have finished seeking, it seems to me that the right side plays faster in the beginning as a catch up. Strange part is that the segments are the exact same, just referencing two separate source files. One located in the asset library, the other in NSTemporaryDirectory
- It seems to me that AVPlayer and AVPlayerItemStatus is 'readyToPlay' before i call play.
- It seems to 'unfreeze' if the player proceeds PAST the point that it locked up.
- I tried to add observers for 'AVPlayerItemPlaybackDidStall' but it was never called.
Cheers!
The problem was in the AVAssetExportSession. To my surprise, changing exporter.canPerformMultiplePassesOverSourceMediaData = true fixed the issue. Although the documentation is quite sparse and even claims 'setting this property to true may have no effect', it did seem to fix the issue. Very, very, very strange! I consider this a bug and will be filing a radar. Here are the docs on the property: canPerformMultiplePassesOverSourceMediaData
It seems possible that in your playAllPlayersFromStart() method that the startTime variable may have changed between the two tasks dispatched (this would be especially likely if that value updates based on scrubbing).
If you make a local copy of startTime at the start of the function, and then use it in both blocks, you may have better luck.
In my application I've to stream videos from server. For that I've used below Code
-(void)playingSong:(NSURL*) url{
AVAsset *asset = [AVAsset assetWithURL:url];
duration = asset.duration;
playerItem = [AVPlayerItem playerItemWithAsset:asset];
player = [AVPlayer playerWithPlayerItem:playerItem];
[player play];
}
All are Global Variables
It's playing all videos when network is good, but unable to play videos with big size, when network is slow.
Means It's not playing for big size videos and it's playing small videos;
I'm using http Server not https;
for ex : 3min video it's playing but for 1hr video it's not.
Why so?
Seems like you have to download the whole video before you can begin playback. It can also be because of your server not AVPlayer.
when you serve videos on a site using plain HTTP – known as
progressive download – the position of the header becomes very
important. Either the header is placed in the beginning of the file or
it’s places in the end of the file. In case of the latter, you’ll have
to download the whole thing before you can begin playback – because
without the header, the player can’t start decoding.
Have look at this guide if your problem is because of videos source.
Have a look at this thread and change you implementation accordingly.
Download video in local and then play in avplayer.
DispatchQueue.global(qos: .background).async {
do {
let data = try Data(contentsOf: url)
DispatchQueue.main.async {
// store "data" in document folder
let fileUrl = URL(fileURLWithPath: <#localVideoURL#>)
let asset = AVAsset(url: fileUrl)
let item = AVPlayerItem(asset: asset)
let player = AVPlayer(playerItem: item)
let layer = AVPlayerLayer(player: player)
layer.bounds = self.view.bounds
self.view.layer.addSublayer(layer)
player.play()
}
} catch {
print(error.localizedDescription)
}
}
I am using a subclass of AVQueuePlayer and when I add new AVPlayerItem with a streaming URL the app freezes for about a second or two. By freezing I mean that it doesn't respond to touches on the UI. Also, if I have a song playing already and then add another one to the queue, AVQueuePlayer automatically starts preloading the song while it is still streaming the first one. This makes the app not respond to touches on the UI for two seconds just like when adding the first song but the song is still playing. So that means AVQueuePlayer is doing something in main thread that is causing the apparent "freeze".
I am using insertItem:afterItem: to add my AVPlayerItem. I tested and made sure that this was the method that was causing the delay. Maybe it could be something that AVPlayerItem does when it gets activated by AVQueuePlayer at the moment of adding it to the queue.
Must point out that I am using the Dropbox API v1 beta to get the streaming URL by using this method call:
[[self restClient] loadStreamableURLForFile:metadata.path];
Then when I receive the stream URL I send it to AVQueuePlayer as follows:
[self.player insertItem:[AVPlayerItem playerItemWithURL:url] afterItem:nil];
So my question is: How do I avoid this?
Should I do the preloading of an audio stream on my own without the help of AVPlayer? If so, how do I do this?
Thanks.
Don't use playerItemWithURL it's sync.
When you receive the response with the url try this:
AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:url options:nil];
NSArray *keys = #[#"playable"];
[asset loadValuesAsynchronouslyForKeys:keys completionHandler:^() {
[self.player insertItem:[AVPlayerItem playerItemWithAsset:asset] afterItem:nil];
}];
Bump, since this is a highly rated question and similar questions online either has outdated answers or aren't great. The whole idea is pretty straight forward with AVKit and AVFoundation, which means no more depending on third party libraries. The only issue is that it took some tinkering around and put the pieces together.
AVFoundation's Player() initialization with url is apparently not thread safe, or rather it's not meant to be. Which means, no matter how you initialize it in a background thread the player attributes are going to be loaded in the main queue causing freezes in the UI especially in UITableViews and UICollectionViews. To solve this issue Apple provided AVAsset which takes a URL and assists in loading the media attributes like track, playback, duration etc. and can do so asynchronously, with a best part being that this loading process is cancellable (unlike other Dispatch queue background threads where ending a task may not be that straight forward). This means, there is no need to worry about lingering zombie threads in the background as you scroll fast on a table view or collection view, ultimately piling up on the memory with a whole bunch of unused objects. This cancellable feature is great, and allows us to cancel any lingering AVAsset async load if it is in progress but only during cell dequeue. The async loading process can be invoked by the loadValuesAsynchronously method, and can be cancelled (at will) at any later time (if still in progress).
Don't forget to exception handle properly using the results of loadValuesAsynchronously. In Swift (3/4), here's how you would would load a video asynchronously and handle situations if the async process fails (due to slow networks, etc.)-
TL;DR
TO PLAY A VIDEO
let asset = AVAsset(url: URL(string: self.YOUR_URL_STRING))
let keys: [String] = ["playable"]
var player: AVPlayer!
asset.loadValuesAsynchronously(forKeys: keys, completionHandler: {
var error: NSError? = nil
let status = asset.statusOfValue(forKey: "playable", error: &error)
switch status {
case .loaded:
DispatchQueue.main.async {
let item = AVPlayerItem(asset: asset)
self.player = AVPlayer(playerItem: item)
let playerLayer = AVPlayerLayer(player: self.player)
playerLayer.videoGravity = AVLayerVideoGravity.resizeAspectFill
playerLayer.frame = self.YOUR_VIDEOS_UIVIEW.bounds
self.YOUR_VIDEOS_UIVIEW.layer.addSublayer(playerLayer)
self.player.isMuted = true
self.player.play()
}
break
case .failed:
DispatchQueue.main.async {
//do something, show alert, put a placeholder image etc.
}
break
case .cancelled:
DispatchQueue.main.async {
//do something, show alert, put a placeholder image etc.
}
break
default:
break
}
})
NOTE:
Based on what your app wants to achieve you may still have to do some amount of tinkering to tune it to get smoother scroll in a UITableView or UICollectionView. You may also need to implement some amount of KVO on the AVPlayerItem properties for it to work and there's plenty of posts here in SO that discuss AVPlayerItem KVOs in detail.
TO LOOP THROUGH ASSETS (video loops/GIFs)
To loop a video, you can use the same method above and introducing AVPlayerLooper. Here's a sample code to loop a video (or perhaps a short video in GIF style). Note the use of duration key which is required for our video loop.
let asset = AVAsset(url: URL(string: self.YOUR_URL_STRING))
let keys: [String] = ["playable","duration"]
var player: AVPlayer!
var playerLooper: AVPlayerLooper!
asset.loadValuesAsynchronously(forKeys: keys, completionHandler: {
var error: NSError? = nil
let status = asset.statusOfValue(forKey: "duration", error: &error)
switch status {
case .loaded:
DispatchQueue.main.async {
let playerItem = AVPlayerItem(asset: asset)
self.player = AVQueuePlayer()
let playerLayer = AVPlayerLayer(player: self.player)
//define Timerange for the loop using asset.duration
let duration = playerItem.asset.duration
let start = CMTime(seconds: duration.seconds * 0, preferredTimescale: duration.timescale)
let end = CMTime(seconds: duration.seconds * 1, preferredTimescale: duration.timescale)
let timeRange = CMTimeRange(start: start, end: end)
self.playerLooper = AVPlayerLooper(player: self.player as! AVQueuePlayer, templateItem: playerItem, timeRange: timeRange)
playerLayer.videoGravity = AVLayerVideoGravity.resizeAspectFill
playerLayer.frame = self.YOUR_VIDEOS_UIVIEW.bounds
self.YOUR_VIDEOS_UIVIEW.layer.addSublayer(playerLayer)
self.player.isMuted = true
self.player.play()
}
break
case .failed:
DispatchQueue.main.async {
//do something, show alert, put a placeholder image etc.
}
break
case .cancelled:
DispatchQueue.main.async {
//do something, show alert, put a placeholder image etc.
}
break
default:
break
}
})
EDIT : As per the documentation, AVPlayerLooper requires the duration property of the asset to be fully loaded in order to be able to loop through videos. Also, the timeRange: timeRange with the start and end timerange in the AVPlayerLooper initialization is really optional if you want an infinite loop. I have also realized since I posted this answer that AVPlayerLooper is only about 70-80% accurate in looping videos, especially if your AVAsset needs to stream the video from a URL. In order to solve this issue there is a totally different (yet simple) approach to loop a video-
//this will loop the video since this is a Gif
let interval = CMTime(value: 1, timescale: 2)
self.timeObserverToken = self.player?.addPeriodicTimeObserver(forInterval: interval, queue: DispatchQueue.main, using: { (progressTime) in
if let totalDuration = self.player?.currentItem?.duration{
if progressTime == totalDuration{
self.player?.seek(to: kCMTimeZero)
self.player?.play()
}
}
})
Gigisommo's answer for Swift 3 including the feedback from the comments:
let asset = AVAsset(url: url)
let keys: [String] = ["playable"]
asset.loadValuesAsynchronously(forKeys: keys) {
DispatchQueue.main.async {
let item = AVPlayerItem(asset: asset)
self.playerCtrl.player = AVPlayer(playerItem: item)
}
}