After loading an AVAsset like this:
AVAsset *asset = [AVAsset assetWithURL:url];
I want to know what the Sampling Rate is of the Audio track. Currently, I am getting the Audio Track like this:
AVAssetTrack *audioTrack = [[asset tracksWithMediaCharacteristic:AVMediaCharacteristicAudible] objectAtIndex:0];
Which works. But I can't seem to find any kind of property, not even after using Google ;-) , that gives me the sampling rate. How does this work normally ? Is it even possible ? (I start doubting more and more, because Googling is not giving me a lot of information ...)
let asset = AVAsset(url: URL(fileURLWithPath: "asset/p2a2.aif"))
let track = asset.tracks[0]
let desc = track.formatDescriptions[0] as! CMAudioFormatDescription
let basic = CMAudioFormatDescriptionGetStreamBasicDescription(desc)
print(basic?.pointee.mSampleRate)
I'm using Swift so it looks a bit different but it should still work with Obj-C.
print(track.naturalTimeScale)
Also seems to give the correct answer but I'm a bit apprehensive because of the name.
Using Swift and AVFoundation :
let url = Bundle.main.url(forResource: "audio", withExtension: "m4a")!
let asset = AVAsset(url: url)
if let firstTrack = asset.tracks.first {
print("bitrate: \(firstTrack.estimatedDataRate)")
}
To find more information in your metadata, you can also consult:
https://developer.apple.com/documentation/avfoundation/avassettrack
https://developer.apple.com/documentation/avfoundation/media_assets_playback_and_editing/finding_metadata_values
Found it. I was using the MTAudioProcessingTap, so in the prepare() function I could just use:
void prepare(MTAudioProcessingTapRef tap, CMItemCount maxFrames, const AudioStreamBasicDescription *processingFormat)
{
sampleRate = processingFormat->mSampleRate;
NSLog(#"Preparing the Audio Tap Processor");
}
Related
I have a video with these specs
Format : H.264 , 1280x544
FPS : 25
Data Size : 26MB
Duration : 3:00
Data Rate : 1.17 Mbit/s
While experimenting ,I performed a removeTimeRange(range : CMTimeRange) on every other frame (total frames = 4225). This results in the video becoming 2x faster , so the duration becomes 1:30.
However when I export the video, the video becomes 12x larger in size i.e. 325MB.This makes sense since this technique is decomposing the video into about 2112 pieces and stitching it back together. Apparently, in doing so the compression among individual frames is lost, thus causing the enormous size.
This causes stuttering in the video when played with an AVPlayer and therefore poor performance.
Question : How can I apply some kind of compression while stitching back the frames so that the video can play smoothly and also be less in size?
I only want a point in the right direction. Thanks!
CODE
1) Creating an AVMutableComposition from Asset & Configuring it
func configureAssets(){
let options = [AVURLAssetPreferPreciseDurationAndTimingKey : "true"]
let videoAsset = AVURLAsset(url: Bundle.main.url(forResource: "Push", withExtension: "mp4")! , options : options)
let videoAssetSourceTrack = videoAsset.tracks(withMediaType: AVMediaTypeVideo).first! as AVAssetTrack
let comp = AVMutableComposition()
let videoCompositionTrack = comp.addMutableTrack(withMediaType: AVMediaTypeVideo, preferredTrackID: kCMPersistentTrackID_Invalid)
do {
try videoCompositionTrack.insertTimeRange(
CMTimeRangeMake(kCMTimeZero, videoAsset.duration),
of: videoAssetSourceTrack,
at: kCMTimeZero)
deleteSomeFrames(from: comp)
videoCompositionTrack.preferredTransform = videoAssetSourceTrack.preferredTransform
}catch { print(error) }
asset = comp }
2) Deleting every other frame.
func deleteSomeFrames(from asset : AVMutableComposition){
let fps = Int32(asset.tracks(withMediaType: AVMediaTypeVideo).first!.nominalFrameRate)
let sumTime = Int32(asset.duration.value) / asset.duration.timescale;
let totalFrames = sumTime * fps
let totalTime = Float(CMTimeGetSeconds(asset.duration))
let frameDuration = Double(totalTime / Float(totalFrames))
let frameTime = CMTime(seconds: frameDuration, preferredTimescale: 1000)
for frame in Swift.stride(from: 0, to: totalFrames, by: 2){
let timing = CMTimeMultiplyByFloat64(frameTime, Float64(frame))
print("Asset Duration = \(CMTimeGetSeconds(asset.duration))")
print("")
let timeRange = CMTimeRange(start: timing, duration : frameTime)
asset.removeTimeRange(timeRange)
}
print("duration after time removed = \(CMTimeGetSeconds(asset.duration))")
}
3) Saving the file
func createFileFromAsset(_ asset: AVAsset){
let documentsDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] as URL
let filePath = documentsDirectory.appendingPathComponent("rendered-vid.mp4")
if let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHighestQuality){
exportSession.outputURL = filePath
exportSession.shouldOptimizeForNetworkUse = true
exportSession.timeRange = CMTimeRangeMake(kCMTimeZero, asset.duration)
exportSession.outputFileType = AVFileTypeQuickTimeMovie
exportSession.exportAsynchronously {
print("finished: \(filePath) : \(exportSession.status.rawValue) ")
if exportSession.status.rawValue == 4{
print("Export failed -> Reason: \(exportSession.error!.localizedDescription))")
print(exportSession.error!)
}
}}}
4) Finally update the ViewController to play the new Composition!
override func viewDidLoad() {
super.viewDidLoad()
// Create the AVPlayer and play the composition
assetConfig.configureAssets()
let snapshot : AVComposition = assetConfig.asset as! AVComposition
let playerItem = AVPlayerItem(asset : snapshot)
player = AVPlayer(playerItem: playerItem)
let playerLayer = AVPlayerLayer(player: player)
playerLayer.frame = CGRect(x : 0, y : 0, width : self.view.frame.width , height : self.view.frame.height)
self.view.layer.addSublayer(playerLayer)
player?.play()
}
If you are using AVMutableComposition,you will notice that each composition may contain one or more AVCompositionTrack(or AVMutableCompositionTrack),and the best way to edit your composition was to operate each track, but not the whole composition.
but if your purpose is to faster your video's rate, editing tracks will not be necessary.
so i will try my best to tell you what i know about your question
About video Stuttering while playing
Possible reason of stuttering
notice that you are using method removeTimeRange(range: CMTimeRange),this method will remove the timeRange on composition yes, but will NOT auto fill the empty of each time range
Visualize Example
[F stand for Frame,E stand for Empty]
org_video --> F-F-F-F-F-F-F-F...
after remove time range, the composition will be like this
after_video --> F-E-F-E-F-E-F-E...
and you might think that the video will be like this
target_video --> F-F-F-F...
this is the most possible reason about stuttering during playback.
Suggested solution
So if you want to shorten your video, make it rate more faster/slower,you possible need to use the method scaleTimeRange:(CMTimeRange) toDuration:(CMTime)
Example
AVMutableComposition * project;//if this video is 200s
//Scale
project.scaleTimeRange:CMTimeRangeMake(kCMTimeZero, project.duration) toDuration:CMTimeMakeWithSeconds(100,kCMTimeMaxTimescale)
this method is to make the video faster/slower.
About the file size
a video file's size might effected by bit rate and format type ,if your using H.264,the most possible reason causing size enlarge will be bit rate.
in your code,you are using AVAssetExportSession
AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHighestQuality
you gave the preset which is AVAssetExportPresetHighestQuality
in my own application project, after i was using this preset, the video's bit rate will be 20~30Mbps,no matter your source video's bit rate. and, well using apple's preset will not allowed you to set the bit rate manually, so.
Possible Solution
There is a third part tool called SDAVAssetExportSession,this session will allowed you to fully config your export session, you might want to try to study this code about custom export session's preset.
here is what i can tell you right now. wish could help :>
Here is a link to a GIF of the problem:
https://gifyu.com/images/ScreenRecording2017-01-25at02.20PM.gif
I'm taking a PHAsset from the camera roll, adding it to a mutable composition, adding another video track, manipulating that added track, and then exporting it through AVAssetExportSession. The result is a quicktime file with .mov file extension saved in the NSTemporaryDirectory():
guard let exporter = AVAssetExportSession(asset: mergedComposition, presetName: AVAssetExportPresetHighestQuality) else {
fatalError()
}
exporter.outputURL = temporaryUrl
exporter.outputFileType = AVFileTypeQuickTimeMovie
exporter.shouldOptimizeForNetworkUse = true
exporter.videoComposition = videoContainer
// Export the new video
delegate?.mergeDidStartExport(session: exporter)
exporter.exportAsynchronously() { [weak self] in
DispatchQueue.main.async {
self?.exportDidFinish(session: exporter)
}
}
I then take this exported file and load it into a mapper object that applies 'slow motion' to the clip based on some time mappings given to it. The result here is an AVComposition:
func compose() -> AVComposition {
let composition = AVMutableComposition(urlAssetInitializationOptions: [AVURLAssetPreferPreciseDurationAndTimingKey: true])
let emptyTrack = composition.addMutableTrack(withMediaType: AVMediaTypeVideo, preferredTrackID: kCMPersistentTrackID_Invalid)
let audioTrack = composition.addMutableTrack(withMediaType: AVMediaTypeAudio, preferredTrackID: kCMPersistentTrackID_Invalid)
let asset = AVAsset(url: url)
guard let videoAssetTrack = asset.tracks(withMediaType: AVMediaTypeVideo).first else { return composition }
var segments: [AVCompositionTrackSegment] = []
for map in timeMappings {
let segment = AVCompositionTrackSegment(url: url, trackID: kCMPersistentTrackID_Invalid, sourceTimeRange: map.source, targetTimeRange: map.target)
segments.append(segment)
}
emptyTrack.preferredTransform = videoAssetTrack.preferredTransform
emptyTrack.segments = segments
if let _ = asset.tracks(withMediaType: AVMediaTypeVideo).first {
audioTrack.segments = segments
}
return composition.copy() as! AVComposition
}
Then I load this file as well as the original file which has also been mapped to slowmo into AVPlayerItems to play in a AVPlayers which is connected to a AVPlayerLayers in my app:
let firstItem = AVPlayerItem(asset: originalAsset)
let player1 = AVPlayer(playerItem: firstItem)
firstItem.audioTimePitchAlgorithm = AVAudioTimePitchAlgorithmVarispeed
player1.actionAtItemEnd = .none
firstPlayer.player = player1
// set up player 2
let secondItem = AVPlayerItem(asset: renderedVideo)
secondItem.seekingWaitsForVideoCompositionRendering = true //tried false as well
secondItem.audioTimePitchAlgorithm = AVAudioTimePitchAlgorithmVarispeed
secondItem.videoComposition = nil // tried AVComposition(propertiesOf: renderedVideo) as well
let player2 = AVPlayer(playerItem: secondItem)
player2.actionAtItemEnd = .none
secondPlayer.player = player2
I then have a start and end time to loop through these videos over and over. I don't use PlayerItemDidReachEnd because i'm not interested in the end, I'm interested in the user inputed time. I even use dispatchGroup to ENSURE that both players have finished seeking before trying to replay the video:
func playAllPlayersFromStart() {
let dispatchGroup = DispatchGroup()
dispatchGroup.enter()
firstPlayer.player?.currentItem?.seek(to: startTime, toleranceBefore: kCMTimeZero, toleranceAfter: kCMTimeZero, completionHandler: { _ in
dispatchGroup.leave()
})
DispatchQueue.global().async { [weak self] in
guard let startTime = self?.startTime else { return }
dispatchGroup.wait()
dispatchGroup.enter()
self?.secondPlayer.player?.currentItem?.seek(to: startTime, toleranceBefore: kCMTimeZero, toleranceAfter: kCMTimeZero, completionHandler: { _ in
dispatchGroup.leave()
})
dispatchGroup.wait()
DispatchQueue.main.async { [weak self] in
self?.firstPlayer.player?.play()
self?.secondPlayer.player?.play()
}
}
}
The strange part here is that the original asset, which has also been mapped via my compose() function loops perfectly fine. However, the renderedVideo which has also been run through the compose() function sometimes freezes when seeking during one of the CMTimeMapping segments. The only difference between the file that freezes and the file that doesnt freeze is that one has been exported to the NSTemporaryDirectory via the AVAssetExportSession to combine the two video tracks into one. They're both the same duration. I'm also sure that it's only the video layer that is freezing and not the audio, because if I add BoundaryTimeObservers to the player that freezes it still hits them and loops. Also the audio loops properly.
To me the strangest part is that the video 'resumes' if it makes it past the spot where it paused to start the seek after a 'freeze'. I've been stuck on this for days and would really love some guidance.
Other odd things to note:
- Even though the CMTimeMapping of the original versus the exported asset are the exact same durations, you'll notice that the rendered asset's slow motion ramp is more 'choppy' than the original.
- Audio continues when video freezes.
- video almost only ever freezes during slow motion sections (caused by CMTimeMapping objects based on segments
- rendered video seems to have to play 'catch up' at the beginning. even though i'm calling play after both have finished seeking, it seems to me that the right side plays faster in the beginning as a catch up. Strange part is that the segments are the exact same, just referencing two separate source files. One located in the asset library, the other in NSTemporaryDirectory
- It seems to me that AVPlayer and AVPlayerItemStatus is 'readyToPlay' before i call play.
- It seems to 'unfreeze' if the player proceeds PAST the point that it locked up.
- I tried to add observers for 'AVPlayerItemPlaybackDidStall' but it was never called.
Cheers!
The problem was in the AVAssetExportSession. To my surprise, changing exporter.canPerformMultiplePassesOverSourceMediaData = true fixed the issue. Although the documentation is quite sparse and even claims 'setting this property to true may have no effect', it did seem to fix the issue. Very, very, very strange! I consider this a bug and will be filing a radar. Here are the docs on the property: canPerformMultiplePassesOverSourceMediaData
It seems possible that in your playAllPlayersFromStart() method that the startTime variable may have changed between the two tasks dispatched (this would be especially likely if that value updates based on scrubbing).
If you make a local copy of startTime at the start of the function, and then use it in both blocks, you may have better luck.
I am currently working on HTTP Live streaming video with AVPlayerViewController / AVPlayer
i am play a video with .m38u file supported
It playing fine but my question is that can i have the data of video like i have set 4 type of resolution while genrating the .m38u file.. , i can varies the resoltion at my end point now the question is how to get the all values which i have setup at my endpoint. I am play this is in android also and using Track i am able to fetch all the video information but in ios how can i fetch all the details containing the video like its height , with, track,resolution supported by video etc..
I have search alot but could not get succeed..
Need help
Thanks in advance
Anita, I start I hope here is slice of code for playing a VIDEO..
self.command.text = "Loading VIDEO"
let videoURL = self.currentSlide.aURL
self.playerItem = AVPlayerItem(URL: videoURL)
self.player = AVPlayer(playerItem: self.playerItem)
self.playerLayer = AVPlayerLayer(player: self.player)
self.streamPlayer = AVPlayerViewController()
self.streamPlayer.player = self.player
self.streamPlayer.view.frame = CGRect(x: 128, y: 222, width: 512, height: 256)
let currentFM = self.streamPlayer.player?.currentItem?.asset
for blah in (currentFM?.metadata.enumerate())! {
print("blah \(blah)")
}
self.presentViewController(self.streamPlayer, animated: true)
self.streamPlayer.player!.play()
}
I added a little extra for showing meta data about the video... it printed...
blah (0, <AVMetadataItem: 0x15e7e4280, identifier=itsk/gsst, keySpace=itsk, key class = __NSCFNumber, key=gsst, commonKey=(null), extendedLanguageTag= (null), dataType=com.apple.metadata.datatype.UTF-8, time={INVALID}, duration= {INVALID}, startDate=(null), extras={
dataType = 1;
dataTypeNamespace = "com.apple.itunes";
}, value=0>)
blah (1, <AVMetadataItem: 0x15e7e1a50, identifier=itsk/gstd, keySpace=itsk, key class = __NSCFNumber, key=gstd, commonKey=(null), extendedLanguageTag=(null), dataType=com.apple.metadata.datatype.UTF-8, time={INVALID}, duration={INVALID}, startDate=(null), extras={
dataType = 1;
dataTypeNamespace = "com.apple.itunes";
}, value=222980>)
Hopefully it means a lot more to you than it does to me :) What your looking I think is classed/called metadata in Applespeak...
Take a look at this post too
Capture still image from AVCaptureSession in Swift
It describes how-to capture a frame of your video, once you have a frame you can take a closer look at its meta data and I suspect find out some of the details you seek.
let metadata = info[UIImagePickerControllerMediaMetadata] as? NSDictionary
let image = info[UIImagePickerControllerOriginalImage] as? UIImage
Are the commands to try and fetch that... let me know if you manage to succeed!
In my application I've to stream videos from server. For that I've used below Code
-(void)playingSong:(NSURL*) url{
AVAsset *asset = [AVAsset assetWithURL:url];
duration = asset.duration;
playerItem = [AVPlayerItem playerItemWithAsset:asset];
player = [AVPlayer playerWithPlayerItem:playerItem];
[player play];
}
All are Global Variables
It's playing all videos when network is good, but unable to play videos with big size, when network is slow.
Means It's not playing for big size videos and it's playing small videos;
I'm using http Server not https;
for ex : 3min video it's playing but for 1hr video it's not.
Why so?
Seems like you have to download the whole video before you can begin playback. It can also be because of your server not AVPlayer.
when you serve videos on a site using plain HTTP – known as
progressive download – the position of the header becomes very
important. Either the header is placed in the beginning of the file or
it’s places in the end of the file. In case of the latter, you’ll have
to download the whole thing before you can begin playback – because
without the header, the player can’t start decoding.
Have look at this guide if your problem is because of videos source.
Have a look at this thread and change you implementation accordingly.
Download video in local and then play in avplayer.
DispatchQueue.global(qos: .background).async {
do {
let data = try Data(contentsOf: url)
DispatchQueue.main.async {
// store "data" in document folder
let fileUrl = URL(fileURLWithPath: <#localVideoURL#>)
let asset = AVAsset(url: fileUrl)
let item = AVPlayerItem(asset: asset)
let player = AVPlayer(playerItem: item)
let layer = AVPlayerLayer(player: player)
layer.bounds = self.view.bounds
self.view.layer.addSublayer(layer)
player.play()
}
} catch {
print(error.localizedDescription)
}
}
NSDictionary* fileAttributes =
[[NSFileManager defaultManager] attributesOfItemAtPath:filename
error:nil]
From the file attribute keys, you can get the date, size, etc. But how do you get the duration?
In the 'File Attribute Keys' of the NSFileManager class reference you can see that there is no key to use that will return the duration of a song. All the information that the NSFileManager instance gets about a file is to do with the properties of the actual file itself within the operating system, such as its file-size. The NSFileManager doesn't actually interpret the file.
In order to get the duration of the file, you need to use a class that knows how to interpret the file. The AVFoundation framework provides the exact class you need, AVAsset. You can instantiate an instance of this abstract class using the concrete subclass AVURLAsset, and then provide it an NSURL which points to the audio file you wish to get the duration. You can then get the duration from the AVAsset instance by querying its duration property.
For example:
AVURLAsset* audioAsset = [AVURLAsset URLAssetWithURL:audioFileURL options:nil];
CMTime audioDuration = audioAsset.duration;
float audioDurationSeconds = CMTimeGetSeconds(audioDuration);
Note that AVFoundation is designed as a heavily asynchronous framework in order to improve performance and the overall user experience. Even performing simple tasks such as querying a media file's duration can potentially take a long period of time and can cause your application to hang. You should use the AVAsynchronousKeyValueLoading protocol to asynchronously load the duration of the song, and then update your UI in a completion handler block. You should take a look at the 'Block Programming Guide' as well as the WWDC2010 video titled, 'Discovering AV Foundation', which is available free at https://developer.apple.com/videos/wwdc/2010.
For anyone still looking for this.
Based on the answer, the code for Swift 4 (including the async loading of values taken from Apple's documentation):
let audioAsset = AVURLAsset.init(url: yourURL, options: nil)
audioAsset.loadValuesAsynchronously(forKeys: ["duration"]) {
var error: NSError? = nil
let status = audioAsset.statusOfValue(forKey: "duration", error: &error)
switch status {
case .loaded: // Sucessfully loaded. Continue processing.
let duration = audioAsset.duration
let durationInSeconds = CMTimeGetSeconds(duration)
print(Int(durationInSeconds))
break
case .failed: break // Handle error
case .cancelled: break // Terminate processing
default: break // Handle all other cases
}
}
You can achieve the same in Swift using :
let audioAsset = AVURLAsset.init(url: audioFileURL, options: nil)
let duration = audioAsset.duration
let durationInSeconds = CMTimeGetSeconds(duration)
For completeness - There is another way to get the duration for a mp3 file:
NSURL * pathToMp3File = ...
NSError *error = nil;
AVAudioPlayer* avAudioPlayer = [[AVAudioPlayer alloc]initWithContentsOfURL:pathToMp3File error:&error];
double duration = avAudioPlayer.duration;
avAudioPlayer = nil;
I have used this with no discernible delay.
Swift 5.0 + iOS 13: This is the only way it worked for me (#John Goodstadt solution in Swift). Currently I'm not sure why, but the there is a difference of average 0.2 seconds between a recorded audio file (in my case a voice memo) and the received audio file using the following code.
do {
let audioPlayer = try AVAudioPlayer(contentsOf: fileURL)
return CGFloat(audioPlayer.duration)
} catch {
assertionFailure("Failed crating audio player: \(error).")
return nil
}
If you can, try this newer method:
let asset = AVURLAsset(url: fileURL, options: nil)
// Returns a CMTime value.
let duration = try await asset.load(.duration)
// An array of AVMetadataItem for the asset.
let metadata = try await asset.load(.metadata)
// A CMTime value and an array of AVMetadataItem.
let (duration, metadata) = try await asset.load(.duration, .metadata)
Note: Loading several properties at the same time enables AVFoundation to optimize performance by batch-loading requests.
Here's a neat guide from Apple documentation Loading media data asynchronously.
I record a linear PCM file (.pcm) through AVAudioRecorder. I get the duration with the help of Farhad Malekpour. Maybe this can help you :
iPhone: get duration of an audio file
NSURL *fileUrl = [NSURL fileURLWithPath:yourFilePath];
AudioFileID fileID;
OSStatus result = AudioFileOpenURL((__bridge CFURLRef)fileUrl,kAudioFileReadPermission, 0, &fileID);
Float64 duration = 0; //seconds. the duration of the audio.
UInt32 ioDataSize = sizeof(Float64);
result = AudioFileGetProperty(fileID, kAudioFilePropertyEstimatedDuration,
&ioDataSize, &duration);
AudioFileClose(fileID);
if(0 == result) {
//success
}
else {
switch (result) {
case kAudioFileUnspecifiedError:{
//
} break;
// other cases...
default:
break;
}
}
Use AVAssetReader to get the duration of the audio file
guard let assetReader = try? AVAssetReader(asset: audioAsset) {
return nil
}
let duration = Double(assetReader.asset.duration.value)
let timescale = Double(assetReader.asset.duration.timescale)
let totalDuration = duration / timescale
print(totalDuration)
I actually found that you can just use the “get details from music” block and set it to “get duration of ” where can be an mp3 input from a prompt, a link to an mp3, an mp3 from the share sheet, etc