I am using AVAssetWriter to save screen recoding to mp4 (Withd H.264 video and aac audio encoding). Everything works perfectly as expected but some users complained about the 0KB video size issue, that is no data is being written to the output file.
By trying over and over again I was able to reproduce this issue. It happened after a few successful writes (sometimes after 2 sometimes after 5 and on some machines it never happens). I compared the successful flow and failure looked at the logs for it. The only thing which I found out was this
CMIO_Unit_Converter_Audio.cpp:588:RebuildAudioConverter AudioConverterSetProperty() failed (1886547824)
Where
kAudioFormatUnsupportedPropertyError = 1886547824
Which leads me to check the audio format in case of failure but the Audio format for AVAssetWriter was perfectly fine.
I am setting up audio writer input as
AVAssetWriter Inputs
[<AVAssetWriterInput: 0x600002047950, mediaType = vide, outputSettings = {
AVVideoCodecKey = avc1;
AVVideoHeightKey = 840;
AVVideoWidthKey = 1360;
}>, <AVAssetWriterInput: 0x60000205e740, mediaType = soun, outputSettings = {
AVFormatIDKey = 1633772320;
AVNumberOfChannelsKey = 2;
AVSampleRateKey = 44100;
}>]
My code to create AVAssetWriterInput for video and audio is as follow
Audio AVAssetWriterInput
settings = [
AVFormatIDKey : kAudioFormatMPEG4AAC,
AVNumberOfChannelsKey : ch,
AVSampleRateKey : rate,
]
audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: settings)
Video AVAssetWriterInput
var settings: [String : Any] = [
AVVideoWidthKey : cx,
AVVideoHeightKey : cy,
AVVideoCodecKey : AVVideoCodecType.h264,
]
videoInput = AVAssetWriterInput(mediaType: .video, outputSettings: settings)
I noticed error in the log appears after AVCaptureSession is being started and before AVCaptureVideoDataOutputSampleBufferDelegate's captureOutput. I logged AVCaptureSession inputs and output as well.
AVCaptureSession Inputs
[<AVCaptureScreenInput: 0x6000022e18e0>, <AVCaptureDeviceInput: 0x6000022e0a00 [Built-in Microphone]>]
AVCaptureSession Outputs
[<AVCaptureAudioDataOutput: 0x6000022e1f40>, <AVCaptureVideoDataOutput: 0x60000228b7a0>]
But again I don't see any problem here. I have been trying to figure this out for a good few days but failed to do so. I don't have any idea what triggers it and even when everything is properly set, it fails to write any output. This is only happening with video+audio if I write video only it works properly.
Related
I have a sample project for resizing videos that works well for most videos. However, AVAssetWriter fails to write the audio from specific videos with the error:
Error Domain=AVFoundationErrorDomain
Code=-11800 "The operation could not be completed"
UserInfo={
NSLocalizedFailureReason=An unknown error occurred (-12780),
NSLocalizedDescription=The operation could not be completed,
NSUnderlyingError=0x282e956e0 {
Error Domain=NSOSStatusErrorDomain Code=-12780 "(null)"
}
}
What is even more problematic is that the same code works fine if I run it on macOS, but it breaks in iOS. I think it isn't a hardware problem because it also breaks in the iOS simulator.
These are the settings I use for (de)compressing the asset tracks:
func audioDecompressionSettings() -> [String: Any] {
return [
AVFormatIDKey: kAudioFormatLinearPCM
]
}
func audioCompressionSettings() -> [String: Any] {
var audioChannelLayout = AudioChannelLayout()
memset(&audioChannelLayout, 0, MemoryLayout<AudioChannelLayout>.size)
audioChannelLayout.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo
return [
AVFormatIDKey: kAudioFormatMPEG4AAC,
AVSampleRateKey: 44100,
AVEncoderBitRateKey: 128000,
AVNumberOfChannelsKey: 2,
AVChannelLayoutKey: NSData(bytes: &audioChannelLayout, length: MemoryLayout<AudioChannelLayout>.size)
]
}
func videoDecompressionSettings() -> [String: Any] {
return [
kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarFullRange,
kCVPixelBufferMetalCompatibilityKey as String: true
]
}
func videoCompressionSettings(size: CGSize) -> [String: Any] {
return [
AVVideoCodecKey: AVVideoCodecType.h264,
AVVideoWidthKey: size.width,
AVVideoHeightKey: size.height
]
}
The complete source code can be found here.
In that project there are two targets, one for Mac and other for iOS, both of them using the same code for resizing the video. I also included two sample video files: fruit.mp4 and rain.mp4. The first one works well in both targets, but the second one breaks in iOS.
Am I missing something here or this is likely to be an Apple bug?
The audio settings for the problematic video are:
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 386 kb/s (default)
and for the other one are:
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 137 kb/s (default)
The important difference between the two is the number of audio channels: 5.1 (5 full bandwidth channels + one low-frequency effects channel) in the first one and stereo (2) in the second one.
When reading the video file, we specify the decompression settings:
[AVFormatIDKey: kAudioFormatLinearPCM]
Which means that the decompressed audio will have the same number of channels as the source file. In our case, we have a 5.1 (actually 6) channels asset and we want to write it to a 2 channels file. It seems that AVAssetWriterInput doesn't handle that case properly in iOS and we get an error.
The solution to the problem is to specify the number of audio channels we want when decompressing the audio from the asset, like this:
[
AVFormatIDKey: kAudioFormatLinearPCM
AVNumberOfChannelsKey: 2
]
Does someone know how to use the AVAssetWriterInput init with more than 2 channels?
I'm trying to init an audioInput, to add it after on AVAssetWriter this way:
let audioInput = AVAssetWriterInput(mediaType: AVMediaTypeAudio, outputSettings: audioOutputSettings)
assetWriter.add(audioInput)
assetWriter.startWriting()
But it crashes when I init the audioInput with the audioOutputSettings dictionary containing the number of channels key greater than 2. The error is:
Terminating app due to uncaught exception ‘NSInvalidArgumentException’, reason: ’*** -[AVAssetWriterInput initWithMediaType:outputSettings:sourceFormatHint:] 6 is not a valid channel count for Format ID ‘aac ’. Use kAudioFormatProperty_AvailableEncodeNumberChannels (<AudioToolbox/AudioFormat.h>) to enumerate available channel counts for a given format.
As you found in the AVAssetWriterInput comment:
If AVNumberOfChannelsKey specifies a channel count greater than 2, the dictionary must also specify a value for AVChannelLayoutKey.
What it fails to mention is that the channel count depends on your format ID, so passing a AudioChannelLayout won't make AAC support anything other than 1 or 2 channels.
Formats that do support 6 channels include LPCM kAudioFormatLinearPCM and, probably more interestingly, High Efficiency AAC (kAudioFormatMPEG4AAC_HE) which supports 2, 4, 6 and 8 channel audio.
The following code creates an AVAssetWriterInput that is ready for 6 channel AAC HE sample buffers:
var channelLayout = AudioChannelLayout()
channelLayout.mChannelLayoutTag = kAudioChannelLayoutTag_MPEG_5_1_D
let audioOutputSettings: [String : Any] = [
AVNumberOfChannelsKey: 6,
AVFormatIDKey: kAudioFormatMPEG4AAC_HE,
AVSampleRateKey: 44100,
AVChannelLayoutKey: NSData(bytes: &channelLayout, length: MemoryLayout.size(ofValue: channelLayout)),
]
let audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioOutputSettings)
Change these two lines:
channelLayout.mChannelLayoutTag = kAudioChannelLayoutTag_MPEG_2_0
AVNumberOfChannelsKey : 2,
I hope it helps you in my code it worked.
I've been scratching my head around this a full day now and don't seem to get closer, so I hope you guys can guide me in the right path :)
Heres's the situation.
I have a AVCaptureSession properly initialized where I add the audio input as follows :
let audioDevice = AVCaptureDevice.defaultDeviceWithMediaType(AVMediaTypeAudio)
let audioIn = try! AVCaptureDeviceInput(device: audioDevice)
if (session.canAddInput(audioIn)) {
session.addInput(audioIn)
}
The audio output is added as follows :
if session.canAddOutput(self.audioOutput) {
self.audioOutput = AVCaptureAudioDataOutput()
session.addOutput(self.audioOutput)
self.audioConnection: AVCaptureConnection = self.audioOutput.connectionWithMediaType(AVMediaTypeAudio)
}
I then setup the recording settings :
if let audioAssetWriterOutput = self.audioOutput.recommendedAudioSettingsForAssetWriterWithOutputFileType(AVFileTypeAppleM4A) {
return audioAssetWriterOutput as? [String: AnyObject]
}
which I assign to my AVAssetWriterInput that is initialized in audio mode, with those settings and the correct format description.
_audioInput = AVAssetWriterInput(mediaType: AVMediaTypeAudio, outputSettings: audioSettings, sourceFormatHint: audioFormatDescription)
_audioInput!.expectsMediaDataInRealTime = true
then I just simply start the AVCaptureSession via startRunning() that will capture the audio data into a .M4A file.
Everything is fine during the audio capture, here are the observations that I made :
The file recorded exists on disk as expected
The file can be played by any player : my mac, my iphone (I imported it via iTunes), seems all good.
The file is at the correct location when I setup my AVAudioPlayer.
Tried to initialize the AVAudioPlayer with NSData or NSURL, same result
Now later in my code, I try to read that audio file via an AVAudioPlayer :
self.audioPlayer = [[AVAudioPlayer alloc] initWithContentsOfURL:[NSURL fileURLWithPath:pathLink] error:&error];
I get thrown an error
Error Domain=NSOSStatusErrorDomain Code=1954115647 "(null)"
which after double checking is "Unsupported type".
What's incorrect in my setup ? Does this come from my AVCaptureSession or can something be wrong with my AVAudioPlayer setup ?
Thanks !
I'm trying to use AVAudioEngine to record sounds from the microphone together with various sound effect files to a AVAudioFile.
I create an AVAudioFile like this:
let settings = self.engine.mainMixerNode.outputFormatForBus(0).settings
try self.audioFile = AVAudioFile(forWriting: self.audioURL, settings: settings, commonFormat: .PCMFormatFloat32, interleaved: false)
I install a tap on the audio engine's mainMixerNode, where I write the buffer to the file:
self.engine.mainMixerNode.installTapOnBus(0, bufferSize: 4096, format: self.engine.mainMixerNode.outputFormatForBus(0)) { (buffer, time) -> Void in
do {
try self.audioFile?.writeFromBuffer(buffer)
} catch let error as NSError {
NSLog("Error writing %#", error.localizedDescription)
}
}
I'm using self.engine.mainMixerNode.outputFormatForBus(0).settingswhen creating the audio file since Apple states that "The buffer format MUST match the file's processing format which is why outputFormatForBus: was used when creating the AVAudioFile object above". In the documentation for installTapOnBus they also say this: " The tap and connection formats (if non-nil) on the specified bus should be identical"
However, this gives me a very large, uncompressed audio file. I want to save the file as .m4a but don't understand where to specify the settings I want to use:
[
AVFormatIDKey: NSNumber(unsignedInt: kAudioFormatMPEG4AAC),
AVSampleRateKey : NSNumber(double: 32000.0), //44100.0
AVNumberOfChannelsKey: NSNumber(int: 1),
AVEncoderBitRatePerChannelKey: NSNumber(int: 16),
AVEncoderAudioQualityKey: NSNumber(int: Int32(AVAudioQuality.High.rawValue))
]
If I pass in these settings instead when creating the audio file, the app crashes when I record.
Any suggestions or ideas on how to solve this?
I am using AVFoundation to record a video and then upload to a server, and use AVPlayer to streaming it to play, but always get these 2 error notifications, is there any thing wrong in my output settings
NSDictionary *settings = #{AVVideoCodecKey : AVVideoCodecH264,
AVVideoWidthKey : #(480),
AVVideoHeightKey : #(480),
AVVideoScalingModeKey : AVVideoScalingModeResizeAspectFill,
AVVideoCompressionPropertiesKey:#{
AVVideoAverageBitRateKey : #(480*480*11.4),
AVVideoMaxKeyFrameIntervalKey : #(24)
}
};
But I can play it very well with AVPlayer locally (means in iPhone)