Does someone know how to use the AVAssetWriterInput init with more than 2 channels?
I'm trying to init an audioInput, to add it after on AVAssetWriter this way:
let audioInput = AVAssetWriterInput(mediaType: AVMediaTypeAudio, outputSettings: audioOutputSettings)
assetWriter.add(audioInput)
assetWriter.startWriting()
But it crashes when I init the audioInput with the audioOutputSettings dictionary containing the number of channels key greater than 2. The error is:
Terminating app due to uncaught exception ‘NSInvalidArgumentException’, reason: ’*** -[AVAssetWriterInput initWithMediaType:outputSettings:sourceFormatHint:] 6 is not a valid channel count for Format ID ‘aac ’. Use kAudioFormatProperty_AvailableEncodeNumberChannels (<AudioToolbox/AudioFormat.h>) to enumerate available channel counts for a given format.
As you found in the AVAssetWriterInput comment:
If AVNumberOfChannelsKey specifies a channel count greater than 2, the dictionary must also specify a value for AVChannelLayoutKey.
What it fails to mention is that the channel count depends on your format ID, so passing a AudioChannelLayout won't make AAC support anything other than 1 or 2 channels.
Formats that do support 6 channels include LPCM kAudioFormatLinearPCM and, probably more interestingly, High Efficiency AAC (kAudioFormatMPEG4AAC_HE) which supports 2, 4, 6 and 8 channel audio.
The following code creates an AVAssetWriterInput that is ready for 6 channel AAC HE sample buffers:
var channelLayout = AudioChannelLayout()
channelLayout.mChannelLayoutTag = kAudioChannelLayoutTag_MPEG_5_1_D
let audioOutputSettings: [String : Any] = [
AVNumberOfChannelsKey: 6,
AVFormatIDKey: kAudioFormatMPEG4AAC_HE,
AVSampleRateKey: 44100,
AVChannelLayoutKey: NSData(bytes: &channelLayout, length: MemoryLayout.size(ofValue: channelLayout)),
]
let audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioOutputSettings)
Change these two lines:
channelLayout.mChannelLayoutTag = kAudioChannelLayoutTag_MPEG_2_0
AVNumberOfChannelsKey : 2,
I hope it helps you in my code it worked.
Related
I have a sample project for resizing videos that works well for most videos. However, AVAssetWriter fails to write the audio from specific videos with the error:
Error Domain=AVFoundationErrorDomain
Code=-11800 "The operation could not be completed"
UserInfo={
NSLocalizedFailureReason=An unknown error occurred (-12780),
NSLocalizedDescription=The operation could not be completed,
NSUnderlyingError=0x282e956e0 {
Error Domain=NSOSStatusErrorDomain Code=-12780 "(null)"
}
}
What is even more problematic is that the same code works fine if I run it on macOS, but it breaks in iOS. I think it isn't a hardware problem because it also breaks in the iOS simulator.
These are the settings I use for (de)compressing the asset tracks:
func audioDecompressionSettings() -> [String: Any] {
return [
AVFormatIDKey: kAudioFormatLinearPCM
]
}
func audioCompressionSettings() -> [String: Any] {
var audioChannelLayout = AudioChannelLayout()
memset(&audioChannelLayout, 0, MemoryLayout<AudioChannelLayout>.size)
audioChannelLayout.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo
return [
AVFormatIDKey: kAudioFormatMPEG4AAC,
AVSampleRateKey: 44100,
AVEncoderBitRateKey: 128000,
AVNumberOfChannelsKey: 2,
AVChannelLayoutKey: NSData(bytes: &audioChannelLayout, length: MemoryLayout<AudioChannelLayout>.size)
]
}
func videoDecompressionSettings() -> [String: Any] {
return [
kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarFullRange,
kCVPixelBufferMetalCompatibilityKey as String: true
]
}
func videoCompressionSettings(size: CGSize) -> [String: Any] {
return [
AVVideoCodecKey: AVVideoCodecType.h264,
AVVideoWidthKey: size.width,
AVVideoHeightKey: size.height
]
}
The complete source code can be found here.
In that project there are two targets, one for Mac and other for iOS, both of them using the same code for resizing the video. I also included two sample video files: fruit.mp4 and rain.mp4. The first one works well in both targets, but the second one breaks in iOS.
Am I missing something here or this is likely to be an Apple bug?
The audio settings for the problematic video are:
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 386 kb/s (default)
and for the other one are:
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 137 kb/s (default)
The important difference between the two is the number of audio channels: 5.1 (5 full bandwidth channels + one low-frequency effects channel) in the first one and stereo (2) in the second one.
When reading the video file, we specify the decompression settings:
[AVFormatIDKey: kAudioFormatLinearPCM]
Which means that the decompressed audio will have the same number of channels as the source file. In our case, we have a 5.1 (actually 6) channels asset and we want to write it to a 2 channels file. It seems that AVAssetWriterInput doesn't handle that case properly in iOS and we get an error.
The solution to the problem is to specify the number of audio channels we want when decompressing the audio from the asset, like this:
[
AVFormatIDKey: kAudioFormatLinearPCM
AVNumberOfChannelsKey: 2
]
I am using AVAssetWriter to save screen recoding to mp4 (Withd H.264 video and aac audio encoding). Everything works perfectly as expected but some users complained about the 0KB video size issue, that is no data is being written to the output file.
By trying over and over again I was able to reproduce this issue. It happened after a few successful writes (sometimes after 2 sometimes after 5 and on some machines it never happens). I compared the successful flow and failure looked at the logs for it. The only thing which I found out was this
CMIO_Unit_Converter_Audio.cpp:588:RebuildAudioConverter AudioConverterSetProperty() failed (1886547824)
Where
kAudioFormatUnsupportedPropertyError = 1886547824
Which leads me to check the audio format in case of failure but the Audio format for AVAssetWriter was perfectly fine.
I am setting up audio writer input as
AVAssetWriter Inputs
[<AVAssetWriterInput: 0x600002047950, mediaType = vide, outputSettings = {
AVVideoCodecKey = avc1;
AVVideoHeightKey = 840;
AVVideoWidthKey = 1360;
}>, <AVAssetWriterInput: 0x60000205e740, mediaType = soun, outputSettings = {
AVFormatIDKey = 1633772320;
AVNumberOfChannelsKey = 2;
AVSampleRateKey = 44100;
}>]
My code to create AVAssetWriterInput for video and audio is as follow
Audio AVAssetWriterInput
settings = [
AVFormatIDKey : kAudioFormatMPEG4AAC,
AVNumberOfChannelsKey : ch,
AVSampleRateKey : rate,
]
audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: settings)
Video AVAssetWriterInput
var settings: [String : Any] = [
AVVideoWidthKey : cx,
AVVideoHeightKey : cy,
AVVideoCodecKey : AVVideoCodecType.h264,
]
videoInput = AVAssetWriterInput(mediaType: .video, outputSettings: settings)
I noticed error in the log appears after AVCaptureSession is being started and before AVCaptureVideoDataOutputSampleBufferDelegate's captureOutput. I logged AVCaptureSession inputs and output as well.
AVCaptureSession Inputs
[<AVCaptureScreenInput: 0x6000022e18e0>, <AVCaptureDeviceInput: 0x6000022e0a00 [Built-in Microphone]>]
AVCaptureSession Outputs
[<AVCaptureAudioDataOutput: 0x6000022e1f40>, <AVCaptureVideoDataOutput: 0x60000228b7a0>]
But again I don't see any problem here. I have been trying to figure this out for a good few days but failed to do so. I don't have any idea what triggers it and even when everything is properly set, it fails to write any output. This is only happening with video+audio if I write video only it works properly.
I want to record audio on iOS in real time, analyze the raw audio data and save parts of the recorded data. I'm recording the data with this code: https://gist.github.com/hotpaw2/ba815fc23b5d642705f2b1dedfaf0107
Now, my data is saved in a Float array and I want to save it to an audio file. I tried doing it with this code:
let fileMgr = FileManager.default
let dirPaths = fileMgr.urls(for: .documentDirectory, in: .userDomainMask)
let recordSettings = [AVEncoderAudioQualityKey: AVAudioQuality.min.rawValue,
AVEncoderBitRateKey: 16,
AVNumberOfChannelsKey: 2,
AVSampleRateKey: 44100] as [String: Any]
let soundFileUrl = dirPaths[0].appendingPathComponent("recording-" + getDate() + ".pcm")
do {
let audioFile = try AVAudioFile(forWriting: soundFileUrl, settings: recordSettings)
let format = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: 44100, channels: 2, interleaved: true)
let audioFileBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: 3000)
for i in 0..<circBuffer.count {
audioFileBuffer.int16ChannelData?.pointee[i] = Int16(circBuffer[i])
}
try audioFile.write(from: audioFileBuffer)
}
On the last line, I get an error which says:
ERROR: >avae> AVAudioFile.mm:306: -[AVAudioFile writeFromBuffer:error:]: error -50
amplitudeDemo(79264,0x70000f7cb000) malloc: * error for object 0x7fc5b9057e00: incorrect checksum for freed object - object was probably modified after being freed.
* set a breakpoint in malloc_error_break to debug
I already searched in a lot of other questions, but I couldn't find anything that helped me.
In your code at this line:
let audioFileBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: 3000)
You declare an AVAudioPCMBuffer with capacity 3000 frames * 2 channels, which means the allocated buffer for your audioFileBuffer can contain 6000 samples. If the index for the channel data exceeds this limit, your code destroys the nearby regions in the heap, which causes object was probably modified error.
So, your circBuffer.count may probably be exceeding this limit. You need to allocate enough buffer size, for the AVAudioPCMBuffer.
do {
//### You need to specify common format
let audioFile = try AVAudioFile(forWriting: soundFileUrl, settings: recordSettings, commonFormat: .pcmFormatInt16, interleaved: true)
let channels = 2
let format = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: 44100, channels: AVAudioChannelCount(channels), interleaved: true)
let audioFileBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(circBuffer.count / channels)) //<-allocate enough frames
//### `stride` removed as it seems useless...
let int16ChannelData = audioFileBuffer.int16ChannelData! //<-cannot be nil for the `format` above
//When interleaved, channel data of AVAudioPCMBuffer is not as described in its doc:
// https://developer.apple.com/reference/avfoundation/avaudiopcmbuffer/1386212-floatchanneldata .
// The following code is modified to work with actual AVAudioPCMBuffer.
//Assuming `circBuffer` as
// Interleaved, 2 channel, Float32
// Each sample is normalized to [-1.0, 1.0] (as usual floating point audio format)
for i in 0..<circBuffer.count {
int16ChannelData[0][i] = Int16(circBuffer[i] * Float(Int16.max))
}
//You need to update `frameLength` of the `AVAudioPCMBuffer`.
audioFileBuffer.frameLength = AVAudioFrameCount(circBuffer.count / channels)
try audioFile.write(from: audioFileBuffer)
} catch {
print("Error", error)
}
Some notes added as comments, please check them before trying this code.
UPDATE
Sorry, for showing untested code, two things fixed:
You need to specify commonFormat: when instantiating AVAudioFile.
int16ChannelData (and other channel data) does not return expected pointers as described in its documentation when interleaved, data filling loop modified to fit for the actual behaviour.
Please try.
You seem to have chosen the frameCapacity arbitrarily to 3000.
Set it to the actual sample count:
let audioFileBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(circBuffer.count))
I've created a function to compress a video file. It uses AVAssetWriter and adds inputs and outputs for video and audio tracks. When it starts writing I'm getting an error when the AVAssetReader for the audio track starts reading, audioReader.startReading(). Here the error, *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[AVAssetReader startReading] cannot be called again after reading has already started'.
The code: https://gist.github.com/jaumevn/9ba329aaf49c81c57a276fd135f53f20
Can anyone see what's the problem here? Thanks!
Line 77 of your code, you're starting a second AVAssetReader on the same file.
You don't need to hook up two readers, instead, you should hook up your AVAudioAssetReader as an Output for the existing AVAssetReader.
Something like this:
let videoReaderSettings : [String : Int] = [kCVPixelBufferPixelFormatTypeKey as String : Int(kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange)]
let videoReaderOutput = AVAssetReaderTrackOutput(track: videoAssetTrack, outputSettings: videoReaderSettings)
let videoReader = try! AVAssetReader(asset: videoAssetUrl)
var settings = [String : AnyObject]()
settings[AVFormatIDKey] = Int(kAudioFormatLinearPCM)
let audioReaderOutput = AVAssetReaderTrackOutput(track: audioAssetTrack, outputSettings: settings)
videoReader.addOutput(videoReaderOutput)
videoReader.addOutput(audioReaderOutput)
videoWriter.startWriting()
videoReader.startReading()
Look into using AVCaptureVideoDataOutputSampleBufferDelegate and AVCaptureAudioDataOutputSampleBufferDelegate to capture and process the buffers from the reader.
I'm trying to use AVAudioEngine to record sounds from the microphone together with various sound effect files to a AVAudioFile.
I create an AVAudioFile like this:
let settings = self.engine.mainMixerNode.outputFormatForBus(0).settings
try self.audioFile = AVAudioFile(forWriting: self.audioURL, settings: settings, commonFormat: .PCMFormatFloat32, interleaved: false)
I install a tap on the audio engine's mainMixerNode, where I write the buffer to the file:
self.engine.mainMixerNode.installTapOnBus(0, bufferSize: 4096, format: self.engine.mainMixerNode.outputFormatForBus(0)) { (buffer, time) -> Void in
do {
try self.audioFile?.writeFromBuffer(buffer)
} catch let error as NSError {
NSLog("Error writing %#", error.localizedDescription)
}
}
I'm using self.engine.mainMixerNode.outputFormatForBus(0).settingswhen creating the audio file since Apple states that "The buffer format MUST match the file's processing format which is why outputFormatForBus: was used when creating the AVAudioFile object above". In the documentation for installTapOnBus they also say this: " The tap and connection formats (if non-nil) on the specified bus should be identical"
However, this gives me a very large, uncompressed audio file. I want to save the file as .m4a but don't understand where to specify the settings I want to use:
[
AVFormatIDKey: NSNumber(unsignedInt: kAudioFormatMPEG4AAC),
AVSampleRateKey : NSNumber(double: 32000.0), //44100.0
AVNumberOfChannelsKey: NSNumber(int: 1),
AVEncoderBitRatePerChannelKey: NSNumber(int: 16),
AVEncoderAudioQualityKey: NSNumber(int: Int32(AVAudioQuality.High.rawValue))
]
If I pass in these settings instead when creating the audio file, the app crashes when I record.
Any suggestions or ideas on how to solve this?