Objective: Dialog Flow Voice Bot Api
I need to send a wav file to the Dialog Flow Api and the format and settings were pre-defined.
So I recorded an audio using AVAudioRecorder in .wav format using
following settings
audioFilename = getDocumentsDirectory().appendingPathComponent("input.wav")
let settings: [String: Any] = [
AVFormatIDKey: Int(kAudioFormatLinearPCM),
AVSampleRateKey: 16000,
AVNumberOfChannelsKey: 2,
AVLinearPCMBitDepthKey: 16,
AVLinearPCMIsBigEndianKey: false,
AVEncoderAudioQualityKey: AVAudioQuality.max.rawValue
]
do {
audioRecorder = try AVAudioRecorder(url: audioFilename!, settings: settings)
audioRecorder.isMeteringEnabled = true
audioRecorder.prepareToRecord()
audioRecorder.delegate = self
audioRecorder.record()
recordButton.setTitle("Tap to Stop", for: .normal)
} catch {
print(error.localizedDescription)
finishRecording(success: false)
}
}
Then I tried to convert it into Base64 audio format
let outputFile = try Data.init(contentsOf: fileUrl)
let base64String = outputFile.base64EncodedString(options: NSData.Base64EncodingOptions.init(rawValue: 0))
print(base64String)
So whenever I try to decode that encoded string, using an online converter, it displays some corrupted bytes
Thoughts??
So I've found the answer to the question.
The reason my byte array wasn't able to maintain correct headers was because of a key which I omitted in the settings variable
AVAudioFileTypeKey: kAudioFileWAVEType
let settings: [String: Any] = [
AVSampleRateKey: 16000,
AVNumberOfChannelsKey: 1,
AVAudioFileTypeKey: kAudioFileWAVEType, //MANDATORY
AVFormatIDKey: kAudioFormatLinearPCM,
AVLinearPCMIsBigEndianKey: false,
AVLinearPCMIsNonInterleaved: true,
AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue
]
It was given in the docs that if you won't provide the settings i.e.
audioRecorder = try AVAudioRecorder(url: audioFilename!, settings: [:] /*empty settings*/)
then
❌ AVAudio recorder will automatically prepare the file from the Format defined in the file. ❌
But turns out, that didn't help either 😫
So whilst I was playing with the settings, I found this very important key AVAudioFileTypeKey, which helped in maintaining the correct headers and thus a valid .wav file 😎
This is how a wav file with Valid headers look like
Related
I try put few exif parameters to image, but only few of them are visible.
When check information about exifDictionary all parametras has a value
[1]: https://i.stack.imgur.com/4OirK.png
But when check this file in exiftool, BodySerialNumber are valid, FileSource is incorrect value, and DateTimeOriginal is not visible, really don't understand what's going on
[2]: https://i.stack.imgur.com/mf4l4.png
This is my code, where i try to save meta to file
static func encodeImage(at url: URL, file: File, completionHandler: ((URL?) -> Void)?) {
let documentsDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] as URL
let filePath = documentsDirectory.appendingPathComponent("\(UUID().uuidString)\(file.fileExtension ?? ".jpg")")
guard let data = NSData(contentsOf: url),
let src = CGImageSourceCreateWithData(data, nil),
let uti = CGImageSourceGetType(src),
let cfPath = CFURLCreateWithFileSystemPath(nil, filePath.path as CFString, CFURLPathStyle.cfurlposixPathStyle, false),
let dest = CGImageDestinationCreateWithURL(cfPath, uti, 1, nil)
else {
completionHandler?(nil)
return
}
let exifProperties = [
kCGImagePropertyExifBodySerialNumber as String: file.device ?? "0",
kCGImagePropertyExifFileSource as String: file.deviceUrl ?? file.path,
kCGImagePropertyExifDateTimeOriginal as String: "\(file.createdAt.milliseconds)"
] as CFDictionary
let exifDictionary = [
kCGImagePropertyExifDictionary as String: exifProperties
] as CFDictionary
CGImageDestinationAddImageFromSource(dest, src, .zero, exifDictionary)
if CGImageDestinationFinalize(dest) {
completionHandler?(filePath)
} else {
completionHandler?(nil)
}
}
How can I fix it ?
Community wiki
The main problem because in Java / Kotlin, they can programatically write what they want in meta keys, maybe they use another version of exif, I really don't know about it, but in Swift you can use special format to key. All keys described in https://www.awaresystems.be/imaging/tiff/tifftags/privateifd/exif.html
And when I try to write String in kCGImagePropertyExifFileSource field I have 0, because this field must outcome integer value 1 ... 3
The same problem with kCGImagePropertyExifDateTimeOriginal here not only a String must be written, but also a special format YYYY: MM: DD HH: MM: SS, other formats cannot be written into this field !!!
What I mean, read about field formats and types!)
What is the process of generating .mov or .m4a file using arrays of Int16 as sterio channel for audio?
I can easily generate raw PCM data as [Int16] from a .mov file and store it in two files leftChannel.pcm and rightChannel.pcm and perform some operations for later use. But I am not able to regenerate the video from these files.
Any process, i.e. direct video generation using raw PCM or using intermediate step of generating m4a from PCM will work.
Update:
I figured out how to convert the PCM array to audio file. But it won't play.
private func convertToM4a(leftChannel leftPath : URL, rightChannel rigthPath : URL, converterCallback : ConverterCallback){
let m4aUrl = FileManagerUtil.getTempFileName(parentFolder: FrameExtractor.PCM_ENCODE_FOLDER, fileNameWithExtension: "encodedAudio.m4a")
if FileManager.default.fileExists(atPath: m4aUrl.path) {
try! FileManager.default.removeItem(atPath: m4aUrl.path)
}
do{
let leftBuffer = try NSArray(contentsOf: leftPath, error: ()) as! [Int16]
let rightBuffer = try NSArray(contentsOf: rigthPath, error: ()) as! [Int16]
let sampleRate = 44100
let channels = 2
let frameCapacity = (leftBuffer.count + rightBuffer.count)/2
let outputSettings = [
AVFormatIDKey : NSInteger(kAudioFormatMPEG4AAC),
AVSampleRateKey : NSInteger(sampleRate),
AVNumberOfChannelsKey : NSInteger(channels),
AVAudioFileTypeKey : NSInteger(kAudioFileAAC_ADTSType),
AVLinearPCMIsBigEndianKey : true,
] as [String : Any]
let audioFile = try AVAudioFile(forWriting: m4aUrl, settings: outputSettings, commonFormat: .pcmFormatInt16, interleaved: false)
let format = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: Double(sampleRate), channels: AVAudioChannelCount(channels), interleaved: false)!
let pcmBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(frameCapacity))!
pcmBuffer.frameLength = pcmBuffer.frameCapacity
for i in 0..<leftBuffer.count {
pcmBuffer.int16ChannelData![0][i] = leftBuffer[i]
}
for i in 0..<rightBuffer.count {
pcmBuffer.int16ChannelData![1][i] = rightBuffer[i]
}
try! audioFile.write(from: pcmBuffer)
converterCallback.m4aEncoded(to: m4aUrl)
} catch {
print(error.localizedDescription)
}
}
Saving it as .m4a with AVAudioFileTypeKey as m4a type was giving malformed file error.
Saving it as .aac with above settings plays the file but with broken sound. Just the buzzing sound with some slow mo effect of the original audio, initially I thought that it is something to do with the input and output of sampling rate but that was not the case.
I assume that something is wrong in Output Dictionary. Any help would be appreciated.
At least the creation of the AAC file with the code you are showing works.
I wrote out two NSArrays with valid Int16 audio data and with your code get a valid result that e.g. when played with (using suffix .aac) in QuickTime Player sounds the same as the input.
How are you creating the input?
Buzzing sound (with lots of noise) is e.g. happening if you reading in audio data using AVAudioFormat with e.g. .pcmFormatInt16 format but the data actually read is in .pcmFormatFloat32 format (most commonly default format). There is unfortunately no runtime warning if you try to do so.
If that's the case try to use .pcmFormatFloat32. If you need it in Int16 you can convert it yourself by basically mapping [-1,1] to [-32768,32767] for both channels.
let fac = Float(1 << 15)
for i in 0..<count {
let val = min(max(inBuffer!.floatChannelData![ch][i] * fac, -fac), fac - 1)
xxx[I] = Int16(val)
}
...
In my app I am using SecKeyGeneratePair to generate RSA key pair. After releasing the app, I started to notice occasional errSecInteractionNotAllowed errors (currently very rare) when using this function, so far only on iOS 10 devices. It is unclear to me why the key pair generation failed, or what I should do to fix that. Also, I could not find any documentation as to why key pair generation should fail with this error.
This is the code I used to generate the key pair:
guard let access = SecAccessControlCreateWithFlags(nil,
kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly,
[],
nil) else {throw Error(description: "Failed to create access control")}
let privateAttributes = [String(kSecAttrIsPermanent): true,
String(kSecAttrApplicationTag): keyTag + self.privateKeyExtension,
String(kSecAttrAccessControl): access] as [String : Any]
let publicAttributes = [String(kSecAttrIsPermanent): true,
String(kSecAttrApplicationTag): keyTag + self.publicKeyExtension] as [String : Any]
let pairAttributes = [String(kSecAttrKeyType): kSecAttrKeyTypeRSA,
String(kSecAttrKeySizeInBits): self.rsaKeySize,
String(kSecPublicKeyAttrs): publicAttributes,
String(kSecPrivateKeyAttrs): privateAttributes] as [String : Any]
var pubKey, privKey: SecKey?
let status = SecKeyGeneratePair(pairAttributes as CFDictionary, &pubKey, &privKey)
After this code, I am checking the status, and if it is not errSecSuccess, I am logging an error with the status returned from the function. This is where I noticed the errSecInteractionNotAllowed error.
So, why does key pair generation or what I could do in order to fix it?
Thanks,
Omer
Two suggestions:
Try to add an other protection class like kSecAttrAccessibleAlways to your call of SecAccessControlCreateWithFlags and test if the behavior still occurs.
Further define a flag for your use case instead of passing an empty array. E.g. userPresence.
Additionally I stumbled across this SO post, maybe you can find some inspiration there.
After discussing this with Apple Developer Support, here is the solution:
let privateAttributes = [String(kSecAttrIsPermanent): true,
String(kSecAttrApplicationTag): keyTag + self.privateKeyExtension,
String(kSecAttrAccessible): kSecAttrAccessibleAlways] as [String : Any]
let publicAttributes = [String(kSecAttrIsPermanent): true,
String(kSecAttrApplicationTag): keyTag + self.publicKeyExtension,
String(kSecAttrAccessible): kSecAttrAccessibleAlways] as [String : Any]
let pairAttributes = [String(kSecAttrKeyType): kSecAttrKeyTypeRSA,
String(kSecAttrKeySizeInBits): self.rsaKeySize,
String(kSecPublicKeyAttrs): publicAttributes,
String(kSecPrivateKeyAttrs): privateAttributes] as [String : Any]
var pubKey, privKey: SecKey?
let status = SecKeyGeneratePair(pairAttributes as CFDictionary, &pubKey, &privKey)
The important part is the kSecAttrAccessible, choose the value that matches your needs from this list. Notice that some of the values will limit the access to the key in KeyVault.
I have searched everywhere for this and i couldn't find proper way of doing it. I have succeeded in recording in .wav format, but the problem is, when i try reading raw data from recorded .wav file, some chunks are in wrong place/aren't there at all.
My code for recording audio:
func startRecording(){
let audioSession = AVAudioSession.sharedInstance()
try! audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord)
try! audioSession.setActive(true)
audioSession.requestRecordPermission({(allowed: Bool) -> Void in print("Accepted")} )
let settings: [String : AnyObject] = [
AVFormatIDKey:Int(kAudioFormatLinearPCM),
AVSampleRateKey:44100.0,
AVNumberOfChannelsKey:1,
AVLinearPCMBitDepthKey:8,
AVLinearPCMIsFloatKey:false,
AVLinearPCMIsBigEndianKey:false,
AVEncoderAudioQualityKey:AVAudioQuality.Max.rawValue
]
let date = NSDate()
let df = NSDateFormatter()
df.dateFormat = "yyyy-MM-dd-HH:mm:ss"
let dfString = df.stringFromDate(date)
let fullPath = documentsPath.stringByAppendingString("/\(dfString).wav")
recorder = try! AVAudioRecorder(URL: NSURL(string: fullPath)!, settings: settings)
recorder.delegate = self
recorder.prepareToRecord()
recorder.record()
}
When i print out data of recorder audio file, i get weird number where 'd' 'a' 't' 'a' should be written, following by zeros. And then, in middle of of data, it appears.
No 64617461 ('d' 'a' 't' 'a') chunk - it should be in place of 464c4c52
64617461 ('d' 'a' 't' 'a') at random spot after a lot of zeros
Is there better way of recording wav file? I am not sure why is this happening, so any help would be appreciated. Should i maybe record in other format then convert it to raw?
Thanks and sorry for so many images.
I think only the fmt chunk is guaranteed to come first. It looks like it's fine to have other chunks before the data chunk, so just skip over non-data chunks.
From http://soundfile.sapp.org/doc/WaveFormat/
A RIFF file starts out with a file header followed by a sequence of data chunks.
You need to update your parser :)
Here is my code for recording audio in my iOS8 Swift app:
var fileName = "/SFRecording-" + String(recordingSequence) + ".caf"
var str = storageLocation + fileName
var url = NSURL.fileURLWithPath(str as String)
audioSession.setCategory(AVAudioSessionCategoryRecord, error: nil)
audioSession.setActive(true, error: nil)
var recordSettings = [
AVFormatIDKey:kAudioFormatAppleIMA4,
AVSampleRateKey:44100.0,
AVNumberOfChannelsKey:2,
AVEncoderBitRateKey:12800,
AVLinearPCMBitDepthKey:16,
AVEncoderAudioQualityKey:AVAudioQuality.Max.rawValue
]
var error: NSError?
realRecorder = AVAudioRecorder(URL:url, settings: recordSettings as [NSObject : AnyObject], error: &error)
It works fine but the resultant CAF file is useless on windows systems. I wanted to record in a more familiar format like MP3 but turns out you cannot in iOS due to licensing issues.
Now I want to record in AAC format for which I have switched the file extension from .CAF to .AAC in above code and also switched value in AVFormatIDKey:kAudioFormatAppleIMA4 to kAudioFormatMPEG4AAC but those settings fail to record anything. Am I suppose to change some other setting too to make AAC recording work?
Remember my objective is to record in a format which is readily playable on mac/win/browser
I figured it out myself. I had to remove below lines for .AAC format to work:
AVSampleRateKey:44100.0,
AVNumberOfChannelsKey:2,
AVEncoderBitRateKey:12800,
AVLinearPCMBitDepthKey:16,