iOS StreamingKit how to record/save the audio? - ios

I am using https://github.com/tumtumtum/StreamingKit
for streaming a live url. works perfectly fine. I want to add recording/save of audio feature to my app. Does anyone know if this library can do this?
If not, are there any alternatives? NOTE that I need to record LIVE streaming audio, not local file / static url.
The page shows that you can Intercept PCM data just before its played:
[audioPlayer appendFrameFilterWithName:#"MyCustomFilter" block:^(UInt32 channelsPerFrame, UInt32 bytesPerFrame, UInt32 frameCount, void* frames)
{
...
}];
However, I am not sure how to convert this into actual recording / mp3 file or even intercept the actual data from this?

You can do something like this, although StreamingKit seems a little bit secretive about the format of the samples it gives you. What's the sample rate? Floats or ints? I suppose you can guess from the sample size. This example assumes 16bit ints.
NSURL *dstUrl = [[NSURL fileURLWithPath:NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES)[0] ] URLByAppendingPathComponent:#"output.m4a"];
NSLog(#"write to %#", dstUrl);
__block AVAudioFile *audioFile = nil;
[audioPlayer appendFrameFilterWithName:#"MyCustomFilter" block:^(UInt32 channelsPerFrame, UInt32 bytesPerFrame, UInt32 frameCount, void* frames)
{
NSError *error;
// what's the sample rate? StreamingKit doesn't seem to tell us
double sampleRate = 44100;
if (!audioFile) {
NSDictionary *settings =
#{
AVFormatIDKey : #(kAudioFormatMPEG4AAC),
AVSampleRateKey : #(sampleRate),
AVNumberOfChannelsKey : #(channelsPerFrame),
};
// need commonFormat?
audioFile = [[AVAudioFile alloc] initForWriting:dstUrl settings:settings commonFormat:AVAudioPCMFormatInt16 interleaved:YES error:&error];
if (!audioFile) {
// error
}
}
AVAudioFormat *format = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatInt16 sampleRate:sampleRate channels:channelsPerFrame interleaved:YES];
AVAudioPCMBuffer *buffer = [[AVAudioPCMBuffer alloc] initWithPCMFormat:format frameCapacity:frameCount];
buffer.frameLength = frameCount;
memmove(buffer.int16ChannelData[0], frames, frameCount*bytesPerFrame);
if (![audioFile writeFromBuffer:buffer error:&error]) {
NSLog(#"write error: %#", error);
}
}];
[self.audioPlayer performSelector:#selector(removeFrameFilterWithName:) withObject:#"MyCustomFilter" afterDelay:10];

Related

iOS: Get Audio from AVAudioRecorder

Disclaimer New to AVAudioRecorder
What I'm doing I'm working on an app that uses the iPhone microphone to record sound. After the sound is recorded, I need to convert the sound (should be AVAsset, right?) into NSData to send to our backend.
What's the issue The issue is I am not sure how to "get" the audio that is supposed to be recorded with the AVAudioRecorder. AVAudioRecorder has a delegate method called - (void)audioRecorderDidFinishRecording:(AVAudioRecorder *) aRecorder successfully:(BOOL)flag. I would have expected the actually AVAsset that contains the audio to be passed from this delegate method, but it does not. What it does give me is the aRecorder object that has a .url property on it. When I NSLog the url from the passed aRecorder, it shows up. In fact I can NSLog the length of the file in the code below:
- (void)audioRecorderDidFinishRecording:(AVAudioRecorder *) aRecorder successfully:(BOOL)flag
{
DLog (#"audioRecorderDidFinishRecording:successfully: %#",aRecorder);
AVURLAsset* audioAsset = [AVURLAsset URLAssetWithURL:aRecorder.url options:nil];
CMTime audioDuration = audioAsset.duration;
float audioDurationSeconds = CMTimeGetSeconds(audioDuration);
NSLog(#"asset length = %f", audioDurationSeconds); //Logs 7.051 seconds, so I know it's "there".
self.audioURL = aRecorder.url;
}
Problem When I pass self.audioURL to the next viewController's self.mediaURL and try to grab the file from the AssetLibrary (similarly to how I did before), the asset is not returned from the AssetLibrary (even though when I po self.mediaURL it indeed logs the correct url:
if (self.mediaURL) {
ALAssetsLibrary *assetLibrary = [[ALAssetsLibrary alloc] init];
[assetLibrary assetForURL:self.mediaURL resultBlock:^(ALAsset *asset) {
if (asset) {
// This block does NOT get called...
ALAssetRepresentation *rep = [asset defaultRepresentation];
Byte *buffer = (Byte*)malloc((long)rep.size);
NSUInteger buffered =[rep getBytes:buffer fromOffset:0.0 length:(long)rep.size error:nil];
NSMutableData *body = [[NSMutableData alloc] init];
body = [NSMutableData dataWithBytesNoCopy:buffer length:buffered freeWhenDone:YES];
[dataToSendToServer setObject:body forKey:#"audioData"];
}
} failureBlock:^(NSError *error) {
NSLog(#"FAILED TO ACCESS AUDIO FROM URL: %#!", self.mediaURL);
}];
}
else {
NSLog(#"NO AUDIO DATA!");
}
}
Because I am new to AVAudioRecorder, perhaps I am just not designing this flow correctly. Could anyone help me out in getting the actual audio data.
Thanks!
AVAudioRecorder records to a file, not to the Asset Library.
So you can simply read the data from that file.

Extract/Record Audio from HLS stream (video) while playing iOS

I am playing HLS streams using AVPlayer. And I also need to record these streams as user presses record button.
The approach I am using is to record audio and video separately then at the end merge these file to make the final video. And It is successful with remote mp4 files.
But now for the HLS (.m3u8) files I am able to record the video using AVAssetWriter but having problems with audio recording.
I am using MTAudioProccessingTap to process the raw audio data and write it to a file. I followed this article. I am able to record remote mp4 audio but its not working with HLS streams.
Initially I wasn't able to extract the audio tracks from the stream using AVAssetTrack *audioTrack = [asset tracksWithMediaType:AVMediaTypeAudio][0];
But I was able to extract the audioTracks using KVO to initialize the MTAudioProcessingTap.
-(void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context{
AVPlayer *player = (AVPlayer*) object;
if (player.status == AVPlayerStatusReadyToPlay)
{
NSLog(#"Ready to play");
self.previousAudioTrackID = 0;
__weak typeof (self) weakself = self;
timeObserverForTrack = [player addPeriodicTimeObserverForInterval:CMTimeMakeWithSeconds(1, 100) queue:nil usingBlock:^(CMTime time)
{
#try {
for(AVPlayerItemTrack* track in [weakself.avPlayer.currentItem tracks]) {
if([track.assetTrack.mediaType isEqualToString:AVMediaTypeAudio])
weakself.currentAudioPlayerItemTrack = track;
}
AVAssetTrack* audioAssetTrack = weakself.currentAudioPlayerItemTrack.assetTrack;
weakself.currentAudioTrackID = audioAssetTrack.trackID;
if(weakself.previousAudioTrackID != weakself.currentAudioTrackID) {
NSLog(#":::::::::::::::::::::::::: Audio track changed : %d",weakself.currentAudioTrackID);
weakself.previousAudioTrackID = weakself.currentAudioTrackID;
weakself.audioTrack = audioAssetTrack;
/// Use this audio track to initialize MTAudioProcessingTap
}
}
#catch (NSException *exception) {
NSLog(#"Exception Trap ::::: Audio tracks not found!");
}
}];
}
}
I am also keeping track of trackID to check if track is changed.
This is how I initialize the MTAudioProcessingTap.
-(void)beginRecordingAudioFromTrack:(AVAssetTrack *)audioTrack{
// Configure an MTAudioProcessingTap to handle things.
MTAudioProcessingTapRef tap;
MTAudioProcessingTapCallbacks callbacks;
callbacks.version = kMTAudioProcessingTapCallbacksVersion_0;
callbacks.clientInfo = (__bridge void *)(self);
callbacks.init = init;
callbacks.prepare = prepare;
callbacks.process = process;
callbacks.unprepare = unprepare;
callbacks.finalize = finalize;
OSStatus err = MTAudioProcessingTapCreate(
kCFAllocatorDefault,
&callbacks,
kMTAudioProcessingTapCreationFlag_PostEffects,
&tap
);
if(err) {
NSLog(#"Unable to create the Audio Processing Tap %d", (int)err);
NSError *error = [NSError errorWithDomain:NSOSStatusErrorDomain
code:err
userInfo:nil];
NSLog(#"Error: %#", [error description]);;
return;
}
// Create an AudioMix and assign it to our currently playing "item", which
// is just the stream itself.
AVMutableAudioMix *audioMix = [AVMutableAudioMix audioMix];
AVMutableAudioMixInputParameters *inputParams = [AVMutableAudioMixInputParameters
audioMixInputParametersWithTrack:audioTrack];
inputParams.audioTapProcessor = tap;
audioMix.inputParameters = #[inputParams];
_audioPlayer.currentItem.audioMix = audioMix;
}
But Now with this audio track MTAudioProcessingTap callbacks "Prepare" and "Process" are never called.
Is the problem with the audioTrack I am getting through KVO?
Now I would really appreciate if some one can help me with this. Or can tell am I using the write approach to record HLS Streams?
I Found solution for this and using it in my app. Wanted to post it earlier but didn't get the time.
So to play with HLS you should have some knowledge what they are exactly. For that please see it here on Apple Website.
HLS Apple
Here are the steps I am following.
1. First get the m3u8 and parse it.
You can parse it using this helpful kit M3U8Kit.
Using this kit you can get the M3U8MediaPlaylist or M3U8MasterPlaylist(if it is a master playlist)
if you get the master playlist you can also parse it to get M3U8MediaPlaylist
(void) parseM3u8
{
NSString *plainString = [self.url m3u8PlanString];
BOOL isMasterPlaylist = [plainString isMasterPlaylist];
NSError *error;
NSURL *baseURL;
if(isMasterPlaylist)
{
M3U8MasterPlaylist *masterList = [[M3U8MasterPlaylist alloc] initWithContentOfURL:self.url error:&error];
self.masterPlaylist = masterList;
M3U8ExtXStreamInfList *xStreamInfList = masterList.xStreamList;
M3U8ExtXStreamInf *StreamInfo = [xStreamInfList extXStreamInfAtIndex:0];
NSString *URI = StreamInfo.URI;
NSRange range = [URI rangeOfString:#"dailymotion.com"];
NSString *baseURLString = [URI substringToIndex:(range.location+range.length)];
baseURL = [NSURL URLWithString:baseURLString];
plainString = [[NSURL URLWithString:URI] m3u8PlanString];
}
M3U8MediaPlaylist *mediaPlaylist = [[M3U8MediaPlaylist alloc] initWithContent:plainString baseURL:baseURL];
self.mediaPlaylist = mediaPlaylist;
M3U8SegmentInfoList *segmentInfoList = mediaPlaylist.segmentList;
NSMutableArray *segmentUrls = [[NSMutableArray alloc] init];
for (int i = 0; i < segmentInfoList.count; i++)
{
M3U8SegmentInfo *segmentInfo = [segmentInfoList segmentInfoAtIndex:i];
NSString *segmentURI = segmentInfo.URI;
NSURL *mediaURL = [baseURL URLByAppendingPathComponent:segmentURI];
[segmentUrls addObject:mediaURL];
if(!self.segmentDuration)
self.segmentDuration = segmentInfo.duration;
}
self.segmentFilesURLs = segmentUrls;
}
You can see that you will get the links to the .ts files from the m3u8 parse it.
Now download all the .ts file into a local folder.
Merge these .ts files in to one mp4 file and Export.
You can do that using this wonderful C library
TS2MP4
and then you can delete the .ts files or keep them if you need them.
This is not good approach what you can do is to Parse M3U8 link .Then try to download segment files (.ts) . If you can get these file you can merge them to generate mp4 file.

Programmatically Record a call and save file in iPhone [duplicate]

Is it theoretically possible to record a phone call on iPhone?
I'm accepting answers which:
may or may not require the phone to be jailbroken
may or may not pass apple's guidelines due to use of private API's (I don't care; it is not for the App Store)
may or may not use private SDKs
I don't want answers just bluntly saying "Apple does not allow that".
I know there would be no official way of doing it, and certainly not for an App Store application, and I know there are call recording apps which place outgoing calls through their own servers.
Here you go. Complete working example. Tweak should be loaded in mediaserverd daemon. It will record every phone call in /var/mobile/Media/DCIM/result.m4a. Audio file has two channels. Left is microphone, right is speaker. On iPhone 4S call is recorded only when the speaker is turned on. On iPhone 5, 5C and 5S call is recorded either way. There might be small hiccups when switching to/from speaker but recording will continue.
#import <AudioToolbox/AudioToolbox.h>
#import <libkern/OSAtomic.h>
//CoreTelephony.framework
extern "C" CFStringRef const kCTCallStatusChangeNotification;
extern "C" CFStringRef const kCTCallStatus;
extern "C" id CTTelephonyCenterGetDefault();
extern "C" void CTTelephonyCenterAddObserver(id ct, void* observer, CFNotificationCallback callBack, CFStringRef name, void *object, CFNotificationSuspensionBehavior sb);
extern "C" int CTGetCurrentCallCount();
enum
{
kCTCallStatusActive = 1,
kCTCallStatusHeld = 2,
kCTCallStatusOutgoing = 3,
kCTCallStatusIncoming = 4,
kCTCallStatusHanged = 5
};
NSString* kMicFilePath = #"/var/mobile/Media/DCIM/mic.caf";
NSString* kSpeakerFilePath = #"/var/mobile/Media/DCIM/speaker.caf";
NSString* kResultFilePath = #"/var/mobile/Media/DCIM/result.m4a";
OSSpinLock phoneCallIsActiveLock = 0;
OSSpinLock speakerLock = 0;
OSSpinLock micLock = 0;
ExtAudioFileRef micFile = NULL;
ExtAudioFileRef speakerFile = NULL;
BOOL phoneCallIsActive = NO;
void Convert()
{
//File URLs
CFURLRef micUrl = CFURLCreateWithFileSystemPath(NULL, (CFStringRef)kMicFilePath, kCFURLPOSIXPathStyle, false);
CFURLRef speakerUrl = CFURLCreateWithFileSystemPath(NULL, (CFStringRef)kSpeakerFilePath, kCFURLPOSIXPathStyle, false);
CFURLRef mixUrl = CFURLCreateWithFileSystemPath(NULL, (CFStringRef)kResultFilePath, kCFURLPOSIXPathStyle, false);
ExtAudioFileRef micFile = NULL;
ExtAudioFileRef speakerFile = NULL;
ExtAudioFileRef mixFile = NULL;
//Opening input files (speaker and mic)
ExtAudioFileOpenURL(micUrl, &micFile);
ExtAudioFileOpenURL(speakerUrl, &speakerFile);
//Reading input file audio format (mono LPCM)
AudioStreamBasicDescription inputFormat, outputFormat;
UInt32 descSize = sizeof(inputFormat);
ExtAudioFileGetProperty(micFile, kExtAudioFileProperty_FileDataFormat, &descSize, &inputFormat);
int sampleSize = inputFormat.mBytesPerFrame;
//Filling input stream format for output file (stereo LPCM)
FillOutASBDForLPCM(inputFormat, inputFormat.mSampleRate, 2, inputFormat.mBitsPerChannel, inputFormat.mBitsPerChannel, true, false, false);
//Filling output file audio format (AAC)
memset(&outputFormat, 0, sizeof(outputFormat));
outputFormat.mFormatID = kAudioFormatMPEG4AAC;
outputFormat.mSampleRate = 8000;
outputFormat.mFormatFlags = kMPEG4Object_AAC_Main;
outputFormat.mChannelsPerFrame = 2;
//Opening output file
ExtAudioFileCreateWithURL(mixUrl, kAudioFileM4AType, &outputFormat, NULL, kAudioFileFlags_EraseFile, &mixFile);
ExtAudioFileSetProperty(mixFile, kExtAudioFileProperty_ClientDataFormat, sizeof(inputFormat), &inputFormat);
//Freeing URLs
CFRelease(micUrl);
CFRelease(speakerUrl);
CFRelease(mixUrl);
//Setting up audio buffers
int bufferSizeInSamples = 64 * 1024;
AudioBufferList micBuffer;
micBuffer.mNumberBuffers = 1;
micBuffer.mBuffers[0].mNumberChannels = 1;
micBuffer.mBuffers[0].mDataByteSize = sampleSize * bufferSizeInSamples;
micBuffer.mBuffers[0].mData = malloc(micBuffer.mBuffers[0].mDataByteSize);
AudioBufferList speakerBuffer;
speakerBuffer.mNumberBuffers = 1;
speakerBuffer.mBuffers[0].mNumberChannels = 1;
speakerBuffer.mBuffers[0].mDataByteSize = sampleSize * bufferSizeInSamples;
speakerBuffer.mBuffers[0].mData = malloc(speakerBuffer.mBuffers[0].mDataByteSize);
AudioBufferList mixBuffer;
mixBuffer.mNumberBuffers = 1;
mixBuffer.mBuffers[0].mNumberChannels = 2;
mixBuffer.mBuffers[0].mDataByteSize = sampleSize * bufferSizeInSamples * 2;
mixBuffer.mBuffers[0].mData = malloc(mixBuffer.mBuffers[0].mDataByteSize);
//Converting
while (true)
{
//Reading data from input files
UInt32 framesToRead = bufferSizeInSamples;
ExtAudioFileRead(micFile, &framesToRead, &micBuffer);
ExtAudioFileRead(speakerFile, &framesToRead, &speakerBuffer);
if (framesToRead == 0)
{
break;
}
//Building interleaved stereo buffer - left channel is mic, right - speaker
for (int i = 0; i < framesToRead; i++)
{
memcpy((char*)mixBuffer.mBuffers[0].mData + i * sampleSize * 2, (char*)micBuffer.mBuffers[0].mData + i * sampleSize, sampleSize);
memcpy((char*)mixBuffer.mBuffers[0].mData + i * sampleSize * 2 + sampleSize, (char*)speakerBuffer.mBuffers[0].mData + i * sampleSize, sampleSize);
}
//Writing to output file - LPCM will be converted to AAC
ExtAudioFileWrite(mixFile, framesToRead, &mixBuffer);
}
//Closing files
ExtAudioFileDispose(micFile);
ExtAudioFileDispose(speakerFile);
ExtAudioFileDispose(mixFile);
//Freeing audio buffers
free(micBuffer.mBuffers[0].mData);
free(speakerBuffer.mBuffers[0].mData);
free(mixBuffer.mBuffers[0].mData);
}
void Cleanup()
{
[[NSFileManager defaultManager] removeItemAtPath:kMicFilePath error:NULL];
[[NSFileManager defaultManager] removeItemAtPath:kSpeakerFilePath error:NULL];
}
void CoreTelephonyNotificationCallback(CFNotificationCenterRef center, void *observer, CFStringRef name, const void *object, CFDictionaryRef userInfo)
{
NSDictionary* data = (NSDictionary*)userInfo;
if ([(NSString*)name isEqualToString:(NSString*)kCTCallStatusChangeNotification])
{
int currentCallStatus = [data[(NSString*)kCTCallStatus] integerValue];
if (currentCallStatus == kCTCallStatusActive)
{
OSSpinLockLock(&phoneCallIsActiveLock);
phoneCallIsActive = YES;
OSSpinLockUnlock(&phoneCallIsActiveLock);
}
else if (currentCallStatus == kCTCallStatusHanged)
{
if (CTGetCurrentCallCount() > 0)
{
return;
}
OSSpinLockLock(&phoneCallIsActiveLock);
phoneCallIsActive = NO;
OSSpinLockUnlock(&phoneCallIsActiveLock);
//Closing mic file
OSSpinLockLock(&micLock);
if (micFile != NULL)
{
ExtAudioFileDispose(micFile);
}
micFile = NULL;
OSSpinLockUnlock(&micLock);
//Closing speaker file
OSSpinLockLock(&speakerLock);
if (speakerFile != NULL)
{
ExtAudioFileDispose(speakerFile);
}
speakerFile = NULL;
OSSpinLockUnlock(&speakerLock);
Convert();
Cleanup();
}
}
}
OSStatus(*AudioUnitProcess_orig)(AudioUnit unit, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inNumberFrames, AudioBufferList *ioData);
OSStatus AudioUnitProcess_hook(AudioUnit unit, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inNumberFrames, AudioBufferList *ioData)
{
OSSpinLockLock(&phoneCallIsActiveLock);
if (phoneCallIsActive == NO)
{
OSSpinLockUnlock(&phoneCallIsActiveLock);
return AudioUnitProcess_orig(unit, ioActionFlags, inTimeStamp, inNumberFrames, ioData);
}
OSSpinLockUnlock(&phoneCallIsActiveLock);
ExtAudioFileRef* currentFile = NULL;
OSSpinLock* currentLock = NULL;
AudioComponentDescription unitDescription = {0};
AudioComponentGetDescription(AudioComponentInstanceGetComponent(unit), &unitDescription);
//'agcc', 'mbdp' - iPhone 4S, iPhone 5
//'agc2', 'vrq2' - iPhone 5C, iPhone 5S
if (unitDescription.componentSubType == 'agcc' || unitDescription.componentSubType == 'agc2')
{
currentFile = &micFile;
currentLock = &micLock;
}
else if (unitDescription.componentSubType == 'mbdp' || unitDescription.componentSubType == 'vrq2')
{
currentFile = &speakerFile;
currentLock = &speakerLock;
}
if (currentFile != NULL)
{
OSSpinLockLock(currentLock);
//Opening file
if (*currentFile == NULL)
{
//Obtaining input audio format
AudioStreamBasicDescription desc;
UInt32 descSize = sizeof(desc);
AudioUnitGetProperty(unit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &desc, &descSize);
//Opening audio file
CFURLRef url = CFURLCreateWithFileSystemPath(NULL, (CFStringRef)((currentFile == &micFile) ? kMicFilePath : kSpeakerFilePath), kCFURLPOSIXPathStyle, false);
ExtAudioFileRef audioFile = NULL;
OSStatus result = ExtAudioFileCreateWithURL(url, kAudioFileCAFType, &desc, NULL, kAudioFileFlags_EraseFile, &audioFile);
if (result != 0)
{
*currentFile = NULL;
}
else
{
*currentFile = audioFile;
//Writing audio format
ExtAudioFileSetProperty(*currentFile, kExtAudioFileProperty_ClientDataFormat, sizeof(desc), &desc);
}
CFRelease(url);
}
else
{
//Writing audio buffer
ExtAudioFileWrite(*currentFile, inNumberFrames, ioData);
}
OSSpinLockUnlock(currentLock);
}
return AudioUnitProcess_orig(unit, ioActionFlags, inTimeStamp, inNumberFrames, ioData);
}
__attribute__((constructor))
static void initialize()
{
CTTelephonyCenterAddObserver(CTTelephonyCenterGetDefault(), NULL, CoreTelephonyNotificationCallback, NULL, NULL, CFNotificationSuspensionBehaviorHold);
MSHookFunction(AudioUnitProcess, AudioUnitProcess_hook, &AudioUnitProcess_orig);
}
A few words about what's going on. AudioUnitProcess function is used for processing audio streams in order to apply some effects, mix, convert etc. We are hooking AudioUnitProcess in order to access phone call's audio streams. While phone call is active these streams are being processed in various ways.
We are listening for CoreTelephony notifications in order to get phone call status changes. When we receive audio samples we need to determine where they come from - microphone or speaker. This is done using componentSubType field in AudioComponentDescription structure. Now, you might think, why don't we store AudioUnit objects so that we don't need to check componentSubType every time. I did that but it will break everything when you switch speaker on/off on iPhone 5 because AudioUnit objects will change, they are recreated. So, now we open audio files (one for microphone and one for speaker) and write samples in them, simple as that. When phone call ends we will receive appropriate CoreTelephony notification and close the files. We have two separate files with audio from microphone and speaker that we need to merge. This is what void Convert() is for. It's pretty simple if you know the API. I don't think I need to explain it, comments are enough.
About locks. There are many threads in mediaserverd. Audio processing and CoreTelephony notifications are on different threads so we need some kind synchronization. I chose spin locks because they are fast and because the chance of lock contention is small in our case. On iPhone 4S and even iPhone 5 all the work in AudioUnitProcess should be done as fast as possible otherwise you will hear hiccups from device speaker which obviously not good.
Yes. Audio Recorder by a developer named Limneos does that (and quite well). You can find it on Cydia. It can record any type of call on iPhone 5 and up without using any servers etc'. The call will be placed on the device in an Audio file. It also supports iPhone 4S but for speaker only.
This tweak is known to be the first tweak ever that managed to record both streams of audio without using any 3rd party severs, VOIP or something similar.
The developer placed beeps on the other side of the call to alert the person you are recording but those were removed too by hackers across the net. To answer your question, Yes, it's very much possible, and not just theoretically.
Further reading
https://stackoverflow.com/a/19413363/202451
http://forums.macrumors.com/showthread.php?t=1566350
https://github.com/nst/iOS-Runtime-Headers
The only solution I can think of is to use the Core Telephony framework, and more specifically the callEventHandler property, to intercept when a call is coming in, and then to use an AVAudioRecorder to record the voice of the person with the phone (and maybe a little of the person on the other line's voice). This is obviously not perfect, and would only work if your application is in the foreground at the time of the call, but it may be the best you can get. See more about finding out if there is an incoming phone call here: Can we fire an event when ever there is Incoming and Outgoing call in iphone?.
EDIT:
.h:
#import <AVFoundation/AVFoundation.h>
#import<CoreTelephony/CTCallCenter.h>
#import<CoreTelephony/CTCall.h>
#property (strong, nonatomic) AVAudioRecorder *audioRecorder;
ViewDidLoad:
NSArray *dirPaths;
NSString *docsDir;
dirPaths = NSSearchPathForDirectoriesInDomains(
NSDocumentDirectory, NSUserDomainMask, YES);
docsDir = dirPaths[0];
NSString *soundFilePath = [docsDir
stringByAppendingPathComponent:#"sound.caf"];
NSURL *soundFileURL = [NSURL fileURLWithPath:soundFilePath];
NSDictionary *recordSettings = [NSDictionary
dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:AVAudioQualityMin],
AVEncoderAudioQualityKey,
[NSNumber numberWithInt:16],
AVEncoderBitRateKey,
[NSNumber numberWithInt: 2],
AVNumberOfChannelsKey,
[NSNumber numberWithFloat:44100.0],
AVSampleRateKey,
nil];
NSError *error = nil;
_audioRecorder = [[AVAudioRecorder alloc]
initWithURL:soundFileURL
settings:recordSettings
error:&error];
if (error)
{
NSLog(#"error: %#", [error localizedDescription]);
} else {
[_audioRecorder prepareToRecord];
}
CTCallCenter *callCenter = [[CTCallCenter alloc] init];
[callCenter setCallEventHandler:^(CTCall *call) {
if ([[call callState] isEqual:CTCallStateConnected]) {
[_audioRecorder record];
} else if ([[call callState] isEqual:CTCallStateDisconnected]) {
[_audioRecorder stop];
}
}];
AppDelegate.m:
- (void)applicationDidEnterBackground:(UIApplication *)application//Makes sure that the recording keeps happening even when app is in the background, though only can go for 10 minutes.
{
__block UIBackgroundTaskIdentifier task = 0;
task=[application beginBackgroundTaskWithExpirationHandler:^{
NSLog(#"Expiration handler called %f",[application backgroundTimeRemaining]);
[application endBackgroundTask:task];
task=UIBackgroundTaskInvalid;
}];
This is the first time using many of these features, so not sure if this is exactly right, but I think you get the idea. Untested, as I do not have access to the right tools at the moment. Compiled using these sources:
Recording voice in background using AVAudioRecorder
http://prassan-warrior.blogspot.com/2012/11/recording-audio-on-iphone-with.html
Can we fire an event when ever there is Incoming and Outgoing call in iphone?
Apple does not allow it and does not provide any API for it.
However, on a jailbroken device I'm sure it's possible. As a matter of fact, I think it's already done. I remember seeing an app when my phone was jailbroken that changed your voice and recorded the call - I remember it was a US company offering it only in the states. Unfortunately I don't remember the name...
I guess some hardware could solve this. Connected to the minijack-port; having earbuds and a microphone passing through a small recorder. This recorder can be very simple. While not in conversation the recorder could feed the phone with data/the recording (through the jack-cable). And with a simple start button (just like the volum controls on the earbuds) could be sufficient for timing the recording.
Some setups
http://www.danmccomb.com/posts/483/how-to-record-iphone-conversations-using-zoom-h4n/
http://forums.macrumors.com/showthread.php?t=346430

How to correctly read decoded PCM samples on iOS using AVAssetReader -- currently incorrect decoding

I am currently working on an application as part of my Bachelor in Computer Science. The application will correlate data from the iPhone hardware (accelerometer, gps) and music that is being played.
The project is still in its infancy, having worked on it for only 2 months.
The moment that I am right now, and where I need help, is reading PCM samples from songs from the itunes library, and playing them back using and audio unit.
Currently the implementation I would like working does the following: chooses a random song from iTunes, and reads samples from it when required, and stores in a buffer, lets call it sampleBuffer. Later on in the consumer model the audio unit (which has a mixer and a remoteIO output) has a callback where I simply copy the required number of samples from sampleBuffer into the buffer specified in the callback. What i then hear through the speakers is something not quite what i expect; I can recognize that it is playing the song however it seems that it is incorrectly decoded and it has a lot of noise! I attached an image which shows the first ~half a second (24576 samples # 44.1kHz), and this does not resemble a normall looking output.
Before I get into the listing I have checked that the file is not corrupted, similarily I have written test cases for the buffer (so I know the buffer does not alter the samples), and although this might not be the best way to do it (some would argue to go the audio queue route), I want to perform various manipulations on the samples aswell as changing the song before it is finished, rearranging what song is played, etc. Furthermore, maybe there are some incorrect settings in the audio unit, however, the graph that displays the samples (which shows the samples are decoded incorrectly) is taken straight from the buffer, thus I am only looking now to solve why the reading from the disk and decoding does not work correctly. Right now i simply want to get a play through working.
Cant post images because new to stackoverflow so heres the link to the image: http://i.stack.imgur.com/RHjlv.jpg
Listing:
This is where I setup the audioReadSettigns which will be used for the AVAssetReaderAudioMixOutput
// Set the read settings
audioReadSettings = [[NSMutableDictionary alloc] init];
[audioReadSettings setValue:[NSNumber numberWithInt:kAudioFormatLinearPCM]
forKey:AVFormatIDKey];
[audioReadSettings setValue:[NSNumber numberWithInt:16] forKey:AVLinearPCMBitDepthKey];
[audioReadSettings setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsBigEndianKey];
[audioReadSettings setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsFloatKey];
[audioReadSettings setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsNonInterleaved];
[audioReadSettings setValue:[NSNumber numberWithFloat:44100.0] forKey:AVSampleRateKey];
Now the following code listing is a method that receives an NSString with the persistant_id of the song:
-(BOOL)setNextSongID:(NSString*)persistand_id {
assert(persistand_id != nil);
MPMediaItem *song = [self getMediaItemForPersistantID:persistand_id];
NSURL *assetUrl = [song valueForProperty:MPMediaItemPropertyAssetURL];
AVURLAsset *songAsset = [AVURLAsset URLAssetWithURL:assetUrl
options:[NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES]
forKey:AVURLAssetPreferPreciseDurationAndTimingKey]];
NSError *assetError = nil;
assetReader = [[AVAssetReader assetReaderWithAsset:songAsset error:&assetError] retain];
if (assetError) {
NSLog(#"error: %#", assetError);
return NO;
}
CMTimeRange timeRange = CMTimeRangeMake(kCMTimeZero, songAsset.duration);
[assetReader setTimeRange:timeRange];
track = [[songAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
assetReaderOutput = [AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks:[NSArray arrayWithObject:track]
audioSettings:audioReadSettings];
if (![assetReader canAddOutput:assetReaderOutput]) {
NSLog(#"cant add reader output... die!");
return NO;
}
[assetReader addOutput:assetReaderOutput];
[assetReader startReading];
// just getting some basic information about the track to print
NSArray *formatDesc = ((AVAssetTrack*)[[assetReaderOutput audioTracks] objectAtIndex:0]).formatDescriptions;
for (unsigned int i = 0; i < [formatDesc count]; ++i) {
CMAudioFormatDescriptionRef item = (CMAudioFormatDescriptionRef)[formatDesc objectAtIndex:i];
const CAStreamBasicDescription *asDesc = (CAStreamBasicDescription*)CMAudioFormatDescriptionGetStreamBasicDescription(item);
if (asDesc) {
// get data
numChannels = asDesc->mChannelsPerFrame;
sampleRate = asDesc->mSampleRate;
asDesc->Print();
}
}
[self copyEnoughSamplesToBufferForLength:24000];
return YES;
}
The following presents the function -(void)copyEnoughSamplesToBufferForLength:
-(void)copyEnoughSamplesToBufferForLength:(UInt32)samples_count {
[w_lock lock];
int stillToCopy = 0;
if (sampleBuffer->numSamples() < samples_count) {
stillToCopy = samples_count;
}
NSAutoreleasePool *apool = [[NSAutoreleasePool alloc] init];
CMSampleBufferRef sampleBufferRef;
SInt16 *dataBuffer = (SInt16*)malloc(8192 * sizeof(SInt16));
int a = 0;
while (stillToCopy > 0) {
sampleBufferRef = [assetReaderOutput copyNextSampleBuffer];
if (!sampleBufferRef) {
// end of song or no more samples
return;
}
CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBufferRef);
CMItemCount numSamplesInBuffer = CMSampleBufferGetNumSamples(sampleBufferRef);
AudioBufferList audioBufferList;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBufferRef,
NULL,
&audioBufferList,
sizeof(audioBufferList),
NULL,
NULL,
0,
&blockBuffer);
int data_length = floorf(numSamplesInBuffer * 1.0f);
int j = 0;
for (int bufferCount=0; bufferCount < audioBufferList.mNumberBuffers; bufferCount++) {
SInt16* samples = (SInt16 *)audioBufferList.mBuffers[bufferCount].mData;
for (int i=0; i < numSamplesInBuffer; i++) {
dataBuffer[j] = samples[i];
j++;
}
}
CFRelease(sampleBufferRef);
sampleBuffer->putSamples(dataBuffer, j);
stillToCopy = stillToCopy - data_length;
}
free(dataBuffer);
[w_lock unlock];
[apool release];
}
Now the sampleBuffer will have incorrectly decoded samples. Can anyone help me why this is so? This happens for different files on my iTunes library (mp3, aac, wav, etc).
Any help would be greatly appreciated, furthermore, if you need any other listing of my code, or perhaps what the output sounds like, I will attach it per request. I have been sitting on this for the past week trying to debug it and have found no help online -- everyone seems to be doign it in my way, yet it seems that only I have this issue.
Thanks for any help at all!
Peter
Currently, I am also working on a project which involves extracting audio samples from iTunes Library into AudioUnit.
The audiounit render call back is included for your reference. The input format is set as SInt16StereoStreamFormat.
I have made use of Michael Tyson's circular buffer implementation - TPCircularBuffer as the buffer storage. Very easy to use and understand!!! Thanks Michael!
- (void) loadBuffer:(NSURL *)assetURL_
{
if (nil != self.iPodAssetReader) {
[iTunesOperationQueue cancelAllOperations];
[self cleanUpBuffer];
}
NSDictionary *outputSettings = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:kAudioFormatLinearPCM], AVFormatIDKey,
[NSNumber numberWithFloat:44100.0], AVSampleRateKey,
[NSNumber numberWithInt:16], AVLinearPCMBitDepthKey,
[NSNumber numberWithBool:NO], AVLinearPCMIsNonInterleaved,
[NSNumber numberWithBool:NO], AVLinearPCMIsFloatKey,
[NSNumber numberWithBool:NO], AVLinearPCMIsBigEndianKey,
nil];
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:assetURL_ options:nil];
if (asset == nil) {
NSLog(#"asset is not defined!");
return;
}
NSLog(#"Total Asset Duration: %f", CMTimeGetSeconds(asset.duration));
NSError *assetError = nil;
self.iPodAssetReader = [AVAssetReader assetReaderWithAsset:asset error:&assetError];
if (assetError) {
NSLog (#"error: %#", assetError);
return;
}
AVAssetReaderOutput *readerOutput = [AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks:asset.tracks audioSettings:outputSettings];
if (! [iPodAssetReader canAddOutput: readerOutput]) {
NSLog (#"can't add reader output... die!");
return;
}
// add output reader to reader
[iPodAssetReader addOutput: readerOutput];
if (! [iPodAssetReader startReading]) {
NSLog(#"Unable to start reading!");
return;
}
// Init circular buffer
TPCircularBufferInit(&playbackState.circularBuffer, kTotalBufferSize);
__block NSBlockOperation * feediPodBufferOperation = [NSBlockOperation blockOperationWithBlock:^{
while (![feediPodBufferOperation isCancelled] && iPodAssetReader.status != AVAssetReaderStatusCompleted) {
if (iPodAssetReader.status == AVAssetReaderStatusReading) {
// Check if the available buffer space is enough to hold at least one cycle of the sample data
if (kTotalBufferSize - playbackState.circularBuffer.fillCount >= 32768) {
CMSampleBufferRef nextBuffer = [readerOutput copyNextSampleBuffer];
if (nextBuffer) {
AudioBufferList abl;
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(nextBuffer, NULL, &abl, sizeof(abl), NULL, NULL, kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment, &blockBuffer);
UInt64 size = CMSampleBufferGetTotalSampleSize(nextBuffer);
int bytesCopied = TPCircularBufferProduceBytes(&playbackState.circularBuffer, abl.mBuffers[0].mData, size);
if (!playbackState.bufferIsReady && bytesCopied > 0) {
playbackState.bufferIsReady = YES;
}
CFRelease(nextBuffer);
CFRelease(blockBuffer);
}
else {
break;
}
}
}
}
NSLog(#"iPod Buffer Reading Finished");
}];
[iTunesOperationQueue addOperation:feediPodBufferOperation];
}
static OSStatus ipodRenderCallback (
void *inRefCon, // A pointer to a struct containing the complete audio data
// to play, as well as state information such as the
// first sample to play on this invocation of the callback.
AudioUnitRenderActionFlags *ioActionFlags, // Unused here. When generating audio, use ioActionFlags to indicate silence
// between sounds; for silence, also memset the ioData buffers to 0.
const AudioTimeStamp *inTimeStamp, // Unused here.
UInt32 inBusNumber, // The mixer unit input bus that is requesting some new
// frames of audio data to play.
UInt32 inNumberFrames, // The number of frames of audio to provide to the buffer(s)
// pointed to by the ioData parameter.
AudioBufferList *ioData // On output, the audio data to play. The callback's primary
// responsibility is to fill the buffer(s) in the
// AudioBufferList.
)
{
Audio* audioObject = (Audio*)inRefCon;
AudioSampleType *outSample = (AudioSampleType *)ioData->mBuffers[0].mData;
// Zero-out all the output samples first
memset(outSample, 0, inNumberFrames * kUnitSize * 2);
if ( audioObject.playingiPod && audioObject.bufferIsReady) {
// Pull audio from circular buffer
int32_t availableBytes;
AudioSampleType *bufferTail = TPCircularBufferTail(&audioObject.circularBuffer, &availableBytes);
memcpy(outSample, bufferTail, MIN(availableBytes, inNumberFrames * kUnitSize * 2) );
TPCircularBufferConsume(&audioObject.circularBuffer, MIN(availableBytes, inNumberFrames * kUnitSize * 2) );
audioObject.currentSampleNum += MIN(availableBytes / (kUnitSize * 2), inNumberFrames);
if (availableBytes <= inNumberFrames * kUnitSize * 2) {
// Buffer is running out or playback is finished
audioObject.bufferIsReady = NO;
audioObject.playingiPod = NO;
audioObject.currentSampleNum = 0;
if ([[audioObject delegate] respondsToSelector:#selector(playbackDidFinish)]) {
[[audioObject delegate] performSelector:#selector(playbackDidFinish)];
}
}
}
return noErr;
}
- (void) setupSInt16StereoStreamFormat {
// The AudioUnitSampleType data type is the recommended type for sample data in audio
// units. This obtains the byte size of the type for use in filling in the ASBD.
size_t bytesPerSample = sizeof (AudioSampleType);
// Fill the application audio format struct's fields to define a linear PCM,
// stereo, noninterleaved stream at the hardware sample rate.
SInt16StereoStreamFormat.mFormatID = kAudioFormatLinearPCM;
SInt16StereoStreamFormat.mFormatFlags = kAudioFormatFlagsCanonical;
SInt16StereoStreamFormat.mBytesPerPacket = 2 * bytesPerSample; // *** kAudioFormatFlagsCanonical <- implicit interleaved data => (left sample + right sample) per Packet
SInt16StereoStreamFormat.mFramesPerPacket = 1;
SInt16StereoStreamFormat.mBytesPerFrame = SInt16StereoStreamFormat.mBytesPerPacket * SInt16StereoStreamFormat.mFramesPerPacket;
SInt16StereoStreamFormat.mChannelsPerFrame = 2; // 2 indicates stereo
SInt16StereoStreamFormat.mBitsPerChannel = 8 * bytesPerSample;
SInt16StereoStreamFormat.mSampleRate = graphSampleRate;
NSLog (#"The stereo stream format for the \"iPod\" mixer input bus:");
[self printASBD: SInt16StereoStreamFormat];
}
I guess it is kind of late, but you could try this library:
https://bitbucket.org/artgillespie/tslibraryimport
After using this to save the audio into a file, you could process the data with render callbacks from MixerHost.
If I were you I would either use kAudioUnitSubType_AudioFilePlayer to play the file and access its samples with the units render callback.
Or
Use ExtAudioFileRef to extract the samples straight to a buffer.

How can I record a conversation / phone call on iOS?

Is it theoretically possible to record a phone call on iPhone?
I'm accepting answers which:
may or may not require the phone to be jailbroken
may or may not pass apple's guidelines due to use of private API's (I don't care; it is not for the App Store)
may or may not use private SDKs
I don't want answers just bluntly saying "Apple does not allow that".
I know there would be no official way of doing it, and certainly not for an App Store application, and I know there are call recording apps which place outgoing calls through their own servers.
Here you go. Complete working example. Tweak should be loaded in mediaserverd daemon. It will record every phone call in /var/mobile/Media/DCIM/result.m4a. Audio file has two channels. Left is microphone, right is speaker. On iPhone 4S call is recorded only when the speaker is turned on. On iPhone 5, 5C and 5S call is recorded either way. There might be small hiccups when switching to/from speaker but recording will continue.
#import <AudioToolbox/AudioToolbox.h>
#import <libkern/OSAtomic.h>
//CoreTelephony.framework
extern "C" CFStringRef const kCTCallStatusChangeNotification;
extern "C" CFStringRef const kCTCallStatus;
extern "C" id CTTelephonyCenterGetDefault();
extern "C" void CTTelephonyCenterAddObserver(id ct, void* observer, CFNotificationCallback callBack, CFStringRef name, void *object, CFNotificationSuspensionBehavior sb);
extern "C" int CTGetCurrentCallCount();
enum
{
kCTCallStatusActive = 1,
kCTCallStatusHeld = 2,
kCTCallStatusOutgoing = 3,
kCTCallStatusIncoming = 4,
kCTCallStatusHanged = 5
};
NSString* kMicFilePath = #"/var/mobile/Media/DCIM/mic.caf";
NSString* kSpeakerFilePath = #"/var/mobile/Media/DCIM/speaker.caf";
NSString* kResultFilePath = #"/var/mobile/Media/DCIM/result.m4a";
OSSpinLock phoneCallIsActiveLock = 0;
OSSpinLock speakerLock = 0;
OSSpinLock micLock = 0;
ExtAudioFileRef micFile = NULL;
ExtAudioFileRef speakerFile = NULL;
BOOL phoneCallIsActive = NO;
void Convert()
{
//File URLs
CFURLRef micUrl = CFURLCreateWithFileSystemPath(NULL, (CFStringRef)kMicFilePath, kCFURLPOSIXPathStyle, false);
CFURLRef speakerUrl = CFURLCreateWithFileSystemPath(NULL, (CFStringRef)kSpeakerFilePath, kCFURLPOSIXPathStyle, false);
CFURLRef mixUrl = CFURLCreateWithFileSystemPath(NULL, (CFStringRef)kResultFilePath, kCFURLPOSIXPathStyle, false);
ExtAudioFileRef micFile = NULL;
ExtAudioFileRef speakerFile = NULL;
ExtAudioFileRef mixFile = NULL;
//Opening input files (speaker and mic)
ExtAudioFileOpenURL(micUrl, &micFile);
ExtAudioFileOpenURL(speakerUrl, &speakerFile);
//Reading input file audio format (mono LPCM)
AudioStreamBasicDescription inputFormat, outputFormat;
UInt32 descSize = sizeof(inputFormat);
ExtAudioFileGetProperty(micFile, kExtAudioFileProperty_FileDataFormat, &descSize, &inputFormat);
int sampleSize = inputFormat.mBytesPerFrame;
//Filling input stream format for output file (stereo LPCM)
FillOutASBDForLPCM(inputFormat, inputFormat.mSampleRate, 2, inputFormat.mBitsPerChannel, inputFormat.mBitsPerChannel, true, false, false);
//Filling output file audio format (AAC)
memset(&outputFormat, 0, sizeof(outputFormat));
outputFormat.mFormatID = kAudioFormatMPEG4AAC;
outputFormat.mSampleRate = 8000;
outputFormat.mFormatFlags = kMPEG4Object_AAC_Main;
outputFormat.mChannelsPerFrame = 2;
//Opening output file
ExtAudioFileCreateWithURL(mixUrl, kAudioFileM4AType, &outputFormat, NULL, kAudioFileFlags_EraseFile, &mixFile);
ExtAudioFileSetProperty(mixFile, kExtAudioFileProperty_ClientDataFormat, sizeof(inputFormat), &inputFormat);
//Freeing URLs
CFRelease(micUrl);
CFRelease(speakerUrl);
CFRelease(mixUrl);
//Setting up audio buffers
int bufferSizeInSamples = 64 * 1024;
AudioBufferList micBuffer;
micBuffer.mNumberBuffers = 1;
micBuffer.mBuffers[0].mNumberChannels = 1;
micBuffer.mBuffers[0].mDataByteSize = sampleSize * bufferSizeInSamples;
micBuffer.mBuffers[0].mData = malloc(micBuffer.mBuffers[0].mDataByteSize);
AudioBufferList speakerBuffer;
speakerBuffer.mNumberBuffers = 1;
speakerBuffer.mBuffers[0].mNumberChannels = 1;
speakerBuffer.mBuffers[0].mDataByteSize = sampleSize * bufferSizeInSamples;
speakerBuffer.mBuffers[0].mData = malloc(speakerBuffer.mBuffers[0].mDataByteSize);
AudioBufferList mixBuffer;
mixBuffer.mNumberBuffers = 1;
mixBuffer.mBuffers[0].mNumberChannels = 2;
mixBuffer.mBuffers[0].mDataByteSize = sampleSize * bufferSizeInSamples * 2;
mixBuffer.mBuffers[0].mData = malloc(mixBuffer.mBuffers[0].mDataByteSize);
//Converting
while (true)
{
//Reading data from input files
UInt32 framesToRead = bufferSizeInSamples;
ExtAudioFileRead(micFile, &framesToRead, &micBuffer);
ExtAudioFileRead(speakerFile, &framesToRead, &speakerBuffer);
if (framesToRead == 0)
{
break;
}
//Building interleaved stereo buffer - left channel is mic, right - speaker
for (int i = 0; i < framesToRead; i++)
{
memcpy((char*)mixBuffer.mBuffers[0].mData + i * sampleSize * 2, (char*)micBuffer.mBuffers[0].mData + i * sampleSize, sampleSize);
memcpy((char*)mixBuffer.mBuffers[0].mData + i * sampleSize * 2 + sampleSize, (char*)speakerBuffer.mBuffers[0].mData + i * sampleSize, sampleSize);
}
//Writing to output file - LPCM will be converted to AAC
ExtAudioFileWrite(mixFile, framesToRead, &mixBuffer);
}
//Closing files
ExtAudioFileDispose(micFile);
ExtAudioFileDispose(speakerFile);
ExtAudioFileDispose(mixFile);
//Freeing audio buffers
free(micBuffer.mBuffers[0].mData);
free(speakerBuffer.mBuffers[0].mData);
free(mixBuffer.mBuffers[0].mData);
}
void Cleanup()
{
[[NSFileManager defaultManager] removeItemAtPath:kMicFilePath error:NULL];
[[NSFileManager defaultManager] removeItemAtPath:kSpeakerFilePath error:NULL];
}
void CoreTelephonyNotificationCallback(CFNotificationCenterRef center, void *observer, CFStringRef name, const void *object, CFDictionaryRef userInfo)
{
NSDictionary* data = (NSDictionary*)userInfo;
if ([(NSString*)name isEqualToString:(NSString*)kCTCallStatusChangeNotification])
{
int currentCallStatus = [data[(NSString*)kCTCallStatus] integerValue];
if (currentCallStatus == kCTCallStatusActive)
{
OSSpinLockLock(&phoneCallIsActiveLock);
phoneCallIsActive = YES;
OSSpinLockUnlock(&phoneCallIsActiveLock);
}
else if (currentCallStatus == kCTCallStatusHanged)
{
if (CTGetCurrentCallCount() > 0)
{
return;
}
OSSpinLockLock(&phoneCallIsActiveLock);
phoneCallIsActive = NO;
OSSpinLockUnlock(&phoneCallIsActiveLock);
//Closing mic file
OSSpinLockLock(&micLock);
if (micFile != NULL)
{
ExtAudioFileDispose(micFile);
}
micFile = NULL;
OSSpinLockUnlock(&micLock);
//Closing speaker file
OSSpinLockLock(&speakerLock);
if (speakerFile != NULL)
{
ExtAudioFileDispose(speakerFile);
}
speakerFile = NULL;
OSSpinLockUnlock(&speakerLock);
Convert();
Cleanup();
}
}
}
OSStatus(*AudioUnitProcess_orig)(AudioUnit unit, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inNumberFrames, AudioBufferList *ioData);
OSStatus AudioUnitProcess_hook(AudioUnit unit, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inNumberFrames, AudioBufferList *ioData)
{
OSSpinLockLock(&phoneCallIsActiveLock);
if (phoneCallIsActive == NO)
{
OSSpinLockUnlock(&phoneCallIsActiveLock);
return AudioUnitProcess_orig(unit, ioActionFlags, inTimeStamp, inNumberFrames, ioData);
}
OSSpinLockUnlock(&phoneCallIsActiveLock);
ExtAudioFileRef* currentFile = NULL;
OSSpinLock* currentLock = NULL;
AudioComponentDescription unitDescription = {0};
AudioComponentGetDescription(AudioComponentInstanceGetComponent(unit), &unitDescription);
//'agcc', 'mbdp' - iPhone 4S, iPhone 5
//'agc2', 'vrq2' - iPhone 5C, iPhone 5S
if (unitDescription.componentSubType == 'agcc' || unitDescription.componentSubType == 'agc2')
{
currentFile = &micFile;
currentLock = &micLock;
}
else if (unitDescription.componentSubType == 'mbdp' || unitDescription.componentSubType == 'vrq2')
{
currentFile = &speakerFile;
currentLock = &speakerLock;
}
if (currentFile != NULL)
{
OSSpinLockLock(currentLock);
//Opening file
if (*currentFile == NULL)
{
//Obtaining input audio format
AudioStreamBasicDescription desc;
UInt32 descSize = sizeof(desc);
AudioUnitGetProperty(unit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &desc, &descSize);
//Opening audio file
CFURLRef url = CFURLCreateWithFileSystemPath(NULL, (CFStringRef)((currentFile == &micFile) ? kMicFilePath : kSpeakerFilePath), kCFURLPOSIXPathStyle, false);
ExtAudioFileRef audioFile = NULL;
OSStatus result = ExtAudioFileCreateWithURL(url, kAudioFileCAFType, &desc, NULL, kAudioFileFlags_EraseFile, &audioFile);
if (result != 0)
{
*currentFile = NULL;
}
else
{
*currentFile = audioFile;
//Writing audio format
ExtAudioFileSetProperty(*currentFile, kExtAudioFileProperty_ClientDataFormat, sizeof(desc), &desc);
}
CFRelease(url);
}
else
{
//Writing audio buffer
ExtAudioFileWrite(*currentFile, inNumberFrames, ioData);
}
OSSpinLockUnlock(currentLock);
}
return AudioUnitProcess_orig(unit, ioActionFlags, inTimeStamp, inNumberFrames, ioData);
}
__attribute__((constructor))
static void initialize()
{
CTTelephonyCenterAddObserver(CTTelephonyCenterGetDefault(), NULL, CoreTelephonyNotificationCallback, NULL, NULL, CFNotificationSuspensionBehaviorHold);
MSHookFunction(AudioUnitProcess, AudioUnitProcess_hook, &AudioUnitProcess_orig);
}
A few words about what's going on. AudioUnitProcess function is used for processing audio streams in order to apply some effects, mix, convert etc. We are hooking AudioUnitProcess in order to access phone call's audio streams. While phone call is active these streams are being processed in various ways.
We are listening for CoreTelephony notifications in order to get phone call status changes. When we receive audio samples we need to determine where they come from - microphone or speaker. This is done using componentSubType field in AudioComponentDescription structure. Now, you might think, why don't we store AudioUnit objects so that we don't need to check componentSubType every time. I did that but it will break everything when you switch speaker on/off on iPhone 5 because AudioUnit objects will change, they are recreated. So, now we open audio files (one for microphone and one for speaker) and write samples in them, simple as that. When phone call ends we will receive appropriate CoreTelephony notification and close the files. We have two separate files with audio from microphone and speaker that we need to merge. This is what void Convert() is for. It's pretty simple if you know the API. I don't think I need to explain it, comments are enough.
About locks. There are many threads in mediaserverd. Audio processing and CoreTelephony notifications are on different threads so we need some kind synchronization. I chose spin locks because they are fast and because the chance of lock contention is small in our case. On iPhone 4S and even iPhone 5 all the work in AudioUnitProcess should be done as fast as possible otherwise you will hear hiccups from device speaker which obviously not good.
Yes. Audio Recorder by a developer named Limneos does that (and quite well). You can find it on Cydia. It can record any type of call on iPhone 5 and up without using any servers etc'. The call will be placed on the device in an Audio file. It also supports iPhone 4S but for speaker only.
This tweak is known to be the first tweak ever that managed to record both streams of audio without using any 3rd party severs, VOIP or something similar.
The developer placed beeps on the other side of the call to alert the person you are recording but those were removed too by hackers across the net. To answer your question, Yes, it's very much possible, and not just theoretically.
Further reading
https://stackoverflow.com/a/19413363/202451
http://forums.macrumors.com/showthread.php?t=1566350
https://github.com/nst/iOS-Runtime-Headers
The only solution I can think of is to use the Core Telephony framework, and more specifically the callEventHandler property, to intercept when a call is coming in, and then to use an AVAudioRecorder to record the voice of the person with the phone (and maybe a little of the person on the other line's voice). This is obviously not perfect, and would only work if your application is in the foreground at the time of the call, but it may be the best you can get. See more about finding out if there is an incoming phone call here: Can we fire an event when ever there is Incoming and Outgoing call in iphone?.
EDIT:
.h:
#import <AVFoundation/AVFoundation.h>
#import<CoreTelephony/CTCallCenter.h>
#import<CoreTelephony/CTCall.h>
#property (strong, nonatomic) AVAudioRecorder *audioRecorder;
ViewDidLoad:
NSArray *dirPaths;
NSString *docsDir;
dirPaths = NSSearchPathForDirectoriesInDomains(
NSDocumentDirectory, NSUserDomainMask, YES);
docsDir = dirPaths[0];
NSString *soundFilePath = [docsDir
stringByAppendingPathComponent:#"sound.caf"];
NSURL *soundFileURL = [NSURL fileURLWithPath:soundFilePath];
NSDictionary *recordSettings = [NSDictionary
dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:AVAudioQualityMin],
AVEncoderAudioQualityKey,
[NSNumber numberWithInt:16],
AVEncoderBitRateKey,
[NSNumber numberWithInt: 2],
AVNumberOfChannelsKey,
[NSNumber numberWithFloat:44100.0],
AVSampleRateKey,
nil];
NSError *error = nil;
_audioRecorder = [[AVAudioRecorder alloc]
initWithURL:soundFileURL
settings:recordSettings
error:&error];
if (error)
{
NSLog(#"error: %#", [error localizedDescription]);
} else {
[_audioRecorder prepareToRecord];
}
CTCallCenter *callCenter = [[CTCallCenter alloc] init];
[callCenter setCallEventHandler:^(CTCall *call) {
if ([[call callState] isEqual:CTCallStateConnected]) {
[_audioRecorder record];
} else if ([[call callState] isEqual:CTCallStateDisconnected]) {
[_audioRecorder stop];
}
}];
AppDelegate.m:
- (void)applicationDidEnterBackground:(UIApplication *)application//Makes sure that the recording keeps happening even when app is in the background, though only can go for 10 minutes.
{
__block UIBackgroundTaskIdentifier task = 0;
task=[application beginBackgroundTaskWithExpirationHandler:^{
NSLog(#"Expiration handler called %f",[application backgroundTimeRemaining]);
[application endBackgroundTask:task];
task=UIBackgroundTaskInvalid;
}];
This is the first time using many of these features, so not sure if this is exactly right, but I think you get the idea. Untested, as I do not have access to the right tools at the moment. Compiled using these sources:
Recording voice in background using AVAudioRecorder
http://prassan-warrior.blogspot.com/2012/11/recording-audio-on-iphone-with.html
Can we fire an event when ever there is Incoming and Outgoing call in iphone?
Apple does not allow it and does not provide any API for it.
However, on a jailbroken device I'm sure it's possible. As a matter of fact, I think it's already done. I remember seeing an app when my phone was jailbroken that changed your voice and recorded the call - I remember it was a US company offering it only in the states. Unfortunately I don't remember the name...
I guess some hardware could solve this. Connected to the minijack-port; having earbuds and a microphone passing through a small recorder. This recorder can be very simple. While not in conversation the recorder could feed the phone with data/the recording (through the jack-cable). And with a simple start button (just like the volum controls on the earbuds) could be sufficient for timing the recording.
Some setups
http://www.danmccomb.com/posts/483/how-to-record-iphone-conversations-using-zoom-h4n/
http://forums.macrumors.com/showthread.php?t=346430

Resources