iOS SoundTouch framework BPM Detection example - ios

I have searched all over the web and cannot find a tutorial on how to use the SoundTouch library for beat detection.
(Note: I have no C++ experience prior to this. I do know C, Objective-C, and Java. So I could have messed some of this up, but it compiles.)
I added the framework to my project and managed to get the following to compile:
NSString *path = [[NSBundle mainBundle] pathForResource:#"song" ofType:#"wav"];
NSData *data = [NSData dataWithContentsOfFile:path];
player =[[AVAudioPlayer alloc] initWithData:data error:NULL];
player.volume = 1.0;
player.delegate = self;
[player prepareToPlay];
[player play];
NSUInteger len = [player.data length]; // Get the length of the data
soundtouch::SAMPLETYPE sampleBuffer[len]; // Create buffer array
[player.data getBytes:sampleBuffer length:len]; // Copy the bytes into the buffer
soundtouch::BPMDetect *BPM = new soundtouch::BPMDetect(player.numberOfChannels, [[player.settings valueForKey:#"AVSampleRateKey"] longValue]); // This is working (tested)
BPM->inputSamples(sampleBuffer, len); // Send the samples to the BPM class
NSLog(#"Beats Per Minute = %f", BPM->getBpm()); // Print out the BPM - currently returns 0.00 for errors per documentation
The inputSamples(*samples, numSamples) song byte information confuses me.
How do I get these pieces of information from a song file?
I tried using memcpy() but it doesn't seem to be working.
Anyone have any thoughts?

After hours and hours of debugging and reading the limited documentation on the web, I modified a few things before stumbling upon this: You need to divide numSamples by numberOfChannels in the inputSamples() function.
My final code is like so:
NSString *path = [[NSBundle mainBundle] pathForResource:#"song" ofType:#"wav"];
NSData *data = [NSData dataWithContentsOfFile:path];
player =[[AVAudioPlayer alloc] initWithData:data error:NULL];
player.volume = 1.0; // optional to play music
player.delegate = self;
[player prepareToPlay]; // optional to play music
[player play]; // optional to play music
NSUInteger len = [player.data length];
soundtouch::SAMPLETYPE sampleBuffer[len];
[player.data getBytes:sampleBuffer length:len];
soundtouch::BPMDetect BPM(player.numberOfChannels, [[player.settings valueForKey:#"AVSampleRateKey"] longValue]);
BPM.inputSamples(sampleBuffer, len/player.numberOfChannels);
NSLog(#"Beats Per Minute = %f", BPM.getBpm());

I've tried this solution to read the BPM from mp3 files (using the TSLibraryImport class to convert to wav) inside the iOS Music Library:
MPMediaItem *item = [collection representativeItem];
NSURL *urlStr = [item valueForProperty:MPMediaItemPropertyAssetURL];
TSLibraryImport* import = [[TSLibraryImport alloc] init];
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSURL* destinationURL = [NSURL fileURLWithPath:[documentsDirectory stringByAppendingPathComponent:#"temp_data"]];
[[NSFileManager defaultManager] removeItemAtURL:destinationURL error:nil];
[import importAsset:urlStr toURL:destinationURL completionBlock:^(TSLibraryImport* import) {
NSString *outPath = [documentsDirectory stringByAppendingPathComponent:#"temp_data"];
NSData *data = [NSData dataWithContentsOfFile:outPath];
AVAudioPlayer *player =[[AVAudioPlayer alloc] initWithData:data error:NULL];
NSUInteger len = [player.data length];
int numChannels = player.numberOfChannels;
soundtouch::SAMPLETYPE sampleBuffer[1024];
soundtouch::BPMDetect *BPM = new soundtouch::BPMDetect(player.numberOfChannels, [[player.settings valueForKey:#"AVSampleRateKey"] longValue]);
for (NSUInteger i = 0; i <= len - 1024; i = i + 1024) {
NSRange r = NSMakeRange(i, 1024);
//NSData *temp = [player.data subdataWithRange:r];
[player.data getBytes:sampleBuffer range:r];
int samples = sizeof(sampleBuffer) / numChannels;
BPM->inputSamples(sampleBuffer, samples); // Send the samples to the BPM class
}
NSLog(#"Beats Per Minute = %f", BPM->getBpm());
}];
The strangeness is that the calculated BMP is always the same value:
2013-10-02 03:05:36.725 AppTestAudio[1464:1803] Beats Per Minute = 117.453835
No matter which track was i.e. number of frames or the buffer size (here I used 2K buffer size as for the SoundTouch example in the source code of the library).

For Swift 3:
https://github.com/Luccifer/BPM-Analyser
And use it like:
guard let filePath = Bundle.main.path(forResource: "TestMusic", ofType: "m4a"),
let url = URL(string: filePath) else {return "error occured, check fileURL"}
BPMAnalyzer.core.getBpmFrom(url, completion: nil)
Feel free to comment!

Related

Best way to save lots of streaming data from iPhone

Currently, I am streaming image and depth data from my iphone X and doing a series of computations on it using a compute metal kernel. The output every cycle (frame) is approximately 900,000 floats. Assuming there are 30 cycles (frames) a second, this represents 27million floats a second.
I would like to save a few seconds (3-5) of this data to a file, which I can then access later.
My current approach of writing the output every cycle is too slow (I/O is expensive) and I am not sure how to buffer the results of a few cycles correctly (I tried creating an array of arrays but then run into memory issues).
Copy data from mtl buffers:
//number of elements per frame approx 300,000
size_t len = elementCount
finalArrayX = new float [len];
finalArrayY = new float [len];
finalArrayZ = new float [len];
//MTLBuffers
finalArrayX = (float*) resultsBufferX.contents;
finalArrayY = (float*) resultsBufferY.contents;
finalArrayZ = (float*) resultsBufferZ.contents;
writeToFile(finalArrayX, finalArrayY, finalArrayZ)
writeToFile:
/*
* Get file handle to append to
*/
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory,NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *documentTXTPath = [documentsDirectory stringByAppendingPathComponent:#"test.txt"];
if(![[NSFileManager defaultManager] fileExistsAtPath: documentTXTPath]) {
[[NSFileManager defaultManager] createFileAtPath: documentTXTPath contents:nil attributes:nil];
}
NSFileHandle *myHandle = [NSFileHandle fileHandleForWritingAtPath:documentTXTPath];
[myHandle seekToEndOfFile];
/*
* Create strings and write line by line
*/
for(int k = 0; k < elementCount; k ++){
NSString *x_i = [NSString stringWithFormat:#"%.2f", finalArrayX[k]];
NSString *y_i = [NSString stringWithFormat:#"%.2f", finalArrayY[k]];
NSString *z_i = [NSString stringWithFormat:#"%.2f", finalArrayZ[k]];
NSString *temp = [line stringByAppendingString: x_i];
temp = [temp stringByAppendingString: #" "];
NSString *tempTwo = [temp stringByAppendingString: y_i];
tempTwo = [tempTwo stringByAppendingString: #" "];
NSString *tempThree = [tempTwo stringByAppendingString: z_i];
NSString *totalLine = [tempThree stringByAppendingString: #"\n"];
[myHandle writeData:[totalLine dataUsingEncoding:NSUTF8StringEncoding]];
}
Whats the best way to go about this?

Can't reverse AVAsset audio properly. The only result is white noise

I'm trying to reverse an AVAsset audio and save it to a file. To make things clear, I've made simple application with the issue https://github.com/ksenia-lyagusha/AudioReverse.git
The application takes mp4 video file from bundle, exports it to Temporary folder in the sandbox as single m4a file, then tries to read it from there, reverse and save result file back.
Temporary m4a file is OK.
The only result of my reverse part is Audio file in the Sandbox with white noise.
There is the part of code below, that is in charge of reversing AVAsset. It is based on related questions
How to reverse an audio file?
iOS audio manipulation - play local .caf file backwards
However, it doesn't work for me.
OSStatus theErr = noErr;
UInt64 fileDataSize = 0;
AudioFileID inputAudioFile;
AudioStreamBasicDescription theFileFormat;
UInt32 thePropertySize = sizeof(theFileFormat);
theErr = AudioFileOpenURL((__bridge CFURLRef)[NSURL URLWithString:inputPath], kAudioFileReadPermission, 0, &inputAudioFile);
thePropertySize = sizeof(fileDataSize);
theErr = AudioFileGetProperty(inputAudioFile, kAudioFilePropertyAudioDataByteCount, &thePropertySize, &fileDataSize);
UInt32 ps = sizeof(AudioStreamBasicDescription) ;
AudioFileGetProperty(inputAudioFile, kAudioFilePropertyDataFormat, &ps, &theFileFormat);
UInt64 dataSize = fileDataSize;
void *theData = malloc(dataSize);
// set up output file
AudioFileID outputAudioFile;
AudioStreamBasicDescription myPCMFormat;
myPCMFormat.mSampleRate = 44100;
myPCMFormat.mFormatID = kAudioFormatLinearPCM;
// kAudioFormatFlagsCanonical is deprecated
myPCMFormat.mFormatFlags = kAudioFormatFlagIsFloat | kAudioFormatFlagIsNonInterleaved;
myPCMFormat.mChannelsPerFrame = 1;
myPCMFormat.mFramesPerPacket = 1;
myPCMFormat.mBitsPerChannel = 32;
myPCMFormat.mBytesPerPacket = (myPCMFormat.mBitsPerChannel / 8) * myPCMFormat.mChannelsPerFrame;
myPCMFormat.mBytesPerFrame = myPCMFormat.mBytesPerPacket;
NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:#"ReverseAudio.caf"];
NSURL *outputURL = [NSURL fileURLWithPath:exportPath];
theErr = AudioFileCreateWithURL((__bridge CFURLRef)outputURL,
kAudioFileCAFType,
&myPCMFormat,
kAudioFileFlags_EraseFile,
&outputAudioFile);
//Read data into buffer
//if readPoint = dataSize, then bytesToRead = 0 in while loop and
//it is endless
SInt64 readPoint = dataSize-1;
UInt64 writePoint = 0;
while(readPoint > 0)
{
UInt32 bytesToRead = 2;
AudioFileReadBytes(inputAudioFile, false, readPoint, &bytesToRead, theData);
// bytesToRead is now the amount of data actually read
UInt32 bytesToWrite = bytesToRead;
AudioFileWriteBytes(outputAudioFile, false, writePoint, &bytesToWrite, theData);
// bytesToWrite is now the amount of data actually written
writePoint += bytesToWrite;
readPoint -= bytesToRead;
}
free(theData);
AudioFileClose(inputAudioFile);
AudioFileClose(outputAudioFile);
If I change file type in AudioFileCreateWithURL from kAudioFileCAFType to another the result file is not created in the Sandbox at all.
Thanks for any help.
You get white noise because your in and out file formats are incompatible. You have different sample rates and channels and probably other differences. To make this work you need to have a common (PCM) format mediating between reads and writes. This is a reasonable job for the new(ish) AVAudio frameworks. We read from file to PCM, shuffle the buffers, then write from PCM to file. This approach is not optimised for large files, as all data is read into the buffers in one go, but is enough to get you started.
You can call this method from your getAudioFromVideo completion block. Error handling ignored for clarity.
- (void)readAudioFromURL:(NSURL*)inURL reverseToURL:(NSURL*)outURL {
//prepare the in and outfiles
AVAudioFile* inFile =
[[AVAudioFile alloc] initForReading:inURL error:nil];
AVAudioFormat* format = inFile.processingFormat;
AVAudioFrameCount frameCount =(UInt32)inFile.length;
NSDictionary* outSettings = #{
AVNumberOfChannelsKey:#(format.channelCount)
,AVSampleRateKey:#(format.sampleRate)};
AVAudioFile* outFile =
[[AVAudioFile alloc] initForWriting:outURL
settings:outSettings
error:nil];
//prepare the forward and reverse buffers
self.forwaredBuffer =
[[AVAudioPCMBuffer alloc] initWithPCMFormat:format
frameCapacity:frameCount];
self.reverseBuffer =
[[AVAudioPCMBuffer alloc] initWithPCMFormat:format
frameCapacity:frameCount];
//read file into forwardBuffer
[inFile readIntoBuffer:self.forwaredBuffer error:&error];
//set frameLength of reverseBuffer to forwardBuffer framelength
AVAudioFrameCount frameLength = self.forwaredBuffer.frameLength;
self.reverseBuffer.frameLength = frameLength;
//iterate over channels
//stride is 1 or 2 depending on interleave format
NSInteger stride = self.forwaredBuffer.stride;
for (AVAudioChannelCount channelIdx = 0;
channelIdx < self.forwaredBuffer.format.channelCount;
channelIdx++) {
float* forwaredChannelData =
self.forwaredBuffer.floatChannelData[channelIdx];
float* reverseChannelData =
self.reverseBuffer.floatChannelData[channelIdx];
int32_t reverseIdx = 0;
//iterate over samples, allocate to reverseBuffer in reverse order
for (AVAudioFrameCount frameIdx = frameLength;
frameIdx >0;
frameIdx--) {
float sample = forwaredChannelData[frameIdx*stride];
reverseChannelData[reverseIdx*stride] = sample;
reverseIdx++;
}
}
//write reverseBuffer to outFile
[outFile writeFromBuffer:self.reverseBuffer error:nil];
}
I wasn't able to find the problem in your code, however I suggest you reversing AVAsset using AVAssetWriter. Following code is based on iOS reverse audio through AVAssetWritet. I've added additional method there to make it work. Finally I've got reversed file.
static NSMutableArray *samples;
static OSStatus sampler(CMSampleBufferRef sampleBuffer, CMItemCount index, void *refcon)
{
[samples addObject:(__bridge id _Nonnull)(sampleBuffer)];
return noErr;
}
- (void)reversePlayAudio:(NSURL *)inputURL
{
AVAsset *asset = [AVAsset assetWithURL:inputURL];
AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:asset error:nil];
AVAssetTrack* audioTrack = [[asset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
NSMutableDictionary* audioReadSettings = [NSMutableDictionary dictionary];
[audioReadSettings setValue:[NSNumber numberWithInt:kAudioFormatLinearPCM]
forKey:AVFormatIDKey];
AVAssetReaderTrackOutput* readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:audioReadSettings];
[reader addOutput:readerOutput];
[reader startReading];
NSDictionary *outputSettings = #{AVFormatIDKey : #(kAudioFormatMPEG4AAC),
AVSampleRateKey : #(44100.0),
AVNumberOfChannelsKey : #(1),
AVEncoderBitRateKey : #(128000),
AVChannelLayoutKey : [NSData data]};
AVAssetWriterInput *writerInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeAudio
outputSettings:outputSettings];
NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:#"reverseAudio.m4a"];
NSURL *exportURL = [NSURL fileURLWithPath:exportPath];
NSError *writerError = nil;
AVAssetWriter *writer = [[AVAssetWriter alloc] initWithURL:exportURL
fileType:AVFileTypeAppleM4A
error:&writerError];
[writerInput setExpectsMediaDataInRealTime:NO];
writer.shouldOptimizeForNetworkUse = NO;
[writer addInput:writerInput];
[writer startWriting];
[writer startSessionAtSourceTime:kCMTimeZero];
CMSampleBufferRef sample;// = [readerOutput copyNextSampleBuffer];
samples = [[NSMutableArray alloc] init];
while (sample != NULL) {
sample = [readerOutput copyNextSampleBuffer];
if (sample == NULL)
continue;
CMSampleBufferCallForEachSample(sample, &sampler, NULL);
CFRelease(sample);
}
NSArray* reversedSamples = [[samples reverseObjectEnumerator] allObjects];
for (id reversedSample in reversedSamples) {
if (writerInput.readyForMoreMediaData) {
[writerInput appendSampleBuffer:(__bridge CMSampleBufferRef)(reversedSample)];
}
else {
[NSThread sleepForTimeInterval:0.05];
}
}
[samples removeAllObjects];
[writerInput markAsFinished];
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
dispatch_async(queue, ^{
[writer finishWritingWithCompletionHandler:^{
// writing is finished
// reversed audio file in TemporaryDirectory in the Sandbox
}];
});
}
Known issues of the code.
There might be some problems with the memory, if the audio is long.
The audio file's duration is longer than original's. (As a quick fix you might cut it down as usual AVAsset).

Whats is the easiest way to randomize sounds

I have 11 sounds for each soundpack. They are named:
testpack1.mp3,
testpack2.mp3 and so on.
My player initialises them with this code:
NSString * strName = [NSString stringWithFormat:#"testpack%ld", (long) (value+1)];
NSString * strPath = [[NSBundle mainBundle] pathForResource:strName ofType:#"mp3"];
NSURL * urlPath = [NSURL fileURLWithPath:strPath];
self.audioplayer = [[AVAudioPlayer alloc] initWithContentsOfURL:urlPath error:NULL];
This sounds will be played by pressing buttons. For example, I have generated 4 buttons, this 4 buttons play every time only testpack1-4.mp3 but I want that my player takes random out of the 11 sounds. What's the easiest solution?
Note: I don't want to repeat the mp3 unless all are played
How about this?
int randNum = rand() % (11 - 1) + 1;
The formuale is like below
int randNum = rand() % (maxNumber - minNumber) + minNumber;
A suggestion:
It declares 3 variables as static variables, played is a simple C-Array
static UInt32 numberOfSounds = 11;
static UInt32 counter = 0;
static UInt32 played[11];
The method playSound() resets the C-Array to zero values if the counter is 0 and sets the counter to the number of sounds.
When the method is called the random generator creates an index number.
If the value at that index in the array is 0 the sound is played, the index in the array is set and the counter is decreased.
If the sound at that index has been played yet, loop until a non-used index is found.
- (void)playSound
{
if (counter == 0) {
for (int i = 0; i < numberOfSounds; i++) {
played[i] = 0;
}
counter = numberOfSounds;
}
BOOL found = NO;
do {
UInt32 value = arc4random_uniform(numberOfSounds) + 1;
if (played[value - 1] != value) {
NSString * strName = [NSString stringWithFormat:#"testpack1-%d", value];
NSString * strPath = [[NSBundle mainBundle] pathForResource:strName ofType:#"mp3"];
NSURL * urlPath = [NSURL fileURLWithPath:strPath];
self.audioplayer = [[AVAudioPlayer alloc] initWithContentsOfURL:urlPath error:NULL];
played[value - 1] = value;
counter--;
found = YES;
}
} while (found == NO);
}

iOS SoundTouch framework song BPM Detection example

I am still looking for a working code which calculates BPM, did any of you solve this code? Appreciate any help.
This is the code I have and it returns either 0 value or wrong value (compared with BPM value online)
-(void) calcBPM {
NSString *path = [[NSBundle mainBundle] pathForResource:#"song_test" ofType:#"mp3"];
NSData *data = [NSData dataWithContentsOfFile:path];
AVAudioPlayer *player =[[AVAudioPlayer alloc] initWithData:data error:NULL];
NSUInteger len = [player.data length];
soundtouch::SAMPLETYPE sampleBuffer[len];
[player.data getBytes:sampleBuffer length:len];
soundtouch::BPMDetect BPM(player.numberOfChannels, [[player.settings valueForKey:#"AVSampleRateKey"] longValue]);
BPM.inputSamples(sampleBuffer, len/player.numberOfChannels);
NSLog(#"Beats Per Minute = %f", BPM.getBpm());
}

Unzipping NSData with ZipArchive

I am using ZipArchive and am reading data in from a device. What I want to do is take the data from the device and unzip it. The only examples I can find are taking the data, writing it to a file on the iOS device, and reading (unzipping) it back again...
This is the code (not working) that I have writing it to disk...
// Read the data in...
if ((recvStringLen = recvfrom(connectSock, recvString, 1025, 0, (struct sockaddr *)&broadcastAddr, &recvStringLen)) < 0) {
NSLog(#"ERROR: Unable to receive user perms message.");
[self showTimeoutError];
return -1;
}
// get the file size...
unsigned int fileSize = (( recvString[16] << 24 ) & 0xff000000) |
(( recvString[17] << 16 ) & 0xff0000) |
(( recvString[18] << 8 ) & 0xff00 ) |
( recvString[19] & 0xff );
NSLog(#"fileSize: %i", fileSize);
recvString[20+fileSize] = '\0'; // terminate the string...
// convert the char data to a string
NSString *stringData = [[[NSString alloc] initWithFormat:#"%s", &recvString[20]] autorelease];
NSData *data = [[[NSData alloc] initWithBytes:stringData length:fileSize] autorelease];
// write the data...
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *appFile = [documentsDirectory stringByAppendingPathComponent:#"userinfo.zip"];
[data writeToFile:appFile atomically:YES];
// exist?
BOOL zipExists = [[NSFileManager defaultManager] fileExistsAtPath:appFile];
NSString *zipFilePath = [documentsDirectory stringByAppendingPathComponent:#"userinfo.zip"];
NSString *output = [documentsDirectory stringByAppendingPathComponent:#"userinfo.dat"];
ZipArchive* za = [[ZipArchive alloc] init];
// unzip it...
BOOL success = [za UnzipOpenFile:zipFilePath];
if( success ) {
BOOL outSuccess = [za UnzipFileTo:output overWrite:YES];
if( outSuccess != NO ) {
NSLog(#"success");
}
[za UnzipCloseFile];
}
[za release];
BOOL datExists = [[NSFileManager defaultManager] fileExistsAtPath:output];
When it gets to 'success', it is always NO... even though 'zipExists' is YES.
The two issues I have:
1. The I haven't successfully written the data to disk, read it back, and unzipped it.
2. I would rather just unzip the data (NSData) instead of going thru writing it to disk and reading it back again...
I searched before posting, but was unable to find a solution...
Skip creating a NSString - that is your problem (or one of them). Create the data object from recvString directly. Think about what happens using %s if the data has a null byte.

Resources