I'm aiming to create an audio-visualisation app for iOS. I need to somehow tap into the current audio output from another app (such as Apple's music app, or Spotify) and get the amplitude of the signal for each sample of the music. I will then perform an FFT algorithm on the data to convert it to the frequency-domain and display the data visually.
Is it possible to read this data from the audio output of other apps? If so, what do I need to use to extract this data?
No. The iOS security sandbox will prevent the reading of any audio samples from other apps via any public API (unless the playing app was coded to explicitly export audio data via inter-app audio or other similar interface).
Related
I'm new to iOS programming and I don't know where to start. I found code examples how to read frequencies from the microphone with AudioKit framework. But this is not what I am looking for. Is it possible to retrieving frequency of the currently playing song in real time without using a microphone?
Thank you for help.
The iOS security sandbox prevents apps from capturing general audio output of any other app, such as the Music app.
Certain music apps, such as GarageBand might share inter-app audio, but this isn't supported by the majority of apps that output "songs".
An app might play the "song" itself, via an AVAudioPlayer, and tap the AVPlayer's output to get raw sample data for spectral frequency and pitch analysis (two very different things, by-the-way).
We're currently using Linphone library to make VOIP calls and they have their own solution for audio playback. However, we would like to display a visualizer for the audio that Linphone is outputting from within our own app. Is there a way that we can intercept this data (maybe through sample buffering) in order to draw up audio waves/volume meter in the user interface?
AVAudioPlayer or AVPlayer is out of the question since we do not have access to those objects. Is there a solution in place for AVAudioSession or in CoreAudio?
Only if the audio output app is exporting the audio data using Inter-App-audio or Audiobus. Otherwise the iOS security sandbox will hide that audio output from your app.
I am looking at the feasibility of getting the current raw audio stream playing and do stuff with it such as stream it over Bluetooth or equalize it, etc. Is there any way to do this in iOS 8?
For example: apps such as Pandora/Spotify are playing music and I want to access the audio they are playing.
To process audio from another app, that app needs to participate in Inter-App Audio.
I don't know if your example apps do that.
AIM: I am trying to integrate my iPhone app with a bluetooth device that can play music and display some visualisation (yet to choose bluetooth hardware). There are three things that puzzles me:
I have seen some sample programs that generate visualisation data based on averagePowerForChannel using AVAudioPlayer. Is it possible to extract such information for a music stored in iPhone filesystem without actually (or before) playing the music? Since iPhone apps cannot execute in background infinitely, extracting this data realtime may not advised?
Once I create this "visualisation" data for an mp3, can how can I send the data to my bluetooth device? I know we can sent custom data only via BLE. Is there a possibility that I can send music data and visualisation data to the bluetooth device at the same time?(or sent visualisation data for a few seconds first and then start streaming the music) - This advice will help me to choose a proper hardware, provided iPhone supports this!
If I cannot sent music data and visualisation data at the same time over bluetooth, I am planning to extract visualisation data first from the mp3(how?), send it over to be stored in the bluetooth device and then display accordingly, in sync with the music data received in real time. Is this possible?
I want to process the stereo output from iOS devices, no matter what application causes them and visualize it in real-time.
Is it possible to use the generic output device (or anything else) to get at the audio data which are currently being played? Maybe as an input to a remoteIO unit?
In other words: I want to do what aurioTouch2 does (FFT only) but instead of using the microphone as input source, I want to process everything which is coming out of the speakers at a given time.
Kind regards
If your own app is playing using the RemoteIO Audio Unit, you can capture that content. You can not capture audio your app is playing using many of the other audio APIs. The iOS security sandbox will prevent your app from capturing audio that any other app is playing (unless that app explicitly exports audio via the Inter-App Audio API or equivalent).