I need the same behaviour as seen in SnapChat, but for a longer duration.
I already have an AVCaptureSession that records video and audio from the built-in microphone.
What I need is to also record music running in the background, i.e. from Spotify, Apple Music etc.
I assume there is two parts to this:
1) Setting the correct category on AVAudionSession (in order to allow background music while recording)
2) Setting some (custom?) input AVCaptureDevice to grap audio playing in the background.
But how do I do it?
Spotify, iTunes etc generally have the music encrypted and the keys protected via a DRM mechanism, so in theory you can't capture or record the music.
Even if you do find a workaround, without the rights for the music you will face copyright issues if this is for an app you want to make available to others, unfortunately.
Related
I am developing an Karaoke app in which you can record your voice while listening to the music. When user uses headphones, everything is great - he can listen to the music and himself in headphones while singing. Then we have his pure voice recorded and we can mix it with playback.
Problem occurs when user does not use headphones. Then we play music via speakers AVAudioSessionCategoryPlayAndRecord and record simultaneously. In final recording we have user's voice and playback from speakers mixed together. Problem is that playback's volume is very big and it's "covering" user's voice. Firstly I thought that this is normal behaviour because speakers are close to microphone so there is nothing I can do.
However when I tried the same thing on Garage Band it somehow lowers playback from speakers making voice more hearable.
I also tried it with Instagram (you can record while playing music e.g. from Spotify) and I noticed that after ~1 sec. playback's volume is decreasing and we can hear voice more precisely.
I don't think that it's post processing because it would be very complicated so maybe there is an option to let "iOS handle it".
To be clear - it does not lowers playback during recording - it's "done" while listening final video.
I use AVCaptureSession for recording and AudioKit Player for playing.
Thanks in advance for any thoughts/tips/advices!
Regards
Ok so I asked Apple TS and the respond was exactly what I wanted: https://developer.apple.com/documentation/avfoundation/avaudiosession/mode/1616455-voicechat You just have to set this mode in AVAudioSession and system will handle it device’s tonal equalization is optimized for voice
iOS cannot 'just handle' that, there is no "filter out the music" function. The fact that it doesn't do it live, but does so later or with a delay strongly implies they are doing some post processing. I'm not a machine learning expert, but I think if you just used an equalizer and a noise gate you could get this effect. It'd be hard to extract an acapella but you could certainly improve it. Likely Instagram takes that second to identify where the voice frequencies are so it knows how to EQ the signal.
My app needs to play some music files, like .mp3. I would like to use MPMoviePlayerController because it has implemented all the UI stuff for me, i.e. I do not want to bother implementing progress slide bar and things like this.
I tested to use it to play a .mp3 file and it worked fine but I do not know if it is fine to use it to do this because its name says "movie player" and it seems it is supposed to play a movie. Would apple reject this? Thank you.
For playing audio from a file or memory, AVAudioPlayer is your best option but unfortunately it doesn't support a network stream while MPMoviePlayerController can
From documentation :
An instance of the AVAudioPlayer class, called an audio player,
provides playback of audio data from a file or memory.
Apple recommends that you use this class for audio playback unless you
are playing audio captured from a network stream or require very low
I/O latency.
For the Apple validation I don't think that your application can be rejected because you're using the Media Player Framework to play an audio file. In fact here they explicitly say that you can do just that:
Choose the right technology for your needs:
To play the audio items in a user’s iPod library, or to play local or
streamed movies, use the Media Player framework. Classes in this
framework automatically support sending audio and video to AirPlay
devices such as Apple TV.
Not sure about performance and memory issues though!
Best of luck.
I want to know -- what happens to a user's music when an AVPlayer starts to play? Does nothing happen, does it stop, or does the action cancel itself? If so, how would I check and see if the user is playing music?
Apple's Documentation for this confused me.
Alright, looking up the info for this, it seems that AVAudioPlayer objects only interrupts the user's audio if the app's AVAudioSession's category is set to Solo Ambient, the default.
I saw this answer: App with AVPlayer plays mp4 interrupt iPod music after launched
I also saw this table in the Apple Developer documentation that explains whether or not different categories interrupt the user's music, with examples of different scenarios, including one for a game: https://developer.apple.com/library/ios/documentation/UserExperience/Conceptual/MobileHIG/Sound.html#//apple_ref/doc/uid/TP40006556-CH44-SW1
In order to not interrupt the user's music, I need to set this to Ambient.
I am looking at the feasibility of getting the current raw audio stream playing and do stuff with it such as stream it over Bluetooth or equalize it, etc. Is there any way to do this in iOS 8?
For example: apps such as Pandora/Spotify are playing music and I want to access the audio they are playing.
To process audio from another app, that app needs to participate in Inter-App Audio.
I don't know if your example apps do that.
I'm designing a music app that plays music from the user's iPhone music library and I'm having problems figuring out which audio player is the right one to use for it (AVAudioPlayer, AVPlayer, or MPMusicPlayer).
My app needs to do the following:
Play music from the iPhone music library
Control app music volume separately from device/system volume
Continue to play app music when app moves to background
Catch events when song changes to next song or finishes
From my research it seems like each of the three audio players mentioned above do SOME of the tasks required for my app, but none of them do ALL of them. AVPlayer seems to get the closest, except its volume is dependent on the device/system volume.
Does anyone have any recommendations or workarounds to accomplish this? I've been wracking my brain over this for quite some time so any help at all would be appreciated.
EDIT
The MPMusicPlayerController class does not actually support playing background audio when getting an instance of it via + applicationMusicPlayer. The best option, then, is to simply use MPMusicPlayer to query the iPod music library. Once a song is selected by the user, the resulting MPMediaItem can be queried for it's asset URL and fed into AVPlayer's +playerWithURL, giving you full control of playback parameters.
--
I would recommend MPMusicPlayerController.
MPMusicPlayerControllerhandles the low-level details of playing audio files in the iTunes library.
You can use the class method -applicationMusicPlayer to get a local copy of the iPod singleton; from there, you can control its volume by setting its volume property for just your application.
I would assume MPMusicPlayerController has background audio support already built in. If it doesn't, you can change your app's audio session context to make it work (see Playing background audio in http://www.sagorin.org/2011/11/29/ios-playing-audio-in-background-audio/)
You can setup any object you choose to be a KVO observer of your application-specific MPMusicPlayerController object. This way, you can be notified whenever the nowPlayingItem property is changed.
Hopefully this high-level description suffices. Let me know if you need any further clarification.