I'd like to crossfade from one track to the next in a Spotify enabled app. Both tracks are Spotify tracks, and since only one data stream at a time can come from Spotify, I suspect I need to buffer (I think I can read ahead 1.5 x playback speed) the last few seconds of the first track, start the stream for track two, fade out one and fade in two using an AudioUnit.
I've reviewed sample apps:
Viva - https://github.com/iKenndac/Viva SimplePlayer with EQ - https://github.com/iKenndac/SimplePlayer-with-EQ and tried to get my mind around the SPCircularBuffer, but I still need help. Could someone point me to another example or help bullet-point a track crossfade game plan?
Update: Thanks to iKenndac, I'm about 95% there. I'll post what I have so far:
in SPPlaybackManager.m: initWithPlaybackSession:(SPSession *)aSession {
added:
self.audioController2 = [[SPCoreAudioController alloc] init];
self.audioController2.delegate = self;
and in
- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context {
...
self.audioController.audioOutputEnabled = self.playbackSession.isPlaying;
// for crossfade, add
self.audioController2.audioOutputEnabled = self.playbackSession.isPlaying;
and added a new method based on playTrack
-(void)crossfadeTrack:(SPTrack *)aTrack callback:(SPErrorableOperationCallback)block {
// switch audiocontroller from current to other
if (self.playbackSession.audioDeliveryDelegate == self.audioController)
{
self.playbackSession.audioDeliveryDelegate = self.audioController2;
self.audioController2.delegate = self;
self.audioController.delegate = nil;
}
else
{
self.playbackSession.audioDeliveryDelegate = self.audioController;
self.audioController.delegate = self;
self.audioController2.delegate = nil;
}
if (aTrack.availability != SP_TRACK_AVAILABILITY_AVAILABLE) {
if (block) block([NSError spotifyErrorWithCode:SP_ERROR_TRACK_NOT_PLAYABLE]);
self.currentTrack = nil;
}
self.currentTrack = aTrack;
self.trackPosition = 0.0;
[self.playbackSession playTrack:self.currentTrack callback:^(NSError *error) {
if (!error)
self.playbackSession.playing = YES;
else
self.currentTrack = nil;
if (block) {
block(error);
}
}];
}
this starts a timer for crossfade
crossfadeTimer = [NSTimer scheduledTimerWithTimeInterval: 0.5
target: self
selector: #selector ( crossfadeCountdown)
userInfo: nil
repeats: YES];
And in order to keep the first track playing after its data has loaded in SPCoreAudioController.m I changed target buffer length:
static NSTimeInterval const kTargetBufferLength = 20;
and in SPSession.m : end_of_track(sp_session *session) {
I removed
// sess.playing = NO;
I call preloadTrackForPlayback: about 15 seconds before end of track, then crossfadeTrack: at 10 seconds before.
Then set crossfadeCountdownTime = [how many seconds you want the crossfade]*2;
I fade volume over the crosssfade with:
- (void) crossfadeCountdown
{
[UIAppDelegate.playbackSPManager setVolume:(1- (((float)crossfadeCountdownTime/ (thisCrossfadeSeconds*2.0)) *0.2) )];
crossfadeCountdownTime -= 0.5;
if (crossfadeCountdownTime == 1.0)
{
NSLog(#"Crossfade countdown done");
crossfadeCountdownTime = 0;
[crossfadeTimer invalidate];
crossfadeTimer = nil;
[UIAppDelegate.playbackSPManager setVolume:1.0];
}
}
I'll keep working on it, and update if I can make it better. Thanks again to iKenndac for his always spot-on help!
There isn't a pre-written crossfade example that I'm aware of that uses CocoaLibSpotify. However, a (perhaps not ideal) game plan would be:
Make two separate audio queues. SPCoreAudioController is an encapsulation of an audio queue, so you should just be able to instantiate two of them.
Play music as normal to one queue. When you're approaching the end of the track, call SPSession's preloadTrackForPlayback:callback: method with the next track to get it ready.
When all audio data for the playing track has been delivered, SPSession will fire the audio delegate method sessionDidEndPlayback:. This means that all audio data has been delivered. However, since CocoaLibSpotify buffers the audio from libspotify, there's still some time before audio stops.
At this point, start playing the new track but divert the audio data to the second audio queue. Start ramping down the volume of the first queue while ramping up the volume of the next one. This should give a pleasing crossfade.
A few pointers:
In SPCoreAudioController.m, you'll find the following line, which defines how much audio CocoaLibSpotify buffers, in seconds. If you want a bigger crossfade, you'll need to increase it.
static NSTimeInterval const kTargetBufferLength = 0.5;
Since you get audio data at a maximum of 1.5x actual playback speed, be careful not to do, for example, a 5 second crossfade when the user has just skipped near to the end of the track. You might not have enough audio data available to pull it off.
Take a good look at SPPlaybackManager.m. This class is the interface between CocoaLibSpotify and Core Audio. It's not too complicated, and understanding it will get you a long way. SPCoreAudioController and SPCircularBuffer are pretty much implementation details of getting the audio into Core Audio, and you shouldn't need to understand their implementations to achieve what you want.
Also, make sure you understand the various delegates SPSession has. The audio delivery delegate only has one job - to receive audio data. The playback delegate gets all other playback events - when audio has finished being delivered to the audio delivery delegate, etc. There's nothing stopping one class being both, but in the current implementation, SPPlaybackManager is the playback delegate, which creates an instance of SPCoreAudioController to be the audio delivery delegate. If you modify SPPlaybackManager to have two Core Audio controllers and alternate which one is the audio delivery delegate, you should be golden.
Related
This is the code I am using to play a sound with AVAudioPlayerNode. It is just playing the last sound of the beats array. On iPhone, all sounds from the beats array are playing simultaneously with the same function.
-(void)playMix{
for (int i = 0; i< mix.beatsArray.count;i++) {
beat = [[WatchBeatObject alloc] initWithFileName:mix.beatsArray[i].fileName];
[beat.audioPlayerNode play];
[beat.audioPlayerNode scheduleBuffer:beat.buffer atTime:nil options:AVAudioPlayerNodeBufferLoops completionHandler:^{
}];
}
}
N.B: Method initWithFileName: handles initializing and creating AVAudioPlayerNode and everything needed.
Thank you in advance.
Does that WatchBeatObject has any category option in AVAudioSession to set AVAudioSessionCategoryAmbient? If yes, set it, they would mix different sound.
I am making an app that plays back audio and I have set it up so that the lock screen gets updated through MPNowPlayingInfoCenter, but I've run into a problem.
At seemingly random times, I get an EXC_BAD_ACCESS error when trying to update the now playing info.
Here's the code that does so:
- (void)updatePlayback
{
if(!active)
return;
NowPlayingController* npc = [AudioController nowPlayingController];
CMTime elapsed = player.currentTime;
Float64 elInterval = CMTimeGetSeconds(elapsed);
[npc setElapsed:elInterval];
CMTime duration = player.currentItem.duration;
Float64 durInterval = CMTimeGetSeconds(duration);
[npc setRemaining:ceilf(durInterval - elInterval)];
[npc setPlayPauseValue:isPlaying];
if(durInterval > 0)
{
[npc setProgressValue:elInterval/durInterval];
[npc setAudioDuration:durInterval];
}
_activeMetadata[MPMediaItemPropertyPlaybackDuration] = #(durInterval);
_activeMetadata[MPNowPlayingInfoPropertyPlaybackRate] = #(isPlaying);
_activeMetadata[MPNowPlayingInfoPropertyElapsedPlaybackTime] = #(elInterval);
MPNowPlayingInfoCenter* npInfoCenter = [MPNowPlayingInfoCenter defaultCenter];
if(npInfoCenter && _activeMetadata)
{
if([npInfoCenter respondsToSelector:#selector(setNowPlayingInfo:)])
{
//////////THE FOLLOWING LINE TRIGGERS EXC_BAD_ACCESS SOMETIMES////////////
[npInfoCenter setNowPlayingInfo:_activeMetadata];
}
}
}
99.9% of the time, this works, but sometimes when resigning the app to the background or when changing audio files, or just randomly,
[npInfoCenter setNowPlayingInfo:_activeMetadata];
throws EXC_BAD_ACCESS.
Also, _activeMetadata is declared as:
#property (atomic, strong, retain) NSMutableDictionary* activeMetadata;
It is instantiated when the AVPlayer is created:
AVAsset* asset = [AVAsset assetWithURL:[NSURL fileURLWithPath:path]];
AVPlayerItem* playerItem = [AVPlayerItem playerItemWithAsset:asset];
player = [AVPlayer playerWithPlayerItem:playerItem];
CMTime duration = player.currentItem.duration;
NSTimeInterval durInterval = CMTimeGetSeconds(duration);
NSLog(#"%f", durInterval);
MPMediaItemArtwork* albumArtwork = [[MPMediaItemArtwork alloc] initWithImage:[downloader useCachedImage:CacheKeySeriesBanners withName:nil withURL:info[#"image"]]];
NSDictionary* nowPlayingInfo = #{MPMediaItemPropertyTitle:ptString,
MPMediaItemPropertyArtist:spString,
MPMediaItemPropertyArtwork:albumArtwork,
MPMediaItemPropertyAlbumTitle:info[#"title"],
MPMediaItemPropertyPlaybackDuration:#(durInterval),
MPNowPlayingInfoPropertyPlaybackRate:#(1),
MPNowPlayingInfoPropertyElapsedPlaybackTime:#(0)};
[[MPNowPlayingInfoCenter defaultCenter] setNowPlayingInfo:nowPlayingInfo];
_activeMetadata = [nowPlayingInfo mutableCopy];
updatePlayback is called via a CADisplayLink on every frame.
Any ideas what could be causing the exception?
I think you're calling setNowPlayingInfo way too often. Granted, it really shouldn't crash but there's no need to use CADisplayLink to call it 60 times a second.
So why are you calling it so often? If it's because you want to progress bar to track smoothly, there's still no need. From the MPNowPlayingInfoPropertyElapsedPlaybackTime declaration:
// The elapsed time of the now playing item, in seconds.
// Note the elapsed time will be automatically extrapolated from the previously
// provided elapsed time and playback rate, so updating this property frequently
// is not required (or recommended.)
p.s. I tried the code with an m4a file and found durInterval was NotANumber. With the correct duration and calling setNowPlayingInfo only once, the progress bar tracked fine & nothing crashed.
Apple fixed this crash in iOS 10.3 and above.
So if you want to support iOS 10.2.1 and below, be sure to throttle how often you set [MPNowPlayingInfoCenter defaultCenter].nowPlayingInfo property. Perhaps limiting setting the property only once a second.
I have following code which performs stop running (video) session, delete last segment of video, store video to directory and starts new video recording. There is (naturally) gap between these two video segments. Is there any way to optimize this code (maybe assync optimization if possible)? By optimization I mean elimination time gap as much as possible between these two video segments, thank you.
- (void) restartVideoRecording {
[captureSession removeOutput:captureMovieOutput];
[captureMovieOutput stopRecording];
if(lastPathWasOne){
captureMoviePath = [[URLPathProvider getUrlPathProvider]videoTwoPathString];
[URLPathProvider deleteFileAtStringPath:captureMoviePath];
lastPathWasOne = NO;
} else {
captureMoviePath = [[URLPathProvider getUrlPathProvider]videoOnePathString];
[URLPathProvider deleteFileAtStringPath:captureMoviePath];
lastPathWasOne = YES;
}
captureMovieURL = [[NSURL alloc] initFileURLWithPath:captureMoviePath];
[captureSession addOutput:captureMovieOutput];
[captureMovieOutput startRecordingToOutputFileURL:captureMovieURL recordingDelegate:self];
}
[NSTimer scheduledTimerWithTimeInterval:loopDuration target:self selector:#selector(restartVideoRecording) userInfo:nil repeats:NO];}
Thank you very much
Yes, you can. Store captured samples in NSTemporaryDirectory and use AVMutableComposition to merge assets at the end of recording session.
there is a Sample code please check this also
I'm trying to make a simple Drumpad app. The app needs to be super fast and play the sound with as little latency as possible. The premise is store audio samples in an array and each one is played when you hit a pad.
The catch is that some pads are chords (and not drums) so they need to stop right away On Touch Up.
Here are the approaches I have tried:
System Sound - Very simple to implement, super responsive, no way to stop sounds without destroying them, sounds cant be more than 30 seconds.
AV Framework - Way to slow.
The Amazing Audio Engine - Seems nice, but not sure what the advantage is over CoreAudio as setup was rather complicated I wasn't able to get a sound to play. Also not sure about latency.
Novocaine - I settled on this for now, it seems very fast, and I can play sounds, but I've seen no way to stop them once they start. I don't know how I can stop a single sound without stopping the whole audioManager?
- (void) playSoundN:(int)padNum
{
__weak ViewController * wself = self;
NSURL *inputFileURL = [[NSBundle mainBundle] URLForResource:#"Drum - Kick" withExtension:#"wav"];
self.fileReader = [[AudioFileReader alloc]
initWithAudioFileURL:inputFileURL
samplingRate:self.audioManager.samplingRate
numChannels:self.audioManager.numOutputChannels];
[self.fileReader play];
self.fileReader.currentTime = 0.0;
[self.audioManager setOutputBlock:^(float *data, UInt32 numFrames, UInt32 numChannels)
{
[wself.fileReader retrieveFreshAudio:data numFrames:numFrames numChannels:numChannels];
NSLog(#"Time: %f", wself.fileReader.currentTime);
}];
[self.audioManager play];
}
Seems like a simple task, starting and stoping a sound with super low latency.
Should I use one of the above approaches or just bite the bullet and dive into Core Audio Units.
Anyone have any ideas?
I just made an app that has similar functionality to what you are trying to accomplish. My recommendation:
Use AVAudioPlayer (and AVAudioRecorder for recording).
Use the prepareToPlay method of the class to preload the sound and reduce lag.
You can read more about the classes in the AVFoundation documentation:
https://developer.apple.com/library/ios/DOCUMENTATION/AVFoundation/Reference/AVFoundationFramework/_index.html
Surprisedly, alexbw accepted the patch quickly. If you updated to newest code, you may simply use following lines in output block:
[weakSelf.reader retrieveFreshAudio:data numFrames:numFrames numChannels:numChannels];
if (!weakSelf.reader.playing) {
weakSelf.audioManager.outputBlock = nil;
dispatch_async(dispatch_get_main_queue(), ^{
// UI stuff
});
}
Novocaine cannot tell EOF of the file, the last buffer is used repeat.
Considering the author think AudioFileReader is just a piece of sugar and not likely to fix it, here is a dirty solution.
__block float lastTimeLeft = 0.0f;
[self.audioManager setOutputBlock:^(float *data, UInt32 numFrames, UInt32 numChannels) {
float timeLeft = weakSelf.reader.duration - [weakSelf.reader getCurrentTime];
if ( timeLeft > 0.01 && timeLeft != lastTimeLeft ) {
lastTimeLeft = timeLeft;
[weakSelf.reader retrieveFreshAudio:data numFrames:numFrames numChannels:numChannels];
} else {
[weakSelf.reader retrieveFreshAudio:data numFrames:numFrames numChannels:numChannels]; //forget this line will cause a buzz at the end.
[weakSelf.reader pause];
weakSelf.audioManager.outputBlock = nil;
}
}];
A better solution would be to patch the AudioFileReader.
I'm using AVPlayer's -addBoundaryTimeOserverForTimes:queue:usingBlock: to execute some code at a specific time in my video (in this case, I want a un-hide a button when my video reaches its duration. Code is as follows:
- (void)viewWillAppear:(BOOL)animated
{
...
_player = [AVPlayer playerWithURL:videoURL];
AVPlayerLayer *newPlayerLayer = [AVPlayerLayer playerLayerWithPlayer:_player];
[newPlayerLayer setFrame:_videoView.bounds];
[_videoView.layer addSublayer:newPlayerLayer];
_observer = [_player addBoundaryTimeObserverForTimes:#[[NSValue valueWithCMTime:_player.currentItem.duration]] queue:NULL usingBlock:^{
[someButton setHidden:NO];
}];
...
}
For whatever reason, sometimes the block of code fires and the button becomes visible, and sometimes it doesn't. Haven't been able to find a pattern in this behavior. It happens very often (almost always) in the simulator, and occasionally when on a device. Has anyone encountered this problem? Any ideas what might be going on?
Edit
Also, if I put a breakpoint on the block, it ALWAYS fires.
Main queue sometimes not call.
You can use Sub queue, and call Main queue in Sub-queue's block.
// dispatch queue setting
dispatch_queue_t subQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
// notification setting
__block id blockObserver;
blockObserver = [self.queuePlayer addBoundaryTimeObserverForTimes:boundary
queue:subQueue // if NULL use mainQueue
usingBlock:^{
// do something
dispatch_queue_t mainQueue = dispatch_get_main_queue();
dispatch_async(mainQueue, ^{
// do something
});
}];
For those wanting to observe when the player ends:
I browse this question about once a year because I always forget about the fix that works for me. This time around I had this issue on macOS. I am seeing the same behavior, the observer block sometimes does not get called. When I switch back from the app that is being debugged to Xcode the block suddenly fires. This might be related to having a breakpoint set in the block as described by the OP.
Here's the fix however: Simply switch to AVPlayerItemDidPlayToEndTimeNotification as described in this answer. Note however, as the name implies the notification's object is the player's current item not the player itself!
Because this notification triggers at the end time of an item, instead of observing some "boundary time" simply set the item's forwardPlaybackEndTime if you need another time than the item's actual end time, i.e. duration.