custom resourceLoader with AVComposition - ios

Is there any way to use a custom resourceLoader with an AVComposition? I get the AVAssetResourceLoaderDelegate callbacks when I set the resourceLoader on an AVURLAsset and create an AVPlayerItem with that asset. But if I use that same AVURLAsset in an AVComposition, I don't get the delegate callbacks. In fact it never returns from the call to insert the asset into the composition.
AVAssetResourceLoaderDelegate methods called with AVURLAsset:
NSURL* mungedURL = [self mungeURL:itemURL];
AVURLAsset* asset = [[AVURLAsset alloc] initWithURL:mungedURL options:nil];
[asset.resourceLoader setDelegate:self queue:dispatch_get_main_queue()];
self.playerItem = [AVPlayerItem playerItemWithAsset:asset];
AVAssetResourceLoaderDelegate methods NOT called with AVURLAsset inserted into AVComposition:
NSURL* mungedURL = [self mungeURL:itemURL];
AVURLAsset* asset = [[AVURLAsset alloc] initWithURL:mungedURL options:nil];
[asset.resourceLoader setDelegate:self queue:dispatch_get_main_queue()];
AVMutableComposition* composition = [AVMutableComposition composition];
CMTimeRange timeRange = CMTimeRangeMake(kCMTimeZero, asset.duration);
[composition insertTimeRange:timeRange ofAsset:asset atTime:kCMTimeZero error:nil];
self.playerItem = [AVPlayerItem playerItemWithAsset:composition];
(actually, it hangs at insertTimeRange:ofAsset:atTime:error. I guess it tries to load from the munged url at that point - but doesn't use delegate.)
Method to tweak url scheme so resource loader delegate is called:
- (NSURL*)mungeURL:(NSURL*)url
{
NSURLComponents* components = [NSURLComponents componentsWithURL:url resolvingAgainstBaseURL:NO];
components.scheme = #"fake-scheme";
return [components URL];
}

Related

Memory Increases When Merging and Playing Audio

I'm trying to merge multiple pieces of audio into one simultaneous sound and then play it. I can merge and play, but my app's memory usage keeps increasing over time.
Looking online, it seems like there's ARC/memory issues with AVPlayer.
I've added all relevant code below for setting up the files, merging them and then playing them.
Setting up sound files
- (void) setUpSoundFiles {
AVURLAsset *songAsset = nil;
AVAssetTrack *sourceAudioTrack = nil;
for (int i = 0; i < numberOfSoundsToMerge; i++){
NSURL *url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:[#(i) stringValue] ofType:#"mp3"]];
songAsset = [AVURLAsset URLAssetWithURL:url options:nil];
sourceAudioTrack = [[songAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
[songTracks insertObject:sourceAudioTrack atIndex:i];
[songAssets insertObject:[AVURLAsset URLAssetWithURL:url options:nil] atIndex:i];
//[sourceAudioTrack.asset cancelLoading];
}
}
Merging Sound Files
- (void) mergeAudio: (AVMutableComposition *)composition{
for(int i = 0; i < numberOfSoundsToMerge; i++) {
AVMutableCompositionTrack *track = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
NSError *error = nil;
BOOL ok = NO;
CMTime startTime = CMTimeMakeWithSeconds(0, 1);
CMTime trackDuration = ((AVURLAsset *)[songAssets objectAtIndex:i]).duration;
CMTimeRange tRange = CMTimeRangeMake(startTime, trackDuration);
//Set Volume
AVMutableAudioMixInputParameters *trackMix = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:track];
[trackMix setVolume:0.8f atTime:startTime];
[audioMixParams addObject:trackMix];
//Insert audio into track
ok = [track insertTimeRange:tRange ofTrack:(AVAssetTrack *)[songTracks objectAtIndex:i] atTime:CMTimeMake(0, 1) error:&error];
[((AVAssetTrack *)[songTracks objectAtIndex:i]).asset cancelLoading];
}
}
Playing Sound Files
- (void) playSounds {
AVMutableComposition *composition = [AVMutableComposition composition];
audioMixParams = [[NSMutableArray alloc] initWithObjects:nil];
[self mergeAudio:composition];
AVMutableAudioMix *audioMix = [AVMutableAudioMix audioMix];
audioMix.inputParameters = [NSArray arrayWithArray:audioMixParams];
AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:composition];
AVPlayer *player = [[AVPlayer alloc] initWithPlayerItem:playerItem];
[playerItem setAudioMix:audioMix];
[player play]; }

AVAssetResourceLoaderDelegate setup issue

I have a weird issue when trying to setup AVAssetResourceLoaderDelegate. When using this code it works most of the time on a device but it doesn't work on the simulator at all:
NSURL *url = [NSURL URLWithString:#"xxx://name"];
AVURLAsset *asset = [AVURLAsset assetWithURL:url];
AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:asset];
[asset.resourceLoader setDelegate:self queue:self.queue];
AVPlayer *avPlayer = [AVPlayer playerWithPlayerItem:playerItem];
[avPlayer play];
The issue is that the delegate method is never getting called:
- (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest
However, this code works perfectly fine:
NSURL *url = [NSURL URLWithString:#"xxx://name"];
AVURLAsset *asset = [AVURLAsset assetWithURL:url];
[asset.resourceLoader setDelegate:self queue:self.queue];
AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:asset];
AVPlayer *avPlayer = [AVPlayer playerWithPlayerItem:playerItem];
[avPlayer play];
The only difference is that we set the delegate first and then we create AVPlayerItem with the asset. In the first case, the asset is not being copied when creating AVPlayerItem and if we check playerItem.asset.resourceLoader.delegate the delegate is set correctly but it's never getting called.
I would appreciate any help with it.
Thanks,
Michal

Playing video from within app?

I am trying to play a local video from my app using AVPlayer. I have looked everywhere but can't find any useful tutorials/demos/documentation. Here is what I am trying, but it is not playing. Any idea why? The URL is valid because I was using the same one to play a video using MPMoviePlayer successfully.
Video *currentVideo = [videoArray objectAtIndex:0];
NSString *filepath = currentVideo.videoURL;
NSURL *fileURL = [NSURL fileURLWithPath:filepath];
AVURLAsset *asset = [AVURLAsset assetWithURL: fileURL];
AVPlayerItem *item = [AVPlayerItem playerItemWithAsset: asset];
self.player = [[AVPlayer alloc] initWithPlayerItem: item];
AVPlayerLayer *layer = [AVPlayerLayer playerLayerWithPlayer:self.player];
self.player.actionAtItemEnd = AVPlayerActionAtItemEndNone;
layer.frame = CGRectMake(0, 0, 1024, 768);
[self.view.layer addSublayer: layer];
[self.player play];
I believe the problem is that you're assuming that after executing this line AVURLAsset *asset = [AVURLAsset assetWithURL: fileURL]; the asset is ready to use. This is not the case as noted in the AV Foundation Programming Guide:
You create an asset from a URL using AVURLAsset. Creating the asset, however,
does not necessarily mean that it’s ready for use. To be used, an asset must
have loaded its tracks.
After creating the asset try loading it's tracks:
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:fileURL options:nil];
NSString *tracksKey = #"tracks";
[asset loadValuesAsynchronouslyForKeys:#[tracksKey] completionHandler:
^{
NSError *error;
AVKeyValueStatus status = [asset statusOfValueForKey:tracksKey error:&error];
if (status == AVKeyValueStatusLoaded) {
// At this point you know the asset is ready
self.playerItem = [AVPlayerItem playerItemWithAsset:asset];
...
}
}];
Please refer to this link to see the complete example.
Hope this helps!

iOS - how to play sound file using avfoundation

I want to play a sound file (which I have dragged into xcode and copy to the project) using av foundation with the following code but fail.
I think NSURL *url = [[NSURL alloc] initWithString:#"sound.caf"]; this is where it goes wrong, but I have no idea other than this way, how I could instantiate an AVAsset with this the sound file (of course, it would be other place that goes wrong). Anyway, can someone offer me some helps? thanks
AVMutableComposition *composition = [AVMutableComposition composition];
CMPersistentTrackID trackID = kCMPersistentTrackID_Invalid;
AVMutableCompositionTrack *compositionTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:trackID];
NSURL *url = [[NSURL alloc] initWithString:#"sound.caf"];
AVAsset *songAsset = [AVURLAsset URLAssetWithURL:url options:nil];
AVAssetTrack *assetTrack = [[songAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
CMTime startTime = CMTimeMakeWithSeconds(0, 1);
CMTime endTime = songAsset.duration;
CMTimeRange tRange = CMTimeRangeMake(startTime, endTime);
NSError *error = nil;
[compositionTrack insertTimeRange:tRange ofTrack:assetTrack atTime:CMTimeMake(0, 44100) error:&error];
AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:composition];
self.player = [[AVPlayer alloc] initWithPlayerItem:playerItem];
[self.player play];
This code should help you:
NSString *soundPath = [[NSBundle mainBundle] pathForResource:#"myfile" ofType:#"wav"];
NSURL *soundURL = [NSURL fileURLWithPath:soundPath];
NSError *error = nil;
AVAudioPlayer *player = [[AVAudioPlayer alloc] initWithContentsOfURL:soundURL error:&error];
if (error) {
NSLog(#"%#",[error localizedDescription]);
}
[player play];
Try this
NSString *shutterplayerPath =
[[NSBundle mainBundle] pathForResource:#"shutter" ofType:#"mp3"];
NSString *tickplayerPath =
[[NSBundle mainBundle] pathForResource:#"tick" ofType:#"wav"];
shutterAudioPlayer =
[[AVAudioPlayer alloc] initWithContentsOfURL:[[NSURL alloc]
initFileURLWithPath: shutterplayerPath] error:NULL];
tickAudioPlayer =
[[AVAudioPlayer alloc] initWithContentsOfURL:[[NSURL alloc];
initFileURLWithPath:tickplayerPath] error:NULL];
[shutterAudioPlayer play];
[tickAudioPlayer play];

AVFoundation - Video didn't play in AVMutablecomposition with AVplayer

I am working on playing a online video file with AVplayer using AVMutablecomposition. The Video plays very fine when I play it on browser but I just cant get it play on my iPad.
Here's the code:
self.composition = [AVMutableComposition composition];
NSURL *urlVideo = [NSURL URLWithString:#"{video location}"];
AVURLAsset *videoAsset = [[AVURLAsset alloc]initWithURL:urlVideo options:nil];
AVMutableCompositionTrack *compositionVideoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
NSError *error = NULL;
AVAssetTrack *videoTrack = [[videoAsset tracksWithMediaType:AVMediaTypeVideo]objectAtIndex:0];
CMTimeRange x = CMTimeRangeMake(kCMTimeZero, [videoAsset duration]);
[compositionVideoTrack insertTimeRange:x ofTrack:videoTrack atTime:kCMTimeZero error:nil];
And the code to set the AVPlayer:
AVPlayerItem *playerItem = [[AVPlayerItem alloc]initWithAsset:composition];
self.player = [[AVPlayer alloc]initWithPlayerItem:playerItem];
AVPlayerLayer *layerVideo = [AVPlayerLayer playerLayerWithPlayer:self.player];
layerVideo.frame = self.view.layer.bounds;
layerVideo.backgroundColor = [UIColor orangeColor].CGColor;
layerVideo.position = CGPointMake(1024/2, 768/2-10);
[self.view.layer addSublayer:layerVideo];
[self.player play];
If I use "AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:videoAsset];" instead of "AVPlayerItem *playerItem = [[AVPlayerItem alloc]initWithAsset:composition];" then I can see the video playing.
Anyone know how to play the video using AVplayer with AVMutablecomposition ?
Your code lacks a couple error checkings but it works as is.
Unless you're in an iPad this line will send the player off-screen:
layerVideo.position = CGPointMake(1024/2, 768/2-10);
If that's the case, try with the right position.

Resources