Stream video while downloading iOS - ios

I am using iOS 7 and I have a .mp4 video that I need to download in my app. The video is large (~ 1 GB) which is why it is not included as part of the app. I want the user to be able to start watching the video as soon as is starts downloading. I also want the video to be able to be cached on the iOS device so the user doesn't need to download it again later. Both the normal methods of playing videos (progressive download and live streaming) don't seem to let you cache the video, so I have made my own web service that chunks up my video file and streams the bytes down to the client. I start the streaming HTTP call using NSURLConnection:
self.request = [[NSMutableURLRequest alloc] initWithURL:self.url];
[self.request setTimeoutInterval:10]; // Expect data at least every 10 seconds
[self.request setHTTPMethod:#"GET"];
self.connection = [[NSURLConnection alloc] initWithRequest:self.request delegate:self startImmediately:YES];
When I receive a data chunk, I append it to the end of the local copy of the file:
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
NSFileHandle *handle = [NSFileHandle fileHandleForWritingAtPath:[self videoFilePath]];
[handle truncateFileAtOffset:[handle seekToEndOfFile]];
[handle writeData:data];
}
If I let the device run, the file is downloaded successfully and I can play it using MPMoviePlayerViewController:
NSURL *url=[NSURL fileURLWithPath:self.videoFilePath];
MPMoviePlayerViewController *controller = [[MPMoviePlayerViewController alloc] initWithContentURL:url];
controller.moviePlayer.scalingMode = MPMovieScalingModeAspectFit;
[self presentMoviePlayerViewControllerAnimated:controller];
However, if I start the player before the file is completely downloaded, the video starts playing just fine. It even has the correct video length displayed at the top scrubber bar. But when the user gets to the position in the video that I had completed downloading before the video started, the video just hangs. If I close and reopen the MPMoviePlayerViewController, then the video plays until it gets to whatever location I was then at when I launched the MPMoviePlayerViewController again. If I wait until the entire video is downloaded, then the video plays without a problem.
I am not getting any events fired, or error messages printed to the console when this happens (MPMoviePlayerPlaybackStateDidChangeNotification and MPMoviePlayerPlaybackDidFinishNotification are never sent after the video starts). It seems like there is something else that is telling the controller what the length of the video is other than what the scrubber is using...
Does anyone know what could be causing this issue? I am not bound to using MPMoviePlayerViewController, so if a different video playback method would work in this situation I am all for it.
Related Unresolved Questions:
AVPlayer and Progressive Video Downloads with AVURLAssets
Progressive Video Download on iOS
How to play an in downloading progress video file in IOS
UPDATE 1
I have found that the video stall is indeed because of the file size when the video starts playing. I can get around this issue by creating a zero-ed out file before I start the download and over overwrite it as I go. Since I have control over the video streaming server, I added a custom header so I know the size of the file being streamed (default file size header for a streaming file is -1). I am creating the file in my didReceiveResponse method as follows:
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
// Retrieve the size of the file being streamed.
NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)response;
NSDictionary *headers = httpResponse.allHeaderFields;
NSNumberFormatter * formatter = [[NSNumberFormatter alloc] init];
[formatter setNumberStyle:NSNumberFormatterDecimalStyle];
self.streamingFileSize = [formatter numberFromString:[headers objectForKey:#"StreamingFileSize"]];
// Check if we need to initialize the download file
if (![[NSFileManager defaultManager] fileExistsAtPath:self.path])
{
// Create the file being downloaded
[[NSData data] writeToFile:self.path atomically:YES];
// Allocate the size of the file we are going to download.
const char *cString = [self.path cStringUsingEncoding:NSASCIIStringEncoding];
int success = truncate(cString, self.streamingFileSize.longLongValue);
if (success != 0)
{
/* TODO: handle errors here. Probably not enough space... See 'man truncate' */
}
}
}
This works great, except that truncate causes the app to hang for about 10 seconds while it creates the ~1GB file on disk (on the simulator it is instant, only a real device has this problem). This is where I am stuck now - does anyone know of a way to allocate a file more efficiently, or a different way to get the video player to recognize the size of the file without needing to actually allocate it? I know some filesystems support "file size" and "size on disk" as two different properties... not sure if iOS has something like that?

I figured out how to do this, and it is much simpler than my original idea.
First, since my video is in .mp4, the MPMoviePlayerViewController or AVPlayer class can play it directly from a web server - I don't need to implement anything special and they can still seek to any point in the video. This must be part of how the .mp4 encoding works with the movie players. So, I just have the raw file available on the server - no special headers required.
Next, when the user decides to play the video I immediately start playing the video from the server URL:
NSURL *url=[NSURL fileURLWithPath:serverVidelFileURLString];
controller = [[MPMoviePlayerViewController alloc] initWithContentURL:url];
controller.moviePlayer.scalingMode = MPMovieScalingModeAspectFit;
[self presentMoviePlayerViewControllerAnimated:controller];
This makes it so the user can watch the video and seek to any location they want. Then, I start downloading the file manually using NSURLConnection like I had been doing above, except now I am not streaming the file, I just download it directly. This way I don't need the custom header since the file size is included in the HTTP response.
When my background download completes, I switch the playing item from the server URL to the local file. This is important for network performance because the movie players only download a few seconds ahead of what the user is watching. Being able to switch to the local file as soon as possible is key to avoid downloading too much duplicate data:
NSTimeInterval currentPlaybackTime = videoController.moviePlayer.currentPlaybackTime;
[controller.moviePlayer setContentURL:url];
[controller.moviePlayer setCurrentPlaybackTime:currentPlaybackTime];
[controller.moviePlayer play];
This method does have the user downloading two video files at the same time initially, but initial testing on the network speeds my users will be using shows it only increases the download time by a few seconds. Works for me!

You gotta create an internal webserver that acts like a proxy! Then set your player to play the movie from the localhost.
When using HTTP protocol to play a video with MPMoviePlayerViewController, the first thing the player does is to ask for the byte-range 0-1 (first 2 bytes) just to obtain the file length. Then, the player asks for "chunks" of the video using the "byte-range" HTTP command (the purpose is to save some battery).
What you have to do is to implement this internal server that delivers the video to the player, but your "proxy" must consider the length of your video as the full length of the file, even if the actual file hasn't been completely downloaded from the internet.
Then you you set your player to play a movie from " http:// localhost : someport "
I've done this before... it works perfectly!
Good luck!

I can only assume that the MPMoviePlayerViewController caches the file length of the file when you started it.
The way to fix (just) this issue is to first determine how large the file is. Then create a file of that length. Keeping an offset pointer, as the file downloads, you can overwrite the "null" values in the file with the real data.
So you get to a specific point in the download, start the MPMoviePlayerViewController, and let it run. I'd also suggest you use the "F_NOCACHE" flag (with fcntl()) so you bypass the file block cache (which means you will lower your memory footprint).
The downside to this architecture is that if you get stalled, and the movie player gets ahead of you, well, the user is going to have a pretty bad experience. Not sure if there is any way for you to monitor and take preemptive action.
EDIT: its quite possible that the video is not read sequentially, but certain information requires the player to essentially look ahead for something. If so, then this is doomed to fail. The only other possible solution is to use some software tool to sequentially order the file (I'm no video expert so cannot comment from experience on any of the above).
To test this out, you can construct a "damaged" video of varying lengths, and test that to see what works and what does not. For instance, suppose you have a 100Meg file. Write a little utility program, and over write the last 50Megs of data with zeros. Now play this video. Its should fail 1/2 through. If it fails right away, well, you now know that its seeking in the file.
If non sequential, its possible that its looking at the last 1000 bytes or so, in which case if you don't overwrite that things work as you want. If you get lucky and this is the case, you would eventually download the last 1000 bytes, then then start from the front of the file.
It really gets down to finding some way before introducing real networking into the picture, to play a partial file. You will surely find it easier to artificially introduce the networking conditions without really doing it real time.

Related

Getting byte Data from File

WHAT IM DOING I am trying to get an audio file (could be up to an hour long. eg. a Podcast) that I've recorded with AVAudioRecorder to be uploaded to our backend. In addition to being uploaded to the server it needs to be able to be "Paused" and "Resumed" if the user chooses. Because of this, I believe, I need to use dataWithBytesNoCopy:buffer on the NSData class to achieve this.
WHERE IM AT I know for a fact I can get the data with using the passed self.mediaURL property:
if (self.mediaURL) {
NSData *audioData = [NSData dataWithContentsOfURL:self.mediaURL];
if (audioData) {
[payloadDic setObject:audioData forKey:#"audioData"];
}
}
However, this will not give me the desired functionality. I am trying to keep track of the bytes uploaded so that I can resume if the user pauses.
QUESTION How can I use the provided self.mediaURL so that I can retrieve the file and be able to calculate the byte length like this example?
Byte *buffer = (Byte*)malloc((long)audioFile.size);
NSUInteger buffered =[rep getBytes:buffer fromOffset:0.0 length:(long)rep.size error:nil];
NSMutableData *body = [[NSMutableData alloc] init];
body = [NSMutableData dataWithBytesNoCopy:buffer length:buffered freeWhenDone:YES];
Instead of making things more complicated for yourself by trying to reinvent the wheel, use what the system gives you. NSURLSession lets you do a background upload. You hand the task to the session (created using the background session configuration) and just walk away. The upload takes place in pieces, when it can. No "pause" or "resume" needed; the system takes care of everything. Your app doesn't even have to be running. If authentication is needed, your app will be woken up in the background as required. This architecture is just made for the situation you describe.
If the problem is that you want random access to file data without having to read the whole thing into a massive NSData, use NSFileHandle.

Playing audio files continually using audio recorded url

I am newcomer in Objective-C and have experience only 12 months in iPhone development.
I am recording audio files in One UIViewController, and playing on another UIViewController. For playing purpose i am saving the date string for generation of url,it is fine working properly,
But ,now my problem is i want to play previous audio record file for some time after that i want to play next audio file using url . i am saving all the data using nsuser dafaults please help me
NSM![enter image description here][1]utableArray *dateString;
NSURL recordFile = [NSURL fileURLWithPath:[NSTemporaryDirectory() stringByAppendingString:[self.dateString objectAtIndex:sender.tag]]];
For playing
player = [[AVAudioPlayer alloc] initWithContentsOfURL:recordFile error:&error];
from fig when i click a tag 2 i want to play first 5 sec tag1 after that i want to play tag2
Record the audio -> Save the audio in the Temp -> Track the saved path of the audio file (You can store this saved path in an array) . Repeat the same steps for the next file.
Use AVQueuePlayerfor playing items one after the other.

Dropbox sync api large video file upload

I am using Dropbox sync api for downloading text file and upload video file from/to dropbox via my ios application.
I am struggling while uploading heavy video file.While i am uploading video file of duration 15 to 20 minutes its uploaded correctly, but if the duration is more than 25 minutes
then it gets memory waring and app crashes.
I am using this code on upload button action
DBPath *paths=[[DBPath root] childPath:[self.allVideoArray objectAtIndex:Selectedvideo]];
DBFile *createfile=[filesystem createFile:paths error:nil];
NSData *data=[[NSData alloc]initWithContentsOfFile:self.path];
[createfile writeData:data error:nil];
[data relese];
Please some body way me out from this problem. Any help should be appreciable, Thanks in advance.
The problem is that you create an NSData instance containing the entire file. If the file is too big to fit into memory your app will crash. There are better ways to write large files to a DBFile.
Since you have a path to the local file you could do:
DBPath *paths=[[DBPath root] childPath:[self.allVideoArray objectAtIndex:Selectedvideo]];
DBFile *createfile=[filesystem createFile:paths error:nil];
[createFile writeContentsOfFile:self.path shouldSteal:NO error:nil];
Another option would be to read the file at self.path in smaller chunks and use DBFile appendData:error:.
Side note - you really need to check return values to make sure these calls are working or not and make use of the error parameter to log the cause of the problem (if any).

SKAction playSoundFileNamed failed at 500 mp3

In my application I need to use a lot of short different mp3s (about 500 items one by one)
So I use SKAction playSoundFileNamed
After ~200 sounds it crashed with 'Failed to load resource - Resource s234.mp3 can not be loaded'. Memory rises to 70mb.
How to avoid this?
What I tried:
recreate sound in every iteration
SKAction *mySound=[SKAction playSoundFileNamed:aa waitForCompletion:YES];
create the one variable in the beggining of .m
SKAction *mySound;
and reuse it in iterations
mySound=[SKAction playSoundFileNamed:aa waitForCompletion:YES];
2.
load all sounds to array once at start
for (int j=0;j<500;j++){
NSString *aa=[NSString stringWithFormat:#"s%d.mp3", j];
[item.sounds addObject:[SKAction playSoundFileNamed:aa waitForCompletion:YES]];
}
...but never changed - it crashes and can't load mp3.
How to clean this memory leaks?
EDITED
I also tried to turn off ARC and manually dealloc it every time. Nothing changed.
This little ditty right here will allow you to use a regular old SKAction but customize the playback in code.
https://github.com/pepelkod/iOS-Examples/tree/master/PlaySoundWithVolume
+(SKAction*)playSoundFileNamed:(NSString*)fileName atVolume:(CGFloat)volume waitForCompletion:(BOOL)wait{
// setup audio
NSString* nameOnly = [fileName stringByDeletingPathExtension];
NSString* extension = [fileName pathExtension];
NSURL *soundPath = [[NSURL alloc] initFileURLWithPath:[[NSBundle mainBundle] pathForResource:nameOnly ofType:extension]];
AVAudioPlayer *player = [[AVAudioPlayer alloc]initWithContentsOfURL:soundPath error:NULL];
[player setVolume:volume];
[player prepareToPlay];
SKAction* playAction = [SKAction runBlock:^{
[player play];
}];
if(wait == YES){
SKAction* waitAction = [SKAction waitForDuration:player.duration];
SKAction* groupActions = [SKAction group:#[playAction, waitAction]];
return groupActions;
}
return playAction;
}
You may be better off with a "proper" sound engine. You could use AVAudioPlayer or even better, ObjectAL (already included in Kobold Kit). That way you have more control over preloading and caching the sound files, and treating streaming audio (MP3) differently from short-lived sound effects (usually CAF or WAV).
For very short sound files, say shorter than 5 seconds, MP3 isn't ideal. You should try CAF/WAV instead.
And do consider how much your sound files use in memory. Say each mp3 file is buffered into a 250 KB buffer, times 500, then that uses over 120 MB of memory. Do not look at the size of the mp3 file because it's a compressed format, and will likely be buffered uncompressed.
I think you're running out of file descriptors. What i've found is that every time playSoundFileNamed is run the sound file is opened and never closed... so the file descriptor is not released.
My testing in instruments leads me to believe there's about 250 available file descriptors. And my research tells me that not only file access uses them up but other things too. So I think your ~200 sound files sounds just about right to crash. Whenever a sound file is the next file that Xcode is trying to access I get nothing from the debugger, but when the next file is a png (I have way more art then sounds) it gives,
error = 24 (Too many open files)
I found the error while play testing and switching back to my menu scene from the game play scene every time I lost. So normally it didn't matter but running the init method for the game play scene over and over was piling up these open sound files.
I've searched and searched for a way to close these files but have come up with nothing. I'm thinking of implementing a singleton to run my sound and moving all of the playSoundFileNamed calls into it so they only ever get called once. I think this is a bug from Apple. These files should be closing with ARC. Has anyone found anything similar?

difference between how AVAssetReader and AudioFileReadPackets reads Audio

consider these two scenarios for reading/writing data from Audio files (for the purpose of sending over a network):
Scenario 1: Audio File Services:
Using AudioFileReadPackets from Audio File Services. This generates audio packets that you can easily send over the network. On the receiving side you use AudioFileStreamOpen and AudioFileStreamParseBytes to parse the data.
AudioFileStreamParseBytes then has two callback functions: AudioFileStream_PropertyListenerProc and AudioFileStream_PacketsProc. These guys are called when a new property is discovered in the stream and when packets are received from the stream, respectively. Once you receive the packets, you can feed it to an audio queue using Audio Queue Service which plays the file just fine.
Note: This method does NOT work with music files stored in the iPod library, which brings us to the 2nd scenario:
Scenario 2: AVAssetReader:
With AVAssetReader you can read from the iPod music library and send packets over the network. Typically you would load the packets directly on an Audio Queue similar to above. However, in this scenario you will have to create a thread to ensure that you block receiving packets when the queue is full, and unblock when queue buffers are available (see this example).
Question:
Is it possible to use AVAssetReader to send packets over, only to have it read by AudioFileStreamParseBytes? (the motive would be that the AudioFileStreamParseBytes's callbacks will handle the threading/blocking business and save you that pain). I tried doing it like so:
1. first read the audio file using AVAssetReader
//NSURL *assetURL = [NSURL URLWithString:#"ipod-library://item/item.m4a?id=1053020204400037178"];
AVURLAsset *songAsset = [AVURLAsset URLAssetWithURL:assetURL options:nil];
NSError * error = nil;
AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:songAsset error:&error];
AVAssetTrack* track = [songAsset.tracks objectAtIndex:0];
// Note: I don't supply an audio format description here, rather I pass on nil to keep the original
// file format. In another piece of code (see here: http://stackoverflow.com/questions/12264799/why-is-audio-coming-up-garbled-when-using-avassetreader-with-audio-queue?answertab=active#tab-top) I can extract the audio format from the track, let's say it's an AAC format.
AVAssetReaderTrackOutput* readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track
outputSettings:nil];
[reader addOutput:readerOutput];
[reader startReading];
2. set up the streamer
// notice how i manually add the audio file type (for the file hint parameter)
// using the info from step one.. If i leave it as 0, this call fails and returns
// the typ? error, which is :The specified file type is not supported.
streamer->err = AudioFileStreamOpen((__bridge void*)streamer,
ASPropertyListenerProc, ASPacketsProc,
kAudioFileAAC_ADTSType, &(streamer->audioFileStream));
3. once I receive the data, I parse the bytes:
streamer->err = AudioFileStreamParseBytes(streamer->audioFileStream, inDataByteSize, inData, 0);
problem: When I do it this way.. I send the bytes and the AudioFileStreamParseBytes does not fail. However, the callbacks *AudioFileStream_PropertyListenerProc* and *AudioFileStream_PacketsProc* are never called. Which makes me think that the parser has failed to parse the bytes and extract any useful information out of them.. in the documentation for AudioStreamParseBytes it states:* You should provide at least more than a single packet’s worth of audio file data, but it is better to provide a few packets to a few seconds data at a time.* I'm sending over 900 bytes, which is just below GKSession's data limit. I'm pretty sure 900 bytes is enough (when testing this under scenario 1, the total bytes was 417 each time and it worked fine).
Any ideas?
The short answer is that it simply doesn't make sense to have packets of audio data be parsed by AudioFileStreamParseBytes.. in the docs AudioFileStreamParseBytes is a function dependent on the existence of an audio file (thus the parameter inAudioFileStream.. which is defined as the ID of the parser to which you wish to pass data. The parser ID is returned by the AudioFileStreamOpen function.)
so lesson learned: don't try to pigeon hole iOS functions to fit your situation.. it should be the other way around.
What I ended up doing was feeding the data directly to an Audio Queue.. without going through all these unnecessary intermediary functions.. a more in depth way would be feeding the data to audio units.. but my application didn't need that level of control

Resources