I'm building a video player that should handle both streaming and non-streaming content and I want it to be playable with AirPlay.
I'm currently using multiple AVPlayer instances (one for each clip), and it works okay, but the problem is it doesn't give a very smooth experience when using AirPlay. The interface jumps back and forth between each clip when switching AVPlayer, so I would like to migrate to using a single AVPlayer. This seems like a trivial task, but I haven't yet found a way to do this.
This is what I've tried so far:
Using a single AVPlayer with multiple AVPlayerItems and switching between those using replaceCurrentItemWithPlayerItem. This works fine when switching between streaming->streaming clips or non-streaming->non-streaming, but AVPlayer doesn't seem to accept replacements between streaming->non-streaming or vice versa. Basically, nothing happens when I try to switch.
Using an AVQueuePlayer with multiple AVPlayerItems fails for the same reason as above.
Using a single AVPlayer with a single AVPlayerItem based on an AVMutableComposition asset. This doesn't work because streaming content is not allowed in an AVMutableComposition (and AVURLAssets created from a streaming url doesn't have any AVAssetTracks and they are required).
So is there anything I am missing? Any other suggestion on how to accomplish this?
I asked this question to Apple's Technical Support and got the answer that it's currently not possible to avoid the short jump back to menu interface, and that no version of AVPlayer supports mixing of streaming and non-streaming content.
Full response:
This is in response to your question about how to avoid the short jump back to the main interface when switching AVPlayers or AVPlayerItems for different media items while playing over AirPlay.
The issue here is the same with AVPlayer and AVQueuePlayer: no instance of AVPlayer (regardless of which particular class) can currently play both streaming and non-streaming content; that is, you can't mix HTTP Live Streaming media (e.g. .m3u8) with non-streaming media (a file-based resource such as an .mp4 file).
And with regard to AVMutableComposition, it does not allow streaming content.
Currently, there is no support for "seamless" video playback across multiple items. I encourage you to file an enhancement request for this feature using the Apple Bug Reporter (http://developer.apple.com/bugreporter/).
AVComposition is probably the best option at present for "seamless" playback. However, it has the limitation just described where streaming content is not allowed.
Related
I have a lot of long (45 mins - 90 mins) MP4 videos in a public S3 bucket and I want to play them in my iOS app using AVPlayer.
I am using AVPlayerViewController to play them but I need to wait several minutes before they start playing as it downloads the whole video rather than streaming it.
I am caching it locally so this is only happening the first time but I would love to stream the video so the user doesn't have to wait for the entire video to download.
Some people are pointing out that I need Cloudfront to stream videos but in the documentation, I've read that this is only necessary when you have many people streaming the same file. I'm building a MVP so I only need a simple solution.
Is there any way to stream an MP4 video from an S3 bucket with AVPlayerViewController without it fully downloading the file before playing it to the user?
TLDR
AVPlayer does not support 'streaming' (HTTP range requests) as you would define it, so either use an alternative video player that does or use a real media streaming protocol like HLS which is supported by AVPlayer & would start the video before downloading it all.
CloudFront is great for delivery in general but is not truly needed - you may have seen it mentioned due to CloudFront RTMP distributions but they now have been discontinued.
Detailed Answer
S3 supports a concept called byte-range fetches using HTTP range requests - you can verify this by doing a HEAD request to your video file & seeing that the Accept-Ranges header exists with a value set to bytes (or not 'none').
Load your MP4 file in the browser & notice that it can start as soon as you click play. You're also able to move to the end of the video file and yet, you haven't really downloaded the entire video file. HTTP range requests are what allow this mechanism to work. Small chunks of the video can be downloaded as & when the user gets to that part of the video. This saves the file server & the user bandwidth while providing a much better user experience than the client downloading the entire file.
The server would need to support byte-range fetches in the first instance before the client can then decide to make range requests (or not to). The key is that, once the server supports it, it is up to the HTTP client to decide whether it wants to fetch the data in chunks or all in one go.
This isn't really 'streaming' as you know it & are referring to in your question but it is more 'downloading the video from the server in chunks and playing it back' using HTTP 206 Partial Content responses.
You can see this in the Network tab of your browser as a series of multiple 206 responses when seeking in the video. The entire video is not downloaded but the video is streamed from whichever position that you skip to.
The problem with AVPlayer
Unfortunately, AVPlayer does not support 'streaming' using HTTP range requests & HTTP 206 Partial Content responses. I've verified this manually by creating a demo iOS app in Xcode.
This has nothing to do with S3 - if you stored these files on any other cloud provider or file server, you'd see that the file is still fully loaded before playing.
The possible solutions
Now that the problem is clear, there are 2 solutions.
Using an alternative video player
The easiest solution is to use an alternative video player which does support byte-range fetches. I'm not an expert in iOS development so I sadly can't help in recommending an alternative but I'm sure there'll be a popular library that the industry prefers over the in-built AVPlayer. This would provide you with your (extremely common) definition of 'streaming'.
Using a video streaming protocol
However, if you must use AVPlayer, the solution is to implement true media streaming with a video streaming protocol - true streaming also allows you to leverage features like adaptive bitrate switching, live audio switching, licensing etc.
There are quite a few of these protocols available like DASH (Dynamic Adaptive Streaming over HTTP), SRT (Secure Reliable Transport) & last but not least, HLS (HTTP Live Streaming).
Today, the most widely used streaming protocol on the internet is HLS, created by Apple themselves (hey, maybe the reason to not support range requests is to force you to use the protocol). Apple's own documentation is really wonderful for delving deeper if you are interested.
Without getting too much into protocol detail, HLS will allow playback to start more quickly in general, fast-forwarding can be much quicker & delivers video as it is being watched for the true streaming experience.
To go ahead with HLS:
Use AWS Elemental MediaConvert to convert your MP4 file to HLS format - the resulting output will be 1 (or more) .M3U8 manifest files in addition to .ts media segment file(s)
Upload the resulting output to S3
Point AVPlayer to the .M3U8 file
let asset = AVURLAsset(url: "https://ermiya.s3.eu-west-1.amazonaws.com/videos/video1playlist.m3u8")
let item = AVPlayerItem(asset: asset)
...
Enjoy near-instant loading of the video
CloudFront
In regards to Amazon CloudFront, it isn't required per se & S3 is sufficient in this case but a quick Google search will mention loads of benefits that it provides, especially caching which can help you save on S3 costs later on.
Conclusion
I would go with converting to HLS if you can, as it will yield more possibilities down the line & is a better true streaming experience in general, but using an alternative video player will work just as well due to iOS AVPlayer restrictions.
Whether to use CloudFront or not, will depend on your user base, usage of S3 and other factors.
As you're creating an MVP, I would recommend just doing a batch conversion of your MP4 files to HLS format & not using CloudFront which would add additional complexity to your cloud configuration.
Like #ErmiyaEskandary said, you could just use HLS to solve your problem, which is probably a good idea, but you should not have to wait for the entire MP4 file to download before playing it with AVPlayer. The issue is actually not with AVPlayer or byte-range requests at all, but rather with how your MP4 files are formatted.
You could have your MP4 file configured incorrectly for streaming. MP4's have a metadata section called the MOOV atom. By default, many encoders put this at the back of the file. In this case, the player would have to download the entire file before it could begin playing.
For streaming usecases, this would need to be put at the front of the file. The player then will only need to buffer the MOOV atom, and it can begin playing the video as the data is loaded.
You can use ffmpeg with the fast start flag enabled to move the MOOV atom to the beginning of the file.
I am using AVPlayer to run a HLS video. The video has no sound. Also i have a audio track url of same format m3u8. Can i somehow change the AVPlayer item asset or something while running my video without sound to add my other audio track so that they are sort of played together.
Disappointingly, you can't create an AVComposition using non local video and audio tracks and play that.
But HLS is at its heart a sort of textual playlist consisting of media pieces that can be played either serially or concurrently. If you inspect both your video and audio m3u8 streams, you should be able to concoct a new single m3u8 stream that includes both video and audio.
HOWEVER, disappointingly, it seems you can't play this resulting stream as a local file (why!?!), so you'd set up an http server to serve it to you, either locally or from or afar, or maybe (!?) you could avoid all that with a clever use of AVAssetResourceLoaderDelegate.
It also seems synchronising two AVPlayers is unsupported too, although perhaps that situation has improved.
Is there a way to do rewind with an HLS implementation?
Here is a link of what HLS is: https://developer.apple.com/streaming/
Wikipedia says yes: "Later versions of the protocol also provide for trick mode fast-forward and rewind and integration of subtitle" http://en.wikipedia.org/wiki/HTTP_Live_Streaming
So how do I implement a rewind in HLS?
You can use AVPlayer to play an HTTP Live Stream video. It will allow you to seek in reverse just like you would seek forward. So in that sense, you can rewind. However, due to the compressed nature of the stream, rewind is quite slow, because it needs to rewind all the way back to the closest keyframe and then interpolate forward to your seek point.
If you're looking for a smooth rewind, it really isn't possible with any stream that is compressed (for the reasons stated above). But you can get decent "rewind" performance if your movie file lives on the device.
I would suggested creating an AVPlayerItem from your HLS and then play that item in an AVPlayer. Experiment a bit to see what the results are and go from there.
I’m working on a small iPhone app which is streaming movie content over a network connection using regular sockets. The video is in H.264 format. I’m however having difficulties with playing/decoding the data. I’ve been considering using FFMPEG, but the license makes it unsuitable for the project. I’ve been looking into Apple’s AVFoundation framework (AVPlayer in particular), which seems to be able to handle h264 content, however I’m only able to find methods to initiate the movie using an url – not by proving a memory buffer streamed from the network.
I’ve been doing some tests to make this happen anyway, using the following approaches:
Play the movie using a regular AVPlayer. Every time data is received on the network, it’s written to a file using fopen with append-mode. The AVPlayer’s asset is then reloaded/recreated with the updated data. There seems to be two issues with this approach: firstly, the screen goes black for a short moment while the first asset is unloaded and the new loaded. Secondly, I do not know exactly where the playing stopped, so I’m unsure how I would find out the right place to start playing the new asset from.
The second approach is to write the data to the file as in the first approach, but with the difference that the data is loaded into a second asset. A AVQueuedPlayer is then used where the second asset is inserted/queued in the player and then called when the buffering has been done. The first asset can then be unloaded without a black screen. However, using this approach it’s even more troublesome (than the first approach) to find out where to start playing the new asset.
Has anyone done something like this and made it work? Is there a proper way of doing this using AVFoundation?
The official method to do this is the HTTP Live Streaming format which supports multiple quality levels (among other things) and automatically switches between them (eg: if the user moves from WiFi to cellular).
You can find the docs here: Apple Http Streaming Docs
Long story short, I am trying to implement a naive solution for streaming video from the iOS camera/microphone to a server.
I am using AVCaptureSession with audio and video AVCaptureOutputs, and then using AVAssetWriter/AVAssetWriterInput to capture video and audio in the captureOutput:didOutputSampleBuffer:fromConnection method and write the resulting video to a file.
To make this a stream, I am using an NSTimer to break the video files into 1 second chunks (by hot-swapping in a different AVAssetWriter that has a different outputURL) and upload these to a server over HTTP.
This is working, but the issue I'm running into is this: the beginning of the .mp4 files appear to always be missing audio in the first frame, so when the video files are concatenated on the server (running ffmpeg) there is a noticeable audio skip at the intersections of these files. The video is just fine - no skipping.
I tried many ways of making sure there were no CMSampleBuffers dropped and checked their timestamps to make sure they were going to the right AVAssetWriter, but to no avail.
Checking the AVCam example with AVCaptureMovieFileOutput and AVCaptureLocation example with AVAssetWriter and it appears the files they generate do the same thing.
Maybe there is something fundamental I am misunderstanding here about the nature of audio/video files, as I'm new to video/audio capture - but thought I'd check before I tried to workaround this by learning to use ffmpeg as some seem to do to fragment the stream (if you have any tips on this, too, let me know!). Thanks in advance!
I had the same problem and solved it by recording audio with a different API, Audio Queue. This seems to solve it, just need to take care of timing in order to avoid sound delay.