I create a project in ASP.NET which uses the mpeg-4 video format. It is possible that it will be a commercial project later. This web application will store and play videos in mpeg-4 format in HTML video and Canvas elements.
As far as I know, the mpeg-4 encoder and decoder are not free to use, and I require a license to use them in my project.
I will record the videos mostly by my smartphone in mpeg-4 format, later I will lower their video bit rate by HandBrake, which is an open source free to use universal encoder, which uses the mpeg-4 encoder/decoder.
The mpeg-4 videos with the lowered bit rate will be displayed by my application in HTML 5 Canvas and Video elements, as I mentioned above.
My question is:
Do I have to pay license fee for using the mpeg-4 video format for this kind of usage?
I would rather use the other 2 free video formats (WebM and OGG), but Internet Explorer and Safari does not support them, so I have to use the mpeg-4 format).
After many hours of further searching, I found out, that distributing MP4 video encoded with H264 codec REQUIRES licensing in the case of commercial web applications, if the users have to pay to access the h264 encoded mp4 video content.
On August 26, 2010 MPEG LA announced that H.264 encoded internet video
that is free to end users will never be charged royalties.
(The above quoted information is copied from Wikipedia, the referenced PDF file is no longer available.)
Fortunately, Cisco is launching a project to create a free to use h264 codec for public use in HTML5.
References:
http://shaver.off.net/diary/2010/01/23/html5-video-and-codecs/
https://blog.mozilla.org/blog/2013/10/30/video-interoperability-on-the-web-gets-a-boost-from-ciscos-h-264-codec/
Related
I have a Roku app and some of the videos come from Youtube. I have no problem retrieving the videos but if I select a video with HD it wants to automatically stream the Dash version. I can prevent Dash if I force a non HD version but who wants to watch a SD version..
SO I am wondering is there any way to force the mp4 stream opposed to a Dash stream?
I have read that XP does not play Dash and so I tried using Windows NT 5.1 as the user-agent but that did not work.
Any help would be greatly appreciated.
DASH and MP4 are not mutually exclusive - they perform different functions in the video delivery.
In simple terms you can view it like this:
Camera captures frames - 'raw video'
The 'raw video' is encoded in some way to store it, generally in a way that balances video size vs the quality. The video is then sometimes refereed to by the encoder used (the codec) - for example if a h.264 codec is used the video may be called a h.264 video.
The video stream, i.e. all the individual frames that make up the video, is packaged into a container. This container may contain video and audio streams, and it may even have multiple video streams. The video is then often referred to by the container format - for example if our h.264 encoded video above is packaged into an MP4 container it is often referred to as an MP4 video, even though the MP4 'container' may contain several video and audio tracks.
To improve the quality of video streaming, a video may also use a streaming protocol like MPEG DASH. The theory here is simple: multiple copies of the video are created with different bit rates, and hence different size and quality. Each of these copies is broken up into, for example, 10 second chunks. An index file is created, called a manifest, and a pointer to each video and audio stream is included. A client playing the video, for example a browser, requests each 10 second chunk as it needs it. It chooses which copy of the video it selects the next chunk from depending on the current network conditions. This means if the network is good it can switch to higher quality copy for the next chunk and if there is a problem it can switch down to a lower quality chunk. If we take our example video encoded by h.264 and put into a MP4 container, we can now package it using DASH streaming format. A video packaged like this is often referred to as a DASH video.
The above is a simplified overview, but it hopefully highlights that your videos may be actually MP4 and DASH, and in fact commonly are.
As an additional note, different devices may support different codecs (and even codec profiles), packaging formats and streaming formats - for example iOS devices tend to support HLS rather than DASH at the time of writing. This changes frequently as devices and standards evolve and is one of the reasons it can be tricky to find a single format that will play on all devices and clients - for this reason servers often will provide the same video in multiple codec and streaming formats to support as many devices and clients as possible.
Problem:
To get an iOS app that streams video accepted into the app store, we need to have a HLS version.
What’s the problem?
Android does not support HLS well, and for other reasons, we need to store MP4 and HLS files of the same content.
What’s the difference between MP4 and HLS and why do you need to store both?
MP4 is a container that stores H.264 video and AAC audio for best compatibility in HTML 5 browsers – jsvideo players often have flash fallback if the browser does not support MP4 video in HTML 5 that use the same MP4 file, but played through flash.
HLS is a protocol where text files (.m3u8) contain references to playlists, which themselves reference .ts files (or m2ts), which are mpeg-2 transport streams, not to be confused with mpeg-2 video. The .ts files are containers for the same H.264 video and AAC audio.
Why am I complaining?
It takes time to create the HLS files and playlists from the MP4 files
(Most importantly) We are now storing twice as much data on S3
Why should I care? If your S3 bill is $10K per month for storing both MP4 and HLS, now it is only $5K. Or put another way, if you are paying $100K for storing data in MP4, it would cost $200K to store the same content in both MP4 and HLS.
What do I want?
I want to store only the .ts files and serve both desktop users, iOS users, and Android users with that single file.
Is it possible?
Doesn’t HLS require 5-10 second .ts segments instead of one big file?
As of IETF draft 7, and version 4 of the protocol, HLS supports the tag EXT-X-BYTERANGE which allows you to specify a media segment as a byte range (subrange) of a larger URL.
Are .ts files compatible with HTML5 video?
Apparently not. They are a different container than MP4, yet contain the same video and audio content. Worth looking into how to store the raw video/audio data once and simply using the correct containers when necessary. If JS video players can convert HTML 5 MP4 files into Flash video on the fly if the browser does not support HTML 5 MP4, then why not be able to do the same with M2TS data?
I might be ignorant on some level, but maybe someone can shed some light on this issue, and possibly present a solution.
There currently is no good solution.
A little background.
Video streaming used to require custom protocols such as RTP/RTMP/RTSP etc. These protocols work fine except, we were basically building two separate networks. One HTTP based for standard web traffic, and the other one. The idea came along to split video into little chunks and serve them to the player over HTTP. This way we do not need special servers/software and we could take advantage of the giant HTTP CDNs that were being built. In addition. because the video was split into chunks, we can can encode the same video into different qualities/file sizes. Then the player can choose the best quality video for its current bandwidth. This was the perfect solution for mobile because of the constant changing network conditions. Several competing standard were developed. Move networks was the first to market [citation needed]. The design was copied by Microsoft (Smooth Streaming) and Apple (HTTP Live streaming aka HLS). Microsoft is phasing out smooth streaming in favor of DASH. DASH looks like it will become the default streaming solution of the future. Except, because of its design-by-comity approach, it has basically been stuck in comity for a few years. Now, in those few years, Apple sold Millions of IOS devices. So HLS can not just be discontinued. Why doesn't everyone just use HLS then? I can think of three reasons 1) Its Apples standard, and people are haters. 2) Transport streams are a complicate file format. and 3) Transport streams a patent encumbered. MP4 is not patent encumbered but it also does not have the adaptive abilities. This make user experience poor on 2G networks. The only network supported by the iPhone 1. Also AT&T at the time did not want full bitrate video streamed over there woefully inadequate celular network. HLS was the compromise. All of this predates HTML5. So the video tag was not even considered in its design.
Addressing your points:
1) It takes time to create the HLS files and playlists from the MP4
files
This is a programing website, Automate it.
2) We are now storing twice as much data on S3
[sic] I want to store only the .ts files and serve both desktop users,
iOS users, and Android users with that single file.
You and me both man :).
Possible solutions.
1) What is specifically wrong with Androids implementation? (except for lack of in older devices)
2) JW player can play HLS (Not sure about on android)
3) Server side transmux on demand.
Doesn’t HLS require 5-10 second .ts segments instead of one big file?
You can do byte-ranges, but you need to make sure all devices you are interested in support it.
If JS video players can convert HTML 5 MP4 files into Flash video on
the fly if the browser does not support HTML 5 MP4, then why not be
able to do the same with M2TS data?
They don't convert. Flash natively supports mp4. It is possible to convert TS in AS3/JS. I have done it. JW player can convert TS in browser. video.js may be able to as well.
I like to stream the web cam pictures wich are captured by opencv. I think about a solution with ffmpeg and live555 (poorly they are not document so well). My problems are:
How can convert the captured images to a H264 format so the picures/second match. If it is in a loop I get more than 25 pictures/sekond and the video is to fast.
How can i directly stream the converted H264 stream over the network via rtp / rtps or similar.
Thanks for your help!
This is a common problem.
if you are not require to distribute your software (private use / server side / open-source), you may use FFMpeg compiled with x264 encoder, there's a config flag for that in FFMpeg's config script.
If you do require to distribute your software, i don't know any LGPL licensed library for that, i believe there is no such library. You'd have to use some paid solution.
You should implement DeviceSource.cpp, see DeviceSource.hh and use it as the FramedSource.
Edit: Apple revealed video encoder API, allowing access to stream of h264 frames in iOS8
For an example of how to use x264 and Live555 to encode and stream frames, see the following:
spyPanda open source project.
How to write a Live555 FramedSource to allow me to stream H.264 live SO question.
My software encodes videos using the standard Windows AVIFile functionality. That means, it can use all the DirectShow Codecs that are installed.
I recommend my users to download the XVid Codec themselves, but that's inconvenient.
I cannot bundle XVid since my software is not GPL. Is there any good direct-show-compatible video codec that can be redistributed with a commercial application? Requirement: The videos must look great on YouTube.
If not, I could switch from AViFile to another library (e.g. FFMpeg). But I'd like to avoid that if possible.
Thanks! :-)
Edit: The videos are computer animation with very sharp details (explosions and sparks). The video should be high-bitrate.
Theora is an excellent open source codec with very good DirectShow support. You also might consider google's newer WebM codecs. You dont say what kind of video you are distributing (from a camcorder, animation/computer rendered graphics, etc), so some of the choice of codec may depend on the exact nature of your video.
I've been reading about Silverlight 4's support for web cam and microphone. I'd like to be able to record audio from the browser and upload it to an ASP.NET MVC site. I've seen some mixed information about whether uploads are supported in Silverlight 4. I'm also looking to encode the audio to MP3 before upload and have seen some references to Speex.
Does anyone have any experience with this scenario that they can share as to whether this will be possible with Silverlight 4?
Your expertise is greatly appreciated!
I'm sure that you can record the audio and upload it to the website. but I don't think there is any Silverlight library that you can use to convert your audio file to mp3. You can always convert it at server-side after uploading as well..
If your SL is trusted OOB application then you can try this http://alvasnet.blogspot.com/2010/02/how-to-save-audio-to-mp3-on-silverlight.html
You can certainly upload any SL4 microphone output to an ASP.NET MVC (or WCF WebAPI) service. I do this in my WP7 project.
As far as encoding, the raw PCM that comes out of the mic is typically huge (for a 16khz sample rate, 16bits/sample, mono channel audio stream, that would net out to 32000 bytes per second of audio).
I have not found an easy (or free) way to encode in MP3 on the SL platform, but fortunately, speech (as opposed to music) is better encoded using the Speex codec. The implementation I use is at nspeex.codeplex.com and has given me excellent compression ratios - e.g. at Quality level 1 (lowest out of 10, which is adequate for my purposes), 64k of audio (the equivalent of 2sec) will encode down to about 1500 bytes.