I am currently in a webcam streaming server project that requires the function of dynamically adjusting the stream's bitrate according to the client's settings (screen sizes, processing power...) or the network bandwidth. The encoder is ffmpeg, since it's free and open sourced, and the codec is MPEG-4 part 2. We use live555 for the server part.
How can I encode MBR MPEG-4 videos using ffmpeg to achieve this?
The multi-bitrate video you are describing is called "Scalable Video Codec". See this wiki link for basic understanding.
Basically, in a scalable video codec, a base layer stream itself has completely decodable; however, additional information is represented in the form of (one or many) enhancement streams. There are couple of techniques to be able to do this including lower/higher resolution, framerate and change in Quantization. The following papers explains in details
of Scalable Video coding for MEPG4 and H.264 respectively. Here is another good paper that explains what you intend to do.
Unfortunately, this is broadly a research topic and till date no open source (ffmpeg and xvid) doesn't support such multi layer encoding. I guess even commercial encoders don't support this as well. This is significantly complex. Probably you can check out if Reference encoder for H.264 supports it.
The alternative (but CPU expensive) way could be transcode in real-time while transmitting the packets. In this case, you should start off with reasonably good quality to start with. If you are using FFMPEG as API, it should not be a problem. Generally multiple resolution could still be a messy but you can keep changing target encoding rate.
Related
I would like to stream a .avi container and not use any codec in the encoding process, that is, I do not want it to encode in H264 or H265, just upload the video and do not encode it, I am using the Azure SDK media services in .NET.
The presets that azure media services has for example in their sdk, they all use h264 or h265 to encode and return an mp4, I just want to upload .avi and see if it is possible that it does not apply any compression and then download the .avi
Thanks!
Adding the answer here. It looks like you were wanting to do a lossless, or near lossless encoding pass using CRF (constant rate factor encoding). There is currently no support for setting CRF encoding in the standard encoder in AMS, but there is work going on to add CRF encoding settings to the SDK in the near future.
For now, you are limited to the settings available in the Transform preset in the H264 or H265 Layers.
You can see all of the available encoding settings most easily in the REST API
https://github.com/Azure/azure-rest-api-specs/blob/main/specification/mediaservices/resource-manager/Microsoft.Media/stable/2021-06-01/Encoding.json
Or if you look at the Transform object in your favorite SDK. Look at the H264Video and H264 Layer classes in the model, as well as the H265 equivalent ones for settings you can control in your code.
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.management.media.models.h264video?view=azure-dotnet
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.management.media.models.h264layer?view=azure-dotnet
UPDATE: SDK for .NET is available now with Exposed RateControlMode for H264 encoding, enabling 2 new ratecontrol modes - CBR (Constant Bit Rate) and CRF (Constant Rate Factor).
See- https://www.nuget.org/packages/Microsoft.Azure.Management.Media
I'm looking mediaconvert service from aws to transcode videos. The value I'm trying to set just now is quality level (QL) for QVBR, according with this it could depends on the platform, for example for 720p/1080p resolution it proposes QL=8/9 (for TV), QL=7 (for tablet), QL=6 (for smartphone).
In fact, the app have a version for the 3 type of devices then I'm asking: I need to keep 3 versions for the same video? I want to save some money in streaming and my app has similar number of users using it in each platform, I want to save in bandwidth but providing good quality videos
Higher QVBR quality levels (QL) correspond to higher bitrates in the output.
For a large display such as a TV, a higher QVBR QL is recommended to help improve the viewer experience. But when viewing the same content on a smaller display such as a phone you may not need all of those extra bits to still have a good experience.
In general, it's recommended to create an output targeted for each of the various devices or resolutions content will be viewed on. This will help save bandwidth for the smaller devices while still delivering high quality for the larger ones.
This concept is referred to as Adaptive Bitrate (ABR) Streaming, and is a common feature of streaming formats such as HLS and DASH (among others). The MediaConvert documentation has a section on how to create ABR outputs as well: https://docs.aws.amazon.com/mediaconvert/latest/ug/video-abr-streaming-outputs.html
I saw that a few video players (e.g. AVPlayerHD) are doing hardware-accelerated playing on iOS for unsupported containers like MKV. How do you think they're achieving that?
I'm thinking reading packet with ffmpeg, decoding with Core Video. Does that make sense? I'm trying to achieve the same.
Thanks!
I think that the HW accelerators for video rendering (decoding) support fixed formats, due to hard wired logic. I don't know of any HW accelerator to be able to transcode from MKV.
Another method of accelerating video playback, would be the usage of OpenCL and make use of the integrated GPU on your device. This method enables HW acceleration of a wider area of applications.
The problem with this approach is that if you are not lucky enough to find a framework that uses OpenCL to do GPU acceleration of transcode / decode, you probably need to do it yourself.
Added info
To implement a fully HW accelerated solution you first need to transcode the MKV into H264 & sub, and from there you can use the HW decoder to render the H264 component.
For the HW accelerated transcode operations you could use GPU (via OpenCL) and/or multithreading.
I found https://handbrake.fr/ that might have some OpenCL transcoding features.
Cheers!
Which video formats can we use in OpenCV? Can anything in addition to AVI and load from camera be used?
If these are the only supported formats, is a video converter required to use other video formats.
I'm not sure how up-to-date it is, but this OpenCV wiki page gives a good overview of what codecs are supported. If looks like AVI is the only format with decent cross-platform support. Your options are either to do the conversion using an external converter (like you suggest) or write code that uses a video library to load the image and create the appropriate cv::Mat or IplImage * header for the data.
Unless you're processing huge quantities of video I suggest taking the path of least resistance and just converting the videos to AVI (see the above link for the details of what OpenCV supports). Just be careful to avoid lossy compression: it will wreck havoc with a lot of image processing algoritms.
OpenCV "farms out" video encoding and decoding to other libraries (e.g., ffmpeg and VFW). Also, have a look at the highgui source directory to see all of the VideoCapture wrappers available (specifically pay attention to the cap_* implementations). AVI is merely a container, and really isn't that critical to what video codecs that OpenCV can read. AVI can contain several different combinations of video, audio, and even subtitle streams. See my other answer about this. Here is also a quick article explaining the differences between containers and codecs.
So, if you're on Linux make sure ffmpeg supports decoding the video codec you are interested in processing. You can check what codecs your version of ffmpeg supports with the following command:
ffmpeg -formats
On Windows, you'll want to make sure you have plenty of codecs available to decode various types of video like the K-Lite Codec Pack.
I am using DSPACK with Delphi 6 Pro.
I am looking for a good sample that shows how to create a filter graph that will convert the sample rate of an audio stream to a desired format (sample rate, bit depth, and number of channels) in real time.
Does anyone know of a good example project that shows how to structure the filter graph with DSPACK to do this? If not with DSPACK, then if you know of a good example or web page that discusses the general DirectX filter graph concepts involved, I can use that.
I also know C/C++ and can follow a C# example well enough.
You need a resampling filter to do this. Options include:
implement a filter which does Audio Resampling
using some resample code/library, see Free Resampling Software
wrapping Media Foundation Audio Resampler DSP, if you are OK with its runtime requirements
use third party filter
Having such filter available, you will need to build a transcoding graph with audio source, resampler and target of your conversion (such as file).
Also as far as I remember, that stock ACM Wrapper Filter is capable of converting PCM audio between standard sample rates.