How can I speed up video encoding (VideoWriter) and decoding (VideoCapture) in opencv? Can I use TBB (threading building blocks) library?
I am working with 1280x720 (720p) video and developing a real-time system but my video frame reading and writing alone are consuming 35 ms per frame. Please help.
Related
This video clip:
https://www.youtube.com/watch?v=wc0PB6Azwn0
What is the max audio bitrate and how to detect real audio bitrate? is it the same? Please no rumors and no guessing.
Does it depend on the video quality I am watching (1080p, 720p, etc)?
If you say - yes, it makes no sense, because the clip was uploaded at one bitrate only.
Hope, someone who knows the subject can answer that questions.
Is it possible to detect the audio bitrate from YouTube video at all?
Stats for nerds is useless, it shows no audio bitrate.
Taking consideration lossy (vs lossless) audio max is 320 Kbps, it cannot be higher than that by definition.
Anyone who knows?
You can use the command line tool youtube-dl to list all available transcodings for a given YouTube video by running youtube-dl -F <url-to-your-video>.
Example output using the URL mentioned in your question:
Studying the output we can see that the audio transcoding with the highest bitrate is "format 251" using the opus codec at an average bitrate of around 145k. Note that YouTube is not using a fixed bitrate but rather a variable bitrate with target of ~160k.
The opus codec is currently supported in most modern browsers (but not Safari). Browsers without support for opus will fallback to the m4a stream at a variable bitrate targeting ~128k.
If you want to make 100% sure which audio transcoding you're currently listening to, you can right click the YouTube video player and select "Stats for nerds" and look for the number mentioned in the codecs section and cross-reference that with the output given by youtube-dl:
Does it depend on the video quality I am watching (1080p, 720p, etc)? If you say - yes, it makes no sense, because the clip was uploaded at one bitrate only.
Yes, it depends on video quality. When you're choosing a video quality, you're not just choosing the video quality... you're choosing the audio quality as well. YouTube isn't giving you the option, but it's part of the package.
Videos aren't served as-is, they're transcoded. You upload your video and it's re-compressed at a variety of different bitrates with different settings.
Your audio bitrate depends on what YouTube decided to encode it as. Each video may have many versions of the stream.
The best thing you can do is get a build of FFmpeg with libquvi enabled, and let it parse the page, find the streams, download the stream, demux, and figure things out for you from there.
Taking consideration lossy (vs lossless) audio max is 320 Kbps, it cannot be higher than that by definition.
Your definition is wrong. There are all kinds of lossy audio codecs, and they can be ran at a variety of bitrates.
I am working on my final project in Software Engineering B.Sc. Our project includes tracking a ball in Foosball game. Actually with the size of the Foosball table I will need at least HD 1080p format (1920x1080 pixels) camera and because of the high speed I will also need 60 fps.
I will use OpenCV opensource to write code in C/C++ and detect a ball on each received frame.
So here is my issue: I need to get steam from the HD camera with 60fps, Wide-angled.
I can't use a web-cam because it will not give me HD format with 60fps
(webcams can't do this, even expensive Logitech or Microsoft while it is written on the package - actually they mean that it can be low resolution with 60 fps OR HD with 30 fps) Also it is not wide-angled.
On the other hand I would like to use a web camera because it is easy to get stream out of it.
The preferred solution is to use extreme camera (something like Go Pro but cheaper version - I have AEE S70 - about 120$) I can use HDMI output of this camera to stream data to PC. But I can't use USB, it will be recognized as a Mass Storage Device. It has micro HDMI output but I have no HDMI Input on my PC.
The question is if it is possible to find some cheap capture device (HDMI->USB3.0/PCI Express) which can stream frames as HD 1080p and 60fps from this extreme camera to PC via HDMI? What device should I use? Maybe you suggest me another camera/or better solution?
Thanks
I've been looking into this for a sport application (Kinovea). It is virtually impossible to find 1080p # 60fps due to the limits of USB 2.0 bandwidth. Actually even for lower bandwidth the camera needs to perform compression on-board.
The closest camera I found is the ELP-USBFHD01M, it's from a Chinese manufacturer and can do 720p # 60fps on the MJPEG stream. I've written a full review in the following blog post.
The nice thing about this camera for computer vision is that it has a removable M12 lens, so you can use a wide angle if you want. They sell various versions of the board with pre-mounted lenses of 140°, 180°, etc.
MJPEG format means that you'll have to decompress on the fly if you want to process each image though.
Other solutions we have explored were USB 3.0 cameras but as you mention they aren't cheap and for me the fact that they don't do on-board compression was a drawback for fast recording to disk.
Another option I haven't had time to fully investigate is the HD capture cards for gamers like AVerMedia. These cards supposedly capture HD at high speed and can stream it to central memory.
Do you really need real-time processing? If you could perform the tracking on video files that you have recorded by other means you could use even 120fps files from the GoPro and get even better results.
Your choice of 1080p with 60 fps is good for the tracking application is good and as you said most of the web cams don't support such high resolution / frame rate combinations. Instead of going for a HDMI->USB3.0/PCI Express converter for your AEE S70 (which will increase the camera latency, cost and time for you to find a solution), you can check See3CAM_CU30 which streams 1080P60 uncompressed data over USB 3.0 off the shelf. Also it costs similar to your AEE S70.
I am working on "upload video & audio to server",I want to know which format is best for upload (consider the quality & file-size)
video formats are just containers, if you want to consider quality and file size you should look into the encoding of video. For ios based devices h264 encoder with high efficiency level 4 provides the good compression, hence you will get good quality in less file size.
If you want to learn about conversion of video data from one format to another please look into ffmpeg.
I saw that a few video players (e.g. AVPlayerHD) are doing hardware-accelerated playing on iOS for unsupported containers like MKV. How do you think they're achieving that?
I'm thinking reading packet with ffmpeg, decoding with Core Video. Does that make sense? I'm trying to achieve the same.
Thanks!
I think that the HW accelerators for video rendering (decoding) support fixed formats, due to hard wired logic. I don't know of any HW accelerator to be able to transcode from MKV.
Another method of accelerating video playback, would be the usage of OpenCL and make use of the integrated GPU on your device. This method enables HW acceleration of a wider area of applications.
The problem with this approach is that if you are not lucky enough to find a framework that uses OpenCL to do GPU acceleration of transcode / decode, you probably need to do it yourself.
Added info
To implement a fully HW accelerated solution you first need to transcode the MKV into H264 & sub, and from there you can use the HW decoder to render the H264 component.
For the HW accelerated transcode operations you could use GPU (via OpenCL) and/or multithreading.
I found https://handbrake.fr/ that might have some OpenCL transcoding features.
Cheers!
I was planning to decode H.264 based RTSP stream using FFMPEG in OpenCV but, when I tried so it gave some errors. Later, I found that many people have faced issues while decoding H.264 stream using ffmpeg (libavcodec). Typically the below mentioned error messages pop-up while using libavcodec:
"[h264 # 0xa766dd0]concealing 1200 DC, 1200 AC, 1200 MV errors"
Has anyone used any other library successfully for decoding H.264 based RTSP. If so, which is the library (I have heard of live555 which is used within vlc player for decoding such streams). I would also like to know the output format and how it can be made compatible with OpenCV (typically within opencv we can use cvQueryFrame to directly extract a frame from a video stream, but in case we are using a library other than ffmpeg how to go about it).
Thanks in advance.
Regards,
Saurabh Gandhi
VLC is using ffmpeg to decode h.264.
the problem can happen when you have the wrong SPS PPS, or don't have.
You need to extract it from the RTSP protocol and pass it to the ffmpeg before trying to decode video.
To Decode your RTSP stream , The best libraries are FFMPEG and Gstreamer.
To decode the stream you need to feed the decoder with the right buffer for which you have to understand your H.264 stream so that you can arrange your SPS, PPS and NAL data before feeding it to the Library Decoder