I'm using OpenCV to handle videos in mp4 format. The image below is a random frame extracted from a video, and you can see the obvious distortion on the sweater.
How can we detect such artifacts? Or can we avoid such artifacts by extracting nearby keyframes and how?
As #VC.One suggested, these distortions are due to video interlacing. Here is a good article about interlacing/deinterlacing: What is Deinterlacing? Facts, solutions, examples.
There are several tools to handle deinterlacing:
[Windows] The one suggested in 100fps.com: Virtualdub + DivX codec + AviSynth
[Windows] MediaCoder suggested by #VC.One.
[Windows/Linux] FFmpeg provides serveral deinterlacing filters, e.g. yadif, kerndeint etc. Here is an example: ffmpeg -i input.mp4 -vf yadif output.mp4
Related
I would like to use the redirection operator to bring the stream from ffmpeg to cv2 so that I can recognize or mark the faces on the stream and redirect this stream again so that it runs under another stream.
One withoutfacedetect and One withfacedetect.
raspivid -w 1920 -h 1080 -fps 30 -o - -t 0 -vf -hf -b 6000000 | ffmpeg -f h264 -i - -vcodec copy -g 50 -strict experimental -f tee -map 0:v "[f=flv]rtmp://xx.xx.xx.xx/live/withoutfacedetect |[f=h264]pipe:1" > test.mp4
I then read up on CV2 and came across the article.
https://www.bogotobogo.com/python/OpenCV_Python/python_opencv3_Image_Object_Detection_Face_Detection_Haar_Cascade_Classifiers.php
I then ran the script with my picture and was very amazed that there was a square around my face.
But now back to business. What is the best way to do this?
thanks to #Mark Setchell, forgot to mention that I'm using a Raspberry Pi 4.
I'm still not 100% certain what you are really trying to do, and have more thoughts than I can express in a comment. I have not tried all of what I think you are trying to do, and I may be over-thinking it, but if I put down my thought-train, maybe others will add in some helpful thoughts/corrections...
Ok, the video stream comes from the camera into the Raspberry Pi initially as RGB or YUV. It seems silly to use ffmpeg to encode that to h264, to pass it to OpenCV on its stdin when AFAIK, OpenCV cannot easily decode it back into BGR or anything it naturally likes to do face detection with.
So, I think I would alter the parameters to raspivid so that it generates RGB data-frames, and remove all the h264 bitrate stuff i.e.
raspivid -rf rgb -w 1920 -h 1080 -fps 30 -o - | ffmpeg ...
Now we have RGB coming into ffmpeg, so you need to use tee and map similar to what you have already and send RGB to OpenCV on its stdin and h264-encode the second stream to rtmp as you already have.
Then in OpenCV, you just need to do a read() from stdin of 1920x1080x3 bytes to get each frame. The frame will be in RGB, but you can use:
cv2.cvtColor(cv2.COLOR_RGB2BGR)
to re-order the channels to BGR as OpenCV requires.
When you read the data from stdin you need to do:
frame = sys.stdin.buffer.read(1920*1080*3)
rather than:
frame = sys.stdin.read(1920*1080*3)
which mangles binary data such as images.
There is a camera that shoots at 20 frame per second. each frame is 4000x3000 pixel.
The frames are sent to a software that contain openCV in it. OpenCV resizes the freames to 1920x1080 then they must be sent to FFMPEG to be encoded to H264 or H265 using Nvidia Nvenc.
The encoded video then got steamed HTTP to a maximum of 10 devices.
The infrastructure is crazy good (10 GB Lan) with state of the art switchers, routers etc...
Right now, i can get 90 FPS when encoding the images from an Nvme SSD. this means that the required encoding speed is achieved.
The question is how to get the images from OpenCV to FFMPEG ?
the stream will be watched on a webapp that was made using MERN stack (assuming that this is relevant).
For cv::Mat you have cv::VideoWriter. If you wish to use FFMpeg, assuming Mat is continuous, which can be enforced:
if (! mat.isContinuous())
{
mat = mat.clone();
}
you can simply feed mat.data into sws_scale
sws_scale(videoSampler, mat.data, stride, 0, mat.rows, videoFrame->data, videoFrame->linesize);
or directly into AVFrame
For cv::cuda::GpuMat, VideoWriter implementation is not available, but you can use NVIDIA Video Codec SDK and similarly feed cv::cuda::GpuMat::data into NvEncoderCuda, just make sure your GpuMat has 4 channels (BGRA):
NV_ENC_BUFFER_FORMAT eFormat = NV_ENC_BUFFER_FORMAT_ABGR;
std::unique_ptr<NvEncoderCuda> pEnc(new NvEncoderCuda(cuContext, nWidth, nHeight, eFormat));
...
cv::cuda::cvtColor(srcIn, srcIn, cv::ColorConversionCodes::COLOR_BG2BGRA);
NvEncoderCuda::CopyToDeviceFrame(cuContext, srcIn.data, 0, (CUdeviceptr)encoderInputFrame->inputPtr,
(int)encoderInputFrame->pitch,
pEnc->GetEncodeWidth(),
pEnc->GetEncodeHeight(),
CU_MEMORYTYPE_HOST,
encoderInputFrame->bufferFormat,
encoderInputFrame->chromaOffsets,
encoderInputFrame->numChromaPlanes);
Here's my complete sample of using GpuMat with NVIDIA Video Codec SDK
I am doing segmentation via deep learning in pytorch. My dataset is a .raw/.mhd format ultrasound images.
I want to input my dataset into the system via data loader.
I faced few important questions:
Does changing the format of the dataset to either .png or .jpg make the segmentation inaccurate?(I think I lost some information in this way!)
Which format is less data lossy?
How should I make a dumpy array if I don't convert the original image format, i.e., .raw/.mhd?
How should I load this dataset?
Knowing nothing about raw and mhd formats, I can give partial answers.
Firstly, jpg is lossy and png is not. So, you're surely losing information in jpg. png is lossless for "normal" images - 1, 3 or 4 channel, with 8 bit precision in each (perhaps also 16 bits are also supported, don't quote me on that). I know nothing about ultrasound images, but if they use higher precision than that, even png will be lossy.
Secondly, I don't know what mhd is and what raw means in the context of ultrasound images. That being said, a simple google search reveals some package for reading the former to numpy.
Finally, to load the dataset, you can use the ImageFolder class from torchvision. You need to write a custom function which loads an image given its path (for instance using the package mentioned above) and pass it to the loader keyword argument.
I hope someone can help me.
I started researching different compression methods to compress Bitmap-Images lossless and lossy. The first methods i used were JPEG, JPEG-2000 and JPEG-XR. Now i want to compare these "standard" ones with H.264 and H.265, maybe they perform as well as they do for video compression.
I tried using ffmpeg, but i can't find out which parameters i need, there are plenty... So maybe someone can help me or link me to an Article/Howto or something else?!
Thanks a lot!
EDIT:
I used the following command:
ffmpeg -i 01.bmp -c:v libx264 -preset veryslow -crf 40 test.avi
but this created an 7kb file from an 76,8 kb input file... not very good compression ratio... is there any possibility to achieve more?
"-crf 40" will choose bitrate around QP = 40, that is somehow low visual quality.
For H.264, QP = 0 ~ 51, where 0 is the best.
So you can consider use "-crf = 16", or even smaller number.
I believe the quality will be much better.
What is possible image format of Y800 which is available in OpenCV? is it always referred to GRAY? Any other options?
Thanks in advance.
Eight years later, I stumbled upon this question. I want to add an answer with regard to OpenCV 4.2.0.
For videos, this version features an ffmpeg backend which natively understands the FOURCC identifier "Y800". Confusingly, it does not take one-channel grayscale (CV_8UC1) frames, but the usual OpenCV three-channel BGR (CV_8UC3):
cv::VideoWriter vw("y8.avi", cv::VideoWriter::fourcc('Y', '8', '0', '0'), 60, frame.size());
vw.write(frame); // note: frame must be 8UC3!
OpenCV supports the grayscale mode in number of image file formats, including, but not limited to PGM, PNG and JPEG.
cv::imwrite("gray.pgm", image);