I have a video of several cv::Mat RGB frames that I need to store as videos with lossless encoding on Jetson Xavier NX. In this process, I need to convert the RGB frames to YUV444 and then use Jetson's inbuilt lossless encoding tools to encode to h264/h265 and then save in an mp4 file format.
I can read cv::Mat frame and then convert it to the YUV444 pixel format (either manually element-wise or using OpenCV's inbuilt converter). Currently, I am writing the YUV444 frames to a binary file per frame using std::ofstream(filename, std::ios::binary). However, when I encode the yuv file to h264 or h265, I get very bad results. Is there something I am missing?
Is there a better method to save a video into a yuv file (such that I can use the encoder on the entire video).
-->> Original image in yuv444 space
-->> Image in the encoded video
I am trying to compress and resize DSLR camera photos. But my observation so far is that webp has noticeable degraded quality when webp file size is about 30% smaller than JPEG.
Command used to generate webp using imagemagick:
convert 1.JPG -strip -quality 80 -resize 800 -define
webp:method=6 1.webp
My goal is to get webp format that is at least 20% samller file size than JPEG while having virtually no difference in quality between JPEG and webp.
Is this achievable or again there is no such thing as free lunch?
Update Since the time of the original posting, I've discovered that using Google's cwebp compressor shows dramatically improved compression over ImageMagik 6.7.8, which is what powered my initial tests. This is especially true for images with transparency when compared to PNGs. Files using the mac version of the webp command (which uses cwebp under the hood) are about 1/4 of the size of the same file compressed with ImageMagik, and do provide a significant performance boost.
According to Google, "WebP typically acheives an average of 30% more compression than JPG" (source) with similar visual quality to a JPG. However, as you suggest, there in never such a thing as a free lunch.
Quality
Quality is largely a subjective measure, but keep in mind that you're comparing a file compressed at quality 80 with a file that doesn't have that level of compression (at least, this is what I understand from your question). Just running the default conversion without specifying a lower quality may give you slightly smaller files without loss of visible quality. 20% smaller might be a bit much of an ask, though, but it may be achievable for certain images.
convert 1.JPG 1.webp # do not specify quality
Size
In practice, it depends a lot on your settings and your source images. For example, I recently ran this command on all jpg images in a folder on a website "in the wild":
convert filename.jpg -quality 80 -strip -define webp:lossless=false -define webp:method=6 filename.webp
The convert command on this particular server is powered by ImageMagick 6.7.8. Some files were dramatically smaller compared to the original JPGs, while others were actually larger. Overall, after running that command, the total file sizes of all JPG images was 49MB, while the total file size of all WebP images was 29MB. That's a pretty good savings, however, when I ran ImageMagick's JPG compression, it was even better:
convert filename.jpg -sampling-factor 4:2:0 -strip -quality 80 -interlace JPEG filename-new.jpg;
The size of all new jpgs in the directory was 21MB. Both are set to compress at quality 80, but the jpg compression appears to be better. This may have to do with some compression already on my set of test images and how that interacts with the WebP conversion process.
As I found, file sizes can even grow, usually if you are converting between lossy and lossless images. On the FAQ page linked above, Google claims: "this is mainly due to the colorspace difference (YUV420 vs ARGB) and the conversion between these."
tl;dr: In the wild, it may or may not improve file size depending on the type of images, if/how the source file was compressed, and what quality you set for the WebP. Visible degradation is harder thing to measure, but try setting a higher quality, or without specifying a quality at all.
Just to add a side-comment to #Pwpwpw's excellent answer, webp is a good PNG replacement, but not great as a JPG replacement.
It beats PNG because it has better lossless compression algorithm than libpng, and it has the great feature of allowing different compression settings for different channels. You can do lossless on the alpha but lossy on the RGB, for example, which is fantastic for overlays.
Against JPG it does less well. It uses the VP8 codec for lossy, which is only somewhat better than JPG.
I would take a look at HEIC. It uses the much more modern h.265 for lossy compression and typically beats JPG by a factor of two at the same quality. It's the format Apple are using by default on iOS now.
libheif have a nice demo here:
https://strukturag.github.io/libheif/
That's libheif, compiled to javascript and running in your browser. You can upload .heic to it and download as jpg. libheif have some basic command-line tools to encode and decode images.
It does sadly have some patent issues, you might need to be a little cautious.
update Looking further out, the current hope is AVIF: it's the same container format as HEIC (called HEIF, confusingly), but swaps the problematic h.265 compressor for AV1. AV1 is roughly equivalent to h.265, but is patent-free.
update for 2022 AVIF seems to be too slow to be practical, so now hope rests on JXL. It's fast, patent free, supports HDR, transparency, etc., and compression is as good as HEIC. Chrome has support, though it's behind a flag for now.
You are re-encoding lossy compressed jpeg; if you have access to the original raw files you should use them as master. Currently you are compressing noise (and other artefacts) introduced by the jpeg encoder so the WebP codec does not have access to the same information that was already lost at the jpeg encoding. Of course it will look worse.
In other words; you should encode the original image data, if possible. I'm aware that some cameras and equipment only outputs HEIC / JPEG and not RAW so you might be out of options - the best thing to do is to keep the JPEGs as JPEGs.
Just did a quick test with a high-res JPG from my DSLR camera converting to a 410x800 background picture for mobile using Photoshop CC 2020 (and the WebPShop plugin).
Lossless:
PNG 100%: 680 KB
JPG 12 (max): 428 KB
WEBP 100%: 537 KB
50% quality:
JPG 6 (50%): 119KB
WEBP 50%: 45KB
At 50% JPG has more detail, but quite comparable quality.
0 quality:
JPG 0 (min): 51KB
WEBP 0%: 6KB
At 0% WEBP is horrible. But imagine the lowest quality JPG, and that's bad image quality and WEBP definitely beats it at 50%, while still being smaller in file size. So for me.. that's free lunch.
I'm trying to capture an image using AVFoundation and save it as JPEG with maximum quality. I'm aware of AVVideoCodecJPEG and AVVideoQualityKey for AVFoundation capture, but it doesn't produce the output I'm looking for. Namely, I need the final JPEG to use the 4:4:4 chroma subsampling. Instead AVFoundation even at quality set to 1.0 (max) produces the image with 4:2:0 chroma subsampling. I tried to capture an image in BGRA32 format and later compress with VideoToolbox - same result. The only way I can get the desired output is by using CGImageDestinationCreateWithData method, but it's too slow - 0.4 sec for 12 MPx image (if I set quality to less than 1.0, the image can be compressed in 0.1 sec for the same size, but than I get again 4:2:0 chroma subsampling).
Did anyone ever succeeded to produce a JPEG with chroma subsampling 4:4:4 using AVFoundation or VideoToolbox? Or maybe there is there any way to accelerate CGImageDestinationCreateWithData method? Thanks in advance
Ok, there seems to be no support for 4:4:4 chroma sub-sampling using hardware accelerated JPG compression.
When I convert png to jpeg , size of image is increasing for 2 percent cases. I am using convert command of imagemagick for this. Also able to handle alpha transparency factor by maintaing white background . But , I am not getting increase in size of image . It is very well known that pngs are lossless and jpegs are lossy , then what is the specific reason and what can be work around for it.
Please refer above mentioned example pic
Try decreasing JPEG quality setting. Jpeg uses Discreet Cosine Transform (DCT) to decrease the size of spatial data, but on high quality setting with high-detailed input images
you will get some size increase due to the fact, that data is not quantized enough and DCT-ed data needs some storage too.
What sort of library would be recommended for lossy png compression while retaining alpha layer.
I use kraken.io a lot but would like to work it more into a workflow.
I know a lot about lossless compression but I'm at a loss with how they do lossy.
If you literally want a library, then libimagequant (I think that's what kraken.io uses).
There's also mediancut posterizer that achieves lossy compression by preprocessing. It's much more effective than posterization in Photoshop or ImageMagick.
Description of lossy PNG methods on ImageAlpha website.
PNG is lossless by design, there's no lossy PNG. All you can do is some lossy preprocessing before saving it as PNG, as converting a true colour image to a palette one, or by reducing the number of colors by posterize - all that will probably reduce the size of the image, but that's not part of PNG.