Are there any image formats for the web with full HDR image support? 10/12-bit channels, DCI-P3/Rec.2020 colour space, etc.
It seems like none of the conventional formats support it, and no one is talking about it, even when YouTube accepts HDR uploads and HDR monitor adoption is increasing.
I am by no means an expert on this topic, but I found this question while working on a 2021/22 solution to the problem and I'd like to share my thoughts and progress. Maybe somebody gets use out of it.
Trigger HDR mode in the browser
It seems, it's possible to trick browsers on Apple platforms to switch to HDR mode, as documented on kidi.ng/wanna-see-a-whiter-white
There, they use a combination of a tiny HDR video and the CSS properties filter/backdrop-filter with brightness(10) to make HTML elements and their colors reach into HDR space. It works and it is a cool trick, if a bit gimmicky.
AVIF HDR support with PQ
As mentioned by Валерий Заподовников, the AVIF file format seems to support HDR in a sense when the image is tagged PQ (Perceptual quantizer).
I found files provided by Netflix (example) demonstrating this on the AVIF codec Github. They do seem to display brighter than regular CSS content in Chrome (see image) with background-color: white;, but I have not been able to create images like these myself. Also: the PNG images didn't yield the same result for me.
Platform limitations
The experiments did not produce any usable results for me, mainly because I have few HDR capable displays to test on and also, Safari does not support AVIF images yet. It seems, it could be a while before it does, but I'll get back to testing then.
My other hope was that the HDR-capable format that Apple does use, .HEIF/.HEIC, would display in Safari and I could work with that, but it doesn't. And it does not look like it will, since it's not a format engineered for web use.
Y. Mano and colleagues at Netflix investigated just this question. They concluded that several commonly supported image formats (notably JPEG2000 and 16-bit PNG) can support HDR images already, as long as a color profile is embedded in the corresponding images. The article I linked to is also a good introduction to HDR and wide color gamut images in general.
Related
I have images that I wanna show it on the web. The extension of the image is .JPG. If you open the file in notepad it starts with LTRI. But that page is not working on browser (even not working on Photos/Paint app of windows 10). It only works on VB6 LeadTools. How can I convert this image? File metadata is below,
You'll need to use the LeadTools libraries. You'll need to register to their site.
I haven't seen anyone else supporting that format, that is proprietary (as written by #Selvin).
Then the code should be easy:
using (RasterCodecs codecs = new RasterCodecs())
{
var image = codecs.Load("yourimage.ext");
// 0 should means "whatever bit format is appropriate"
codecs.Save(image, "out.jpg", Leadtools.RasterImageFormat.Jpeg, 0);
}
The answer posted by xanatos is correct. This appears to be a 1-bit LEAD CMP image file.
We would like to add the following details:
If the images are indeed 1-bit CMP, they would be pure black-and-white, and they would be lossless (unlike color CMP and common JPEG, which are usually lossy formats).
Naming the file aa.JPG is incorrect, since the recommended extension for LEAD images is “.CMP”. However, when using our SDK, the correct format is detected regardless of the file name or its extension.
Converting 1-bit images to JPEG is not recommended, since JPEG files are more suited for full-color images. It’s better to save them as PNG, TIFF or other lossless formats. (The LEADTOOLS SDK supports many such formats).
Using lossless compression shouldn’t produce large files since 1-bit images contain a lot less data than full-color images.
When using the free evaluation of our SDK, technical support is free through chat and email, so feel free to contact us with any questions you have.
I'm wondering if it is possible to interactively edit a photo in GIMP and then export a script to programmatically transform a series of images in the same way?
Chris
No. I have seen plenty of attempts to write something to do this, but all failed, because there is nothing in Gimp to really support this.
If all you do is global changes (color balance, contrast, exposure, sharpening...) then the demosaicing apps typically export "recipes" that can be re-applied to other pictures (sometimes even in batch). Despite their names demosaicing apps do not work only on sensor data, the usually also accept JPEG files as input.
Most (all?) FOSS demosaicing apps are discussed (and supported by their authors) on pixls.us (which has a list of apps here.
I'm looking for a way to inspect the the ICC color profile data provided CGColorSpace's copyICCData() method.
Specifically, I'm loading PNG images into UIImages on iOS, and trying to find a way to use the let iccData:CFData? = aUIImage.cgImage!.colorSpace!.copyICCData() to determine the gamma for the image file. This is for a game that uses 3D rendering— if the source image has a standard 2.2 gamma, I'll load the image data into a texture as sRGB (e.g. MTLPixelFormatRGBA8Unorm_sRGB) and if it has a gamma of 1.0 I'll instead load it as a linear texture (e.g. MTLPixelFormatRGBA8Unorm).
Note: The solution of just passing a UIImage/CGImage to the rendering system (SceneKit/Metal) and letting it sort it out won't work here because: 1. Some of the rendering I'm doing is assembling 2D images into a 3D texture, so that's something I need to do with raw data, not something I can just read from a standard image file format; 2. I'm specifically trying to pass gamma-1.0 images into the rendering system to avoid the overhead of sRGB→linear conversion (rendering is in linear space).
Also: Manual ICC-parsing solutions, Apple-API-using solutions, and open-source library suggestions are all acceptable answers. This is not specifically a query for tool recommendation — any solution that'll work is a good one — but in my research, manual ICC parsing would be unwieldy and Apple's APIs don't seem to expose any ICC properties. So I believe the most likely answer is a pointer to some library out there that I haven't been able to find via Google or GitHub or CocoaPods or StockOverflow, and will be gladly accepted.
Your best bet is to use sample icc 'https://sourceforge.net/projects/sampleicc/'. Just get the profile data as you described, then use OpenIccProfile to load it up. From there, get a ref to the header structure (.m_Header) and pull the info you need. I'd also recommend you take a look at RefIccMax 'https://github.com/InternationalColorConsortium/RefIccMAX' which is a newer version of the same lib, but not ready for primetime.
Is it possible to process(filter) HDR images through Core Image? I couldn't find much documentation on this, so I was wondering if someone possibly had an answer to it. I do know that it is possible to do the working space computations with RGBAh when you initialize a CIContext, so I figured that if we could do computations with floating point image formats, that it should be possible..
What, if it is not possible, are alternatives if you want to produce HDR effects on iOS?
EDIT: I thought I'd try to be a bit more concise. It is to my understanding that HDR images can be clamped and saved as .jpg, .png, and other image formats by clamping the pixel values. However, I'm more interested in doing tone mapping through Core Image on a HDR image that has not been converted yet. The issue is encoding a CIImage with a HDR image, supposedly with the .hdr extention.
EDIT2: Maybe it would be useful to useful to use CGImageCreate , along with CGDataProviderCreateWithFilename ?
I hope you have basic understanding of how HDR works. An HDR file is generated by capturing 2 or more images at different exposures and combining it. So even if there's something like .HDR file, it would be a container format with more than one jpg in it. Technically you can not give two image files at once as an input to a generic CIFilter.
And in iOS, as I remember, it's not possible to access original set of photos of an HDR but the processed final output. Even if you could, you'd have to manually do the HDR process and generate a single HDR png/jpg anyway before feeding it to a CIFilter.
Since there are people who ask for a CI HDR Algorithm, I decided to share my code on github. See:
https://github.com/schulz0r/CoreImage-HDR
It is the Robertson HDR algorithm, so you cannot use RAW images. Please see the unit tests if you want to know how to get the camera response and obtain the hdr image. CoreImage saturates pixel values outside [0.0 ... 1.0], so the HDR is scaled into said interval.
Coding with metal always causes messy code for me, so I decided to use MetalKitPlus which you have to include in your project. You can find it here:
https://github.com/LRH539/MetalKitPlus
I think you have to check out the dev/v2.0.0 branch. I will merge this into master in the future.
edit: Just clone the master branch of MetalKitPlus. Also, I added a more detailed description to my CI-HDR project.
You can now(iOS 10+) capture Raw images(coded on 12 bits) and then filter them the way you like using CIFilter. You might not get a dynamic range as wide as the one you get by using bracketed captures; nevertheless, it is still wider than capturing 8-bits images.
Check Apple's documentation for capturing and processing RAW images.
I also recommend you watch wwdc2016 video by Apple(move to the raw processing part).
I have a JPEG image stored in memory as a blob and am looking to apply some basic transformations to it (e.g. resize, convert to greyscale, rotate etc.)
I am currently using Google Scripts which doesn't have a native image library as far as I can tell.
Are there standard algorithms or similar which would allow me to work with the raw binary array, knowing it represents a JPEG image, to achieve such a transformation?
Not the answer you are looking for I guess, but...
To be able to do image processing using JPEG files as input, you need to decode the images. Well, actually, 90/180/270 degree rotation, flipping and cropping is possible as lossless operations, and thus without decoding the image data. But for anything more advanced, like resizing, you need to work with a decoded image.
Both the file structure (JIF/JFIF) and algorithms used to compress the image data in standard JPEG format are well defined and properly documented. But at the same time, the specification is quite complex. It's certainly doable if you have the time and know what you are doing. And if you are lucky, and your JPEG blobs are all written the same way, you might get away with implementing only some of the spec. But even then, you will need to (re-)implement large parts of the spec, and it might just not be worth it.
Using a 3rd party service to convert it for you, or create your own using a known library, like libjpeg or Java's ImageIO, etc. might be your best bets, if you need a quick solution, and don't have too strict requirements for performance.
There are no straightfoward image processing capabilities available in Apps Script. You'll have either expose your Python as a web service and call it from Apps Script or use the Drive REST API to access the files from your Python app or use any api webservices.
GAE Python has Image processing capabilities check the below url:
https://developers.google.com/appengine/docs/python/images/
Available image transformations
The Images service can resize, rotate, flip, and crop images, and enhance photographs. It can also composite multiple images into a single image.