How to compute the naturalness of image with python?
Naturalness is the degree of correspondence between images and human perception of reality. Huang et al. [15] proposed an approach to obtain a quantitative description of naturalness by grouping the pixels with 20L80 and S>0.1 in HSL color space according to their hue (H coordinate) value into three sets: Skin, Grass and Sky. The naturalness score (https://maths-people.anu.edu.au/~johnm/courses/mathdm/talks/dimitri-clickadvert.pdf)
Related
I am implementing a classifier that is capable of recognizing vehicle color, and I am using the 3D color histograms of the region of interest as a feature vector, computed using openCV's method, calcHist(). Specifically, to calculate the histograms, I use hist = cv.calcHist([hsv_image], [0, 1, 2], None, (8,8,8), [0, 180, 0, 256, 0, 256]). With these parameters, I got, by doing the flatten() of the histogram, a feature vector of 8x8x8 = 512, and with these feature vectors the classifier works pretty well, but I'm looking to further improve the accuracy of my model. So, what I would like to know is if there is any correlation between the number of bins and the range of color channel values, so that I can choose the best number of bins possible.
I'd suggest if you want to improve accuracy, perhaps try a perceptually accurate colorspace — and HSV isn't one. As color is not "real", and only a perception, it would follow that using a perceptually accurate appearance model is a best practice for your application.
Perceptual Appearance Models
CIECAM02, CIECAM16, Jzazbz are pretty much state of the art, and there is ZCAM for HDR imagery, and also image appearance models such as iCAM. These might not be available in a library for OpenCV, but most aren't that difficult to implement.
CIELAB
A simpler model is CIELAB which is part of OpenCV, and is a better choice than HSV or HSL, particularly if you are goal is to judge or select colors in a manner similar to human perception.
L*a*b* breaks the colors down based on human perception and the opponent process of vision. The channels are perceptual lightness, L* as a value from 0 to 100, and a* and b* which encode red/green and blue/yellow respectively, and are each nominally -128 to 127 if using signed 8bit integers.
LAB is a 3D Cartesian space. To determine the color difference, it is simply the euclidian distance between two colors, in other words, the square root of the sum of the squared differences, so:
∆ = ((L*1 - L*2)2 + (a1 - a2)2 + (b1 - b2)2 )0.5
Polar Version
CIELAB is also available with polar coordinates, LCh, for Lightness, Chroma, and hue.
Saturation is not available with LAB, only Chroma. The other color appearance models I mentioned above do have a saturation correlate, as well as brightness in addition to lightness. (Brightness being a different perception than lightness/darkness).
I want to cluster images based on colour similarity. For that I need a good similarity metric between two 3D histograms. A 3D histogram of an image is just a 3 dimensional space where each axis represents one of the base colours. The range of each axis is 0-255 since this are the possible values of the base colours for each pixel.
The histogram is represented as a 256X256X256 matrix and each entry in the matrix represents the count of pixels with that specific colour in the image. For example:
If the value of the matrix element M[0][0][0] = 1150 it means that there are 1150 black pixels in the image (RGB(0,0,0) represents the colour black)
I am looking for the most sensible similarity metric for this kind of problem. The metric will be used in the clustering algorithm (DBSCAN probably) to evaluate image similarity.
Use the L*a*b* (CIELAB) color space, where euclidean distance is indeed similarity, as it is designed to model human eye perceptions non-linearities.
I am not sure if the "Symmetric mean absolute surface distance" (SMAD) is used as a metric to evaluate the segmentation algorithm for 2D images or 3D images? Thanks!
What are surface Distance based metrics?
Hausdorff distance
Hausdorff distance 95% percentile
Mean (Average) surface distance
Median surface distance
Std surface distance
Note: These metrics are symmetric, which means the distance from A to B is the same as the distance from B to A.
For each contour voxel of the segmented volume (A), the Euclidean distance from the closest contour voxel of the reference volume (B) is computed and stored as list1. This computation is also performed for the contour voxels of the reference volume (B), stored as list2. list1 and list2 are merged to get list3.
Hausdorff distance is the maximum value of list3.
Hausdorff distance 95% percentile is the 95% percentile of list3.
Mean (Average) surface distance is the mean value of list3.
Median surface distance is the median value of list3.
Std surface distance is the standard deviation of list3.
References:
Heimann T, Ginneken B, Styner MA, et al. Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets. IEEE Transactions on Medical Imaging. 2009;28(8):1251–1265.
Yeghiazaryan, Varduhi, and Irina D. Voiculescu. "Family of boundary overlap metrics for the evaluation of medical image segmentation." Journal of Medical Imaging 5.1 (2018): 015006.
Ruskó, László, György Bekes, and Márta Fidrich. "Automatic segmentation of the liver from multi-and single-phase contrast-enhanced CT images." Medical Image Analysis 13.6 (2009): 871-882.
How to calculate the surface distance based metrics?
seg-metrics is a simple package to compute different metrics for Medical image segmentation(images with suffix .mhd, .mha, .nii, .nii.gz or .nrrd), and write them to csv file.
I am the author of this package.
I found two papers which address the definitions of SMAD:
https://www.cs.ox.ac.uk/files/7732/CS-RR-15-08.pdf (see section 7.1 on average symmetric surface distance, but this is the same thing)
http://ai2-s2-pdfs.s3.amazonaws.com/2b8b/079f5eeb31a555f1a48db3b93cd8c73b2b0c.pdf (page 1439, see equation 7)
I tried to post the equation here but I do not have enough reputation. Basically look at equation 15 in the first reference listed above.
It appears that this is defined for 3D voxels, though I don't see why it should not work in 2D.
I want to segment an image but someone told me that the Euclidean distance for RGB is not as good as HSV -- but for HSV, as not all H, S, V are of the same range so I need to normalize it. Is it a good idea to normalize HSV and then do clustering? If so, how should I normalize on HSV scale?
Thanks
As HSV components are signify Hue, Saturation and gray intensity of a pixel they are not correlated to each other in terms of color, each component have its own role in defining the property of that pixel, like Hue will give you information regarding color (wavelength in other terms) Saturation always shows how much percentage of white is mixed with that color and Value is nothing but magnitude of that color(in other term Intensity), that is why all components of HSV space not follow same scale for representation of the values while hue can goes negative(because these are cyclic values) on the scale as well but intensity (V) will never goes negative, so normalization will not help in clustering much, the Better idea is you should apply clustering only on Hue if you want to do color clustering.
Now why Euclidean is not good for multi-channel clustering is because its distribution along mean is spherical(for 2D circular) so if it can not make any difference between (147,175,208) and (208,175,147) both will have same distance from the center, its better to use Mahalanobis Distance for distance calculation because it uses Co-variance matrix of the components which makes this distance distribution Parabolic along the mean.
so if you want to do color segmentation in RGB color space use mahalanobis distance(but it will computationally extensive so it will slows down clustering process) and if you want to do clustering in HSV color space use Hue for the segmentation of colors and than use V for fine tuning of segmentation output.
Hope it will help. Thank You
Hue is cyclic.
Do not use the mean (and thus, k-means) on such data.
Firstly you need to know why HSV is more preffered than RGB in image segmentation. HSV separates color information (Chroma) and image intensity or brightness level (Luma) which is very useful if you want to do image segmentation. For example if you try to use RGB approach for a photo with sea as the background there is a big chance the dominant RGB component in the sea is not blue (usually because of shadow or illumination). But if you are using HSV, value is separated and you can construct a histogram or thresholding rules using only saturation and hue.
There is a really good paper that compared both RGB and HSV approach and I think it will be a good read for you -> http://www.cse.msu.edu/~pramanik/research/papers/2002Papers/icip.hsv.pdf
I have just started studying the Viola-Jones face detection algorithm to design a face recognition system. off all the things i understood, I have confusion regarding the phrase:"sum of pixels". Does it means sum of colors at given pixels or the sum of distance of the given pixels?
Generally if you see something like that they're talking about the value of a pixel (its intensity). According to OpenCV, the value of a pixel is calculated as 0.299 R + 0.587 G + 0.114 B. This is how OpenCV converts to grayscale, btw. So when they talk about the sum of pixels, they're likely talking about the sum of the pixel values in a given region (i.e. for a 3x3 region, take the value of each pixel and sum it up).
The sum of pixels is the cumulative sum of pixels along the 2 dimensions of the image
Please check cumsum function in Matlab.
For example:
I = cumsum(cumsum(double(image)),2)
Check out this link for some good info on the Viola Jones Face Detection technique
Good Luck!