I've read about the power law (Gamma) Transformations so let's look to the equation: s = c*r^γ
Suppose that I have one pixel which has intensity of 37. If the gamma is 0.4 and c is 1, then the output intensity is 37^(0.4) which is 4.2. Thus it's darker, not brighter. But then why does it look brighter in the example in my textbook?
The gamma transformation applies to data in the range [0,1]. So, for your typical unsigned 8-bit integer image, you would have to scale it first to that range. The equation, including the scaling, then would be:
s = 255 * (r/255)^γ
Now you'd have, for r = 37 and γ = 0.4: s = 255 * (37/255)^0.4 = 117.8. This is brighter.
I have read the BT.709 spec a number of times and the thing that is just not clear is should an encoded H.264 bitstream actually apply any gamma curve to the encoded data? Note the specific mention of a gamma like formula in the BT.709 spec. Apple provided examples of OpenGL or Metal shaders that read YUV data from CoreVideo provided buffers do not do any sort of gamma adjustment. YUV values are being read and processed as though they are simple linear values. I also examined the source code of ffmpeg and found no gamma adjustments being applied after the BT.709 scaling step. I then created a test video with just two linear grayscale colors 5 and 26 corresponding to 2% and 10% levels. When converted to H.264 with both ffmpeg and iMovie, the output BT.709 values are (YCbCr) (20 128 128) and (38 128 128) and these values exactly match the output of the BT.709 conversion matrix without any gamma adjustment.
A great piece of background on this topic can be found at Quicktime Gamma Bug. It seems that some historical issues with Quicktime and Adobe encoders were improperly doing different gamma adjustments and the results made video streams look awful on different players. This is really confusing because if you compare to sRGB, it clearly indicates how to apply a gamma encoding and then decode it to convert between sRGB and linear. Why does BT.709 go into so much detail about the same sort of gamma adjustment curve if no gamma adjustment is applied after the matrix step when creating a h.264 data stream? Are all the color steps in a h.264 stream meant to be coded as straight linear (gamma 1.0) values?
In case specific example input would make things more clear, I am attaching 3 color bar images, the exact values of different colors can be displayed in an image editor with these image files.
This first image is in the sRGB colorspace and is tagged as sRGB.
This second image has been converted to the linear RGB colorspace and is tagged with a linear RGB profile.
This third image has been converted to REC.709 profile levels with Rec709-elle-V4-rec709.icc from elles_icc_profiles
. This seems to be what one would need to do to simulate "camera" gamma as described in BT.709.
Note how the sRGB value in the lower right corner (0x555555) becomes linear RGB (0x171717) and the BT.709 gamma encoded value becomes (0x464646). What is unclear is if I should be passing a linear RGB value into ffmpeg or if I should be passing an already BT.709 gamma encoded value which would then need to be decoded in the client before the linear conversion Matrix step to get back to RGB.
Update:
Based on the feedback, I have updated my C based implementation and Metal shader and uploaded to github as an iOS example project MetalBT709Decoder.
Encoding a normalized linear RGB value is implemented like this:
static inline
int BT709_convertLinearRGBToYCbCr(
float Rn,
float Gn,
float Bn,
int *YPtr,
int *CbPtr,
int *CrPtr,
int applyGammaMap)
{
// Gamma adjustment to non-linear value
if (applyGammaMap) {
Rn = BT709_linearNormToNonLinear(Rn);
Gn = BT709_linearNormToNonLinear(Gn);
Bn = BT709_linearNormToNonLinear(Bn);
}
// https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.709-6-201506-I!!PDF-E.pdf
float Ey = (Kr * Rn) + (Kg * Gn) + (Kb * Bn);
float Eb = (Bn - Ey) / Eb_minus_Ey_Range;
float Er = (Rn - Ey) / Er_minus_Ey_Range;
// Quant Y to range [16, 235] (inclusive 219 values)
// Quant Eb, Er to range [16, 240] (inclusive 224 values, centered at 128)
float AdjEy = (Ey * (YMax-YMin)) + 16;
float AdjEb = (Eb * (UVMax-UVMin)) + 128;
float AdjEr = (Er * (UVMax-UVMin)) + 128;
*YPtr = (int) round(AdjEy);
*CbPtr = (int) round(AdjEb);
*CrPtr = (int) round(AdjEr);
return 0;
}
Decoding from YCbCr to linear RGB is implemented like so:
static inline
int BT709_convertYCbCrToLinearRGB(
int Y,
int Cb,
int Cr,
float *RPtr,
float *GPtr,
float *BPtr,
int applyGammaMap)
{
// https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.709_conversion
// http://www.niwa.nu/2013/05/understanding-yuv-values/
// Normalize Y to range [0 255]
//
// Note that the matrix multiply will adjust
// this byte normalized range to account for
// the limited range [16 235]
float Yn = (Y - 16) * (1.0f / 255.0f);
// Normalize Cb and CR with zero at 128 and range [0 255]
// Note that matrix will adjust to limited range [16 240]
float Cbn = (Cb - 128) * (1.0f / 255.0f);
float Crn = (Cr - 128) * (1.0f / 255.0f);
const float YScale = 255.0f / (YMax-YMin);
const float UVScale = 255.0f / (UVMax-UVMin);
const
float BT709Mat[] = {
YScale, 0.000f, (UVScale * Er_minus_Ey_Range),
YScale, (-1.0f * UVScale * Eb_minus_Ey_Range * Kb_over_Kg), (-1.0f * UVScale * Er_minus_Ey_Range * Kr_over_Kg),
YScale, (UVScale * Eb_minus_Ey_Range), 0.000f,
};
// Matrix multiply operation
//
// rgb = BT709Mat * YCbCr
// Convert input Y, Cb, Cr to normalized float values
float Rn = (Yn * BT709Mat[0]) + (Cbn * BT709Mat[1]) + (Crn * BT709Mat[2]);
float Gn = (Yn * BT709Mat[3]) + (Cbn * BT709Mat[4]) + (Crn * BT709Mat[5]);
float Bn = (Yn * BT709Mat[6]) + (Cbn * BT709Mat[7]) + (Crn * BT709Mat[8]);
// Saturate normalzied linear (R G B) to range [0.0, 1.0]
Rn = saturatef(Rn);
Gn = saturatef(Gn);
Bn = saturatef(Bn);
// Gamma adjustment for RGB components after matrix transform
if (applyGammaMap) {
Rn = BT709_nonLinearNormToLinear(Rn);
Gn = BT709_nonLinearNormToLinear(Gn);
Bn = BT709_nonLinearNormToLinear(Bn);
}
*RPtr = Rn;
*GPtr = Gn;
*BPtr = Bn;
return 0;
}
I believe this logic is implemented correctly, but I am having a very difficult time validating the results. When I generate a .m4v file that contains gamma adjusted color values (osxcolor_test_image_24bit_BT709.m4v), the result come out as expected. But a test case like (bars_709_Frame01.m4v) that I found here does not seem to work as the color bar values seem to be encoded as linear (no gamma adjustment).
For a SMPTE test pattern, the 0.75 graylevel is linear RGB (191 191 191), should this RGB be encoded with no gamma adjustment as (Y Cb Cr) (180 128 128) or should the value in the bitstream appear as the gamma adjusted (Y Cb Cr) (206 128 128)?
(follow up)
After doing additional research into this gamma issue, it has become clear that what Apple is actually doing in AVFoundation is using a 1.961 gamma function. This is the case when encoding with AVAssetWriterInputPixelBufferAdaptor, when using vImage, or with CoreVideo APIs. This piecewise gamma function is defined as follows:
#define APPLE_GAMMA_196 (1.960938f)
static inline
float Apple196_nonLinearNormToLinear(float normV) {
const float xIntercept = 0.05583828f;
if (normV < xIntercept) {
normV *= (1.0f / 16.0f);
} else {
const float gamma = APPLE_GAMMA_196;
normV = pow(normV, gamma);
}
return normV;
}
static inline
float Apple196_linearNormToNonLinear(float normV) {
const float yIntercept = 0.00349f;
if (normV < yIntercept) {
normV *= 16.0f;
} else {
const float gamma = 1.0f / APPLE_GAMMA_196;
normV = pow(normV, gamma);
}
return normV;
}
Your original question: Does H.264 encoded video with BT.709 matrix include any gamma adjustment?
The encoded video only contains gamma adjustment - if you feed the encoder gamma adjusted values.
A H.264 encoder doesn't care about the transfer characteristics.
So if you compress linear and then decompress - you'll get linear.
So if you compress with gamma and then decompress - you'll get gamma.
Or if your bits are encoded with a Rec. 709 transfer function - the encoder won't change the gamma.
But you can specify the transfer characteristic in the H.264 stream as metadata. (Rec. ITU-T H.264 (04/2017) E.1.1 VUI parameters syntax). So the encoded streams carries the color space information around but it is not used in encoding or decoding.
I would assume that 8 bit video always contains a non linear transfer function. Otherwise you would use the 8 bit fairly unwisely.
If you convert to linear to do effects and composition - I'd recommend increasing the bit depth or linearizing into floats.
A color space consists of primaries, transfer function and matrix coefficients.
The gamma adjustment is encoded in the transfer function (and not in the matrix).
There is a lot of information on the internet about the differences between YUV4:4:4 to YUV4:2:2 formats, however, I can not find anything that tells how to convert the YUV4:4:4 to YUV4:2:2. Since such conversion is performed using software, I was hoping that there should be some developers that have done it and could direct me to the sources that describe the conversion algorithm. Of course, the software code would be nice to have, but having the access to the theory would be sufficient enough to write my own software. Specifically, I would like to know pixel structure and how the bytes are managed during conversion.
I found several similar questions like this and this, however, could not get my question answered. Also, I posted this question on the Photography forum, and they considered it as a software question.
The reason why you can't find specific description, is that there are many ways to do it.
Lets start from Wikipedia: https://en.wikipedia.org/wiki/Chroma_subsampling#4:2:2
4:4:4:
Each of the three Y'CbCr components have the same sample rate, thus there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and cinematic post production.
and
4:2:2:
The two chroma components are sampled at half the sample rate of luma: the horizontal chroma resolution is halved. This reduces the bandwidth of an uncompressed video signal by one-third with little to no visual difference.
Note: Terms YCbCr and YUV are used interchangeably.
https://en.wikipedia.org/wiki/YCbCr
Y′CbCr is often confused with the YUV color space, and typically the terms YCbCr and YUV are used interchangeably, leading to some confusion; when referring to signals in video or digital form, the term "YUV" mostly means "Y′CbCr".
Data memory ordering:
Again there is more than one format.
Intel IPP documentation defines two main categories: "Pixel-Order Image Formats" and "Planar Image Formats".
There is a nice documentation here: https://software.intel.com/en-us/node/503876
Refer here: http://www.fourcc.org/yuv.php#NV12 for YUV pixel arrangement formats.
Refer here: http://scc.ustc.edu.cn/zlsc/sugon/intel/ipp/ipp_manual/IPPI/ippi_ch6/ch6_image_downsampling.htm#ch6_image_downsampling for downsampling description.
Let's assume "Pixel-Order" format:
YUV 4:4:4 data order: Y0 U0 V0 Y1 U1 V1 Y2 U2 V2 Y3 U3 V3
YUV 4:2:2 data order: Y0 U0 Y1 V0 Y2 U1 Y3 V1
Each element is a single byte, and Y0 is the lower byte in memory.
The 4:2:2 data order described above is named UYVY or YUY2 pixel format.
Conversion algorithms:
"Naive sub-sampling":
"Throw" every second U/V component:
Take U0, and throw U1, take V0 and throw V1...
Source: Y0 U0 V0 Y1 U1 V1 Y2 U2 V2
Destination: Y0 U0 Y1 V0 Y2 U2 Y3 V2
I can't recommend it, since it causes aliasing artifacts.
Average each U/V pair:
Take Destination U0 equals source (U0+U1)/2, same for V0...
Source: Y0 U0 V0 Y1 U1 V1 Y2 U2 V2
Destination: Y0 (U0+U1)/2 Y1 (V0+V1)/2 Y2 (U2+U3)/2 Y3 (V2+V3)/2
Use other interpolation method for down-sampling U and V (cubic interpolation for example).
Usually you will not be able to see any differences compared to simple average.
C implementation:
The question is not tagged as C, but I think the following C implementation may be helpful.
The following code converts pixel-ordered YUV 4:4:4 to pixel-ordered YUV 4:2:2 by averaging each U/V pair:
//Convert single row I0 from pixel-ordered YUV 4:4:4 to pixel-ordered YUV 4:2:2.
//Save the result in J0.
//I0 size in bytes is image_width*3
//J0 size in bytes is image_width*2
static void ConvertRowYUV444ToYUV422(const unsigned char I0[],
const int image_width,
unsigned char J0[])
{
int x;
//Process two Y,U,V triples per iteration:
for (x = 0; x < image_width; x += 2)
{
//Load source elements
unsigned char y0 = I0[x*3]; //Load source Y element
unsigned int u0 = (unsigned int)I0[x*3+1]; //Load source U element (and convert from uint8 to uint32).
unsigned int v0 = (unsigned int)I0[x*3+2]; //Load source V element (and convert from uint8 to uint32).
//Load next source elements
unsigned char y1 = I0[x*3+3]; //Load source Y element
unsigned int u1 = (unsigned int)I0[x*3+4]; //Load source U element (and convert from uint8 to uint32).
unsigned int v1 = (unsigned int)I0[x*3+5]; //Load source V element (and convert from uint8 to uint32).
//Calculate destination U, and V elements.
//Use shift right by 1 for dividing by 2.
//Use plus 1 before shifting - round operation instead of floor operation.
unsigned int u01 = (u0 + u1 + 1) >> 1; //Destination U element equals average of two source U elements.
unsigned int v01 = (v0 + v1 + 1) >> 1; //Destination U element equals average of two source U elements.
J0[x*2] = y0; //Store Y element (unmodified).
J0[x*2+1] = (unsigned char)u01; //Store destination U element (and cast uint32 to uint8).
J0[x*2+2] = y1; //Store Y element (unmodified).
J0[x*2+3] = (unsigned char)v01; //Store destination V element (and cast uint32 to uint8).
}
}
//Convert image I from pixel-ordered YUV 4:4:4 to pixel-ordered YUV 4:2:2.
//I - Input image in pixel-order data YUV 4:4:4 format.
//image_width - Number of columns of image I.
//image_height - Number of rows of image I.
//J - Destination "image" in pixel-order data YUV 4:2:2 format.
//Note: The term "YUV" referees to "Y'CbCr".
//I is pixel ordered YUV 4:4:4 format (size in bytes is image_width*image_height*3):
//YUVYUVYUVYUV
//YUVYUVYUVYUV
//YUVYUVYUVYUV
//YUVYUVYUVYUV
//
//J is pixel ordered YUV 4:2:2 format (size in bytes is image_width*image_height*2):
//YUYVYUYV
//YUYVYUYV
//YUYVYUYV
//YUYVYUYV
//
//Conversion algorithm:
//Each element of destination U is average of 2 original U horizontal elements
//Each element of destination V is average of 2 original V horizontal elements
//
//Limitations:
//1. image_width must be a multiple of 2.
//2. I and J must be two separate arrays (in place computation is not supported).
static void ConvertYUV444ToYUV422(const unsigned char I[],
const int image_width,
const int image_height,
unsigned char J[])
{
//I0 points source row.
const unsigned char *I0; //I0 -> YUYVYUYV...
//J0 and points destination row.
unsigned char *J0; //J0 -> YUYVYUYV
int y; //Row index
//In each iteration process single row.
for (y = 0; y < image_height; y++)
{
I0 = &I[y*image_width*3]; //Input row width is image_width*3 bytes (each pixel is Y,U,V).
J0 = &J[y*image_width*2]; //Output row width is image_width*2 bytes (each two pixels are Y,U,Y,V).
//Process single source row into single destination row
ConvertRowYUV444ToYUV422(I0, image_width, J0);
}
}
Planar representation of YUV 4:2:2
Planar representation may be more intuitive than "Pixel-Order" format.
In planar representation each color channel is represented as a separate matrix, which can be displayed as an image.
Example:
Original image in RGB format (before converting to YUV):
Image channels in YUV 4:4:4 format:
(Left YUV triple is represented in gray levels, and right YUV triple is represented using false colors).
Image channels in YUV 4:2:2 format (after horizontal Chroma subsampling):
(Left YUV triple is represented in gray levels, and right YUV triple is represented using "false colors").
As you can see, in 4:2:2 format, the U an V channels are down-sampled (shrunk) in the horizontal axis.
Remark:
The "false colors" representation of U and V channels is used for emphasizing that Y is the Luma channel and U and V are the Chrominance channels.
Higher order interpolation and Anti-Aliasing filter:
Following MATLAB code sample shows how to perform down-sampling with higher order interpolation and Anti-Aliasing filter.
The sample also shows the down-sampling method used by FFMPEG.
Note: you don't need to know MATLAB programming in order to understand the samples.
You do need some knowledge of image filtering by convolution between a Kernel and an image.
%Prepare the input:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
load('mandrill.mat', 'X', 'map'); %Load input image
RGB = im2uint8(ind2rgb(X, map)); %Convert to RGB (the mandrill sample image is an indexed image)
YUV = rgb2ycbcr(RGB); %Convert from RGB to YUV (MATLAB function rgb2ycbcr uses BT.601 conversion formula)
%Separate YUV to 3 planes (Y plane, U plane and V plane)
Y = YUV(:, :, 1);
U = YUV(:, :, 2);
V = YUV(:, :, 3);
U = double(U); %Work in double precision instead of uint8.
[M, N] = size(Y); %Image size is N columns by M rows.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Linear interpolation without Anti-Aliasing filter:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Horizontal down-sampling U plane using Linear interpolation (without Anti-Aliasing filter).
%Simple averaging is equivalent to linear interpolation.
U2 = (U(:, 1:2:end) + U(:, 2:2:end))/2;
refU2 = imresize(U, [M, N/2], 'bilinear', 'Antialiasing', false); %Use MATLAB imresize function as reference
disp(['Linear interpolation max diff = ' num2str(max(abs(double(U2(:)) - double(refU2(:)))))]); %Print maximum difference.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Cubic interpolation without Anti-Aliasing filter:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Horizontal down-sampling U plane using Cubic interpolation (without Anti-Aliasing filter).
%Following operations are equivalent to cubic interpolation:
%1. Convolution with filter kernel [-0.125, 1.25, -0.125]
%2. Averaging pair elements
fU = imfilter(U, [-0.125, 1.25, -0.125], 'symmetric');
U2 = (fU(:, 1:2:end) + fU(:, 2:2:end))/2;
U2 = max(min(U2, 240), 16); %Limit to valid range of U elements (valid range of U elements in uint8 format is [16, 240])
refU2 = imresize(U, [M, N/2], 'cubic', 'Antialiasing', false); %Use MATLAB imresize function as reference
refU2 = max(min(refU2, 240), 16); %Limit to valid range of U elements
disp(['Cubic interpolation max diff = ' num2str(max(abs(double(U2(:)) - double(refU2(:)))))]); %Print maximum difference.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Linear interpolation with Anti-Aliasing filter:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Horizontal down-sampling U plane using Linear interpolation with Anti-Aliasing filter.
%Remark: The Anti-Aliasing filter is the filter used by MATLAB specific implementation of 'bilinear' imresize.
%Following operations are equivalent to Linear interpolation with Anti-Aliasing filter:
%1. Convolution with filter kernel [0.25, 0.5, 0.25]
%2. Averaging pair elements
fU = imfilter(U, [0.25, 0.5, 0.25], 'symmetric');
U2 = (fU(:, 1:2:end) + fU(:, 2:2:end))/2;
refU2 = imresize(U, [M, N/2], 'bilinear', 'Antialiasing', true); %Use MATLAB imresize function as reference
disp(['Linear interpolation with Anti-Aliasing max diff = ' num2str(max(abs(double(U2(:)) - double(refU2(:)))))]); %Print maximum difference.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Cubic interpolation with Anti-Aliasing filter:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Horizontal down-sampling U plane using Cubic interpolation with Anti-Aliasing filter.
%Remark: The Anti-Aliasing filter is the filter used by MATLAB specific implementation of 'cubic' imresize.
%Following operations are equivalent to Linear interpolation with Anti-Aliasing filter:
%1. Convolution with filter kernel [-0.0234375, -0.046875, 0.2734375, 0.59375, 0.2734375, -0.046875, -0.0234375]
%2. Averaging pair elements
h = [-0.0234375, -0.046875, 0.2734375, 0.59375, 0.2734375, -0.046875, -0.0234375];
fU = imfilter(U, h, 'symmetric');
U2 = (fU(:, 1:2:end) + fU(:, 2:2:end))/2;
U2 = max(min(U2, 240), 16); %Limit to valid range of U elements
refU2 = imresize(U, [M, N/2], 'cubic', 'Antialiasing', true); %Use MATLAB imresize function as reference
refU2 = max(min(refU2, 240), 16); %Limit to valid range of U elements
disp(['Cubic interpolation with Anti-Aliasing max diff = ' num2str(max(abs(double(U2(:)) - double(refU2(:)))))]); %Print maximum difference.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%FFMPEG implementation of horizontal down-sampling U plane.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%FFMPEG uses cubic interpolation with Anti-Aliasing filter (different filter kernel):
%Remark: I didn't check the source code of FFMPEG to verify the values of the filter kernel.
%I can't tell how FFMPEG actually implements the conversion.
%Following operations are equivalent to FFMPEG implementation (with minor differences):
%1. Convolution with filter kernel [-115, -231, 1217, 2354, 1217, -231, -115]/4096
%2. Averaging pair elements
h = [-115, -231, 1217, 2354, 1217, -231, -115]/4096;
fU = imfilter(U, h, 'symmetric');
U2 = (fU(:, 1:2:end) + fU(:, 2:2:end))/2;
U2 = max(min(U2, 240), 16); %Limit to valid range of U elements (FFMPEG actually doesn't limit the result)
%Save Y,U,V planes to file in format supported by FFMPEG
f = fopen('yuv444.yuv', 'w');
fwrite(f, Y', 'uint8');
fwrite(f, U', 'uint8');
fwrite(f, V', 'uint8');
fclose(f);
%For executing FFMPEG within MATLAB, download FFMPEG and place the executable in working directory (ffmpeg.exe for Windows)
%FFMPEG converts source file in YUV444 format to destination file in YUV422 format.
if isunix
[status, cmdout] = system(['./ffmpeg -y -s ', num2str(N), 'x', num2str(M), ' -pix_fmt yuv444p -i yuv444.yuv -pix_fmt yuv422p yuv422.yuv']);
else
[status, cmdout] = system(['ffmpeg.exe -y -s ', num2str(N), 'x', num2str(M), ' -pix_fmt yuv444p -i yuv444.yuv -pix_fmt yuv422p yuv422.yuv']);
end
f = fopen('yuv422.yuv', 'r');
refY = (fread(f, [N, M], '*uint8'))';
refU2 = (fread(f, [N/2, M], '*uint8'))'; %Read down-sampled U plane (FFMPEG result from file).
refV2 = (fread(f, [N/2, M], '*uint8'))';
fclose(f);
%Limit to valid range of U elements.
%In FFMPEG down-sampled U and V may exceed valid range (there is probably a way to tell FFMPEG to limit the result).
refU2 = max(min(refU2, 240), 16);
%Difference exclude first column and last column (FFMPEG treats the margins different than MATLAB)
%Remark: There are minor differences due to rounding (I guess).
disp(['FFMPEG Cubic interpolation with Anti-Aliasing max diff = ' num2str(max(max(abs(double(U2(:, 2:end-1)) - double(refU2(:, 2:end-1))))))]);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Examples for different kind of down-sampling methods.
Linear interpolation versus Cubic interpolation with Anti-Aliasing filter:
In the first example (mandrill) there are no visible differences.
In the second example (circle and rectangle) there are minor visible differences.
The third example (lines) demonstrates aliasing artifacts.
Remark: displayed images where up-sampled from YUV422 to YUV444 using Cubic interpolation and converted from YUV444 to RGB.
Linear interpolation versus Cubic with Anti-Aliasing (mandrill):
Linear interpolation versus Cubic with Anti-Aliasing (circle and rectangle):
Linear interpolation versus Cubic with Anti-Aliasing (demonstrates Aliasing artifacts):
I want to calculate histogram of an image hows pixels are of type 32F (32 bit floating point). What should be the parameter values of "calcHist" function for:
- dims
- bins
- range
Well I've done this many times. Something like so:
cv::Mat matSrc; // this is a CV_32FC1 normalised image
int nHistSize = 65536;
float fRange[] = { 0.0f, 1.0f };
const float* fHistRange = { fRange };
cv::Mat matHist;
cv::calcHist(&matSrc, 1, 0, cv::Mat(), matHist, 1, &nHistSize, &fHistRange);
As it says in the documentation describing the source arrays:
Source arrays. They all should have the same depth, CV_8U or CV_32F ,
and the same size. Each of them can have an arbitrary number of
channels.
So CV_32F is supported. In this situation, the range (in my example 0.0 to 1.0) is binned into the number of bins required (in my example 65536).
I need to exchange data between FreeImage (FIBITMAP) and OpenCV format (IplImage and/or Mat). I'm fine with getting data from a FIBITMAP into an IplImage or Mat since FreeImage gives you a function FreeImage_GetScanLine which you can set the OPenCV imageData ptr equal to.
However, I'm stuck on how to do the reverse, i.e. once I have an OpenCV image, how do I get its data into a FreeImage image?
Here is a more detailed code for the conversion. There are many image data types in either library, I tried supporting most common ones. This assumes you are passing a cv::Mat as the source. FreeImage has the view perspective of lower left!
/* These are openCV types
#define CV_8U 0
#define CV_8S 1
#define CV_16U 2
#define CV_16S 3
#define CV_32S 4
#define CV_32F 5
#define CV_64F 6
*/
/* these are FI types
FIT_UNKNOWN = 0, // unknown type
FIT_BITMAP = 1, // standard image : 1-, 4-, 8-, 16-, 24-, 32-bit
FIT_UINT16 = 2, // array of unsigned short : unsigned 16-bit
FIT_INT16 = 3, // array of short : signed 16-bit
FIT_UINT32 = 4, // array of unsigned long : unsigned 32-bit
FIT_INT32 = 5, // array of long : signed 32-bit
FIT_FLOAT = 6, // array of float : 32-bit IEEE floating point
FIT_DOUBLE = 7, // array of double : 64-bit IEEE floating point
FIT_COMPLEX = 8, // array of FICOMPLEX : 2 x 64-bit IEEE floating point
FIT_RGB16 = 9, // 48-bit RGB image : 3 x 16-bit
FIT_RGBA16 = 10, // 64-bit RGBA image : 4 x 16-bit
FIT_RGBF = 11, // 96-bit RGB float image : 3 x 32-bit IEEE floating point
FIT_RGBAF = 12 // 128-bit RGBA float image : 4 x 32-bit IEEE floating point
*/
if(_dib) // get rid of the current dib.
FreeImage_Unload(_dib);
int width = src.size().width;
int height = src.size().height;
switch(src.type())
{
case CV_8U :{_dib = FreeImage_AllocateT(FIT_BITMAP,width, height, 8) ;}break; // 8 bit grayscale
case CV_8UC3:{_dib = FreeImage_AllocateT(FIT_BITMAP,width, height, 24);}break; // 24 bit RGB
case CV_16U :{_dib = FreeImage_AllocateT(FIT_UINT16,width, height, 16);}break; // 16 bit grayscale
case CV_16S :{_dib = FreeImage_AllocateT(FIT_INT16 ,width, height, 16);}break;
case CV_32S :{_dib = FreeImage_AllocateT(FIT_INT32 ,width, height, 32);}break;
case CV_32F :{_dib = FreeImage_AllocateT(FIT_FLOAT ,width, height, 32);}break;
case CV_64F :{_dib = FreeImage_AllocateT(FIT_DOUBLE,width, height, 32);}break;
default:ASSERT(FALSE);
}
if(_dib==NULL)
return FALSE;
int srcRowBytes = width * src.elemSize();
for (int ih=0;ih<height;ih++)
{
BYTE* ptr2Line = FreeImage_GetScanLine(_dib,(height-1)-ih);
memcpy(ptr2Line,src.ptr(ih),srcRowBytes);
}
_bHasChanged = TRUE;
return TRUE;
You have to be a little careful just copying the data pointer, a lot of image formats have padding to start each new line on eg a 4byte boundary.
If your image has a GetScanLine() function then it's probably safer to make an empty plImage*/cv::Mat and memcopy each row with the pointer returned from GetScanLine() and the .ptr() member of cv::MAt
cv::Mat &src
int srcRowBytes = width * src.elemSize();
for (int ih=0;ih<height;ih++) {
memcpy(dest.pointer_to_row(ih),src.ptr(ih),srcRowBytes);
}
Well, if you don't mind the copy, you can just create an IplImage*/cv::Mat header for the FIBITMAP and then copy (using the opencv function), like this:
cv::Mat src; // your source image
FIBITMAP whatever; // allocate space for your FIBITMAP here
cv::Mat wrapper(height, width, CV_8UC3, ptr_from_FIBITMAP, step);
src.copyTo(wrapper);