I am trying to convert a YUV image to RGB using OpenCV. I am a complete novice at this. I have created a function which takes a YUV image as source and converts it into RGB. It is like this :
void ConvertYUVtoRGBA(const unsigned char *src, unsigned char *dest, int width, int height)
{
cv::Mat myuv(height + height/2, width, CV_8UC1, &src);
cv::Mat mrgb(height, width, CV_8UC4, &dest);
cv::cvtColor(myuv, mrgb, CV_YCrCb2RGB);
return;
}
Should this work? Do I need to convert the Mat into char* again? I am in a loss and any help will be greatly appreciated.
There is not enough detail in your question to give a certain answer but below is my best guess. I'll assume you want RGBA output (not RGB, BGR or BGRA) and that your YUV is yuv420sp (as this is what comes out of an Android camera, and it is consistent with your Mat sizes)
void ConvertYUVtoRGBA(const unsigned char *src, unsigned char *dest, int width, int height)
{
//cv::Mat myuv(height + height/2, width, CV_8UC1, &src);
cv::Mat myuv(height + height/2, width, CV_8UC1, src); // pass buffer pointer, not its address
//cv::Mat mrgb(height, width, CV_8UC4, &dest);
cv::Mat mrgb(height, width, CV_8UC4, dest);
//cv::cvtColor(myuv, mrgb, CV_YCrCb2RGB);
cv::cvtColor(myuv, mrgb, CV_YUV2RGBA_NV21); // are you sure you don't want BGRA?
return;
}
Do I need to convert the Mat into char again?*
No the Mat mrgb is a wrapper around dest and, the way you have arranged it, the RGBA data will written directly into the dest buffer.
Related
I would like to convert a OpenCV CV_32FC1 Mat into a png to save it and later use it in a Unity Shader.
I would like to decode it so that the first channel contains the highest 8 bits, the second channel the next 8 bits and the third channel the next 8 bits.
Edit> I actually mean the highest bits of the mantissa. Otherwise discarding 8 Bit (since I need 3 channels for imwrite) would destroy the float-representation.
I already have this working the other way around with this function:
Mat prepareLUT(char* filename){
Mat first;
first = imread(filename, CV_LOAD_IMAGE_COLOR);
Mat floatmat;
first.convertTo(floatmat, CV_32F);
std::vector<Mat> channels(3);
split(floatmat, channels);
Mat res(Size(960,1080), CV_32FC1);
res = channels[2]/255 + channels[1]/(255.0*255.0) + channels[0]/(255.0*255.0*255.0);
return res;
}
but I am unable to do this the other way around.
My first idea was the following:
void saveLUT(Mat in, char const* filename){
Mat m1 = Mat(imageSize, CV_8UC1);
Mat m2 = Mat(imageSize, CV_8UC1);
Mat m3 = Mat(imageSize, CV_8UC1);
m1 = (in*(0xFF*0xFF*0xFF-1));
m2 = (in*(0xFF*0xFF-1));
m3 = (in*(0xFF-1));
std::vector<Mat> channels;
channels.push_back(m1);
channels.push_back(m2);
channels.push_back(m3);
Mat out;
merge(channels, out);
imwrite(filename, out);
}
I thought all the bits left and right of my 8 Bit range would be cut-off, giving me the right Mat, but it always outputs some gray image.
The second approach was to work with float mats, then convert them to Char Mats to cut-off the trailing numbers:
void saveLUT(Mat in, char const* filename){
Mat m1f(imageSize, CV_32FC1);
Mat m2f(imageSize, CV_32FC1);
Mat m3f(imageSize, CV_32FC1);
Mat m1, m2, m3;
m3f = in*255;
m3f.convertTo(m3, CV_8UC1);
m3.convertTo(m3f, CV_32FC1);
m2f = (in*255-m3f)*255;
m2f.convertTo(m2, CV_8UC1);
m2.convertTo(m2f, CV_32FC1);
m1f = ((in*255-m3f)*255-m2f)*255;
m1f.convertTo(m1, CV_8UC1);
std::vector<Mat> channels;
channels.push_back(m1);
channels.push_back(m2);
channels.push_back(m3);
Mat out;
merge(channels, out);
imwrite(filename, out);
}
This way I always subtract the numbers that are too high by subtracting the result for the previous channel before multiplying, but this still gives me a gray result as the one below.
Any Idea how to tacle this?
What you want to achieve is essentially a conversion from type CV_32FC1 to type CV_8UC4 that you can then save as a PNG file.
This can be achieved in one line in C++ using the data pointers:
cv::Mat floatImage; // Your CV_32FC1 Mat
cv::Mat pngImage(floatImage.rows, floatImage.cols, CV_8UC4, (cv::Vec4b*)floatImage.data);
What you obtain is a 4-channel 8-bit precision image where each pixel contains one of the floating-point values in your original image separated in 4 blocks of 8 bits.
The inverse transformation is also possible:
cv::Mat pngImage;
cv::Mat floatImage(pngImage.rows, pngImage.cols, CV_32FC1, (float*)pngImage.data);
I found a way to do this, but it is not very pretty.
I just perform the conversion on each value, with a few bit operations and bit-shifting like so:
void saveLUT(Mat in, char const* filename){
int i,j;
Mat c1(imageSize, CV_8UC1);
Mat c2(imageSize, CV_8UC1);
Mat c3(imageSize, CV_8UC1);
for(i = 0; i < in.cols; i++){
for(j = 0; j < in.rows; j++){
float orig = in.at<float>(j,i);
uint32_t orig_int = orig*(256.0*256.0*256.0-1);
c1.at<uint8_t>(j,i) = (uint8_t)((orig_int&0xFF0000) >> 16);
c2.at<uint8_t>(j,i) = (uint8_t)((orig_int&0x00FF00) >> 8);
c3.at<uint8_t>(j,i) = (uint8_t)((orig_int&0x0000FF));
}
}
std::vector<Mat> channels;
channels.push_back(c1);
channels.push_back(c2);
channels.push_back(c3);
Mat out;
merge(channels, out);
imwrite(filename, out);
Mat encoded(imageSize, CV_8UC4);
}
It's not pretty to look at and I have to assume there are faster methods to do this, but I did not find any and it runs fast enough for my purpose.
I am trying to convert BGR image into YUV420P, but when I try to view each of the YUV planes separately, this is what I see.
Shouldn't cv::Mat::data and AVFrame::data[0] be packed in the same way? I should be able to do a direct memcpy. Am I missing something?
Any ideas?
Mat frame;
VideoCapture cap;
if(!cap.open(0)){
return 0;
}
// capture a frame
cap >> frame;
if( frame.empty() ) return 0;
cv::Size s = frame.size();
int height = s.height;
int width = s.width;
// Creating two frames for conversion
AVFrame *pFrameYUV =av_frame_alloc();
AVFrame *pFrameBGR =av_frame_alloc();
// Determine required buffer size and allocate buffer for YUV frame
int numBytesYUV=av_image_get_buffer_size(AV_PIX_FMT_YUV420P, width,
height,1);
// Assign image buffers
avpicture_fill((AVPicture *)pFrameBGR, frame.data, AV_PIX_FMT_BGR24,
width, height);
uint8_t* bufferYUV=(uint8_t *)av_malloc(numBytesYUV*sizeof(uint8_t));
avpicture_fill((AVPicture *)pFrameYUV, bufferYUV, AV_PIX_FMT_YUV420P,
width, height);
// Initialise Software scaling context
struct SwsContext *sws_ctx = sws_getContext(width,
height,
AV_PIX_FMT_BGR24,
width,
height,
AV_PIX_FMT_YUV420P,
SWS_BILINEAR,
NULL,
NULL,
NULL
);
// Convert the image from its BGR to YUV
sws_scale(sws_ctx, (uint8_t const * const *)pFrameBGR->data,
pFrameYUV->linesize, 0, height,
pFrameYUV->data, pFrameYUV->linesize);
// Trying to see the different planes of YUV
Mat MY = Mat(height, width, CV_8UC1);
memcpy(MY.data,pFrameYUV->data[0], height*width);
imshow("Test1", MY); // fail
Mat MU = Mat(height/2, width/2, CV_8UC1);
memcpy(MU.data,pFrameYUV->data[1], height*width/4);
imshow("Test2", MU); // fail
Mat MV = Mat(height/2, width/2, CV_8UC1);
memcpy(MV.data,pFrameYUV->data[2], height*width/4);
imshow("Test3", MV); // fail
waitKey(0); // Wait for a keystroke in the window
For sws_scale() third parameter should not be pFrameYUV->linesize but rather pFrameBGR->linesize, i.e. pointer to width*3.
My code is working fine with showing brightness in the image using below code
jint* _in = env->GetIntArrayElements(in, 0);
jint* _out = env->GetIntArrayElements(out, 0);
Mat mSrc(height, width, CV_8UC4, (unsigned char*)_in);
Mat bgra(height, width, CV_8UC4, (unsigned char*)_out);
vector<Mat> sChannels;
split(mSrc, sChannels);
for(int i=0; i<sChannels.size(); i++)
{
Mat channel = sChannels[i];
equalizeHist(channel, channel);
}
merge(sChannels, bgra);
env->ReleaseIntArrayElements(in, _in, 0);
env->ReleaseIntArrayElements(out, _out, 0);
jint retVal;
int ret = 1;
retVal = jint(retVal);
return retVal;
It work for me too for changing image into grayscale but in this way :
Mat mSrc(height, width, CV_8UC4, (unsigned char*)_in);
Mat gray(height, width, CV_8UC1);
Mat bgra(height, width, CV_8UC4, (unsigned char*)_out);
cvtColor(mSrc , gray , CV_BGRA2GRAY);
cvtColor(gray , bgra , CV_GRAY2BGRA);
But when i am trying to use bilateralfilter with it , which only work with 3 channels as given here , how to deal with it ? because java bitmap accept RGBA format ,and when i change the above into
Mat mSrc(height, width, CV_8UC3, (unsigned char*)_in);
Mat bgra(height, width, CV_8UC3, (unsigned char*)_in);
somehow bilateral filter show me output , but what if I have apply all these filter on one image ? how can I handle this problem ? because there may be other algorithms too which only deal with 3 channels.
If you need to convert CV_8UC3 to CV_8UC4 you just need to call:
cvtColor(src, dst, CV_BGR2BGRA);
since it outputs a 4-channel image (R, G, B and Alpha)
Inversely,
cvtColor(src, dst, CV_BGRA2BGR);
should convert CV_8UC4 to CV_8UC3
I have a buffer which contains an image in YV12 format. Now I want to either convert this buffer to RGB format or create a Mat object from it directly! Can someone help me? I tried this code :
cv::Mat input(widthOfImg, heightOfImg, CV_8UC1, vy12Buffer);
cv::Mat converted;
cv::cvtColor(input, converted, CV_YUV2RGB_YV12);
That's possible.
cv::Mat picYV12 = cv::Mat(nHeight * 3/2, nWidth, CV_8UC1, yv12DataBuffer);
cv::Mat picBGR;
cv::cvtColor(picYV12, picBGR, CV_YUV2BGR_YV12);
cv::imwrite("test.bmp", picBGR); //only for test
Opencv color conversion flags
The height is multiplied by 3/2 because there are 4 Y samples, and 1 U and 1 V sample stored for every 2x2 square of pixels. This results in a byte sample to pixel ratio of 3/2
4*1+1+1 samples per 2*2 pixels = 6/4 = 3/2
YV12 Format
Correction: In the last version of OpenCV (i use oldest 2.4.13 version) is color conversion code changed to
COLOR_YUV2BGR_YV12
cv::cvtColor(picYV12, picBGR, COLOR_YUV2BGR_YV12);
here is the corresponding version in java (Android)...
This method was faster than other techniques like renderscript or opengl(glReadPixels) for getting bitmap from yuv12/i420 data stream (tested with webrtc i420 ).
long startTimei = SystemClock.uptimeMillis();
Mat picyv12 = new Mat(768,512,CV_8UC1); //(im_height*3/2,im_width), should be even no...
picyv12.put(0,0,return_buff); // buffer - byte array with i420 data
Imgproc.cvtColor(picyv12,picyv12,COLOR_YUV2RGB_YV12);// or use COLOR_YUV2BGR_YV12 depending on output result
long endTimei = SystemClock.uptimeMillis();
Log.d("i420_time", Long.toString(endTimei - startTimei));
Log.d("picyv12_size", picyv12.size().toString()); // Check size
Log.d("picyv12_type", String.valueOf(picyv12.type())); // Check type
Utils.matToBitmap(picyv12,tbmp2); // Convert mat to bitmap (height, width) i.e (512,512) - ARGB_888
save(tbmp2,"itest"); // Save bitmap
That's impossible.
Y'UV420p is a planar format, meaning that the Y', U, and V values are
grouped together instead of interspersed. The reason for this is that
by grouping the U and V values together, the image becomes much more
compressible. When given an array of an image in the Y'UV420p format,
all the Y' values come first, followed by all the U values, followed
finally by all the V values.
but cv::Mat is a RGB color model, and arranged like B0 G0 R0 B1 G1 R1... So,we can't create a Mat object from a YV12 buffer directly.
Here is an example:
cv::Mat Yv12ToRgb( uchar *pBuffer,long bufferSize, int width,int height )
{
cv::Mat result(height,width,CV_8UC3);
uchar y,cb,cr;
long ySize=width*height;
long uSize;
uSize=ySize>>2;
assert(bufferSize==ySize+uSize*2);
uchar *output=result.data;
uchar *pY=pBuffer;
uchar *pU=pY+ySize;
uchar *pV=pU+uSize;
uchar r,g,b;
for (int i=0;i<uSize;++i)
{
for(int j=0;j<4;++j)
{
y=pY[i*4+j];
cb=ucharpU[i];
cr=ucharpV[i];
//ITU-R standard
b=saturate_cast<uchar>(y+1.772*(cb-128));
g=saturate_cast<uchar>(y-0.344*(cb-128)-0.714*(cr-128));
r=saturate_cast<uchar>(y+1.402*(cr-128));
*output++=b;
*output++=g;
*output++=r;
}
}
return result;
}
You can try as YUV_I420 array
char filePath[3000];
int width, height;
cout << "file path = ";
cin >> filePath;
cout << "width = ";
cin >> width;
cout << "height = ";
cin >> height;
FILE *pFile = fopen(filePath, "rb");
unsigned char* buff = new unsigned char[width * height *3 / 2];
fread(buff, 1, width * height* 3 / 2, pFile);
fclose(pFile);
cv::Mat imageRGB;
cv::Mat picI420 = cv::Mat(height * 3 / 2, width, CV_8UC1, buff);
cv::cvtColor(picI420, imageRGB, CV_YUV2BGRA_I420);
imshow("imageRGB", imageRGB);
waitKey(0);
I want to write data directly into the imageData array of an IplImage, but I can't find a lot of information on how it's formatted. One thing that's particularly troubling me is that, despite creating an image with three channels, there are four bytes to each pixel.
The function I'm using to create the image is:
IplImage *frame = cvCreateImage(cvSize(1, 1), IPL_DEPTH_8U, 3);
By all indications, this should create a three channel RGB image, but that doesn't seem to be the case.
How would I, for example, write a single red pixel to that image?
Thanks for any help, it's get me stumped.
If you are looking at frame->imageSize keep in mind that it is frame->height * frame->widthStep, not frame->height * frame->width.
BGR is the native format of OpenCV, not RGB.
Also, if you're just getting started, you should consider using the C++ interface (where Mat replaces IplImage) since that is the future direction and it's a lot easier to work with.
Here's some sample code that accesses pixel data directly:
int main (int argc, const char * argv[]) {
IplImage *frame = cvCreateImage(cvSize(41, 41), IPL_DEPTH_8U, 3);
for( int y=0; y<frame->height; y++ ) {
uchar* ptr = (uchar*) ( frame->imageData + y * frame->widthStep );
for( int x=0; x<frame->width; x++ ) {
ptr[3*x+2] = 255; //Set red to max (BGR format)
}
}
cvNamedWindow("window", CV_WINDOW_AUTOSIZE);
cvShowImage("window", frame);
cvWaitKey(0);
cvReleaseImage(&frame);
cvDestroyWindow("window");
return 0;
}
unsigned char* imageData = [r1, g1, b1, r2, g2, b2, ..., rN, bn, gn]; // n = height*width of image
frame->imageData = imageData.
Take Your image that is a dimensional array of height N and width M and arrange it into a row-wise vector of length N*M. Make this of type unsigned char* for IPL_DEPTH_8U images.
Straight to your answer, painting the pixel red:
IplImage *frame = cvCreateImage(cvSize(1, 1), IPL_DEPTH_8U, 3);
int y,x;
x=0;y=0; //Pixel coordinates. Use this for bigger images than a single pixel.
int C=2; //0 for blue, 1 for green and 2 for red (BGR is the default format).
frame->imageData[y*frame->widthStep+3*x+C]=(uchar)255;