I can get BGR data from Mat.data pointer, but I don't know how to calculate the data size. Could somebody help me? Thanks.
If your matrix is continuous, I'd go with cv::Mat::total()
to get the number of elements and cv::Mat::elemSize() to get the matrix element size in bytes:
Mat m;
//...
uchar* data = m.data();
auto datasize = m.total() * m.elemSize();
An alternative could be (but I'm not so sure, so double check this) to take the difference between cv::Mat::dataend and cv::Mat::datastart
auto datasize = m.dataend - m.datastart;
If your matrices are not continuous, I guess that you can still use the first method to obtain the size, but don't memcpy() that amount of bytes, because it won't be your image data.
Related
I know this question has been asked quite a bit, but none of the solutions really fit my case. I am looking to add a second type of object to the canvas with the code shown below. I know I didn't provide much but its a quick start. Just ask for more if you think you have a hunch. This code below is in my render function.
So far I have checked that
I have enough vertices in my points array
I have enough normal vectors in my normals array
I have enough texture coordinates in my texCoords array
There are no mismatches between the vectors added when creating my terrain and my propeller.
The terrain renders just fine with the texture, lighting and all but,I am unable to get the propeller to render. I get the error I listed above. I have added multiple objects to canvases before and never run into an error like this.
//----------------------------------------- Draw Terrain ------------------------------------
var i = 0;
for(var row=0-dimension; row<dimension; row+=3){
for(var col=0-dimension; col<dimension; col+=3, i++){
var mv = mult(viewer, mult(translate(row, -1, col), mult(scale[i],rot[i])));
gl.uniformMatrix4fv(modelViewLoc, false, flatten(mv));
gl.uniformMatrix3fv(normalLoc, false, flatten(normalMatrix(mv, true)));
gl.drawArrays( gl.TRIANGLES, 0, index);
}
}
//----------------------------------------- Draw Propeller ------------------------------------
mv = mult(viewer, mult( translate(-2.1, -2.9, -.2), scalem(4,5,5)));
gl.uniformMatrix4fv(modelViewLoc, false, flatten(mv));
gl.uniformMatrix3fv(normalLoc, false, flatten(normalMatrix(mv, true)));
gl.drawArrays( gl.TRIANGLES, propellerStart, points.length);
Is there any way i can use the "Attribute 2" in the error message to track down the variable giving me this issue?
Appreciate the help!
What part don't you understand? The error is clear, whatever buffer you have attached to attribute 2 is not big enough to handle the propellerStart, points.length draw request.
So first thing is figure out which attribute is attribute 2. Do this by printing out your attribute locations. Is your points, normals, or texcoords?
You should already be looking them up somewhere with gl.getAttribLocation so print out those values, find out which one is #2.
Then go look at the size of the buffer you attached to that attribute. To do that somewhere you would have called.
gl.bindBuffer(gl.ARRAY_BUFFER, someBuffer);
gl.vertexAttribPointer(locationForAttribute2, size, type, normalize, stride, offset);
So we know it's someBuffer from the above code. We also need to know size, type, stride, and offset
Somewhere else you filled that buffer with data using
gl.bindBuffer(gl.ARRAEY_BUFFER, someBuffer);
gl.bufferData(gl.ARRAY_BUFFER, someData, ...);
So you need to find the size of someData.
sizeOfBuffer = someData.length * someData.BYTES_PER_ELEMENT
Let's it's a 1000 element Float32Array so it someData.length is 1000 and someData.BYTES_PER_ELEMENT is 4 therefore sizeOfBuffer is 4000.
Using all of that you can now check if your buffer is too small. (note: we already know it's too small since the browser told us so but if you want know how to compute it yourself)
Let's say size is 3, type is gl.FLOAT, stride is 32, offset is 12 (note: I personally never use anything but stride = 0 and offset = 0)
Let's say points.length = 50
numPoints = points.length;
bytesPerElement = size * 4; // because a gl.FLOAT is 4 bytes
realStride = stride === 0 ? bytesPerElement : stride;
bytesNeeded = realStride * (numPoints - 1) + bytesPerElement;
bytesNeeded in this case is (64 * 49) + 12 = 3148
So now we know how many bytes are needed. Does are buffer have enough data? We'll when you called draw you passed in an offset propellerStart. Let's assume it's 900 and there's the offset in the attribute so.
bufferSizeNeeded = offset + propellerStart + bytesNeeded
so bufferSizeNeeded = 12 + 900 + 3148 which is 4060. Since 4060 is > sizeOfBuffer which was 4000 you're going to get the error you got.
In any case the point is really it's up to you to figure out which buffer is used by attribute #2, then go look at why your buffer is too small. Is your offset to drawArrays wrong? Is your stride too big? Is your offset wrong in vertexAttribPointer (it's in number of bytes not number of units). Do you put the wrong size (1,2,3,4). Do you mis-calculate the number of points?
I am trying to find an efficient way to see if one image is a subset of another (meaning that each unique pixel in one image is also found in the other.) The repetition or ordering of the pixels do not matter.
I am working in Java, so I would like all of my operations to be completed in OpenCV for efficiency's sake.
My first idea was to export a list of unique pixel values, and compare it to the list from the second image.
As there is not a built in function to extract unique pixels, I abandoned this approach.
I also understand that I can find the locations of a particular color with the inclusive inRange, and findNonZero operations.
Core.inRange(image, color, color, tempMat); // inclusive
Core.findNonZero(tempMat, colorLocations);
Unfortunately, this does not provide an adequate answer, as it would need to be executed per color, and would still require extracting unique pixels.
Essentially, I'm asking if there is a clever way to use the built in OpenCV functions to see if an image is comprised of the pixels found in another image.
I understand that this will not work for slight color differences. I am working on a limited dataset, and care about the exact pixel values.
To put the question more mathematically:
Because the only think you are interested in is the pixel values i would suggest to do the following.
Compute the histogram of image 1 using hist1 = calcHist()
Compute the histogram of image 2 using hist2 = calcHist()
Calculate the difference vector diff = hist1 - hist2
Check if each bin of the hist of the subimage is less or equal than the corresponding bin in the hist of the bigger image
Thanks to Miki for the fix.
I will keep Amitay's as the accepted answer, as he absolutely lead me down the correct path. I wanted to also share my exact answer for anyone who finds this in the future.
As I stated in my question, I was looking for an efficient way to see if the RGB values of one image were a subset of the RGB values of another image.
I made a function to the following specification:
The Java code is as follows:
private boolean isSubset(Mat subset, Mat subMask, Mat superset) {
// Get unique set of pixels from both images
subset = getUniquePixels(subset, subMask);
superset = getUniquePixels(superset, null);
// See if the superset pixels encapsulate the subset pixels
// OR the unique pixels together
Mat subOrSuper = new Mat();
Core.bitwise_or(subset, superset, subOrSuper);
//See if the ORed matrix is equal to the superset
Mat notEqualMat = new Mat();
Core.compare(superset, subOrSuper, notEqualMat, Core.CMP_NE);
return Core.countNonZero(notEqualMat) == 0;
}
subset and superset are assumed to be CV_8UC3 matricies, while subMask is assumed to be CV_8UC1.
private Mat getUniquePixels(Mat img, Mat mask) {
if (mask == null) {
mask = new Mat();
}
// int bgrValue = (b << 16) + (g << 8) + r;
img.convertTo(img, CvType.CV_32FC3);
Vector<Mat> splitImg = new Vector<>();
Core.split(img, splitImg);
Mat flatImg = Mat.zeros(img.rows(), img.cols(), CvType.CV_32FC1);
Mat multiplier;
for (int i = 0; i < splitImg.size(); i++) {
multiplier = Mat.ones(img.rows(), img.cols(), CvType.CV_32FC1);
// set powTwo = to 2^i;
int powTwo = (1 << i);
// Set multiplier matrix equal to powTwo;
Core.multiply(multiplier, new Scalar(powTwo), multiplier);
// n<<i == n * 2^i;
// I'm shifting the RGB values into separate parts of the same 32bit
// integer.
Core.multiply(multiplier, splitImg.get(i), splitImg.get(i));
// Add the shifted RGB components together.
Core.add(flatImg, splitImg.get(i), flatImg);
}
// Create a histogram of the pixel values.
List<Mat> images = new ArrayList<>();
images.add(flatImg);
MatOfInt channels = new MatOfInt(0);
Mat hist = new Mat();
// 16777216 == 256*256*256
MatOfInt histSize = new MatOfInt(16777216);
MatOfFloat ranges = new MatOfFloat(0f, 16777216f);
Imgproc.calcHist(images, channels, mask, hist, histSize, ranges);
Mat uniquePixels = new Mat();
Core.inRange(hist, new Scalar(1), new Scalar(Float.MAX_VALUE), uniquePixels);
return uniquePixels;
}
Please feel free to ask questions, or point out problems!
I have an FFMPEG AVFrame in YUVJ420P and I want to convert it to a CVPixelBufferRef with CVPixelBufferCreateWithBytes. The reason I want to do this is to use AVFoundation to show/encode the frames.
I selected kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange and tried converting it since the AVFrame has the data in three planes
Y480 Cb240 Cr240. And according to what I've researched this matches the selected kCVPixelFormatType. By being biplanar I need to convert it into a buffer that contains Y480 and CbCr480 Interleaved.
I tried to create a buffer with 2 planes:
frame->data[0] on the first plane,
frame->data[1] and frame->data[2] interleaved on the second plane.
However, I'm getting return error -6661 (invalid a) from CVPixelBufferCreateWithBytes:
"Invalid function parameter. For example, out of range or the wrong type."
I don't have expertise on image processing at all, so any pointers to documentation that can get me started in the right approach to this problem are appreciated. My C skills aren't top of the line either so maybe I'm making a basic mistake here.
uint8_t **buffer = malloc(2*sizeof(int *));
buffer[0] = frame->data[0];
buffer[1] = malloc(frame->linesize[0]*sizeof(int));
for(int i = 0; i<frame->linesize[0]; i++){
if(i%2){
buffer[1][i]=frame->data[1][i/2];
}else{
buffer[1][i]=frame->data[2][i/2];
}
}
int ret = CVPixelBufferCreateWithBytes(NULL, frame->width, frame->height, kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, buffer, frame->linesize[0], NULL, 0, NULL, cvPixelBufferSample)
The frame is the AVFrame with the rawData from FFMPEG Decoding.
My C skills aren't top of the line either so maybe im making a basic mistake here.
You're making several:
You should be using CVPixelBufferCreateWithPlanarBytes(). I do not know if CVPixelBufferCreateWithBytes() can be used to create a planar video frame; if so, it will require a pointer to a "plane descriptor block" (I can't seem to find the struct in the docs).
frame->linesize[0] is the bytes per row, not the size of the whole image. The docs are unclear, but the usage is fairly unambiguous.
frame->linesize[0] refers to the Y plane; you care about the UV planes.
Where is sizeof(int) from?
You're passing in cvPixelBufferSample; you might mean &cvPixelBufferSample.
You're not passing in a release callback. The documentation does not say that you can pass NULL.
Try something like this:
size_t srcPlaneSize = frame->linesize[1]*frame->height;
size_t dstPlaneSize = srcPlaneSize *2;
uint8_t *dstPlane = malloc(dstPlaneSize);
void *planeBaseAddress[2] = { frame->data[0], dstPlane };
// This loop is very naive and assumes that the line sizes are the same.
// It also copies padding bytes.
assert(frame->linesize[1] == frame->linesize[2]);
for(size_t i = 0; i<srcPlaneSize; i++){
// These might be the wrong way round.
dstPlane[2*i ]=frame->data[2][i];
dstPlane[2*i+1]=frame->data[1][i];
}
// This assumes the width and height are even (it's 420 after all).
assert(!frame->width%2 && !frame->height%2);
size_t planeWidth[2] = {frame->width, frame->width/2};
size_t planeHeight[2] = {frame->height, frame->height/2};
// I'm not sure where you'd get this.
size_t planeBytesPerRow[2] = {frame->linesize[0], frame->linesize[1]*2};
int ret = CVPixelBufferCreateWithPlanarBytes(
NULL,
frame->width,
frame->height,
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
NULL,
0,
2,
planeBaseAddress,
planeWidth,
planeHeight,
planeBytesPerRow,
YOUR_RELEASE_CALLBACK,
YOUR_RELEASE_CALLBACK_CONTEXT,
NULL,
&cvPixelBufferSample);
Memory management is left as an exercise to the reader, but for test code you might get away with passing in NULL instead of a release callback.
I need to process images which i get from OpenCV.
I wrote so far:
IplImage* img=0;
img=cvLoadImage("paket2.tif");
api.SetRectangle(0,0,img->width, img->height);
api.SetImage((uchar*)img->imageData,img->width,img->height,img->depth/8,img->width*(img->depth/8));
//i tried also below line
//api.SetImage((uchar*)img->imageData,img->width,img->height,img->depth/8,img->widthStep);
int left,top,right,bottom;
left=0;top=0;right=0;bottom=0;
api.Recognize(NULL);
tesseract::ResultIterator *ri=api.GetIterator();
char * sonuc=(*ri).GetUTF8Text(tesseract::RIL_SYMBOL);
if((*ri).BoundingBox(tesseract::RIL_SYMBOL,&left,&top,&right,&bottom))
{printf("bb dogru\n");printf("%d,%d,%d,%d",left,top,right,bottom);}
printf("sonuc:%s",sonuc);
if i pass IplImage->widthStep to bytes perline, i have "wrong" boundingBox in left and right values and can not read all the text in the image.
if i pass IplImage->width*(IplImage->depth/8), boundingBox function returns false.
I hope you have some idea.
Thanks in advance.
Copy your submatrix to a new IplImage. Create a tesseract image header with the correct info(width, height, step). Link the tesseract data pointer to the iplImage data pointer.
I can't remember how to access tesseract pointer, but for opencv is image->data.ptr
This code here worked for me:
tesseract::TessBaseAPI tess;
tess.Init(argv[0], "eng", tesseract::OEM_DEFAULT);
cv::Mat image = cv::imread("...");
tess.SetImage((uchar*)image .data, image.size().width, image.size().height, image.channels(), image.step1());
tess.Recognize(0);
const char* out = tess.GetUTF8Text();
Is there a way to convert IplImage pointer to float pointer? Basically converting the imagedata to float.
Appreciate any help on this.
Use cvConvert(src,dst) where src is the source image and dst is the preallocated floating point image.
E.g.
dst = cvCreateImage(cvSize(src->width,src->height),IPL_DEPTH_32F,1);
cvConvert(src,dst);
// Original image gets loaded as IPL_DEPTH_8U
IplImage* colored = cvLoadImage("coins.jpg", CV_LOAD_IMAGE_UNCHANGED);
if (!colored)
{
printf("cvLoadImage failed!\n");
return;
}
// Allocate a new IPL_DEPTH_32F image with the same dimensions as the original
IplImage* img_32f = cvCreateImage(cvGetSize(colored),
IPL_DEPTH_32F,
colored->nChannels);
if (!img_32f)
{
printf("cvCreateImage failed!\n");
return;
}
cvConvertScale(colored, img_32f);
// quantization for 32bit. Without it, this img would not be displayed properly
cvScale(img_32f, img_32f, 1.0/255);
cvNamedWindow("test", CV_WINDOW_AUTOSIZE);
cvShowImage ("test", img_32f);
You can't convert the image to float by simply casting the pointer. You need to loop over every pixel and calculate the new value.
Note that most float image types assume a range of 0-1 so you need to divide each pixel by whatever you want the maximum to be.