I am using an OpenCV 1.0 based calibration toolbox to which I am making small additions. My additions require the use of the FFTW library (OpenCV has DFT functions but they aren't to my liking).
I have been trying to access the pixel values of an image and store those pixel values into a FFTW_complex type variable. I have tried a lot of the different suggestions (including openCV documentation) but I have been unable to do this properly.
The code below doesn't bring up any inconsistencies with variable types during the build or whilst debugging; however, the pixel values obtained and stored in "testarray" are a repetition of the values [13, 240, 173, 186]. Does anyone know how to access the pixel values and store them into FFTW compliant matrices/containers?
//.....................................//
//For image manipulation
IplImage* im1 = cvCreateImage(cvSize(400,400),IPL_DEPTH_8U,1);
int width = im1 -> width;
int height = im1 -> height;
int step = im1 -> widthStep/sizeof(uchar);
int fft_size = width *height;
//Setup pointers to images
uchar *im_data = (uchar *)im1->imageData;
//......................................//
fftw_complex testarray[subIM_size][subIM_size]; //size of complex FFTW array
im1= cvLoadImage(FILEname,0);
if (!im1)printf("Could not load image file");
//Load imagedata into FFTW arrays
for( i = 0 ; i < height ; i++ ) {
for( j = 0 ; j < width ; j++) {
testarray[i][j].re = double (im_data[i * step + j]);
testarray[i][j].im = 0.0;
}
}
I found out the problem. I had been using the wrong approach to access it.
This is what I used:
testarray[i][j].re = ((uchar*)(im1->imageData + i *im1->widthStep))[j]; //double (im_data[i * step + j]);
I am using C++ in Visual Studio 2008 and this is the way is use:
If we have a loop like that for going through the image:
for (int y = 0 ; y < height; y++){
for (int x = 0 ; x < width ; x++){
Then, the access to the fftw variable ( let's call it A) will be done as follows:
A [ height * y + x][0] = double (im_data[height * y + x]);
A [ height * y + x][1] = 0;
Hope it helps!
Antonio
Related
I have searched internet and stackoverflow thoroughly, but I haven't found answer to my question:
How can I get/set (both) RGB value of certain (given by x,y coordinates) pixel in OpenCV? What's important-I'm writing in C++, the image is stored in cv::Mat variable. I know there is an IplImage() operator, but IplImage is not very comfortable in use-as far as I know it comes from C API.
Yes, I'm aware that there was already this Pixel access in OpenCV 2.2 thread, but it was only about black and white bitmaps.
EDIT:
Thank you very much for all your answers. I see there are many ways to get/set RGB value of pixel. I got one more idea from my close friend-thanks Benny! It's very simple and effective. I think it's a matter of taste which one you choose.
Mat image;
(...)
Point3_<uchar>* p = image.ptr<Point3_<uchar> >(y,x);
And then you can read/write RGB values with:
p->x //B
p->y //G
p->z //R
Try the following:
cv::Mat image = ...do some stuff...;
image.at<cv::Vec3b>(y,x); gives you the RGB (it might be ordered as BGR) vector of type cv::Vec3b
image.at<cv::Vec3b>(y,x)[0] = newval[0];
image.at<cv::Vec3b>(y,x)[1] = newval[1];
image.at<cv::Vec3b>(y,x)[2] = newval[2];
The low-level way would be to access the matrix data directly. In an RGB image (which I believe OpenCV typically stores as BGR), and assuming your cv::Mat variable is called frame, you could get the blue value at location (x, y) (from the top left) this way:
frame.data[frame.channels()*(frame.cols*y + x)];
Likewise, to get B, G, and R:
uchar b = frame.data[frame.channels()*(frame.cols*y + x) + 0];
uchar g = frame.data[frame.channels()*(frame.cols*y + x) + 1];
uchar r = frame.data[frame.channels()*(frame.cols*y + x) + 2];
Note that this code assumes the stride is equal to the width of the image.
A piece of code is easier for people who have such problem. I share my code and you can use it directly. Please note that OpenCV store pixels as BGR.
cv::Mat vImage_;
if(src_)
{
cv::Vec3f vec_;
for(int i = 0; i < vHeight_; i++)
for(int j = 0; j < vWidth_; j++)
{
vec_ = cv::Vec3f((*src_)[0]/255.0, (*src_)[1]/255.0, (*src_)[2]/255.0);//Please note that OpenCV store pixels as BGR.
vImage_.at<cv::Vec3f>(vHeight_-1-i, j) = vec_;
++src_;
}
}
if(! vImage_.data ) // Check for invalid input
printf("failed to read image by OpenCV.");
else
{
cv::namedWindow( windowName_, CV_WINDOW_AUTOSIZE);
cv::imshow( windowName_, vImage_); // Show the image.
}
The current version allows the cv::Mat::at function to handle 3 dimensions. So for a Mat object m, m.at<uchar>(0,0,0) should work.
uchar * value = img2.data; //Pointer to the first pixel data ,it's return array in all values
int r = 2;
for (size_t i = 0; i < img2.cols* (img2.rows * img2.channels()); i++)
{
if (r > 2) r = 0;
if (r == 0) value[i] = 0;
if (r == 1)value[i] = 0;
if (r == 2)value[i] = 255;
r++;
}
const double pi = boost::math::constants::pi<double>();
cv::Mat distance2ellipse(cv::Mat image, cv::RotatedRect ellipse){
float distance = 2.0f;
float angle = ellipse.angle;
cv::Point ellipse_center = ellipse.center;
float major_axis = ellipse.size.width/2;
float minor_axis = ellipse.size.height/2;
cv::Point pixel;
float a,b,c,d;
for(int x = 0; x < image.cols; x++)
{
for(int y = 0; y < image.rows; y++)
{
auto u = cos(angle*pi/180)*(x-ellipse_center.x) + sin(angle*pi/180)*(y-ellipse_center.y);
auto v = -sin(angle*pi/180)*(x-ellipse_center.x) + cos(angle*pi/180)*(y-ellipse_center.y);
distance = (u/major_axis)*(u/major_axis) + (v/minor_axis)*(v/minor_axis);
if(distance<=1)
{
image.at<cv::Vec3b>(y,x)[1] = 255;
}
}
}
return image;
}
I have a huge image ( about 63000 x 63000 pixels = 3969 Megapixels )
what i have done so far is i decided to make "tiles" of (1024 x 1024) and do my calculations based on these tiles, resulting in an 62 x 62 image tile grid!
(this works out very well and has the advantage of making the image viewable with zoom-in and zoom out, only viewn tiles are downsized for example)
But what i need now are the contours from the huge image!
i use the OpenCV function "findContours" to detect contours on each
one of the tiles.
i have added some overlap in the tiles so i get
overlapping contours ( 1 pixel overlap )
i used the offset parameter
of "findContours" to shift the contours to the right position
into the "virtual total image"
Here are some screenshot's i made from a demo application
What I want is this:
Now my questions:
is it possible to stitch the contours, my worst case is a contour which covers the total image... is there some library that can do this?
is there a library which works on a compressed version of the total image ( like rle for example )
is there a way to make opencv findcontours work on 1 bit binary images ?
Here's the code used by findcontours:
// Surf2DTiledData ...a gobject based class used for 2d tile management and viewing..
Surf2DTiledData* td = (Surf2DTiledData*)in_td;
int nr_hor_tiles = surf2_d_tiled_data_get_nr_hor_tiles(td);
int nr_ver_tiles = surf2_d_tiled_data_get_nr_ver_tiles(td);
int tile_size_x = surf2_d_tiled_data_get_tile_width(td);
int tile_size_y = surf2_d_tiled_data_get_tile_height(td);
contouring_data_obj = surf2_d_tiled_data_get_ContouringData(td);
p_contours = contouring_data_obj->p_contours;
p_border_contours = contouring_data_obj->p_border_contours;
g_return_if_fail(p_border_contours != NULL);
g_return_if_fail(p_contours != NULL);
for (y = 0; y < nr_ver_tiles; y++){
int x;
for (x = 0; x < nr_hor_tiles; x++){
int idx = x + y*nr_hor_tiles;
CvMemStorage *mem = contouring_data_obj->contour_storage[idx];
CvMat _src;
CvSeq *contours = NULL;
uchar* dataBuffer = (uchar*)p_data[x][y];
// the idea is to have some extra space available for the overlap
// detection of contours!
// the extra space is needed for the algorithm to check for
// overlaps of contours later on!
#define VIRT_BORDER_EXTEND 2
int virtual_x = x * tile_size_x - VIRT_BORDER_EXTEND;
int virtual_y = y * tile_size_y - VIRT_BORDER_EXTEND;
int virtual_width = tile_size_x + VIRT_BORDER_EXTEND * 2;
int virtual_height = tile_size_y + VIRT_BORDER_EXTEND * 2;
int x_off = -VIRT_BORDER_EXTEND;
int y_off = -VIRT_BORDER_EXTEND;
if (virtual_x < 0) {
virtual_width += virtual_x;
virtual_x = 0;
x_off = 0;
}
if (virtual_y < 0) {
virtual_height += virtual_y;
virtual_y = 0;
y_off = 0;
}
if ((virtual_x + virtual_width) > (nr_hor_tiles*tile_size_x)) {
virtual_width = nr_hor_tiles*tile_size_x - virtual_x;
}
if ((virtual_y + virtual_height) > (nr_ver_tiles*tile_size_y)) {
virtual_height = nr_ver_tiles*tile_size_y - virtual_y;
}
CvMat* _roi_mat = get_roi_mat(td,
virtual_x, virtual_y,
virtual_width, virtual_height);
// Use either this:
//mem = cvCreateMemStorage(0);
if (_roi_mat){
// CV_LINK_RUNS => different algorithm!!!!
int tile_off_x = tile_size_x * x;
int tile_off_y = tile_size_y * y;
CvPoint contour_shift = cvPoint(x_off + tile_off_x, y_off + tile_off_y);
int n = cvFindContours(_roi_mat, mem, &contours, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, contour_shift);
cvReleaseMat(&_roi_mat);
p_contours[x][y] = contours;
}
//cvReleaseMemStorage(&mem);
}
}
later i used opengl to make textures out of the tiles and for every tile there is a quad !
the opencv contours are not drawn as this could be too slow for now, but i draw their bounding boxes... which are drawn in opengl too..
I'm trying to make a mobile fast version of Gaussian Blur image filter.
I've read other questions, like: Fast Gaussian blur on unsigned char image- ARM Neon Intrinsics- iOS Dev
For my purpose i need only a fixed size (7x7) fixed sigma (2) Gaussian filter.
So, before optimizing for ARM NEON, I'm implementing 1D Gaussian Kernel in C++, and comparing performance with OpenCV GaussianBlur() method directly in mobile environment (Android with NDK). This way it will result in a much simpler code to optimize.
However the result is that my implementation is 10 times slower then OpenCV4Android version. I've read that OpenCV4 Tegra have optimized GaussianBlur implementation, but I don't think that standard OpenCV4Android have those kind of optimizations, so why is my code so slow?
Here is my implementation (note: reflect101 is used for pixel reflection when applying filter near borders):
Mat myGaussianBlur(Mat src){
Mat dst(src.rows, src.cols, CV_8UC1);
Mat temp(src.rows, src.cols, CV_8UC1);
float sum, x1, y1;
// coefficients of 1D gaussian kernel with sigma = 2
double coeffs[] = {0.06475879783, 0.1209853623, 0.1760326634, 0.1994711402, 0.1760326634, 0.1209853623, 0.06475879783};
//Normalize coeffs
float coeffs_sum = 0.9230247873f;
for (int i = 0; i < 7; i++){
coeffs[i] /= coeffs_sum;
}
// filter vertically
for(int y = 0; y < src.rows; y++){
for(int x = 0; x < src.cols; x++){
sum = 0.0;
for(int i = -3; i <= 3; i++){
y1 = reflect101(src.rows, y - i);
sum += coeffs[i + 3]*src.at<uchar>(y1, x);
}
temp.at<uchar>(y,x) = sum;
}
}
// filter horizontally
for(int y = 0; y < src.rows; y++){
for(int x = 0; x < src.cols; x++){
sum = 0.0;
for(int i = -3; i <= 3; i++){
x1 = reflect101(src.rows, x - i);
sum += coeffs[i + 3]*temp.at<uchar>(y, x1);
}
dst.at<uchar>(y,x) = sum;
}
}
return dst;
}
A big part of the problem, here, is that the algorithm is overly precise, as #PaulR pointed out. It's usually best to keep your coefficient table no more precise than your data. In this case, since you appear to be processing uchar data, you would use roughly an 8-bit coefficient table.
Keeping these weights small will particularly matter in your NEON implementation because the narrower you have the arithmetic, the more lanes you can process at once.
Beyond that, the first major slowdown that stands out is that having the image edge reflection code within the main loop. That's going to make the bulk of the work less efficient because it will generally not need to do anything special in that case.
It might work out better if you use a special version of the loop near the edges, and then when you're safe from that you use a simplified inner loop that doesn't call that reflect101() function.
Second (more relevant to prototype code) is that it's possible to add the wings of the window together before applying the weighting function, because the table contains the same coefficients on both sides.
sum = src.at<uchar>(y1, x) * coeffs[3];
for(int i = -3; i < 0; i++) {
int tmp = src.at<uchar>(y + i, x) + src.at<uchar>(y - i, x);
sum += coeffs[i + 3] * tmp;
}
This saves you six multiplies per pixel, and it's a step towards some other optimisations around controlling overflow conditions.
Then there are a couple of other problems related to the memory system.
The two-pass approach is good in principle, because it saves you from performing a lot of recomputation. Unfortunately it can push the useful data out of L1 cache, which can make everything quite a lot slower. It also means that when you write the result out to memory, you're quantising the intermediate sum, which can reduce precision.
When you convert this code to NEON, one of the things you will want to focus on is trying to keep your working set inside the register file, but without discarding calculations before they've been fully utilised.
When people do use two passes, it's usual for the intermediate data to be transposed -- that is, a column of input becomes a row of output.
This is because the CPU will really not like fetching small amounts of data across multiple lines of the input image. It works out much more efficient (because of the way the cache works) if you collect together a bunch of horizontal pixels, and filter those. If the temporary buffer is transposed, then the second pass also collects together a bunch of horizontal points (which would vertical in the original orientation) and it transposes its output again so it comes out the right way.
If you optimise to keep your working set localised, then you might not need this transposition trick, but it's worth knowing about so that you can set yourself a healthy baseline performance. Unfortunately, localisation like this does force you to go back to the non-optimal memory fetches, but with the wider data types that penalty can be mitigated.
If this is specifically for 8 bit images then you really don't want floating point coefficients, especially not double precision. Also you don't want to use floats for x1, y1. You should just use integers for coordinates and you can use fixed point (i.e. integer) for the coefficients to keep all the filter arithmetic in the integer domain, e.g.
Mat myGaussianBlur(Mat src){
Mat dst(src.rows, src.cols, CV_8UC1);
Mat temp(src.rows, src.cols, CV_16UC1); // <<<
int sum, x1, y1; // <<<
// coefficients of 1D gaussian kernel with sigma = 2
double coeffs[] = {0.06475879783, 0.1209853623, 0.1760326634, 0.1994711402, 0.1760326634, 0.1209853623, 0.06475879783};
int coeffs_i[7] = { 0 }; // <<<
//Normalize coeffs
float coeffs_sum = 0.9230247873f;
for (int i = 0; i < 7; i++){
coeffs_i[i] = (int)(coeffs[i] / coeffs_sum * 256); // <<<
}
// filter vertically
for(int y = 0; y < src.rows; y++){
for(int x = 0; x < src.cols; x++){
sum = 0; // <<<
for(int i = -3; i <= 3; i++){
y1 = reflect101(src.rows, y - i);
sum += coeffs_i[i + 3]*src.at<uchar>(y1, x); // <<<
}
temp.at<uchar>(y,x) = sum;
}
}
// filter horizontally
for(int y = 0; y < src.rows; y++){
for(int x = 0; x < src.cols; x++){
sum = 0; // <<<
for(int i = -3; i <= 3; i++){
x1 = reflect101(src.rows, x - i);
sum += coeffs_i[i + 3]*temp.at<uchar>(y, x1); // <<<
}
dst.at<uchar>(y,x) = sum / (256 * 256); // <<<
}
}
return dst;
}
This is the code after implementing all the suggestions of #Paul R and #sh1, summarized as follows:
1) use only integer arithmetic (with precision to taste)
2) add the values of the pixels at the same distance from the mask center before applying the multiplications, to reduce the number of multiplications.
3) apply only horizontal filters to take advantage of the storage by rows of the matrices
4) separate cycles around the edges from those inside the image not to make unnecessary calls to reflection functions. I totally removed the functions of reflection, including them inside the loops along the edges.
5) In addition, as a personal observation, to improve rounding without calling a (slow) function "round" or "cvRound", I've added to both temporary and final pixel results 0.5f (= 32768 in integers precision) to reduce the error / difference compared to OpenCV.
Now the performance is much better from about 15 to about 6 times slower than OpenCV.
However, the resulting matrix is not perfectly identical to that obtained with the Gaussian Blur of OpenCV. This is not due to arithmetic length (sufficient) as well as removing the error remains. Note that this is a minimum difference, between 0 and 2 (in absolute value) of pixel intensity, between the matrices resulting from the two versions. Coefficient are the same used by OpenCV, obtained with getGaussianKernel with same size and sigma.
Mat myGaussianBlur(Mat src){
Mat dst(src.rows, src.cols, CV_8UC1);
Mat temp(src.rows, src.cols, CV_8UC1);
int sum;
int x1;
double coeffs[] = {0.070159, 0.131075, 0.190713, 0.216106, 0.190713, 0.131075, 0.070159};
int coeffs_i[7] = { 0 };
for (int i = 0; i < 7; i++){
coeffs_i[i] = (int)(coeffs[i] * 65536); //65536
}
// filter horizontally - inside the image
for(int y = 0; y < src.rows; y++){
uchar *ptr = src.ptr<uchar>(y);
for(int x = 3; x < (src.cols - 3); x++){
sum = ptr[x] * coeffs_i[3];
for(int i = -3; i < 0; i++){
int tmp = ptr[x+i] + ptr[x-i];
sum += coeffs_i[i + 3]*tmp;
}
temp.at<uchar>(y,x) = (sum + 32768) / 65536;
}
}
// filter horizontally - edges - needs reflect
for(int y = 0; y < src.rows; y++){
uchar *ptr = src.ptr<uchar>(y);
for(int x = 0; x <= 2; x++){
sum = 0;
for(int i = -3; i <= 3; i++){
x1 = x + i;
if(x1 < 0){
x1 = -x1;
}
sum += coeffs_i[i + 3]*ptr[x1];
}
temp.at<uchar>(y,x) = (sum + 32768) / 65536;
}
}
for(int y = 0; y < src.rows; y++){
uchar *ptr = src.ptr<uchar>(y);
for(int x = (src.cols - 3); x < src.cols; x++){
sum = 0;
for(int i = -3; i <= 3; i++){
x1 = x + i;
if(x1 >= src.cols){
x1 = 2*src.cols - x1 - 2;
}
sum += coeffs_i[i + 3]*ptr[x1];
}
temp.at<uchar>(y,x) = (sum + 32768) / 65536;
}
}
// transpose to apply again horizontal filter - better cache data locality
transpose(temp, temp);
// filter horizontally - inside the image
for(int y = 0; y < src.rows; y++){
uchar *ptr = temp.ptr<uchar>(y);
for(int x = 3; x < (src.cols - 3); x++){
sum = ptr[x] * coeffs_i[3];
for(int i = -3; i < 0; i++){
int tmp = ptr[x+i] + ptr[x-i];
sum += coeffs_i[i + 3]*tmp;
}
dst.at<uchar>(y,x) = (sum + 32768) / 65536;
}
}
// filter horizontally - edges - needs reflect
for(int y = 0; y < src.rows; y++){
uchar *ptr = temp.ptr<uchar>(y);
for(int x = 0; x <= 2; x++){
sum = 0;
for(int i = -3; i <= 3; i++){
x1 = x + i;
if(x1 < 0){
x1 = -x1;
}
sum += coeffs_i[i + 3]*ptr[x1];
}
dst.at<uchar>(y,x) = (sum + 32768) / 65536;
}
}
for(int y = 0; y < src.rows; y++){
uchar *ptr = temp.ptr<uchar>(y);
for(int x = (src.cols - 3); x < src.cols; x++){
sum = 0;
for(int i = -3; i <= 3; i++){
x1 = x + i;
if(x1 >= src.cols){
x1 = 2*src.cols - x1 - 2;
}
sum += coeffs_i[i + 3]*ptr[x1];
}
dst.at<uchar>(y,x) = (sum + 32768) / 65536;
}
}
transpose(dst, dst);
return dst;
}
According to Google document, on Android device, using float/double is twice slower than using int/uchar.
You may find some solutions to speed up your C++ code on this Android documents.
https://developer.android.com/training/articles/perf-tips
I am trying to get and set pixels of a gray scale image by using emgu Cv with C#.
If I use a large image size this error message occurs: "Index was outside the bounds of the array."
If I use an image 200x200 or less then there is no error but I don't understand why.
Following is my code:
Image<Gray , byte> grayImage;
--------------------------------------------------------------------
for (int v = 0; v < grayImage.Height; v++)
{
for (int u = 0; u < grayImage.Width; u++)
{
byte a = grayImage.Data[u , v , 0]; //Get Pixel Color | fast way
byte b = (byte)(myHist[a] * (K - 1) / M);
grayImage.Data[u , v , 0] = b; //Set Pixel Color | fast way
}
}
--------------------------------------------------------------------
http://i306.photobucket.com/albums/nn262/neji1909/9-6-25565-10-39.png
Please help me and sorry I am not good at English.
you are not indexing by (x,y) but by (row, col) - inverted. When you used 200x200 image it was the same whether you used width or height.
you could do that by using pointers (much faster) because if you are using indexing EmguCV internally uses calls to opencv for an every pixel.
so:
byte* ptr = (byte*)image.MIplImage.imageData;
int stride = image.MIplImage.widthStep;
int width = image.Width;
int height = image.Height;
for(int j = 0; j < height; j++)
{
for(int i = 0; i < width; i++)
{
ptr[i] = (byte)(myHist[a] * (K - 1) / M);
}
ptr += stride;
}
That's because the x and y are inverted in the Data array. You should change your code this way (invert u and v):
for (int v = 0; v < grayImage.Height; v++)
{
for (int u = 0; u < grayImage.Width; u++)
{
byte a = grayImage.Data[v , u , 0]; //Get Pixel Color | fast way
byte b = (byte)(myHist[a] * (K - 1) / M);
grayImage.Data[v , u , 0] = b; //Set Pixel Color | fast way
}
}
See also Iterate over pixels of an image with emgu cv
So i have a program written already that captures the image from a webcam, into a vector called pBuffer.
I can easily acess the RGB pixel information of each pixel, simply by
pBuffer[i]=R;pBuffer[i+1]=G;Buffer[i+2]=B.
No problem in here.
The next step is now create an IplImage* img, and fill it in with the information of the pBuffer...some sort of SetPixel.
There is a SetPixel Function on the web, that is :
(((uchar*)(image>imageData + image>widthStep*(y))))[x * image>nChannels + channel] = (uchar)value;
where the value is the pBuffer information, x and y the pixel coordinates.However i simply cannot put this to work. Any ideas?? I am working with C++.
What you are trying to do you can do like this (assuming width and height are the image dimensions):
CvSize size;
size.height = height;
size.width = width;
IplImage* ipl_image_p = cvCreateImage(size, IPL_DEPTH_8U, 3);
for (int y = 0; y < height; ++y)
for (int x = 0; x < width; ++x)
for (int channel = 0; channel < 3; ++channel)
*(ipl_image_p->imageData + ipl_image_p->widthStep * y + x * ipl_image_p->nChannels + channel) = pBuffer[x*y*3+channel];
However, you don't have to copy the data. You can also use your image data by IplImage (assuming pBuffer is of type char*, otherwise you need possibly to cast it):
CvSize size;
size.height = height ;
size.width = width;
IplImage* ipl_image_p = cvCreateImageHeader(size, IPL_DEPTH_8U, 3);
ipl_image_p->imageData = pBuffer;
ipl_image_p->imageDataOrigin = ipl_image_p->imageData;