Is there a way to directly copy previously allocated CUDA device data into an OpenCV GPU Mat? I would like to copy my data, previously initialized and filled by CUDA, into the OpenCV GPU mat. I would like to do so because I want solve a linear system of equations Ax = B by computing the inverse of the matrix A using OpenCV.
What I want to do is something like this:
float *dPtr;
gpuErrchk( cudaMalloc( (void**) &dPtr, sizeof(float) * height * width));
gpuErrchk( cudaMemset(dPtr, 0, sizeof(float) * height * width));
// modify dPtr in some way on the GPU
modify_dPtr();
// copy previously allocated and modified dPtr into OpenCV GPU mat?
// process GPU mat later - e.x. do a matrix inversion operation.
// extract raw pointer from GPU mat
EDIT:
The OpenCV documentation provides a GPU upload function.
Can the device pointer just be passed into that function as a parameter? If not, is there no other way to do such a data transfer? I don't want to copy data back and forth between the host and device memory, do my computation on a normal OpenCV Mat container, and copy back the results; my application is real-time. I am assuming that since there is no .at() function for a GPU Mat, as in the normal OpenCV Mat, there is no way to access the element at a particular location in the matrix? Also, does an explicit matrix inversion operation exist for the GPU Mat? The documentation does not provide a GPU Mat inv() function.
Just as talonmies posted in the comments, there is a constructor in the header of the GPU mat structure that allows the creation of a GPUMat header pointing to my previously allocated CUDA device data. This is what I had used:
cv::gpu::GpuMat dst(height, width, CV_32F, d_Ptr);
There is no need to figure out the step size because the constructor automatically evaluates it, given the width and height of the image.
Hopefully, when the support for OpenCV GPU functions becomes better, this post may be useful to someone.
EDIT
Another (probably) useful way is to utilize unified memory in CUDA. Pass the data into an OpenCV GPU and CPU mat, and continue operations from there.
Related
I use cv::cuda::remap instead of cv::remap to take advantage of CUDA acceleration to speed up video undistortion. Both versions of the program can run normally, but, while the camera matrix, distortion coefficients, map1 and map2 which come from cv::initUndistortRectifyMap() are all the same, the undistorted result image of the CPU version cv::remap is correct as follow:
but the CUDA version cv::cuda::remap results in a problem:
The code snippet for the CPU version is as follows:
cv::cuda::GpuMat gpuMat(m_height, m_width, CV_8UC4, (void *)dpFrame);
cv::Mat mat;
gpuMat.download(mat);
cv::remap(mat, mat, m_map1, m_map2, cv::INTER_LINEAR);
gpuMat.upload(mat);
GPU version:
cv::cuda::GpuMat gpuMat(m_height, m_width, CV_8UC4, (void *)dpFrame);
cv::cuda::remap(gpuMat, gpuMat, m_gpuMap1, m_gpuMap2, cv::INTER_LINEAR);
Among them, dpFrame is of type CUdeviceptr, m_map1 and m_map2 are calculated by cv::initUndistortRectifyMap, m_gpuMap1 and m_gpuMap2 are of type cv::cuda::GpuMat obtained by uploading m_map1 and m_map2 to GPU.
cv::remap and cv::cuda::remap are the same algorithm, why are their results different? I tried both versions of OpenCV 455 and 460, and neither works.
I'm stuck here and don't know how to go forward. Any suggestions are really appreciated. Thanks.
Okey, I also tried your code and got the similar results. I ended up with getting a correct result after a few tests.
My code simply flips an image with remap. Here is your code result to my input:
Code
cv::cuda::GpuMat gpuMat(m_height, m_width, CV_8UC4, (void *)dpFrame);
cv::cuda::remap(gpuMat, gpuMat, m_gpuMap1, m_gpuMap2, cv::INTER_LINEAR);
Input
Output
Then I just add a new declaration of cv::cuda::GpuMat and put it to output of resize function. Here is the code.
cv::cuda::GpuMat gpuMat(m_height, m_width, CV_8UC4, (void *)dpFrame);
cv::cuda::GpuMat gpuMat2;
cv::cuda::remap(gpuMat, gpuMat2, m_gpuMap1, m_gpuMap2, cv::INTER_LINEAR);
gpuMat2.download(mat);
New Output
I dont have a clear answer to the question why. Since we deal with gpu, it seems better to define different types for input and output of resize
I have a program that uses opencv and oclMat.
When I tried to run this my PC gets slow and sometimes get freezed.
I guess there are two much memory in GPU. So, my question is how can I release the memory allocated by opencv ocl mat. I execute 4 kernels. Something like this:
I create the oclmat and call the kernel and pass the matrices to the kernel.The result is a ocl mat which is used in the following kernel. M3,M4,M8,M9,M5,M10 are matrices that hold data inside the kernel. I am not using local memory(as the target device does not support local memory). SO, I am using the above mentioned ocl mat as data holder. All the temporary calculated data inside the kernel is stored in those matrix. They work the way a local memory would have worked here.I do not need them in the next kernel. SO I want to free them. What is the process to do that?
oclmat M1,M2,M3,M4
kernel1 (M1,M2,M3,M4)
oclmat M5,M6
kernel2(M4,M5,M6)
oclmat M7,M8,M9
kernel3(M6,M7,M8,M9)
oclmat M10,M11
kernel2(M9,M10,M11)
I am trying to implement computer vision algorithm on my NVidia GPU with openCV. I am using openCV 2.4 and I am currently writing very simple programs to get accustomed to openCV. I wrote a simple code of transposing a matrix and also to implementing canny edge detection on GPU. The program is running perfectly but I need to deallocate the memory in both the CPU and GPU. So I am posting my code below :
int main(int argc,char *argv[])
{
int k;
cv::Mat src;
cv::Mat dest;
cv::Mat dest_1;
cv::gpu::GpuMat im_source;
cv::gpu::GpuMat im_dest;
cv::gpu::GpuMat im_dest_1;
cv::gpu::Stream::Null;
k = cv::gpu :: getCudaEnabledDeviceCount();
printf("%d\n",k);
src = cv::imread("lena.jpg",0);
cv::imshow("lena_org",src);
im_source.upload(src);
cv::gpu::transpose(im_source,im_dest);
im_dest.download(dest);
cv::imshow("lena_trans",dest);
cv::gpu::Canny(im_source,im_dest_1,100,100,3,false);
im_dest_1.download(dest_1);
cv::imshow("lena_edge",dest_1);
cv::waitKey();
}
So from the code above I believe the memory is not getting freed in both the CPU and GPU. I was searching the internet a bit and I came across with cv::Mat::Release for cpu and cv::gpu::GpuMat::Release for the GPU side. But I am not getting how to use them or how I should use this functions in my code so that I could free bot my CPU and GPU memories. It would be very much helpful if someone could guide me through correct usage of the Release apis so that I could free the memory successfully. Thanks for all your support.
The destructor for cv::Mat objects automatically frees the memory, making calls to the release function you describe. At the level of your code, you shouldn't have to worry about that. Once the matrix leaves scope, it gets destroyed.
If you want to manually destroy your reference to the data, you can call, for example, src.release(). There is a good tutorial on memory management in the OpenCV documentation, available here
I am trying to extract features using OpenCV's HoG API, however I can't seem to find the API that allow me to do that.
What I am trying to do is to extract features using HoG from all my dataset (a set number of positive and negative images), then train my own SVM.
I peeked into HoG.cpp under OpenCV, and it didn't help. All the codes are buried within complexities and the need to cater for different hardwares (e.g. Intel's IPP)
My question is:
Is there any API from OpenCV that I can use to extract all those features / descriptors to be fed into a SVM ? If there's how can I use it to train my own SVM ?
If there isn't, are there any existing libraries out there, which could accomplish the same thing ?
So far, I am actually porting an existing library (http://hogprocessing.altervista.org/) from Processing (Java) to C++, but it's still very slow, with detection taking around at least 16 seconds
Has anyone else successfully to extract HoG features, how did you go around it ? And do you have any open source codes which I could use ?
Thanks in advance
You can use hog class in opencv as follows
HOGDescriptor hog;
vector<float> ders;
vector<Point> locs;
This function computes the hog features for you
hog.compute(grayImg, ders, Size(32, 32), Size(0, 0), locs);
The HOG features computed for grayImg are stored in ders vector to make it into a matrix, which can be used later for training.
Mat Hogfeat(ders.size(), 1, CV_32FC1);
for(int i=0;i<ders.size();i++)
Hogfeat.at<float>(i,0)=ders.at(i);
Now your HOG features are stored in Hogfeat matrix.
You can also set the window size, cell size and block size by using object hog as follows:
hog.blockSize = 16;
hog.cellSize = 4;
hog.blockStride = 8;
// This is for comparing the HOG features of two images without using any SVM
// (It is not an efficient way but useful when you want to compare only few or two images)
// Simple distance
// Consider you have two HOG feature vectors for two images Hogfeat1 and Hogfeat2 and those are same size.
double distance = 0;
for(int i = 0; i < Hogfeat.rows; i++)
distance += abs(Hogfeat.at<float>(i, 0) - Hogfeat.at<float>(i, 0));
if (distance < Threshold)
cout<<"Two images are of same class"<<endl;
else
cout<<"Two images are of different class"<<endl;
Hope it is useful :)
I also wrote the program of 2 hog feature comparing with the help of the above article.
And I apply this method to check ROI region changing or not.
Please refer to the page here.
source code and simple introduction
Here is GPU version as well.
cv::Mat temp;
gpu::GpuMat gpu_img, descriptors;
cv::gpu::HOGDescriptor gpu_hog(win_size, Size(16, 16), Size(8, 8), Size(8, 8), 9,
cv::gpu::HOGDescriptor::DEFAULT_WIN_SIGMA, 0.2, gamma_corr,
cv::gpu::HOGDescriptor::DEFAULT_NLEVELS);
gpu_img.upload(img);
gpu_hog.getDescriptors(gpu_img, win_stride, descriptors, cv::gpu::HOGDescriptor::DESCR_FORMAT_ROW_BY_ROW);
descriptors.download(temp);
OpenCV 3 provides some changes to the way GPU algorithms (i.e. CUDA) can be used by the user, see the Transition Guide - CUDA.
To update the answer from user3398689 to OpenCV 3, here is a snipped code:
#include <opencv2/core/cuda.hpp>
#include <opencv2/cudaimgproc.hpp>
[...]
/* Suppose you load an image in a cv::Mat variable called 'src' */
int img_width = 320;
int img_height = 240;
int block_size = 16;
int bin_number = 9;
cv::Ptr<cv::cuda::HOG> cuda_hog = cuda::HOG::create(Size(img_width, img_height),
Size(block_size, block_size),
Size(block_size/2, block_size/2),
Size(block_size/2, block_size/2),
bin_number);
/* The following commands are optional: default values applies */
cuda_hog->setDescriptorFormat(cuda::HOG::DESCR_FORMAT_COL_BY_COL);
cuda_hog->setGammaCorrection(true);
cuda_hog->setWinStride(Size(img_width_, img_height_));
cv::cuda::GpuMat image;
cv::cuda::GpuMat descriptor;
image.upload(src);
/* May not apply to you */
/* CUDA HOG works with intensity (1 channel) or BGRA (4 channels) images */
/* The next function call convert a standard BGR image to BGRA using the GPU */
cv::cuda::GpuMat image_alpha;
cuda::cvtColor(image, image_alpha, COLOR_BGR2BGRA, 4);
cuda_hog->compute(image_alpha, descriptor);
cv::Mat dst;
image_alpha.download(dst);
You can then use the descriptors in 'dst' variable as you prefer like, e.g., as suggested by G453.
I noticed that a new data structure cv::Matx was added to the new OpenCV version, intended for small matrices of known size at compilation time, for example
cv::Matx31f // matrix 3x1 of float type
Checking the documentation I saw that most of matrix operations are available, but still I don't see the advantages of using this new type instead of the old cv::Mat.
When should I use Matx instead of Mat?
Short answer: cv::Mat uses the heap to store its data, while cv::Matx uses the stack.
A cv::Mat uses dynamic memory allocation (on the heap). This is appropriate for big matrices (like images) and lets you do things like shallow copies of a matrix, which is the default behavior of cv::Mat.
However, for the small matrices that cv::Matx is designed for, heap allocation would be very expensive compared to doing the same thing on the stack. I have seen a block of math reduce processing time by over 75% by switching to using stack-allocated types (e.g. cv::Point and cv::Matx) instead of cv::Mat.
It's about memory management and not wasting (in some cases important) memory or just reservation of memory for an object you'll use later.
That's how I understand it – may be someone else can give a better explanation.
This is a late late answer, but it is still an interesting question!
dom's answer is quite accurate, and the heap/stack reference in user1460044's is also interesting.
From a practical point of view, I wouldn't use Matx (or Vec), except if it were completely necessary. The major advantages of Matx are
Using the stack (efficient![1])
Initialization.
The problem is, at the end you will have to move your Matx data to a Mat to do most of stuff, and so, you will be back at the heap again.
On the other hand, the "cool initialization" of a Matx can be done in a normal Mat:
// Matx initialization:
Matx31f A(1.f,2.f,3.f);
// Mat initialization:
Mat B = (Mat_<float>(3,1) << 1.f, 2.f, 3.f);
Also, there is a difference in initialization (beyond the heap/stack) stuff. If you try to put 5 values into the Matx31, it will crash (runtime exception), while calling the Mat_::operator<< with 5 values will only store the first three.
[1] Efficient if your program has to create lots of matrices of less than ~10 elements. In that case use Matx matrices.
There are 2 other reasons why I prefer Matx to Mat:
Readability: people reading the code can immediately see the size of the matrices, for example:
cv::Matx34d transform = ...;
It's clear that this is a 3x4 matrix, so it contains a 3D transformation of type (R,t), where R is a rotation matrix (as opposed to say, axis-angle).
Similarly, accessing an element is more natural with transform(i,j) vs transform.at<double>(i,j).
Easy debugging. Since the elements for Matx are allocated on the stack in an array of known length, IDEs or debuggers can display the entire contents nicely when stepping through the code.