I have a 3D matrix (very large, let call it L) and a 3D small one (very small, let call it S) and want to use OpenCV to find the closest pattern in L.
Does OpenCV do it for me? If yes, how I should use it?
Thanks.
What you need is the Point Cloud Library, which is an open source library to work with 3D data. I can tell you from my experience, that learning to use this library is very similar to learning OpenCV because many developers work for Willow Garage, the main sponsor of OpenCV.
If you go to the PCL tutorials you will find three useful sections to solve your problem:
1) finding features in your 3D point cloud, that you can later use for matching
2) 3D object recognition based on correspondence grouping
3) Point cloud registration using methods like iterative closest point, and feature matching
No, OpenCV doesn't have anything for this.
Do you have sparse pointcloud or just 3-dim matrix?
For 3-dim matrix you can use phase correlation using FFT. Good library is FFTW
OpenCV has added some neat tools to accomplish this kind of task
Surface Matching https://docs.opencv.org/master/d9/d25/group__surface__matching.html
Silhouette based 3D tracking https://docs.opencv.org/master/d4/dc4/group__rapid.html
Convolutional Neural Network https://docs.opencv.org/master/d9/d02/group__cnn__3dobj.html
Related
I am working on identifying an object by using Kinect sensor so to get x,y,z coordinates of the object.
I am trying to find the related information for this but could not able to find much. I have seen the videos as well but nobody is sharing the information or any sample code?
This is what I want to achieve https://www.youtube.com/watch?v=nw3yix3XomY
Proabably, few people may asked same question but as I am new to the Kinect and these libraries due to which I need little more guidance.
I read somewhere that object detection is not possible using Kinect v1. We need to use 3rd party libraries like open CV or point-clouds (pcl).
Can somebody help me that even by using third party libraries how exactly can I identify object via a Kinect sensor?
It will be really helpful.
Thank you.
As the author of the video you linked stated in the comment, following this PCL tutorial will help you. As you found out already, realizing this may not be possible using the standalone SDK. Relying on PCL will help you not reinvent the wheel.
The idea there is to:
Downsample the cloud to have less data to deal with in the next steps (this also reduces noise a bit).
Identify keypoints/features (i.e. points, areas, textures that remain somehow invariant to some transformations).
Compute the keypoint descriptors, mathematical representations of these features.
For each scene keypoint descriptor, find nearest neighbor into the model keypoints descriptor cloud and add it to the correspondences vector.
Perform clustering on the keypoints and detect the model in the scene.
The software in the tutorial needs the user to manually feed in the model and scene files. It doesn't do that on live feed, as the video you linked.
The process should be pretty similar though. I'm not sure how cpu-intensive the detection is, so it might require additional performance tweaking.
Once you have frame-by-frame detection in place, you could start thinking about actually tracking an object across the frames. But that's another topic.
Currently I'm trying to develop a program which can skeleton track
From the research I've been doing, I found out that the best way to tackle this problem is by using an RGB-Depth Camera such as Kinect.
Challenge : MS Kinect does not support skeleton tracking therefore I need to build a custom skeleton tracking
First Problem : How to Detect with a RGB-Depth camera?
What I found : Use a machine learning algorithm
Question: Is machine learning the only option to detect ? do I need the depth information for detection?
You can use the depth channel for the purpose of segmentation of the target silhouette, and then extract descriptive features and classify with them.
I'm not sure if what you need is to classify a target as human or animal, or whether you need to find what type of animal. If you need the former, features such as aspect ratio are very simple and good separators. If you need to classify which animal, it depends on the list of classes. Some cases are easier and some are harder.
Problem: I have a photo of an object (a manufactured part like the attached photo below), using my Andoird phone camera I want to verify if the object in camera preview matches to the template or not. (in other words, is it the same part as the template or not)
I can make the user to move the camera in order to have similar view of the template in camera preview as the template however there will be different noise level and/or lighting and maybe different background.
Question: What do you recommend me to use for solving this problem? I was thinking of Canny edge extraction and then matching the camera frames towards the canny edge extract from template? is this a good idea? if yes would you please tell me how can I implement this? any resources? samples? (I can do the Canny edge extraction but couldn't find a way to do the matching)
if Not a good idea then what do you recommend?
Things I have tried:
Feature Extract and Matching: I used few different extractor and matcher implementations from OpenCV and my app is working and drawing the detected feature points and matches, etc. however being a beginner with image processing I cannot make sense of the result and also how to know what is a match. any idea, help, good resources?
Template Matching: I used OpenCV template matching however the performance was horrible and I decided that this cannot be the solution.
I tried object recognition with my phone on your test image and the results were positive.
Detector used :ORB(Binary Detector).
Descriptor used :ORB.
Matching Technique : Brute-force matching .
Image Size 640x480.
I was able to detect around 500 feature points (number of keypoints is around sufficient but it might produce false matches when you have more images with similar looking objects.you need to refine your matching to avoid false matches).
Result of object recognition on two different scales.
Regarding you finding difficulties in understanding object recognition. What exactly did you not understand(Specific topic).
I recommend you to go thru the these two books
Learning OpenCV by By Adrian Kaehler, Gary Bradski
OpenCV 2 Computer Vision Application Programming Cookbook by by Robert Laganière(chapter 8 & 9).
Cheers!
from what I understand canny edge detection might not be an optimal solution. according to me after some basic pre-processing of the test image find its sift features and compare it with the sift features of the template. sift being really versatile should work here too.
you can also try opensurf feature they are faster than sift but i havent had an opportunity to work alot with them to be able to comment on its accuracy
I am trying to use the OpenCV's cascade classifier based on Histogram of Oriented Objects (HOGs) feature type -- such as the paper "Fast Human Detection Using a Cascade of Histograms of Oriented Gradients".
Searching in the web, I found that the Cascade Classificator of OpenCV only supports HAAR/LBP feature type (OpenCV Cascade Classification).
Is there a way to use HOGs with the OpenCV cascade classifier? What
do you suggest?
Is there a patch or another library that I can use?
Thanks in advance!
EDIT 1
I've kept my search, when I finally found in android-opencv that there is a trunk in Cascade Classifier which allows it to work with HOG features. But I don't know if it works...
Link: http://code.opencv.org/projects/opencv/repository/revisions/6853
EDIT 2
I have not tested the fork above because my problem has changed. But I found an interesting link which may be very useful in the future (when I come back to this problem).
This page contains the source code of the paper "Histograms of Oriented Gradients for
Human Detection". Also, more information. http://pascal.inrialpes.fr/soft/olt/
If you use OpenCV-Python, then you have the option of using some additional libraries, such as scikits.image, that have Histogram of Oriented Gradient built-ins.
I had to solve exactly this same problem a few months ago, and documented much of the work (including very basic Python implementations of HoG, plus GPU implementations of HoG using PyCUDA) at this project page. There is code available there. The GPU code should be reasonably easy to modify for use in C++.
It now seems to be available also in the non-python code. opencv_traincascade in 2.4.3 has a HOG featuretype option (which I did not try):
[-featureType <{HAAR(default), LBP, HOG}>]
Yes, you can use cv::CascadeClassifier with HOG features. To do this just load it with hogcascade_pedestrians.xml that you may find in opencv_src-dir/data/hogcascades.
The classifier works faster and its results are much better when it trained with hogcascade in compare with haarcascade...
I was going the K-means algorithm as a technique for clustering images, as part of my project in the area of Image Processing. In the process, I decided to code using OpenCV libraries. So, can somebody help me in coding the algorithm in C++ using OpenCV. I can manage well, if some code snippet is provided. I just started learning about the OpenCV environment. Thank you for reading this. I ll be grateful for your help. waiting for a reply!
Before you implement clustering (k-means or otherwise), you need to figure out how to compare one image vs the other. This is usually dependent on the type of images you have, what features you can extract (colors, edges, shapes etc.).
A simple way of comparing 2 images would be to use the SURF features and compare to other images' SURF features. The OpenCV samples should get you started on how to extract SURF.
Other approaches would be to extract edges, shapes, color histograms and compare to others. Once you have done so, clustering should be fairly straightforward.
Check mlem.cpp and cxmatrix.cpp.
If you're new to OpenCV, have a look at my website.. http://aishack.in/ I've put up some tutorials/articles on vision and vision algorithms. Hopefully they'll help!