I am using Drake to implement a visual servoing scheme. For that I need to process color images and extract features.
I am using the ManipulationStation class, and I have already published the color images of the rgbd_sensor objects to LCM channels (in c++). Now I want to process the images during the simulation.
I know that it would be best to process images internally (without using the ImageWriter), and for that, I cannot use the image_array_t on LCM channels or ImageRgbaU8, I have to convert images to an Eigen or OpenCV type.
I will then use feature extraction functions in these libraries. These features will be used in a control law.
Do you have any examples on how to write a system or code in c++ that could convert my Drake images to OpenCV or Eigen types? What is the best way to do it?
Thanks in advance for the help!
Arnaud
There's no current example converting the drake::Image to Eigen or OpenCV but it should be pretty straightforward.
What you should do is create a system akin to ImageWriter. It takes an image as an input and does whatever processing you need on the image. You connect its input port to the RgbdSensor's output port (again, like ImageWriter.) The only difference, instead of taking the image and converting it to an lcm data type, you convert it to your OpenCV or Eigen type and apply your logic to that.
If you look, ImageWriter declares a periodic publish event. Change that to (probably) a discrete update event (still periodic, I'd imagine) and then change ImageWriter::WriteImage callback into your "Image-to-OpenCV" callback.
Finally, you could also declare an output port that would make your converted image to other systems.
Related
I'm looking for an implementation of thickening and thinning of 1-bit color bitmaps such that the topology, or connectivity, of the images is not altered. That is, if you do thickening just using dilation, two blobs that are near to each other may be joined into a single blob: the type of thickening operation I am looking for would never do this.
Does OpenCV have this out of the box or is there a way to implement it with what OpenCV does offer?
I believe this operation is implemented in Leptonica but would rather not add an additional dependency beyond OpenCV if I do not need to.
I'm using OpenCV from C++ if that matters.
I'm wondering if it is possible to use the OpenCV framework to recognise a building?
For example, if I store an image of a building, is it possible to use OpenCV to detect this building through the iPhone camera?
Thanks!
Detecting known objects such as your building in an image can be done using the features2d module in OpenCV.
It works by detecting key points in the known image and computing a set of descriptors for these that can be compared to the key points and descriptors computed from the unknown scene image by a process known as matching.
The find_obj.py demo in the samples/python2 folder of OpenCV shows how to detect a known object in an image.
There is also a tutorial in the user guide, see http://docs.opencv.org/doc/user_guide/ug_features2d.html
Note that some of the algorithms often used (e.g. SURF and SIFT) are not free, and need to be licensed separately if you use them.
Is possible, but, you have a long road to go.
One way to do this: use visual keypoints to recognise objects.
OpenCV Sift Documentation
I want to use Hadoop to process segmentation on large scale images (Pleiades image tif format)
The idea is to split the image into tiles and distribute them to each node.
The map task will be the segmentation process. I developped the segmentation algorithm using the OTB library written in C++.
I created an otb application which can be launched since a java program by giving the path of the tile and it returned the segmented tile.
I really don't know how to implement the split mechanism, I have to custom the fileInputFormat class and the RecorReader class but with what types of input keys and input values.
Actually I have to get the path to the tile to segment...
If someone has a suggestion ?
Best Regards,
Solution strategy may depend on amount of images and variety of their sizes.
If you have a lot of images (much more then amount of free mapper slots in your cluster) you probably may consider process each image inside of mapper with StreamInputFormat and use your segmentation algorithm. Otherwise you may need to implement your own InputFormat that will form correct InputSplits.
I am trying to implement an algorithm for a system which the camera get 1000fps, and I need to get the value of each pixel in all images and do the different calculation on the evolution of pixel[i][j] in N number of images, for all the pixels in the images. I have the (unsigned char *ptr) I want to transfer them to the GPU and start implementing the algorithm.but I am not sure what would be the best option for realtime processing.
my system:
CPU Intel Xeon x5660 2.8Ghz(2 processors)
GPU NVIDIA Quadro 5000
I got the following questions:
I do I need to add any Image Processing library addition to CUDA? if yes what do you suggest?
can I create a matrix for pixel[i,j] containing values for images [1:n] for each pixel in the image size? for example for 1000 images with 200x200 size I will end up with 40000 matrix each
containing 1000 values for one pixel? Does CUDA gives me some options like OpenCV to have a Matrices? or Vector?
1 - Do I need to add any Image Processing library addition to CUDA?
Apples and oranges. Each has a different purpose. An image processing library like OpenCV offers a lot more than simple accelerated matrix computations. Maybe you don't need OpenCV to do the processing in this project as you seem to rather use CUDA directly. But you could still have OpenCV around to make it easier to load and write different image formats from the disk.
2 - Does CUDA gives me some options like OpenCV to have a Matrices?
Absolutely. Some time ago I wrote a simple (educational) application that used OpenCV to load an image from the disk and use CUDA to convert it to its grayscale version. The project is named cuda-grayscale. I haven't tested it with CUDA 4.x but the code shows how to do the basic when combining OpenCV and CUDA.
It sounds like you will have 40000 independent calculations, where each calculation works only within one (temporal) pixel. If so, this should be a good task for the GPU. Your 352 core Fermi GPU should be able to beat your 12 hyperthreaded Xeon cores.
Is the algorithm you plan to run a common operation? It sounds like it might not be, in which case you will likely have to write your own kernels.
Yes, you can have arrays of elements of any type in CUDA.
Having this being a "streaming oriented" approach is good for a GPU implementation in that it maximizes number of calculations as compared to transfers over the PCIe bus. It it might also introduce difficulties in that, if you want to process the 1000 values for a given pixel in a specific order (oldest to newest, for instance), you will probably want to avoid continuously shifting all the frames in memory (to make room for the newest frame). It will slightly complicate your addressing of the pixel values, but the best approach, to avoid shifting the frames, may be to overwrite the oldest frame with the newest frame each time a new frame is added. That way, you end up with a "stack of frames" that is fairly well ordered but has a discontinuity between old and new frames somewhere within it.
I do I need to add any Image Processing library addition to CUDA ???
if yes what do you suggest?
Disclosure: My company develop & market CUVILib
There are very few options when it comes to GPU Accelerated Imaging libraries which also offer general-purpose functionality. CUVILib is one of those options which offers the following, very suited for your specific needs:
CuviImage object which holds your image data and image as a 2D matrix
You can write your own GPU function and use CuviImage as a 2D GPU matrix.
CUVILib already provides a rich set of Imaging functionality like Color Operations, Image Statistics, Feature detection, Motion estimation, FFT, Image Transforms etc so chances are that you will find your desired functionality.
As for the question of whether GPUs are suited for your application: Yes! Imaging is one of those domains which are ideal for parallel computation.
Links:
CUVILib: http://www.cuvilib.com
TunaCode: http://www.tunacode.com
I am interested in studying some image processing. I imagine matlab is the best way to go about that but right now I don't have access to matlab. I tried octave but for some reason it can't even load a png, bmp or anything other than 1 specific format. R doesn't seem to be the key here either.
What is the language of choice here? Perl?
Also can anyone point me to any other good tutorials that I may have missed on image processing?
Opencv is an excellent image processing library. Although written in C it comes with some high level tools to display images handle image files, mouse events etc so you can experiment without writing a lot of windows code.
It also works with python, although I haven't used it with the PIL.
If you are interested in how the algorithms work then implementing them yourself using python and numpy for the matrix ops is easy.
I guess it depends on what you want to do. Matlab certainly is a high end choice, but for a lot of things the image modules of general purpose programming languages do the trick.
I did some pixel mangling and image processing with PIL, the python image library. It is perfectly sufficient for processing single RGB images of reasonable size (say, what a consumer digital camera delivers). It can handle alpha channels, has some filters, more or less quick methods of accessing the pixel information - and it is python, a very straightforward and readable language.
The recommended language in my computer vision class was Ch with the OpenCV library. Ch is basically an interpreted version of C, the syntax is quite similar but has a few nice features, like treating arrays as matrices. OpenCV will house pretty much any image processing function you could need.
I think any free programming environments will do basic image processing well. If speed is not an issue, Processing will work fine and you can easily extend your code to Java in the future.
Have a look at Adobe Pixel Bender. It's really fun to play with.