ICP doesn't do much for my image registration of hands, why? - image-processing

I am trying to align a left and a right hand using ICP.
I tried to align the images as far as I could, using flipping and cropping the images. hands
To be able to use ICP (used this one: https://github.com/ClayFlannigan/icp/blob/master/icp.py), I made sure the matrices of both hands had the same shape.
I am performing the ICP as follows:
icptest = icp(c_cvimgl, c_cvimgr, init_pose=None, max_iterations=20, tolerance=0.001)
img_tr = np.dot(c_cvimgr, icptest[0][-1,-1])
But when displaying the hands nothing has changed. What am I doing wrong?hands after ICP
Eventually I would like to be able to align the left and right hand to a reference-hand to be able to check certain variables.

Related

Built in way to convert from screen coordinates to image coordinates?

I have an app where users can scale and position images in a number of ways. They can drag an entire layer of images around, scale that layer, drag around individual images inside the layer, and scale those individual images.
For some unrelated functionality, I need to generate the image coordinates that a user is pointing to on a given image (ie (0,0) for the top left & (width,height) for the bottom right), independent of how much it has been moved around and scaled. Is there a built in method for tranforming an absolute mouse position to it's relative position on an image (and vice versa) that takes into account any scaling/panning? I have started building my own methods for this tranformation but before I got too deep I wanted to see if it was already built in somewhere that I'm not seeing.
Konva doesn't have such methods yet. You have to implement them manually.
You can subscribe to this related issue: https://github.com/konvajs/konva/issues/303

How to separate the query and the train image from the Mat object returned from DrawMatches() method

I am trying to detect an object in a video. i am using SURF as feature detection and descriptor extractor, and BRUTFORCE as matcher. i tested my work with faces, i captured a picture of me and when i run the camera and direct it toward me, my face gets detected and a rectangle is drawn around it. i tried to make another test, i captured an image of my mouse and resized it, and when i run the cam, it is not getting detected
the problems i am facing are:
1-is the size of the query/object image matters in such cases,? i am asking this question because the image i captured of my self is bigger than the one of the mouse, and the face is getting detected and the mouse not.
2-regardless of which image i am using as a query/object iamge, how to display camera preview of only the train/scene image without the query/object image. i am asking this question because, what i am getting is something as shown in the below posted images, while what i want to do is something as it is shown here, i checked the code in that link, it is in C++ but i followed the same thing and also the tutorial uses 'drawMatches' method which has a peer in java which is Features2D.DrawMatches() and both of them returns a Mat object with the query/object image on the left side and the train/scene image on the right side as also shown in the image i posted below.
what i want to do is, to display on the the camera output without the query/object image, i want the area designated for the camera output is to show only the train/scene image captured from the camera.
please let me know how to solve this issues, i want to do something as shown in the tutorial i cited in the link.
1 - size matters but in your case, I think the most crucial problem is "textureness". SURF detect the interest points where the "texture gradient" is strong. In the case of your mouse, the gradient is mainly smooth, except aroud the logo (fujitsu), the button and at the border of the image. In the tutorial you point to, you notice it uses a very textured object to demonstrate the effect.
2 - to the best of my knowledge, there is fully automatic method to do what you want, but it can be done with a few steps. Basically, you must determine the surrounding box of your object then draw it. To draw, the easier is to use cv::rectangle but you can be more precise with four (or more....) cv::line. To determine the surrounding box, you can estimate the extreme points among the filtered matches.
Good luck!

How to add rain effect to a picture?

Given a picture, I would like to modify it to create the effect of rain on glass. What steps should I take to achieve this goal?
Suppose we want to add the effect of a single drop of water on a given point in an image, some pixels around that point should be modified in some way: how these pixels should be modified?
Simple way is to just make transparent image that is actually image overlay. That looks like common approach of water drop effects in gimp.
example:
http://natural-drops.deviantart.com/art/drop-of-rain-373710307
I think that in order to make optically correct image one need to have full 3D info of environment, because most optics equations that one needs to simulate correct image includes each object distance.

Should I use a sprite(image) or should I draw?

Below is a screenshot from a famous structure building game.
As you can see rods are attached together with ball joints. I'm trying to achieve something similar with Cosos2d on iPhone. I've got two questions here:
Should I draw those ball joints or should I just add them as sprites?
I guess the rods are not drawn bur are rather images. But considering that the length of the rods will change depending on need, how do I make rods with different lengths from just one image?
I am not sure what would be the best approach for one. I would imagine that either way would be fine, but it would be harder to achieve whatever visual affect you want through drawing them. I think the hardest part would be making sure that they are anchored to vertices in a grid system that has the dimension that you want.
As for (2) you can always scale your image.
CCSprite *rod = [CCSprite spriteWithFile:#"rod.png"];
rod.scaleY = 2.0; //or scaleX
Then rotate and change its anchor points as needed.
Again I imagine the most difficult part would be anchoring them and positioning them correctly, so that align properly with the vertices.

WebGL z-buffer artifacts?

We are working on a Three.js based WebGL project, and have trouble understanding how transparency is handled in WebGL. The image shows a doublesided surface drawn with alpha = 0.7, which behaves correctly on its right side. However closer to the middle strange artifacts appear, and on the left side the transparency does not seem to work at all.
http://emilaxelsson.se/sandbox/vis1/alpha.png
The problem can also be seen here:
http://emilaxelsson.se/sandbox/vis1/
Has anyone seen anything similar before? What could the reason be?
Your problem is that transparent objects needs to be sorted and rendered in a back-to-front order (if you try to change the opacity of your mesh from 0.7 (transparent) to 1.0 (opaque), you can see that the z-buffer works just fine).
See:
http://www.opengl.org/wiki/Transparency_Sorting
http://www.opengl.org/archives/resources/faq/technical/transparency.htm (15.050)
In your case it might be less trivial to solve, since I assume that you only have one mesh.
Edit: Just to summarize the discussion below. It is possible to achieve correct rendering of such a double-sided transparent mesh. To do this, you need to create 6 versions of the mesh, corresponding to 6 sides of a cube. Each version needs to be sorted in a back-to-front order based on the 'side of the cube' (front, back, left, right, top, bottom).
When rendering choose the correct mesh (based on the camera viewing direction) and render that single mesh.
The easy solution for your case (based on the picture you attached), without going to expensive sorting and multiple meshes, is to disable depth test and enable face culling. That produces acceptable results if you do not have any opaque objects in front of the mesh.

Resources