I am rendering a 3D Obj model in iOS. http://metalbyexample.com/picking-hit-testing/. I am following the above Tutorial written by WM to find the hit point in the 3D model. Using the above link I can find the intersecting point with 3D model. I want to find the triangular polygon where I touched and I want to find the UV coordinates in the Image texture. How can I do that..
Related
I am trying to draw a curved 3D line (doesn't matter if it made by a cylinder, rectangle, etc) on a ARSCNView (Renders using SceneKit).
I currently have something like this which renders lines using SCNCylinder but not sure how to curve them.
I tried using SCNShape from UIBezierPath to no success. Because I already have 3d point points not 2D points and using this method I can't have different Y axis (aka up) in ARKit.
I've written a little app using CoreMotion, AV and SceneKit to make a simple panorama. When you take a picture, it maps that onto a SK rectangle and places it in front of whatever CM direction the camera is facing. This is working fine, but...
I would like the user to be able to click a "done" button and turn the entire scene into a single image. I could then map that onto a sphere for future viewing rather than re-creating the entire set of objects. I don't need to stitch or anything like that, I want the individual images to remain separate rectangles, like photos glued to the inside of a ball.
I know about snapshot and tried using that with a really wide FOV, but that results in a fisheye view that does not map back properly (unless I'm doing it wrong). I assume there is some sort of transform I need to apply? Or perhaps there is an easier way to do this?
The key is "photos glued to the inside of a ball". You have a bunch of rectangles, suspended in space. Turning that into one image suitable for projection onto a sphere is a bit of work. You'll have to project each rectangle onto the sphere, and warp the image accordingly.
If you just want to reconstruct the scene for future viewing in SceneKit, use SCNScene's built in serialization, write(to:options:delegate:progressHandler:) and SCNScene(named:).
To compute the mapping of images onto a sphere, you'll need some coordinate conversion. For each image, convert the coordinates of the corners into spherical coordinates, with the origin at your point of view. Change the radius of each corner's coordinate to the radius of your sphere, and you now have the projected corners' locations on the sphere.
It's tempting to repeat this process for each pixel in the input rectangular image. But that will leave empty pixels in the spherical output image. So you'll work in reverse. For each pixel in the spherical output image (within the 4 corner points), compute the ray (trivially done, in spherical coordinates) from POV to that point. Convert that ray back to Cartesian coordinates, compute its intersection with the rectangular image's plane, and sample at that point in your input image. You'll want to do some pixel weighting, since your output image and input image will have different pixel dimensions.
OpenCV docs for solvePnp
In an augmented reality app, I detect the image in the scene so I know imagePoints, but the object I'm looking for (objectPoints) is a virtual marker just stored in memory to search for in the scene, so I don't know where it is in space. The book I'm reading(Mastering OpenCV with Practical Computer Vision Projects ) passes it as if the marker is a 1x1 matrix and it works fine, how? Doesn't solvePnP needs to know the size of the object and its projection so we know who much scale is applied ?
Assuming you're looking for a physical object, you should pass the 3D coordinates of the points on the model which are mapped (by projection) to the 2D points in the image. You can use any reference frame, and the results of the solvePnp will give you the position and orientation of the camera in that reference frame.
If you want to get the object position/orientation in camera space, you can then transform both by the inverse of the transform you got from solvePnp, so that the camera is moved to the origin.
For example, for a cube object of size 2x2x2, the visible corners may be something like: {-1,-1,-1},{1,-1,-1},{1,1,-1}.....
You have to pass the 3D coordinates of the real-world object that you want to map with the image. The scaling and rotation values will depend on the coordinate system that you use.
This is not as difficult as it sounds. See this blog post on head pose estimation. for more details with code.
In order to debug my shader, I am trying to display the just the front face of the cube map.
The cube map is a 125x750 image with the 6 faces on top of each other:
First, I load the cube map with GLKit:
_cubeTexture = [GLKTextureLoader cubeMapWithContentsOfFile:[[NSBundle mainBundle] pathForResource:#"uffizi_cube_map_ios" ofType:#"png"] options:kNilOptions error:&error];
Then I load it into the shader:
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_CUBE_MAP, self.cubeTexture.name);
glUniform1i( glGetUniformLocation( self.shaderProgram, "cube"), 0);
Then in the fragment shader:
gl_FragColor = textureCube(cube, vec3(-1.0+2.0*(gl_FragCoord.x/resolution.x),-1.0+2.0*(gl_FragCoord.y/resolution.y),1.0));
This displays a distorted image which seems to be a portion of the top of the cube map:
It shouldn't be distorted, and it should show the right face, not the top face.
I can't find any documentation that describes how the coordinates map to the cube, so what am I doing wrong?
It seems that there is a problem with cubeMapWithContentsOfFile. The cubeMapWithContentsOfFiles method (the one that takes an array of 6 images) works perfectly on the simulator. (There is a different issue with both methods on device).
To visualize how texture coordinates work for cube maps, picture a cube centered at the origin, with the faces at distance 1 from the origin, and with the specified cube map image on each face.
The texture coordinates can then be seen as direction vectors. Starting at the origin, the 3 components define a vector that can point in any direction. The ray defined by the vector will then intersect one of the 6 cube faces at a given point. This is the point where the corresponding cube map image is sampled during texturing.
For example, take a vector that points in a direction that is closest to the positive z axis. The ray defined by this vector intersects the top face of the cube. Therefore, the top (POSITIVE_Z) image of the cube map is sampled, at the point where the ray intersects the face.
Equivalent rules applies to all other directions. The face corresponding to the largest absolute value of one of the vector components determines which face is sampled, and the intersection point determines the position within the image.
The exact rules and formula can be found in the spec document. For example in the latest spec (OpenGL 4.5), see Section 8.13 "Cube Map Texture Selection", with the matching table 8.19. But as long as you understand that the texture coordinates define a direction vector, you have the main aspect covered.
How you determine the texture coordinates really depends on what you want to achieve. Common cases include:
Using normal vector as the cube map texture coordinates. This can for example be used for pre-computed lighting effects, where the content of the cube map image contains pre-computed lighting results for each possible normal direction.
Using the reflection vector as the cube map texture coordinate. This supports the implementation of environment mapping. The content of the cube map is a picture of the environment.
I am facing a problem while working in XNA 3.1 framework.
Actually I created a terrain model from DEM file. Terrain model is having a X,Y,Z coordinates. The terrain is created in the lower half of the screen. In xna domain 0,0 is the centre of the screen.
My problem is that, I want to identify the terrain coordinates when user click on the screen (using mouse). In screen view 0,0 is the left top most corner. I am getting difficulty in maping screen coordinates with my terrain coordinates and vice versa.
Is any one has faced the similar problem or let me suggest any solution for this any pointer would be helpful for me.
Thanks
Adee
What you're trying to do is called "picking", and there is an infinity of resources about that on the net. In short, it consists of:
generating a ray in 3D space that goes from front to back of your view frustum, using the mouse coordinates as X,Y
apply the reverse transformation of your view-projection matrix to get this vector in world space
test for intersection between this vector and each polygon of your terrain
use z-order to choose the closest intersecting polygon
See XNA Picking Tutorial for an in-depth explanation. Or just google "picking 3D".