I'm struggling with a texture-baking process with 3DSmax software. I have a white 3D mesh with 2 image textures. I'm trying to get a diffusemap (see target_diffuse_map.jpg). To do this, I exectue the following steps:
1) Affect image-texture1 and image-texture2 to face1 and face2 of the objet.
2) Clone the object to get the white colors when baking texture.
3) unwrap UVM.
4) Rendering Texture to obtain the diffuse map.
5) Projection of the texture + white colors on the cloned object.
Please, find these steps on this small video I made: https://drive.google.com/file/d/1h4v2CrL8OCLwdeVtLmpQwD250cawgJpi/view
I obtain a bad sampled and weird diffuse map (please see obtained_diffuse_map.jpg). What I want is target_diffuse_map.jpg.
I'm I forgetting some steps?
Thank you for your help.
You need to either:
Add a small amount of "Push" in the Projection Modifier
Uncheck "Use Cage" in the Projection Options dialog, while setting a very small value for the offset
Projection Mapping works by casting rays from points on the cage towards corresponding model points on your mesh. You did not push the cage out at all, therefore rays are not well defined; rays are cast from a point toward a direction which is the exact same point. This causes numerical errors and z-fighting. The there needs to be some time amount of push so the "from" and "to" points of each ray are different giving them a well-defined direction to travel.
The second option, instead of using the cage defined in the projection modifier, is to use the offset method (you probably still need to apply projection modifier though). This method defines each rays as starting from a point defined by taking the model point of the mesh and moving outward by a fixed offset amount in the direction of the normal. The advantage is that for curved objects with large polygons, it produces less distortion because the system uses the smoothed shading normal at each point. The disadvantage you can't have different cage distances at different points of the model, for better control. Use this method for round wooden barrels and other simplistic objects with large, smooth curves.
Also, your situation is made difficult by having different parts of the model very close to each other (touching) and embedded within each other - namely how the mouth of the bottle is inside the cap and the cap it touching the base. For this case, it might make sense to break the objects apart after you have the overall UV mapping, run projection mapping separately on each one separately, and then combine the maps back together in an image editor.
Related
I'm using python kivy to render meshes with opengl onto a canvas. I want to return vertex data from the fragment shader so i can build a collider (to use on my cpu event listeners after doing projection and model view transforms). I can replicate the matrix multiplications on the cpu (i guess that's the easy way out), but then i would have to do the same calculations twice (not good).
The only way I can think of doing this (after some browsing) is to imprint an object id onto my rendered mesh alpha channel (wouldn't affect much if i'd keep data coding near value 1 for alpha ). And create some kind of 'color picker' on the cpu side to decode it (I'm guessing that's not hard to do using kivy).
Anyone has a better idea to deal with this? Or a better approach?
First criterion here is: do you need collision for picking or for physics simulation?
If it is for physics: you almost never want the same mesh for rendering and for physics collisions. Typically, you use a very rough approximation for the physics shape, nearly always a convex shape, or a union of convex shapes. (Colliding arbitrary concave meshes is something that no physics engine can do well, and if they attempt it at all, performance will be poor.)
If it is for the purpose of picking an object with a mouse-click: you can go two different ways for this:
You replicate the geometry on the CPU, and use the mouse-location plus camera-view to create a ray that intersects this geometry, to see what is hit first.
After rendering your scene, you read back a single pixel from the depth buffer. (The pixel that your mouse is over.) With the depth value you get back, plus camera info, you can reconstruct a corresponding 3D position in your world. Once you have a 3D location, you can query your world to see which object is the closest to this point, and you will have your hit.
I'm currently working on an augmented reality application using a medical imaging program called 3DSlicer. My application runs as a module within the Slicer environment and is meant to provide the tools necessary to use an external tracking system to augment a camera feed displayed within Slicer.
Currently, everything is configured properly so that all that I have left to do is automate the calculation of the camera's extrinsic matrix, which I decided to do using OpenCV's solvePnP() function. Unfortunately this has been giving me some difficulty as I am not acquiring the correct results.
My tracking system is configured as follows:
The optical tracker is mounted in such a way that the entire scene can be viewed.
Tracked markers are rigidly attached to a pointer tool, the camera, and a model that we have acquired a virtual representation for.
The pointer tool's tip was registered using a pivot calibration. This means that any values recorded using the pointer indicate the position of the pointer's tip.
Both the model and the pointer have 3D virtual representations that augment a live video feed as seen below.
The pointer and camera (Referred to as C from hereon) markers each return a homogeneous transform that describes their position relative to the marker attached to the model (Referred to as M from hereon). The model's marker, being the origin, does not return any transformation.
I obtained two sets of points, one 2D and one 3D. The 2D points are the coordinates of a chessboard's corners in pixel coordinates while the 3D points are the corresponding world coordinates of those same corners relative to M. These were recorded using openCV's detectChessboardCorners() function for the 2 dimensional points and the pointer for the 3 dimensional. I then transformed the 3D points from M space to C space by multiplying them by C inverse. This was done as the solvePnP() function requires that 3D points be described relative to the world coordinate system of the camera, which in this case is C, not M.
Once all of this was done, I passed in the point sets into solvePnp(). The transformation I got was completely incorrect, though. I am honestly at a loss for what I did wrong. Adding to my confusion is the fact that OpenCV uses a different coordinate format from OpenGL, which is what 3DSlicer is based on. If anyone can provide some assistance in this matter I would be exceptionally grateful.
Also if anything is unclear, please don't hesitate to ask. This is a pretty big project so it was hard for me to distill everything to just the issue at hand. I'm wholly expecting that things might get a little confusing for anyone reading this.
Thank you!
UPDATE #1: It turns out I'm a giant idiot. I recorded colinear points only because I was too impatient to record the entire checkerboard. Of course this meant that there were nearly infinite solutions to the least squares regression as I only locked the solution to 2 dimensions! My values are much closer to my ground truth now, and in fact the rotational columns seem correct except that they're all completely out of order. I'm not sure what could cause that, but it seems that my rotation matrix was mirrored across the center column. In addition to that, my translation components are negative when they should be positive, although their magnitudes seem to be correct. So now I've basically got all the right values in all the wrong order.
Mirror/rotational ambiguity.
You basically need to reorient your coordinate frames by imposing the constraints that (1) the scene is in front of the camera and (2) the checkerboard axes are oriented as you expect them to be. This boils down to multiplying your calibrated transform for an appropriate ("hand-built") rotation and/or mirroring.
The basic problems is that the calibration target you are using - even when all the corners are seen, has at least a 180^ deg rotational ambiguity unless color information is used. If some corners are missed things can get even weirder.
You can often use prior info about the camera orientation w.r.t. the scene to resolve this kind of ambiguities, as I was suggesting above. However, in more dynamical situation, of if a further degree of automation is needed in situations in which the target may be only partially visible, you'd be much better off using a target in which each small chunk of corners can be individually identified. My favorite is Matsunaga and Kanatani's "2D barcode" one, which uses sequences of square lengths with unique crossratios. See the paper here.
This is an image from apple's documentation. They show a transform from a cube to sphere and also to some random geometry.
Only a few lines lower they state:
A morpher and its target geometries may be loaded from a scene file or
created programmatically. The base geometry and all target geometries
must be topologically identical—that is, they must contain the same
number and structural arrangement of vertices.
Could someone explain this paragraph because apparently I don't understand it.
Since a sphere will never have the same structural arrangement of vertices as cube(at least I think so), it is impossible to make transformation. But hey, we all see it in the picture. I also tried do to the transformation and I don't get the expected results. So how do you go from sphere to cube or vice versa?
"Topologically identical" means that the relationships between vertices in a mesh must be preserved, but their locations in space can change. Here's an example of that in 2D:
These two meshes have the same eight vertices, connected to each other in the same ways, but their positions (and thus the shape they form) differ.
To do the same in 3D with SceneKit, you need custom vertex data — the primitive shapes that SceneKit can generate for you (like SCNSphere, SCNBox, and whatnot) all have different topologies, so they can't be used as morpher targets.
If you want to morph a box into a sphere, you'll need to generate your own box and sphere with identical topology. The "some random shape" in Apple's illustration is a hint at how you might do that — it appears to be one of the variants of a superellipsoid. If you use the equations in that Wikipedia page you can generate a set of points that can be either on a sphere or on a cube depending on other parameters. Vary those parameters to generate a couple of meshes, create SCNGeometry from those meshes, and you've got valid SCNMorpher targets.
You can see a simpler example of morphing in Apple's SceneKit WWDC 2014 Slides sample app.
You can't presume the locations of each vertex in the given images; the cube doesn't neccesarily have eight and the left-most doesn't guaruntee to have 6.
Admittedly, I've not played with SCNMorpher but from that description I imagine it will interpolate on a per-vertex basis (so they will have to match up).
If it helps, picture the sphere as having a lot of 'dots' spread equally along its surface and those are pushed or squeezed to make the other surfaces
My question maybe a bit too broad but i am going for the concept. How can i create surface as they did in "Cham Cham" app
https://itunes.apple.com/il/app/cham-cham/id760567889?mt=8.
I got most of the stuff done in the app but the surface change with user touch is quite different. You can change its altitude and it grows and shrinks. How this can be done using sprite kit what is the concept behind that can anyone there explain it a bit.
Thanks
Here comes the answer from Cham Cham developers :)
Let me split the explanation into different parts:
Note: As the project started quite a while ago, it is implemented using pure OpenGL. The SpiteKit implementation might differ, but you just need to map the idea over to it.
Defining the ground
The ground is represented by a set of points, which are interpolated over using Hermite Spline. Basically, the game uses a bunch of points defining the surface, and a set of points between each control one, like the below:
The red dots are control points, and eveyrthing in between is computed used the metioned Hermite interpolation. The green points in the middle have nothing to do with it, but make the whole thing look like boobs :)
You can choose an arbitrary amount of steps to make your boobs look as smooth as possible, but this is more to do with performance.
Controlling the shape
All you need to do is to allow the user to move the control points (or some of them, like in Cham Cham; you can define which range every point could move in etc). Recomputing the interpolated values will yield you an changed shape, which remains smooth at all times (given you have picked enough intermediate points).
Texturing the thing
Again, it is up to you how would you apply the texture. In Cham Cham, we use one big texture to hold the background image and recompute the texture coordinates at every shape change. You could try a more sophisticated algorithm, like squeezing the texture or whatever you found appropriate.
As for the surface texture (the one that covers the ground – grass, ice, sand etc) – you can just use the thing called Triangle Strips, with "bottom" vertices sitting at every interpolated point of the surface and "top" vertices raised over (by offsetting them against "bottom" ones in the direction of the normal to that point).
Rendering it
The easiest way is to utilize some tesselation library, like libtess. What it will do it covert you boundary line (composed of interpolated points) into a set of triangles. It will preserve texture coordinates, so that you can just feed these triangles to the renderer.
SpriteKit note
Unfortunately, I am not really familiar with SpriteKit engine, so cannot guarantee you will be able to copy the idea over one-to-one, but please feel free to comment on the challenging aspects of the implementation and I will try to help.
There is very little contrast in my model.
The model frankly looks like one big blob, despite its having a complex shape. I can really only see its silhouette shape. The only details that I can see inside of the shape are created by the spot light, which is always very faint no matter how I tweak it.
What is the key to getting the shape of an object to appear clearly and to get some contrast?
Did you create normals for your triangles and pass those in? You never mentioned those, and surface normals are required before directional lighting works (A normal is a ray that is perpendicular to each surface, usually from the center of the surface.
I don't have my OpenGL references handy, but I remember the 3 kinds of lighting as directional, ambient, and specular, where ambient is global, and directional and diffuse depend on the lighting direction and the surface normals.
I would suggest starting with 1 light source first and figuring that out, then adding more lights once you get your single light working correctly.
The problem was, although I was creating normals and providing them with glNormalArray, I had not enabled GL_NORMALIZE so they weren't being used. I had been under the impression that GL_NORMALIZE automatically created normals, which is the opposite of what I wanted.