I have a 3d mesh in blender, textured with UV mapping.
I would like to add some material effects like wood with nodes, so I went to the node editor, but clicking on "use nodes" my mesh turn yellow (there's a yellow color applied to my mesh).
How can I do to apply UV mapping and material to my mesh?
Thank you in advance
It sounds like you are using the blender internal render engine. While this has support for node based materials, development of it was limited and it wasn't something that caught on much, so you will not find much in the way of tutorials.
To use a texture you will need to add a texture node and select the texture you setup before, most likely a geometry node to get the uv's and connect them to the output.
You will find that the newer Cycles render engine is fully node based and has much better support and documentation with a larger range of nodes available.
Related
I’m interested in the issue of data processing from TrueDepth Camera. It is necessary to obtain the data of a person’s face, build a 3D model of the face and save this model in an .obj file.
Since in the 3D model needed presence of the person’s eyes and teeth, then ARKit / SceneKit is not suitable, because ARKit / SceneKit do not fill these areas with data.
But with the help of the SceneKit.ModelIO library, I managed to export ARSCNView.scene (type SCNScene) in the .obj format.
I tried to take this project as a basis:
https://developer.apple.com/documentation/avfoundation/cameras_and_media_capture/streaming_depth_data_from_the_truedepth_camera
In this project, working with TrueDepth Camera is done using Metal, but if I'm not mistaken, MTKView, rendered using Metal, is not a 3D model and cannot be exported as .obj.
Please tell me if there is a way to export MTKView to SCNScene or directly to .obj?
If there is no such method, then how to make a 3D model from AVDepthData?
Thanks.
It's possible to make a 3D model from AVDepthData, but that probably isn't what you want. One depth buffer is just that — a 2D array of pixel distance-from-camera values. So the only "model" you're getting from that isn't very 3D; it's just a height map. That means you can't look at it from the side and see contours that you couldn't have seen from the front. (The "Using Depth Data" sample code attached to the WWDC 2017 talk on depth photography shows an example of this.)
If you want more of a truly-3D "model", akin to what ARKit offers, you need to be doing the work that ARKit does — using multiple color and depth frames over time, along with a machine learning system trained to understand human faces (and hardware optimized for running that system quickly). You might not find doing that yourself to be a viable option...
It is possible to get an exportable model out of ARKit using Model I/O. The outline of the code you'd need goes something like this:
Get ARFaceGeometry from a face tracking session.
Create MDLMeshBuffers from the face geometry's vertices, textureCoordinates, and triangleIndices arrays. (Apple notes the texture coordinate and triangle index arrays never change, so you only need to create those once — vertices you have to update every time you get a new frame.)
Create a MDLSubmesh from the index buffer, and a MDLMesh from the submesh plus vertex and texture coordinate buffers. (Optionally, use MDLMesh functions to generate a vertex normals buffer after creating the mesh.)
Create an empty MDLAsset and add the mesh to it.
Export the MDLAsset to a URL (providing a URL with the .obj file extension so that it infers the format you want to export).
That sequence doesn't require SceneKit (or Metal, or any ability to display the mesh) at all, which might prove useful depending on your need. If you do want to involve SceneKit and Metal you can probably skip a few steps:
Create ARSCNFaceGeometry on your Metal device and pass it an ARFaceGeometry from a face tracking session.
Use MDLMesh(scnGeometry:) to get a Model I/O representation of that geometry, then follow steps 4-5 above to export it to an .obj file.
Any way you slice it, though... if it's a strong requirement to model eyes and teeth, none of the Apple-provided options will help you because none of them do that. So, some food for thought:
Consider whether that's a strong requirement?
Replicate all of Apple's work to do your own face-model inference from color + depth image sequences?
Cheat on eye modeling using spheres centered according to the leftEyeTransform/rightEyeTransform reported by ARKit?
Cheat on teeth modeling using a pre-made model of teeth, composed with the ARKit-provided face geometry for display? (Articulate your inner-jaw model with a single open-shut joint and use ARKit's blendShapes[.jawOpen] to animate it alongside the face.)
so I have this mesh below which I created. It was originally a .obj, but I converted it over to a .scn. This is how the mesh looks with default ambient lighting when viewing in SceneKit editor.
My issue is, that when I add a light to the scene it very badly shows off the mesh's geometry. For example, in the mesh below I add 4 omni lights around the mesh.
As you can tell, the top of the Apple shows of the individual polygons that make up the mesh. My main question is: is there something I can do to reduce this? There has to be some setting where I can feather how the light interacts with the mesh. I have attempted to turn off shadows but that didn't do anything.
Adding a bunch more lights also didn't fix the problem. You can still see those little polygons standing out upon getting closer.
Here is another example where I add a spot light looking directly down at the mesh. Here the polygon problems exists, but the entire apple also becomes extremely dark. Is there someway to light the apple up so it doesn't go completely dark so quickly without adding more lights?
It's very likely that your mesh only has positions. You will also need per-vertex normals to get a smooth shading. Blender almost certainly has a tool to generate missing normals.
The lower part of the apple is dark because it doesn't get any light. You can add an .ambient light to your scene so that every object gets at least a minimum amount of light.
Try the following:
myNode.geometry.subdivisionLevel = 1
(where myNode is the SCNNode, that holds the Apple Geometry from your SCN File)
This should at least smooth the surface.
I've been trying without success to extract face features, for instance the mouth, from ARSCNFaceGeometry in order to change their color or add a different material.
I understand I need to create an SCNGeometry for which I have the SCNGeometrySource but haven't been able to create the SCNGeometryElement.
Have tried creating it from ARFaceAnchor in update(from faceGeometry: ARFaceGeometry) but so far have been unable.
Would really appreciate someone help
ARSCNFaceGeometry is a single mesh. If you want different areas of it to be different colors, your best bet is to apply a texture map (which you do in SceneKit by providing images for material property contents).
There’s no semantic information associated with the vertices in the mesh — that is, there’s nothing that says “this point is the tip of the nose, these points are the edge of the upper lip, etc”. But the mesh is topologically stable, so if you create a texture image that adds a bit of color around the lips or a lightning bolt over the eye or whatever, it’ll stay there as the face moves around.
If you need help getting started on painting a texture, there are a couple of things you could try:
Create a dummy texture first
Make a square image and fill it with a double gradient, such that the red and blue component for each pixel is based on the x and y coordinate of that pixel. Or some other distinctive pattern. Apply that texture to the model, and see how it looks — the landmarks in the texture will guide you where to paint.
Export the model
Create a dummy ARSCNFaceGeometry using the init(blendShapes:) initializer and an empty blendShapes dictionary (you don’t need an active ARFaceTracking session for this, but you do need an iPhone X). Use SceneKit’s scene export APIs (or Model I/O) to write that model out to a 3D file of some sort (.scn, which you can process further on the Mac, or something like .obj).
Import that file into your favorite 3D modeling tool (Blender, Maya, etc) and use that tool to paint a texture. Then use that texture in your app with real faces.
Actually, the above is sort of an oversimplification, even though it’s the simple answer for common cases. ARSCNFaceGeometry can actually contain up to four submeshes if you create it with the init(device:fillMesh:) initializer. But even then, those parts aren’t semantically labeled areas of the face — they’re the holes in the regular face model, flat fill-ins for the places where eyes and mouth show through.
I am creating a model of Saturn and I'm having problems when creating the rings. I found this asset
but when I try to set it as a diffuse, it projects like this
How can I control the way a texture projects over a geometry?
I found the solution. By replacing the cylinder with a torus and rotating the image 90 degrees, XCode did the mapping itself.
But there must be a better way.
This isn’t specifically a SceneKit or IOS issue, the same would apply in any 3D package.
You can control the way a texture projects over a geometry by using UV mapping. In practice that means you map the vertices and faces of the model on to the texture in software such a Blender. The texture you use now is meant to be tiled but because the lines on the texture are perfectly straight it will never look optimal.
To save yourself some trouble, use a texture that shows the entire ring from the top/above.
I think the best way is to use SCNTube.
EDIT: I've solved the issue below the tilde line -- the missing chunks -- by fixing an elementary error in my for-loop dealing with calculating face normals. I now have a new problem though: strange, unwanted shadows on the surface itself. Some areas appear darker than others... See the next picture for the current issue.
I have an omni light added to my scene's root node as well as a directional light added in the same manner. For some reason I can't seem to light the underbelly of the surface otherwise. Notice the strange shadow on the inside of the concave surface (it's more pronounced when I remove the subdivision effect as I have done here) -->
Here is the surface from above -- notice how some areas seem strangely darker.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is a concave surface.
I have these smooth, curved planes in 3D space. Right now, they look rather cartoonish -- I would like to utilize some form of lighting to make them look more "3D-ish."
I have tried various combinations of ambient lighting, omni lighting, and default lighting, but nothing seems to work right. I get something quite strange when I apply something like a basic omni light --
Here is another look at a better angle using omni lighting. Looks like someone took a bite out of it --
Am I overlooking a specific type of light or lighting strategy?
I'd like to avoid used baked lighting, because the scene is rather simple. Thanks.
I'll outline my steps for the bold.
1: I specify the vertices for each of the four faces of a pyramid-like shape. Like this (apologies for my lack of artistic ability) -->
2: I specify the indices for the face, i.e., [0,1,2, 0,2,3, etc.]
I create a dictionary mapping each vertex to the sum of that vertex's adjacent, normalized face normals.
I append each of these summed up normalized per-vertex normals to a vector.
I combine the vertices, indices, and vector of normals to create an SCN Geometry.
To get the rounded look, I increase the subdivision count.
Pray that it works.
I'm new to the 3D world, so I could be way out in left field and not even know it.
This should give you a reasonable result with minimal effort and the least possible need to understand 3D lighting.
Open the Fox game example/sample from Apple:
https://developer.apple.com/library/prerelease/ios/samplecode/Fox/Introduction/Intro.html
Delete everything from the level.scn Scene Graph other than Lights, Camera and the Mountain.
And then add your geometry object to a node at the bottom, where I have the sphere highlighted at the bottom of the Scene Graph....
Now the material needs a bit of work, to make it useful.
Select the Mountain by clicking on it in the View, and goto the material editor and make it look like this, just keep checking against this image until yours matches the few (weird) changes I've made. And trust me this will work out fine:
When you want to get that lovely red you have, you simply change this property: DIFFUSE : It's right at the top of the Material settings.
Now you have a material and lighting setup that gives a reasonable approximation of curvature in a 3D space.
Applying this material to your object is a little weird, and unintuitive, you go here, and click on the add button, and pick the material with the same name as the one in the above image, that’s on the mountain.
You can improve this by adding two more lights in what’s known as a “3 point lighting setup”, google this phrase to see it explained.
Further, you can add off screen (out of camera) placards, usually white, to manage key reflections to further assist in users getting a feel for what’s being presented.