How to specify two-sided material for RealityKit / ARView? - ios

Question

I'm trying to load a model and texture in RealityKit (set up in an ARView instance), but I can't seem to figure out how to specify the material should be two-sided.
I have the model loaded up as a ModelEntity, the texture loaded up as a TextureResource. The model and texture are loading up, but are rending one-sided. Since the model is open (i.e., back faces are visible), there are gaps on how it is rendered.
So far, I have,
let entity: ModelEntity = try .loadModel(named: "model.obj")
var material = SimpleMaterial()
material.baseColor = try .texture(.load(named: "texture.png"))
entity.model?.materials = [material]
I was hoping to find a property such as
material.twoSided = true
but so far, I have not found the equivalent thing in RealityKit.
Anyone know how to set two-sided materials in RealityKit?

Answer 1

There doesn't seem to be any way to do this programmatically at the moment via the RealityKit APIs.
Can you change your model definition so it doesn't do back face culling? For example in a USDZ file I am importing it defines one part of the mesh as:
def Mesh "Plane_1"
{
uniform bool doubleSided = 1
You might be able to convert your obj file to a used file using usdzconvert first (https://developer.apple.com/download/more/?=USDPython) then edit the file manually, then import that in to your scene.
It might also be depending on how the model is setup that you can pass in more than one material to the materials array that are applied to different parts of the model, you can see how many materials the model expects:
entity.model?.mesh.expectedMaterialCount

Answer 2

As others already answered you can't do it programmatically.
You can however do it manually for each model via the inspection panel.
See the image below. Near the bottom, you have "Double Sided" checkbox.

Answer 3

I don't think there is a way to do this. RealityKit is still in early days. Material support in RealityKit is very limited right now. I think there are plans to change this in iOS 14 or beyond. There are comments in the documentation that describe features that don't yet exist such as Material protocol says "Describes a material (colors, shaders, textures, etc.) that define the look of a mesh part." There currently is no way to define custom shaders. If you look at the RealityKit framework bundle, there are shader graph definitions and new material features that are not yet exposed in the public API. I suspect there will be a shader graph editor, support for custom shaders, and double-sided materials coming.

Answer 4

It can be done, but it takes a change of perspective: instead of making a double sided material, you create a double sided mesh. This is accomplished by taking every part in every model and creating a double with the normals inverted (and the triangles reversed). Using the code below, the solution to the stated question becomes:
do {
let entity: ModelEntity = try .loadModel(named: "model.obj")
if let model = entity.model {
try model.mesh.addInvertedNormals()
// or alternatively, since this model isn't onscreen yet:
// model.mesh = try model.mesh.addingInvertedNormals()
var material = SimpleMaterial()
material.baseColor = try .texture(.load(named: "texture.png"))
model.materials = [material]
entity.model = model
}
} catch {}
And the entity will now display the material on both sides. Here is the code to do it:
import Foundation
import RealityKit
public extension MeshResource {
// call this to create a 2-sided mesh that will then be displayed
func addingInvertedNormals() throws -> MeshResource {
return try MeshResource.generate(from: contents.addingInvertedNormals())
}
// call this on a mesh that is already displayed to make it 2 sided
func addInvertedNormals() throws {
try replace(with: contents.addingInvertedNormals())
}
static func generateTwoSidedPlane(width: Float, depth: Float, cornerRadius: Float = 0) -> MeshResource {
let plane = generatePlane(width: width, depth: depth, cornerRadius: cornerRadius)
let twoSided = try? plane.addingInvertedNormals()
return twoSided ?? plane
}
}
public extension MeshResource.Contents {
func addingInvertedNormals() -> MeshResource.Contents {
var newContents = self
newContents.models = .init(models.map { $0.addingInvertedNormals() })
return newContents
}
}
public extension MeshResource.Model {
func partsWithNormalsInverted() -> [MeshResource.Part] {
return parts.map { $0.normalsInverted() }.compactMap { $0 }
}
func addingParts(additionalParts: [MeshResource.Part]) -> MeshResource.Model {
let newParts = parts.map { $0 } + additionalParts
var newModel = self
newModel.parts = .init(newParts)
return newModel
}
func addingInvertedNormals() -> MeshResource.Model {
return addingParts(additionalParts: partsWithNormalsInverted())
}
}
public extension MeshResource.Part {
func normalsInverted() -> MeshResource.Part? {
if let normals, let triangleIndices {
let newNormals = normals.map { $0 * -1.0 }
var newPart = self
newPart.normals = .init(newNormals)
// ordering of points in the triangles must be reversed,
// or the inversion of the normal has no effect
newPart.triangleIndices = .init(triangleIndices.reversed())
// id must be unique, or others with that id will be discarded
newPart.id = id + " with inverted normals"
return newPart
} else {
print("No normals to invert, returning nil")
return nil
}
}
}
So, calling addingInvertedNormals() creates a mesh that will show the same material on both sides. I have used this to create a 2-sided grid plane.
With a little extra work (left as an exercise!), you could give the created parts different material indexes, and show different materials on each side.

Answer 5

What you describe is called culling. Check MTLCullMode for example. From there you can jump to various points where you can set culling mode (you are interested in no culling).