I would like to display a mesh with iOS ARKit with SceneKit and not RealityKit.
I am able to add a material to the mesh, like so which displays the mesh just fine
let edgesMaterial = SCNMaterial()
edgesMaterial.fillMode = .lines
edgesMaterial.lightingModel = .constant
edgesMaterial.transparency = 1.0
edgesMaterial.diffuse.contents = UIColor.red
scnGeom.materials = [edgesMaterial]
However, this adds overlapping lines of the wireframes in the scene. With RealityKit the fix is rather simple by just adding
arView.environment.sceneUnderstanding.options.insert(.occlusion)
How can I get the same effect with a scenekit scene? Basically I want to add a occlusion material with lines. Any pointers?
UPDATE:
I can get something close but not quite the same visualization by using this:
sceneView.debugOptions.insert([.showWireframe])
let occMaterial = SCNMaterial()
occMaterial.colorBufferWriteMask = SCNColorMask(rawValue: 0)
scnGeom.materials = [occMaterial]
The lines appear white and also any object I place in the scene will render the wireframe automatically now with this
Related
The goal is to recreate the lighting for this OBJ file: https://poly.google.com/view/cKryD9VnDEZ
Code to load OBJ file into SceneKit (can download file from above link):
let modelPath = "model.obj"
let url = NSURL(string: modelPath)
let scene = SCNScene(named: modelPath)!
sceneView.autoenablesDefaultLighting = true
sceneView.allowsCameraControl = true
sceneView.scene = scene
sceneView.backgroundColor = UIColor.white
Options tried so far:
1) The default ambient lighting is much harsher than the Google Poly lighting. Removing the ambient lighting rendered everything too flat.
2) Using four directional lights: one in front, one behind, one below, and one above the model. All lights are angled to point at the model. This was the best, but still left some shadows and harsher areas not seen on Google Polymer.
3) Added two more lights to option #2, this time adding lights to the left and right. This one was worse than option #2 since the extra lights combined with the four existing lights and whitewashed the model.
UPDATE AFTER FOLLOWING SUGGESTIONS:
The code now implements an ambient light and a directional light.
Adding the directional light to the camera node, versus the scene root node, made no difference for some reason.
The light code is below.
There are two problems:
1) In Screenshot 1, the right side of the chest is too bright and shows no edges. The far left face of the chest is too dark. The face with the best lighting is in the center. How can you get the lighting to be like this for all faces (or better match the Google Poly lighting)?
2) In Screenshot 2, the directional light appears to have no effect. How can you ensure the back of the model is as light as the front with the suggested architecture of one ambient light and one directional light?
SCREENSHOT 1:
SCREENSHOT 2:
CODE:
// Create ambient light
let ambientLightNode = SCNNode()
ambientLightNode.light = SCNLight()
ambientLightNode.light!.type = .ambient
ambientLightNode.light!.color = UIColor(white: 0.50, alpha: 1.0)
// Add ambient light to scene
scene.rootNode.addChildNode(ambientLightNode)
// Create directional light
let directionalLight = SCNNode()
directionalLight.light = SCNLight()
directionalLight.light!.type = .directional
directionalLight.light!.color = UIColor(white: 0.40, alpha: 1.0)
directionalLight.eulerAngles = SCNVector3(x: Float.pi, y: 0, z: 0)
// Add directional light
scene.rootNode.addChildNode(directionalLight)
OBJ files loaded through Model I/O use physically based lighting by default. This model has a cartoonish look and uses a lot of ambient lighting with a few specular highlights.
You should start by converting all your materials to the lambert lighting model.
Then add an ambient light to you scene. There's a lot of ambient lighting in this scene, every part of the object is lit. A color of 75% white will do.
Finally attach a directional light to the camera to highlight the polygons facing the user. A color of 50% white sounds about right.
In addition to MNuages answer, try to enable screen space ambient occlusion (on the camera). The following enables it for the current camera:
scnView.pointOfView.camera.screenSpaceAmbientOcclusionIntensity = 1.7;
scnView.pointOfView.camera.screenSpaceAmbientOcclusionNormalThreshold = 0.1;
scnView.pointOfView.camera.screenSpaceAmbientOcclusionDepthThreshold = 0.08;
scnView.pointOfView.camera.screenSpaceAmbientOcclusionBias = 0.33;
scnView.pointOfView.camera.screenSpaceAmbientOcclusionRadius = 3.0;
You will probably have to tweak the values a bit to get the for you desired results, the above is just what works for me in a certain scene.
My ultimate goal is to have an SCNNode representing an image floating in space. This is more or less easily accomplished with the current code I have below, but the problem is that the back side of the image isn't rendered and is thus transparent from the back. I want to be able to display a different image on the back so that there is something to see from both sides. the isDoubleSided property doesn't work here because it simply mimics what's on the front. Any Ideas? I looked into the idea of creating my own geometry from Sources and Elements but it seemed very complex for what should be really simple.
My current code:
private func createNode() -> SCNNode{
let scaleFactor = image.size.width/0.2
let width = image.size.width/scaleFactor
let height = image.size.height/scaleFactor
let geometry = SCNPlane(width: width, height: height)
let material = SCNMaterial()
material.diffuse.contents = image
geometry.materials = [material]
return SCNNode(geometry: geometry)
}
Thanks!
Since you want different images, you need to use different materials. SceneKit allows specifying material per geometry element. SCNPlane has only one element, that's why isDoubleSided just mirrors image on the back side. You have two options here:
Create two SCNPlane geometries, orient them back to back and assign different images to each geometry.firstMaterial.diffuse.contents
Create custom SCNGeometry from SCNGeometrySource (4 vertices of plane) and two SCNGeometryElements (one for each side: 2 triangles, 6 indices), and assign array of two materials (different images) to geometry.
The first option is easier, but looks more like a workaround.
I have used shader modifiers for Plane but its not working. Can anyone suggest me how to solve it?
let myShaderfragment = "#pragma transparent;\n" + "_output.color.a = 0.0;"
let myShaderSurface = "#pragma transparent;\n" + "_surface.diffuse.a = 0.0;"
material.shaderModifiers = [SCNShaderModifierEntryPoint.fragment : myShaderfragment, SCNShaderModifierEntryPoint.surface : myShaderSurface]
The SceneKit: What's New session from WWDC 2017 explains how to do that.
For the plane, use a material with constant as its lightingModel. It's the cheapest one.
This material will have writesToDepthBuffer set to true and colorBufferWriteMask set to [] (empty option set). That way the plane will write in the depth buffer, but won't draw anything on screen.
Set the light's shadowMode to deferred so that shadows are not applied when rendering the objects themselves, but as a final post-process.
There's a dedicated lighting model now (SCNLightingModelShadowOnly) to only render shadows
Here below my code and camera image to load 2d UIView into sceneView. I already tried to load into material content but getting node many times blank.
let annotationNode = SCNNode()
let planeGeoMetry:SCNPlane = SCNPlane()
planeGeoMetry.firstMaterial?.fillMode = .fill
planeGeoMetry.firstMaterial?.diffuse.contents = view
annotationNode.name = name
annotationNode.geometry = planeGeoMetry
self.sceneView.scene.rootNode.addChildNode(annotationNode)
From Apples docs on SCNMaterialProperty.contents:
SceneKit cannot use a layer that is already being displayed elsewhere (for example, the backing layer of a
UIView
object)
Personally the easiest way of displaying information on a plane is using a SpriteKit Scene.
I'd like to be able to add shapes to the surface of a sphere using SceneKit. I started with a simple example where I'm just trying to color a portion of the sphere's surface another color. I'd like this to be an object that can be tapped, selected, etc... so my thought was to add shapes as SCNNodes using custom SCNShape objects for the geometry.
What I have now is a blue square that I'm drawing from a series of points and adding to the scene containing a red sphere. It basically ends up tangent to a point on the sphere, but the real goal is to draw it on the surface. Is there anything in SceneKit that will allow me to do this? Do I need to do some math/geometry to make it the same shape as the sphere or map to a sphere's coordinates? Is what I'm trying to do outside the scope of SceneKit?
If this question is way too broad I'd be glad if anyone could point me towards books or resources to learn what I'm missing. I'm totally new to SceneKit and 3D in general, just having fun playing around with some ideas.
Here's some playground code for what I have now:
import UIKit
import SceneKit
import XCPlayground
class SceneViewController: UIViewController {
let sceneView = SCNView()
private lazy var sphere: SCNSphere = {
let sphere = SCNSphere(radius: 100.0)
sphere.materials = [self.surfaceMaterial]
return sphere
}()
private lazy var testScene: SCNScene = {
let scene = SCNScene()
let sphereNode: SCNNode = SCNNode(geometry: self.sphere)
sphereNode.addChildNode(self.blueChildNode)
scene.rootNode.addChildNode(sphereNode)
//scene.rootNode.addChildNode(self.blueChildNode)
return scene
}()
private lazy var surfaceMaterial: SCNMaterial = {
let material = SCNMaterial()
material.diffuse.contents = UIColor.redColor()
material.specular.contents = UIColor(white: 0.6, alpha: 1.0)
material.shininess = 0.3
return material
}()
private lazy var blueChildNode: SCNNode = {
let node: SCNNode = SCNNode(geometry: self.blueGeometry)
node.position = SCNVector3(0, 0, 100)
return node
}()
private lazy var blueGeometry: SCNShape = {
let points: [CGPoint] = [
CGPointMake(0, 0),
CGPointMake(50, 0),
CGPointMake(50, 50),
CGPointMake(0, 50),
CGPointMake(0, 0)]
var pathRef: CGMutablePathRef = CGPathCreateMutable()
CGPathAddLines(pathRef, nil, points, points.count)
let bezierPath: UIBezierPath = UIBezierPath(CGPath: pathRef)
let shape = SCNShape(path: bezierPath, extrusionDepth: 1)
shape.materials = [self.blueNodeMaterial]
return shape
}()
private lazy var blueNodeMaterial: SCNMaterial = {
let material = SCNMaterial()
material.diffuse.contents = UIColor.blueColor()
return material
}()
override func viewDidLoad() {
super.viewDidLoad()
sceneView.frame = self.view.bounds
sceneView.backgroundColor = UIColor.blackColor()
self.view.addSubview(sceneView)
sceneView.autoenablesDefaultLighting = true
sceneView.allowsCameraControl = true
sceneView.scene = testScene
}
}
XCPShowView("SceneKit", view: SceneViewController().view)
If you want to map 2D content into the surface of a 3D SceneKit object, and have the 2D content be dynamic/interactive, one of the easiest solutions is to use SpriteKit for the 2D content. You can set your sphere's diffuse contents to an SKScene, and create/position/decorate SpriteKit nodes in that scene to arrange them on the face of the sphere.
If you want to have this content respond to tap events... Using hitTest in your SceneKit view gets you a SCNHitTestResult, and from that you can get texture coordinates for the hit point on the sphere. From texture coordinates you can convert to SKScene coordinates and spawn nodes, run actions, or whatever.
For further details, your best bet is probably Apple's SceneKitReel sample code project. This is the demo that introduced SceneKit for iOS at WWDC14. There's a "slide" in that demo where paint globs fly from the camera at a spinning torus and leave paint splashes where they hit it — the torus has a SpriteKit scene as its material, and the trick for leaving splashes on collisions is basically the same hit test -> texture coordinate -> SpriteKit coordinate approach outlined above.
David Rönnqvist's SceneKit book (available as an iBook) has an example (the EarthView example, a talking globe, chapter 5) that is worth looking at. That example constructs a 3D pushpin, which is then attached to the surface of a globe at the location of a tap.
Your problem is more complicated because you're constructing a shape that covers a segment of the sphere. Your "square" is really a spherical trapezium, a segment of the sphere bounded by four great circle arcs. I can see three possible approaches, depending on what you're ultimately looking for.
The simplest way to do it is to use an image as the material for the sphere's surface. That approach is well illustrated in the Ronnqvist EarthView example, which uses several images to show the earth's surface. Instead of drawing continents, you'd draw your square. This approach isn't suitable for interactivity, though. Look at SCNMaterial.
Another approach would be to use hit test results. That's documented on SCNSceneRenderer (which SCNView complies with) and SCNHitTest. Using the hit test results, you could pull out the face that was tapped, and then its geometry elements. This won't get you all the way home, though, because SceneKit uses triangles for SCNSphere, and you're looking for quads. You will also be limited to squares that line up with SceneKit's underlying wireframe representation.
If you want full control of where the "square" is drawn, including varying its angle relative to the equator, I think you'll have to build your own geometry from scratch. That means calculating the latitude/longitude of each corner point, then generating arcs between those points, then calculating a bunch of intermediate points along the arcs. You'll have to add a fudge factor, to raise the intermediate points slightly above the sphere's surface, and build up your own quads or triangle strips. Classes here are SCNGeometry, SCNGeometryElement, and SCNGeometrySource.