I have an idea for AR app, and I noticed that in most AR apps the objects are not hided through the wall, is it possible for example I pinned the AR portrait in my room, and it is only can be seen if I go there, not through the walls?
I have no experience in AR, just about to learn it.
Thank you.
it's simple way to do this, you have to detect plane (in your case a wall) and setup for
node.geometry.firstMaterial?.colorBufferWriteMask = []
node.renderingOrder = -1
You can easily hide all your objects behind a wall using three different approaches for wall creation: SCNBox(), SCNShape() with extrusion, or SCNGeometry(). Whatever you choose, just assign empty instance property .colorBufferWriteMask to it. And .renderingOrder instance property must be -1. Node with a negative value of rendering order is rendered first.
let wallNode = SCNNode()
wallNode.geometry = SCNBox(width: 15.0, height: 3.0, length: 0.1, chamferRadius: 0)
wallNode.position = SCNVector3(x: 0, y: 0, z: 5)
//wallNode.geometry = SCNShape(path: NSBezierPath?, extrusionDepth: CGFloat)
//wallNode.geometry = SCNGeometry(sources: [SCNGeometrySource], elements: [SCNGeometryElement]?)
wallNode.geometry?.firstMaterial?.colorBufferWriteMask = []
wallNode.renderingOrder = -1
scene.rootNode.addChildNode(wallNode)
Hope this helps.
I've written this function to attempt to scale UIBezierPaths that I have as they are being drawn...
func fit(into:CGRect) -> Self {
let bounds = self.cgPath.boundingBox
let sw = into.size.width/bounds.width
let sh = into.size.height/bounds.height
let factor = min (5, min(sw, max(sh, 0.0)))
return scale(x: factor, y: factor, into: into)
}
It works ok. The problem however exists. There are dashes inbetween points sometimes. If I'm not making sense please tell me and I'll try to rephrase my question.
I have the following code that creates a SCNBox and shoots it on the screen. This works but as soon as I turn the phone in any other direction then the force impulse does not get updated and it always shoots the box in the same old position.
Here is the code:
#objc func tapped(recognizer :UIGestureRecognizer) {
guard let currentFrame = self.sceneView.session.currentFrame else {
return
}
/
let box = SCNBox(width: 0.2, height: 0.2, length: 0.2, chamferRadius: 0)
let material = SCNMaterial()
material.diffuse.contents = UIColor.red
material.lightingModel = .constant
var translation = matrix_identity_float4x4
translation.columns.3.z = -0.01
let node = SCNNode()
node.geometry = box
node.geometry?.materials = [material]
print(currentFrame.camera.transform)
node.physicsBody = SCNPhysicsBody(type: .dynamic, shape: nil)
node.simdTransform = matrix_multiply(currentFrame.camera.transform, translation)
node.physicsBody?.applyForce(SCNVector3(0,2,-10), asImpulse: true)
self.sceneView.scene.rootNode.addChildNode(node)
}
Line 26 is where I apply the force but it does not take into account the user's current phone orientation. How can I fix that?
On line 26 you're passing a constant vector to applyForce. That method takes a vector in world space, so passing a constant vector means you're always applying a force in the same direction — if you want a direction that's based on the direction the camera or something else is pointing, you'll need to calculate a vector based on that direction.
The (new) SCNNode property worldFront might prove helpful here — it gives you the direction a node is pointing, automatically converted to world space, so it's useful with physics methods. (Though you might want to scale it.)
The code below is used to create a scene and create blocks in SceneKit. The blocks come out looking flat and not "3D enough" according to our users. Screenshots 1-2 show our app.
Screenshots 3-5 show what users expect the blocks to look like, that is more 3D-like.
After speaking to different people, there are different opinions about how to render blocks that look more like screenshots 3-5. Some people say use ambient occlusion, others say voxel lighting, some say use spot lighting and use shadows, or directional lighting.
We previously tried adding omni lighting, but that didn't work so it was removed. As you can see in the code, we also experimented with an ambient light node but that also didn't yield the right results.
What is the best way to render our blocks and achieve a comparable look to screenshots 3-5?
Note: we understand the code is not optimized for performance, i.e., that polygons are shown that should not be shown. That is okay. The focus is not on performance but rather on achieving more 3D-like rendering. You can assume some hard limit on nodes, like no more than 1K or 10K in a scene.
Code:
func createScene() {
// Set scene view
let scene = SCNScene()
sceneView.jitteringEnabled = true
sceneView.scene = scene
// Add camera node
sceneView.pointOfView = cameraNode
// Make delegate to capture screenshots
sceneView.delegate = self
// Set ambient lighting
let ambientLightNode = SCNNode()
ambientLightNode.light = SCNLight()
ambientLightNode.light!.type = SCNLightTypeAmbient
ambientLightNode.light!.color = UIColor(white: 0.50, alpha: 1.0)
//scene.rootNode.addChildNode(ambientLightNode)
//sceneView.autoenablesDefaultLighting = true
// Set floor
setFloor()
// Set sky
setSky()
// Set initial position for user node
userNode.position = SCNVector3(x: 0, y: Float(CameraMinY), z: Float(CameraZoom))
// Add user node
scene.rootNode.addChildNode(userNode)
// Add camera to user node
// zNear fixes white triangle bug while zFar fixes white line bug
cameraNode.camera = SCNCamera()
cameraNode.camera!.zNear = Double(0.1)
cameraNode.camera!.zFar = Double(Int.max)
cameraNode.position = SCNVector3(x: 0, y: 0, z: 0) //EB: Add some offset to represent the head
userNode.addChildNode(cameraNode)
}
private func setFloor() {
// Create floor geometry
let floorImage = UIImage(named: "FloorBG")!
let floor = SCNFloor()
floor.reflectionFalloffEnd = 0
floor.reflectivity = 0
floor.firstMaterial!.diffuse.contents = floorImage
floor.firstMaterial!.diffuse.contentsTransform = SCNMatrix4MakeScale(Float(floorImage.size.width)/2, Float(floorImage.size.height)/2, 1)
floor.firstMaterial!.locksAmbientWithDiffuse = true
floor.firstMaterial!.diffuse.wrapS = .Repeat
floor.firstMaterial!.diffuse.wrapT = .Repeat
floor.firstMaterial!.diffuse.mipFilter = .Linear
// Set node & physics
// -- Must set y-position to 0.5 so blocks are flush with floor
floorLayer = SCNNode(geometry: floor)
floorLayer.position.y = -0.5
let floorShape = SCNPhysicsShape(geometry: floor, options: nil)
let floorBody = SCNPhysicsBody(type: .Static, shape: floorShape)
floorLayer.physicsBody = floorBody
floorLayer.physicsBody!.restitution = 1.0
// Add to scene
sceneView.scene!.rootNode.addChildNode(floorLayer)
}
private func setSky() {
// Create sky geometry
let sky = SCNFloor()
sky.reflectionFalloffEnd = 0
sky.reflectivity = 0
sky.firstMaterial!.diffuse.contents = SkyColor
sky.firstMaterial!.doubleSided = true
sky.firstMaterial!.locksAmbientWithDiffuse = true
sky.firstMaterial!.diffuse.wrapS = .Repeat
sky.firstMaterial!.diffuse.wrapT = .Repeat
sky.firstMaterial!.diffuse.mipFilter = .Linear
sky.firstMaterial!.diffuse.contentsTransform = SCNMatrix4MakeScale(Float(2), Float(2), 1);
// Set node & physics
skyLayer = SCNNode(geometry: sky)
let skyShape = SCNPhysicsShape(geometry: sky, options: nil)
let skyBody = SCNPhysicsBody(type: .Static, shape: skyShape)
skyLayer.physicsBody = skyBody
skyLayer.physicsBody!.restitution = 1.0
// Set position
skyLayer.position = SCNVector3(0, SkyPosY, 0)
// Set fog
/*sceneView.scene?.fogEndDistance = 60
sceneView.scene?.fogStartDistance = 50
sceneView.scene?.fogDensityExponent = 1.0
sceneView.scene?.fogColor = SkyColor */
// Add to scene
sceneView.scene!.rootNode.addChildNode(skyLayer)
}
func createBlock(position: SCNVector3, animated: Bool) {
...
// Create box geometry
let box = SCNBox(width: 1.0, height: 1.0, length: 1.0, chamferRadius: 0.0)
box.firstMaterial!.diffuse.contents = curStyle.getContents() // "curStyle.getContents()" either returns UIColor or UIImage
box.firstMaterial!.specular.contents = UIColor.whiteColor()
// Add new block
let newBlock = SCNNode(geometry: box)
newBlock.position = position
blockLayer.addChildNode(newBlock)
}
Screenshots 1-2 (our app):
Screenshots 3-5 (ideal visual representation of blocks):
I still think there's a few easy things you can do that will make a big difference to how your scene is rendered. Apologies for not using your code, this example is something I had lying around.
Right now your scene is only lit by an ambient light.
let aLight = SCNLight()
aLight.type = SCNLightTypeAmbient
aLight.color = UIColor(red: 0.2, green: 0.2, blue: 0.2, alpha: 1.0)
let aLightNode = SCNNode()
aLightNode.light = aLight
scene.rootNode.addChildNode(aLightNode)
If I use only this light in my scene I see the following. Note how all faces are lit the same irrespective of the direction they face. Some games do pull off this aesthetic very well.
The following block of code adds a directional light to this scene. The transformation applied in this light won't be valid for your scene, it's important to orientate the light according to where you want the light coming from.
let dLight = SCNLight()
dLight.type = SCNLightTypeDirectional
dLight.color = UIColor(red: 0.6, green: 0.6, blue: 0.6, alpha: 1.0)
let dLightNode = SCNNode()
dLightNode.light = dLight
var dLightTransform = SCNMatrix4Identity
dLightTransform = SCNMatrix4Rotate(dLightTransform, -90 * Float(M_PI)/180, 1, 0, 0)
dLightTransform = SCNMatrix4Rotate(dLightTransform, 37 * Float(M_PI)/180, 0, 0, 1)
dLightTransform = SCNMatrix4Rotate(dLightTransform, -20 * Float(M_PI)/180, 0, 1, 0)
dLightNode.transform = dLightTransform
scene.rootNode.addChildNode(dLightNode)
Now we have shading on each of the faces based on their angle relative to the direction of the light.
Currently SceneKit only supports shadows if you're using the SCNLightTypeSpot. Using a spotlight means we need to both orientate (as per directional light) and position it. I use this as a replacement for the directional light.
let sLight = SCNLight()
sLight.castsShadow = true
sLight.type = SCNLightTypeSpot
sLight.zNear = 50
sLight.zFar = 120
sLight.spotInnerAngle = 60
sLight.spotOuterAngle = 90
let sLightNode = SCNNode()
sLightNode.light = sLight
var sLightTransform = SCNMatrix4Identity
sLightTransform = SCNMatrix4Rotate(sLightTransform, -90 * Float(M_PI)/180, 1, 0, 0)
sLightTransform = SCNMatrix4Rotate(sLightTransform, 65 * Float(M_PI)/180, 0, 0, 1)
sLightTransform = SCNMatrix4Rotate(sLightTransform, -20 * Float(M_PI)/180, 0, 1, 0)
sLightTransform = SCNMatrix4Translate(sLightTransform, -20, 50, -10)
sLightNode.transform = sLightTransform
scene.rootNode.addChildNode(sLightNode)
In the above code we first tell the spotlight to cast a shadow, by default all nodes in your scene will then cast a shadow (this can be changed). The zNear and zFar settings are also important and must be specified so that the nodes casting shadows are within this range of distance from the light source. Nodes outside this range will not cast a shadow.
After shading/shadows there's a number of other effects you can apply easily. Depth of field effects are available for the camera. Fog is similarly easy to include.
scene.fogColor = UIColor.blackColor()
scene.fogStartDistance = 10
scene.fogEndDistance = 110
scenekitView.backgroundColor = UIColor(red: 0.2, green: 0.2, blue: 0.2, alpha: 1.0)
Update
Turns out you can get shadows from a directional light. Modifying the spotlight code from above by changing its type and setting the orthographicScale. Default value for orthographicScale seems to be 1.0, obviously not suitable for scenes much larger than 1.
let dLight = SCNLight()
dLight.castsShadow = true
dLight.type = SCNLightTypeDirectional
dLight.zNear = 50
dLight.zFar = 120
dLight.orthographicScale = 30
let dLightNode = SCNNode()
dLightNode.light = dLight
var dLightTransform = SCNMatrix4Identity
dLightTransform = SCNMatrix4Rotate(dLightTransform, -90 * Float(M_PI)/180, 1, 0, 0)
dLightTransform = SCNMatrix4Rotate(dLightTransform, 65 * Float(M_PI)/180, 0, 0, 1)
dLightTransform = SCNMatrix4Rotate(dLightTransform, -20 * Float(M_PI)/180, 0, 1, 0)
dLightTransform = SCNMatrix4Translate(dLightTransform, -20, 50, -10)
dLightNode.transform = dLightTransform
scene.rootNode.addChildNode(dLightNode)
Produces the following image.
The scene size is 60x60, so in this case setting the orthographic scale to 30 produces shadows for the objects close to the light. The directional light shadows appear different to the spot light due to the difference in projections (orthographic vs perspective) used when rendering the shadow map.
Ambient occlusion calculations will give you the best results, but is very expensive, particularly in a dynamically changing world, which it looks like this is.
There are several ways to cheat, and get the look of Ambient occlusion.
Here's one:
place transparent, gradient shadow textures on geometry "placards" used to place/present the shadows at the places required. This will involve doing checks of geometry around the new block before determining what placards to place, with which desired texture for the shadowing. But this can be made to look VERY good, at a very low cost in terms of polygons, draw calls and filtrate. It's probably the cheapest way to do this, and have it look good/great, and can only really be done (with a good look) in a world of blocks. A more organic world rules this technique out. Please excuse the pun.
Or, another, similar: Place additional textures onto/into objects that have the shadow, and blend this with the other textures/materials in the object. This will be a bit fiddly, and I'm not an expert on the powers of materials in Scene Kit, so can't say for sure this is possible and/or easy in it.
Or: Use a blend of textures with a vertex shader that's adding a shadow from the edges that touch or otherwise need/desire a shadow based on your ascertaining what and where you want shadows and to what extent. Will still need the placards trick on the floors/walls unless you add more vertices inside flat surfaces for the purpose of vertex shading for shadows.
Here's something I did for a friend's CD cover... shows the power of shadows. It's orthographic, not true 3D perspective, but the shadows give the impression of depths and create the illusions of space:
all answers above (or below) seem to be good ones (at the time of this writing) however,
what I use (just for setting up a simple scene) is one ambient light (lights everything in all directions) to make things visible.And then one omnidirectional light positioned somewhere in the middle of your scene, the omni light can be raised up (Y up I mean) to light the whole of your scene. The omni light gives the user a sense of shading and the ambient light makes it more like a sun light.
for example:
Imagine sitting in a living room (like I am right now) and the sun-light peers through the window to your right.
You can obviously see a shadow of an area that the couch is not getting sun light, however you can still see details of what is in the shadow.
Now! all the sudden your wold gets rid of ambient light BOOM! The shadow is now pitch black, you can't anymore see what is in the shadow.
Or say the ambient light came back again (what a relief), but all the sudden the omni light stopped working. (probably my fault :( ) Everything now is lighted the same, no shadow, no difference, but if you lay a paper on the table, and look at it from above, there is no shadow! So you think it is part of the table! In a world like this your rely on the contour of something in order to see it- you would have to look at the table from side view, to see the thickness of the paper.
Hope this helps (at least a little)
Note: ambient lighting give a similar effect to emissive material
On some iOS devices (iPhone 6s Plus) there is a partial and arbitrary disappearance of object parts.
How to avoid this?
All sticks must be the same and are clones of one SCNNode.
16 complex SCNNodes, from 3 SCNNode: box, ball and stick. Node containing a geometry by node.flattenedClone().
It must be like this:
Сode fragment:
func initBox()
{
var min: SCNVector3 = SCNVector3()
var max: SCNVector3 = SCNVector3()
let geom1 = SCNBox(width: boxW, height: boxH, length: boxL, chamferRadius: boxR)
geom1.firstMaterial?.reflective.contents = UIImage(data: BoxData)
geom1.firstMaterial?.reflective.intensity = 1.2
geom1.firstMaterial?.fresnelExponent = 0.25
geom1.firstMaterial?.locksAmbientWithDiffuse = true
geom1.firstMaterial?.diffuse.wrapS = SCNWrapMode.Repeat
let geom2 = SCNSphere(radius: 0.5 * boxH)
geom2.firstMaterial?.reflective.contents = UIImage(data: BalData)
geom2.firstMaterial?.reflective.intensity = 1.2
geom2.firstMaterial?.fresnelExponent = 0.25
geom2.firstMaterial?.locksAmbientWithDiffuse = true
geom2.firstMaterial?.diffuse.wrapS = SCNWrapMode.Repeat
let geom3 = SCNCapsule(capRadius: stickR, height: stickH)
geom3.firstMaterial?.reflective.contents = UIImage(data: StickData)
geom3.firstMaterial?.reflective.intensity = 1.2
geom3.firstMaterial?.fresnelExponent = 0.25
geom3.firstMaterial?.locksAmbientWithDiffuse = true
geom3.firstMaterial?.diffuse.wrapS = SCNWrapMode.Repeat
let box = SCNNode()
box.castsShadow = false
box.position = SCNVector3Zero
box.geometry = geom1
Material.setFirstMaterial(box, materialName: Materials[boxMatId])
let bal = SCNNode()
bal.castsShadow = false
bal.position = SCNVector3(0, 0.15 * boxH, 0)
bal.geometry = geom2
Material.setFirstMaterial(bal, materialName: Materials[balMatId])
let stick = SCNNode()
stick.castsShadow = false
stick.position = SCNVector3Zero
stick.geometry = geom3
stick.getBoundingBoxMin(&min, max: &max)
stick.pivot = SCNMatrix4MakeTranslation(0, min.y, 0)
Material.setFirstMaterial(stick, materialName: Materials[stickMatId])
box.addChildNode(bal)
box.addChildNode(stick)
boxmain = box.flattenedClone()
boxmain.name = "box"
}
Add nodes to the scene:
func Boxesset()
{
let Boxes = SCNNode()
Boxes.name = "Boxes"
var z: Float = -4.5 * radius
for _ in 0..<4
{
var x: Float = -4.5 * radius
for _ in 0..<4
{
let B: SCNNode = boxmain.clone()
B.position = SCNVector3(x: x, y: radius, z: z)
Boxes.addChildNode(B)
x += 3 * Float(radius)
}
z += 3 * Float(radius)
}
self.rootNode.addChildNode(Boxes)
}
This is tested and works great on the simulator - all devices,
on the physical devices - iPad Retina and iPhone 5.
Glitch is observed only at ultra modern iPhone 6s Plus (128 Gb).
The problem is clearly visible on the video ->
The problem with graphics can be solved by changing the Default rendering API to OpenGL ES...
...but you may have unexpected problems in pure computing modules that are not associated with graphics on iPhone 6S Plus. (the iPhone 6 has no such problems).
What's wrong?
TL;DR
Add scnView.prepareObject(boxmain, shouldAbortBlock: nil) to the end of your initBox.
I had a quick look at your code running on my 6s Plus and saw similar results. One of the corner nodes was missing, and was consistently missing each run. But we're not running the same code, mine's missing the materials data...
SceneKit is lazy, often things are not done until an object is added to a scene. I first came across this extracting geometry from a SceneKit primitive (SCNSphere etc), you're finding it when you clone a clone of something via the following lines.
let B: SCNNode = boxmain.clone()
...
boxmain = box.flattenedClone()
I'd say SceneKit is simply not completing the clone before the second clone occurs consistently. I have no way of knowing this for sure.
Removing the first clone fixes this issue for me. For example replace boxmain = box.flattenedClone() with boxmain = box. But I'd say what you've done is best practice, flattening these nodes will reduce the number of draw calls and improve performance (probably not an issue on the 6s).
SceneKit also provides a method - prepareObject:shouldAbortBlock: that will perform the operations required before an object is added to a scene (in this case the .flattenedClone()).
Adding the following line to the end of your initBox function also fixes the problem and is a better solution.
scnView.prepareObject(boxmain, shouldAbortBlock: nil)
Just say, I don't know the right answer to my question, but I found an acceptable solution for myself.
It turned out, it's all in the "diffuse" property of the SCNMaterial.
For whatever reason, Metal does not very like when diffuse = UIColor(...)
But if at least one element in a compound SCNNode (as in my case) is diffuse.contents = UIImage(...), then everything begins to work perfectly.
it works
diffuse=<SCNMaterialProperty: 0x7a6d50a0 | contents=<UIImage: 0x7a6d5b40> size {128, 128} orientation 0 scale 1.000000>
it doesn't work
diffuse=<SCNMaterialProperty: 0x7e611a50 | contents=UIDeviceRGBColorSpace 0.25 0.25 0.25 0.99>
I have found the solution of the problem is simply:
I just added one small, inconspicuous element with diffuse.contents = UIImage(...) to the existing 3 elements with diffuse.contents = UIColor(...) and it worked great.
So, my recommendations:
be careful when working with Metal. (I have a problems on the 5S devices and above)
thoroughly test the SceneKit applications on real devices, don't trust only the simulator
I hope, it's temporary bugs and it will be fix in future releases of Xcode.
Have a nice apps!
P.S. By the way, the finished app is completely free in the AppStore now
Qubic: tic-tac-toe 4x4x4