Removing planeDetection planes prevents placing additional planes - ios

I am generating planes using ARKit and if I manually remove the plane I am unable to detect planes it the same position.
To detect planes I set the scene config
let config = ARWorldTrackingConfiguration()
config.worldAlignment = .gravity
config.providesAudioData = false
config.isLightEstimationEnabled = true
...
else if ARMode == .floor {
self.ARMode = .floor
scannerBox.isHidden=true
focusNode?.isHidden=false
plusButton.isHidden=true
config.planeDetection = .horizontal self.planeTexture="test.scnassets/Textures/Surface_Texture.png"
}
...
sceneView.session.run(config)
Add plane to the scene in renderer function
let planeNode = self.createARPlaneNode(planeAnchor: planeAnchor,
color: UIColor.yellow.withAlphaComponent(0.5))
node.addChildNode(planeNode)
If a certain button is pushed the plane and other objects should be removed
scene.rootNode.enumerateChildNodes { (node, _) in
print(node.name)
if (node.name == "sphere" ) {
node.removeFromParentNode()
}
if ( node.name == "surfacePlane") {
node.removeFromParentNode()
}
if(node.name != nil){
if (node.name?.contains("ProductModel_"))! {
node.removeFromParentNode()
}
}
}
When the above code fires the planes, spheres, and products disappear as expected.
If I try to scan the surface elsewhere in the room it works as expected but if I try to scan where the plane was and generate a new one it will not work. It will not generate a new plane and if nearby planes expand to cover the same area they disappear.
I believe the problem is likely the scene geometry of the removed planes is somehow still present and preventing new planes in the same space.
As a temporary work around I and stopping and starting the AR sessions and that removes the planes.
let config = sceneView.session.configuration as!
ARWorldTrackingConfiguration
config.planeDetection = .horizontal
sceneView.session.run(config,
options: [.resetTracking, .removeExistingAnchors])
I am trying to determine why this is the case and the planes aren't treated the same as objects I explicitly placed in the scene.

I believe you also need to remove the ARPlaneAnchor your plane node was attached to so that ARKit will create a new ARPlaneAnchor and kick off the renderer function that is creating your plane node.
Something like
if ( node.name == "surfacePlane") {
if let planeAnchor = sceneView.anchor(for: node) as? ARPlaneAnchor {
sceneView.session.remove(anchor: planeAnchor)
}
node.removeFromParentNode()
}

Related

Can't detect collision between rootNode and pointOfView child nodes in SceneKit / ARKit

In an AR app, I want to detect collisions between the user walking around and the walls of an AR node that I construct. In order to do that, I create an invisible cylinder right in front of the user and set it all up to detect collisions.
The walls are all part of a node which is a child of sceneView.scene.rootNode.
The cylinder, I want it to be a child of sceneView.pointOfView so that it would always follow the camera.
However, when I do so, no collisions are detected.
I know that I set it all up correctly, because if instead I set the cylinder node as a child of sceneView.scene.rootNode as well, I do get collisions correctly. In that case, I continuously move that cylinder node to always be in front of the camera in a renderer(updateAtTime ...) function. So I do have a workaround, but I'd prefer it to be a child of pointOfView.
Is it impossible to detect collisions if nodes are children of different root nodes?
Or maybe I'm missing something in my code?
The contactDelegate is set like that:
sceneView.scene.physicsWorld.contactDelegate = self so maybe this only includes sceneView.scene, but will exclude sceneView.pointOfView???
Is that the issue?
Here's what I do:
I have a separate file to create and configure my cylinder node which I call pov:
import Foundation
import SceneKit
func createPOV() -> SCNNode {
let pov = SCNNode()
pov.geometry = SCNCylinder(radius: 0.1, height: 4)
pov.geometry?.firstMaterial?.diffuse.contents = UIColor.blue
pov.opacity = 0.3 // will be set to 0 when it'll work correctly
pov.physicsBody = SCNPhysicsBody(type: .kinematic, shape: nil)
pov.physicsBody?.isAffectedByGravity = false
pov.physicsBody?.mass = 1
pov.physicsBody?.categoryBitMask = BodyType.cameraCategory.rawValue
pov.physicsBody?.collisionBitMask = BodyType.wallsCategory.rawValue
pov.physicsBody?.contactTestBitMask = BodyType.wallsCategory.rawValue
pov.simdPosition = simd_float3(0, -1.5, -0.3) // this position only makes sense when setting as a child of pointOfView, otherwise the position will always be changed by renderer
return pov
}
Now in my viewController.swift file I call this function and set is as a child of either root nodes:
pov = createPOV()
sceneView.pointOfView?.addChildNode(pov!)
(Don't worry right now about not checking and unwrapping).
The above does not detect collisions.
But if instead I add it like so:
sceneView.scene.rootNode.addChildNode(pov!)
then collisions are detected just fine.
But then I need to always move this cylinder to be in front of the camera and I do it like that:
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
guard let pointOfView = sceneView.pointOfView else {return}
let currentPosition = pointOfView.simdPosition
let currentTransform = pointOfView.simdTransform
let orientation = SCNVector3(-currentTransform.columns.2.x, -currentTransform.columns.2.y, -currentTransform.columns.2.z)
let currentPositionOfCamera = orientation + SCNVector3(currentPosition)
DispatchQueue.main.async {
self.pov?.position = currentPositionOfCamera
}
}
For completeness, here's the code I use to configure the node of walls in ViewController (they're built elsewhere in another function):
node?.physicsBody = SCNPhysicsBody(type: .dynamic, shape: SCNPhysicsShape(node: node!, options: nil))
node?.physicsBody?.isAffectedByGravity = false
node?.physicsBody?.mass = 1
node?.physicsBody?.damping = 1.0 // remove linear velocity, needed to stop moving after collision
node?.physicsBody?.angularDamping = 1.0 // remove angular velocity, needed to stop rotating after collision
node?.physicsBody?.velocityFactor = SCNVector3(1.0, 0.0, 1.0) // will allow movement only in X and Z coordinates
node?.physicsBody?.angularVelocityFactor = SCNVector3(0.0, 1.0, 0.0) // will allow rotation only around Y axis
node?.physicsBody?.categoryBitMask = BodyType.wallsCategory.rawValue
node?.physicsBody?.collisionBitMask = BodyType.cameraCategory.rawValue
node?.physicsBody?.contactTestBitMask = BodyType.cameraCategory.rawValue
And here's my physycsWorld(didBegin contact) code:
func physicsWorld(_ world: SCNPhysicsWorld, didBegin contact: SCNPhysicsContact) {
if contact.nodeA.physicsBody?.categoryBitMask == BodyType.wallsCategory.rawValue || contact.nodeB.physicsBody?.categoryBitMask == BodyType.wallsCategory.rawValue {
print("Begin COLLISION")
contactBeginLabel.isHidden = false
}
}
So I print something to the console and I also turn on a label on the view so I'll see that collision was detected (and the walls indeed move as a whole when it works).
So Again, it all works fine when the pov node is a child of sceneView.scene.rootNode, but not if it's a child of sceneView.pointOfView.
Am I doing something wrong or is this a limitation of collision detection?
Is there something else I can do to make this work, besides the workaround I already implemented?
Thanks!
Regarding the positioning of your cyliner:
instead to use the render update at time, you better use a position constraint for your cylinder node to move with the point of view. the result will be the same, as if it were a child of the point of view, but collisions will be detected, because you add it to the main rootnode scenegraph.
let constraint = SCNReplicatorConstraint(target: pointOfView) // must be a node
constraint.positionOffset = positionOffset // some SCNVector3
constraint.replicatesOrientation = false
constraint.replicatesScale = false
constraint.replicatesPosition = true
cylinder.constraints = [constraint]
There is also an influence factor you can configure. By default the influence is 100%, the position will immediatly follow.

Node clones not maintaining geometry despite copying it over in iOS 12

I have a little painting app in SceneKit that I originally developed on iOS 9. There's a part where I take a collection of nodes, clone them, then flatten them to make one single node that can be moved around.
Problem is, in iOS 9 I was able to successfully do the "deep clone" technique where I copy over the node's geometry and materials, and these properties were retained. This is not the case in iOS 12, however. The geometry gets all wonky despite running the exact same code. I've spent a few hours trying different variations of the process below, to no avail. Here's the code:
func sendPainting() {
// a parent node that contains the canvas and frame nodes as children
guard let configPainting = sceneView.scene!["configpainting"] else {
return
}
guard let configCanvas = configPainting.childNode(withName: "canvas", recursively: true) else {
return
}
guard let configFrame = configPainting.childNode(withName: "frame", recursively: true) else {
return
}
let configClone = configPainting.clone()
// copy over geometry and material
configClone.childNode(withName: "canvas", recursively: true)?.geometry = configCanvas.geometry?.copy() as? SCNGeometry
configClone.childNode(withName: "canvas", recursively: true)?.geometry?.firstMaterial = configCanvas.geometry?.firstMaterial?.copy() as? SCNMaterial
configClone.childNode(withName: "frame", recursively: true)?.geometry = configFrame.geometry?.copy() as? SCNGeometry
configClone.childNode(withName: "frame", recursively: true)?.geometry?.firstMaterial = configFrame.geometry?.firstMaterial?.copy() as? SCNMaterial
// make a flattened clone to now have the canvas and frame consolidated into 1 node
let localClone = configClone.flattenedClone()
...
// add to my scene
sceneView.scene!["localpaintings"]?.addChildNode(localClone)
}
Here's what the group of nodes look like before cloning and flattening, in both iOS 9 and 12:
And after:
In iOS 9, geometry is retained (sorry, different image, but you get the idea):
In iOS 12, the frame gets shrunk and rotated:

How to drag SCNode with finger irrespective of axis using ARKit?

I am working on an AR based application using ARKit. I am using https://developer.apple.com/documentation/arkit/handling_3d_interaction_and_ui_controls_in_augmented_reality as base for this. Using this i am able to move or rotate the whole Virtual Object.
Now there are lot of child nodes in the Virtual Object. I want to drag/move any child node with user finger irrespective of the axis. The child SCNode may be in ground or floating. I want to move the object wherever the user finger goes irrespective of the axis or irrespective of the euler angles of the child node. Is this even possible?
I followed the below links but it is just moving along a particular axis.
ARKit - Drag a node along a specific axis (not on a plane)
Dragging SCNNode in ARKit Using SceneKit
I tried using the below code and it is not at all helping,
let tapPoint: CGPoint = gesture.location(in: sceneView)
let result = sceneView.hitTest(tapPoint, options: nil)
if result.count == 0 {
return
}
let scnHitResult: SCNHitTestResult? = result.first
movedObject = scnHitResult?.node //.parent?.parent
let hitResults = self.sceneView.hitTest(tapPoint, types: .existingPlane)
if !hitResults.isEmpty{
guard let hitResult = hitResults.last else { return }
movedObject?.position = SCNVector3Make(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
}

Create UIBezierPath shape in 3D world ARKit

I'm making an app where the user can create some flat shapes by positioning some points on a 3D space with ARKit, but it seems that the part where I create the UIBezierPath using these points is problematic.
In my app, the user starts by positioning a virtual transparent wall in AR at the same place that his device by pressing a button:
guard let currentFrame = sceneView.session.currentFrame else {
return
}
let imagePlane = SCNPlane(width: sceneView.bounds.width, height: sceneView.bounds.height)
imagePlane.firstMaterial?.diffuse.contents = UIColor.black
imagePlane.firstMaterial?.lightingModel = .constant
var windowNode = SCNNode()
windowNode.geometry = imagePlane
sceneView.scene.rootNode.addChildNode(windowNode)
windowNode.simdTransform = currentFrame.camera.transform
windowNode.opacity = 0.1
Then, the user place some points (some sphere nodes) on that wall to determine the shape of the flat object that he wants to create by pressing a button. If the user points back to the first sphere node created, I close the shape, create a node of it and place it at the same position that the wall:
let hitTestResult = sceneView.hitTest(self.view.center, options: nil)
if let firstHit = hitTestResult.first {
if firstHit.node == windowNode {
let x = Double(firstHit.worldCoordinates.x)
let y = Double(firstHit.worldCoordinates.y)
let pointCoordinates = CGPoint(x: x , y: y)
let sphere = SCNSphere(radius: 0.02)
sphere.firstMaterial?.diffuse.contents = UIColor.white
sphere.firstMaterial?.lightingModel = .constant
let sphereNode = SCNNode(geometry: sphere)
sceneView.scene.rootNode.addChildNode(sphereNode)
sphereNode.worldPosition = firstHit.worldCoordinates
if points.isEmpty {
windowPath.move(to: pointCoordinates)
} else {
windowPath.addLine(to: pointCoordinates)
}
points.append(sphereNode)
if undoButton.alpha == 0 {
undoButton.alpha = 1
}
} else if firstHit.node == points.first {
windowPath.close()
let windowShape = SCNShape(path: windowPath, extrusionDepth: 0)
windowShape.firstMaterial?.diffuse.contents = UIColor.white
windowShape.firstMaterial?.lightingModel = .constant
let tintedWindow = SCNNode(geometry: windowShape)
let worldPosition = windowNode.worldPosition
tintedWindow.worldPosition = worldPosition
sceneView.scene.rootNode.addChildNode(tintedWindow)
//removing all the sphere nodes from points and reinitializing the UIBezierPath windowPath
removeAllPoints()
}
}
That code works when I create a first invisible wall and a first shape, but when I create a second wall, when I'm done to draw my shape, the shape appears to be deformed and not at the right place like really not at the right place at all. So I think that I'm missing something with the coordinates of my UIBezierPath points but what ?
EDIT
Ok so after several tests, it seems that it depends on the orientation of the device at the launch of the AR session. When the device, at launch, faces the first wall that the user will create, the shape is created and places as expected. But if the user for exemple launch the app with his device pointed in one direction, then do a rotation of 90 degrees on himself, place the first wall and create his shape, the shape will be deformed and not at the right place.
So it seems that it's a problem of 3D coordinates but I still don't figure it out.
Ok I just found the problem ! I was just using the wrong vectors and coordinates... I've never been a math/geometry guy haha
So instead of using:
let x = Double(firstHit.worldCoordinates.x)
let y = Double(firstHit.worldCoordinates.y)
I now use:
let x = Double(firstHit.localCoordinates.x)
let y = Double(firstHit.localCoordinates.y)
And instead of using:
let worldPosition = windowNode.worldPosition
I now use:
let worldPosition = windowNode.transform
That's why the position of my shape node was depending of the initialisation of the AR session, I was working with world coordinates, seems obvious to me now.

ARKit Place a SCNNode facing the camera

I'm using ARKit to display 3D objects. I managed to place the nodes in the real world in front of the user (aka the camera). But I don't manage to make them to face the camera when I drop them.
let tap_point=CGPoint(x: x, y: y)
let results=arscn_view.hitTest(tap_point, types: .estimatedHorizontalPlane)
guard results.count>0 else{
return
}
guard let r=results.first else{
return
}
let hit_tf=SCNMatrix4(r.worldTransform)
let new_pos=SCNVector3Make(hit_tf.m41, hit_tf.m42+Float(0.2), hit_tf.m43)
guard let scene=SCNScene(named: file_name) else{
return
}
guard let node=scene.rootNode.childNode(withName: "Mesh", recursively: true) else{
return
}
node.position=new_pos
arscn_view.scene.rootNode.addChildNode(node)
The nodes are well positioned on the plane, in front of the camera. But they are all looking in the same direction. I guess I should rotate the SCNNode but I didn't manage to do this.
First, get the rotation matrix of the camera:
let rotate = simd_float4x4(SCNMatrix4MakeRotation(sceneView.session.currentFrame!.camera.eulerAngles.y, 0, 1, 0))
Then, combine the matrices:
let rotateTransform = simd_mul(r.worldTransform, rotate)
Lastly, apply a transform to your node, casting as SCNMatrix4:
node.transform = SCNMatrix4(rotateTransform)
Hope that helps
EDIT
here how you can create SCNMatrix4 from simd_float4x4
let rotateTransform = simd_mul(r.worldTransform, rotate)
node.transform = SCNMatrix4(m11: rotateTransform.columns.0.x, m12: rotateTransform.columns.0.y, m13: rotateTransform.columns.0.z, m14: rotateTransform.columns.0.w, m21: rotateTransform.columns.1.x, m22: rotateTransform.columns.1.y, m23: rotateTransform.columns.1.z, m24: rotateTransform.columns.1.w, m31: rotateTransform.columns.2.x, m32: rotateTransform.columns.2.y, m33: rotateTransform.columns.2.z, m34: rotateTransform.columns.2.w, m41: rotateTransform.columns.3.x, m42: rotateTransform.columns.3.y, m43: rotateTransform.columns.3.z, m44: rotateTransform.columns.3.w)
guard let frame = self.sceneView.session.currentFrame else {
return
}
node.eulerAngles.y = frame.camera.eulerAngles.y
here's my code for the SCNNode facing the camera..hope help for someone
let location = touches.first!.location(in: sceneView)
var hitTestOptions = [SCNHitTestOption: Any]()
hitTestOptions[SCNHitTestOption.boundingBoxOnly] = true
let hitResultsFeaturePoints: [ARHitTestResult] = sceneView.hitTest(location, types: .featurePoint)
let hitTestResults = sceneView.hitTest(location)
guard let node = hitTestResults.first?.node else {
if let hit = hitResultsFeaturePoints.first {
let rotate = simd_float4x4(SCNMatrix4MakeRotation(sceneView.session.currentFrame!.camera.eulerAngles.y, 0, 1, 0))
let finalTransform = simd_mul(hit.worldTransform, rotate)
sceneView.session.add(anchor: ARAnchor(transform: finalTransform))
}
return
}
Do you want the nodes to always face the camera, even as the camera moves? That's what SceneKit constraints are for. Either SCNLookAtConstraint or SCNBillboardConstraint can keep a node always pointing at the camera.
Do you want the node to face the camera when placed, but then hold still (so you can move the camera around and see the back of it)? There are a few ways to do that. Some involve fun math, but a simpler way to handle it might just be to design your 3D assets so that "front" is always in the positive Z-axis direction. Set a placed object's transform based on the camera transform, and its initial orientation will match the camera's.
Here's how I did it:
func faceCamera() {
guard constraints?.isEmpty ?? true else {
return
}
SCNTransaction.begin()
SCNTransaction.animationDuration = 5
SCNTransaction.completionBlock = { [weak self] in
self?.constraints = []
}
constraints = [billboardConstraint]
SCNTransaction.commit()
}
private lazy var billboardConstraint: SCNBillboardConstraint = {
let constraint = SCNBillboardConstraint()
constraint.freeAxes = [.Y]
return constraint
}()
As stated earlier a SCNBillboardConstraint will make the node always look at the camera. I am animating it so the node doesn't just immediately snap into place, this is optional. In the SCNTransaction.completionBlock I remove the constraint, also optional.
Also I set the SCNBillboardConstraint's freeAxes, which customizes on what axis the node follows the camera, again optional.
I want the node to face the camera when I place it then keep it here (and be able to move around). – Marie Dm
Blockquote
You can put object facing to camera, using this:
if let rotate = sceneView.session.currentFrame?.camera.transform {
node.simdTransform = rotate
}
This code will save you from gimbal lock and other troubles.
The four-component rotation vector specifies the direction of the rotation axis in the first three components and the angle of rotation (in radians) in the fourth. The default rotation is the zero vector, specifying no rotation. Rotation is applied relative to the node’s simdPivot property.
The simdRotation, simdEulerAngles, and simdOrientation properties all affect the rotational aspect of the node’s simdTransform property. Any change to one of these properties is reflected in the others.
https://developer.apple.com/documentation/scenekit/scnnode/2881845-simdrotation
https://developer.apple.com/documentation/scenekit/scnnode/2881843-simdtransform

Resources