ARkit - Camera Position and 3D model Positions - ios

I am trying to put several models in the scene.
for candidate in selectedCandidate {
sceneView.scene.rootNode.addChildNode(selectedObjects[candidate])
}
The candidate and selectedCandidate stands for the index of the model I want to use. Each model contains a rootNode and nodes attached to it. I use the API worldPosition and position of SCNNode to get and modify 3D model's position.
The thing I want to do is put those models right in front users' eyes. It means I need to get the camera's position and orientation vector to put the models in the right position I want. I also use these codes to get the camera's position according to this solution https://stackoverflow.com/a/47241952/7772038:
guard let pointOfView = sceneView.pointOfView else { return }
let transform = pointOfView.transform
let orientation = SCNVector3(-transform.m31, -transform.m32, transform.m33)
let location = SCNVector3(transform.m41, transform.m42, transform.m43)
The PROBLEM is that the camera's position and the model's position I printed out directly are severely different in order of magnitude. Camera's position is 10^-2 level like {0.038..., 0.047..., 0.024...} BUT the model's position is 10^2 level like {197.28, 100.29, -79.25}. From my point of view when I run the program, I am in the middle of those models and models are very near, but the positions are so different. So can you tell me how to modify the model's position to whatever I want? I really need to put the model right in front of user's eyes. If I simply do addChildNode() the models are behind me or somewhere else, while I need the model just be in front of users' eyes. Thank you in advance!

If you want to place an SCNNode infront of the camera you can do so like this:
/// Adds An SCNNode 3m Away From The Current Frame Of The Camera
func addNodeInFrontOfCamera(){
guard let currentTransform = augmentedRealitySession.currentFrame?.camera.transform else { return }
let nodeToAdd = SCNNode()
let boxGeometry = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
boxGeometry.firstMaterial?.diffuse.contents = UIColor.red
nodeToAdd.geometry = boxGeometry
var translation = matrix_identity_float4x4
//Change The X Value
translation.columns.3.x = 0
//Change The Y Value
translation.columns.3.y = 0
//Change The Z Value
translation.columns.3.z = -3
nodeToAdd.simdTransform = matrix_multiply(currentTransform, translation)
augmentedRealityView?.scene.rootNode.addChildNode(nodeToAdd)
}
And you can change any of the X,Y,Z values as you need.
Hope it points you in the right direction...
Update:
If you have multiple nodes e.g. in a scene, in order to use this function, it's probably best to create a 'holder' node, and then add all your content as a child.
Which means then you can simply call this function on the holder node.

Related

Align a SCNNode along an axis by rotating its parent

Using Apple's RoomPlan API, I managed to create a series of nodes representing the walls of a room. However, after applying their transform, the yaw (eulerAngles.y) ends up being a random number which makes aligning other nodes not generated as part of the room tricky.
My thought was to add all the walls as children to a parent node, then rotate the parent until the largest wall had a eulerAngle.y of 90. However, I am having trouble getting it to work right.
Code for generating the walls from a RoomPlan CapturedRoom
for scannedWall in roomScan.walls {
//Generate new wall geometry
let length = 0.01
let width = scannedWall.dimensions.x
let height = scannedWall.dimensions.y
//Generate new SCNNode
let newWallGeometry = SCNBox(
width: CGFloat(width),
height: CGFloat(height),
length: CGFloat(length),
chamferRadius: 0
)
newWallGeometry.firstMaterial?.transparency = 0.6
let newWall = SCNNode(geometry: newWallGeometry)
newWall.simdTransform = scannedWall.transform
parentNode.addChildNode(newWall)
}
My initial thought was just to rotate the parent node by the difference between the longest wall's rotation and 90 degrees, but that does not seem to be correct:
parentNode.eulerAngles.y = (.pi/2 - longestWall.eulerAngles.y)
Any help would be greatly appreciated, I'm fairly new to swift/scenekit and am banging my head against a wall

How to scale and position a SCNNode that has a SCNSkinner attached?

Using SceneKit, I'm loading a very simple .dae file consisting of a large cylinder with three associated bones. I want to scale the cylinder down and position it on the ground. Here's the code
public class MyNode: SCNNode {
public convenience init() {
self.init()
let scene = SCNScene(named: "test.dae")
let cylinder = (scene?.rootNode.childNode(withName: "Cylinder", recursively: true))!
let scale: Float = 0.1
cylinder.scale = SCNVector3Make(scale, scale, scale)
cylinder.position = SCNVector3(0, scale, 0)
self.addChildNode(cylinder)
}
}
This doesn't work; the cylinder is still huge when I view it. The only way I can get the code to work is to remove associated SCNSKinner.
cylinder.skinner = nil
Why does this happen and how can I properly scale and position the model, bones and all?
when a geometry is skinned it is driven by its skeleton. Which means that the transform of the skinned node is no longer used, it's the transforms of the bones that are important.
For this file Armature is the root of the skeleton. If you translate/scale this node instead of Cylinder you'll get what you want.

Align 3D object parallel to vertical plane detected by estametedVerticalPlane

I have this book, but I'm currently remixing the furniture app from the video tutorial that was free on AR/VR week.
I would like to have a 3D wall canvas aligned with the wall/vertical plane detected.
This is proving to be harder than I thought. Positioning isn't an issue. Much like the furniture placement app you can just get the column3 of the hittest.worldtransform and provide the new geometry this vector3 for position.
But I do not know what I have to do to get my 3D object rotated to face forward on the aligned detected plane. As I have a canvas object, the photo is on one side of the canvas. On placement, the photo is ALWAYS facing away.
I thought about applying a arbitrary rotation to the canvas to face forward but that then was only correct if I was looking north and place a canvas on a wall to my right.
I'v tried quite a few solutions on line all but one always use .existingPlaneUsingExtent. for vertical plane detections. This allows for you to get the ARPlaneAnchor from the
hittest.anchor? as ARPlaneAnchor.
If you try this when using .estimatedVerticalPlane the anchor? is nil
I also didn't continue down this route as my horizontal 3D objects started getting placed in the air. This maybe down to a control flow logic but I am ignoring it until the vertical canvas placement is working.
My current train of thought is to get the front vector of the canvas and rotate it towards the front facing vector of the vertical plane detected UIImage or the hittest point.
How would I get a forward vector from a 3D point. OR get the front vector from the grid image, that is a UIImage that is placed as an overlay when ARKit detects a vertical wall?
Here is an example. The canvas is showing the back of the canvas and is not parallel with the detected vertical plane that is the column. But there is a "Place Poster Here" grid which is what I want the canvas to align with and I'm able to see the photo.
Things I have tried.
using .estimatedVerticalPlane
ARKit estimatedVerticalPlane hit test get plane rotation
I don't know how to correctly apply this matrix and eular angle results from the SO answer.
my add picture function.
func addPicture(hitTestResult: ARHitTestResult) {
// I would like to convert estimate hitTest to a anchorpoint
// it is easier to rotate a node to a anchorpoint over calculating eularAngles
// we have all detected anchors in the _Renderer SCNNode. however there are
// Get the current furniture item, correct its position if necessary,
// and add it to the scene.
let picture = pictureSettings.currentPicturePiece()
//look for the vertical node geometry in verticalAnchors
if let hitPlaneAnchor = hitTestResult.anchor as? ARPlaneAnchor {
if let anchoredNode = verticalAnchors[hitPlaneAnchor]{
//code removed as a .estimatedVerticalPlane hittestResult doesn't get here
}
}else{
// Transform hitresult to world coords
let worldTransform = hitTestResult.worldTransform
let anchoredNodeOrientation = worldTransform.eulerAngles
picture.rotation.y =
-.pi * anchoredNodeOrientation.y
//set the transform matirs
let positionMatris = worldTransform.columns.3
let position = SCNVector3 (
positionMatris.x,
positionMatris.y,
positionMatris.z
)
picture.position = position + pictureSettings.currentPictureOffset();
}
//parented to rootNode of the scene
sceneView.scene.rootNode.addChildNode(picture)
}
Thanks for any help available.
Edited:
I have notice the 'handness' or the 3D model isn't correct/ is opposite?
Positive Z is pointing to the Left and Positive X is facing the camera for what I would expects is the front of the model. Is this a issue?
You should try to avoid adding node directly into the scene using world coordinates. Rather you should notify the ARSession of an area of interest by adding an ARAnchor then use the session callback to vend an SCNNode for the added anchor.
For example your hit test might look something like:
#objc func tapped(_ sender: UITapGestureRecognizer) {
let location = sender.location(in: sender.view)
guard let hitTestResult = sceneView.hitTest(location, types: [.existingPlaneUsingGeometry, .estimatedVerticalPlane]).first,
let planeAnchor = hitTestResult.anchor as? ARPlaneAnchor,
planeAnchor.alignment == .vertical else { return }
let anchor = ARAnchor(transform: hitTestResult.worldTransform)
sceneView.session.add(anchor: anchor)
}
Here a tap gesture recognized is used to detect taps within an ARSCNView. When a tap is detected a hit test is performed looking for existing and estimated planes. If the plane is vertical, we add an ARAnchor is added with the worldTransform of the hit test result, and we add that anchor to the ARSession. This will register that point as an area of interest for the ARSession, so we'll receive better tracking and less drift after our content is added there.
Next, we need to vend our SCNNode for the newly added ARAnchor. For example
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
if anchor is ARPlaneAnchor {
let anchorNode = SCNNode()
anchorNode.name = "anchor"
return anchorNode
} else {
let plane = SCNPlane(width: 0.67, height: 1.0)
plane.firstMaterial?.diffuse.contents = UIImage(named: "monaLisa")
let planeNode = SCNNode(geometry: plane)
planeNode.eulerAngles = SCNVector3(CGFloat.pi * -0.5, 0.0, 0.0)
let node = SCNNode()
node.addChildNode(planeNode)
return node
}
}
Here we're first checking if the anchor is an ARPlaneAnchor. If it is, we vend an empty node for debugging purposes. If it is not, then it is an anchor that was added as the result of a hit test. So we create a geometry and node for the most recent tap. Because it is a vertical plane and our content is lying flat need to rotate it about the x axis. So we adjust it's eulerAngles to have it be upright. If we were to return planeNode directly adjustment to eulerAngles would be removed so we add it as a child node of an empty node and return it.
Should result in something like the following.

SceneKit move object on top of another one

I am new to SceneKit and I am programming a game.
I have loaded two objects into my scene. The first object doesn't move, only the second one. The two objects always have to stick together but the second object can move completely free on the first object's surface depending on user input (basically like two magnets with infinity power but no friction).
My approach is to take the second object's x and y coordinates and look what object one's z coordinate is at given x and y coordinates. Then I move object two to the exact same z-coordinate.
I tried using a SCNDistanceConstraint but it didn't have any effect:
let cnstrnt = SCNDistanceConstraint(target: object1)
cnstrnt.maximumDistance = 1
cnstrnt.minimumDistance = 0.99
object2?.constraints?.append(cnstrnt)
I also tried using a SCNTransformConstraint without any effect either:
let transform = SCNTransformConstraint.positionConstraint(inWorldSpace: true) { (object2, vector) -> SCNVector3 in
let z = object1?.worldPosition.z
return SCNVector3(object2.worldPosition.x, object2.worldPosition.y, z!)
}
object2?.constraints?.append(transform)
Using a hitTest only returns results that are positioned on the bounding box of the object and not its actual surface:
let hitTest = mySceneView.scene?.physicsWorld.rayTestWithSegment(from: SCNVector3((object2?.position.x)!, (object2?.position.y)!, -10), to: (object2?.position)!, options: nil)
So how can I get the z-coordinate of an 3d object's surface from a x and y coordinate? Because then I'd be able to set the new position of object2 manually.
Maybe you have another approach that is more elegant and faster than mine?
Thanks beforehand!

How to drag SCNode with finger irrespective of axis using ARKit?

I am working on an AR based application using ARKit. I am using https://developer.apple.com/documentation/arkit/handling_3d_interaction_and_ui_controls_in_augmented_reality as base for this. Using this i am able to move or rotate the whole Virtual Object.
Now there are lot of child nodes in the Virtual Object. I want to drag/move any child node with user finger irrespective of the axis. The child SCNode may be in ground or floating. I want to move the object wherever the user finger goes irrespective of the axis or irrespective of the euler angles of the child node. Is this even possible?
I followed the below links but it is just moving along a particular axis.
ARKit - Drag a node along a specific axis (not on a plane)
Dragging SCNNode in ARKit Using SceneKit
I tried using the below code and it is not at all helping,
let tapPoint: CGPoint = gesture.location(in: sceneView)
let result = sceneView.hitTest(tapPoint, options: nil)
if result.count == 0 {
return
}
let scnHitResult: SCNHitTestResult? = result.first
movedObject = scnHitResult?.node //.parent?.parent
let hitResults = self.sceneView.hitTest(tapPoint, types: .existingPlane)
if !hitResults.isEmpty{
guard let hitResult = hitResults.last else { return }
movedObject?.position = SCNVector3Make(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
}

Resources