Need starting and ending position of MeshResource/Box in RealityKit - ios

let material = SimpleMaterial.init(color: .red,roughness: 1,isMetallic: false)
let doorBox = MeshResource.generateBox(width: 0.02,height: 1, depth: 0.5)
let doorEntity = ModelEntity(mesh: doorBox, materials: [material])
let anchor = ARAnchorEntity()
anchor.addChild(doorEntity)
In RealityKit, I am having box which is MeshResource, Box looks like a line. I have added this box in ARView, and have set realtime camera position. In one scenario I want to know Box/Line’s starting and ending position.
Lets say box with entity has middle/current position (0.1,0.23,-1.3) then what will be box’s left and right position ? Anchor with box is keep changing it's position with camera movement.
Thanks in advance.
Check explanation with the image

You can use this extension.
extension Entity {
func getDistancedPosition(x: Float, y: Float, z: Float) -> SIMD3<Float> {
let referenceNodeTransform = transform.matrix
var translationMatrix = matrix_identity_float4x4
translationMatrix.columns.3.x = x
translationMatrix.columns.3.y = y
translationMatrix.columns.3.z = z
let updatedTransform = matrix_multiply(referenceNodeTransform,
translationMatrix)
return .init(updatedTransform.columns.3.x,
updatedTransform.columns.3.y,
updatedTransform.columns.3.z)
}
}
To get left and right for your box, Use below code:
let side1Position = door.getDistancedPosition(x: 0, y: 0, z: self.viewModel.doorDepth/2)
let side2Position = door.getDistancedPosition(x: 0, y: 0, z: -(self.viewModel.doorDepth/2))
To make the box look like the line you must have used depth. If not then you can change the parameter accordingly. e.g. door.getDistancedPosition(x: -0.1, y: 0, z: 0)
You can also refer to this question and its accepted answer:
Position a SceneKit object in front of SCNCamera's current orientation

Related

SceneKit unprojectPoint() with custom camera

I am developing a molecular visualizer for macOS / iPadOS with SceneKit. Long story short, I want that when the user clicks (or touches) the screen at a certain position, a new atom is placed (in this example just a SCNSphere).
Previously, I had the allowsCameraControl property of the SCNView active, which allowed me to freely move the camera and with the unprojectPoint() method, I could successfully place a new node at touch location. The limitation of the default camera controller is that it does not zoom. When you pinch the screen, it changes the FOV property of the camera instead of moving it through the Z axis.
Therefore, I made a custom camera node with a SCNCamera. I succesfully recreated the default camera behaviour (movement, rotation) and furthermore I am able to correclty zoom into the scene. The downside of this is that the unprojectPoint() method no longer works as expeced, as the new nodes are placed at a very close position of the camera node itself. No matter where I click on the scene, that the unprojected point will always be very close to 0, 0, 10
internal func newNodeAt(point: CGPoint) {
let pointVector = SCNVector3(point.x, point.y, 0.8)
let position = self.unprojectPoint(pointVector)
print("x:\(position.x), y: \(position.y), z: \(position.z)")
let newSphere = SCNSphere(radius: 1)
let newNode = SCNNode(geometry: newSphere)
self.scene?.rootNode.addChildNode(newNode)
}
The camera node is setup as folows and its directly attached to the scene root node.
internal func setupCameraNode() -> SCNNode {
let cam = SCNCamera()
cam.name = "camera"
cam.zFar = 200
cam.zNear = 0.1
let camNode = SCNNode()
camNode.camera = cam
camNode.position = SCNVector3(0, 0, 5)
camNode.name = "Camera node"
return camNode
}
These are the printed positions after clicking on random positions of the scene.
x:-0.1988764852285385, y: -0.05589345842599869, z: 10.920427322387695
x:-0.18989555537700653, y: 0.14564114809036255, z: 10.920427322387695
x: 0.2168566882610321, y: 0.13085339963436127, z: 10.920427322387695
x: 0.24202580749988556, y: -0.15493911504745483, z: 10.920427322387695
x:-0.06516486406326294, y: -0.1781780868768692, z: 10.920427322387695
x:-0.08134553581476212, y: 0.12478446960449219, z: 10.920427322387695
x:-0.25866374373435974, y: 0.1456427276134491, z: 10.920427322387695
x: 0.217658132314682, y: 0.16270162165164948, z: 10.920427322387695
x: 0.2053154855966568, y: -0.12679903209209442, z: 10.920427322387695
I suppose that the unprojectPoint() is somehow related to the point of view? But I do not know how to fix this. Thanks.
I think you are on the right track, you just have to provide some kind of depth reference for the user. This is my code for similar, but when I call airStrike, I deal with the depth based on a plane facing the user and that's how I know where Z needs to be.
Just a guess without a visual, but seems there are a couple of options. Create a reference plane in the middle of the molecule and ++/-- that to show where the tap will land from a depth perpective.
Or just let them put it anywhere, then select it and depth++/depth-- to get it in the right position.
#objc func handleTap(recognizer: UITapGestureRecognizer)
{
let location: CGPoint = recognizer.location(in: gameScene)
if(data.isAirStrikeModeOn == true)
{
let projectedPoint = gameScene.projectPoint(SCNVector3(0, 0, 0))
let scenePoint = gameScene.unprojectPoint(SCNVector3(location.x, location.y, CGFloat(projectedPoint.z)))
gameControl.airStrike(position: scenePoint)
}
}
After days of testing I figured out a workaround and now I can place the nodes correctly where they should be.
My node tree was is like this:
RootNode
CameraNode
atomNodes
atom (individual spheres)
Therefore, all I had to do was to convert the unprojected position from the RootNode (which I suppose is the one that the camera takes the reference from) to the atomNodes, thus:
let unprojected = unprojectPoint(SCNVector3(location.x, location.y, 0.99))
let position = atomNodes.convertPosition(unprojected, from: rootNode)
The 0.99 is just a nice Z position in my view for the spheres to be placed. (More info here)
My advice would be to always check the node tree because the positions are relative to each other.

Change "field of view" in SceneKit

I have some camera with perspective projection and object in the (0,0,0).
I computed "bounding frame"(size in pixels on screen) of this object according to this camera and scaled it to fit the screen size. And now it in the middle of screen.
Frame of the scnView is equal to the screen size and I have some UI components above this view(e.g. navigation bar and some big transparent view in the bottom of view). The scnView has some fullscreen background and this object can be scaled/moved/... therefore I need scnview to be fullscreen.
Then I move this object in the middle of "clear area" (the area with absence of UIComponents) and see the bottom of this object (as if it's above me), because my projection is perspective.
I want to move this object in the center of clear area and see like it's in front of me without any distortion. How can I achieve that?
I see 2 solutions.
1) Draw my object offscreen in texture and after that draw this texture on screen in required position.
2) create scnView a bit taller. For example set frame size to = (0, (bottomHeight - topHeight) / 2, width, height + (bottomHeight - topHeight) / 2) to move the center of scnView in required position.
But I don't like first solution due to addition draw call and second solution even sounds crappy.
P.S. sorry for that pictures
Update: Basically I want to use perspective projection and move 3D object like a 2D image in SceneKit.
Sorry if I'm not be understanding correctly, but can you strafe camera up and center it, or will that not be clear enough?
func strafeY(vAmount: XFloat)
{
nPosition = cgVecAdd(v1: nPosition, v2: cgVecScalarMult(v: nTarget, s: vAmount))
}
Try this... generate a new scenekit game project, then replace the default rotation with the code below.
Rotate X slightly as you move the object up to to "somewhat" maintain the relationship to the eye.
You could create a similar but separate routine to do smaller increments to smooth it out.
ship.runAction(SCNAction.rotateBy(x: CGFloat(GLKMathDegreesToRadians(-90)), y: 0, z: 0, duration: 0))
let vRotateAmount: Float = 3
let vAction1 = SCNAction.move(to: SCNVector3Make(0, 1, 0), duration: 1)
let vAction1a = SCNAction.rotateBy(x: CGFloat(GLKMathDegreesToRadians(vRotateAmount)), y:0, z:0, duration: 1)
let vAction2 = SCNAction.move(to: SCNVector3Make(0, 2, 0), duration: 1)
let vAction2a = SCNAction.rotateBy(x: CGFloat(GLKMathDegreesToRadians(vRotateAmount)), y:0, z:0, duration: 1)
let vAction3 = SCNAction.move(to: SCNVector3Make(0, 1, 0), duration: 1)
let vAction3a = SCNAction.rotateBy(x: CGFloat(GLKMathDegreesToRadians(-vRotateAmount)), y:0, z:0, duration: 1)
let vAction4 = SCNAction.move(to: SCNVector3Make(0, 0, 0), duration: 1)
let vAction4a = SCNAction.rotateBy(x: CGFloat(GLKMathDegreesToRadians(-vRotateAmount)), y:0, z:0, duration: 1)
let seq = SCNAction.sequence([vAction1, vAction1a, vAction2, vAction2a, vAction3, vAction3a, vAction4, vAction4a])
let allSeq = SCNAction.repeatForever(seq)
ship.runAction(allSeq)
Solved it by changing camera projectionTransform. Smth like:
let newMatrix = SCNMatrix4Mult(initialProjectionCameraMatrix, translationMatrix)

SceneKit matrix transformation to match camera angle

I'm building a UIPanGestureRecognizer so I can move nodes in 3D space.
Currently, I have something that works, but only when the camera is exactly perpendicular to the plane, my UIPanGestureRecognizer looks like this:
#objc func handlePan(_ sender:UIPanGestureRecognizer) {
let projectedOrigin = self.sceneView!.projectPoint(SCNVector3Zero)
let viewCenter = CGPoint(
x: self.view!.bounds.midX,
y: self.view!.bounds.midY
)
let touchlocation = sender.translation(in: self.view!)
let moveLoc = CGPoint(
x: CGFloat(touchlocation.x + viewCenter.x),
y: CGFloat(touchlocation.y + viewCenter.y)
)
let touchVector = SCNVector3(x: Float(moveLoc.x), y: Float(moveLoc.y), z: Float(projectedOrigin.z))
let worldPoint = self.sceneView!.unprojectPoint(touchVector)
let loc = SCNVector3( x: worldPoint.x, y: 0, z: worldPoint.z )
worldHandle?.position = loc
}
The problem happens when the camera is rotated, and the coordinates are effected by the perspective change. Here is you can see the touch position drifting:
Related SO post for which I used to get to this position:
How to use iOS (Swift) SceneKit SCNSceneRenderer unprojectPoint properly
It referenced these great slides: http://www.terathon.com/gdc07_lengyel.pdf
The tricky part of going from 2D touch position to 3D space is obviously the z-coordinate. Instead of trying to convert the touch position to an imaginary 3D space, map the 2D touch to a 2D plane in that 3D space using a hittest. Especially when movement is required only in two direction, for example like chess pieces on a board, this approach works very well. Regardless of the orientation of the plane and the camera settings (as long as the camera doesn't look at the plane from the side obviously) this will map the touch position to a 3D position directly under the finger of the touch and follow consistently.
I modified the Game template from Xcode with an example.
https://github.com/Xartec/PrecisePan/
The main parts are:
the pan gesture code:
// retrieve the SCNView
let scnView = self.view as! SCNView
// check what nodes are tapped
let p = gestureRecognize.location(in: scnView)
let hitResults = scnView.hitTest(p, options: [SCNHitTestOption.searchMode: 1, SCNHitTestOption.ignoreHiddenNodes: false])
if hitResults.count > 0 {
// check if the XZPlane is in the hitresults
for result in hitResults {
if result.node.name == "XZPlane" {
//NSLog("Local Coordinates on XZPlane %f, %f, %f", result.localCoordinates.x, result.localCoordinates.y, result.localCoordinates.z)
//NSLog("World Coordinates on XZPlane %f, %f, %f", result.worldCoordinates.x, result.worldCoordinates.y, result.worldCoordinates.z)
ship.position = result.worldCoordinates
ship.position.y += 1.5
return;
}
}
}
The addition of a XZ plane node in viewDidload:
let XZPlaneGeo = SCNPlane(width: 100, height: 100)
let XZPlaneNode = SCNNode(geometry: XZPlaneGeo)
XZPlaneNode.geometry?.firstMaterial?.diffuse.contents = UIImage(named: "grid")
XZPlaneNode.name = "XZPlane"
XZPlaneNode.rotation = SCNVector4(-1, 0, 0, Float.pi / 2)
//XZPlaneNode.isHidden = true
scene.rootNode.addChildNode(XZPlaneNode)
Uncomment the isHidden line to hide the helper plane and it will still work. The plane obviously needs to be large enough to fill the screen or at least the portion where the user is allowed to pan.
By setting a global var to hold a startWorldPosition of the pan (in state .began) and comparing it to the hit worldPosition in the state .change you can determine the delta/translation in world space and translate other objects accordingly.

How to apply SCNVector3 force/impulse from an orientation in scenekit?

In ARKit/SceneKit, when the user taps the button, I want to apply an impulse to my node. I want the impulse to come from the current user's perspective. This means the node would be moving away from the user's perspective. I'm able to get the current orientation/direction, thanks to this code:
func getUserVector() -> (SCNVector3, SCNVector3) { // (direction, position)
if let frame = self.sceneView.session.currentFrame {
let mat = SCNMatrix4(frame.camera.transform) // 4x4 transform matrix describing camera in world space
let dir = SCNVector3(-1 * mat.m31, -1 * mat.m32, -1 * mat.m33) // orientation of camera in world space
let pos = SCNVector3(mat.m41, mat.m42, mat.m43) // location of camera in world space
return (dir, pos)
}
return (SCNVector3(0, 0, -1), SCNVector3(0, 0, -0.2))
}
via https://github.com/farice/ARShooter/blob/master/ARViewer/ViewController.swift#L191
I have an arbitrary SCNVector, that I've created. It contains info on how high (Y axis), how much to the left or right, and how much forward to apply to the node.
I want to convert/translate my SCNVector3 to come from the orientation/direction of the camera.
Meaning, I have
let (direction, position) = self.getUserVector()
let force = SCNVector3(x: 1.67, y: 13.83, z: -18.3)
How do I apply the force from the location/origin of the direction?
Figured it out after lots of googling. To convert the impulse vector3 to the direction I need, I used something like this:
let original = SCNVector3(x: 1.67, y: 13.83, z: -18.3)
let force = simd_make_float4(original.x, original.y, original.z, 0)
let rotatedForce = simd_mul(currentFrame.camera.transform, force)
let vectorForce = SCNVector3(x:rotatedForce.x, y:rotatedForce.y, z:rotatedForce.z)
node.physicsBody?.applyForce(vectorForce, asImpulse: true)

checking to see if location in scene is occupied by sprite

I am building my first iPhone game using Xcode, SpriteKit and Swift. I am new to these technologies but I am familiar with general programming concepts.
Here is what I am trying to do in English. I want circles to randomly appear on the screen and then begin to expand in size. However, I do not want a circle to appear in a location where a circle currently exists. I am having trouble determining each circle's position.
Inside GameScene.swift I have the following code inside the didMoveToView:
runAction(SKAction.repeatActionForever(
SKAction.sequence([
SKAction.runBlock(addCircle), SKAction.waitForDuration(3, withRange: 2)]
)))
The piece of code above calls my "addCircle" method:
func addCircle() {
// Create sprite.
let circle = SKSpriteNode(imageNamed: "grad640x640_circle")
circle.name = "circle"
circle.xScale = 0.1
circle.yScale = 0.1
// Determine where to position the circle.
let posX = random(min: 50, max: 270)
let posY = random(min: 50, max: 518)
// ***Check to see if position is currently occupied by another circle here.
circle.position = CGPoint(x: posX, y: posY)
// Add circle to the scene.
addChild(circle)
// Expand the circle.
let expand = SKAction.scaleBy(2, duration: 0.5)
circle.runAction(expand)
}
The random function above just chooses a random number within the given range. How can I check to see if my random functions are generating a location that is currently occupied by another circle?
I was thinking of using a do..while loop to randomly generate a set of x and y coordinates and then check to see if a circle is at that location but I cannot find how to check for that condition. Any help would be greatly appreciated.
There are a few methods which can help you in this regard:
(BOOL) intersectsNode: (SKNode*)node, available with SKNode, and
CGRectContainsPoint() as well as the CGRectContainsRect() methods.
For instance the loop to check for intersection can look as follows:
var point: CGPoint
var exit:Bool = false
while (!exit) {
let posX = random(min: 50, max: 270)
let posY = random(min: 50, max: 518)
point = CGPoint(x: posX, y: posY)
var pointFound: Bool = true
self.enumerateChildNodesWithName("circle", usingBlock: {
node, stop in
let sprite:SKSpriteNode = node as SKSpriteNode
if (CGRectContainsPoint(sprite.frame, point))
{
pointFound = false
stop.memory = true
}
})
if (pointFound)
{
exit = true
}
}
//point contains CGPoint where no other circle exists
//Declare new circle at point

Resources