I'm new to ARKit. I want to get the direction from anchor 1 to anchor 2. Currently, I can get the position from transform.columns.3. However, this works only for fixed axis.(z-axis always toward user)
How can I compare two anchor with respect to 6 axes (pitch, yaw, roll)? What should I read to get more detail information about this?
func showDirection(of object: ARAnchor) { // only work for fixed axis
if let currentFrame = sceneView.session.currentFrame {
print("diff(x) = \(currentFrame.camera.transform.columns.3.x - object.transform.columns.3.x)")
print("diff(y) = \(currentFrame.camera.transform.columns.3.y - object.transform.columns.3.y)")
print("diff(z) = \(currentFrame.camera.transform.columns.3.z - object.transform.columns.3.z)")
}
}
I think my answer to this other user's question may be helpful. Basically, using SceneKit or ARKit, you can find the orientations of the camera and of your target anchor, and do some quaternion math to find the axis and angle of the relative rotation between them on x, y and z axes. My example assumed a SceneKit/ARKit app, which allows you to use quaternions instead of matrices, but the math should essentially be the same for ARKit transforms. If you use ARKit's simd_float4x4 transform matrices, you could find one matrix in the space of the other (A.inverse * B) and use the resulting matrix to glean relative position and orientation.
Your question was a little hard to follow, as I'm not sure if the orientation of the anchor you're targeting matters in your case, but this should help as far as comparing two anchors with respect to pitch, yaw and roll.
Related
I'd like to simulate the shift of a tilt-shift/perspective-control lens in Scene Kit on MacOS.
Imagine the user has the camera facing a tall building at ground level, I'd like to be able to shift the 'lens' so that the projective distortion shifts (see e.g. Wikipedia).
Apple provides lots of physically-based parameters for SCNCamera (sensor height, aperture blade count), but I can't see anything obvious for this. It seems to exist in Unity.
Crucially I'd like to shift the lens so that the object stays in the same position relative to the camera. Obviously I could move the camera to get the effect, but the object needs to stay centred in the viewport (and I can't see a way to modify the viewport either). I've tried to modify the .projectionTransform matrix directly, but it was unsuccessful.
Thanks!
There's is no API on SCNCamera that does that out of the box. As you guessed one has to create a custom projection matrix and set it to the projectionTransform property.
I finally worked out the correct adjustment to the projection matrix – it's quite confusing to follow the maths, because it is a 4x4 matrix rather than 3x4 or 4x3 as you'd use for a plain camera projection matrix, which additionally makes it especially confusing to work out whether it is expecting row vectors or column vectors.
Anyway, the correct element is .m32 for the y axis
let camera = SCNNode()
camera.camera = SCNCamera()
let yShift: CGFloat = 1.0
camera.camera!.projectionTransform.m32 = yShift
Presumably .m31 will shift in the x axis, but I have to admit I haven't tested this.
When I thought about it a bit more, I also realised that the effect I actually wanted involves moving the camera too. Adjusting .m32 simulates moving the sensor, which will appear to move the subject relative to the camera, as if you had a wide angle lens and you were moving the crop. To keep the subject centred in frame, you need to move the camera's position too.
With a bit (a lot) of help from this blog post and in particular this code, I implemented this too:
let distance: CGFloat = 1.0 // calculate distance from subject here
let fovRadians = camera.camera!.fieldOfView * CGFloat.pi / 180.0
let yAdjust = tan(fovRadians / 2) * distance * yShift
camera.position = camera.position - camera.worldUp * yAdjust
(any interested readers could presumably work out the x axis shift from the source above)
i'm using arkit and realitykit,first load a room ,then put the picture on the wall ,but how to let it stick on the wall?
how to get the right rotation of the picture?
There are four possibilities,wall's front: z+, z-, x+, x-,
SceneKit
Answering your first question, to make the node stick to a particular place you need to just keep a desired location and add the node, it should not disappear unless you remove it. It may move up and down a little when you change phone orientation and position, but not much
node.position = SCNVector3(x, y, z)
If you are using touches to define the location, I would recommend you considering sceneView.hitTest() and touchesBegan() functions
As for rotation, you simply can use Euler angles on a node of your interest along any axis, and in any needed direction
node.eulerAngles.x = -.pi / 2
Also, I would highly recommend you “App Development with Swift” by Apple Education, 2019. In this book there is a whole chapter on ARKit, besides answering your questions, it has numerous useful techniques and ideas
Here you can find the implementation of the end-of-the-chapter guided project from the App Development with Swift book, but doing it yourself would be much more useful
RealityKit
As given here, you can use one of two options for rotation
let boxAnchor = try! Experience.loadBox()
boxAnchor.steelBox?.orientation = simd_quatf(angle: .pi/4, axis: [0, 0, 1])
For angle property you insert how much you want to rotate the object in radians, and for axis property you select the axis that you want to rotate around.
Another option is to use transform
boxAnchor.steelBox?.transform = Transform(pitch: 0, yaw: 0, roll: .pi/4)
Roll, pitch, and yaw represent rotation along a particular axis, more is written here.
To change the position you can again use transform's translation
steelBox.transform.translation = [0, 0, -0.5]
This will translate your object according to the given parameters. This function heavily relies on affine transforms.
Regarding transformations w.r.t. other objects
Nodes in SceneKit as well as Entities in RealityKit are transformed and rotated with respect to the parent Node or Entity.
So, in your case, you have a big model where you plan to put smaller objects.
You have two options, either use touches detection on the big model (house) and them manually calculate where the object should be placed, which may be pretty cumbersome.
Another option is to add small transparent planes at predefined positions and then adding touches detection on them. This way you can omit calculation of where the smaller object should be placed
Good luck!
I am using ARKit's ARFaceTrackingConfiguration with ARConfiguration.WorldAlignment.camera alignment, but I found that the documentation (seemingly) does not reflect the reality;
Based on the excerpt of documentation below, I would expect that the face anchor's transform is expressed in right handed coordinate system. However, when I tried moving my head, I noticed that the Z coordinate of the face anchor is always negative (i.e. faceAnchor.transform.columns.3.z < 0). Note that moving head in the X and Y directions corresponds to expected outcome (unlike Z coordinate).
Camera alignment defines a coordinate system based on the native sensor orientation of the device camera. Relative to a AVCaptureVideoOrientation.landscapeRight-oriented camera image, the x-axis points to the right, the y-axis points up, and the z-axis points out the front of the device (toward the user).
I want the transform to behave as per the documentation, i.e. the Z coordinate of face anchor should be positive given that documentation says "the z-axis points out the front of the device (toward the user)". So far it seems the Z-axis points out the back of the device…
Am I missing something obvious?
I tried to repair the rotation by the following code, but I am not sure if it's correct way to fix this:
// Repair rotation
let oldFaceRotation = simd_quatf(face.transform) // get quaternion from
let repairedFaceRotation = simd_quatf(ix: oldFaceRotation.axis.y, iy: oldFaceRotation.axis.x, iz: -oldFaceRotation.axis.z, r: oldFaceRotation.real)
// Repair translation
var repairedPosition = face.transform.columns.3
repairedPosition.z *= -1
// Combine
var correctedFaceTransform = float4x4(repairedFaceRotation)
correctedFaceTransform.columns.3 = repairedPosition
It seems quite obvious:
When ARSession is running and ARCamera begins tracking environment, it places WorldOriginAxis in front of your face at (x: 0, y: 0, z: 0). Just check it using:
sceneView.debugOptions = [.showWorldOrigin]
So your face's position must be at positive part of Z axis of World Coordinates.
Thus, ARFaceAnchor will be placed at positive Z-axis direction, as well.
And when you use ARFaceTrackingConfiguration vs ARWorldTrackingConfiguration there's two things to consider:
Rear Camera moves towards objects along negative Z-axes (positive X-axis is on the right).
Front Camera moves towards faces along positive Z-axes (positive X-axis is on the left).
Hence, when you are "looking" through TrueDepth Camera, a 4x4 Matrix is mirrored.
Although I still don't know why does not the face anchor behave as described in the documentation, I can at least answer how to correct its left-handed system into the Metal- and SceneKit-friendly right-handed system (X axis to the right, Y axis up, Z axis from the screen towards user):
func faceAnchorPoseToRHS(_ mat: float4x4) -> float4x4 {
let correctedPos = float4(x: mat.columns.3.x, y: mat.columns.3.y, z: -mat.columns.3.z, w: 1)
let quat = simd_quatf(mat)
let newQuat = simd_quatf(angle: -quat.angle, axis: float3(quat.axis.x, quat.axis.y, -quat.axis.z))
var newPose = float4x4(newQuat)
newPose.columns.3 = correctedPos
return newPose
}
the goal of the project is to create a drawing app. i want it so that when i touch the screen and move my finger it will follow the finger and leave a cyan color paint. i did created it BUT there is one problem. the paint DEPTH is always randomly placed.
here is the code, just need to connect the sceneView with the storyboard.
https://github.com/javaplanet17/test/blob/master/drawingar
my question is how do i make the program so that the depth will always be consistent, by consistent i mean there is always distance between the paint and the camera.
if you run the code above you will see that i have printed out all the SCNMatrix4, but i none of them is the DEPTH.
i have tried to change hitTransform.m43 but it only messes up the x and y.
If you want to get a point some consistent distance in front of the camera, you don’t want a hit test. A hit test finds the real world surface in front of the camera — unless your camera is pointed at a wall that’s perfectly parallel to the device screen, you’re always going to get a range of different distances.
If you want a point some distance in front of the camera, you need to get the camera’s position/orientation and apply a translation (your preferred distance) to that. Then to place SceneKit content there, use the resulting matrix to set the transform of a SceneKit node.
The easiest way to do this is to stick to SIMD vector/matrix types throughout rather than converting between those and SCN types. SceneKit adds a bunch of new accessors in iOS 11 so you can use SIMD types directly.
There’s at least a couple of ways to go about this, depending on what result you want.
Option 1
// set up z translation for 20 cm in front of whatever
// last column of a 4x4 transform matrix is translation vector
var translation = matrix_identity_float4x4
translation.columns.3.z = -0.2
// get camera transform the ARKit way
let cameraTransform = view.session.currentFrame.camera.transform
// if we wanted, we could go the SceneKit way instead; result is the same
// let cameraTransform = view.pointOfView.simdTransform
// set node transform by multiplying matrices
node.simdTransform = cameraTransform * translation
This option, using a whole transform matrix, not only puts the node a consistent distance in front of your camera, it also orients it to point the same direction as your camera.
Option 2
// distance vector for 20 cm in front of whatever
let translation = float3(x: 0, y: 0, z: -0.2)
// treat distance vector as in camera space, convert to world space
let worldTranslation = view.pointOfView.simdConvertPosition(translation, to: nil)
// set node position (not whole transform)
node.simdPosition = worldTranslation
This option sets only the position of the node, leaving its orientation unchanged. For example, if you place a bunch of cubes this way while moving the camera, they’ll all be lined up facing the same direction, whereas with option 1 they’d all be in different directions.
Going beyond
Both of the options above are based only on the 3D transform of the camera — they don’t take the position of a 2D touch on the screen into account.
If you want to do that, too, you’ve got more work cut out for you — essentially what you’re doing is hit testing touches not against the world, but against a virtual plane that’s always parallel to the camera and a certain distance away. That plane is a cross section of the camera projection frustum, so its size depends on what fixed distance from the camera you place it at. A point on the screen projects to a point on that virtual plane, with its position on the plane scaling proportional to the distance from the camera (like in the below sketch):
So, to map touches onto that virtual plane, there are a couple of approaches to consider. (Not giving code for these because it’s not code I can write without testing, and I’m in an Xcode-free environment right now.)
Make an invisible SCNPlane that’s a child of the view’s pointOfView node, parallel to the local xy-plane and some fixed z distance in front. Use SceneKit hitTest (not ARKit hit test!) to map touches to that plane, and use the worldCoordinates of the hit test result to position the SceneKit nodes you drop into your scene.
Use Option 1 or Option 2 above to find a point some fixed distance in front of the camera (or a whole translation matrix oriented to match the camera, translated some distance in front). Use SceneKit’s projectPoint method to find the normalized depth value Z for that point, then call unprojectPoint with your 2D touch location and that same Z value to get the 3D position of the touch location with your camera distance. (For extra code/pointers, see my similar technique in this answer.)
I am using 3ds max for a long time and I know xyz axis. what I see in xcode in rotation the scnnode what made my mind blowed up is w component of scnvector4.
Can someone explain in detail how to use this method because I searched a lot of time but I can't make my object spin as I desire. anyone can help to make him spin to his back in 180 degree but I will appreciate if someone explain more for further rotations, Knowing that I saw this link but I didn't understand something.
http://www.cprogramming.com/tutorial/3d/rotationMatrices.html
I believe that your are trying to rotate nodes (rotation property).
From the documentation :
The four-component rotation vector specifies the direction of the rotation axis in the first three components and the angle of rotation (in radians) in the fourth.
You might find it easier to use eulerAngles :
The node’s orientation, expressed as pitch, yaw, and roll angles, each in radians
Use .transform to rotate a node
node.transform = SCNMatrix4Mult(node.transform, SCNMatrix4MakeRotation(angle, x, y, z))
if you want to rotate your node 180 degree by x axis
node.transform = SCNMatrix4Mult(node.transform, SCNMatrix4MakeRotation(Float(M_PI), 1, 0, 0))