Consider the default Augmented Realiy app created by Xcode.
ARSession has this delegate method called
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
that triggers when a new anchor is added to the scene.
The anchor is an ARPlaneAnchor.
This anchor has a geometry property that is an ARPlaneGeometry that has nothing to do with the tradicional ARSPlaneAhchor.
Because there is zero documentation about it, I ask: this geometry corresponds to a triangle mesh that corresponds to the plane detected by RealityKit.
How do I display that mesh on the scene?
Remember that I am using ARView not SceneKit or other stuff.
Related
I know that if I'm using ARSCNView (i.e. SceneKit), I can draw a line using something like a cylinder since SceneKit has a 3D environment and drawing something between two 3d points is supported natively.
However, I'm trying to create an interface like Apple's measure app:
The rendered elements (e.g. the line and the 3'3") don't look 3D, so I was thinking I should use ARSKView (i.e. SpriteKit) for this. Placing a single element such as the 👾 emoji in Apple's ARKit + SpriteKit template is pretty trivial, since you simply need to implement the method:
func view(_ view: ARSKView, nodeFor anchor: ARAnchor) -> SKNode? {
return SKLabelNode(text: "👾")
}
This associates a 2D object (SKNode) with a 3D position (ARAnchor).
What I can't figure out how to do is how to draw a line between two nodes.
Is there any way to draw a line between two nodes like in the image above using SpriteKit, or must we switch over to SceneKit to achieve this effect?
Note: This is not a duplicate of the following questions since those questions talk about SceneKit (3D), not SpriteKit (2D):
Draw straight line from start value to end value ARKit
ARKit line between two SCNNode
I want to display a SCNNode above the tracked image which is the closest to the center of the camera.
func renderer(_ renderer: SCNSceneRenderer, willUpdate node: SCNNode, for anchor: ARAnchor)
I tried to hide all others nodes/anchors and only keep current one in this renderer function but nothing worked.
My goal is to display a 3D object when an image is tracked and when another image is found, the tracking is enabled on the image closest to the center of the camera.
Is there a way to specify which ARAnchor on the camera to track or to hide other ARAnchors?
I want to place a figure straight on the floor. I see two options where to put it:
inside anchor's SCNNode with anchor's coordinates
inside rootNode, in global coordinates, with height == anchor.transform[3][1]
I don't turn off tracking because I see that stability of tracking improves in first 10-20 second.
In the first case, my figure rotates randomly (because anchor tends to increase the extent and wants to fit extent's rectangle in the tracking area). In the second case, the figure may be upper or lower than the actual floor (I can see it by adding extra "floor" inside anchor's SCNNode).
I can use the first case and make transformations to compensate rotation but it does not look like a right solution.
What is the right way to place a figure on the floor?
I guess you have the anchor from a callback from something like arConfiguration.planeDetection = .horizontal where arConfiguration is defined as let arConfiguration = ARWorldTrackingConfiguration().
When a callback like func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) is called by ARKit you should add the node to the ARKit scene.rootNode. For the same plane this callback will be called: func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor). Then, to have the same object in the same position during the ARKit scene exploration you should act in the "add" callback.
Did i get it right?
Hope this helps
Hi I need to take a set of photos with my iphone and read the corresponding projection matrix, rotation and translation matrix for post-processing. I have never used ARKit or programmed in Swift/Object C before. What is the best way to get started?
If you create a default AR project in Xcode then you'll be able to study the basics of getting an AR app running in Swift.
To read the camera parameters you need to implement this function:
func session(_ session: ARSession, didUpdate frame: ARFrame) {
guard let transform = frame.camera.transform else {return}
let position = SCNVector3Make(transform.columns.3.x, transform.columns.3.y, transform.columns.3.z)
let rotation = frame.camera.eulerAngles
let projection = frame.camera.projectionMatrix
print(position)
print(rotation)
print(projectionMatrix)
print("================="
}
(I.e copy / paste that code as another function in your ViewController class in the default app. The rotation is given in euler angles.
Given you've not used Swift before this may not be that helpful. But in theory it's all you need.
ARKit grabs frames at 60 frames per second (you can change this) - this function is called automatically for every new frame.
If you're using it to take photos you'll have to add more code to get all the timing right etc.
Hey I'm trying to figure out. How to keep a simple node in place. As I walk around it in ARKit
Code:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if let planeAnchor = anchor as? ARPlaneAnchor {
if planeDetected == false { // Bool only allows 1 plane to be added
planeDetected = true
self.addPlane(node: node, anchor: planeAnchor)
}
}
}
This adds the SCNNode
func addPlane(node: SCNNode, anchor: ARPlaneAnchor) {
// We add the anchor plane here
let showDebugVisuals = Bool()
let plane = Plane(anchor, showDebugVisuals)
planes[anchor] = plane
node.addChildNode(plane)
// We add our custom SCNNode here
let scene = SCNScene(named: "art.scnassets/PlayerModel.scn")!
let Body = scene.rootNode.childNode(withName: "Body", recursively: true)!
Body.position = SCNVector3.positionFromTransform(anchor.transform)
Body.movabilityHint = .movable
wrapperNode.position = SCNVector3.positionFromTransform(anchor.transform)
wrapperNode.addChildNode(Body)
scnView.scene.rootNode.addChildNode(wrapperNode)
Ive tried adding a Plane/Anchor Node and putting the "Body" node in that but it still moves. I thought maybe it has something to do with the update function.
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
}
Or most likely the position setting
wrapperNode.position = SCNVector3.positionFromTransform(anchor.transform)
Iv'e looked through every source / project file / video on the internet and nobody has a simple solution to this simple problem.
There are two kinds of "moving around" that could be happening here.
One is that ARKit is continuously refining its estimate of how the device's position in the real world maps to the abstract coordinate space you're placing virtual content in. For example, suppose you put a virtual object at (0, 0, -0.5), and then move your device to the left by exactly 10 cm. The virtual object will appear to be anchored in physical space only if ARKit tracks the move precisely. But visual-inertial odometry isn't an exact science, so it's possible that ARKit thinks you moved to the left by 10.5 cm — in that case, your virtual object will appear to "slip" to the right by 5 mm, even though its position in the ARKit/SceneKit coordinate space remains constant.
You can't really do much about this, other than hope Apple makes devices with better sensors, better cameras, or better CPUs/GPUs and improves the science of world tracking. (In the fullness of time, that's probably a safe bet, though that probably doesn't help with your current project.)
Since you're also dealing with plane detection, there's another wrinkle. ARKit is continuously refining its estimates of where a detected plane is. So, even though the real-world position of the plane isn't changing, its position in ARKit/SceneKit coordinate space is.
This kind of movement is generally a good thing — if you want your virtual object to appear anchored to the real-world surface, you want to be sure of where that surface is. You'll see some movement as plane detection gets more sure of the surface's position, but after a short time, you should see less "slip" as you move the camera around for plan-anchored virtual objects than those that are just floating in world space.
In your code, though, you're not taking advantage of plane detection to make your custom content (from "PlayerModel.scn") stick to the plane anchor:
wrapperNode.position = SCNVector3.positionFromTransform(anchor.transform)
wrapperNode.addChildNode(Body)
scnView.scene.rootNode.addChildNode(wrapperNode)
This code uses the initial position of the plane anchor to position wrapperNode in world space (because you're making it a child of the root node). If you instead make wrapperNode a child of the plane anchor's node (the one you received in renderer(_:didAdd:for:)), it'll stay attached to the plane as ARKit refines its estimate of the plane's position. You'll get a little bit more movement initially, but as plane detection "settles", your virtual object will "slip" less.
(When you make the node a child of the plane, you don't need to set its position — a position of zero means it's right where the plane is. Inf anything, you need to set its position only relative to the plane — i.e. how far above/below/along it.)
To keep an SCNNode in place you can disable sceneView plane detection once you get the result you desired.
let configuration = ARWorldTrackingConfiguration();
configuration.planeDetection = []
self.sceneView.session.run(configuration)
The reason for this is that ARKit constantly reestimates the position of the detected plane resulting in your SCNNode moving around.