I'm pretty new to AR Kit but recently I found that the image tracking feature is quite awesome. I found it's as simple as:
let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: Bundle.main)
let configuration = ARImageTrackingConfiguration()
configuration.trackingImages = referenceImages
configuration.maximumNumberOfTrackedImages = 1
sceneView.session.run(configuration)
which works beautifully! However, I want to further the experience by identifying which image has been tracked and display different AR objects / nodes based on the image that was tracked. Is there a way to get more information on the specific image that is currently being tracked?
In you AR Reference Group in your assets catalog, when you click the reference image, you can open the attributes inspector and enter a "Name."
This name is then reflected in the name property of the ARImageAnchor for the anchor that is created when the AR session begins to track that specific image.
Then in
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode?
You can inspect the anchor and respond accordingly. For example:
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
guard let anchor = anchor as? ARImageAnchor else { return nil }
if anchor.name == "calculator" {
print("tracking calculator image")
return SCNNode.makeMySpecialCalculatorNode()
}
return nil
}
Related
I'm building AR Scanner application where users are able to scan different images and receive rewards for this.
When they point camera at some specific image - I place SCNNode on top of that image and after they remove camera from that image - SCNNode get's dismissed.
But when image disappears and camera stays at the same position SCNNode didn't get dismissed.
How can I make it disappear together with Reference image disappearance?
I have studied lot's of other answers here, on SO, but they didn't help me
Here's my code for adding and removing SCNNode's:
extension ARScannerScreenViewController: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
DispatchQueue.main.async { self.instructionLabel.isHidden = true }
if let imageAnchor = anchor as? ARImageAnchor {
handleFoundImage(imageAnchor, node)
imageAncors.append(imageAnchor)
trackedImages.append(node)
} else if let objectAnchor = anchor as? ARObjectAnchor {
handleFoundObject(objectAnchor, node)
}
}
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
guard let pointOfView = sceneView.pointOfView else { return }
for (index, item) in trackedImages.enumerated() {
if !(sceneView.isNode(item, insideFrustumOf: pointOfView)) {
self.sceneView.session.remove(anchor: imageAncors[index])
}
}
}
private func handleFoundImage(_ imageAnchor: ARImageAnchor, _ node: SCNNode) {
let name = imageAnchor.referenceImage.name!
print("you found a \(name) image")
let size = imageAnchor.referenceImage.physicalSize
if let imageNode = showImage(size: size) {
node.addChildNode(imageNode)
node.opacity = 1
}
}
private func showImage(size: CGSize) -> SCNNode? {
let image = UIImage(named: "InfoImage")
let imageMaterial = SCNMaterial()
imageMaterial.diffuse.contents = image
let imagePlane = SCNPlane(width: size.width, height: size.height)
imagePlane.materials = [imageMaterial]
let imageNode = SCNNode(geometry: imagePlane)
imageNode.eulerAngles.x = -.pi / 2
return imageNode
}
private func handleFoundObject(_ objectAnchor: ARObjectAnchor, _ node: SCNNode) {
let name = objectAnchor.referenceObject.name!
print("You found a \(name) object")
}
}
I also tried to make it work using ARSession, but I couldn't even get to prints:
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
for anchor in anchors {
for myAnchor in imageAncors {
if let imageAnchor = anchor as? ARImageAnchor, imageAnchor == myAnchor {
if !imageAnchor.isTracked {
print("Not tracked")
} else {
print("tracked")
}
}
}
}
}
You have to use ARWorldTrackingConfiguration instead of ARImageTrackingConfiguration. It's quite bad idea to use both configurations in app because each time you switch between them – tracking state is reset and you have to track from scratch.
Let's see what Apple documentation says about ARImageTrackingConfiguration:
With ARImageTrackingConfiguration, ARKit establishes a 3D space not by tracking the motion of the device relative to the world, but solely by detecting and tracking the motion of known 2D images in view of the camera.
The basic differences between these two configs are about how ARAnchors behave:
ARImageTrackingConfiguration allows you get ARImageAnchors only if your reference images is in a Camera View. So if you can't see a reference image – there's no ARImageAnchor, thus there's no a 3D model (it's resetting each time you cannot-see-it-and-then-see-it-again). You can simultaneously detect up to 100 images.
ARWorldTrackingConfiguration allows you track a surrounding environment in 6DoF and get ARImageAnchor, ARObjectAnchor, or AREnvironmentProbeAnchor. If you can't see a reference image – there's no ARImageAnchor, but when you see it again ARImageAnchor is still there. So there's no reset.
Conclusion:
ARWorldTrackingConfiguration's cost of computation is much higher. However this configuration allows you perform not only image tracking but also hit-testing and ray-casting for detected planes, object detection, and a restoration of world maps.
Use nodeForAnchor to load your nodes, so when the anchors disappear, the nodes will go as well.
I want to preface this with I am a beginner to Swift but need to get this ARKit project finished already.
I use the function.
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
let trackedNode = node
if let imageAnchor = anchor as? ARImageAnchor{
if (imageAnchor.isTracked) {
trackedNode.isHidden = false
offScreen = false
print("Visible")
}else {
trackedNode.isHidden = true
//print("\(trackedImageName)")
offScreen = true
print("No image in view")
}
}
}
This detects if the anchor is on screen and sets the global variable offScreen to the appropriate value.
I want to take the new value of the variable and use it in my createdVideoPlayerNodeFor function. If offScreen is true, then set AVPlayer to pause.
However, I have my AVPlayer declared in my createdVideoPlayerNodeFor function so I can't contain it in one function.
I know I am referring to fragments of my code at a time so I have full code posted below.
var offScreen = false
let videoNode = SCNNode()
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
offScreen = false
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
node.addChildNode(createdVideoPlayerNodeFor(referenceImage))
}
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
let trackedNode = node
if let imageAnchor = anchor as? ARImageAnchor{
if (imageAnchor.isTracked) {
trackedNode.isHidden = false
offScreen = false
print("Visible")
}else {
trackedNode.isHidden = true
//print("\(trackedImageName)")
offScreen = true
print("No image in view")
}
}
}
func createdVideoPlayerNodeFor(_ target: ARReferenceImage) -> SCNNode {
let videoPlayerGeometry = SCNPlane(width: target.physicalSize.width, height: target.physicalSize.height)
var player = AVPlayer()
if let targetName = target.name,
let awsURL:NSURL = NSURL(string: "my video url :).mp4") {
player = AVPlayer(url: awsURL as URL)
player.play()
NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: player.currentItem, queue: nil) { (notification) in
player.seek(to: CMTime.zero)
player.pause()
}
}
videoPlayerGeometry.firstMaterial?.diffuse.contents = player
videoNode.geometry = videoPlayerGeometry
videoNode.eulerAngles.x = -Float.pi / 2
return videoNode
}
I am in dire need of help with this so if anyone can help me to figure this out, It would be greatly appreciated.
Please ask questions if I didn't explain well enough or anything, I really just need to figure this out :)
Edit: In my testing, I found that when the variable was changed in either function, it was almost like the variable had 2 different values, 1 for each of the functions. So if it was set to true in the didUpdate function, it didn't matter because the createdVideo function would use the value set at the beginning of the variable's declaration. Is this even possible to set the value of the variable in one func and have it carry over to another?
offScreen is an instance variable, which means that it is in scope for both functions. You should be able to both read and set it from either. Be careful, however, that you don't read/write that variable from different threads as then the value can be unpredictable. You might want to set up an offscreenQueue, a private DispatchQueue, that will restrict access to this variable.
I'm trying to add a 3D-object properly on a reference image. To add the 3D-object on the image in real world I'm using the imageAnchor.transform property as seen below.
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
updateQueue.async {
// Add a virtual cup at the position of the found image
self.virtualObject.addVirtualObjectWith(sceneName: "cup.dae",
childNodeName: nil,
position: SCNVector3(x: imageAnchor.transform.columns.3.x,
y: imageAnchor.transform.columns.3.y,
z: imageAnchor.transform.columns.3.z),
recursively: true,
imageAnchor: imageAnchor)
}
}
The problem is when I move the device orientation the cup won't stay nicely in the middle on the image. I would also like to have the cup on the same spot even when I remove the image. I don't get the problem because when you add an object using plane detection and hit testing there is also a ARAnchor used for the plane.
Update 05/14/2018
func addVirtualObjectWith(sceneName: String, childNodeName: String, objectName: String, position: SCNVector3, recursively: Bool, node: SCNNode?){
print("VirtualObject: Added virtual object with scene name: \(sceneName)")
let scene = SCNScene(named: "art.scnassets/\(sceneName)")!
var sceneNode = scene.rootNode.childNode(withName: childNodeName, recursively: recursively)!
sceneNode.name = objectName
sceneNode.position = position
add(object: sceneNode, toNode: node)
}
func add(object: SCNNode, toNode: SCNNode?){
if toNode != nil {
toNode?.addChildNode(object)
}
else {
sceneView.scene.rootNode.addChildNode(object)
}
}
I finally found the solution, turns out that the size of the AR Reference Image was not set correctly in the attributes inspector. When the size is not correct, the anchor of the image will be shaky.
I am trying to return a previously created node in my Session Delegate:
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
guard let planeAnchor = anchor as? ARPlaneAnchor else {return SCNNode()}
let anchorNode = sceneView.anchor(for: earthNode)
if anchorNode == nil || anchorNode == planeAnchor {
return earthNode
}
return nil
}
I am trying to see whether there is already an anchor assigned to my node, and if not, return earthNode.
My problem is that let anchorNode = sceneView.anchor(for: earthNode) is either freezing, or infinite looping.
My working theory is that this is due to the fact that earthNode isn't yet placed in the scene. But that seems like a wonky explanation. I also, of course, presume that my usage of ARkit reeks of ignorance.
Does anybody else also experience non-working ARKit scene on an iPhone8?
When I download Apple's ARKit example and run it on an iPhone8, it stays on Initializing - when I check the ARSCNViewDelegate implementation:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let planeAnchor = anchor as? ARPlaneAnchor else { return }
DispatchQueue.main.async {
self.statusViewController.cancelScheduledMessage(for: .planeEstimation)
self.statusViewController.showMessage("SURFACE DETECTED")
[..]
}
[..]
}
It seems as if it never goes beyond the guard, so a ARPlaneAnchor is never being added to the scene...
The same project on an iPhone6s / iPhone7 runs just fine though...
Does anyone else know how to fix this?