I'm trying to add a 3D-object properly on a reference image. To add the 3D-object on the image in real world I'm using the imageAnchor.transform property as seen below.
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
updateQueue.async {
// Add a virtual cup at the position of the found image
self.virtualObject.addVirtualObjectWith(sceneName: "cup.dae",
childNodeName: nil,
position: SCNVector3(x: imageAnchor.transform.columns.3.x,
y: imageAnchor.transform.columns.3.y,
z: imageAnchor.transform.columns.3.z),
recursively: true,
imageAnchor: imageAnchor)
}
}
The problem is when I move the device orientation the cup won't stay nicely in the middle on the image. I would also like to have the cup on the same spot even when I remove the image. I don't get the problem because when you add an object using plane detection and hit testing there is also a ARAnchor used for the plane.
Update 05/14/2018
func addVirtualObjectWith(sceneName: String, childNodeName: String, objectName: String, position: SCNVector3, recursively: Bool, node: SCNNode?){
print("VirtualObject: Added virtual object with scene name: \(sceneName)")
let scene = SCNScene(named: "art.scnassets/\(sceneName)")!
var sceneNode = scene.rootNode.childNode(withName: childNodeName, recursively: recursively)!
sceneNode.name = objectName
sceneNode.position = position
add(object: sceneNode, toNode: node)
}
func add(object: SCNNode, toNode: SCNNode?){
if toNode != nil {
toNode?.addChildNode(object)
}
else {
sceneView.scene.rootNode.addChildNode(object)
}
}
I finally found the solution, turns out that the size of the AR Reference Image was not set correctly in the attributes inspector. When the size is not correct, the anchor of the image will be shaky.
Related
I'm building AR Scanner application where users are able to scan different images and receive rewards for this.
When they point camera at some specific image - I place SCNNode on top of that image and after they remove camera from that image - SCNNode get's dismissed.
But when image disappears and camera stays at the same position SCNNode didn't get dismissed.
How can I make it disappear together with Reference image disappearance?
I have studied lot's of other answers here, on SO, but they didn't help me
Here's my code for adding and removing SCNNode's:
extension ARScannerScreenViewController: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
DispatchQueue.main.async { self.instructionLabel.isHidden = true }
if let imageAnchor = anchor as? ARImageAnchor {
handleFoundImage(imageAnchor, node)
imageAncors.append(imageAnchor)
trackedImages.append(node)
} else if let objectAnchor = anchor as? ARObjectAnchor {
handleFoundObject(objectAnchor, node)
}
}
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
guard let pointOfView = sceneView.pointOfView else { return }
for (index, item) in trackedImages.enumerated() {
if !(sceneView.isNode(item, insideFrustumOf: pointOfView)) {
self.sceneView.session.remove(anchor: imageAncors[index])
}
}
}
private func handleFoundImage(_ imageAnchor: ARImageAnchor, _ node: SCNNode) {
let name = imageAnchor.referenceImage.name!
print("you found a \(name) image")
let size = imageAnchor.referenceImage.physicalSize
if let imageNode = showImage(size: size) {
node.addChildNode(imageNode)
node.opacity = 1
}
}
private func showImage(size: CGSize) -> SCNNode? {
let image = UIImage(named: "InfoImage")
let imageMaterial = SCNMaterial()
imageMaterial.diffuse.contents = image
let imagePlane = SCNPlane(width: size.width, height: size.height)
imagePlane.materials = [imageMaterial]
let imageNode = SCNNode(geometry: imagePlane)
imageNode.eulerAngles.x = -.pi / 2
return imageNode
}
private func handleFoundObject(_ objectAnchor: ARObjectAnchor, _ node: SCNNode) {
let name = objectAnchor.referenceObject.name!
print("You found a \(name) object")
}
}
I also tried to make it work using ARSession, but I couldn't even get to prints:
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
for anchor in anchors {
for myAnchor in imageAncors {
if let imageAnchor = anchor as? ARImageAnchor, imageAnchor == myAnchor {
if !imageAnchor.isTracked {
print("Not tracked")
} else {
print("tracked")
}
}
}
}
}
You have to use ARWorldTrackingConfiguration instead of ARImageTrackingConfiguration. It's quite bad idea to use both configurations in app because each time you switch between them – tracking state is reset and you have to track from scratch.
Let's see what Apple documentation says about ARImageTrackingConfiguration:
With ARImageTrackingConfiguration, ARKit establishes a 3D space not by tracking the motion of the device relative to the world, but solely by detecting and tracking the motion of known 2D images in view of the camera.
The basic differences between these two configs are about how ARAnchors behave:
ARImageTrackingConfiguration allows you get ARImageAnchors only if your reference images is in a Camera View. So if you can't see a reference image – there's no ARImageAnchor, thus there's no a 3D model (it's resetting each time you cannot-see-it-and-then-see-it-again). You can simultaneously detect up to 100 images.
ARWorldTrackingConfiguration allows you track a surrounding environment in 6DoF and get ARImageAnchor, ARObjectAnchor, or AREnvironmentProbeAnchor. If you can't see a reference image – there's no ARImageAnchor, but when you see it again ARImageAnchor is still there. So there's no reset.
Conclusion:
ARWorldTrackingConfiguration's cost of computation is much higher. However this configuration allows you perform not only image tracking but also hit-testing and ray-casting for detected planes, object detection, and a restoration of world maps.
Use nodeForAnchor to load your nodes, so when the anchors disappear, the nodes will go as well.
I want to preface this with I am a beginner to Swift but need to get this ARKit project finished already.
I use the function.
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
let trackedNode = node
if let imageAnchor = anchor as? ARImageAnchor{
if (imageAnchor.isTracked) {
trackedNode.isHidden = false
offScreen = false
print("Visible")
}else {
trackedNode.isHidden = true
//print("\(trackedImageName)")
offScreen = true
print("No image in view")
}
}
}
This detects if the anchor is on screen and sets the global variable offScreen to the appropriate value.
I want to take the new value of the variable and use it in my createdVideoPlayerNodeFor function. If offScreen is true, then set AVPlayer to pause.
However, I have my AVPlayer declared in my createdVideoPlayerNodeFor function so I can't contain it in one function.
I know I am referring to fragments of my code at a time so I have full code posted below.
var offScreen = false
let videoNode = SCNNode()
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
offScreen = false
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
node.addChildNode(createdVideoPlayerNodeFor(referenceImage))
}
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
let trackedNode = node
if let imageAnchor = anchor as? ARImageAnchor{
if (imageAnchor.isTracked) {
trackedNode.isHidden = false
offScreen = false
print("Visible")
}else {
trackedNode.isHidden = true
//print("\(trackedImageName)")
offScreen = true
print("No image in view")
}
}
}
func createdVideoPlayerNodeFor(_ target: ARReferenceImage) -> SCNNode {
let videoPlayerGeometry = SCNPlane(width: target.physicalSize.width, height: target.physicalSize.height)
var player = AVPlayer()
if let targetName = target.name,
let awsURL:NSURL = NSURL(string: "my video url :).mp4") {
player = AVPlayer(url: awsURL as URL)
player.play()
NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: player.currentItem, queue: nil) { (notification) in
player.seek(to: CMTime.zero)
player.pause()
}
}
videoPlayerGeometry.firstMaterial?.diffuse.contents = player
videoNode.geometry = videoPlayerGeometry
videoNode.eulerAngles.x = -Float.pi / 2
return videoNode
}
I am in dire need of help with this so if anyone can help me to figure this out, It would be greatly appreciated.
Please ask questions if I didn't explain well enough or anything, I really just need to figure this out :)
Edit: In my testing, I found that when the variable was changed in either function, it was almost like the variable had 2 different values, 1 for each of the functions. So if it was set to true in the didUpdate function, it didn't matter because the createdVideo function would use the value set at the beginning of the variable's declaration. Is this even possible to set the value of the variable in one func and have it carry over to another?
offScreen is an instance variable, which means that it is in scope for both functions. You should be able to both read and set it from either. Be careful, however, that you don't read/write that variable from different threads as then the value can be unpredictable. You might want to set up an offscreenQueue, a private DispatchQueue, that will restrict access to this variable.
I am implementing a demo project using SceneKit and ARKit where i keep camera on some picture and play video from url. Everything works fine but i could not be able to stop previous video and it starts another video. Previous video plays on background. It happens sometimes when i switch image. My implementation is :
#IBOutlet var sceneView: ARSCNView!
var videosNames = [String]()
var player = AVPlayer()
var videoNode = SKVideoNode()
In viewDidload,
sceneView.delegate = self
// Create a session configuration
let configuration = ARImageTrackingConfiguration()
// first see if there is a folder called "ARImages" Resource Group in our Assets Folder
if let trackedImages = ARReferenceImage.referenceImages(inGroupNamed: "ARImages", bundle: Bundle.main) {
// if there is, set the images to track
configuration.trackingImages = trackedImages
// at any point in time, only 1 image will be tracked
configuration.maximumNumberOfTrackedImages = 1
}
// Run the view's session
sceneView.session.run(configuration)
The callback method looks like :
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
// if the anchor is not of type ARImageAnchor (which means image is not detected), just return
if let _ : Bool = videosNames.contains(anchor.name!) {
guard let imageAnchor = anchor as? ARImageAnchor, var fileUrlString = Bundle.main.path(forResource: "baby1", ofType: "mp4") else {return}
print("didAdd node:\(anchor.name!)")
//find our video file
fileUrlString = "https://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4" // i set the file URL path directly here
let videoItem = AVPlayerItem(url: URL(fileURLWithPath: fileUrlString))
self.player.seek(to: CMTime.zero)
self.player.pause() // Tried to pause the video, set seek time zero. Nothing works.
self.videoNode.pause()
player = AVPlayer(playerItem: videoItem)
//initialize video node with avplayer
videoNode = SKVideoNode(avPlayer: self.player)
//videoNode.removeAllChildren()
self.player.play()
//self.videoNode.play()
// add observer when our player.currentItem finishes player, then start playing from the beginning
NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: self.player.currentItem, queue: nil) { (notification) in
self.player.seek(to: CMTime.zero)
self.player.play()
//self.videoNode.play()
print("Looping Video")
}
// set the size (just a rough one will do)
let videoScene = SKScene(size: CGSize(width: 480, height: 360))
// center our video to the size of our video scene
self.videoNode.position = CGPoint(x: videoScene.size.width / 2, y: videoScene.size.height / 2)
// invert our video so it does not look upside down
self.videoNode.yScale = -1.0
// add the video to our scene
videoScene.addChild(self.videoNode)
// create a plan that has the same real world height and width as our detected image
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
// set the first materials content to be our video scene
plane.firstMaterial?.diffuse.contents = videoScene
// create a node out of the plane
let planeNode = SCNNode(geometry: plane)
// since the created node will be vertical, rotate it along the x axis to have it be horizontal or parallel to our detected image
planeNode.eulerAngles.x = -Float.pi / 2
// finally add the plane node (which contains the video node) to the added node
node.addChildNode(planeNode)
}
}
When any node gets update
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
if node.isHidden == true {
print("Node is out of view:\(anchor.name!) :")
self.player.seek(to: CMTime.zero)
videoNode.pause()
} else {
print("Node is not out of view:\(anchor.name!)")
}
}
Here i get log when any new video start playing , but i could not stop previously started video which plays in background (only audio) with the new video. It totally messed up my process. If i put camera away from both image, but the sounds keeps playing. Could anyone please help me for the issue ?
Thanks in advance.
Node must be removed from ARSCNView's session. Following method does not removed the node from session.
node.removeFromParentNode()
Only sceneView.session.remove() works perfectly in didUpdate() method.
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
if node.isHidden == true {
if let imageAnchor = anchor as? ARImageAnchor {
sceneView.session.remove(anchor: imageAnchor)
}
} else {
if !isNodeAdded {
isNodeAdded = true
}
}
}
sceneView.session.remove() works to stop the video in the didUpdate() method, but you also need to pause the player, otherwise the audio still will be playing. This works for me:
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
if node.isHidden == true {
if let imageAnchor = anchor as? ARImageAnchor {
sceneView.session.remove(anchor: imageAnchor)
player.pause()
}
} else {
player.play()
}
}
I'm pretty new to AR Kit but recently I found that the image tracking feature is quite awesome. I found it's as simple as:
let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: Bundle.main)
let configuration = ARImageTrackingConfiguration()
configuration.trackingImages = referenceImages
configuration.maximumNumberOfTrackedImages = 1
sceneView.session.run(configuration)
which works beautifully! However, I want to further the experience by identifying which image has been tracked and display different AR objects / nodes based on the image that was tracked. Is there a way to get more information on the specific image that is currently being tracked?
In you AR Reference Group in your assets catalog, when you click the reference image, you can open the attributes inspector and enter a "Name."
This name is then reflected in the name property of the ARImageAnchor for the anchor that is created when the AR session begins to track that specific image.
Then in
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode?
You can inspect the anchor and respond accordingly. For example:
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
guard let anchor = anchor as? ARImageAnchor else { return nil }
if anchor.name == "calculator" {
print("tracking calculator image")
return SCNNode.makeMySpecialCalculatorNode()
}
return nil
}
I am trying to return a previously created node in my Session Delegate:
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
guard let planeAnchor = anchor as? ARPlaneAnchor else {return SCNNode()}
let anchorNode = sceneView.anchor(for: earthNode)
if anchorNode == nil || anchorNode == planeAnchor {
return earthNode
}
return nil
}
I am trying to see whether there is already an anchor assigned to my node, and if not, return earthNode.
My problem is that let anchorNode = sceneView.anchor(for: earthNode) is either freezing, or infinite looping.
My working theory is that this is due to the fact that earthNode isn't yet placed in the scene. But that seems like a wonky explanation. I also, of course, presume that my usage of ARkit reeks of ignorance.