ARKit - Object stuck to camera after tap on screen - ios

I started out with the template project which you get when you choose ARKit project. As you run the app you can see the ship and view it from any angle.
However, once I allow camera control and tap on the screen or zoom into the ship through panning the ship gets stuck to camera. Now wherever I go with the camera the ship is stuck to the screen.
I went through the Apple Guide and seems like the don't really consider this as unexpected behavior as there is nothing about this behavior.
How to keep the position of the ship fixed after I zoom it or touch the screen?

Well, looks like allowsCameraControl is not the answer at all. It's good for SceneKit but not for ARKit(maybe it's good for something in AR but I'm not aware of it yet).
In order to zoom into the view a UIPinchGestureRecognizer is required.
// 1. Find the touch location
// 2. Perform a hit test
// 3. From the results take the first result
// 4. Take the node from that first result and change the scale
#objc private func handlePan(recognizer: UIPinchGestureRecognizer) {
if recognizer.state == .changed {
// 1.
let location = recognizer.location(in: sceneView)
// 2.
let hitTestResults = sceneView.hitTest(location, options: nil)
// 3.
if let hitTest = hitTestResults.first {
let shipNode = hitTest.node
let newScaleX = Float(recognizer.scale) * shipNode.scale.x
let newScaleY = Float(recognizer.scale) * shipNode.scale.y
let newScaleZ = Float(recognizer.scale) * shipNode.scale.z
// 4.
shipNode.scale = SCNVector3(newScaleX, newScaleY, newScaleZ)
recognizer.scale = 1
}
}
Regarding #2. I got confused a little with another hitTest method called hitTest(_:types:)
Note from documentation
This method searches for AR anchors and real-world objects detected by
the AR session, not SceneKit content displayed in the view. To search
for SceneKit objects, use the view's hitTest(_:options:) method
instead.
So that method cannot be used if you want to scale a node which is a SceneKit content

Related

How to limit SCNNode from moving when camera moves along the x-axis?

I'm working a simple app using ARKit where a user can tap their screen and place a node (SCNNode) on a given location. I want the user to be able to place nodes that stay in place no matter where the camera is so that when they pan back to the location where they placed the node, it's still there.
I've gotten the tap functionality to work, but I've noticed that when I physically move my device along the x-axis, the placed node moves along with it. I've tried to anchor the nodes to something other than the root node, but it hasn't worked as expected. I tried to look up documentation on how the root node is placed and if it's calculated based on the camera which would explain why the nodes are moving along with the camera, but no luck there either.
Here's the code for placing the nodes. The node position is placed using scenePoint which is a projection from the touch location to the scene that was done using SceneKit: unprojectPoint returns same/similar point no matter where you touch screen.
let nodeImg = SCNNode(geometry: SCNSphere(radius: 0.05))
nodeImg.physicsBody? = .static()
nodeImg.geometry?.materials.first?.diffuse.contents = hexColor
nodeImg.geometry?.materials.first?.specular.contents = UIColor.white
nodeImg.position = SCNVector3(scenePoint.x, scenePoint.y, scenePoint.z)
print(nodeImg.position)
sceneView.scene.rootNode.addChildNode(nodeImg)
I think this has something to do with the fact that I'm adding the nodeImg node as a child to the rootNode, but I'm not sure what else to anchor it to.
On tap you need to set the 'worldPosition' of the node and not just 'position'
Check this link : ARKit: position vs worldposition vs simdposition
Assuming you have set sceneView.allowsCameraControl = false
#objc func handleTapGesture(withGestureRecognizer recognizer: UITapGestureRecognizer) {
let location: CGPoint = recognizer.location(in: self.sceneView)
let hits = self.sceneView.hitTest(location, options: nil)
if let tappednode = hits.first?.node {
nodeImg.worldPosition = tappednode.worldPosition
self.sceneView.scene.rootNode.addChildNode(nodeImg)
}
}

How to drag SCNode with finger irrespective of axis using ARKit?

I am working on an AR based application using ARKit. I am using https://developer.apple.com/documentation/arkit/handling_3d_interaction_and_ui_controls_in_augmented_reality as base for this. Using this i am able to move or rotate the whole Virtual Object.
Now there are lot of child nodes in the Virtual Object. I want to drag/move any child node with user finger irrespective of the axis. The child SCNode may be in ground or floating. I want to move the object wherever the user finger goes irrespective of the axis or irrespective of the euler angles of the child node. Is this even possible?
I followed the below links but it is just moving along a particular axis.
ARKit - Drag a node along a specific axis (not on a plane)
Dragging SCNNode in ARKit Using SceneKit
I tried using the below code and it is not at all helping,
let tapPoint: CGPoint = gesture.location(in: sceneView)
let result = sceneView.hitTest(tapPoint, options: nil)
if result.count == 0 {
return
}
let scnHitResult: SCNHitTestResult? = result.first
movedObject = scnHitResult?.node //.parent?.parent
let hitResults = self.sceneView.hitTest(tapPoint, types: .existingPlane)
if !hitResults.isEmpty{
guard let hitResult = hitResults.last else { return }
movedObject?.position = SCNVector3Make(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
}

ARKit: How to place one imported 3D model above another?

I am working on a AR project using ARKit.
If I touch only the imported 3D object on a point, I want to place another 3D object above it.
(For example I have placed a table above which I have to place something else like a flower vase on the touched point).
How can I solve the problem that the second object should only be placed, when I touch the first 3D object?
The surface of the object is not flat, so I can not use hittest with bounding box.
One approach is to give the first imported 3D object a node name.
firstNode.name = “firstObject”
Inside you tapped gesture function you can do a hitTest like this
let tappedNode = self.sceneView.hitTest(location, options: [:])
let node = tappedNode[0].node
if node.name == “firstObject” {
let height = firstNode.boundingBox.max.y -firstNode.boundingBox.min.y
let position2ndNode = SCNVector3Make(firstNode.worldPosition.x, (firstNode.worldPosition.y + height), firstNode.worldPosition.z)
2ndNode.position = position2ndNode
sceneView.scene.rootNode.addChildNode(2ndNode)
} else {
return
}
This way when you tap anywhere else the 2nd object won’t get placed. It will only place when you tap on the node itself. It doesn’t matter where you tap on the node, because we only want the height & we can determine that from its boundingBox max - min which we then add to the firstnode.worldPosition.y
Make sure you set at the top of ARSCNView class
var firstNode = SCNNode!
this way we can access the firstNode in the tap gesture function.
Edit: If the first 3D model has many nodes. You can flattenNode on the parent Node in the sceneGraph (best illustrated with photo below). This removes all the childNodes and wraps from the sceneGraph. You can then just work with the parentNode.

Move objects around, with gesture recognizer for multiple Objects

I am trying to make an app where you can use Stickers like on Snapchat and Instagram. It fully worked to find a technique, that adds the images, but now I want that if you swipe the object around the object changes its position (I also want to make the scale / rotate function).
My code looks like this:
#objc func StickerLaden() {
for i in 0 ..< alleSticker.count {
let imageView = UIImageView(image: alleSticker[i])
imageView.frame = CGRect(x: StickerXScale[i], y:StickerYScale[i], width: StickerScale[i], height: StickerScale[i])
ImageViewsSticker.append(imageView)
ImageView.addSubview(imageView)
imageView.isUserInteractionEnabled = true
let aSelector : Selector = "SlideFunc"
let slideGesture = UISwipeGestureRecognizer(target: self, action: aSelector)
imageView.addGestureRecognizer(slideGesture)
}
}
func SlideFunc(fromPoint:CGPoint, toPoint: CGPoint) {
}
Here are the high-level steps you need to take:
Add one UIPanGestureRecognizer to the parent view that has the images on it.
Implement UIGestureRecognizerDelegate methods to keep track of the user touching and releasing the screen.
On first touch, loop through all your images and call image.frame.contains(touchPoint). Add all images that are under the touch point to an array.
Loop through the list of touched images and calculate the distance of the touch point to the center of the image. Chose the image whose center is closest to the touched point.
Move the chosen image to the top of the view stack. [You now have selected an image and made it visible.]
Next, when you receive pan events, change the frame of the chosen image accordingly.
Once the user releases the screen, reset any state variables you may have, so that you can start again when the next touch is done.
The above will give you a nicely working pan solution. It's a good amount of things you need to sort out, but it's not very difficult.
As I said in my comment, scale and rotate are very tricky. I advise you to forget that for a bit and first implement other parts of your app.

How can I set text orientation in ARKit?

I am creating a simple app with ARKit in which I add some text to the scene to the tapped position:
#objc func tapped(sender: UITapGestureRecognizer){
let sceneView = sender.view as! ARSCNView
let tapLocation = sender.location(in: sceneView)
let hitTest = sceneView.hitTest(tapLocation, types: .featurePoint)
if !hitTest.isEmpty{
self.addTag(tag: "A", hitTestResult: hitTest.first!)
}
else{
print("no match")
}
}
func addTag(tag: String, hitTestResult: ARHitTestResult){
let tag = SCNText(string:tag, extrusionDepth: 0.1)
tag.font = UIFont(name: "Optima", size: 1)
tag.firstMaterial?.diffuse.contents = UIColor.red
let tagNode = SCNNode(geometry: tag)
let transform = hitTestResult.worldTransform
let thirdColumn = transform.columns.3
tagNode.position = SCNVector3(thirdColumn.x,thirdColumn.y - tagNode.boundingBox.max.y / 2,thirdColumn.z)
print("\(thirdColumn.x) \(thirdColumn.y) \(thirdColumn.z)")
self.sceneView.scene.rootNode.addChildNode(tagNode)
}
It works, but I have problem with the orientation of the text. When I add it with the camera's original position, the text orientation is ok, I can see the text frontwise (Sample 1). But when I turn camera to the left / right, and add the text by tapping, I can see the added text from the side (Sample 2).
Sample 1:
Sample 2:
I know there should be some simple trick to solve it, but as a beginner in this topic I could not find it so far.
You want the text to always face the camera? SCNBillboardConstraint is your friend:
tagNode.constraints = [SCNBillboardConstraint()]
Am I correct in saying that you want the text to face the camera when you tap (wherever you happen to be facing), but then remain stationary?
There are a number of ways of adjusting the orientation of any node. For this case I would suggest simply setting the eulerAngles of the text node to be equal to those of the camera, at the point in which you instantiate the text.
In your addTag() function you add:
let eulerAngles = self.sceneView.session.currentFrame?.camera.eulerAngles
tagNode.eulerAngles = SCNVector3(eulerAngles.x, eulerAngles.y, eulerAngles.z + .pi / 2)
The additional .pi / 2 is there to ensure the text is in the correct orientation, as the default with ARKit is for a landscape orientation and therefore the text comes out funny. This applies a rotation around the local z axis.
It's also plausible (and some may argue it's better) to use .localRotate() of the node, or to access its transform property, however I like the approach of manipulating both the position and eulerAngles directly.
Hope this helps.
EDIT: replaced Float(1.57) with .pi / 2.

Resources