I'm making an AR app that's a ball toss game using Swift's ARKit.
Click here for my repo
The point of the game is to toss the ball and make it land in the hat. However, whenever I try to toss the ball, it always appear to fall to infinity instead of landing in the hat or on the floor plane that I've created.
Here's the code for tossing the ball:
#IBAction func throwBall(_ sender: Any) {
// Create ball
let ball = SCNSphere(radius: 0.02)
currentBallNode = SCNNode(geometry: ball)
currentBallNode?.physicsBody = .dynamic()
currentBallNode?.physicsBody?.allowsResting = true
currentBallNode?.physicsBody?.isAffectedByGravity = true
// Apply transformation
let camera = sceneView.session.currentFrame?.camera
let cameraTransform = camera?.transform
currentBallNode?.simdTransform = cameraTransform!
// Add current ball node to balls array
balls.append(currentBallNode!)
// Add ball node to root node
sceneView.scene.rootNode.addChildNode(currentBallNode!)
// Set force to be applied
let force = simd_make_float4(0, 0, -3, 0)
let rotatedForce = simd_mul(cameraTransform!, force)
let vectorForce = SCNVector3(x:rotatedForce.x, y:rotatedForce.y, z:rotatedForce.z)
// Apply force to ball
currentBallNode?.physicsBody?.applyForce(vectorForce, asImpulse: true)
}
And here's the physics body setting for the floor:
Look at below screenshot for get more idea.
Nevermind, I managed to resolve this by adding the following function:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let planeAnchor = anchor as? ARPlaneAnchor, planeAnchor.center == self.planeAnchor?.center || self.planeAnchor == nil else { return }
// Set the floor's geometry to be the detected plane
let floor = sceneView.scene.rootNode.childNode(withName: "floor", recursively: true)
let plane = SCNPlane(width: CGFloat(planeAnchor.extent.x), height: CGFloat(planeAnchor.extent.y))
floor?.geometry = plane
}
Related
I created a fun little AR app that allows me to point my phone at 3 index cards, that have 2D drawings (xmas tree, ginger bread man, gift), and have 3D versions of the images pop up.
However, I have limited my app to just 3 images for now, as each picture/3D combo has its own block of code (shown below) but I would like to somehow consolidate to one block (even if I need to rename the image files to a numbered format, but would appreciate any advice).
I have tried two methods:
one in which I have 3 blocks of code, one for each picture (this method works - a translucent plane appears just over the index card and a 3d Object appears)
one in which I have consolidated to one block of code (this method results in only the translucent plane appearing when pointing the camera at the index card)
3 blocks of code method
Screenshots of the app, along with the code are as follows:
// MARK: - ARSCNViewDelegate
//the anchor will be the image detected and the node will be a 3d object
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
//the 3d object
let node = SCNNode()
//if this detects a plane or point or anything other then an image, will not run this code
if let imageAnchor = anchor as? ARImageAnchor {
//this plane is created from the image detected (card). want the width and height in the physical world to be the same as the card, so effectively saying "look at the anchor image found, and get its size properties”
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
//once dimension is given to the plane, use it to create a planenode which is a 3d object that will be rendered on top of the card
let planeNode = SCNNode(geometry: plane)
//makes the plane translucent
plane.firstMaterial?.diffuse.contents = UIColor(white: 1.0, alpha: 0.5)
//by default the plane created is vertical so need to flip it to be flat with the card detected
planeNode.eulerAngles.x = -.pi / 2
// tap into the node created above and add a child node which will be the plane node
node.addChildNode(planeNode)
// ------------3 BLOCKS OF CODE BELOW THAT NEED TO SIMPLIFY TO 1----------------
// This is the first block of code for the ChristmasTree.png image/ChristmasTree.scn 3D object – note that the 2D image is not detected if “.png” is included at the end of the image anchor, however the “.scn” appears to be required for the 3D object).
if imageAnchor.referenceImage.name == "ChristmasTree" {
//create the 3d model on top of the card
if let cardScene = SCNScene(named: "art.scnassets/ChristmasTree.scn") {
//create a node that will represent the 3d object.
if let cardNode = cardScene.rootNode.childNodes.first {
//Since the 3d image is rotated the other way, need to bring it forward so same as above only with positive
cardNode.eulerAngles.x = .pi / 2
planeNode.addChildNode(cardNode)
}
}
}
// This is the second block of code for the Gift.png image/Gift.scn 3D object
if imageAnchor.referenceImage.name == "Gift" {
//create the 3d model on top of the card
if let cardScene = SCNScene(named: "art.scnassets/Gift.scn") {
//create a node that will represent the 3d object
if let cardNode = cardScene.rootNode.childNodes.first {
//Since the 3d image is rotated the other way, need to bring it forward so same as above only with positive
cardNode.eulerAngles.x = .pi / 2
planeNode.addChildNode(cardNode)
}
}
}
// This is the third block of code for the GingerbreadMan.png image/GingerbreadMan.scn 3D object
if imageAnchor.referenceImage.name == "GingerbreadMan" {
//create the 3d model on top of the card
if let cardScene = SCNScene(named: "art.scnassets/GingerbreadMan.scn") {
//create a node that will represent the 3d object
if let cardNode = cardScene.rootNode.childNodes.first {
//Since the 3d image is rotated the other way, need to bring it forward so same as above only with positive
cardNode.eulerAngles.x = .pi / 2
planeNode.addChildNode(cardNode)
}
}
}
}
//this method is expecting an output of SCNNode and it needs to send that back onto the scene so it can render the 3d object
return node
}
}
One block of code method
I have revised the code as follows, however it has resulted in only the translucent plane appearing when pointing the camera at the index card:
// MARK: - ARSCNViewDelegate
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
let node = SCNNode()
if let imageAnchor = anchor as? ARImageAnchor {
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
let planeNode = SCNNode(geometry: plane)
plane.firstMaterial?.diffuse.contents = UIColor(white: 1.0, alpha: 0.5)
planeNode.eulerAngles.x = -.pi / 2
node.addChildNode(planeNode)
//------------single block of code------------
let name = imageAnchor.referenceImage.name
if ["ChristmasTree", "Gift", "GingerbreadMan"].contains(name) {
if let cardScene = SCNScene(named: "art.scnassets/\(name).scn") {
if let cardNode = cardScene.rootNode.childNodes.first {
cardNode.eulerAngles.x = .pi / 2
planeNode.addChildNode(cardNode)
}
}
}
//------------single block of code------------
}
return node
}
}
Your issue there is that name is an optional String and when you do string interpolation it results in a string like "Optional("whatever"). What you need is to unwrap your optional String:
if let name = imageAnchor.referenceImage.name {
// this condition is not needed considering that you are already safely unwrapping SCNScene fallible initializer
// if ["ChristmasTree", "Gift", "GingerbreadMan"].contains(name) {
if let cardScene = SCNScene(named: "art.scnassets/\(name).scn"),
let cardNode = cardScene.rootNode.childNodes.first {
cardNode.eulerAngles.x = .pi / 2
planeNode.addChildNode(cardNode)
}
// }
}
This is the first time I am creating an ARKit app, and also not very familiar with iOS development, however I am trying to achieve something relitively simple...
All the the app needs to do is get the world position of the phone and send it continously to a rest api.
I am using the default ARKit project in Xcode, and am able to get the phone's position with the following function in ViewController.swift:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if let planeAnchor = anchor as? ARPlaneAnchor {
let plane = SCNPlane(width: CGFloat(planeAnchor.extent.x), height: CGFloat(planeAnchor.extent.z))
plane.firstMaterial?.diffuse.contents = UIColor(white: 1, alpha: 0.75)
let planeNode = SCNNode(geometry: plane)
planeNode.position = SCNVector3Make(planeAnchor.center.x, planeAnchor.center.x, planeAnchor.center.z)
planeNode.eulerAngles.x = -.pi / 2
node.addChildNode(planeNode)
}
guard let pointOfView = sceneView.pointOfView else { return }
let transform = pointOfView.transform
let orientation = SCNVector3(-transform.m31, -transform.m32, transform.m33)
let location = SCNVector3(transform.m41, transform.m42, transform.m43)
let currentPositionOfCamera = SCNVector3(orientation.x + location.x, orientation.y + location.y, orientation.z + location.z)
print(currentPositionOfCamera)
}
this function works as expected, in renders a plane in the view and prints the position, but only once
I need to get the phone's position as it is moved, I tried to add:
updateAtTime time: NSTimeInterval
to the function definition, but after doing this even the plane was not rendered any longer.
How can I continuously get the phone's position vector?
TIA
On image marker detection, I want to play animation of walking guy within that marker boundary only using ARKit. For that I want to find out the position of that 3D object while it is walking on marker. Animation is created using external 3D authoring tool and saved in .scnassets as .dae file. I have added node and start animation using below code:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if let imageAnchor = anchor as? ARImageAnchor {
DispatchQueue.main.async {
//let translation = imageAnchor.transform.columns.3
let idleScene = SCNScene(named: "art.scnassets/WalkAround/WalkAround.dae")!
// This node will be parent of all the animation models
let node1 = SCNNode()
// Add all the child nodes to the parent node
for child in idleScene.rootNode.childNodes {
node1.addChildNode(child)
}
node1.scale = SCNVector3(0.2, 0.2, 0.2)
let physicalSize = imageAnchor.referenceImage.physicalSize
let size = CGSize(width: 500, height: 500)
let skScene = SKScene(size: size)
skScene.backgroundColor = .white
let plane = SCNPlane(width: self.referenceImage!.physicalSize.width, height: self.referenceImage!.physicalSize.height)
let material = SCNMaterial()
material.lightingModel = SCNMaterial.LightingModel.constant
material.isDoubleSided = true
material.diffuse.contents = skScene
plane.materials = [material]
let rectNode = SCNNode(geometry: plane)
rectNode.eulerAngles.x = -.pi / 2
node.addChildNode(rectNode)
node.addChildNode(node1)
self.loadAnimation(withKey: "walking", sceneName: "art.scnassets/WalkAround/SambaArmtr", animationIdentifier: "SambaArmtr-1")
}
}
}
func loadAnimation(withKey: String, sceneName:String, animationIdentifier:String) {
let sceneURL = Bundle.main.url(forResource: sceneName, withExtension: "dae")
let sceneSource = SCNSceneSource(url: sceneURL!, options: nil)
if let animationObject = sceneSource?.entryWithIdentifier(animationIdentifier, withClass: CAAnimation.self) {
// The animation will only play once
animationObject.repeatCount = 1
}
}
I tried using node.presentation.position in both below methods to get current position of object.
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval)
// Or
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor)
If I will not move device anymore once animation has been started, those methods will not get called and till the time I am getting same position of node. Thats why I am not getting where I am wrong. Or is there any way to get current position of object while animation is going on in ARKit?
I don't know of any way to get the current frame within an embedded animation. With that said, the animation embedded within a model uses CoreAnimation to run the animation. You could use the CAAimationDelegate to listen to the start/end events of your animation and run a timer. The timer would give you the best estimate of which frame the animation is on.
References:
SceneKit Animating Content Documentation: https://developer.apple.com/documentation/scenekit/animation/animating_scenekit_content
CAAnimationDelegate Documentation: https://developer.apple.com/documentation/quartzcore/caanimationdelegate
With ARKit 2 a new configuration was added: ARImageTrackingConfiguration which according to the SDK can have better performance and some new use cases.
Experimenting with it on Xcode 10b2 (see https://forums.developer.apple.com/thread/103894 how to fix the asset loading) my code now correctly calls the delegate that an image was tracked and hereafter a node was added but I could not find any documentation where the coordinate system is located, hence does anybody know how to put the node into the scene for it to overlay the detected image:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
DispatchQueue.main.async {
if let imageAnchor = anchor as? ARImageAnchor {
let imageNode = SCNNode.createImage(size: imageAnchor.referenceImage.physicalSize)
imageNode.transform = // ... ???
node.addChildNode(imageNode)
}
}
}
ps: in contrast to ARWorldTrackingConfiguration the origin seems to constantly move around (most likely putting the camera into 0,0,0).
pps: SCNNode.createImage is a helper function without any coordinate calculations.
Assuming that I have read your question correctly, you can do something like the following:
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
let nodeToReturn = SCNNode()
//1. Check We Have Detected Our Image
if let validImageAnchor = anchor as? ARImageAnchor {
//2. Log The Information About The Anchor & Our Reference Image
print("""
ARImageAnchor Transform = \(validImageAnchor.transform)
Name Of Detected Image = \(validImageAnchor.referenceImage.name)
Width Of Detected Image = \(validImageAnchor.referenceImage.physicalSize.width)
Height Of Detected Image = \(validImageAnchor.referenceImage.physicalSize.height)
""")
//3. Create An SCNPlane To Cover The Detected Image
let planeNode = SCNNode()
let planeGeometry = SCNPlane(width: validImageAnchor.referenceImage.physicalSize.width,
height: validImageAnchor.referenceImage.physicalSize.height)
planeGeometry.firstMaterial?.diffuse.contents = UIColor.white
planeNode.geometry = planeGeometry
//a. Set The Opacity To Less Than 1 So We Can See The RealWorld Image
planeNode.opacity = 0.5
//b. Rotate The PlaneNode So It Matches The Rotation Of The Anchor
planeNode.eulerAngles.x = -.pi / 2
//4. Add It To The Node
nodeToReturn.addChildNode(planeNode)
//5. Add Something Such As An SCNScene To The Plane
if let modelScene = SCNScene(named: "art.scnassets/model.scn"), let modelNode = modelScene.rootNode.childNodes.first{
//a. Set The Model At The Center Of The Plane & Move It Forward A Tad
modelNode.position = SCNVector3Zero
modeNode.position.z = 0.15
//b. Add It To The PlaneNode
planeNode.addChildNode(modelNode)
}
}
return nodeToReturn
}
Hopefully this will point you in the right direction...
I have been trying to move car body along with wheels. I have creating augmented reality project, where placing car model in horizontal plane and Car are controlled by four buttons namelu acceleration, steering, reverse and brake. car left, right are controller by steering while acceleration and reverse.
Actually now i can able to placing 3d car model in horizontal plane but dont know how to rotate car wheels like forward and backward along with car body.
Here is the code where i have been trying for placing 3d object:
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
if let touch = touches.first {
// gives us the location of where we touched on the 2D screen.
let touchLocation = touch.location(in: sceneView)
// hitTest is performed to get the 3D coordinates corresponding to the 2D coordinates that we got from touching the screen.
// That 3d coordinate will only be considered when it is on the existing plane that we detected.
let results = sceneView.hitTest(touchLocation, types: .existingPlaneUsingExtent)
// if we have got some results using the hitTest then do this.
if let hitResult = results.first {
let boxScene = SCNScene(named: "art.scnassets/porsche.scn")!
if let boxNode = boxScene.rootNode.childNode(withName: "car", recursively: true) {
print("box:::\(boxNode.childNodes)")
boxNode.position = SCNVector3(x: hitResult.worldTransform.columns.3.x, y: hitResult.worldTransform.columns.3.y + 0.15, z: hitResult.worldTransform.columns.3.z)
// finally the box is added to the scene.
sceneView.scene.rootNode.addChildNode(boxNode)
}
}
}
}
Detecting horizontal plane functionality code:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if anchor is ARPlaneAnchor {
// anchors can be of many types, as we are just dealing with horizontal plane detection we need to downcast anchor to ARPlaneAnchor
let planeAnchor = anchor as! ARPlaneAnchor
// creating a plane geometry with the help of dimentions we got using plane anchor.
let plane = SCNPlane(width: CGFloat(planeAnchor.extent.x), height: CGFloat(planeAnchor.extent.z))
// a node is basically a position.
let planeNode = SCNNode()
// setting the position of the plane geometry to the position we got using plane anchor.
planeNode.position = SCNVector3(x: planeAnchor.center.x, y: 0, z: planeAnchor.center.z)
// when a plane is created its created in xy plane instead of xz plane, so we need to rotate it along x axis.
planeNode.transform = SCNMatrix4MakeRotation(-Float.pi/2, 1, 0, 0)
//create a material object
let gridMaterial = SCNMaterial()
//setting the material as an image. A material can also be set to a color.
gridMaterial.diffuse.contents = UIImage(named: "art.scnassets/grid.png")
// assigning the material to the plane
plane.materials = [gridMaterial]
// assigning the position to the plane
planeNode.geometry = plane
//adding the plane node in our scene
node.addChildNode(planeNode)
}
else {
return
}
}
Any help much appreciated!!!
This is quite comprehensive task.
You should make your car as SCNPhysicsVehicle subclass.
SCNPhysicsVehicle is a physics behavior that modifies a physics body to behave like a car, motorcycle, or other wheeled vehicle.
To build a vehicle, designate an SCNPhysicsBody object as its chassis and an array of SCNPhysicsVehicleWheel objects as its wheels. For each wheel, you define physical characteristics such as suspension and traction, and associate a node in your scene to provide the wheel’s size and visual representation. After you construct a vehicle, you can control it in terms of acceleration, braking, and steering.
General: https://developer.apple.com/documentation/scenekit/scnphysicsbody
For wheels: https://developer.apple.com/documentation/scenekit/scnphysicsvehiclewheel
Example:
var vehicle = SCNPhysicsVehicle()
You need to create ground with static body.
func createGround(planeAnchor: ARPlaneAnchor) -> SCNNode {
let ground = SCNNode(geometry: SCNPlane(width: CGFloat(planeAnchor.extent.x), height: CGFloat(CGFloat(planeAnchor.extent.z))))
ground.position = SCNVector3(planeAnchor.center.x,planeAnchor.center.y,planeAnchor.center.z)
ground.eulerAngles = SCNVector3(90.degreesToRadians, 0, 0)
let staticBody = SCNPhysicsBody.static() // it must be static
ground.physicsBody = staticBody
return ground
}
You need to setup your car.
func setupCar() {
let scene = SCNScene(named: "YourScene.scn")
let chassis = (scene?.rootNode.childNode(withName: "chassis", recursively: true))! // Your chassis
let frontLeftWheel = chassis.childNode(withName: "frontLeftWheel", recursively: true)!
let frontRightWheel = chassis.childNode(withName: "frontRightWheel", recursively: true)!
let rearLeftWheel = chassis.childNode(withName: "rearLeftWheel", recursively: true)!
let rearRightWheel = chassis.childNode(withName: "rearRightWheel", recursively: true)!
// physic behavior for wheels
let v_frontLeftWheel = SCNPhysicsVehicleWheel(node: frontLeftWheel)
let v_frontRightWheel = SCNPhysicsVehicleWheel(node: frontRightWheel)
let v_rearRightWheel = SCNPhysicsVehicleWheel(node: rearLeftWheel)
let v_rearLeftWheel = SCNPhysicsVehicleWheel(node: rearRightWheel)
chassis.position = SCNVector(0,0,0) // insert your desired position
let body = SCNPhysicsBody(type: .dynamic, shape: SCNPhysicsShape(node: chassis, options: [SCNPhysicsShape.Option.keepAsCompound: true]))
body.mass = 1
chassis.physicsBody = body
self.vehicle = SCNPhysicsVehicle(chassisBody: chassis.physicsBody!, wheels: [v_rearRightWheel, v_rearLeftWheel, v_frontRightWheel, v_frontLeftWheel])
self.sceneView.scene.physicsWorld.addBehavior(self.vehicle)
self.sceneView.scene.rootNode.addChildNode(chassis)
}
Now you can manipulate with your car.
func renderer(_ renderer: SCNSceneRenderer, didSimulatePhysicsAtTime time: TimeInterval) {
var engineForce: CGFloat = 5
var brakingForce: CGFloat = 0
// here you can manipulate your car steering angle
self.vehicle.setSteeringAngle(0, forWheelAt: 2)
self.vehicle.setSteeringAngle(0, forWheelAt: 3)
self.vehicle.applyEngineForce(engineForce, forWheelAt: 0)
self.vehicle.applyEngineForce(engineForce, forWheelAt: 1)
self.vehicle.applyBrakingForce(0, forWheelAt: 0)
self.vehicle.applyBrakingForce(0, forWheelAt: 1)
}
P.S. Don't forget to add to your viewDidLoad():
self.sceneView.delegate = self
self.sceneView.scene?.physicsWorld.contactDelegate = self
P.P.S This is only a general idea, you must do some research and find a working solution to your specific situation.
I hope it helped!