Draw custom polygon dynamically from manually added verticies in real time - ios

I want to create app where you can set points in real world in detected surface and after you have 4 or more points it creates plane/polygon between them with texture. Basic controller configuration:
#IBOutlet weak var sceneView: ARSCNView!
let configuration = ARWorldTrackingConfiguration()
var vectors: [SCNVector3] = []
var mostBottomYAxis: Float? {
didSet {
for (index, _) in vectors.enumerated() {
vectors[index].y = self.mostBottomYAxis ?? vectors[index].y
}
}
}
var identifier: UUID?
override func viewDidLoad() {
super.viewDidLoad()
self.sceneView.debugOptions = [ARSCNDebugOptions.showFeaturePoints, ARSCNDebugOptions.showWorldOrigin]
self.sceneView.showsStatistics = true
self.configuration.worldAlignment = .gravity // Y axis is Up and Down
self.configuration.planeDetection = .horizontal // Detect horizontal surfaces
self.sceneView.session.run(configuration)
self.sceneView.autoenablesDefaultLighting = true
self.sceneView.delegate = self
self.registerGestureRecognizers()
}
Tap gesture method for adding vertex to surface:
#objc func handleTap(sender: UITapGestureRecognizer) {
let sceneView = sender.view as! ARSCNView
let tapLocation = sender.location(in: sceneView)
if let raycast = sceneView.raycastQuery(from: tapLocation, allowing: .estimatedPlane, alignment: .horizontal),
let result = sceneView.session.raycast(raycast).first {
self.addItem(raycastResult: result)
}
}
func addItem(raycastResult: ARRaycastResult) {
let transform = raycastResult.worldTransform
let thirdColumn = transform.columns.3
let vector = SCNVector3(x: thirdColumn.x, y: mostBottomYAxis ?? thirdColumn.y, z: thirdColumn.z)
self.vectors.append(vector)
self.addItem(toPosition: vector)
}
and methods for drawing polygon:
func getGeometry(forVectors vectors: [SCNVector3]) -> SCNGeometry {
let polygonIndices = Array(0...vectors.count).map({ Int32($0) })
let indices: [Int32] = [Int32(vectors.count)] + /* We have a polygon with this count of points */
polygonIndices /* The indices for our polygon */
let source = SCNGeometrySource(vertices: vectors)
let indexData = Data(bytes: indices,
count: indices.count * MemoryLayout<Int32>.size)
let element = SCNGeometryElement(data: indexData,
primitiveType: .polygon,
primitiveCount: 1,
bytesPerIndex: MemoryLayout<Int32>.size)
// MARK: - Texture
let textureCoordinates = [
CGPoint(x: 0, y: 0),
CGPoint(x: 1, y: 0),
CGPoint(x: 0, y: 1),
CGPoint(x: 1, y: 1)
]
let uvSource = SCNGeometrySource(textureCoordinates: textureCoordinates)
let geometry = SCNGeometry(sources: [source, uvSource],
elements: [element])
return geometry
}
func getNode(forVectors vectors: [SCNVector3]) -> SCNNode {
let polyDraw = getGeometry(forVectors: vectors)
let material = SCNMaterial()
material.isDoubleSided = true
material.diffuse.contents = UIImage(named: "Altezo_colormix_brilant")
material.diffuse.wrapS = .repeat
material.diffuse.wrapT = .repeat
polyDraw.materials = [material]
let node = SCNNode(geometry: polyDraw)
return node
}
extension PlacePointsForPlaneViewController: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
guard let planeAnchor = anchor as? ARPlaneAnchor else { return }
let y = planeAnchor.transform.columns.3.y
if mostBottomYAxis == nil {
mostBottomYAxis = y
identifier = planeAnchor.identifier
}
if mostBottomYAxis! > y {
mostBottomYAxis = y
identifier = planeAnchor.identifier
}
// Remove existing plane nodes
node.enumerateChildNodes { (childNode, _) in
childNode.removeFromParentNode()
}
if planeAnchor.identifier == self.identifier {
if self.vectors.count > 3 {
let modelNode = getNode(forVectors: self.vectors)
node.addChildNode(modelNode)
}
}
}
}
I hope it's mostly self explained. I have vectors which contains manually added points. I want to be working only with one surface that is most bottom (thats what do mostBottomYAxis and identifier).
What am I doing wrong? Why polygon is not drawn exactly between points? When I tried to draw line between two points it's working.
One more problem. How to set texture coordinates to correctly draw texture in polygon and to get it working not only for 4 verticies but for more (dynamically as user add more points and can change their positions)?
Thanks for help
Edit: After I post question I tried to different nodes to add my custom polygon and it's working fine if I added to sceneView rootNode:
sceneView.scene.rootNode.addChildNode(modelNode)
So this helps with position of polygon. But still I have problems with texture. How to set texture offset/transform to get it working? To have texture fill this custom geometry and repeat texture to edges.

Related

How can I dynamically track SCNNodes in a scene for removal?

Obligatory: first time writing an app, relevant code is below. My code runs mostly as I want it to, but does not achieve my goal of dynamic object tracking.
I'm using Swift and Scenekit to build a simple puzzle game, similar to a 3d-version of candy crush.
I have a class Cube that extends SCNNode. On initialization, this class will randomly draw a 5x5 cube of SCNBoxes with each box being red, green, or blue (all 6 sides of a box are 1 color).
The goal of the game is to get the highest score by removing "chains" of SCNBoxes of like-colors. When a chain is removed, cubes should recognize gravity and drop to fill in the voids created by the removed-chain. This is where I need to dynamically track position. As the cubes fall into the gaps, their neighbors change.
My approach: build a struct CubeDetails that has properties var color: String and var location: SCNVector3. Next, build a dictionary masterCubeDict = [SCNNode: CubeDetails] that has all of the cubes of 1 color (the color is provided by a hittestresult).
Every time a user taps a cube, grab its color, refresh the masterCubeDict, and then use math on the SCNVector3 position to determine which cubes are neighbors.
I think my algorithm for finding 'cube neighbors' using math on the scnvector3 is where I'm off. There must be a better way for scenekit nodes to identity/find each other, right?
Also -- I would like the physics of the cubes to let them fall and have no bounce/sliding at all. They should only move straight up/down. Collisions should never happen. I thought I implemented that properly through friction, restituion, and mass of the cubes but I'm not getting the outcome I want.
class Cube
import SceneKit
class Cube : SCNNode {
let cubeWidth:Float = 0.95
let spaceBetweenCubes:Float = 0.05
var cubecolor:UIColor = UIColor.black
var masterCubeDict: [SCNNode: CubeDetails] = [:]
struct CubeDetails {
var color:String
var position:SCNVector3
}
override init() {
super.init()
let cubeOffsetDistance = self.cubeOffsetDistance()
var cubeColorString: String = ""
var xPos:Float = -cubeOffsetDistance
var yPos:Float = -cubeOffsetDistance
var zPos:Float = -cubeOffsetDistance
let xFloor:Float = -1.5
let yFloor:Float = -1.5
let zFloor:Float = -1.5
let floorGeo = SCNBox(width: 20, height: 0, length: 20, chamferRadius: 0)
let floor = SCNNode(geometry: floorGeo)
floor.position = SCNVector3(x: xFloor, y: yFloor, z: zFloor)
floor.name = "floor"
floor.opacity = 0.0
floor.physicsBody = SCNPhysicsBody(type: .kinematic, shape: nil)
floor.physicsBody?.collisionBitMask = 1
floor.physicsBody?.friction = 1.0
self.addChildNode(floor)
for _ in 0..<5 {
for _ in 0..<5 {
for _ in 0..<5 {
let cubeGeometry = SCNBox(width: CGFloat(cubeWidth), height: CGFloat(cubeWidth), length: CGFloat(cubeWidth), chamferRadius: 0)
let material = SCNMaterial()
material.diffuse.contents = randomColor()
//unwrap material (type any) and cast to uicolor for switch
if let unwrapColor: UIColor = material.diffuse.contents as? UIColor {
switch unwrapColor {
case UIColor.red:
cubeColorString = "red"
case UIColor.green:
cubeColorString = "green"
case UIColor.blue:
cubeColorString = "blue"
default:
cubeColorString = "black"
}
} else { print("Error unwrapping color") }
cubeGeometry.materials = [material, material, material, material, material, material]
let cube = SCNNode(geometry: cubeGeometry)
cube.name = cubeColorString
cube.physicsBody = SCNPhysicsBody(type: .dynamic, shape: nil)
cube.physicsBody?.restitution = 0.0
cube.physicsBody?.isAffectedByGravity = true
cube.physicsBody?.mass = 25.0
cube.physicsBody?.friction = 1.0
cube.physicsBody?.collisionBitMask = 1
cube.position = SCNVector3(x: xPos, y: yPos, z: zPos)
let details = CubeDetails(color: cubeColorString, position: cube.position)
//add cube details to the master dict
masterCubeDict[cube] = details
//print(masterCubeDict)
xPos += cubeWidth + spaceBetweenCubes
self.addChildNode(cube)
}
xPos = -cubeOffsetDistance
yPos += cubeWidth + spaceBetweenCubes
}
xPos = -cubeOffsetDistance
yPos = -cubeOffsetDistance
zPos += cubeWidth + spaceBetweenCubes
}
}
private func cubeOffsetDistance()->Float {
return (cubeWidth + spaceBetweenCubes) / 2
}
private func randomColor() -> UIColor{
var tmpColor: UIColor
let num = Int.random(in:0...2)
switch num {
case 0:
tmpColor = UIColor.red
case 1:
tmpColor = UIColor.blue
case 2:
tmpColor = UIColor.green
default:
tmpColor = UIColor.black
}
return tmpColor
}
required init?(coder aDecoder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
}
GameViewController
import UIKit
import QuartzCore
import SceneKit
var myMasterCubeDict: [SCNNode: Cube.CubeDetails] = [:]
class GameViewController: UIViewController {
let gameCube = Cube()
override func viewDidLoad() {
super.viewDidLoad()
// create a new scene
// let scene = SCNScene(named: "art.scnassets/ship.scn")!
let scene = SCNScene()
// create and add a camera to the scene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
scene.rootNode.addChildNode(cameraNode)
// place the camera
cameraNode.position = SCNVector3(x: 2, y: 0, z: 20)
// create and add a light to the scene
let lightNode = SCNNode()
lightNode.light = SCNLight()
lightNode.light!.type = .omni
lightNode.position = SCNVector3(x: 0, y: 10, z: 10)
scene.rootNode.addChildNode(lightNode)
// create and add an ambient light to the scene
let ambientLightNode = SCNNode()
ambientLightNode.light = SCNLight()
ambientLightNode.light!.type = .ambient
ambientLightNode.light!.color = UIColor.darkGray
scene.rootNode.addChildNode(ambientLightNode)
// init cube
myMasterCubeDict = gameCube.masterCubeDict
scene.rootNode.addChildNode(gameCube)
// retrieve the SCNView
let scnView = self.view as! SCNView
// set the scene to the view
scnView.scene = scene
// allows the user to manipulate the camera
scnView.allowsCameraControl = true
// show statistics such as fps and timing information
scnView.showsStatistics = true
// configure the view
scnView.backgroundColor = UIColor.black
// add a tap gesture recognizer
let tapGesture = UITapGestureRecognizer(target: self, action: #selector(handleTap(_:)))
scnView.addGestureRecognizer(tapGesture)
}
#objc
func handleTap(_ gestureRecognize: UIGestureRecognizer) {
// retrieve the SCNView
let scnView = self.view as! SCNView
// check what nodes are tapped
let p = gestureRecognize.location(in: scnView)
let hitResults = scnView.hitTest(p, options: [:])
// check that we clicked on at least one object
if hitResults.count > 0 {
// retrieved the first clicked object
let result = hitResults[0]
//get dict of same-color node
var dictOfSameColor = findAndReturnChain(boi: result.node)
// print(dictOfSameColor)
var finalNodes: [SCNNode] = [result.node]
var resFlag = 1
repeat {
var xSame: Bool = false
var ySame: Bool = false
var zSame: Bool = false
resFlag = 0
for node in finalNodes {
// var nodeX = node.position.x
for (key, value) in dictOfSameColor {
if(abs(node.position.x - value.position.x) < 0.7) {
xSame = true
}
if(abs(node.position.y - value.position.y) < 0.7) {
ySame = true
}
if(abs(node.position.z - value.position.z) < 0.7) {
zSame = true
}
//print("X-val: \(xDif) \nY-val: \(yDif) \nZ-val: \(zDif) \nColor: \(key.name) \n\n\n\n")
if (xSame && ySame ) {
if !(zSame) {
if (abs((node.position.z-value.position.z)) < 2) {
finalNodes.append(key)
dictOfSameColor.removeValue(forKey: key)
resFlag = 1
}
}
}
if (xSame && zSame) {
if !(ySame) {
if (abs((node.position.y-value.position.y)) < 2) {
finalNodes.append(key)
dictOfSameColor.removeValue(forKey: key)
resFlag = 1
}
}
}
if (ySame && zSame) {
if !(xSame) {
if (abs((node.position.x-value.position.x)) < 2) {
finalNodes.append(key)
dictOfSameColor.removeValue(forKey: key)
resFlag = 1
}
}
}
xSame = false
ySame = false
zSame = false
}
}
//print(finalNodes)
} while resFlag == 1
//print(finalNodes)
for node in finalNodes {
if node.name != "floor" {
node.removeFromParentNode()
}
}
//IMPLEMENT: Reset dicts to current state of the cube
myMasterCubeDict = updateMasterCubeDict(cube: gameCube)
dictOfSameColor.removeAll()
}
}
func findAndReturnChain(boi: SCNNode) -> [SCNNode:Cube.CubeDetails] {
var ret: [SCNNode:Cube.CubeDetails] = [:]
//find cubes with the same color
for (key, value) in myMasterCubeDict {
if value.color == boi.name {
ret[key] = value
}
}
return ret
}
func updateMasterCubeDict(cube: Cube) -> [SCNNode:Cube.CubeDetails] {
myMasterCubeDict.removeAll()
var newNode: SCNNode = SCNNode()
var newDetails = Cube.CubeDetails(color: "", position: SCNVector3Zero)
cube.enumerateChildNodes { (cube, stop) in
newNode = cube
if let newName = cube.name {
newDetails.color = newName
}
newDetails.position = cube.position
myMasterCubeDict[newNode] = newDetails
}
return myMasterCubeDict
}
override var shouldAutorotate: Bool {
return true
}
override var prefersStatusBarHidden: Bool {
return true
}
override var supportedInterfaceOrientations: UIInterfaceOrientationMask {
if UIDevice.current.userInterfaceIdiom == .phone {
return .allButUpsideDown
} else {
return .all
}
}
}
I did a game somewhat like this. You could probably get the math to work, but the way I did it was to map out each node and have an array containing its adjacent nodes. Doing it this way, I'm sure that when I remove a node and loop through its adjacent[array] nodes, then I got the right ones.
I don't subclass SCNNodes - some do, but I create the class I want that contains info about my node - I add the node to Scenekit there, that's separates the actual node from other work I may want to do with the class. Some nodes have a lot of detail that I may want to manage separately (multiple particle systems, movements, etc). Then I just keep my classes of nodes in an array and each class has direct access to it's own node.
Sorry - I don't know about the bounce, there are a lot of choices with the physics engine.

Using Multiple AR Image Anchors to display dynamic 3D Overlay

I'm working on a project wherein we have to detect a certain number of custom QR codes (as ARImageAnchors) and then using the position of these anchors to dynamically display a 3D overlay. To be exact, we are planning to dynamically display a 3D model of human anatomy over the anchors which will be placed on a mannequin. For example, the mannequin we are placing the QR codes on is smaller or bigger than the default size of the 3D model, we would like it to adapt based on the distances between the images. Below is the sample code I'm thinking of working off from (source: https://www.appcoda.com/arkit-image-recognition/).
import UIKit
import ARKit
class ViewController: UIViewController {
#IBOutlet weak var sceneView: ARSCNView!
#IBOutlet weak var label: UILabel!
let fadeDuration: TimeInterval = 0.3
let rotateDuration: TimeInterval = 3
let waitDuration: TimeInterval = 0.5
lazy var fadeAndSpinAction: SCNAction = {
return .sequence([
.fadeIn(duration: fadeDuration),
.rotateBy(x: 0, y: 0, z: CGFloat.pi * 360 / 180, duration: rotateDuration),
.wait(duration: waitDuration),
.fadeOut(duration: fadeDuration)
])
}()
lazy var fadeAction: SCNAction = {
return .sequence([
.fadeOpacity(by: 0.8, duration: fadeDuration),
.wait(duration: waitDuration),
.fadeOut(duration: fadeDuration)
])
}()
lazy var treeNode: SCNNode = {
guard let scene = SCNScene(named: "tree.scn"),
let node = scene.rootNode.childNode(withName: "tree", recursively: false) else { return SCNNode() }
let scaleFactor = 0.005
node.scale = SCNVector3(scaleFactor, scaleFactor, scaleFactor)
node.eulerAngles.x = -.pi / 2
return node
}()
lazy var bookNode: SCNNode = {
guard let scene = SCNScene(named: "book.scn"),
let node = scene.rootNode.childNode(withName: "book", recursively: false) else { return SCNNode() }
let scaleFactor = 0.1
node.scale = SCNVector3(scaleFactor, scaleFactor, scaleFactor)
return node
}()
lazy var mountainNode: SCNNode = {
guard let scene = SCNScene(named: "mountain.scn"),
let node = scene.rootNode.childNode(withName: "mountain", recursively: false) else { return SCNNode() }
let scaleFactor = 0.25
node.scale = SCNVector3(scaleFactor, scaleFactor, scaleFactor)
node.eulerAngles.x += -.pi / 2
return node
}()
override func viewDidLoad() {
super.viewDidLoad()
sceneView.delegate = self
configureLighting()
}
func configureLighting() {
sceneView.autoenablesDefaultLighting = true
sceneView.automaticallyUpdatesLighting = true
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
resetTrackingConfiguration()
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
sceneView.session.pause()
}
#IBAction func resetButtonDidTouch(_ sender: UIBarButtonItem) {
resetTrackingConfiguration()
}
func resetTrackingConfiguration() {
guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else { return }
let configuration = ARWorldTrackingConfiguration()
configuration.detectionImages = referenceImages
let options: ARSession.RunOptions = [.resetTracking, .removeExistingAnchors]
sceneView.session.run(configuration, options: options)
label.text = "Move camera around to detect images"
}
}
extension ViewController: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
DispatchQueue.main.async {
guard let imageAnchor = anchor as? ARImageAnchor,
let imageName = imageAnchor.referenceImage.name else { return }
// TODO: Comment out code
// let planeNode = self.getPlaneNode(withReferenceImage: imageAnchor.referenceImage)
// planeNode.opacity = 0.0
// planeNode.eulerAngles.x = -.pi / 2
// planeNode.runAction(self.fadeAction)
// node.addChildNode(planeNode)
// TODO: Overlay 3D Object
let overlayNode = self.getNode(withImageName: imageName)
overlayNode.opacity = 0
overlayNode.position.y = 0.2
overlayNode.runAction(self.fadeAndSpinAction)
node.addChildNode(overlayNode)
self.label.text = "Image detected: \"\(imageName)\""
}
}
func getPlaneNode(withReferenceImage image: ARReferenceImage) -> SCNNode {
let plane = SCNPlane(width: image.physicalSize.width,
height: image.physicalSize.height)
let node = SCNNode(geometry: plane)
return node
}
func getNode(withImageName name: String) -> SCNNode {
var node = SCNNode()
switch name {
case "Book":
node = bookNode
case "Snow Mountain":
node = mountainNode
case "Trees In the Dark":
node = treeNode
default:
break
}
return node
}
}
I know that the 3D overlay is displayed in the renderer function above, but it is only displaying on top of a single detected image anchor. Now my question is, is it possible to reference multiple ARImage anchors to dynamically display a single 3D model?
Being a novice in ARKit and Swift in general, I'm not sure how to go about this problem yet. I'm hoping someone might have an idea of how to work around this and point me to the right direction. Any help will be greatly appreciated!
Thanks in advance!

ARkit image recognition and AR visualization of the image

In the company where I work I need to do this application. I have to recognize an image of a painting, and to visualize it in AR once recognized (in practice I find the real picture and above the painting in AR) I visualize the text or the selectable points with various characteristics of the picture in question. At the moment I have this code for the AR that recognizes the image in question and I visualize a plan above it. can you help me to create maybe a view above the picture with the features listed above?
import ARKit
import SceneKit
import UIKit
class ViewController: UIViewController, ARSCNViewDelegate {
#IBOutlet var sceneView: ARSCNView!
#IBOutlet weak var blurView: UIVisualEffectView!
/// The view controller that displays the status and "restart experience" UI.
lazy var statusViewController: StatusViewController = {
return childViewControllers.lazy.compactMap({ $0 as? StatusViewController }).first!
}()
/// A serial queue for thread safety when modifying the SceneKit node graph.
let updateQueue = DispatchQueue(label: Bundle.main.bundleIdentifier! +
".serialSceneKitQueue")
/// Convenience accessor for the session owned by ARSCNView.
var session: ARSession {
return sceneView.session
}
// MARK: - View Controller Life Cycle
override func viewDidLoad() {
super.viewDidLoad()
sceneView.delegate = self
sceneView.session.delegate = self
// Hook up status view controller callback(s).
statusViewController.restartExperienceHandler = { [unowned self] in
self.restartExperience()
}
}
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
// Prevent the screen from being dimmed to avoid interuppting the AR experience.
UIApplication.shared.isIdleTimerDisabled = true
// Start the AR experience
resetTracking()
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
session.pause()
}
// MARK: - Session management (Image detection setup)
/// Prevents restarting the session while a restart is in progress.
var isRestartAvailable = true
/// Creates a new AR configuration to run on the `session`.
/// - Tag: ARReferenceImage-Loading
func resetTracking() {
guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else {
fatalError("Missing expected asset catalog resources.")
}
let configuration = ARWorldTrackingConfiguration()
configuration.detectionImages = referenceImages
session.run(configuration, options: [.resetTracking, .removeExistingAnchors])
statusViewController.scheduleMessage("Look around to detect images", inSeconds: 7.5, messageType: .contentPlacement)
}
// MARK: - ARSCNViewDelegate (Image detection results)
/// - Tag: ARImageAnchor-Visualizing
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
updateQueue.async {
// Create a plane to visualize the initial position of the detected image.
let plane = SCNPlane(width: referenceImage.physicalSize.width,
height: referenceImage.physicalSize.height)
let planeNode = SCNNode(geometry: plane)
planeNode.opacity = 0.25
/*
`SCNPlane` is vertically oriented in its local coordinate space, but
`ARImageAnchor` assumes the image is horizontal in its local space, so
rotate the plane to match.
*/
planeNode.eulerAngles.x = -.pi / 2
/*
Image anchors are not tracked after initial detection, so create an
animation that limits the duration for which the plane visualization appears.
*/
planeNode.runAction(self.imageHighlightAction)
// Add the plane visualization to the scene.
node.addChildNode(planeNode)
}
DispatchQueue.main.async {
let imageName = referenceImage.name ?? ""
self.statusViewController.cancelAllScheduledMessages()
self.statusViewController.showMessage("Detected image “\(imageName)”")
}
}
var imageHighlightAction: SCNAction {
return .sequence([
.wait(duration: 0.25),
.fadeOpacity(to: 0.85, duration: 0.25),
.fadeOpacity(to: 0.15, duration: 0.25),
.fadeOpacity(to: 0.85, duration: 0.25),
.fadeOut(duration: 0.5),
.removeFromParentNode()
])
}
}
One way to tackle your problem is to create an SKScene and render it as an SCNMaterial.
Here is a fully commented example for you which makes use of the following ARSCNViewDelegate method:
//--------------------------
// MARK: - ARSCNViewDelegate
//--------------------------
extension ViewController: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
//1. Check We Have An ARImageAnchor And Have Detected Our Reference Image
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
//2. Get The Physical Width & Height Of Our Reference Image
let width = CGFloat(referenceImage.physicalSize.width)
let height = CGFloat(referenceImage.physicalSize.height)
//3. Create An SKScene Which We Will Display Above Our Target
let overlayNode = SCNNode()
let spriteKitScene = SKScene(size: CGSize(width: 600, height: 300))
spriteKitScene.backgroundColor = .clear
let imageName = SKLabelNode(text: "Line Friends")
imageName.position = CGPoint(x: 600/2, y: 240)
imageName.horizontalAlignmentMode = .center
imageName.fontSize = 60
imageName.fontName = "San Fransisco"
spriteKitScene.addChild(imageName)
let artistName = SKLabelNode(text: "By Line Coorporation")
artistName.position = CGPoint(x: 600/2, y: 180)
artistName.horizontalAlignmentMode = .center
artistName.fontSize = 60
artistName.fontName = "San Fransisco"
spriteKitScene.addChild(artistName)
let creationDate = SKLabelNode(text: "Created 2011")
creationDate.position = CGPoint(x: 600/2, y: 120)
creationDate.horizontalAlignmentMode = .center
creationDate.fontSize = 60
creationDate.fontName = "San Fransisco"
spriteKitScene.addChild(creationDate)
let planeHeight = height/2
let overlayGeometry = SCNPlane(width: width, height: planeHeight)
overlayNode.geometry = overlayGeometry
//4. Add The SpriteKit Scene As The SCNPlane's Geometry
overlayGeometry.firstMaterial?.diffuse.contents = spriteKitScene
//5. Rotate The Material Contents So It Isn't Backwards
overlayGeometry.firstMaterial?.diffuse.contentsTransform = SCNMatrix4Translate(SCNMatrix4MakeScale(1, -1, 1), 0, 1, 0)
//6. Rotate The Node
overlayNode.transform = SCNMatrix4MakeRotation(-Float.pi / 2, 1, 0, 0)
//7. Place It About The Target
let zPosition = height - (planeHeight/2)
overlayNode.position = SCNVector3(0, 0, -zPosition)
//8. Add It To The Scene
node.addChildNode(overlayNode)
}
}
Which yields something like the following:
Obviously if you have multiple image targets, you would create a func to dynamically create your 'info' overlay...
Hope it points you in the right direction... And apologies for the heinous spelling of corporation! ^_______*

Is it possible to detect floor with objects placed on it using ARKit?

I want to detect floor using ARkit , when objects are present such as table etc. and want to color the detected surface, is this possible using ARkit, I am a newbie in ARKit world and not sure if this is possible and where to start.
Your session configuration function should be like this, you call it on viewWillAppear
func setUpSceneView() {
let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = .horizontal
sceneView.session.run(configuration)
sceneView.delegate = self
sceneView.debugOptions = [ARSCNDebugOptions.showFeaturePoints]
}
you can also use .vertical for walls or whatever..
on view did load add a tapGestureRecognizer to the scene so that you can add the SCNNode where you tap to the detected surface
addTapGestureToSceneView() inside viewDidLoad
func addTapGestureToSceneView() {
let tapGestureRecognizer = UITapGestureRecognizer(target: self, action: #selector(placeOnSurface.addNodeToSceneView(withGestureRecognizer:)))
sceneView.addGestureRecognizer(tapGestureRecognizer)
}
the method addNodeToSceneView will place SNNode on surface when it detects the plane..
#objc func addShipToSceneView(withGestureRecognizer recognizer: UIGestureRecognizer) {
let tapLocation = recognizer.location(in: sceneView)
let hitTestResults = sceneView.hitTest(tapLocation, types: .existingPlaneUsingExtent)
guard let hitTestResult = hitTestResults.first else { return }
let translation = hitTestResult.worldTransform.translation
let x = translation.x
let y = translation.y
let z = translation.z
var name:String = "YourModel.obj" //you can add node whatever
let group = DispatchGroup()
group.enter()
DispatchQueue.global(qos: .background).async {
print("This is run on the background queue")
let tempScene = SCNScene(named: name)!
var geom:SCNGeometry = tempScene.rootNode.childNodes[0].geometry!
var boxNode:SCNNode = SCNNode(geometry: geom)
boxNode.position = SCNVector3(x,y,z)
self.sceneView.scene.rootNode.addChildNode(boxNode)
group.leave()
}
group.notify(queue: .main){
//Here you know that the node is has been put
}
}
and you need some functions for render
extension float4x4 {
var translation: float3 {
let translation = self.columns.3
return float3(translation.x, translation.y, translation.z)
}
}
extension placeOnSurface: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
// 1
guard let planeAnchor = anchor as? ARPlaneAnchor else { return }
// 2
let width = CGFloat(planeAnchor.extent.x)
let height = CGFloat(planeAnchor.extent.z)
let plane = SCNPlane(width: width, height: height)
// 3
plane.materials.first?.diffuse.contents = UIColor.transparentLightBlue
// 4
let planeNode = SCNNode(geometry: plane)
// 5
let x = CGFloat(planeAnchor.center.x)
let y = CGFloat(planeAnchor.center.y)
let z = CGFloat(planeAnchor.center.z)
planeNode.position = SCNVector3(x,y,z)
planeNode.eulerAngles.x = -.pi / 2
// 6
node.addChildNode(planeNode)
}
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
// 1
guard let planeAnchor = anchor as? ARPlaneAnchor,
let planeNode = node.childNodes.first,
let plane = planeNode.geometry as? SCNPlane
else { return }
// 2
let width = CGFloat(planeAnchor.extent.x)
let height = CGFloat(planeAnchor.extent.z)
plane.width = width
plane.height = height
// 3
let x = CGFloat(planeAnchor.center.x)
let y = CGFloat(planeAnchor.center.y)
let z = CGFloat(planeAnchor.center.z)
planeNode.position = SCNVector3(x, y, z)
}
}
I also recommend you in viewDidLoad to call the function configureLighting() for better user experience
func configureLighting() {
sceneView.autoenablesDefaultLighting = true
sceneView.automaticallyUpdatesLighting = true
}

Why is SCNNode "jiggling" when dropped onto SCNPlane?

I have a SCNPlane that is added to the scene when a sufficient area is detected for a horizontal surface. The plane appears to be placed in a correct spot, according to the floor/table it's being placed on. The problem is when I drop a SCNNode(this has been consistent whether it was a box, pyramid, 3D-model, etc.) onto the plane, it will eventually find a spot to land and 99% start jiggling all crazy. Very few times has it just landed and not moved at all. I also think this may be cause by the node being dropped and landing slightly below the plane surface. It is not "on top" neither "below" the plane. Maybe the node is freaking out because it's kind of teetering between both levels?
Here is a video of what's going on, you can see at the beginning that the box is below and above the plane and the orange box does stop when it collides with the dark blue box, but does go back to its jiggling ways when the green box collides with it at the end:
The code is here on github
I will also show some of the relevant parts embedded in code:
I just create a Plane class to add to the scene when I need to
class Plane: SCNNode {
var anchor :ARPlaneAnchor
var planeGeometry :SCNPlane!
init(anchor :ARPlaneAnchor) {
self.anchor = anchor
super.init()
setup()
}
func update(anchor: ARPlaneAnchor) {
self.planeGeometry.width = CGFloat(anchor.extent.x)
self.planeGeometry.height = CGFloat(anchor.extent.z)
self.position = SCNVector3Make(anchor.center.x, 0, anchor.center.z)
let planeNode = self.childNodes.first!
planeNode.physicsBody = SCNPhysicsBody(type: .static, shape: SCNPhysicsShape(geometry: self.planeGeometry, options: nil))
}
private func setup() {
//plane dimensions
self.planeGeometry = SCNPlane(width: CGFloat(self.anchor.extent.x), height: CGFloat(self.anchor.extent.z))
//plane material
let material = SCNMaterial()
material.diffuse.contents = UIImage(named: "tronGrid.png")
self.planeGeometry.materials = [material]
//plane geometry and physics
let planeNode = SCNNode(geometry: self.planeGeometry)
planeNode.physicsBody = SCNPhysicsBody(type: .static, shape: SCNPhysicsShape(geometry: self.planeGeometry, options: nil))
planeNode.physicsBody?.categoryBitMask = BodyType.plane.rawValue
planeNode.position = SCNVector3Make(anchor.center.x, 0, anchor.center.z)
planeNode.transform = SCNMatrix4MakeRotation(Float(-Double.pi / 2.0), 1, 0, 0)
//add plane node
self.addChildNode(planeNode)
}
This is the ViewController
enum BodyType: Int {
case box = 1
case pyramid = 2
case plane = 3
}
class ViewController: UIViewController, ARSCNViewDelegate, SCNPhysicsContactDelegate {
//outlets
#IBOutlet var sceneView: ARSCNView!
//globals
var planes = [Plane]()
var boxes = [SCNNode]()
//life cycle
override func viewDidLoad() {
super.viewDidLoad()
//set sceneView's frame
self.sceneView = ARSCNView(frame: self.view.frame)
//add debugging option for sceneView (show x, y , z coords)
self.sceneView.debugOptions = [ARSCNDebugOptions.showFeaturePoints, ARSCNDebugOptions.showWorldOrigin]
//give lighting to the scene
self.sceneView.autoenablesDefaultLighting = true
//add subview to scene
self.view.addSubview(self.sceneView)
// Set the view's delegate
sceneView.delegate = self
//subscribe to physics contact delegate
self.sceneView.scene.physicsWorld.contactDelegate = self
//show statistics such as fps and timing information
sceneView.showsStatistics = true
//create new scene
let scene = SCNScene()
//set scene to view
sceneView.scene = scene
//setup recognizer to add scooter to scene
let tapGestureRecognizer = UITapGestureRecognizer(target: self, action: #selector(tapped))
sceneView.addGestureRecognizer(tapGestureRecognizer)
}
//MARK: helper funcs
#objc func tapped(recognizer: UIGestureRecognizer) {
let scnView = recognizer.view as! ARSCNView
let touchLocation = recognizer.location(in: scnView)
let touch = scnView.hitTest(touchLocation, types: .existingPlaneUsingExtent)
//take action if user touches box
if !touch.isEmpty {
guard let hitResult = touch.first else { return }
addBox(hitResult: hitResult)
}
}
private func addBox(hitResult: ARHitTestResult) {
let boxGeometry = SCNBox(width: 0.1,
height: 0.1,
length: 0.1,
chamferRadius: 0)
let material = SCNMaterial()
material.diffuse.contents = UIColor(red: .random(),
green: .random(),
blue: .random(),
alpha: 1.0)
boxGeometry.materials = [material]
let boxNode = SCNNode(geometry: boxGeometry)
//adding physics body, a box already has a shape, so nil is fine
boxNode.physicsBody = SCNPhysicsBody(type: .dynamic, shape: nil)
//set bitMask on boxNode, enabling objects with diff categoryBitMasks to collide w/ each other
boxNode.physicsBody?.categoryBitMask = BodyType.plane.rawValue | BodyType.box.rawValue
boxNode.position = SCNVector3(hitResult.worldTransform.columns.3.x,
hitResult.worldTransform.columns.3.y + 0.3,
hitResult.worldTransform.columns.3.z)
self.sceneView.scene.rootNode.addChildNode(boxNode)
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = .horizontal
//track objects in ARWorld and start session
sceneView.session.run(configuration)
}
//MARK: - ARSCNViewDelegate
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
//if no anchor found, don't render anything!
if !(anchor is ARPlaneAnchor) {
return
}
DispatchQueue.main.async {
//add plane to scene
let plane = Plane(anchor: anchor as! ARPlaneAnchor)
self.planes.append(plane)
node.addChildNode(plane)
//add initial scene object
let pyramidGeometry = SCNPyramid(width: CGFloat(plane.planeGeometry.width / 8), height: plane.planeGeometry.height / 8, length: plane.planeGeometry.height / 8)
pyramidGeometry.firstMaterial?.diffuse.contents = UIColor.white
let pyramidNode = SCNNode(geometry: pyramidGeometry)
pyramidNode.name = "pyramid"
pyramidNode.physicsBody = SCNPhysicsBody(type: .dynamic, shape: nil)
pyramidNode.physicsBody?.categoryBitMask = BodyType.pyramid.rawValue | BodyType.plane.rawValue
pyramidNode.physicsBody?.contactTestBitMask = BodyType.box.rawValue
pyramidNode.position = SCNVector3(-(plane.planeGeometry.width) / 3, 0, plane.planeGeometry.height / 3)
node.addChildNode(pyramidNode)
}
}
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
let plane = self.planes.filter {
plane in return plane.anchor.identifier == anchor.identifier
}.first
if plane == nil {
return
}
plane?.update(anchor: anchor as! ARPlaneAnchor)
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
//pause session
sceneView.session.pause()
}
}
I think i followed the same tutorial. I also had same result. Reason is because when the cube drops from higher place, it accelerates and doesnot exactly hit on the plane but passes through. If you scale down the cube to '1 mm' you can see box completely passes through plane and continue falling below plane. You can try droping cube from nearer to the plane, box drops slower and this 'jiggling' will not occur. Or you can try with box with small height instead of plane.
I had the same problem i found out one solution.I was initializing the ARSCNView programmatically.I just removed those code and just added a ARSCNView in the storyboard joined it in my UIViewcontroller class using IBOutlet it worked like a charm.
Hope it helps anyone who is going through this problem.
The same code is below.
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
self.sceneView.debugOptions = [ARSCNDebugOptions.showFeaturePoints,ARSCNDebugOptions.showWorldOrigin]
sceneView.delegate = self
sceneView.showsStatistics = true
let scene = SCNScene()
sceneView.scene = scene
}
The "jiggling" is probably caused by an incorrect gravity vector. Try experimenting with setting the gravity of your scene.
For example, add this to your viewDidLoad function:
sceneView.scene.physicsWorld.gravity = SCNVector3Make(0.0, -1.0, 0.0)
I found that setting the gravity - either through code, or by loading an empty scene - resolves this issue.

Resources