Swift -What is a SCNCamera's Definition and Purpose - ios

I can't find a good explanation of what a SCNCamera is and it's purpose. This is Apple's definition:
A set of camera attributes that can be attached to a node to provide a
point of view for displaying the scene.
This definition isn't clear because I set up the scene and added a SCNNode without attaching a SCNCamera to it. The point of view from the device's camera shows the SCNNode at the location I positioned it at with no problem and the scene is displayed fine.
What is the difference between the device's camera and a SCNCamera?
What is the benefit of attaching a SCNCamera to a SCNNode vs not using one?
If I have multiple SCNNodes (all detached no hierarchy amongst each other) does each node need it's own SCNCamera?
If I have multiple SCNNodes in a hierarchy (parent node with child nodes) does each node need it's own SCNCamera or does just the parent node?
lazy var sceneView: ARSCNView = {
let sceneView = ARSCNView()
sceneView.translatesAutoresizingMaskIntoConstraints = false
sceneView.delegate = self
return sceneView
}()
let configuration = ARWorldTrackingConfiguration()
override func viewDidLoad() {
super.viewDidLoad()
// pin sceneView to the view
let material = SCNMaterial()
material.diffuse.contents = UIImage(named: "earth")
let plane = SCNPlane(width: 0.33, height: 0.33)
plane.materials = [material]
plane.firstMaterial?.isDoubleSided = true
let myNode = SCNNode(geometry: plane)
myNode.name = "earth"
myNode.position = SCNVector3(0.0, 0.6, -0.9)
sceneView.scene.rootNode.addChildNode(myNode)
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
sceneView.session.run(configuration, options: [])
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillAppear(animated)
sceneView.session.pause()
sceneView.session.run(configuration, options: [.resetTracking, .removeExistingAnchors])
}

In SceneKit, the SCNCamera represents the point of view from which the user sees a scene. Ray Wenderlich provides a good explanation:
Think back to the analogy of the movie set from Chapter 1: to shoot a
scene, you’d position a camera looking at the scene and the resulting
image of that scene would be from the camera’s perspective.
Scene Kit
works in a similar fashion; the position of the node that contains the
camera determines the point of view from which you view the scene.
You do not need to have a SCNCamera for each node. You should only need to have one camera for each angle that you want to show, or even just one. You can move one camera throughout the scene using its parent's position property.
It looks like you're working with ARKit, which behaves a little differently. When using an ARSCNView, as opposed to a non-AR SCNView, you get the following behvior:
The view automatically renders the live video feed from the device camera as the scene background.
The world coordinate system of the view's SceneKit scene directly responds to the AR world coordinate system established by the session
configuration.
The view automatically moves its SceneKit camera to match the real-world movement of the device.
You do not need to worry as much about the scene's camera in this case, as it is automatically being controlled by the system so that it matches the device's movement for AR.
For more detail, see Apple's documentation on SCNCamera: SCNCamera - SceneKit

I got the answer to my question from within this answer. Basically in ARKit using ARSCNView the camera comes from sceneView.pointOfView but in SceneKit you need to create a camera to get the camera pov (code below).
Getting the camera node
To get the camera node, it depends if you're using SCNKit, ARKit, or other framework. Below are examples for ARKit and SceneKit.
With ARKit, you have ARSCNView to render the 3D objects of an SCNScene overlapping the camera content. You can get the camera node from ARSCNView's pointOfView property:
let cameraNode = sceneView.pointOfView
For SceneKit, you have an SCNView that renders the 3D objects of an SCNScene. You can create camera nodes and position them wherever you want, so you'd do something like:
let scnScene = SCNScene()
// (Configure scnScene here if necessary)
scnView.scene = scnScene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(0, 5, 10) // For example
scnScene.rootNode.addChildNode(cameraNode)
Once a camera node has been setup, you can access the current camera in the same way as ARKit:
let cameraNode = scnView.pointOfView

Related

SceneKit camera causes objects to Not Show Up

I want to be able to create a custom camera node in SceneKit and view my scene from it (instead of the default camera).
However, I've been encountering a very strange issue with SceneKit:
If I use SCNCamera, nothing shows up in my scene.
If I don't use SCNCamera, the objects in my scene render
correctly.
This is the code I am using (very simple code; from a tutorial):
import UIKit
import SceneKit
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
let sceneView = SCNView()
sceneView.frame = self.view.frame
self.view.addSubview(sceneView)
let scene = SCNScene()
sceneView.autoenablesDefaultLighting = true
sceneView.allowsCameraControl = true
let cameraNode = SCNNode()
// If the below line of code is commented out (so no SCNCamera is added), everything shows up
cameraNode.camera = SCNCamera()
scene.rootNode.addChildNode(cameraNode)
let sphere = SCNSphere(radius: 5)
sphere.firstMaterial?.diffuse.contents = UIColor.red
let sphereNode = SCNNode(geometry: sphere)
cameraNode.addChildNode(sphereNode)
sceneView.backgroundColor = UIColor.green
sceneView.scene = scene
}
}
This seems pretty straightforward, yet I can't find any reason why this is happening on SO, etc.
Strangely, I also observe that if I try to access the camera node via sceneView.pointOfView, I get nil, even though sceneView.allowsCameraControl is set to true
Any help is appreciated!
The sphere is a child node of the camera, without any offset (its position is (0, 0, 0)) and so the camera is inside the sphere. And if the sphere's material isn't doubleSided then you won't see anything.

How to place 3D object (.scn file) on detected vertical plane which should be parallel to plane in ARKIt?

I am going to detect horizontal and vertical plane in ARKit. After detecting whether it is horizontal or vertical surface, respectively add plane of gray color on detected surface.On tap on detected plane I am going to add 3D object of .scn file.
My code is working fine for placing 3D object (.scn file) on horizontal plane but not working correctly with vertical plane.
3D object (.scn file) for vertical plane like photo frame is facing right in SceneKit editor. So I changed it’s EularAngleY to -0 and it’s facing front now in SceneKit Editor. When I tap on detected vertical plane which is facing to front then also photo frame is facing to right and If I move device facing right and place photo frame then it’s correct.
I want to place 3D object .scn file which should be parallel to plane (If vertical plane is facing front then it should be face front even in .scn file it faces to any direction).That 3D object is not parallel to detected plane.
Have I needed to change rotation also with respect to detected plane’s angle or need to do any changes in .scn file in SceneKit editor? How can I achieve it?
Please check below code on hitting detected vertical plane. Is there anything wrong?
#objc func addObjectToSceneView1(withGestureRecognizer recognizer: UIGestureRecognizer){
let tapLocation = recognizer.location(in: sceneView)
let hitTestResults = sceneView.hitTest(tapLocation, types: .existingPlaneUsingExtent)
guard let hitTestResult = hitTestResults.first, let anchor = hitTestResult.anchor as? ARPlaneAnchor else { return }
let translation = hitTestResult.worldTransform.columns.3
let x = translation.x
let y = translation.y
let z = translation.z
guard let shipScene = SCNScene(named: "art.scnassets/frame/frame.scn"),
let shipNode = shipScene.rootNode.childNode(withName: "frame", recursively: true)
else { return }
shipNode.position = SCNVector3(x,y,z)
sceneView.scene.rootNode.addChildNode(shipNode)
}
.scn file is like below. Is this .scn file correct? Or x should be with frame's depth? Everytime whenever I tap on plane it will show image like this only.
As I mentioned in the answer here, it is better to add an ARAnchor to the ARSession rather than directly adding an SCNNode into the scene graph after doing a hit test. Currently the code you posted doesn't take into account the rotation of the detected plane. For the code to work you would need to determine the normal of the detected plane take the dot product and cross product with the desired orientation of the model calculate the rotation, then apply the rotation. However, the ARSession will do all of that for you. By using ARAnchor(transform: hitTestResult.worldTransform) the rotation is encoded into the anchor. So you will only need to deal with transformations with the models own local coordinate space.
For example:
#objc func addObjectToSceneView1(withGestureRecognizer recognizer: UIGestureRecognizer){
let tapLocation = recognizer.location(in: sceneView)
let hitTestResults = sceneView.hitTest(tapLocation, types: .existingPlaneUsingExtent)
guard let hitTestResult = hitTestResults.first, let anchor = hitTestResult.anchor as? ARPlaneAnchor else { return }
// create anchor and add to session and wait for callback
let anchor = ARAnchor(transform: hitTestResult.worldTransform)
sceneView.session.add(anchor: anchor)
}
Then in your session delegate call back:
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
if anchor is ARPlaneAnchor {
// node for plane anchor
let anchorNode = SCNNode()
return anchorNode
} else {
// must be node for most recent hit test
guard let frameScene = SCNScene(named: "art.scnassets/frame/frame.scn"),
let frameNode = frameScene.rootNode.childNode(withName: "frame", recursively: true) else { return nil }
return frameNode
}
}
In you're scn file you'll want the model to be located at the origin laying flat without any transforms. This means you'll likely need to nest nodes and position the underlying model relative to a parent empty node.
Here "frame" is the outer node with no transform that is returned from nodeForAnchor and "picture" is rotated to be flat and scaled to the size of the content.
Final result:

SCNCamera made programmatically not in the correct position

I am trying to programmatically create a camera for my MainScene.scn file.
I need to create the camera in code, as I am wanting to make a camera orbit node, and this is the only way I can think of. I also would like to keep using my scene file.
This is my code (simplified) from my view controller:
import UIKit
import SceneKit
class GameViewController: UIViewController {
// MARK: Scene objects
private var gameView: SCNView!
private var gameScene: SCNScene!
private var gameCameraNode: SCNNode!
// MARK: View controller overrides
override func viewDidLoad() {
super.viewDidLoad()
// Setup game
initView()
initScene()
initCamera()
}
}
private extension GameViewController {
// Initialise the view and scene
private func initView() {
self.view = SCNView(frame: view.bounds) // Create an SCNView to play the game within
gameView = self.view as? SCNView // Assign the view
gameView.showsStatistics = true // Show game statistics
gameView.autoenablesDefaultLighting = true // Allow default lighting
gameView.antialiasingMode = .multisampling2X // Use anti-aliasing for a smoother look
}
private func initScene() {
gameScene = SCNScene(named: "art.scnassets/MainScene.scn")! // Assign the scene
gameView.scene = gameScene // Set the game view's scene
gameView.isPlaying = true // The scene is playing (not paused)
}
private func initCamera() {
gameCameraNode = SCNNode()
gameCameraNode.camera = SCNCamera()
gameCameraNode.position = SCNVector3(0, 3.5, 27)
gameCameraNode.eulerAngles = SCNVector3(-2, 0, 0)
gameScene.rootNode.addChildNode(gameCameraNode)
gameView.pointOfView = gameCameraNode
}
}
This code can easily be pasted to replace the default view controller code. All you need to do is add the MainScene.scn and drag in something like a box.
If you try the code, the camera is in the wrong position. If I use the same properties for the camera in the scene, it works, but that is not what I am looking for.
From what I have read, SceneKit may be creating a default camera as said here and here. However, I am setting the pointOfView property just as they said in those answers, but it still does not work.
How can I place my camera in the correct position in the scene programmatically?
After a while, I discovered that you can actually add empty nodes directly within the Scene Builder. I originally only wanted a programmatic answer, as I wanted to make a camera orbit node like the questions I linked to. Now I can add an empty node, I can make the child of the orbit the camera.
This requires no code, unless you want to access the nodes (e.g. changing position or rotation):
gameCameraNode = gameView.pointOfView // Use camera object from scene
gameCameraOrbitNode = gameCameraNode.parent // Use camera orbit object from scene
Here are the steps to create an orbit node:
1) Drag it in from the Objects Library:
2) Setup up your Scene Graph like so:

How to tap and move scene nodes in ARKit

I'm currently trying to build an AR Chess app and I'm having trouble getting the movement of the pieces working.
I would like to be able to tap on a chess piece, then the legal moves it can make on the chess board will be highlighted and it will move to whichever square the user tapped on.
Pic of the chess board design and nodes:
https://gyazo.com/2a88f9cda3f127301ed9b4a44f8be047
What I would like to implement:
https://imgur.com/a/IGhUDBW
Would greatly appreciate any suggestions on how to get this working.
Thanks!
ViewController Code:
import UIKit
import SceneKit
import ARKit
class ViewController: UIViewController, ARSCNViewDelegate {
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
// Set the view's delegate
sceneView.delegate = self
// Show statistics such as fps and timing information
sceneView.showsStatistics = true
// Add lighting to the scene
sceneView.autoenablesDefaultLighting = true
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
// Create a session configuration to track an external image
let configuration = ARImageTrackingConfiguration()
// Image detection
// Reference which group to find the image to detect in the Assets folder e.g. "Detection Card"
if let imageDetect = ARReferenceImage.referenceImages(inGroupNamed: "Detection Card", bundle: Bundle.main) {
// Sets image tracking properties to the image in the referenced group
configuration.trackingImages = imageDetect
// Amount of images to be tracked
configuration.maximumNumberOfTrackedImages = 1
}
// Run the view's session
sceneView.session.run(configuration)
}
// Run when horizontal surface is detected and display 3D object onto image
// ARAnchor - tells a certain point in world space is relevant to your app, makes virtual content appear "attached" to some real-world point of interest
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode {
// Creates 3D object
let obj = SCNNode()
// Check if image detected through camera is an ARImageAnchor - which contains position and orientation data about the image detected in the session
if let imageAnchor = anchor as? ARImageAnchor {
// Set dimensions of the horizontal plane to be displayed onto the image to be the same as the image uploaded
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
// Display mild transparent layer onto detected image
// This is to ensure image detection works by display a faint layer on the image
plane.firstMaterial?.diffuse.contents = UIColor(white: 1.0, alpha: 0.2)
// Set geometry shape of the plane
let planeNode = SCNNode(geometry: plane)
// Flip vertical plane to horizontal plane
planeNode.eulerAngles.x = -Float.pi / 2
obj.addChildNode(planeNode)
// Initialise chess scene
if let chessBoardSCN = SCNScene(named: "art.scnassets/chess.scn") {
// If there is a first in the scene file
if let chessNodes = chessBoardSCN.rootNode.childNodes.first {
// Displays chessboard upright
chessNodes.eulerAngles.x = Float.pi / 2
// Adds chessboard to the overall 3D scene
obj.addChildNode(chessNodes)
}
}
}
return obj
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
// Pause the view's session
sceneView.session.pause()
}
}
You will need to add gestures on to your view and use the ARSceneViews hitTest method to detect what the gesture is touching in your scene. You can then update the positions based on the movement from the gestures.
Here is a question that deals with roughly the same requirement of dragging nodes around.
Placing, Dragging and Removing SCNNodes in ARKit
First, you need to add a gesture recognizer for tap into your viewDidLoad, like this:
let tapGesture = UITapGestureRecognizer(target: self, action: #selector(handleTap(_:)))
myScnView.addGestureRecognizer(tapGesture)
Then realize the handler function:
#objc
func handleTap(_ gestureRecognize: UIGestureRecognizer) {
// HERE YOU NEED TO DETECT THE TAP
// check what nodes are tapped
let location = gestureRecognize.location(in: myScnView)
let hitResults = myScnView.hitTest(location, options: [:])
// check that we clicked on at least one object
if hitResults.count > 0 {
// retrieved the first clicked object
let tappedPiece = hitResults[0].node
// HERE YOU CAN SHOW POSSIBLE MOVES
//Ex. showPossibleMoves(for: tappedPiece)
}
}
Now, to show the possible moves, you need to identify all quadrants and your node position on the chessboard.
To do this, you can assign a name or a number, or a combination of letter and number, or moreover a combination of numbers. (I suggest combination of number, like row 1 column 1, like a matrix).
let's take my suggestion, so you can name each quadrant 1.1 1.2 ... 2.1 2.2 and so on.
Now, to detect where your piece is, you can check contact with the PhysicsContactDelegate.
Now you have the tappedPiece and the place where it is, so you have to define the rule for the pieces, for example:
let rules = ["tower":"cross"] //add the others
N.B You can choose what you want to define the rules.
Let's take my suggestion for good, now you should create the function to highlight:
func highlight(quadrant: SCNNode){
quadrant.geometry?.firstMaterial?.emission.contents = UIColor.yellow
}
Finally the showPossibleMoves(for: tappedPiece) could be something this:
func showPossibleMoves(for piece: SCNNode){
let pieceType = piece.name //You have to give the name as you did into your rules variable
//ex. if you have rules like ["tower":"cross"] you have to set all towers name to "tower"
let rule = rules[pieceType]
switch rule{
case "cross":
//you have to highlight all nodes on the right, left, above and bottom
// you can achieve this by selecting the start point and increase it
//assuming you named your quadrants like 1.1 1.2 or 11 12 13 ecc...
let startRow = Int(startQuadrant.name.first)
let startColumn = Int(startQuadrant.name.last)
//Now loop the highlight on right
for column in startColumn+1...MAX_COLUMN-1{
let quadrant = myScnView.scene.rootNode.childNode(withName:"\(startRow).\(column)" , recursively: true)
// call highlight function
highlight(quadrant: quadrant)
}
//Now loop for above quadrants
for row in startRow+1...MAX_ROW-1{
let quadrant = myScnView.scene.rootNode.childNode(withName:"\(row).\(startColumn)" , recursively: true)
// call highlight function
highlight(quadrant: quadrant)
}
//DO THE SAME FOR ALL DIRECTIONS
}
// ADD ALL CASES, like bishop movements "diagonals" and so on
}
NOTE: In the handlerTap function you have to check what you're tapping, for example, to check if you're tapping on a quadrant after selecting a piece (you want to move you're piece) you can check a boolean value and the name of the selected node
//assuming you have set the boolean value after selecting a piece
if pieceSelected && node.name != "tower"{
//HERE YOU CAN MOVE YOUR PIECE
}

marker detection and 3d model display for corresponding markers in ios(using ARkit)

Previously i have used vuforia(unity) to developed an AR app for iOS. Now i have to implement the same app using ARKit.
ARKit is awesome except there is no marker detection.
I have tried to use vision to detect markers and not successful so far.
can i have some samples for marker detection and displaying 3d models on the markers for iOS ?
Thanks in Advance.
There are a number of ways to achieve what your are looking for, although arguably the most simple is using images are markers.
As of ARKit 1.5 you are able to use ReferenceImages to place AR Content which are essentially the same as the markers you would use in Vuforia or EasyAR.
For your information a referenceImage is simply:
An image to be recognized in the real-world environment during a
world-tracking AR session.
To make use of this function you need to pass in a :
collection of reference images to your session configuration's
detectionImages property.
Which can be set like so:
var detectionImages: Set<ARReferenceImage>! { get set }
An important thing to note with ARKit 1.5 is that unlike Vuforia which can allow extended tracking of images:
Image detection doesn't continuously track real-world movement of the
image or track when the image disappears from view. Image detection
works best for cases where AR content responds to static images in the
scene—for example, identifying art in a museum or adding animated
elements to a movie poster.
As #Alexander said, your best bet for learning how this may be appropriate to your situation is looking at the SampleCode and Documentation online, which is available here:
Recognizing Images In An AR Experience
The core points however are these:
To Enable Image Detection:
You need to first provide one or more ARReferenceImage resources. These can be added manually using the AR asset catalog in Xcode, remembering that your must enter the physical size of the image in Xcode as accurately as possible since ARKit relies on this information to determine the distance of the image from the camera.
ARReferenceImages can also be created on the fly using the following methods:
init(CGImage, orientation: CGImagePropertyOrientation, physicalWidth: CGFloat)
Which creates a new reference image from a Core Graphics image
object.
init(CVPixelBuffer, orientation: CGImagePropertyOrientation, physicalWidth: CGFloat)
Which creates a new reference image from a Core Video pixel buffer.
Having done this, you then need to create a world tracking configuration in which you pass in your ARReferenceImages before running your ARSession e.g:
guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else { return }
let configuration = ARWorldTrackingConfiguration()
configuration.detectionImages = referenceImages
session.run(configuration, options: [.resetTracking, .removeExistingAnchors])
Handling Detection Of Images:
When an ARReferenceImage is detected by your ARSession and ARImageAnchor is created which simply provides:
Information about the position and orientation of an image detected in
a world-tracking AR session.
If image detection is successful therefore, you will need to use the following ARSCNViewDelegate callback to handle placement of your objects etc:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { }
An example of using this to handle placement of your 3D content would be like so:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
//1. If Out Target Image Has Been Detected Than Get The Corresponding Anchor
guard let currentImageAnchor = anchor as? ARImageAnchor else { return }
//2. Get The Targets Name
let name = currentImageAnchor.referenceImage.name!
//3. Get The Targets Width & Height
let width = currentImageAnchor.referenceImage.physicalSize.width
let height = currentImageAnchor.referenceImage.physicalSize.height
//4. Log The Reference Images Information
print("""
Image Name = \(name)
Image Width = \(width)
Image Height = \(height)
""")
//5. Create A Plane Geometry To Cover The ARImageAnchor
let planeNode = SCNNode()
let planeGeometry = SCNPlane(width: width, height: height)
planeGeometry.firstMaterial?.diffuse.contents = UIColor.white
planeNode.opacity = 0.25
planeNode.geometry = planeGeometry
//6. Rotate The PlaneNode To Horizontal
planeNode.eulerAngles.x = -.pi/2
//7. The Node Is Centered In The Anchor (0,0,0)
node.addChildNode(planeNode)
//8. Create AN SCNBox
let boxNode = SCNNode()
let boxGeometry = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
//9. Create A Different Colour For Each Face
let faceColours = [UIColor.red, UIColor.green, UIColor.blue, UIColor.cyan, UIColor.yellow, UIColor.gray]
var faceMaterials = [SCNMaterial]()
//10. Apply It To Each Face
for face in 0 ..< 5{
let material = SCNMaterial()
material.diffuse.contents = faceColours[face]
faceMaterials.append(material)
}
boxGeometry.materials = faceMaterials
boxNode.geometry = boxGeometry
//11. Set The Boxes Position To Be Placed On The Plane (node.x + box.height)
boxNode.position = SCNVector3(0 , 0.05, 0)
//12. Add The Box To The Node
node.addChildNode(boxNode)
}
Hope it helps...

Resources