I am trying to make an app where the user enters some text in a textfield and then the app displays this text in front of the ar camera to the user. I positioned the text correctly in front of the camera and I have changed the anchor of the text to be in the center of the text. but when I add the text into the scene the text is rotated 90 degrees around the z-axis. And I know why but I don't know how to solve it. The reason is that the camera of the arscene.session has a rotation of 0 for all x, y, z when the device is in landscape but since I want my app to be in portrait I rotate the device 90 degrees which rotates the camera as well and since the text has the same camera rotation, it's rotated as well. I tried correcting the rotation of the text by rotating it again around the z-axis but that doesn't solve the entire issue because when I change the direction of my phone, that affects the camera axis which will affect different axis of the text(not the same axis because I rotated the axis in the correction step). so I think the only way to solve the issue is to rotate the camera to be in consistent with the portrait mode from the beginning but I haven't found any way to set the rotation of the camera
here is the code of adding the text:
private func createTextNode(text:String?)
{
guard let text = text else {return}
let arText = SCNText(string: text, extrusionDepth: 1)
arText.font = UIFont(name: arText.font.fontName, size: 2)
arText.firstMaterial?.diffuse.contents = selectedColor
//making the node
let node = SCNNode()
node.geometry = arText
center(node: node)
guard let currentFrame = sceneView.session.currentFrame else {return}
let camera = currentFrame.camera
let cameraTransform = camera.transform
var newTransform = matrix_identity_float4x4
newTransform.columns.3.z = -0.2
let modifiedTransform = matrix_multiply(cameraTransform, newTransform)
node.transform = SCNMatrix4(modifiedTransform)
node.scale = SCNVector3(0.02, 0.02, 0.02)
self.sceneView.scene.rootNode.addChildNode(node)
node.eulerAngles.x = 90.degrees
}
and that's how the output looks like..
output
any help will be appreciated
You cannot use the matrix identity for any orientation, it has to be rotated depending on device orientation. I have a function in my apps that I call to update that before I perform the matrix multiplication :
var translation = matrix_identity_float4x4
func updateTranslationMatrix() {
switch UIDevice.current.orientation{
case .portrait, .portraitUpsideDown, .unknown, .faceDown, .faceUp:
print("portrait ")
translation.columns.0.x = -cos(.pi/2)
translation.columns.0.y = sin(.pi/2)
translation.columns.1.x = -sin(.pi/2)
translation.columns.1.y = -cos(.pi/2)
case .landscapeLeft :
print("landscape left")
translation.columns.0.x = 1
translation.columns.0.y = 0
translation.columns.1.x = 0
translation.columns.1.y = 1
case .landscapeRight :
print("landscape right")
translation.columns.0.x = cos(.pi)
translation.columns.0.y = -sin(.pi)
translation.columns.1.x = sin(.pi)
translation.columns.1.y = cos(.pi)
}
translation.columns.3.z = -0.6 //60cm in front of the camera
}
If you mean that you don't want the orientation of the device to change along with the rotation, then:
Go to Project > General > Deployment Info
Under device orientation, uncheck all boxes except 'Portrait'.
If this does not solve your problem and what you really want is to fix the euler angles of your text node, let me know, I'll be happy to help.
Related
I'm making an app where the user can create some flat shapes by positioning some points on a 3D space with ARKit, but it seems that the part where I create the UIBezierPath using these points is problematic.
In my app, the user starts by positioning a virtual transparent wall in AR at the same place that his device by pressing a button:
guard let currentFrame = sceneView.session.currentFrame else {
return
}
let imagePlane = SCNPlane(width: sceneView.bounds.width, height: sceneView.bounds.height)
imagePlane.firstMaterial?.diffuse.contents = UIColor.black
imagePlane.firstMaterial?.lightingModel = .constant
var windowNode = SCNNode()
windowNode.geometry = imagePlane
sceneView.scene.rootNode.addChildNode(windowNode)
windowNode.simdTransform = currentFrame.camera.transform
windowNode.opacity = 0.1
Then, the user place some points (some sphere nodes) on that wall to determine the shape of the flat object that he wants to create by pressing a button. If the user points back to the first sphere node created, I close the shape, create a node of it and place it at the same position that the wall:
let hitTestResult = sceneView.hitTest(self.view.center, options: nil)
if let firstHit = hitTestResult.first {
if firstHit.node == windowNode {
let x = Double(firstHit.worldCoordinates.x)
let y = Double(firstHit.worldCoordinates.y)
let pointCoordinates = CGPoint(x: x , y: y)
let sphere = SCNSphere(radius: 0.02)
sphere.firstMaterial?.diffuse.contents = UIColor.white
sphere.firstMaterial?.lightingModel = .constant
let sphereNode = SCNNode(geometry: sphere)
sceneView.scene.rootNode.addChildNode(sphereNode)
sphereNode.worldPosition = firstHit.worldCoordinates
if points.isEmpty {
windowPath.move(to: pointCoordinates)
} else {
windowPath.addLine(to: pointCoordinates)
}
points.append(sphereNode)
if undoButton.alpha == 0 {
undoButton.alpha = 1
}
} else if firstHit.node == points.first {
windowPath.close()
let windowShape = SCNShape(path: windowPath, extrusionDepth: 0)
windowShape.firstMaterial?.diffuse.contents = UIColor.white
windowShape.firstMaterial?.lightingModel = .constant
let tintedWindow = SCNNode(geometry: windowShape)
let worldPosition = windowNode.worldPosition
tintedWindow.worldPosition = worldPosition
sceneView.scene.rootNode.addChildNode(tintedWindow)
//removing all the sphere nodes from points and reinitializing the UIBezierPath windowPath
removeAllPoints()
}
}
That code works when I create a first invisible wall and a first shape, but when I create a second wall, when I'm done to draw my shape, the shape appears to be deformed and not at the right place like really not at the right place at all. So I think that I'm missing something with the coordinates of my UIBezierPath points but what ?
EDIT
Ok so after several tests, it seems that it depends on the orientation of the device at the launch of the AR session. When the device, at launch, faces the first wall that the user will create, the shape is created and places as expected. But if the user for exemple launch the app with his device pointed in one direction, then do a rotation of 90 degrees on himself, place the first wall and create his shape, the shape will be deformed and not at the right place.
So it seems that it's a problem of 3D coordinates but I still don't figure it out.
Ok I just found the problem ! I was just using the wrong vectors and coordinates... I've never been a math/geometry guy haha
So instead of using:
let x = Double(firstHit.worldCoordinates.x)
let y = Double(firstHit.worldCoordinates.y)
I now use:
let x = Double(firstHit.localCoordinates.x)
let y = Double(firstHit.localCoordinates.y)
And instead of using:
let worldPosition = windowNode.worldPosition
I now use:
let worldPosition = windowNode.transform
That's why the position of my shape node was depending of the initialisation of the AR session, I was working with world coordinates, seems obvious to me now.
I am creating a simple app with ARKit in which I add some text to the scene to the tapped position:
#objc func tapped(sender: UITapGestureRecognizer){
let sceneView = sender.view as! ARSCNView
let tapLocation = sender.location(in: sceneView)
let hitTest = sceneView.hitTest(tapLocation, types: .featurePoint)
if !hitTest.isEmpty{
self.addTag(tag: "A", hitTestResult: hitTest.first!)
}
else{
print("no match")
}
}
func addTag(tag: String, hitTestResult: ARHitTestResult){
let tag = SCNText(string:tag, extrusionDepth: 0.1)
tag.font = UIFont(name: "Optima", size: 1)
tag.firstMaterial?.diffuse.contents = UIColor.red
let tagNode = SCNNode(geometry: tag)
let transform = hitTestResult.worldTransform
let thirdColumn = transform.columns.3
tagNode.position = SCNVector3(thirdColumn.x,thirdColumn.y - tagNode.boundingBox.max.y / 2,thirdColumn.z)
print("\(thirdColumn.x) \(thirdColumn.y) \(thirdColumn.z)")
self.sceneView.scene.rootNode.addChildNode(tagNode)
}
It works, but I have problem with the orientation of the text. When I add it with the camera's original position, the text orientation is ok, I can see the text frontwise (Sample 1). But when I turn camera to the left / right, and add the text by tapping, I can see the added text from the side (Sample 2).
Sample 1:
Sample 2:
I know there should be some simple trick to solve it, but as a beginner in this topic I could not find it so far.
You want the text to always face the camera? SCNBillboardConstraint is your friend:
tagNode.constraints = [SCNBillboardConstraint()]
Am I correct in saying that you want the text to face the camera when you tap (wherever you happen to be facing), but then remain stationary?
There are a number of ways of adjusting the orientation of any node. For this case I would suggest simply setting the eulerAngles of the text node to be equal to those of the camera, at the point in which you instantiate the text.
In your addTag() function you add:
let eulerAngles = self.sceneView.session.currentFrame?.camera.eulerAngles
tagNode.eulerAngles = SCNVector3(eulerAngles.x, eulerAngles.y, eulerAngles.z + .pi / 2)
The additional .pi / 2 is there to ensure the text is in the correct orientation, as the default with ARKit is for a landscape orientation and therefore the text comes out funny. This applies a rotation around the local z axis.
It's also plausible (and some may argue it's better) to use .localRotate() of the node, or to access its transform property, however I like the approach of manipulating both the position and eulerAngles directly.
Hope this helps.
EDIT: replaced Float(1.57) with .pi / 2.
I'm using ARKit to display 3D objects. I managed to place the nodes in the real world in front of the user (aka the camera). But I don't manage to make them to face the camera when I drop them.
let tap_point=CGPoint(x: x, y: y)
let results=arscn_view.hitTest(tap_point, types: .estimatedHorizontalPlane)
guard results.count>0 else{
return
}
guard let r=results.first else{
return
}
let hit_tf=SCNMatrix4(r.worldTransform)
let new_pos=SCNVector3Make(hit_tf.m41, hit_tf.m42+Float(0.2), hit_tf.m43)
guard let scene=SCNScene(named: file_name) else{
return
}
guard let node=scene.rootNode.childNode(withName: "Mesh", recursively: true) else{
return
}
node.position=new_pos
arscn_view.scene.rootNode.addChildNode(node)
The nodes are well positioned on the plane, in front of the camera. But they are all looking in the same direction. I guess I should rotate the SCNNode but I didn't manage to do this.
First, get the rotation matrix of the camera:
let rotate = simd_float4x4(SCNMatrix4MakeRotation(sceneView.session.currentFrame!.camera.eulerAngles.y, 0, 1, 0))
Then, combine the matrices:
let rotateTransform = simd_mul(r.worldTransform, rotate)
Lastly, apply a transform to your node, casting as SCNMatrix4:
node.transform = SCNMatrix4(rotateTransform)
Hope that helps
EDIT
here how you can create SCNMatrix4 from simd_float4x4
let rotateTransform = simd_mul(r.worldTransform, rotate)
node.transform = SCNMatrix4(m11: rotateTransform.columns.0.x, m12: rotateTransform.columns.0.y, m13: rotateTransform.columns.0.z, m14: rotateTransform.columns.0.w, m21: rotateTransform.columns.1.x, m22: rotateTransform.columns.1.y, m23: rotateTransform.columns.1.z, m24: rotateTransform.columns.1.w, m31: rotateTransform.columns.2.x, m32: rotateTransform.columns.2.y, m33: rotateTransform.columns.2.z, m34: rotateTransform.columns.2.w, m41: rotateTransform.columns.3.x, m42: rotateTransform.columns.3.y, m43: rotateTransform.columns.3.z, m44: rotateTransform.columns.3.w)
guard let frame = self.sceneView.session.currentFrame else {
return
}
node.eulerAngles.y = frame.camera.eulerAngles.y
here's my code for the SCNNode facing the camera..hope help for someone
let location = touches.first!.location(in: sceneView)
var hitTestOptions = [SCNHitTestOption: Any]()
hitTestOptions[SCNHitTestOption.boundingBoxOnly] = true
let hitResultsFeaturePoints: [ARHitTestResult] = sceneView.hitTest(location, types: .featurePoint)
let hitTestResults = sceneView.hitTest(location)
guard let node = hitTestResults.first?.node else {
if let hit = hitResultsFeaturePoints.first {
let rotate = simd_float4x4(SCNMatrix4MakeRotation(sceneView.session.currentFrame!.camera.eulerAngles.y, 0, 1, 0))
let finalTransform = simd_mul(hit.worldTransform, rotate)
sceneView.session.add(anchor: ARAnchor(transform: finalTransform))
}
return
}
Do you want the nodes to always face the camera, even as the camera moves? That's what SceneKit constraints are for. Either SCNLookAtConstraint or SCNBillboardConstraint can keep a node always pointing at the camera.
Do you want the node to face the camera when placed, but then hold still (so you can move the camera around and see the back of it)? There are a few ways to do that. Some involve fun math, but a simpler way to handle it might just be to design your 3D assets so that "front" is always in the positive Z-axis direction. Set a placed object's transform based on the camera transform, and its initial orientation will match the camera's.
Here's how I did it:
func faceCamera() {
guard constraints?.isEmpty ?? true else {
return
}
SCNTransaction.begin()
SCNTransaction.animationDuration = 5
SCNTransaction.completionBlock = { [weak self] in
self?.constraints = []
}
constraints = [billboardConstraint]
SCNTransaction.commit()
}
private lazy var billboardConstraint: SCNBillboardConstraint = {
let constraint = SCNBillboardConstraint()
constraint.freeAxes = [.Y]
return constraint
}()
As stated earlier a SCNBillboardConstraint will make the node always look at the camera. I am animating it so the node doesn't just immediately snap into place, this is optional. In the SCNTransaction.completionBlock I remove the constraint, also optional.
Also I set the SCNBillboardConstraint's freeAxes, which customizes on what axis the node follows the camera, again optional.
I want the node to face the camera when I place it then keep it here (and be able to move around). – Marie Dm
Blockquote
You can put object facing to camera, using this:
if let rotate = sceneView.session.currentFrame?.camera.transform {
node.simdTransform = rotate
}
This code will save you from gimbal lock and other troubles.
The four-component rotation vector specifies the direction of the rotation axis in the first three components and the angle of rotation (in radians) in the fourth. The default rotation is the zero vector, specifying no rotation. Rotation is applied relative to the node’s simdPivot property.
The simdRotation, simdEulerAngles, and simdOrientation properties all affect the rotational aspect of the node’s simdTransform property. Any change to one of these properties is reflected in the others.
https://developer.apple.com/documentation/scenekit/scnnode/2881845-simdrotation
https://developer.apple.com/documentation/scenekit/scnnode/2881843-simdtransform
I'm having a hard time setting boundaries and positioning camera properly inside my view after panning. So here's my scenario.
I have a node that is bigger than the screen and I want to let user pan around to see the full map. My node is 1000 by 1400 when the view is 640 by 1136. Sprites inside the map node have the default anchor point.
Then I've added a camera to the map node and set it's position to (0.5, 0.5).
Now I'm wondering if I should be changing the position of the camera or the map node when the user pans the screen ? The first approach seems to be problematic, since I can't simply add translation to the camera position because position is defined as (0.5, 0.5) and translation values are way bigger than that. So I tried multiplying/dividing it by the screen size but that doesn't seem to work. Is the second approach better ?
var map = Map(size: CGSize(width: 1000, height: 1400))
override func didMove(to view: SKView) {
(...)
let pan = UIPanGestureRecognizer(target: self, action: #selector(panned(sender:)))
view.addGestureRecognizer(pan)
self.anchorPoint = CGPoint.zero
self.cam = SKCameraNode()
self.cam.name = "camera"
self.camera = cam
self.addChild(map)
self.map.addChild(self.cam!)
cam.position = CGPoint(x: 0.5, y: 0.5)
}
var previousTranslateX:CGFloat = 0.0
func panned (sender:UIPanGestureRecognizer) {
let currentTranslateX = sender.translation(in: view!).x
//calculate translation since last measurement
let translateX = currentTranslateX - previousTranslateX
let xMargin = (map.nodeSize.width - self.frame.width)/2
var newCamPosition = CGPoint(x: cam.position.x, y: cam.position.y)
let newPositionX = cam.position.x*self.frame.width + translateX
// since the camera x is 320, our limits are 140 and 460 ?
if newPositionX > self.frame.width/2 - xMargin && newPositionX < self.frame.width - xMargin {
newCamPosition.x = newPositionX/self.frame.width
}
centerCameraOnPoint(point: newCamPosition)
//(re-)set previous measurement
if sender.state == .ended {
previousTranslateX = 0
} else {
previousTranslateX = currentTranslateX
}
}
func centerCameraOnPoint(point: CGPoint) {
if cam != nil {
cam.position = point
}
}
Your camera is actually at a pixel point 0.5 points to the right of the centre, and 0.5 points up from the centre. At (0, 0) your camera is dead centre of the screen.
I think the mistake you've made is a conceptual one, thinking that anchor point of the scene (0.5, 0.5) is the same as the centre coordinates of the scene.
If you're working in pixels, which it seems you are, then a camera position of (500, 700) will be at the top right of your map, ( -500, -700 ) will be at the bottom left.
This assumes you're using the midpoint anchor that comes default with the Xcode SpriteKit template.
Which means the answer to your question is: Literally move the camera as you please, around your map, since you'll now be confident in the knowledge it's pixel literal.
With one caveat...
a lot of games use constraints to stop the camera somewhat before it gets to the edge of a map so that the map isn't half off and half on the screen. In this way the map's edge is showing, but the furthest the camera travels is only enough to reveal that edge of the map. This becomes a constraints based effort when you have a player/character that can walk/move to the edge, but the camera doesn't go all the way out there.
I'm trying to rotate an SCNBox I created using swipe gestures. For example, when I swipe right the box should rotate 90degs in the Y-axis and -90degs when I swipe left. To achieve this I have been using the node's SCNAction.rotateByX method to perform the rotation animation. Now the problem I'm having is when rotating along either the X-axis or Z-axis after a rotation in the Y-axis and vice-versa is that the positions of the axes change.
What I have notice is that any rotation perform on either of the X,Y,Z axes changes the direction in which the other axes point.
Example: Default position
Then after a rotation in the Z-axis:
Of course this pose a problem because now when I swipe left or right I no longer get the desire effect because the X-axis and Y-axis have now swapped positions. What I would like to know is why does this happen? and is there anyway to perform the rotation animation without it affecting the other axes?
I apologize for my lack of understanding on this subject as this is my first go at 3d graphics.
Solution:
func swipeRight(recognizer: UITapGestureRecognizer) {
// rotation animation
let action = SCNAction.rotateByX(0, y: CGFloat(GLKMathDegreesToRadians(90)), z: 0, duration: 0.5)
boxNode.runAction(action)
//repositoning of the x,y,z axes after the rotation has been applied
let currentPivot = boxNode.pivot
let changePivot = SCNMatrix4Invert(boxNode.transform)
boxNode.pivot = SCNMatrix4Mult(changePivot, currentPivot)
boxNode.transform = SCNMatrix4Identity
}
I haven't ran into any problems yet but it may be safer to use a completion handler to ensure any changes to X,Y,Z axes are done before repositioning them.
I had the same issue, here's what I use to give the desired behavior:
func panGesture(sender: UIPanGestureRecognizer) {
let translation = sender.translationInView(sender.view!)
let pan_x = Float(translation.x)
let pan_y = Float(-translation.y)
let anglePan = sqrt(pow(pan_x,2)+pow(pan_y,2))*(Float)(M_PI)/180.0
var rotVector = SCNVector4()
rotVector.x = -pan_y
rotVector.y = pan_x
rotVector.z = 0
rotVector.w = anglePan
// apply to your model container node
boxNode.rotation = rotVector
if(sender.state == UIGestureRecognizerState.Ended) {
let currentPivot = boxNode.pivot
let changePivot = SCNMatrix4Invert(boxNode.transform)
boxNode.pivot = SCNMatrix4Mult(changePivot, currentPivot)
boxNode.transform = SCNMatrix4Identity
}
}